feat(io): support way to ensure `read_*` apis create physical tables in the backend about ibis HOT 4 CLOSED

I think having separate methods makes sense for this, while keyword arguments clutter the API.

How about load_* equivalents that unconditionally persist data and mirror the read_* APIs?

from ibis.

I think best distinction we can do between any approach here is:

1. Things persist after the session ends
2. Things do not persist after the session ends

The former almost certainly requires physical storage (ruling out views), while the latter can be more less or whatever's well-suited to backend.

If people need something specific, they have to call create_{view,table}

from ibis.

I think having separate methods makes sense for this, while keyword arguments clutter the API.

If these APIs are identical except for how the backend loads the data (a binary option), I'd argue that a keyword argument makes more sense than duplicate methods which do the same thing for most backends.

How about

# cache=False is whatever the backend currently does, cache=True always uses a table
t = con.read_parquet(..., cache=True)

How about load_* equivalents that unconditionally persist data and mirror the read_* APIs?

I hadn't considered the temporary/persistent choice, but I suppose that is another axis. Since our read_* methods generate a unique name by default (and anything that persists permanently should probably have an explicit name), I'm not sure if a binary modality here would make sense, and agree a new load_* method might make more sense (so we could better indicate and enforce that table_name is required).

In my case though, I don't really want a persistent table, but do want the load to happen only once for the lifetime of the session. In this case a simple kwarg (that really only needs to be implemented by the few backends that use views by default) seems like the simpler option.

from ibis.

After talking this through, I think this issue would be better resolved through fixing #6195, and documenting this pattern for backends where it's relevant (mostly just duckdb). Closing.

from ibis.