feat: profiling? about ibis HOT 2 OPEN

Hey Nick!

This is a super interesting problem, but I think it's not feasible for us to correlate an ibis expression with a particular duration (or some proxy of that like instructions, or even database plan node) because the thing that the database executes often looks very different from the thing it was asked to execute.

It's one of the trade-offs associated with using SQL and/or a database: you generally get better performance at the cost of debugability.

That said, what do you think about a backend.explain(expr) method? We had this a while ago but it was removed due to lack of implementation across our backends.

This would at least allow you to get some insight into what the database thinks is happening.

from ibis.

what do you think about a backend.explain(expr) method?

I think that sounds nice, even if it returns teh raw string. However, to avoid needing to do a private method on a non-explicit-backend expression

e = ibis.read_csv().<sdfgdsfg>
e._find_backend().explain(e)

I would love to either (1 is my favorite)

1. have ibis.explain(expr) in addition to backend.explain(expr)
2. just have expr.explain() (though we don't want to fill up that namespace TOO much)
3. have expr.explain() in addition to backend.explain(expr)

Possible things to support (duckdb for inspiration)

• (need) show physical plan
• (need) show profiling/timing results
• (nice to have) show logical plan
• (nice to have) show optimized logical plan

from ibis.