Tracking source code of an experiment about ploomber-engine HOT 3 CLOSED

There's an important detail here that I hadn't considered before:

Currently, we create a fixed schema. so we first need to tackle the versioning issue (ploomber/sklearn-evaluation#119).

Then, we need to figure out how to support this.

When people are using SQLiteTracker directly, they make explicit API calls. e.g.: experiment.log_metric("accuracy", 0.9)). However, when they use the tracking features in this package, we do the auto-logging for them (see the example). The challenge is that both of them use the same underlying logic (SQLiteTracker).

So we need to find a way to manage both use cases (with source code and without source code)

I think that maybe this package can create a subclass:

class PloomberEngineTracker(SQLiteTracker):
    pass

And support adding the source code as a new column. But this means that we should register somewhere a new versioning schema. This can become kind of messy. Maybe we should push this a bit? I'm unsure what the best solution here is and I don't know how relevant this feature will be.

from ploomber-engine.

this should be opened in sklearn-evaluation

from ploomber-engine.

Moved

from ploomber-engine.