cmu-db / noisepage-pilot Goto Github PK

We currently send SIGTERM to shutdown TScout in dodos/tscout.py.
We should send SIGINT instead.

Currently, Clusterer in forecaster/cluster.py doesn't allow online clustering (the function _cluster_online is defined but is never called). How should we use that function? Is it enough to just replace the last line in __init__ from

self.assignment_df = self._cluster_offline()

to

self.assignment_df = self._cluster_online()

?

ExplainMicroserviceCost processes the explain "tree" for a given query and estimates the cost of the query using our models via the microservice. We currently make a network request for each node in the explain tree.

The overall goals of this task to restructure ExplainMicroserviceCost is as follows:

1. Insert some tooling/instrumentation to extract microservice inference times (can also be logged to sqlite)
2. Make 1 network call to perform model inference for an entire query explain tree
3. Allow for partial query explain tree cost inference (i.e., failing 1 node does not fail the entire tree)
4. Batch logging the inference + real query results

Currently ExplainMicroserviceCost::PrepareCostRequest still only sends a small curated set of features. ExplainMicroserviceCost::PrepareCostRequest should send all valid features as returned from hutch.