ctuning / ck-analytics Goto Github PK

when retrieving experiment table (ck get experiment:), add option to convert tables (currently as list of vectors) into pandas or numpy objects

Moved from CK: mlcommons/ck#21

We have various tables to show results (particularly when crowdsourcing optimization):
http://cknowledge.org/repo/web.php?wcid=208c21b837924601:bc0409fb61f0aa82:3b2d453798564d5a
We should unify them (split them when there are too many results, prune them, sort them, etc). We may use pandas for that ...

In Collective Mind, we had a universal plotter, where you could select any available dimension X,Y,Z and customize graph via web. We should add something similar to CK. Preliminary support is available via JSON:
http://cknowledge.org/repo/web.php?wcid=208c21b837924601:2d41f89bcf32d4d4:3b1b261d30f3934a

So we need to convert JSON into web selectors ...

We currently have a simple way of analyzing variation of experimental results in math.variation in ck-analytics repo. We use density via gaussian_kde and can detect multiple expected values, i.e. to warn that there are several states and features to separate them should be found. However, we do not calculate confidence interval, etc (since which such approach we do not need it, but instead we look at the delta and if there is more than 1 expected value).

We should check that it works correctly and possibly improve it (calculate confidence interval, etc)...

Moved from CK: mlcommons/ck#26

We now collect all points on Pareto frontier. There can be too many points so we should fix a number (say 4) to leave on the frontier and leave optimal ones (by distance).

Moved from CK: mlcommons/ck#13

To avoid wasting time reading repetitive papers, and find unique ideas:
https://github.com/ctuning/ck/wiki/Advanced_usage_auto_paper

Moved from CK: mlcommons/ck#23