roma-glushko / morty Goto Github PK
View Code? Open in Web Editor NEW๐ฌ Lightweight experiment and configuration manager for small ML/DL projects and Kaggling
๐ฌ Lightweight experiment and configuration manager for small ML/DL projects and Kaggling
For small to middle (side) projects and experiments, it's useful to have a simple, straightforward and flexible configuration system. It's not convenient to copy-past even simple plain dictionary/class based configuration loading code.
Also, when using configs, we need to be able to:
In the same time, it may be helpful to have a simple and straightforward experiment management system. We can't focus too deep on this part as we probably never replace Neptune.ai or W&B functionality.
However, in some cases, installing them may be an overhead and all that scientist may need is a just plain and quick way to understand what is the best scored experiment so far and how it was produced.
In other cases, they may not be compatible with ML/DL framework version you need (like TF 2.5 RC Ampere-compatible version may not be compatible with Neptune dependency list)
Add a straightforward and simple way to manage configs/hyperparams and track experiment outcomes.
We may based our configs on python dictionaries specified as separate files/modules. This should be a flexible approach that allows to make configurable such a pieces of code as augmentation pipelines. This would be annoying to add another layer of abstraction to just be able to experiment with augmentation.
Another useful thing would be to have reusable factories which allows to register different "moving" components (e.g. losses). So we could combine configured class/type value and create an instance of it via the factories.
It's a bit harder to imagine a simple experiment management system to keep straightforward and still useful. The system would be local and file-based. All information could be logged in separated directories where experiment details could be logged.
We would like to see the following information logged:
We are not going to focus on more advanced way of reusing logged information like plotting learning curves or embeddings.
However, DVC provides a simple way to plot some kind of information and we could use the system compatible with their plotting functionality.
Let's call it an MVP.
The main source of inspiration came from Kaggle master's pipelines and Hydra config system:
Experiment Indexer should collect information that is needed to build the leaderboard. The system should be able to identify index inconsistencies and the recent changes in order to perform an automatic partial reindex of the experiment information.
TBU
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.