rust-ml / tsap Goto Github PK
View Code? Open in Web Editor NEWTree Structured Argument Parser for Rust
License: MIT License
Tree Structured Argument Parser for Rust
License: MIT License
In the last half year I used hydra extensively for machine learning at my university job. It provides a configuration tool for complex applications, remote capability (e.g. executing on SLURM cluster) and multiruns (e.g. doing hyperparameter search).
This issue proposes a similar crate for Rust and linfa to build and configure machine learning applications. Similar to structopt and clap it consists of two elements, a macro system to derive structs and a argument parser to override default hierarchies. Compared to established argument parsers, we don't follow Unix argument patterns and combine a default from file with override by arguments step. The crate is therefore geared towards ML instead of command line applications.
To keep dependencies low we make most of the API surface optional. In minimal setting the parameter macro generates only a builder pattern; allowing programmatical control of the algorithm. This is geared for consumer crates, which uses simple parameter sets and don't have to make configuration accessible. In more complex settings we also want to take files and command line arguments into account and extend our API surface to that of Hydra.
Because most of the infrastructure already exist, we use serde for serialization, TOML as configuration format and add a small template engine to rewrite TOML's configuration tree. Extending configuration from other files is a typical usecase for a resolver. In total tsap
exists to give a unified macro for generating highly configurable parameter sets for algorithms we have at Linfa.
We will start with a simple example to illustrate a usecase:
use tsap::param;
#[param]
struct RandomForest {
ntrees: usize,
}
#[param]
struct SVClassifier {
regul_term: f32,
}
#[param]
enum Model {
RandomForest(RandomForest),
SVClassifier(SVClassifier),
}
#[param]
struct Main {
seed: usize,
#[check]
model: Model,
}
fn main() {
// load default configuration from entry file
let conf: Main = Main::from_file("config/main.toml")
.amend_args()
.seed(40)
.try_into()?;
// start experiment, do other stuff...
conf.run_experiment();
}
The following binary calls are then valid:
cargo run
cargo run model=svclassifier
cargo run -m model=randomforest seed=10,20,30,40
Default values are defined with a hierarchy of TOML files, typical parametrizations can be shaped by using using default values. For example the upper example could have one for the main entry and one for the model:
seed = 42
[model]
from_file = { base_path="models/", default="randomforest" }
variant = "randomforest"
ntrees = 128
The keys variant
and from_file
are reserved keys and indicate enum variant and where to look for the configuration. When overloading the model
with a different configuration, the application will automatically look in that folder for a TOML file with the same name.
WIP
?
for buildingA declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.