devyonz / epsilon Goto Github PK

Add tf saver and associated boiler plate code to save and load old models (possibly only neural net)

Currently we publish only loss or cv loss values. Needs an accuracy percentage value .

Currently the only method of comparison is cross-validation scores. Once accuracy is added, that will be another method.

Cross-entropy loss = Sum ( y * log yhat)
Create a table accuracy and result of the 10 fold cross entropy loss.
https://www.openml.org/a/estimation-procedures/1
https://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html#cross-entropy

Gets confusing with CV.

1. Standard NLTK stuff like lematize, stop words
2. BOW with n-gram or n-gram skip models
3. word2vec

Port code and make sure it runs on a GCP/Azure/AWS instance.

Once done, take an input json of experiment configs and execute them. We will have to arrive on an experiment config to use.
Off the top of my head it should have:

1. Feature Engineering Flags.
   Example:
   StopWords: On/Off
   Stem: On/Off
   n-grams: 4 (This will be a continuous value)
2. Model Type (Bayes, NN, SVM. We can assume that the hyperparameters get played with internally)

This coupled with model saving / serialization. Should hold us in good stead - come December.