Ch11_06_alphalens_signals_quality about machine-learning-for-trading HOT 8 CLOSED

The next cell has
best_predictions= pd.read_hdf(results_path / 'predictions.h5', f'train/{lookahead:02}')

Just move this cell up so it runs before the one that threw the error. Will update the notebook accordingly.

from machine-learning-for-trading.

Hi Stefan,

Thanks for your reply. Maybe I was not very clear at very beginning. The key point is that in "predictions.h5" there is no 'train/01' at this time. There is only 'test/01'. According to the notebook, "We generate the train predictions in the notebook alphalens_signal_quality and the test predictions in the notebook random_forest_return_signals.", but it seems that I cannot find where 'train' predictions are made throughout 06_alphalens_signal_quality and 05_random_forest_return_signals.

Thanks again!

from machine-learning-for-trading.

Correct, should be test or whichever way you choose to name the file containing your predictions unless you want to evaluate the training result. Updated accordingly.

from machine-learning-for-trading.

Then in 07_backtesting_with_zipline,
"
def load_predictions(bundle):
t = 1
df = pd.concat([pd.read_hdf(results_path / 'predictions.h5', 'train/{:02}'.format(t)),
pd.read_hdf(results_path / 'predictions.h5', 'test/{:02}'.format(t))])
df = df[~df.index.duplicated()].drop('y_test', axis=1)
....
"
we should remove the 'train/{:02}'.format(2) as well, right. Basically remove every 'train'?

from machine-learning-for-trading.

Not necessarily, you may want to compare in-sample vs out-of-sample performance. It depends on what you want to evaluate. In many cases, the train predictions are the result of time-series cross-validation to select the best model, whereas test refers to the out-of-sample predictions for the best model during a period following the cross-validation sample. You may want to evaluate the performance of both 'train' and test periods results.

from machine-learning-for-trading.

Yes I got the train and test separation part. Since there is no code to aggregate in-sample cross validation results to generate the 'train/01' predictions (only 'test/01' is explicitly generated), would you please add it? Thanks a lot!

from machine-learning-for-trading.

The train predictions are created during cross validation and stored in the last line of cell curently labeled 43 with pd.concat(predictions).to_hdf(cv_store, 'predictions/' + key).

You would select the version that corresponds to the best parameters you identify upon comparison. To 'explicitly' generate the file you are requesting, you would load those predictions and save under a the 'train' name of your choosing. I'm currently a bit busy but will see if I can add more comments to clarify.

from machine-learning-for-trading.

Thanks Stefan. No need for that. I can find myself. Thanks for your time and patience! Your book is very helpful!

from machine-learning-for-trading.