Train/test split does not take into account carryover effects in test set about mamimo HOT 3 CLOSED

Hello! Thanks for your kind words, very happy that the library helped you :)

To assess the model quality, you should always split first and then do the processing and training only on the training dataset, otherwise the model (or preprocessor) might be able to cheat if you give it testing data a well.

Sure, it does not see the last part of the time series, but that's how it out also on real life: you train the model, gut it has to deal with unseen data tomorrow.

If you really just want to fit on the whole dataset, you could write a cv splitter that outputs not train/ test, but 2 times the complete dataset. That will lead to overfitting though, and you can trust the carryovers etc. even less.

Best
Robert

from mamimo.

Sure, it does not see the last part of the time series, but that's how it out also on real life: you train the model, gut it has to deal with unseen data tomorrow.

I am getting confused with this part. Even in real life, you know what happened in the past. In this case for example, you know the carryover strength parameter that you applied in the previous days, so why not apply it in the past data to update current or future data as well?

Edit: expanded question

from mamimo.

Yes, and the hyper parameter optimization learns from the past and checks how well the same parameters with in the future. Once you have the best hyper parameters, you can predict into the future with them for unseen data

from mamimo.