Serialize Dataframe about datumbox-framework HOT 4 CLOSED

Looking at the latest code, it seems that whenever Dataframe is created backed by MapDB, it creates a temp DB file. The temp files get deleted whenever the JVM shuts down.

However, there are many use cases where one would like to deserialize a large dataset into a disk backed map. Then they may later try out multiple algorithms on that dataframe. This is especially helpful during experimentation phase if the input dataset takes long to translate into a Dataframe.

What are you thought? If implemented what should the design look like? I can contribute this features if it makes sense.

from datumbox-framework.

What you describing makes sense but unfortunately it is not currently supported. A way around it would be to store the records externally from the Dataframe and then read them and load them back to the Dataframe when necessary.

Adding the feature is not that straightforward, I'll need to check out the design as you said. I'll keep you posted.

from datumbox-framework.

This feature is implemented in the experimental branch. It will be released on version 0.8.0.

from datumbox-framework.

All the tests passed and the feature was added to the develop branch. A stable release is expected within a couple of weeks.

from datumbox-framework.