weezy20 / kvs Goto Github PK

Currently for get tries to fetch all values that are in memory (store.map). This map is built via either replaying all log commands or by loading a pre-built map from disk. The latter requires better guarantees in relation to the disk log that it was built against. Having an empty map or a plain different map that doesn't reflect the current status of the database log issued by KvStore::open indicates a problem, the wrong values will be loaded into memory.

The following suggestions would need an implementation:

1. Make the memory map load values from the same on-disk memory index as in the path specified by KvStore::open. If one doesn't exist, create one in the same location so as to maintain consistency. Also, the file name should be generated deterministically based on the current hash of the db log file so as to compute the file path of the on-disk in memory index (even though we already know it's going to be in the same place as the database), just to guarantee that it's the file we want.
2. The hash of the current on-disk memory index should be stored somewhere to identify it as being the correct memory index that was generated during the time of a replay of log commands. This could mean either using a third file to maintain these integrity checks or some other mechanism. I prefer this option as it's simpler to understand and possibly implement.
3. Optimize the two steps away if the cost of hashing and doing these lookups outweigh the benefit gained, for example, if the db log is small enough that it's cheaper to replay them to build the in memory index, do that!