tarides / notafs Goto Github PK

Errors are not properly exposed to end-users, as there's a catch-all on the way that hides the precise error. It would at least help the CLI to figure out the correct sector size if this could be cleaned up. Also a proper "disk is full" error should be added rather than raised with failwith!

It would be nice if we could format and prepare disks from the terminal, to be used later by an unikernel! Probably by using the mirage-kv interface, to enable copying files from and out of a notafs disk?

There are few settings that we should store on the disk (in the first sector? in each root generation?) to avoid using a disk with the wrong parameters later:

• a version number (in case we need to upgrade our schema later)
• the sector size (to detect that the block device was opened with the wrong page size)
• the disk size (to detect that a resize happened)
• the number of root generations (which should be configurable when calling format?)
• ...?

We need some mutexes to protect against concurrent mutations. Currently it's possible for an lwt promise to be reading a file, and have another lwt process concurrently delete/write to the same rope... which immediately frees related sectors, so the first promise could end up reading garbage!

We have some benchmarks for the unix backend, but it would be interesting to know how it behaves on solo5 where I/O has a different performance profile. To interpret the numbers, it makes sense to compare reads/writes speed with ocaml-tar as it has the maximum throughput possible.

Currently, reads are batched only when reading a rope as this requires structuring the lwt code in a way that allows concurrency. A similar optimization should be done to verify_checksum to speed up the validation at boot.

edit: Same thing for Rope.free! (as we need to traverse the full rope to free all reachable sectors)

It might be helpful to have a more explicit error if we add a new empty named file to the Notafs file system. For the moment, it is just an assertion in src/files.ml.

Error (File "src/files.ml", line 223, characters 14-20: Assertion failed)

Currently the root generations are not checksummed. Only the sectors containing filesystem data are, as their checksum is stored in their parent pointer. Since there are no pointers to the root generations, their checksum should be stored in their own sector.

• If we don't checksum the roots, then this could lead to a correctness issue! (data pointed from a root could checksum just right, but the filesystem might be out-of-sync with the free queue)
• Furthermore, this would enable detecting that the disk was never formatted (by discovering that no root generation have a valid checksum)

The wear-leveling story is looking pretty alright... except for the root generations that might be written much more often than the other sectors. There are two potential directions to reduce the amount of writes performed on the first sectors of the disk:

• Use those first sectors to preallocate where the root generations will be. Then we would only need to write to those first sectors once all preallocated generations have been used. This looks simple and reasonable...
• Adding a header to each data sector, to store pointers to the next generations. At boot, this requires following a trail of pointers in memory... and I'm not convinced that it always works! (when some writes didn't complete)

The support for Mirage_kv.RW is missing some stuff (like the ability to connect to an existing disk, rather than format it!)... and the implementation hasn't been stress-tested (I have some doubts regarding the way we simulate the hierarchical directories in list)

Add some documentation to the project (via mli files and such)