mirage / io-page Goto Github PK

We should deprecate the library. There is no good reasons to expose io-pages to code which is not a device driver.

@djs55 do you think that's doable?

And thus, the library fails to link on windows:

# ** Cannot resolve symbols for C:/cygwin/home/appveyor/.opam/4.02.3+mingw64c/lib/io-page\libio_page_unix_stubs.a(stub_alloc_pages.o/
# ):
#  posix_memalign
# File "caml_startup", line 1:
# Error: Error during linking

See https://ci.appveyor.com/project/samoht/ocaml-git-7vmk0/build/1.0.33#L637

due to stub_alloc_pages.c

#if !defined(__APPLE__)
#include <malloc.h>
#endif

where my c compiler complains (errors out!) that malloc.h has been replaced by stdlib.h.
A conjunct !defined(__FREEBSD__) should fix this issue.

For some reasons (I guess: because of the page alignment constraint) io-page uses its own allocator for bigarrays, which seems to replace OCaml caml_ba_alloc. But caml_ba_alloc was itself calling caml_alloc_custom, which would instruct the GC the amount of memory that could be reclaimed when finalizing that bloc. As imperfect as this is, it is still preferable than not letting the GC know that there are potentially a lot of memory to be reclaimed by GCing the bigarrays.

In my test program that does a lot of IO within 128MiB of RAM and dies from out-of-memoriness after a few minutes, this caml_alloc_pages holds around 20MiB alone, making it the second memory holder after caml_aligned_malloc (the mallocer for the heap) which holds 70MiB.

I think calling caml_alloc_custom or something to the same effect would improves the situation.
I'm not even sure those blocs are freed at all (and I'm setting up another experiment to check this).

see: https://ci.appveyor.com/project/djs55/io-page/build/1.0.29#L437

I am not sure why it tries to install my-package, or who decided jq was the best way to parse the output, but the current state is not working anyway.

Windows is picky about memory allocators and we ought to use _aligned_free to deallocate this memory. Unfortunately the OCaml finaliser always calls free so we can't use it.

I encounter to run in a failed assertion (line 20) in https://github.com/hannesm/test -- which reproduces the code ocaml-crunch contains for reading data from its key-value store.

a string is put into an io_page, and a cstruct of the given string length is exposed -- the cstruct is not equal (using to_bigarray) to a cstruct produced by of_string of the initial string...

this problem occurs here on FreeBSD (seems to work fine on linux) with io-page 1.4.0 and 1.2.0 at least...

$ docker run --rm -it ocaml/opam:debian-9_ocaml-4.04.0
$ opam install utop io-page-unix
$ utop
utop # #require "io-page-unix";;
Error: The external function `mirage_alloc_pages' is not available
utop # #require "io-page";;
Error: The external function `mirage_alloc_pages' is not available      

Is this supposed to work?