forking breaks lwt-io if happens after Lwt_main.run about lwt HOT 3 OPEN

If you insert Lwt_unix.set_pool_size 0 ; before the first Lwt_main.run it will work. I Imagine Lwt starts a worker thread, which will not exist in the child process after the fork. With Lwt < 5, Lwt_unix.(set_default_async_method Async_none) would also have done the trick.

More generally I have no idea why Lwt provides Lwt_unix.fork given that (i) Lwt starts worker threads on encountering blocking system calls and (ii) threads and fork don't mix (all you can do in the child process after fork in a multi-threaded program is call async-signal-safe functions and then exec). Note that Unix.execv* is not thread safe and cannot be used in multi-threaded programs either.

from lwt.

Possibly Lwt_unix.fork is not thread safe either, because the documentation says that (as opposed to Unix.fork) "in the child process all pending jobs are canceled", and I doubt that that can be achieved only by means of applying async-signal-safe functions.

from lwt.

If you insert Lwt_unix.set_pool_size 0 ; before the first Lwt_main.run it will work. I Imagine Lwt starts a worker thread, which will not exist in the child process after the fork. With Lwt < 5, Lwt_unix.(set_default_async_method Async_none) would also have done the trick.

Yes, it was our workaround to use Lwt_unix.(set_default_async_method Async_none) in the child process.

More generally I have no idea why Lwt provides Lwt_unix.fork given that (i) Lwt starts worker threads on encountering blocking system calls and (ii) threads and fork don't mix (all you can do in the child process after fork in a multi-threaded program is call async-signal-safe functions and then exec). Note that Unix.execv* is not thread safe and cannot be used in multi-threaded programs either.

Yep, in the end we had to switch to lwt-parallel library instead of using fork directly. This library addresses this issue by creating a process snapshot before lwt is used. Probably, this is the only safe way of using Lwt_unix.fork with lwt. We pushed some new features to lwt-parallel (including explicit snapshots). See ocaml/opam-repository#22611

from lwt.