memory not-release issue for large BS on FastSeq about fastseq HOT 3 CLOSED

This issue is not specific to Fastseq. It occurs because of memory held by zombie processes (either from data loader or from postprocess) when main process is aborted due to OOM.

https://discuss.pytorch.org/t/pytorch-doesnt-free-gpus-memory-of-it-gets-aborted-due-to-out-of-memory-error/13775/5

from fastseq.

This issue is not specific to Fastseq. It occurs because of memory held by zombie processes (either from data loader or from postprocess) when main process is aborted due to OOM.

https://discuss.pytorch.org/t/pytorch-doesnt-free-gpus-memory-of-it-gets-aborted-due-to-out-of-memory-error/13775/5

When main process exits, we can ask kernel to send signal to child processes. So they can detect such signal and exit elegantly without be zombies.
https://stackoverflow.com/questions/284325/how-to-make-child-process-die-after-parent-exits

from fastseq.

solved by adding "--shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864" for gpu docker run.

from fastseq.