Comments (3)
This issue is not specific to Fastseq. It occurs because of memory held by zombie processes (either from data loader or from postprocess) when main process is aborted due to OOM.
from fastseq.
This issue is not specific to Fastseq. It occurs because of memory held by zombie processes (either from data loader or from postprocess) when main process is aborted due to OOM.
When main process exits, we can ask kernel to send signal to child processes. So they can detect such signal and exit elegantly without be zombies.
https://stackoverflow.com/questions/284325/how-to-make-child-process-die-after-parent-exits
from fastseq.
solved by adding "--shm-size=1g --ulimit memlock=-1 --ulimit stack=67108864" for gpu docker run.
from fastseq.
Related Issues (20)
- Does fastseq support cpu HOT 3
- NMT models speedup abnormally related to batch size HOT 2
- Can the fastseq install on windows? HOT 3
- Where to read EL-Attention source code for huggingface-transformers HOT 4
- In which file to read the source code implementation of El-Attention for self-attention HOT 1
- EL-attention GPT-2 HOT 2
- Running error with PyTorch 1.12.1 HOT 2
- fairseq/transformers unit test modify local environment HOT 3
- Transformers unit tests failure
- ACTION REQUIRED: Microsoft needs this private repository to complete compliance info
- RuntimeError: CUDA error: no kernel image is available for execution on the device HOT 7
- ModuleNotFoundError: No module named 'fastseq.models' HOT 2
- fairseq eval_lm HOT 2
- Does it support Tensorflow 2? HOT 3
- Any end-end inference example with Google Colab & HuggingFace HOT 2
- Does it support model seq2seq with encoder, and decoder base on lstm, bi-lstm? HOT 2
- Support for ONNX models & INT8 quantization HOT 2
- Support for HF's transformers 3.1+ HOT 6
- Support for Hugging Face's PEGASUS Model HOT 6
- Support for current fairseq 0.10.2 HOT 13
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastseq.