Hi there! When I was playing around with cylinder-flow dataset, there was an error:
Traceback (most recent call last):
File "/home/liluo/anaconda3/lib/python3.9/multiprocessing/queues.py", line 244, in _feed
File "/home/liluo/anaconda3/lib/python3.9/multiprocessing/reduction.py", line 51, in dumps
File "/home/liluo/anaconda3/lib/python3.9/site-packages/torch/multiprocessing/reductions.py", line 359, in reduce_storage
File "/home/liluo/anaconda3/lib/python3.9/multiprocessing/reduction.py", line 198, in DupFd
File "/home/liluo/anaconda3/lib/python3.9/multiprocessing/resource_sharer.py", line 48, in __init__
OSError: [Errno 24] Too many open files
I notice that there are 100 pt files for these train/valid/test folders on cylinder-flow dataset. I'm wondering is the problem caused by reading so many files? I adjust my num_workers
to 4.