RuntimeError: unable to write to file </torch_31911_952036340_6686>: No space left on device (28)

[saulgooodman]

Hello! I encountered the following error while running your code,

I used the nvidia-smi command to find out that my GPU's memory is sufficient.

I suspect this is due to an error with torch.multiprocessing because the use of shared memory causes the device to be unable to continue running once there are many other tasks running on the host, butit seems that the file-sharing strategy is irreplaceable. Can you tell me how to solve this problem? Thank you!

[haoyanan2024]

I have the same problem. Have you solved it?

[saulgooodman]

I guess you are running it inside Docker. You need to increase the --shm-size. After I set --shm-size="32g" in Docker, it worked. For example:

nvidia-docker run -it --name 'eslam' -v /data0/saulgoodman/:/home/data0 --gpus all --shm-size="32g" -p 1212:22 ufoym/deepo:all-py38-cu113 bash