HW9 Disk quota exceeded error in nohup.out about cmc-csci046 HOT 8 CLOSED

Everyone is allocated only 200MB of disk space on the lambda server, and the

OSError: [Errno 122] Disk quota exceeded

error message indicates that you have sused more than this. Therefore, when you try to write to a file, you cannot, and python generates an error message. This causes your python program to crash and stop processing more input.

All of the print calls in your code get redirected to a file using output redirection (>), and so these print statements are what are causing you to exceed your storage, and you should remove them.

Part of the lesson I want you to learn from this assignment is how to process a large amount of data (~400gb) while using only limited resources.

PS. In the future, please copy and paste the error messages into github rather than using images.

from cmc-csci046.

I am getting the same error

OSError: [Errno 122] Disk quota exceeded

when trying to reduce the .lang files and .country files into consolidated folders using:

./src/reduce.py --input_paths outputs/geoTwitter*.lang --output_path=reduced.lang

and

./src/reduce.py --input_paths outputs/geoTwitter*.country --output_path=reduced.country

I used the du --si -s $HOME command to check my disk usage and it seems like I have 125MB of data left. Are there any suggestions for how I could free up space (it doesn't look like I have any unnecessary print statements) or any consensus on how much space is needed to run these commands?

Edit: I cannot edit files either, as I'm getting an error message saying that my disc quota has been exceeded!

from cmc-csci046.

@vik-jhun

This was a problem with the lambda server's disk being full, and not you having exceeded your quota limits (see #79 ). It should be fixed now.

Sorry about that.

from cmc-csci046.

@mikeizbicki I am still getting the error of Disk quota exceeded when trying to create a new directory or cloning a directory from git. I deleted a few directories to see if that can help free up space but it did not work.

from cmc-csci046.

@tadizov There is currently 150Gb free on the hard drive you are on, and you have only used ~15Mb of your 200Mb allocation, so you should be fine. You may have to log in/out for the new settings to take effect.

from cmc-csci046.

@mikeizbicki I am still getting an error when trying to clone a git repository in the form as @tadizov described :
fatal: could not create work tree dir 'trees': Disk quota exceeded

I also ran my initial commands from yesterday's comment again and they're still facing the same errors. I ran these commands just a few minutes before posting this comment.

from cmc-csci046.

@vik-jhun You have a file called .vscode-server that is taking up your entire disk space allotment. This one reason among many that you should not be using vscode for these assignments.

from cmc-csci046.

@mikeizbicki I understand. Thank you for looking that up so promptly. I've removed the file and it seems to be working now.

from cmc-csci046.