"OverflowError: Python int too large to convert to C long" about glacier-upload HOT 3 CLOSED

In this line which is mentioned:

C:\users\palmerf\appdata\local\programs\python\python38\lib\site-packages\glacier_upload\get_job_output.py", line 41, in get_job_output

Can you try changing

file.write(response['body'].read())

while 1:
    data = file_output.read(3.2E7) # 32 MB chunk at a time
    if not data:
        break

    file.write(data)

This should provide a batching like read from the read rather than downloading it all at once. Referred to this example here: https://stackoverflow.com/a/4971681

If this works, I can submit a pull request to fix it.

from glacier-upload.

This is what worked:

with open(file_name, "xb") as file:
     while 1:
     data = response["body"].read(int(2**25)  # 32 MB chunk at a time
     if not data:
         break

    file.write(data)

This is not ideal - especially if the archive was created from a list of files. In that case "upload.py"
creates a tarfile.

Debugging this program is either time-consuming - a 4 hour wait for a download - or expensive if you add provisioned capacity. I've decided to go with one of the Windows solutions.

from glacier-upload.

This should be solved in version 2.0 with chunked downloading of archives.

from glacier-upload.