Comments (9)
Good catch!
It seems like there are 3 possible solutions to this:
- Limiting the upload speed.
- Using the
retry_after
parameter after the blocked request. - Using the
x-ratelimit-*
headers before the blocked request.
If there's no penalty on passing the rate limit too often (which I will research about), I'm opting for the second option, as it will be a cleaner and simpler solution imo. What do you think?
Either way, please share your upload speed so we can get a general sense of the rate limit
Thanks!
from web.
Upload speed should be around 125mbps to 150mbps max, but is usually a bit lower than that. Speed test link: https://www.speedtest.net/result/13941504387
After looking those two things up, it does seem like retry_after is a good option. Not really sure though.
from web.
Hi again!
I ended up fixing the issue using the third option (x-ratelimit-* headers) over at #16
However, my upload speed isn't high enough to properly test this.
Could you clone the branch and test it locally (As explained in #10) and comment there if it fixed your issue?
You can also review the changes if you want toπ
from web.
Oddly, it was working quite well, but then at around 50% of the file (it's about 11.4 GB) it started doing this again.
Checking the discord channel also shows that chunks are being skipped
(Also, I would review them but I know next to nothing about coding lol)
Link to the file I used is here: https://placedata.reddit.com/data/canvas-history/2022_place_canvas_history.csv.gzip (it's just the 2022 history for the Reddit Place thing)
from web.
Thank you! I really appreciate your help with this.
I'll invesitgate this and will probably ask you to test a new version.
I suspect the fix might not be perfect because they x-ratelimit-retry-after
only displays seconds and not milliseconds.
It is weird though, because I would expect it to round up. Just to make sure, you cloned the bugfix/upload-ratelimit branch, right?
from web.
i downloaded the zip from github, but i tried cloning it just now using git (i made sure it was the right branch) and its doing the same thing
from web.
Thanks again, I'll look into it and let you know!
from web.
Just wanted to add, I'm also getting this from trying to download a file.
from web.
The PR #16 for this seems to fix this issue for me.
from web.
Related Issues (20)
- Cant download 8 gb files HOT 2
- docker, docker compose and rclone support? HOT 2
- Can't use The Chrome extension on Disbox which is hosted by local HOT 2
- dark theme bug HOT 1
- unable to rename my files HOT 5
- Expiration time for Attachment URLs decreasing from 14 days to 1 day HOT 1
- Extension HOT 1
- Unable multiple webhooks HOT 2
- Upload fail (Zip File 385Mo) HOT 4
- My file are not displayed HOT 4
- All Files gone all of sudden HOT 2
- Suggestion: Add multiple files at once HOT 1
- Unable to generate share link HOT 1
- Disbox HOT 1
- Directly uploading files from MEGA. HOT 1
- Can not share file HOT 2
- Files not downloading properly HOT 4
- Disbox firefox extension HOT 1
- File Uploading Cancelation HOT 1
- Upload File and New Folder button not working HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from web.