Comments (1)
thisisaaronland
commented on July 21, 2024
Re-opening not because of filehandles but because scheduling 600K goroutines, in general, just makes the computer sad...
[wof-sync-files] 19:19:11.551084 [info] Scheduled 73924 Completed 445 Success 0 Error 0 Skipped 445 Retried 0 Goroutines 75133 Time 10.288574844s
[wof-sync-files] 19:19:21.827454 [info] Scheduled 190523 Completed 1096 Success 0 Error 0 Skipped 1096 Retried 0 Goroutines 191267 Time 20.564949727s
[wof-sync-files] 19:19:33.128454 [info] Scheduled 273171 Completed 2070 Success 0 Error 0 Skipped 2070 Retried 0 Goroutines 275799 Time 31.865956141s
[wof-sync-files] 19:19:43.141135 [info] Scheduled 379669 Completed 3100 Success 0 Error 0 Skipped 3100 Retried 0 Goroutines 380113 Time 41.878603692s
[wof-sync-files] 19:19:53.566032 [info] Scheduled 470369 Completed 4208 Success 0 Error 0 Skipped 4208 Retried 0 Goroutines 467256 Time 52.303533563s
[wof-sync-files] 19:20:04.338812 [info] Scheduled 654022 Completed 4990 Success 0 Error 0 Skipped 4990 Retried 0 Goroutines 651170 Time 1m3.075065831s
/usr/local/data/whosonfirst-data-venue-us-california$ fatal error: runtime: cannot allocate memory
from go-whosonfirst-s3.
Related Issues (16)
- Investigate go-tunny...
- Is maxprocs being set / honoured correctly? HOT 1
- Don't treat 404 as an error
- Optionally write errors to a file...
- status monitor is not being updated correctly HOT 3
- Make it work with brands
- Less confusing logging when PUT-ing objects
- Why do HTTP HEAD requests request in a freak-out-and-die loop? HOT 1
- Add a -debug flag
- Add support for -prefix
- Make numprocs configurable HOT 1
- Use rcrowley's metrics package HOT 2
- Check lastmodified dates
- Add error/retry code
- Investigate offical AWS go bindings now that they are blessed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-whosonfirst-s3.