Comments (3)
We are using createRemoteFileNode
which does all of the fetching for us at L269-L275. We then use Promise.all to let these process concurrently.
In reality though, if you have one picture per row, that means each node will have one picture, and we are running the Promise.all for each node (at L269-L275) because Airtable attachment fields house multiple attachments. (So throttling around those lines of code wouldn't do much, nor would adding throttling in createRemoteFileNode
.)
I believe if you want to try running the downloading serially (or throttle in someway), you would need to do that around the node creation itself. See L158-167.
from gatsby-source-airtable.
Thank you for the context, @jbolda. I will try starting there when I’m able to get back to this.
from gatsby-source-airtable.
I've submitted a solution that uses the concurrency limit in Promise.map() from bluebird in #50
from gatsby-source-airtable.
Related Issues (20)
- Gatsby v3 DeprecationWarning HOT 4
- Combining fileNode and text/markdown mappings HOT 3
- faster build when sourcing a lot of data HOT 2
- getting base name in graphql
- Field type mismatch in Airtable means field doesn't show up at all in Gatsby HOT 1
- Query on Collaborator type does not recognise the profilePicUrl
- is it possible to filter by a field in a Linked Table? HOT 1
- Dependency Dashboard
- Slow build times HOT 1
- Mp3 files get converted to Mpga HOT 3
- Unused variable `nodeType`
- Can this plugin be used to link lookup fields instead of linked records?
- How do add default tableLinks?
- update for Gatsby V4
- base name issue in Airtable?
- Upcoming updates to Airtable Attachment URLs that are fetched via API HOT 8
- error Cannot read properties of undefined (reading 'id')
- Early heads up: switching away from user API keys HOT 1
- Accelerated need to limit large url requests to “list records”, with alternative POST version HOT 5
- Build take alot of time.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gatsby-source-airtable.