directus / gatsby-source-directus Goto Github PK
View Code? Open in Web Editor NEWSource plugin for pulling data into Gatsby from a Directus API.
Source plugin for pulling data into Gatsby from a Directus API.
Gatsby v5
Directus 10.7.2
When grabbing JSON files from Directus into Gatsby, we are given this error message.
When we change the filename (Download) from name.json
to name.svg
we can bypass this error for now.
Error:
GraphQL request:1:3
1 | 72A0Ac7F369Cb2618D58E1849Da49629Json
| ^'. Stacktrace was 'GraphQLError: Syntax Error: Invalid number, expected digit but got: "A".
at syntaxError
(/path/node_modules/graphql/error/syntaxError.js:15:10)
at readNumber
(/path/node_modules/graphql/language/lexer.js:550:40)
at readNextToken
(/path/node_modules/graphql/language/lexer.js:413:14)
at Lexer.lookahead
(/path/node_modules/graphql/language/lexer.js:84:29)
at Lexer.advance
(/path/node_modules/graphql/language/lexer.js:67:38)
at Parser.advanceLexer
(/path/node_modules/graphql/language/parser.js:1536:31)
at Parser.expectToken
(/path/node_modules/graphql/language/parser.js:1393:12)
at Parser.many
(/path/node_modules/graphql/language/parser.js:1507:10)
at Parser.parseDocument
(/path/node_modules/graphql/language/parser.js:122:25)
at parse (/path/node_modules/graphql/language/parser.js:32:17)
at TypeMapper.convertSDLTypeDefinition
(/path/node_modules/graphql-compose/src/TypeMapper.ts:249:44)
at Function.createTemp (/path/node_modules/graphql-compose/src/
ObjectTypeComposer.ts:301:28)
at Function.create (/path/node_modules/graphql-compose/src/Obje
ctTypeComposer.ts:271:21)
at forEach
(/path/node_modules/gatsby/src/schema/infer/index.js:41:41)
at Array.forEach (<anonymous>)
at addInferredTypes
(/path/node_modules/gatsby/src/schema/infer/index.js:27:13)'```
Gatsby 5 requires Node 18, but as soon as I upgrade Node from v16.18.1 to v18.12.1, gatsby-source-directus fails with this error:
"@directus/gatsby-source-directus" threw an error while running the createSchemaCustomization lifecycle:
gatsby-source-directus: authentication failed with: connect ECONNREFUSED ::1:8055
Are credentials valid?
In Node 16 this error doesn't occur, and the project builds properly. In gatsby-config.js
, the plugin is configured like this:
{
resolve: '@directus/gatsby-source-directus',
options: {
url: 'http://localhost:8055',
auth: {
token: process.env.TOKEN
}
}
}
I am trying to build my gatsby application (running v4). My app uses @directus/gatsby-source-directus as a gatsby plugin to access CMS stored data for my app. When running both the development and the production build (gatsby develop and gatsby build), I encounter a rate limiting error message, thrown at the line the directus query is used. The app throws on trying to create a production build (gatsby build). The development build also shows this error at buildtime, but the app runs after the errors appear in the console.
My app also uses image processing and transforming plugins like:
Run a gatsby application that queries a larger amount of documents (over 200), documents that each contain their own set of images.
You can run the application by using:
Error: Too many requests, retry after 278ms.
sdk.cjs.js:696 Transport.request
[website]/[@directus]/sdk/dist/sdk.cjs.js:696:23
runMicrotasks
task_queues:96 processTicksAndRejections
node:internal/process/task_queues:96:5
sdk.cjs.js:710 Transport.get
[website]/[@directus]/sdk/dist/sdk.cjs.js:710:16
sdk.cjs.js:37 FilesHandler.readOne
[website]/[@directus]/sdk/dist/sdk.cjs.js:37:26
gatsby-node.js:88 resolve
[website]/[@directus]/gatsby-source-directus/gatsby-node.js:88:36
async Promise.all
async Promise.all
async Promise.all
async Promise.all
async Promise.all
graphql-runner.ts:220 GraphQLRunner.query
[website]/[gatsby]/src/query/graphql-runner.ts:220:14
create-pages.ts:36 wrappedGraphQL
[website]/[gatsby]/src/services/create-pages.ts:36:25
gatsby-node.js:109 Object.exports.createPages
/Users/limbo/Documents/BLOCKTALK/website/gatsby-node.js:109:20
api-runner-node.js:487 runAPI
[website]/[gatsby]/src/utils/api-runner-node.js:487:16
latest
16.6.0
Chrome
That's my first github issue ever, so be patient with me :)
Upon running gatsby build
i get the following console error, right after success createSchemaCustomization - 0.995s
:
ERROR #11321 API.NODE.EXECUTION
"@directus/gatsby-source-directus" threw an error while running the sourceNodes lifecycle:
Unable to fetch:
http://0.0.0.0:8055/apiassets/fa38e044-7051-478f-bb3e-60a9203cad49
---
Reason: Response code 404 (Not Found)
---
Fetch details:
{
"attempt": 3,
"method": "GET",
"errorCode": "ERR_NON_2XX_3XX_RESPONSE",
"responseStatusCode": 404,
"responseStatusMessage": "Not Found",
"requestHeaders": {
"user-agent": "got (https://github.com/sindresorhus/got)",
"authorization": "Bearer XXX", #hidden ;)
"accept-encoding": "gzip, deflate, br"
},
"responseHeaders": {
"content-security-policy": "script-src 'self' 'unsafe-eval';worker-src 'self' blob:;child-src 'self' blob:;img-src 'self' data: blob: https://cdn.directus.io;media-src 'self' https://cdn.directus.io;connect-src 'self' https://*;default-src 'self';base-uri 'self';font-src 'self' https: data:;form-action 'self';frame-ancestors 'self';object-src
'none';script-src-attr 'none';style-src 'self' https: 'unsafe-inline'",
"x-powered-by": "Directus",
"vary": "Origin",
"access-control-allow-credentials": "true",
"access-control-expose-headers": "Content-Range",
"content-type": "application/json; charset=utf-8",
"content-length": "135",
"etag": "W/\"87-z71CK3Cz69C2v+wQIOJA8Sei+HE\"",
"date": "Wed, 19 Apr 2023 09:18:30 GMT",
"connection": "keep-alive",
"keep-alive": "timeout=5"
}
}
---
HTTPError: Response code 404 (Not Found)
- index.js:913 Request._onResponseBase
[frontend]/[got]/dist/source/core/index.js:913:31
- index.js:948 Request._onResponse
[frontend]/[got]/dist/source/core/index.js:948:24
- index.js:962 ClientRequest.<anonymous>
[frontend]/[got]/dist/source/core/index.js:962:23
- node:events:628 Object.onceWrapper
node:events:628:26
- node:events:525 ClientRequest.emit
node:events:525:35
- index.js:43 ClientRequest.origin.emit
[frontend]/[@szmarczak]/http-timer/dist/source/index.js:43:20
- node:_http_client:693 HTTPParser.parserOnIncomingClient
node:_http_client:693:27
- node:_http_common:119 HTTPParser.parserOnHeadersComplete
node:_http_common:119:17
- node:_http_client:535 Socket.socketOnData
node:_http_client:535:22
- node:events:513 Socket.emit
node:events:513:28
- readable:324 addChunk
node:internal/streams/readable:324:12
- readable:297 readableAddChunk
node:internal/streams/readable:297:9
- readable:234 Socket.Readable.push
node:internal/streams/readable:234:10
- stream_base_commons:190 TCP.onStreamRead
node:internal/stream_base_commons:190:23
not finished source and transform nodes - 3.339s
error Command failed with exit code 1.
As you can see the asset url http://0.0.0.0:8055/apiassets/fa38e044-7051-478f-bb3e-60a9203cad49
has a missing /
after api
.
The issue may come come from gatsby-node.js
line 91
:
const fileUrl = ${plugin.url}assets/${file.id};
There you see the missing /
.
Even if add the /
to the options.url
value in my gatsby-config.ts
(like localhost/api/
), the trailing slash gets replaced by line 171
in the gatsby-node.js
:
const basePath = baseUrl.pathname.endsWith('/') ? baseUrl.pathname.slice(0, -1) : baseUrl.pathname;
Im not sure if im doing anything wrong since im curious about the fact that im the first reporting this.
"@directus/gatsby-source-directus": "^9.15.1"
"gatsby": "^5.8.1"
"directus": "^9.20.4"
node
v19.9.0
Thanks for your help guys :) greets from germany <3
reopen of #16 because it's closed and don't want it to get lost.
Following issue persists after updating everything
ERROR #85923 GRAPHQL.VALIDATION
There was an error in your GraphQL query:
Cannot query field "directus" on type "Query".
If you don't expect "directus" to exist on the type "Query" it is most likely a typo. However, if you expect "directus" to exist there are a couple of solutions to common problems:
- If you added a new data source and/or changed something inside gatsby-node/gatsby-config, please try a restart of your development server.
- You want to optionally use your field "directus" and right now it is not used anywhere.
It is recommended to explicitly type your GraphQL schema if you want to use optional fields.
File: src\pages\index.jsx:41:5
See our docs page for more info on this error: https://gatsby.dev/creating-type-definitions
When Directus is not installed in the root of the domain but in a sub directory, such as https://example.com/directus the plugin fails to initialise due to a (I believe) typo.
It is looking for files in https://example.com/directusassets/
directory that does not exists and is the result of a missing trailing slash in:
https://github.com/directus/gatsby-source-directus/blob/bfd4f40136dc760a8a9239f19bc691bcf4fb4b40/gatsby-node.js#L93C11-L93C18
In my opinion the line should be (note the slash before assets, that is missing in the main code):
const fileUrl = `${plugin.url}/assets/${file.id}`;
Thank you
Gatsby 5.9 and 5.10, Directus 10.2.1, gatsby-source-filesystem 5.10, @directus/gatsby-source-directus 9.15.1
Highlighting the same issue that has been reported by strapi users has affected me using gatsby-source-directus too.
Added as a placeholder while tracking gatsby-uc/plugins#426 (comment)
Reverting to gatsby 5.8 and gatsby-source-filesystem 5.8 things work again.
Using gatsby-source-directus, when requesting a specific image using a GraphQL query in a static page query and in gatsby-node.js/page query of page created by gatsby-node.js, the children of the imageFile property are null on the second query.
The issue only happens on gatsby build
, everything works fine when using gatsby develop
.
https://github.com/jarne/gatsby-source-directus-bug-repro
gatsby build
, not when running gatsby develop
ERROR
(node:35867) [DEP0148] DeprecationWarning: Use of deprecated folder mapping "./public/" in the "exports" field module
resolution of the package at /Users/jarne/Downloads/gatsby-source-directus-bug-repro/node_modules/extract-files/package.json.
Update this package.json to use a subpath pattern like "./public/*".
(Use `node --trace-deprecation ...` to show where the warning was created)
success Building production JavaScript and CSS bundles - 5.278s
success Building HTML renderer - 7.714s
success Execute page configs - 0.016s
success Caching Webpack compilations - 0.001s
success run queries in workers - 0.480s - 2/2 4.17/s
success Merge worker state - 0.003s
success Rewriting compilation hashes - 0.001s
success Writing page-data.json files to public directory - 0.012s - 3/4 339.23/s
ERROR
Page data from page-data.json for the failed page "/": {
"componentChunkName": "component---src-pages-index-js",
"path": "/",
"result": {
"pageContext": {}
},
"staticQueryHashes": [
"1673888159"
]
}
failed Building static HTML for pages - 1.895s
ERROR #95313
Building static HTML failed for path "/"
See our docs page for more info on this error: https://gatsby.dev/debug-html
36 | return (
37 | <GatsbyImage
> 38 | image={pl.cover.imageFile.childImageSharp.gatsbyImageData}
| ^
39 | style={{
40 | width: "80%",
41 | maxWidth: 175,
WebpackError: TypeError: Cannot read properties of null (reading 'gatsbyImageData')
- index.js:38
gatsby-starter-minimal/src/pages/index.js:38:67
- index.js:31
gatsby-starter-minimal/src/pages/index.js:31:31
- index.js:25
[gatsby-starter-minimal]/[camelcase]/index.js:25:1
- extends.js:8
[gatsby-starter-minimal]/[@babel]/runtime/helpers/extends.js:8:1
- extends.js:14
[gatsby-starter-minimal]/[@babel]/runtime/helpers/extends.js:14:40
- extends.js:13
[gatsby-starter-minimal]/[@babel]/runtime/helpers/extends.js:13:1
- index.js:44
[gatsby-starter-minimal]/[camelcase]/index.js:44:1
- static-entry.js:295
gatsby-starter-minimal/.cache/static-entry.js:295:22
- history.js:22
[gatsby-starter-minimal]/[@gatsbyjs]/reach-router/es/lib/history.js:22:1
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
jarne@Jarnes-iMac gatsby-source-directus-bug-repro %
9.7.0
16.14.2
Postgres 13.6
Firefox, Safari
Docker
Untouched example from
https://github.com/directus/examples (both the directus setup & gatsby starter)
Running it and visiting the index page produces the following error
success onPreExtractQueries - 0.001s
ERROR #85923 GRAPHQL
There was an error in your GraphQL query:
Cannot query field "directus" on type "Query".
If you don't expect "directus" to exist on the type "Query" it is most likely a typo. However, if you expect "directus" to exist there are a couple of solutions to common problems:
- If you added a new data source and/or changed something inside gatsby-node/gatsby-config, please try a restart of your development server.
- You want to optionally use your field "directus" and right now it is not used anywhere.
It is recommended to explicitly type your GraphQL schema if you want to use optional fields.
File: src\pages\index.jsx:41:5
See our docs page for more info on this error: https://gatsby.dev/creating-type-definitions
failed extract queries from components - 0.249s
Gatsby DSG fails using gatsby-source-directus.
I set up a directus with gatsby-source-directus. What do I want to achieve is to have some of my collections being built only at the first request, by using the deferred static generation.
Here, in short, the defer : true
should identify the products as DSG items. You can find the option at the end of this snippet
exports.createPages = async ({ graphql, actions }) => {
const { createPage } = actions
await graphql(`
{
directus {
product(limit: -1, filter: {status: {_eq: "published"}}) {
id
status
product_code
}
}
}
`).then(result => {
const templatePaths = {
product: path.resolve(`./src/templates/product.js`),
};
products.forEach((node) => {
createPage({
path: `${path}/${node.product_code}`,
component: templatePaths.product,
context: {
language,
productID: parseInt(node.id),
},
/*
When set to true, it tells Gatsby to exclude the page from the build step and instead generate it during the first HTTP request:
See https://www.gatsbyjs.com/docs/reference/rendering-options/deferred-static-generation/
*/
defer: true, // <-- this is the option to set the DSG generation for certain contents
})
})
})
}
batch: true
in graphql options must be set to true, in order to work with the defer: true
in gatsby-config.js.
module.exports = {
plugins: [
{
resolve: '@directus/gatsby-source-directus',
options: {
url: `http://localhost:8055`,
auth: {
token: `${token}`,
},
dev: {
refresh: '5s'
},
//[Optional] - defines extra options to be passed into gatsby-source-graphql. Useful for advanced use cases
graphql: {
batch: true,
// See https://github.com/graphql/dataloader#new-dataloaderbatchloadfn--options
// for a full list of DataLoader options
dataLoaderOptions: {
maxBatchSize: 512,
},
}
},
},
],
};
Once the setup is completed, if you build
everything, nothing fails, and every page is generated.
┌ src/templates/family.js
│ ├ D /prodotti/product_name
│ └ D ...355 more pages available
├ src/templates/product.js
│ ├ D /prodotti/ANOTHER-PRODUCT
│ └ D ...17875 more pages available
├ src/pages/404.js
│ ├ /en/404.html
│ └ ...7 more pages available
└ src/pages/index.js
├ /en/
└ ...3 more pages available
╭────────────────────────────────────────────────────────────────╮
│ │
│ (SSG) Generated at build time │
│ D (DSG) Deferred static generation - page generated at runtime │
│ ∞ (SSR) Server-side renders at runtime (uses getServerData) │
│ λ (Function) Gatsby function │
│ │
╰────────────────────────────────────────────────────────────────╯
✨ Done in 43.97s.
Once you serve
what's in the build, and you try to access at your i.e. localhost:9000/your-item, you get this:
http://localhost:9000/prodotti/my-product
Internal server error.
Which leads to this error:
Rendering html for "/prodotti/my-product" failed. nodeFetch is not a function
**TypeError: nodeFetch is not a function**
I already investigated for a possible solution
First, i tried to use only the gatsby-source-graphql plugin, i.e.
resolve: "gatsby-source-graphql",
options: {
typeName: "Directus",
fieldName: "directus",
url: "http://localhost:8055/graphql",
headers: {
Authorization: `Bearer ${token}`,
},
batch: true,
fetchOptions: {},
dataLoaderOptions: {
maxBatchSize: 512,
},
},
},
This way, the pages are all correctly built as with gatsby-source-directus, and they are generated properly in deferred static generation, while on serve
. Only, the graphql obviously does not contains all the necessary graphql items, such as dyrectus_system.
So I managed that the problem should be in the gatsby-source-directus.
If you check at this answer you can find that this was a problem common to gatsby-source-graphql too.
Therefore, I tried to change /node_modules/@directus/gatsby-source-directus/node_modules/gatsby-source-graphql/fetch.js
like this:
//BEFORE
// const nodeFetch = require(`node-fetch`); // this is passed to the Apollo Link
//AFTER
const nodeFetch = require(`node-fetch`).default
And now it somehow works. Pages are correctly generated in DSG BUT it is a dirty, quick not and resolutive working fix.
In example, with this line, gatsby images are not fetched. And there should be many other issues which I had not investigated yet
9.7.0
16.14.0
MySql
Chrome, Brave
Linux 5.10.76-linuxkit
Docker
gatsby throws an error when reloading a page file.
info changed file at /home/user/project/src/pages/index.js
ERROR
Missing onError handler for invocation 'building-schema', error was 'Error: Schema must contain uniquely named types but contains multiple types named
"File".'. Stacktrace was 'Error: Schema must contain uniquely named types but contains multiple types named "File".
at new GraphQLSchema (/home/user/project/node_modules/graphql/type/schema.js:194:15)
at SchemaComposer.buildSchema (/home/user/project/node_modules/graphql-compose/src/SchemaComposer.ts:242:12)
at buildSchema (/home/user/project/node_modules/gatsby/src/schema/schema.js:84:33)
at build (/home/user/project/node_modules/gatsby/src/schema/index.js:111:18)
at buildSchema (/home/user/project/node_modules/gatsby/src/services/build-schema.ts:19:3)'
When I start the development server there are no issue but when I reload any page existing in ./src/pages/
I got this error.
1 - Init a fresh gatsby project (Typescript or javascript) I tested both
2 - In package.json
{
...
"keywords": [
"gatsby"
],
"scripts": {
"develop": "gatsby develop",
"start": "gatsby develop",
"build": "gatsby build",
"serve": "gatsby serve -p 5000",
"clean": "gatsby clean",
"lighthouse": "lhci autorun"
},
"dependencies": {
"@directus/gatsby-source-directus": "^9.14.1",
"@fortawesome/fontawesome-free": "^6.2.0",
"dotenv": "^16.0.3",
"gatsby": "^4.24.4",
"gatsby-plugin-google-gtag": "^4.24.0",
"gatsby-plugin-image": "^2.24.0",
"gatsby-plugin-manifest": "^4.24.0",
"gatsby-plugin-sass": "^5.24.0",
"gatsby-plugin-sharp": "^4.24.0",
"gatsby-plugin-sitemap": "^5.24.0",
"gatsby-source-filesystem": "^4.24.0",
"gatsby-transformer-sharp": "^4.24.0",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-helmet": "^6.1.0",
"sass": "^1.55.0"
},
"devDependencies": {
"@lhci/cli": "^0.8.2"
}
}
3 - In gatsby-config
require('dotenv').config()
module.exports = {
siteMetadata: {
...
},
plugins: [
'gatsby-plugin-image',
'gatsby-plugin-sitemap',
{
resolve: `gatsby-plugin-google-gtag`,
options: {
trackingIds: [process.env.GTAG_MEASUREMENT_ID],
pluginConfig: {
head: true,
},
},
},
{
resolve: 'gatsby-plugin-manifest',
options: {
icon: 'src/images/icon.png',
},
},
'gatsby-plugin-sharp',
'gatsby-transformer-sharp',
`gatsby-plugin-sass`,
{
resolve: 'gatsby-source-filesystem',
options: { name: 'images', path: './src/images/' },
__key: 'images',
},
{
resolve: 'gatsby-source-filesystem',
options: { name: 'pages', path: './src/pages/' },
__key: 'pages',
},
{
resolve: 'gatsby-source-filesystem',
options: { name: 'static', path: './src/static/' },
__key: 'static',
},
{
resolve: '@directus/gatsby-source-directus',
options: {
url: process.env.DIRECTUS_API_URL,
auth: { token: process.env.DIRECTUS_API_KEY },
},
},
],
}
4 - Start the development server (it will work)
5 - Make a minor change in src/pages/index.js
and save
6 - You will get the error
9.18.1
v16.17.1
v16.16.0
MySql
Linux 5.10.76-linuxkit
On a VPS with pm2
and nginx
for reverse proxy
I have an mp4 video stored in the file library. In gatsby-source-directus v9.14.1 I can make this call in graphiQL:
query MyQuery {
directus {
home {
marquee_video {
id
imageFile {
id
}
}
}
}
}
and it returns this:
{
"data": {
"directus": {
"home": {
"marquee_video": {
"id": "44e55b25-ce2c-49cd-bf41-362f16d55f62",
"imageFile": {
"id": "6a9b883e-80c5-5e1c-a8d5-ebe154e03790"
}
}
}
}
},
"extensions": {
"enableRefresh": "true"
}
}
however, in v9.14.3 or 9.14.4 it returns this:
{
"data": {
"directus": {
"home": {
"marquee_video": {
"id": "44e55b25-ce2c-49cd-bf41-362f16d55f62",
"imageFile": null
}
}
}
},
"extensions": {
"enableRefresh": "true"
}
}
imageFile
always returns null
for videos, at least mp4 videos. Doing a build exhibits the same behavior. Directus itself is at version 9.20.4.
It all worked before, but I had to reinstall packages, and now I am lost somewhere in either dependency hell, though that shouldn't be it, and authentication failed.
The token 100% works, I changed it 3 times now in Directus to make sure there's no weird caching happening.
Tried deleting cache folder etc etc. Nothing works.
These are my deps:
"@directus/gatsby-source-directus": "^9.15.1", "@hookform/error-message": "^2.0.1", "@mdx-js/react": "^2.3.0", "@react-icons/all-files": "^4.1.0", "gatsby": "^5.7.0", "gatsby-cookie-notice": "^1.0.6", "gatsby-plugin-gdpr-cookies": "^2.0.9", "gatsby-plugin-google-gtag": "^5.7.0", "gatsby-plugin-image": "^3.7.0", "gatsby-plugin-manifest": "^5.7.0", "gatsby-plugin-mdx": "^5.7.0", "gatsby-plugin-offline": "^6.8.0", "gatsby-plugin-robots-txt": "^1.8.0", "gatsby-plugin-sass": "^6.7.0", "gatsby-plugin-sharp": "^5.7.0", "gatsby-plugin-sitemap": "^6.7.0", "gatsby-source-filesystem": "^5.7.0", "gatsby-source-graphql": "^5.8.0", "gatsby-transformer-sharp": "^5.7.0", "react": "^18.2.0", "react-dom": "^18.2.0", "react-hook-form": "^7.43.5", "react-hook-form-persist": "^3.0.0", "sass": "^1.59.2"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.