Error while uploading files "Failed to index document" about langchain-chat-with-documents HOT 17 CLOSED

I Found the solution, remove the commented line after the WEAVITE_HOST property in .ENV file

WEAVIATE_HOST= xxxxx.weaviate.network THIS ----> [# do not use https:// just the domain like bellingcat-xxx.weaviate.network} <--- THIS

from langchain-chat-with-documents.

I have the same issue...

from langchain-chat-with-documents.

Same here...

from langchain-chat-with-documents.

Does it fail every time or only with some documents?

from langchain-chat-with-documents.

Every time unfortunately. With both PDFs and TXT files

from langchain-chat-with-documents.

Could you add some console screenshots, please? (make sure to remove any secret keys if there any)

from langchain-chat-with-documents.

The debug errors in the server are the same as the above comment:

• info Loaded env from C:\Users\r.ramos\Documents\GitHub\langchain-chat-with-documents.env
  "pnpm" no se reconoce como un comando interno o externo,
  Error: 'Error in addDocuments' TypeError [ERR_INVALID_URL]: Invalid URL
  at WeaviateStore.addVectors (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/langchain/dist/vectorstores/weaviate.js:103:19)
  at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
  at async WeaviateStore.fromDocuments (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/langchain/dist/vectorstores/weaviate.js:155:9)
  at async eval (webpack-internal:///(api)/./src/server/api/routers/weaviate.ts:81:13)
  at async resolveMiddleware (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/index.mjs:416:30)
  at async callRecursive (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/index.mjs:452:32)
  at async callRecursive (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/index.mjs:452:32)
  at async resolve (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/index.mjs:482:24)
  at async file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/resolveHTTPResponse-e1286cb3.mjs:87:32
  at async Promise.all (index 0)
  at async resolveHTTPResponse (file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/resolveHTTPResponse-e1286cb3.mjs:84:28)
  at async file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/nodeHTTPRequestHandler-f0efcff4.mjs:32:24
  at async file:///C:/Users/r.ramos/Documents/GitHub/langchain-chat-with-documents/node_modules/@trpc/server/dist/adapters/next.mjs:42:9
  ❌ tRPC failed on weaviate.index: Failed to index document`

The browser console shows this:

from langchain-chat-with-documents.

I have the same error. Trying to follow the pre-signed URL generated [form: https://doc-user-01234abc.0123456789xyz.r2.cloudflarestorage.com/brief.txt?X-Amz-Algorithm=AWS4-HMAC-SHA256&....], I get:
"This XML file does not appear to have any style information associated with it. The document tree is shown below.

SignatureDoesNotMatch
The request signature we calculated does not match the signature you provided. Check your secret access key and signing method.
"
I believe it might be related to this bug, but I cannot find a way around it. At Cloudflare Discrord's R2 channel they mention SignatureDoesNotMatch in multiple places, including this gist for inspiration. Trying to apply the gist insights did not help me either, but I am not an expert on the underlying tech stack, so you might have more luck.

Enabling us to manually upload the documents via CloudFlare could be a workaround - my files end at CloudFlare bucket as intended, but pre-signing fails.

from langchain-chat-with-documents.

On the R2 account page -> [Manage R2 API Tokens] make sure you have all these permissions enabled:
Edit: Allow edit access of all objects and List, Write, and Delete operations of all buckets

from langchain-chat-with-documents.

In my case, the R2 details are correct, however I have not been able to successfully parse any documents. All documents are uploaded to R2, but there's a parsing error with Weaviate. Please see: https://share.cleanshot.com/WkMB3LnN

from langchain-chat-with-documents.

In my case, the R2 details are correct, however I have not been able to successfully parse any documents. All documents are uploaded to R2, but there's a parsing error with Weaviate. Please see: https://share.cleanshot.com/WkMB3LnN

Same. FIles are correctly uploaded it seems

from langchain-chat-with-documents.

I can confirm deleting everything after "... xxxxx.weaviate.network" worked for me too! Great find, thanks!

from langchain-chat-with-documents.

Unfortunately I'm still getting the same errors. I didn't have that comment in my .env file in the first place.

This is what I'm getting:

Warning: Indexing all PDF objects
Error
    at InvalidPDFExceptionClosure (/path/to/pdf.js:452:35)
    at Object.<anonymous> (/path/to/pdf.js:455:2)
    at __w_pdfjs_require__ (/path/to/pdf.js:45:30)
    at Object.<anonymous> (/path/to/pdf.js:7939:23)
    at __w_pdfjs_require__ (/path/to/pdf.js:45:30)
    at /path/to/pdf.js:88:18
    at /path/to/pdf.js:91:10
    at webpackUniversalModuleDefinition (/path/to/pdf.js:18:20)
    at Object.<anonymous> (/path/to/pdf.js:25:3)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:168:29)
    at ModuleJob.run (node:internal/modules/esm/module_job:197:25)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:337:24)
    at async PDFLoaderImports [as pdfjs] (/path/to/pdf.js:80:34)
    at async PDFLoader.parse (/path/to/pdf.js:22:42)
    at async eval (webpack-internal:///(api)/./src/server/api/routers/weaviate.ts:73:29)
    at async resolveMiddleware (/path/to/@trpc/server/dist/index.mjs:416:30)
    at async callRecursive (/path/to/@trpc/server/dist/index.mjs:452:32)
    at async next (/path/to/@trpc/server/dist/index.mjs:460:32)
    at async callRecursive (/path/to/@trpc/server/dist/index.mjs:452:32)
    at async resolve (/path/to/@trpc/server/dist/index.mjs:480:24)
    at async /path/to/@trpc/server/dist/resolveHTTPResponse-edf72fc7.mjs:123:32
    at async Promise.all (index 0)
    at async resolveHTTPResponse (/path/to/@trpc/server/dist/resolveHTTPResponse-edf72fc7.mjs:120:28)
    at async /path/to/@trpc/server/dist/nodeHTTPRequestHandler-c9ddbeeb.mjs:61:24
    at async /path/to/@trpc/server/dist/adapters/next.mjs:39:9 {
  message: 'Invalid PDF structure'
}
❌ tRPC failed on weaviate.index: Failed to index document

from langchain-chat-with-documents.

(/path/to/pdf.js:45:30) this path doesnt look normal to me. Did you add anything to the original project?

from langchain-chat-with-documents.

I obfuscated the error text to remove the personal details. Everything is untouched from the latest pull.

from langchain-chat-with-documents.

Ah that makes more sense 😅 well unfortunately I cant help. But a few console.log would help shed a better light. I think logging the client and the method this error is refering to would be a good first step.

from langchain-chat-with-documents.

In my case, the R2 details are correct, however I have not been able to successfully parse any documents. All documents are uploaded to R2, but there's a parsing error with Weaviate. Please see: https://share.cleanshot.com/WkMB3LnN

Please try updating node to the latest stable release and running npm install pdf-parse@latest to update the library. I have fixed the same error by this way.

from langchain-chat-with-documents.