[BUG]Error: 400 Bad Request: Payload error: JSON payload (95879532 bytes) is larger than allowed (limit: 33554432 bytes). about flowise HOT 9 CLOSED

duplicate issue with this #1824

from flowise.

Updated the flowise to version 1.6.5 and still the same problem persists.

however after the flowise version update to 1.6.5 pdf & txt file which initially was showing unknown error are now giving the same error of JSON file limit error. By default the default file limit is 50mb (#96) however i am getting an error where in the limit is specified as 33.55 MB

PDF file upsert error:

txt file upsert error:

docx file upsert error:

from flowise.

Followed the instruction as per this issue (#96) and increased the file limit to 300 mb.

still the error persists...plz help

from flowise.

@mrabbah: Thanx for the comment...i thought that i am the only one facing this issue....interestingly i am not getting this error with a chunk size of 1000 for the same document and it works fine with that. My preferred chunk size is 250-300 which produces this error.

from flowise.

The problem has been sorted. The issue was with Qdrant which had a default value of 32 MB (33554432 byte) limitation for Maximum size of POST data in a single request in megabytes. Increasing the same in qdrant config.yaml has sorted this problem and upserting is now working fine. BDW the document that i was trying to upsert had more than 4500 vectors (for 250 chunk size with 10 overlap). No change in flowise required to solve this issue

from flowise.

@mrabbah: plz configure Qdrant with required file size for a single upload and the problem goes away..

from flowise.

@prithvi151080 I didn't find any flags or environment variable that I can set to change the Max JSON payload size accepted by Qdrant!

from flowise.

This change u have to do in the qdrant vector DB...not in flowise...create a custom_config.yaml file and under services set Maximum size of POST data in a single request in megabytes to the required file size as per ur use case and the problem goes away. use this config file as the master config for ur Qdrant instance.
Qdrant config link details: https://qdrant.tech/documentation/guides/configuration/

from flowise.

@mrabbah: Hope this helps :)

from flowise.