Comments (5)
Hi @komi786 , the problem is probably about the fact that you're keeping your hnsw graph on disk and using not fast enough disks.
In general, we recommend to store hnsw graph on disk only as a last attempt to reduce memory footprint, when you are ready to give up on the speed.
I would recommend to remove on_disk option for hnsw graph, and either keep everything in memory, if you have enough resources, or enable on_disk option for vectors only, not for hnsw graph.
from qdrant-client.
Also, your client instantiation seems to be controversial, you provide url with http
, however set https
flag to true
If you want to speed up the upserting/querying, I would recommend to switch to grpc if possible.
from qdrant-client.
@joein i added https when i found discussion on time out error in an another issue here . just wanted to check if this makes any difference but no. you are right . Can you tell me what is the difference between on_disk_payload and on_disk options?
from qdrant-client.
I removed on_disk_payload as well as on_disk from HNSW config but time out issue still persists
from qdrant-client.
you don't need to remove on_disk_payload
could you send output of your collection info once again please?
what is the batch size you're using?
if it is large or you have long documents, it might be reasonable to reduce it
from qdrant-client.
Related Issues (20)
- healthz return raw str, but decode use .json() HOT 2
- Indeterministic lock results HOT 1
- QdrantClient seems to crash Gunicorn HOT 2
- I don't understand how to extract all documents HOT 1
- Unable to use metadata filter values containing spaces HOT 3
- Missing update_payload method
- ModuleNotFoundError: No module named 'dbm' during connection HOT 3
- invalid PayloadIndexParams model: integer_index_params in gRPC mode HOT 2
- qdrant cloud: retrieve collection metrics via python client HOT 1
- NotImplementedError: cannot instantiate 'WindowsPath' on your system HOT 2
- pydantic <=2.6.0 converts bool to 0/1 and breaks bool filters HOT 2
- Qdrant Scroll Method Timeout HOT 8
- Mounting Local Volume for Langchain Embeddings in Docker Container HOT 8
- can not get "time" from api using the sdk HOT 4
- Extra inputs are not permitted HOT 5
- `init_from is deprecated` warnings on create collection calls HOT 2
- fix orderby conversion with pydantic 1.10.x HOT 1
- The problem I encountered when using celery task to execute qdrant in fastApi HOT 1
- Hybrid search throws error when `prefer_grpc=True` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from qdrant-client.