Comments (1)
monatis
commented on May 23, 2024
hi @hyunmokky, there's no hard limit on the number of vectors you can upload to one collection, so yes, you can, considering the following:
- Before uploading, set
indexing_thresholdto a very large number to disable indexing temporarily. You can set it a reasonable value again after the upload is complete. - Make sure that you have enough storage. The amount of storage required is heavily dependent on the vector dimension, amount of payload data if any, and HNSW params you choose. Be aware that 1Kb = 1 vector of size 256.
- You can use either
.upload_records()or.upload_collection()methods for parallel uploading, depending on which one suits better on the rest of your code, e.g., whether you produce vectors and/or payload data one by one or as a large Numpy array, for example.
from qdrant-client.
Related Issues (20)
- how to get shard keys for collection when using custom sharding HOT 1
- upload_collection can't be launched with parallel and without explicit ids
- Regression: payloads cannot contain python builtin datetime objects in 1.7.1 HOT 2
- Error: Payload Limit Exceeded HOT 16
- Unable to close grpc_channel. Connection was interrupted on the server side HOT 12
- grpc options are not parsed correctly when https is set
- PointStruct is very slow HOT 8
- update scoring in local mode in discovery api HOT 1
- Missing import statement in documentation (Get Started) HOT 2
- Local Qdrant db Error on loading: KeyError: '__pydantic_fields_set__' HOT 4
- query_text param not working for qdrant_client.search HOT 8
- Upgrade fastembed version from 0.1.1 to 0.2.1 (latest) HOT 3
- Deleting points by ID not working HOT 3
- Trigger nighly tests against latest qdrant dev build
- Tracking issue: local mode for Qdrant v1.8 HOT 3
- Feature Request: Progress bar for batch upload_points function HOT 2
- Check vectors for NaN in local mode HOT 2
- qdrant_client.get_fastembed_vector_params() with upload_collection HOT 4
- Python Application Crashes on Attempting to Retrieve Non-existent Collection via QdrantClient in GRPC Mode HOT 2
- Add note about batching into README.md HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from qdrant-client.