Local client becomes unusable after first run about qdrant-client HOT 4 CLOSED

Yes, there are several ways to access your points

1. As it is persisted in sqlite, you can just read the data from sqlite using sqlite3

Something like this:

import sqlite3
import pickle
connection = sqlite3.connect('asdf/collection/asdf/storage.sqlite')
cursor = connection.cursor()
cursor.execute("SELECT point FROM points")
for row in cursor.fetchall():
     rows = pickle.loads(row[0])

print(rows)

2. The same thing but using QdrantClient classes

from qdrant_client.local.persistence import CollectionPersistence

col = CollectionPersistence("asdf/collection/asdf")
for point in col.load():
    print(point)

3. You can modify load_vectors method in qdrant_client.local.local_collection

vectors[name].append(
    np.ones(self.config.vectors[name].size, dtype=np.float32)
)

->

size = self.config.vectors[name].size if isinstance(self.config.vectors, dict) else self.config.vectors.size
vectors[name].append(np.ones(size, dtype=np.float32))

4. You can wait for sometime for the next release of qdrant-client where we will fix it (I hope it will be available this week)

P.S. I don't think it can spoil your db, but be sure to make a backup before experiments
P.P.S. Pay attention to the path of your db during these experiments, you might need to set it different in different options (e.g. 1) and 2))

from qdrant-client.

Hi @hdmi

Thank you for pointing it out and for a code sample (which occurred to be really valuable :) )!

It seems that there is a bug on our side, which does not correctly load unnamed vectors

As a workaround until we fix it, I can suggest you to make dense vectors named:

vectors_config=models.VectorParams(
            size=1024, 
            distance=models.Distance.COSINE)

->

vectors_config={"text": models.VectorParams(
            size=1024, 
            distance=models.Distance.COSINE)
}

By the way, there is another bug in local mode, which is also highlighted by your code sample :(

operation_info = db.upsert(
            collection_name=collectionName,
            wait=True,
            points=[PointStruct(id = i, 
                                vector = { "text": models.SparseVector(indices=[0,100,123,3213], values=[0.12321, -1.3123, 0.9321, -1.33333])},
                                payload = {"name": f"Name_{i}"})
                    ],
            )

Here you are trying to upsert vector text instead of text-sparse. text vector does not exist and should throw an error

However with correct configuration it should work as intended.

We will fix these bugs in the upcoming days, sorry for the inconvenience

from qdrant-client.

Ok, thanks for the insights!

Kind of offtopic but related to the bug, is there a way of recovering the already stored embeddings?
I have days of generated sparse and dense vectors that I would like to recover.

Maybe I can build a qdrant server and place the sqlite files there.

Thank you for the support!

from qdrant-client.

available as of qdrant-client==1.7.1

from qdrant-client.