Using as a library about faiss HOT 5 CLOSED

@sbdk82
This conversion is a way to define an embedding. "Hashing" could mean many things in this context, including what's Faiss does internally for representing the vectors it indexes.

Directly using a q-gram as a sum of 1-hot vector may be ok-ish but not taking into account the inter-word similarity. This is why I suggested taking the sum of word2vec (over q-grams for instance) instead. We have done some tests in a similar setting with Faiss and it seemed that this was giving decent results.

from faiss.

Faiss is not a string matching library.
If you can cast your problem to vector search (i.e., by using string embeddings such as sums of word2vec), then you may consider Faiss, but the hard part will be to define a proper string embedding in the first place that would be adapted to your problem. Otherwise consider using a specific string matching algorithm.

from faiss.

How about converting the string to a vector of hash values of its q-grams?

from faiss.

@jegou Thanks !! I will try to figure out how could I use word2vec for my data.

from faiss.

How can i use word2vev with FAISS

from faiss.