Git Product home page Git Product logo

java-client's Introduction

Qdrant ย  Java

Java library for the Qdrant vector search engine.

Javadoc Tests Apache 2.0 License Discord Roadmap 2023

Qdrant Java Client

Java client library with handy utility methods and overloads for interfacing with Qdrant.

๐Ÿ“ฅ Installation

Important

Requires Java 8 or above.

To install the library, add the following lines to your build config file.

Maven

<dependency>
  <groupId>io.qdrant</groupId>
  <artifactId>client</artifactId>
  <version>1.9.0</version>
</dependency>

SBT

libraryDependencies += "io.qdrant" % "client" % "1.9.0"

Gradle

implementation 'io.qdrant:client:1.9.0'

๐Ÿ“– Documentation

๐Ÿ”Œ Getting started

Creating a client

A client can be instantiated with

QdrantClient client = 
  new QdrantClient(QdrantGrpcClient.newBuilder("localhost").build());

which creates a client that will connect to Qdrant on https://localhost:6334.

Internally, the high level client uses a low level gRPC client to interact with Qdrant. Additional constructor overloads provide more control over how the gRPC client is configured. The following example configures a client to use TLS, validating the certificate using the root CA to verify the server's identity instead of the system's default, and also configures API key authentication:

ManagedChannel channel = Grpc.newChannelBuilder(
  "localhost:6334",
  TlsChannelCredentials.newBuilder()
    .trustManager(new File("ssl/ca.crt"))
    .build())
.build();

QdrantClient client = new QdrantClient(
  QdrantGrpcClient.newBuilder(channel)
    .withApiKey("<apikey>")
    .build());

The client implements AutoCloseable, though a client will typically be created once and used for the lifetime of the application. When a client is constructed by passing a ManagedChannel, the client does not shut down the channel on close by default. The client can be configured to shut down the channel on close with

ManagedChannel channel = Grpc.newChannelBuilder(
  "localhost:6334", 
  TlsChannelCredentials.create())
.build();

QdrantClient client = new QdrantClient(
  QdrantGrpcClient.newBuilder(channel, true)
    .withApiKey("<apikey>")
    .build());

All client methods return ListenableFuture<T>.

Working with collections

Once a client has been created, create a new collection

client.createCollectionAsync("my_collection",
  VectorParams.newBuilder()
    .setDistance(Distance.Cosine)
    .setSize(4)
    .build())
  .get();

Insert vectors into a collection

// import static convenience methods
import static io.qdrant.client.PointIdFactory.id;
import static io.qdrant.client.ValueFactory.value;
import static io.qdrant.client.VectorsFactory.vectors;

List<PointStruct> points =
    List.of(
        PointStruct.newBuilder()
            .setId(id(1))
            .setVectors(vectors(0.32f, 0.52f, 0.21f, 0.52f))
            .putAllPayload(
                Map.of(
                    "color", value("red"),
                    "rand_number", value(32)))
            .build(),
        PointStruct.newBuilder()
            .setId(id(2))
            .setVectors(vectors(0.42f, 0.52f, 0.67f, 0.632f))
            .putAllPayload(
                Map.of(
                    "color", value("black"),
                    "rand_number", value(53),
                    "extra_field", value(true)))
            .build());

UpdateResult updateResult = client.upsertAsync("my_collection", points).get();

Search for similar vectors

List<ScoredPoint> anush =
    client
        .searchAsync(
            SearchPoints.newBuilder()
                .setCollectionName("my_collection")
                .addAllVector(List.of(0.6235f, 0.123f, 0.532f, 0.123f))
                .setLimit(5)
                .build())
        .get();

Search for similar vectors with filtering condition

// import static convenience methods
import static io.qdrant.client.ConditionFactory.range;

List<ScoredPoint> points = client.searchAsync(SearchPoints.newBuilder()
  .setCollectionName("my_collection")
  .addAllVector(List.of(0.6235f, 0.123f, 0.532f, 0.123f))
  .setFilter(Filter.newBuilder()
    .addMust(range("rand_number", Range.newBuilder().setGte(3).build()))
    .build())
  .setLimit(5)
  .build()
).get();

โš–๏ธ LICENSE

Apache 2.0 ยฉ 2023

java-client's People

Contributors

anush008 avatar eddumelendez avatar russcam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

java-client's Issues

Support for recovery via snapshot

Unless I'm missing it somewhere, I don't see support for collection recovery via a snapshot. Is this support planned? Ideally it would support both reading the snapshot from a URI or uploading the snapshot via HTTP multi-part.

Vector value seem rounded

Hello,
I'm currently trying to add a QDrantEmbeddingStore to Lanchain4J (https://github.com/langchain4j) but I have a problem.
IT with testContainer keep failing because some Vector values seem rounded :
expected: Embedding { vector = [-0.086097434, 0.033505224, 0.017902793, ...
but was: Embedding { vector = [-0.08609743, 0.03350522, 0.017902792, ...

I don't know if it can be related to quantization, I did'nt enable quantization (but maybe it is enabled by default)
If anyone have a hint that would be appreciated.
Thanks

Default demo does not work

I am following the example in the readme.md running against a local docker instance of Quadrant. My code throws an exception as soon as I fire a command, e.g. create a collection. I see the following message:


java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]

	at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566)
	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547)
	at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:88)
	at qdrant.CrudTest.createCollection(CrudTest.java:31)


...

Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
	at io.grpc.Status.asRuntimeException(Status.java:537)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 000012040000000000000400100000000500004000000601000000
	at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1308)
	at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1378)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)

My code looks like

 QdrantGrpcClient lowLevelClient = QdrantGrpcClient
                .newBuilder("localhost")
                .build();
 client = new QdrantClient(lowLevelClient);

 Collections.VectorParams params = Collections.VectorParams
                .newBuilder()
                .setDistance(Collections.Distance.Cosine)
                .setSize(dim)
                .build();

Collections.CollectionOperationResponse response = client
                .createCollectionAsync(name, params)
                .get();

Handler dispatch failed: java.lang.NoSuchMethodError: 'com.google.protobuf.Internal$ProtobufList io.qdrant.client.grpc.Points$Vector.makeMutableCopy(com.google.protobuf.Internal$ProtobufList)'

I add the pom and write a test case to search points. And replace io.grpc with version 1.60.1 .

         <dependency>
            <groupId>io.qdrant</groupId>
            <artifactId>client</artifactId>
            <version>1.7.0</version>
            <exclusions>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-netty</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-stub</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-protobuf</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-services</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-netty-shaded</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.guava</groupId>
                    <artifactId>guava</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>33.0.0-jre</version>
        </dependency>

But I got the error as below.

Handler dispatch failed: java.lang.NoSuchMethodError: 'com.google.protobuf.Internal$ProtobufList io.qdrant.client.grpc.Points$Vector.makeMutableCopy(com.google.protobuf.Internal$ProtobufList)'

Search code

            Points.SearchPoints.Builder searchBuilder = Points.SearchPoints.newBuilder();
            searchBuilder.setCollectionName(COLLECTION_NAME)
                    .addAllVector(vector.stream().map(Double::floatValue).toList())
                    .setShardKeySelector(Points.ShardKeySelector.newBuilder()
                            .addShardKeys(Collections.ShardKey.newBuilder()
                                    .setKeyword(SHARD_KEY)
                                    .build())
                            .build())
                    .setLimit(topK)
                    .setScoreThreshold((float) minSimilarity)
                    .setWithPayload(Points.WithPayloadSelector.newBuilder()
                            .setEnable(true)
                            .build());

Cannot set shardKey when deleting vectors

For some reason, I want to delete the vectors.

Here is the code.

 qdrantClient.deleteVectorsAsync(collectionName, vectors, ids).get()

I get below error message.

io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Wrong input: No shards found for shard key
	at io.grpc.Status.asRuntimeException(Status.java:533)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
2024-02-05 13:16:38.540 [] ERROR i.q.c.QdrantClient[2628] -- Delete vectors operation failed

But I didn't find how to set shardKey. Anyone know it?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.