qdrant / java-client Goto Github PK

View Code? Open in Web Editor NEW

27.0 6.0 5.0 346 KB

Official Java client for Qdrant

Home Page: https://qdrant.github.io/java-client

License: Apache License 2.0

Java 100.00%

grpc-client qdrant-client vector-database vector-search vector-search-engine

java-client's Issues

Http client instead of grpc

I am currently working on a project that requires the use of an HTTP client, and I have already done the implementation using langchain4j. However, I noticed that httpclient is not currently included in the library, and I wanted to inquire if there are any plans to include it in the future.

If it is not planned to be included, I would be grateful if you could recommend an alternative HTTP client that would work well with qdrant. I appreciate any insight or guidance you can provide in this regard.

Cannot set shardKey when deleting vectors

For some reason, I want to delete the vectors.

Here is the code.

 qdrantClient.deleteVectorsAsync(collectionName, vectors, ids).get()

I get below error message.

io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Wrong input: No shards found for shard key
	at io.grpc.Status.asRuntimeException(Status.java:533)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
2024-02-05 13:16:38.540 [] ERROR i.q.c.QdrantClient[2628] -- Delete vectors operation failed

But I didn't find how to set shardKey. Anyone know it?

Can not create Payload Index with datetime Type

Now I can not create Payload Index with datatime Type using javaclient, because of the following code.

public ListenableFuture<UpdateResult> createPayloadIndexAsync(
		String collectionName,
		String field,
		PayloadSchemaType schemaType,
		@Nullable PayloadIndexParams indexParams,
		@Nullable Boolean wait,
		@Nullable WriteOrderingType ordering,
		@Nullable Duration timeout
	) {
		CreateFieldIndexCollection.Builder requestBuilder = CreateFieldIndexCollection.newBuilder()
			.setCollectionName(collectionName)
			.setFieldName(field)
			.setWait(wait == null || wait);

		switch (schemaType) {
			case Keyword:
				requestBuilder.setFieldType(FieldType.FieldTypeKeyword);
				break;
			case Integer:
				requestBuilder.setFieldType(FieldType.FieldTypeInteger);
				break;
			case Float:
				requestBuilder.setFieldType(FieldType.FieldTypeFloat);
				break;
			case Geo:
				requestBuilder.setFieldType(FieldType.FieldTypeGeo);
				break;
			case Text:
				requestBuilder.setFieldType(FieldType.FieldTypeText);
				break;
			case Bool:
				requestBuilder.setFieldType(FieldType.FieldTypeBool);
				break;
			default:
				throw new IllegalArgumentException("Invalid schemaType: '" + schemaType + "'");
		}

Allow custom strings as IDs

Hi,

it seems only UUID and Long are supported as Point IDs. I need however to bring my own Strings as ids. What is the best solution here?

Thanks

Caused by: io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS

Hi. I run my qdrant service by docker and without ssl. when operate with java client it throws NotSslRecordException exception but python client is ok. How can I use java client without ssl? (because my product service did not use ssl)

--------------------------this java code does not work-----------------------------
@test
public void testCreateCollection() {
try {
QdrantClient client = new QdrantClient(QdrantGrpcClient.newBuilder("localhost").build());
Collections.CollectionOperationResponse response = client.createCollectionAsync("my_collection",
Collections.VectorParams.newBuilder()
.setDistance(Collections.Distance.Cosine)
.setSize(4)
.build()).get();
} catch (Exception e) {
//here catch io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS
e.printStackTrace();
}
}

--------------------------this python code works-----------------------------
from qdrant_client import QdrantClient
from qdrant_client.http.models import Distance, VectorParams

client = QdrantClient(host="127.0.0.1", port=6333, grpc_port=6334, prefer_grpc=False)
client.create_collection(
collection_name="test_collection1",
vectors_config=VectorParams(size=4, distance=Distance.DOT),
)

Cannot access com.google.protobuf.GeneratedMessageV3

Getting the following error.

Cannot access com.google.protobuf.GeneratedMessageV3

io.qdrant client 1.7.1 org.springframework.boot spring-boot-starter-parent 3.3.0-SNAPSHOT

Handler dispatch failed: java.lang.NoSuchMethodError: 'com.google.protobuf.Internal$ProtobufList io.qdrant.client.grpc.Points$Vector.makeMutableCopy(com.google.protobuf.Internal$ProtobufList)'

I add the pom and write a test case to search points. And replace io.grpc with version 1.60.1 .

         <dependency>
            <groupId>io.qdrant</groupId>
            <artifactId>client</artifactId>
            <version>1.7.0</version>
            <exclusions>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-netty</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-stub</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-protobuf</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-services</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-netty-shaded</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.guava</groupId>
                    <artifactId>guava</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>33.0.0-jre</version>
        </dependency>

But I got the error as below.

Handler dispatch failed: java.lang.NoSuchMethodError: 'com.google.protobuf.Internal$ProtobufList io.qdrant.client.grpc.Points$Vector.makeMutableCopy(com.google.protobuf.Internal$ProtobufList)'

Search code

            Points.SearchPoints.Builder searchBuilder = Points.SearchPoints.newBuilder();
            searchBuilder.setCollectionName(COLLECTION_NAME)
                    .addAllVector(vector.stream().map(Double::floatValue).toList())
                    .setShardKeySelector(Points.ShardKeySelector.newBuilder()
                            .addShardKeys(Collections.ShardKey.newBuilder()
                                    .setKeyword(SHARD_KEY)
                                    .build())
                            .build())
                    .setLimit(topK)
                    .setScoreThreshold((float) minSimilarity)
                    .setWithPayload(Points.WithPayloadSelector.newBuilder()
                            .setEnable(true)
                            .build());

Unable to create a Native image (GraalVM) using Qdrant Java libraries.

The Qdrant Java client libraries rely on the grpc-java project (https://github.com/grpc/grpc-java), which lacks support for native compilation due to a shaded Netty dependency.
This issue is widespread and affects multiple Java libraries.

An issue was filed in the grpc-java project about this, but eventually closed as not planned: grpc/grpc-java#10601.

It would be good to find a general solution for this.

GraalVM version used:

native-image --version
native-image 21.0.3 2024-04-16
GraalVM Runtime Environment Liberica-NIK-23.1.3-1 (build 21.0.3+10-LTS)
Substrate VM Liberica-NIK-23.1.3-1 (build 21.0.3+10-LTS, serial gc)

Errors while trying to compile a native image:

11:07:23.637 [ForkJoinPool-2-worker-2] ERROR io.grpc.netty.shaded.io.netty.handler.ssl.BouncyCastleAlpnSslUtils -- Unable to initialize BouncyCastleAlpnSslUtils.
java.lang.ClassNotFoundException: org.bouncycastle.jsse.BCSSLEngine
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoader.loadClass(NativeImageClassLoader.java:637)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:421)
	at java.base/java.lang.Class.forName(Class.java:412)
	at io.grpc.netty.shaded.io.netty.handler.ssl.BouncyCastleAlpnSslUtils.<clinit>(BouncyCastleAlpnSslUtils.java:63)
	at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
	at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationSupport.ensureClassInitialized(ClassInitializationSupport.java:177)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ProvenSafeClassInitializationSupport.computeInitKindAndMaybeInitializeClass(ProvenSafeClassInitializationSupport.java:348)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ProvenSafeClassInitializationSupport.computeInitKindAndMaybeInitializeClass(ProvenSafeClassInitializationSupport.java:76)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationSupport.maybeInitializeAtBuildTime(ClassInitializationSupport.java:161)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationSupport.maybeInitializeAtBuildTime(ClassInitializationSupport.java:150)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.isInitialized(SVMHost.java:318)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.isInitialized(AnalysisType.java:911)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.maybeEagerlyInitialize(BytecodeParser.java:4423)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1679)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1672)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5416)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3426)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.iterateBytecodesForBlock(SharedGraphBuilderPhase.java:743)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3386)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3228)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1137)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.build(SharedGraphBuilderPhase.java:162)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1029)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:101)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:116)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:146)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.parseGraph(AnalysisMethod.java:895)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsedHelper(AnalysisMethod.java:860)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:843)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:186)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:621)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:167)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:153)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraphInfo(MethodTypeFlow.java:111)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:114)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.lambda$addObserver$0(TypeFlow.java:475)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:187)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:171)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
[2/8] Performing analysis...  []                                                                       (324.2s @ 3.61GB)
   43,829 reachable types   (89.1% of   49,183 total)
   81,782 reachable fields  (65.2% of  125,390 total)
  218,772 reachable methods (61.0% of  358,401 total)
   12,695 types, 1,137 fields, and 10,037 methods registered for reflection
       12 native libraries: -framework Accelerate, -framework Carbon, -framework Cocoa, -framework CoreServices, -framework JavaRuntimeSupport, -framework Metal, -framework OpenGL, -framework QuartzCore, -framework Security, m, stdc++

Error: Classes that should be initialized at run time got initialized during image building:
 org.apache.commons.logging.LogFactory was unintentionally initialized at build time. To see why org.apache.commons.logging.LogFactory got initialized use --trace-class-initialization=org.apache.commons.logging.LogFactory
ch.qos.logback.core.util.StatusPrinter was unintentionally initialized at build time. To see why ch.qos.logback.core.util.StatusPrinter got initialized use --trace-class-initialization=ch.qos.logback.core.util.StatusPrinter
io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator'). To see why io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator
io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf
io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf'). To see why io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf
io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf
ch.qos.logback.core.util.Loader was unintentionally initialized at build time. To see why ch.qos.logback.core.util.Loader got initialized use --trace-class-initialization=ch.qos.logback.core.util.Loader
ch.qos.logback.core.status.StatusBase was unintentionally initialized at build time. To see why ch.qos.logback.core.status.StatusBase got initialized use --trace-class-initialization=ch.qos.logback.core.status.StatusBase
io.grpc.netty.shaded.io.netty.handler.ssl.PemValue the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted). To see why io.grpc.netty.shaded.io.netty.handler.ssl.PemValue got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.handler.ssl.PemValue
ch.qos.logback.classic.Level was unintentionally initialized at build time. To see why ch.qos.logback.classic.Level got initialized use --trace-class-initialization=ch.qos.logback.classic.Level
org.slf4j.LoggerFactory was unintentionally initialized at build time. To see why org.slf4j.LoggerFactory got initialized use --trace-class-initialization=org.slf4j.LoggerFactory
io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf
ch.qos.logback.core.CoreConstants was unintentionally initialized at build time. To see why ch.qos.logback.core.CoreConstants got initialized use --trace-class-initialization=ch.qos.logback.core.CoreConstants
ch.qos.logback.core.status.InfoStatus was unintentionally initialized at build time. To see why ch.qos.logback.core.status.InfoStatus got initialized use --trace-class-initialization=ch.qos.logback.core.status.InfoStatus
io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted). To see why io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey
io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf
io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator'). To see why io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator
org.apache.commons.logging.impl.SLF4JLogFactory was unintentionally initialized at build time. To see why org.apache.commons.logging.impl.SLF4JLogFactory got initialized use --trace-class-initialization=org.apache.commons.logging.impl.SLF4JLogFactory
io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf
io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf
io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil'). To see why io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil
ch.qos.logback.classic.Logger was unintentionally initialized at build time. To see why ch.qos.logback.classic.Logger got initialized use --trace-class-initialization=ch.qos.logback.classic.Logger
io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-common/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted'). To see why io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted
io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf
io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf
To see how the classes got initialized, use --trace-class-initialization=org.apache.commons.logging.LogFactory,ch.qos.logback.core.util.StatusPrinter,io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator,io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf,io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf,io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf,ch.qos.logback.core.util.Loader,ch.qos.logback.core.status.StatusBase,io.grpc.netty.shaded.io.netty.handler.ssl.PemValue,ch.qos.logback.classic.Level,org.slf4j.LoggerFactory,io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf,ch.qos.logback.core.CoreConstants,ch.qos.logback.core.status.InfoStatus,io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey,io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf,io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator,org.apache.commons.logging.impl.SLF4JLogFactory,io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf,io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf,io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil,ch.qos.logback.classic.Logger,io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted,io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf,io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf
------------------------------------------------------------------------------------------------------------------------
                       82.6s (8.0% of total time) in 147 GCs | Peak RSS: 7.76GB | CPU load: 1.12
========================================================================================================================
Finished generating 'data-loader' in 5m 57s.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

PROPOSAL: Move from ListenableFuture<V> to CompletableFuture<T>

Originally posted by @Nowheresly in #30

Default demo does not work

I am following the example in the readme.md running against a local docker instance of Quadrant. My code throws an exception as soon as I fire a command, e.g. create a collection. I see the following message:


java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]

	at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566)
	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547)
	at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:88)
	at qdrant.CrudTest.createCollection(CrudTest.java:31)


...

Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
	at io.grpc.Status.asRuntimeException(Status.java:537)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 000012040000000000000400100000000500004000000601000000
	at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1308)
	at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1378)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)

My code looks like

 QdrantGrpcClient lowLevelClient = QdrantGrpcClient
                .newBuilder("localhost")
                .build();
 client = new QdrantClient(lowLevelClient);

 Collections.VectorParams params = Collections.VectorParams
                .newBuilder()
                .setDistance(Collections.Distance.Cosine)
                .setSize(dim)
                .build();

Collections.CollectionOperationResponse response = client
                .createCollectionAsync(name, params)
                .get();

Support for recovery via snapshot

Unless I'm missing it somewhere, I don't see support for collection recovery via a snapshot. Is this support planned? Ideally it would support both reading the snapshot from a URI or uploading the snapshot via HTTP multi-part.

Vector value seem rounded

Hello,
I'm currently trying to add a QDrantEmbeddingStore to Lanchain4J (https://github.com/langchain4j) but I have a problem.
IT with testContainer keep failing because some Vector values seem rounded :
expected: Embedding { vector = [-0.086097434, 0.033505224, 0.017902793, ...
but was: Embedding { vector = [-0.08609743, 0.03350522, 0.017902792, ...

I don't know if it can be related to quantization, I did'nt enable quantization (but maybe it is enabled by default)
If anyone have a hint that would be appreciated.
Thanks

I had to add "implementation("com.google.protobuf:protobuf-java:3.24.0")"

I use kotlin. I had to add line implementation("com.google.protobuf:protobuf-java:3.24.0") to my build.gradle.kts to make qdrant java-client works.

Now I have this 2 lines:

implementation("io.qdrant:client:1.9.1")
implementation("com.google.protobuf:protobuf-java:3.24.0")

Secund think, I had to use exacly version 3.24.0, witch is pointed in https://github.com/qdrant/java-client/blob/master/build.gradle#L81

Maybe it because kotlin, but I think it is worth to say it in documentation.

create collection fail when use java-client

Using the Qdrant Cloud Free tier

my code is ：
client.createCollectionAsync("MyCollecton", Collections.VectorParams.newBuilder() .setDistance(Collections.Distance.Cosine).setSize(1536).build()).get();

and the following exception was thrown after execution.

io.qdrant.client.QdrantClient : Create collection operation failed

io.grpc.StatusRuntimeException: UNIMPLEMENTED: HTTP status code 404
invalid content-type: text/plain; charset=utf-8
headers: Metadata(:status=404,content-type=text/plain; charset=utf-8,x-content-type-options=nosniff,date=Mon, 13 May 2024 10:30:18 GMT,content-length=19)
DATA-----------------------------
404 page not found

at io.grpc.Status.asRuntimeException(Status.java:537)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 common frames omitted

Is there something i missed?

Maven
io.grpc 1.59.1
io.qdrant 1.9.0
com.google.protobuf 3.24.0
guava 33.2.0-jre

Provide Testcontainers modules

Hi, there is community interest about Qdrant module for Testcontainers Java and .NET. I just found Testcontainers is being used to test the client libraries for Java and .Net. It would be nice to create a separate project so the community can use it as well. Nowadays, Testcontainers offers the Module Catalog for discoverability.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.