ydb-platform / ydb-java-sdk Goto Github PK
View Code? Open in Web Editor NEWYDB Java SDK
Home Page: https://ydb.tech
License: Apache License 2.0
YDB Java SDK
Home Page: https://ydb.tech
License: Apache License 2.0
Hi guys,
There is a field keyRanges in TableDescription. I don't see that it is used in CreateTableRequest, though I would expect that somehow it would be mapped to partition_at_keys (though semantics differs).
Similar to Clickhouse JDBC driver
tech.ydb:ydb-sdk-core:2.0.1
Provides transitive vulnerable dependency maven:io.netty:netty-codec-http:4.1.63.Final
We have already query stats for method executeDataQuery #122, but don't have for executeScanQuery
It is necessary to modify the SessionRetryContext class to work correctly with transactions, considering autocommit and interactive modes
When a SyncReader is shutdown with full messageQueue, there is a single thread left which infinately waits trying to put a message into blocking queue which is full. This thread is not stopped on shutDown and holds references to messages and PartitionSessions. That leads to memory leaks.
SDK examples have a few helpers for integration tests with YDB service in docker container. These helpers contain similar code and do similar work.
It will be a good decision to make a reference implementation of a helper for integration tests. It would also be nice to have bindings to JUnit4/JUnit5 for easier use of the helper in tests.
https://github.com/ydb-platform/ydb-java-sdk/actions/runs/4832565129
Also we can reproduce freezing if we enable option TESTCONTAINERS_REUSE_ENABLE
We would like to have some additional settings for GrpcTransport
It is necessary to split the separate functionality into separate modules
There was attempt in v1, but not completed.
It would be great to have jpa support and not to write full queries by hand.
Some of grpc-java options may have various restrictions like grpc/grpc-java#7599
We need to describe all of options with recommendations how to use them
R2DBC (Reactive Relational Database Connectivity) is a reactive API specification for SQL databases. It provides a non-blocking and reactive way to interact with relational databases.
Roadmap below:
- SQL syntax
- YDB Data types map to Java Types, implement io.r2dbc.spi.Type
- other language-specific features YQL
- PostgreSQL R2DBC - https://github.com/pgjdbc/r2dbc-postgresql. The perfect solution from the founders of R2DBC.
- Clickhouse R2DBC - https://github.com/ClickHouse/clickhouse-java/tree/main/clickhouse-r2dbc. Community implementation. Don't support transactions, and more features.
- The driver implementation will be built using the YDB Java SDK v2.
- Connection != YDB Session. Because the connection is active until the close method is called.
Connection have next state - InTransaction, OutTransaction, Close.
- io.r2dbc.spi.ConnectionFactory
- io.r2dbc.spi.ConnectionFactoryMetadata
- io.r2dbc.spi.ConnectionFactoryProvider
- io.r2dbc.spi.Result
- io.r2dbc.spi.Row
- io.r2dbc.spi.RowMetadata
- io.r2dbc.spi.Batch
- io.r2dbc.spi.Connection
- io.r2dbc.spi.Statement
Testing
Documentation.
Release candidate.
Document the usage, configuration options, and any database-specific features of YDB R2DBC driver to help users and developers understand how to use it effectively.
CI example is here: https://github.com/ydb-platform/ydb-nodejs-sdk/blob/main/.github/workflows/slo.yml
Workload sample and description is here: https://github.com/ydb-platform/ydb-nodejs-sdk/tree/main/slo-workload
As you can see from the repository, you will need to create a directory for the workload package and provide a Dockerfile inside it.
Important note: Do not run SLO tests in pull requests. You will either have to open secrets for pull requests, or give rights to each contributor (which is not a safe method either). The way I see it, the best option is to run tests on the main branch
In the current development version (2.1.0-SNAPSHOT) the URL-style endpoints (e.g. those with grpc:// and grpcs:// prefixes) do not work as expected.
Sometimes we get a broken session and should close it gracefully, but now there are a few places where this is not done.
We already have the old implementation based on the experimental JDBC driver
https://github.com/yandex-cloud/ydb-java-sdk/tree/master/spring-data-jpa
I think it's possible to implement this connector using only pure YDB Java SDK
We have two types of table indexes in the table serivce
https://github.com/ydb-platform/ydb-api-protos/blob/master/protos/ydb_table.proto#L84
But in Java SDK we can't have api for creating asynchronous indexes
Maybe it will be a good solution to use https://micrometer.io/
Support the consistent snapshot-style copying on the set of tables, based on the available YDB CopyTables GRPC method.
A pre-requisite for Spark datasource support.
If a thread is waiting infinitely for a message to read from SyncReader and its transport is already stopped in another thread, reader stream will get transport error and won't be able to retry it because it uses the scheduler from stopped transport which is also already shut down. Therefore, it won't be shut down and the SyncReader.receive() method will run infinitely without any chance to complete
We want to add more easy way to use the coordination service, like it made in analogs (ZooKeeper, Curator etc. )
There is a possible situation where user response to a callback does not occur immediately. And before it actually happens, reader could restart reading session stream and there could even be other partition sessions with the same ids
Current implementation don't support polling operations
https://github.com/ydb-platform/ydb-java-sdk/blob/develop/core/src/main/java/tech/ydb/core/Operations.java#L55
We need to create API to work with long operations, including:
In the develop branch, build fails on test execution due to org.bouncycastle classes missing:
Errors:
ManagedChannelFactoryTest.customSslFactory:138 » Certificate No provider succeeded to generate a self-signed certificate. See debug log for the root cause.
Tests run: 50, Failures: 0, Errors: 1, Skipped: 0
------------------------------------------------------------------------
Reactor Summary:
Java SDK for YDB 2.1.0-SNAPSHOT .................... SUCCESS [ 0.996 s]
Java SDK Bill of Materials 2.1.0-SNAPSHOT .......... SUCCESS [ 0.069 s]
Auth API module 1.0.0 .............................. SUCCESS [ 1.427 s]
Core module of Java SDK for YDB 2.1.0-SNAPSHOT ..... FAILURE [ 3.661 s]
Tests common module 2.1.0-SNAPSHOT ................. SKIPPED
JUnit 4 support module 2.1.0-SNAPSHOT .............. SKIPPED
Table client implementation 2.1.0-SNAPSHOT ......... SKIPPED
Scheme client implementation 2.1.0-SNAPSHOT ........ SKIPPED
JUnit 5 support module 2.1.0-SNAPSHOT .............. SKIPPED
------------------------------------------------------------------------
BUILD FAILURE
The stacktraces contain the following:
Caused by: java.lang.NoClassDefFoundError: org/bouncycastle/jce/provider/BouncyCastleProvider
at io.grpc.netty.shaded.io.netty.handler.ssl.util.SelfSignedCertificate.<init>(SelfSignedCertificate.java:246)
... 33 more
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.jce.provider.BouncyCastleProvider
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
... 34 more
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.