Git Product home page Git Product logo

ydb-dotnet-sdk's Introduction

Nuget

YDB .NET SDK

YDB client libraries for .NET.

Prerequisites

.NET 6 or .NET 7

Versioning

We follow the SemVer 2.0.0. In particular, we provide backward compatibility in the MAJOR releases. New features without loss of backward compatibility appear on the MINOR release. In the minor version, the patch number starts from 0. Bug fixes and internal changes are released with the third digit (PATCH) in the version.

Major version zero (0.y.z) is considered prerelease and do not guarantee any backward compatibility.

Installation

dotnet add package Ydb.Sdk

Usage

To begin your work with YDB, create an instance of Ydb.Sdk.Driver class:

var config = new DriverConfig(
    endpoint: endpoint, // Database endpoint, "grpcs://host:port"
    database: database, // Full database path
    credentials: credentialsProvider // Credentials provider, see "Credentials" section
);

using var driver = new Driver(
    config: config,
    loggerFactory: loggerFactory
);

await driver.Initialize(); // Make sure to await driver initialization

Credentials

YDB SDK provides several standard ways for authentication:

  1. Ydb.Sdk.Auth.AnonymousProvider. Anonymous YDB access, mainly for tests purposes.
  2. Ydb.Sdk.Auth.TokenProvider. Token authentication for OAuth-like tokens.
  3. Ydb.Sdk.Auth.StaticCredentialsProvider. Username and password based authentication.

For Yandex.Cloud specific authentication methods, consider using ydb-dotnet-yc.

TableClient

After you have driver instance, you can use it to create clients for different YDB services. The straightforward example of querying data may look similar to the following example:

// Create Ydb.Sdk.Table.TableClient using Driver instance.
using var tableClient = new TableClient(driver, new TableClientConfig());

// Execute operation on arbitrary session with default retry policy
var response = await tableClient.SessionExec(async session =>
{
    var query = @"
        DECLARE $id AS Uint64;

        SELECT
            series_id,
            title,
            release_date
        FROM series
        WHERE series_id = $id;
    ";

    return await session.ExecuteDataQuery(
        query: query,
        parameters: new Dictionary<string, YdbValue>
        {
            { "$id", YdbValue.MakeUint64(id) }
        },
        // Begin serializable transaction and commit automatically after query execution
        txControl: TxControl.BeginSerializableRW().Commit(),
    );
});

response.Status.EnsureSuccess();

var queryResponse = (ExecuteDataQueryResponse)response;
var resultSet = queryResponse.Result.ResultSets[0];

foreach (var row in resultSet.Rows)
{
    Console.WriteLine($"> Series, " +
        $"series_id: {(ulong?)row["series_id"]}, " +
        $"title: {(string?)row["title"]}, " +
        $"release_date: {(DateTime?)row["release_date"]}");
}

Examples

See examples folder

ydb-dotnet-sdk's People

Contributors

alexv-smirnov avatar asmyasnikov avatar humman avatar kirillkurdyukov avatar lipatovalexander avatar quantulion avatar rekby avatar sfsharapov avatar spuchin avatar xmasapple avatar zork33 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

ydb-dotnet-sdk's Issues

Error ClientTransportTimeout

When creating a session through Createsession, I get a ClientTransportTimeout error, regardless of the set timeout (was 20 minutes), there is no response from the server.

Through ydb cli, requests are executed successfully, the server responds. Initialize driver method works.
Version is Ydb.Sdk 0.1.0

var poolConfig = new SessionPoolConfig();
var fieldInfo = typeof(SessionPoolConfig).GetField("<CreateSessionTimeout>k__BackingField", BindingFlags.Instance | BindingFlags.NonPublic);
fieldInfo?.SetValue(poolConfig, TimeSpan.FromMinutes(20));

var tableClient = new TableClient(driver, new TableClientConfig(poolConfig));

var session = await tableClient.CreateSession(new CreateSessionSettings
{
    OperationTimeout = TimeSpan.FromMinutes(20),
    TransportTimeout = TimeSpan.FromMinutes(20),
});

bug: Transaction with more than 2 queries is always committed after second query

Bug Report

YDB dotnet SDK version: 0.2.0

Environment: Yandex Cloud Function, dotnet8

Current behavior:

Firstly, I faced the problem with logger. See https://github.com/ydb-platform/ydb-dotnet-sdk/blob/main/src/Ydb.Sdk/src/Services/Table/ExecuteDataQuery.cs#L68 and https://github.com/ydb-platform/ydb-dotnet-sdk/blob/main/src/Ydb.Sdk/src/Services/Table/Transaction.cs#L121. logger.LogTrace($"Using tx #{_txNum}"); throws ArgumentNullException. I fixed this and published new package, but it still doesn't work.

Now it seems that transaction is always committed after second query, even without calling .Commit() in TxControl.Tx(..) in second query.

Expected behavior:

I can do multiple (more than two) queries in one transaction using TxControl.

Steps to reproduce:

Add an intermediate query to this example. Like this:

var execResponse = await Client.SessionExec(async session =>
            {
                var query1 = "<some_query>";

                // Execute first query (no transaction commit)
                var response = await session.ExecuteDataQuery(
                    query: query1,
                    txControl: TxControl.BeginSerializableRW(),
                    parameters: new Dictionary<string, YdbValue>()
                );

                if (!response.Status.IsSuccess || response.Tx is null)
                {
                    return response;
                }

                var query2 = "<some_query>";

                // Execute second query (no transaction commit)
                response = await session.ExecuteDataQuery(
                    query: query2,
                    TxControl.Tx(response.Tx),
                    parameters: new Dictionary<string, YdbValue>()
                );

                # here transaction is committed and response.Tx is null 

                if (!response.Status.IsSuccess || response.Tx is null)
                {
                    return response;
                }

                var query3 = "<some_query>";

                // Execute third query and commit transaction
                response = await session.ExecuteDataQuery(
                    query: query3,
                    TxControl.Tx(response.Tx).Commit(),
                    parameters: new Dictionary<string, YdbValue>()
                );

                return response;
            });

            execResponse.Status.EnsureSuccess();

Remark: I use TableClient here. I tried to use QueryClient, but it doesn't work for me too. There was some error about endpoints (not related to this issue and transactions).

YdbValue.MakeBool

Since the YDB supports Bool value and YdbValue class has fields for the bools values, the method YdbValue.MakeBool is missing or I can create bool value with the different way?

Add SLO test workload and CI

CI example is here: https://github.com/ydb-platform/ydb-nodejs-sdk/blob/main/.github/workflows/slo.yml
Workload sample and description is here: https://github.com/ydb-platform/ydb-nodejs-sdk/tree/main/slo-workload
As you can see from the repository, you will need to create a directory for the workload package and provide a Dockerfile inside it.

Important note: Do not run SLO tests in pull requests. You will either have to open secrets for pull requests, or give rights to each contributor (which is not a safe method either). The way I see it, the best option is to run tests on the main branch

Migration to .NET 6 / Переход на .NET 6

I propose multitargeting this library to both .NET Core 3.1 and .NET 6.

I've not discovered any significant changes yet. However, it will allow developers to use the library with current .NET 6 framework, which is LTS version.

--

Предлагаю добавить поддержку шестой LTS версии .NET каркаса (фреймворка).

Пока не прекратится поддержка версии 3.1, можно собирать библиотеку и для .NET Core 3.1 и для .NET 6. Предварительно проблем не обнаружил.

MakeBool

Missing method to create boolean value

ClientTransferTimeoutException without additional information

`
var credentials = new ServiceAccountProvider(saFilePath: "Keys/key.json");

        await credentials.Initialize();
        var driver = new Driver(
            new DriverConfig(
                "endpoint",
                "database",
                credentials));
        await driver.Initialize();
        var tableClient = new TableClient(driver);
        var response = await tableClient.SessionExec(async session =>
        {
            var query = $@"
                        DECLARE $id AS String;
                        SELECT *
                        FROM <table>
                        WHERE id = $id;";

            return await session.ExecuteDataQuery(
                query: query,
                txControl: TxControl.BeginSerializableRW().Commit(),
                parameters: new Dictionary<string, YdbValue> { { "$id", YdbValue.MakeUtf8("id") } });
        });

        response.Status.EnsureSuccess();`

Throw exception
image

Topic control plane

Цель проекта

Реализация методов управления топиками в сервисе ydb topic:

Результат:

В SDK реализованы методы:
CreateTopic
DescribeTopic
DescribeConsumer
AlterTopic
DropTopic

Критерии приёмки:

  1. Реализованы все методы API
  2. Реализованы интеграционные тесты
  3. Интеграционные тесты проверяют правильность работы методов
  4. PR "зелёный" - проходят и новые интеграционные тесты и уже существующие тесты для SDK.

Особенности

Требуется написать обёртку над GRPC-методами, для удобной работы с ними из .NET.
Уровень сложности: простой

Проект предполагает участие 2 человек:

  1. Реализация методов и сопутствующих структур данных
  2. Тестирование кода

feat: ADO.NET

Community reference

ADO.NET must be used with QueryService (a.k.a API 2.0 OLTP)

Roadmap of Refactoring:

Roadmap QueryService 1.0.0 / fix bugs:

Roadmap ADO.NET (Directory Driver):

Backlog:

  • #99
  • feat: make min / max size in SessionPool

feat: implement query service

With two methods:

  1. Query("query text", query_params, optional transaction mode, lambda (result stream))
  2. QueryTx(optional transaction mode, lambda(tx))

lambda is retriable user callback. When use queryTx - transaction will commit by default and has way for rollback (exception + tx.Rollback()).

And see to spanner.

Roadmap:

  • Implement .Query() - method for single SQL execution without transaction creation
  • Implement .QueryTx() method for multiple SQL executions on same transaction
    waiting for Begin/Commit/Rollback transaction methods will be realized on server (at the moment transactions works using TableService methods)
  • Session attach/detach
  • Merge implementation of SessionPools from Query and Table services
    • Attach
    • Detach
  • Retryer
  • Examples
  • helper methods such as Select, SelectFirstOrDefault, NonQuery etc.

YdbValue.MakeDecimal

The decimal type is supported in YDB, but for some reason it is not possible to work with it from the driver.

dev: Fix query stream example

for sum something without allocate memory for full result: result.Add(Series.FromRow(row));

need change example: private async Task StreamSelect()

Send hint session-balancer

Sent hint "session-balancer" in header "x-ydb-client-capabilities" if server side balancing supported.

Error: Failed to discover initial endpoints

Ydb.Sdk.Driver+InitializationFailureException: Failed to discover initial endpoints\n ---> Grpc.Core.RpcException: Status(StatusCode="Unavailable", Detail="Error starting gRPC call. HttpRequestException: Resource temporarily unavailable (***secret***) SocketException: Resource temporarily unavailable", DebugException="System.Net.Http.HttpRequestException: Resource temporarily unavailable (***secret***)\n ---> System.Net.Sockets.SocketException (11): Resource temporarily unavailable\n at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)\n at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)\n at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)\n --- End of inner exception stack trace ---\n at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.AddHttp2ConnectionAsync(HttpRequestMessage request)\n at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.GetHttp2ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)\n at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)\n at Grpc.Shared.TelemetryHeaderHandler.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)\n at System.Net.Http.HttpMessageInvoker.<SendAsync>g__SendAsyncWithTelemetry|6_0(HttpMessageHandler handler, HttpRequestMessage request, CancellationToken cancellationToken)\n at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)")\n at Ydb.Sdk.Driver.DiscoverEndpoints()\n at Ydb.Sdk.Driver.Initialize()\n --- End of inner exception stack trace ---\n at Ydb.Sdk.Driver.Initialize()

That is, if you could not initialize the connection, then only restarting the application will help.

YDB Topic

Currently, there is no way to work with topics in the dotnet-sdk. Tell me, when can I expect this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.