ydb-platform / ydb-go-sdk Goto Github PK
View Code? Open in Web Editor NEWPure Go native and database/sql driver for YDB
Home Page: https://ydb.tech
License: Apache License 2.0
Pure Go native and database/sql driver for YDB
Home Page: https://ydb.tech
License: Apache License 2.0
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
ydb-go-sdk/internal/conn/pool.go
Line 74 in 710f840
I think about counter with implicit close object when counter = 0 is un-intuitive.
What about:
Drop coroutine with res.Recv() loop
Call res.Recv() as sub-call of res.NextResultSet()
Describe the Feature Request
I'd like to be able to configure already running Connection
with CustomOption
s and not allocate table.Client
on each call afterwards.
Describe Preferred Solution
New method that will return configured Conneciton
, that will have all underlying clients configured to the options provided.
Also, I'd like to make this Connection.Close
method to act on its own clients, and not on the base clients/connection.
This applies to custom clients (e.g., allocated by providing options to the Connection.Table
func) as well.
Describe Alternatives
I'll have to implement such wrapper myself. However, I'd like to ensure that custom clients will not close he ones related to underlying Connection
.
If the feature request is approved, would you be willing to submit a PR?
Yes
This need for jaeger traces
scanner.Scanner interface must check manually Is null for raw value
Алексей Мясников, [11 Jan 2022, 18:35:31]:
А расскажи кейс зачем это надо?
Я распросил про плюсовый клиент - там сложно как для пользователя, так и реализация
Как бы вопроизвести смогу, но с точки зрения пользовательского АПИ точно сверхсложно. Как просто сделать - думаю пока.
Точнее даже так: надо выдумать простое пользовательское АПИ. Я бы предложил что-то типа
db.Coordination().Lock("some_resource", count) // спим пока не залочимся
// делаем что надо
db.Coordination().Unlock("some_resource", count)
Ибо тема с сессиями асинхронными запросами-ответами - такое себе
Yaroslav Leonov, [11 Jan 2022, 22:12:13]:
слушай, ну тема стандартная. Есть некий процесс, который должен быть синглтоном в глобальном смысле. Примеры:
Алексей Мясников, [11 Jan 2022, 23:47:31]:
Yaroslav Leonov, [11 Jan 2022, 23:36:44]:
zk = KazooClient()
zk.start()
lock = zk.Lock("/lockpath", "my-identifier")
with lock: # blocks waiting for lock acquisition
# do something with the lock
https://github.com/apache/curator/tree/master/curator-examples
в репку куратора можешь посмотреть, там неплохо с звёздами
Алексей Мясников, [11 Jan 2022, 22:32:29]:
start
кажется лишний. Можно совместить с lock
Yaroslav Leonov, [11 Jan 2022, 22:33:41]:
не знаю почему так сделано. у zk очень сложный API и bug prone API
насколько я помню start там асинхронный, т.е. он future возвращает
типа код
f = zk.start()
f.wait()
spawn 100 threads
можно поменять на
zk.get('/')
spawn 100 threads
если тебе почему-то хочется стартовать треды после коннкета.
а, ну вот наверное разумный пример:
мне дёрнули GET /stop в API, я хочу приостановить работу. в zk я могу сделать zk.stop()
и лишние треды попадают. при неявном start()
в api мне код писать нужно
короче тут скорее дело вкуса
recipe package - go.etcd.io/etcd/client/v3/experimental/recipes - pkg.go.dev
etcd: https://pkg.go.dev/go.etcd.io/etcd/client/[email protected]/experimental/recipes
но у них есть election session и mutex
https://pkg.go.dev/go.etcd.io/etcd/client/[email protected]/concurrency
Ticket: KIKIMR-9116
_Значения ydb.Value возвращаются, например, в KeyRanges из DescribeTable. Сейчас нет никакой публичной функциональности для работы с этим значением (определение и приведение типа) - можно лишь "не глядя" прокинуть значение в другой запрос. Однако подобная функциональность есть в internal.
Дополнительно:
Типы данных YDB в go sdk удобно оформить как публичные интерфейсы (вытащить из internal, чтобы было и to, и from), чтобы было удобно работать со значениями_
приведи кусочек кода, который бы делал вам хорошо
Что-нибудь вот такое, например:
desc, err := session.DescribeTable(ctx, tableName)
if err != nil {
return err
}
for _, keyRange := range desc.KeyRanges {
// Value.String() сейчас отсутствует
from, err := keyRange.From.ToString()
if err != nil {
return err
}
to, err := keyRange.To.ToString()
if err != nil {
return err
}
// do something with from, to
}
или
...
from, ok := keyRange.From.(ydb.StringValue)
if !ok {
return xerrors.New("Invalid type")
}
fromString := from.Get()
...
Сейчас из ydb.Value невозможно получить что-нибудь, с чем можно работать. Можно только наоборот - из чего-нибудь сделать ydb.Value
А какой практический смысл в string представлении?
Вывести в лог или веб-морду пользователю?
String это к примеру. Понятно, что может быть и что-то другое.
В нашем случае мы хотели более-менее равномерно разбить весь интервал ключей, чтобы распараллелить обработку целой таблицы.
Кажется From и To в этом случае можно и нужно использовать As Is
Я сделал PR #75 поддержку String() для KeyRange и Value
(не замерджено в мастер, на ревью, тестировании)
Кажется понял о чем ты: если description возвращает 3 интервала [0,1],[2,99],[100,1000], ты вы хотите распилить его как [0,333],[334,666],[667,1000]
Тогда As Is не получится. Но надо ли хотеть такое?
Сериализация в string как ad-hoc поможет, но вам надо (если я правильно понимаю) пачка методов типа:
For now scanning into type, that defined as:
type MyCustomID uint64
Ends up with error:
scan row failed: types *MyCustomType is unknown
It would be cool to support such types (like json.Unmarshal does).
For now workaround for this is to typecast variable before sending to Scan:
type RowStruct struct {
ID MyCustomID
}
res.Scan((*uint64)(&row.ID))
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Let I have an optional string values
var my_nullable_string *string
and I need to insert or update a number of rows in a table.
I should put a structure declaration in the query:
DECLARE $values AS List<Struct<
id: String,
...
my_nullable_string: Optional<String>
>>;
and pass typed values in a slice to be passed to YDB. The problem is there's no straight way to make a types.StructFieldValue
out of nullable string value. So the only way I found is rather quirky and too verbose:
var val types.Value
if my_nullable_string == nil {
val = types.NullValue(types.TypeString)
} else {
val = types.OptionalValue(types.StringValueFromString(*my_nullable_string))
}
values = append(values, types.StructValue(
types.StructFieldValue("id", types.StringValueFromString(id)),
...
types.StructFieldValue("my_nullable_string", val),
)
Describe the Feature Request
It would be great if one could pass nullable values as easy as not nullable.
Describe Preferred Solution
Something like this:
values = append(values, types.StructValue(
types.StructFieldValue("id", types.StringValueFromString(id)),
...
types.StructFieldValue("my_nullable_string", types.OptionalStringValueFromNullableString(my_nullable_string))),
)
or even more universal solution like
values = append(values, types.StructValue(
types.StructFieldValue("id", types.StringValueFromString(id)),
...
types.StructFieldValue("my_nullable_string", types.OptionalValueFromNullable(types.TypeString, my_nullable_string)),
)
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
It helps with IDE driven development: IDE provide choice if type ydb
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Describe the Feature Request
I think that that if there could be some kind of ORM for YDB it will give significant boost to the project.
Describe Preferred Solution
I would like to have a look at the gorm.io. It will be great to have ydb driver for it.
Thanks! Great project!
At now we have two mutually exclusive options:
This feature provide Retry using middleware functions and provide changing context from parent to child middleware (it need for opentracing tree of calls)
This helps to support go1.18 typed parameters in the future
YDB GO SDK version:
Environment
Current behavior:
Expected behavior:
Steps to reproduce:
Related code:
insert short code snippets here
Other information:
YDB GO SDK version:. ydb-go-sdk/3.14.4
Environment
Current behavior: linux, macos
Expected behavior:
endpoints, err := conn.Discovery().Discover(ctx) // <-- return list of endpoints and no error
Steps to reproduce:
package main
import (
"context"
"errors"
"fmt"
"os"
"github.com/ydb-platform/ydb-go-sdk/v3"
"github.com/ydb-platform/ydb-go-sdk/v3/table"
)
func p(err error) {
if err != nil {
panic(err)
}
}
func main() {
ctx := context.Background()
options := []ydb.Option{
ydb.WithConnectionString(os.Getenv("YDB_CONN")),
ydb.WithInsecure(), // without TLS
ydb.WithAccessTokenCredentials(os.Getenv("YDB_TOKEN")),
}
conn, err := ydb.New(ctx, options...)
p(err)
var val int32
err = conn.Table().DoTx(ctx, func(ctx context.Context, tx table.TransactionActor) error {
res, err := tx.Execute(ctx, "SELECT 1 AS ping", nil)
p(err)
if !res.NextResultSet(ctx, "ping") {
return errors.New("no result set")
}
if !res.NextRow() {
return errors.New("no rows")
}
return res.Scan(&val)
},
table.WithTxSettings(table.TxSettings(table.WithSerializableReadWrite())),
)
p(err)
fmt.Println(val) // print result - for show about connection is ok
endpoints, err := conn.Discovery().Discover(ctx)
fmt.Println(len(endpoints)) // print 0
fmt.Println(err) // print error
}
//Output
1
0
transport error: unauthenticated: unauthenticated at `github.com/ydb-platform/ydb-go-sdk/v3/internal/conn.(*conn).Invoke(conn.go:328)`
YDB GO SDK version:
Environment
Current behavior:
Expected behavior:
Steps to reproduce:
Related code:
insert short code snippets here
Other information:
Related ticket: KIKIMR-14082
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
YDB GO SDK version:
v3.8.8
Current behavior:
Printing debug string "Exit 3" to stdout
Expected behavior:
Not print anything to stdout
Steps to reproduce:
Create new table session
Related code:
ctx := context.Background()
db, err := ydb.New(ctx,
ydb.WithDialTimeout(10*time.Second),
ydb.WithConnectionString("grpcs://ydb.serverless.yandexcloud.net:2135"),
yc.WithServiceAccountKeyFileCredentials("./SA-FILE", yc.WithSystemCertPool()),
ydb.WithDatabase(os.Getenv("YDB_DATABASE")),
)
if err != nil {
panic(err)
}
client := db.Table()
session, _ := client.CreateSession(ctx)
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Both sql.Scanner
and ydb.table.types.Scanner
provide scan to custom types
This action will broke back compatibility but make simple type switch in internal scanner logic
What do you think about it?
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
moved from yandex-cloud/ydb-go-sdk#6
YDB GO SDK version:
github.com/ydb-platform/ydb-api-protos v0.0.0-20210921091122-f697ac767e19
github.com/ydb-platform/ydb-go-genproto v0.0.0-20211103074319-526e57659e16
github.com/ydb-platform/ydb-go-sdk-metrics v0.4.1
github.com/ydb-platform/ydb-go-sdk-prometheus v0.4.1
github.com/ydb-platform/ydb-go-sdk-zap v0.3.0
github.com/ydb-platform/ydb-go-sdk-zerolog v0.3.0
github.com/ydb-platform/ydb-go-sdk/v3 v3.3.3
github.com/ydb-platform/ydb-go-yc v0.2.1
Environment
Linux x64, local ydb installation with enabled ydb zap logging
Current behavior:
simple insert causes panic at zap.String("tx", tx.ID()),
Expected behavior:
no panic
Steps to reproduce:
Other information:
t.OnSessionQueryExecute = func(info trace.ExecuteDataQueryStartInfo) func(trace.SessionQueryPrepareDoneInfo) {
session := info.Session
query := info.Query
tx := info.Tx
params := info.Parameters
log.Debug("executing",
zap.String("version", version),
zap.String("id", session.ID()),
zap.String("status", session.Status()),
zap.String("tx", tx.ID()), // PANIC
zap.String("yql", query.String()),
zap.String("params", params.String()),
)
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0xa522f1]
goroutine 1 [running]:
github.com/ydb-platform/ydb-go-sdk-zap.Table.func6({{0xecdff8, 0xc000392500}, {0xec77b8, 0xc000316000}, {0x0, 0x0}, {0xeca178, 0xc00038d8c0}, {0xebae80, 0xc0001162b8}})
/-S/vendor/github.com/ydb-platform/ydb-go-sdk-zap/table.go:183 +0x1f1
github.com/ydb-platform/ydb-go-sdk/v3/trace.Table.onSessionQueryExecute({0xc000389100, 0xc000389110, 0xc000389120, 0xc000389130, 0xc000389140, 0xc000389150, 0xc000389160, 0xc000389170, 0xc000389180, 0xc000389190, ...}, ...)
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/trace/table_gtrace.go:559 +0x55
github.com/ydb-platform/ydb-go-sdk/v3/trace.TableOnSessionQueryExecute({0xc000389100, 0xc000389110, 0xc000389120, 0xc000389130, 0xc000389140, 0xc000389150, 0xc000389160, 0xc000389170, 0xc000389180, 0xc000389190, ...}, ...)
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/trace/table_gtrace.go:854 +0x158
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.(*session).Execute(0xc000316000, {0xecdff8, 0xc000392500}, 0xbfcf80, {0xc0003c2000, 0x9a}, 0xc000316000, {0x0, 0x0, 0x0})
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/session.go:478 +0x1f4
a.yandex-team.ru/tasklet/experimental/internal/storage/ydb.(*Client).ExecuteWriteQuery.func1({0xecdff8, 0xc000392500}, {0x7f158c3c6b48, 0xc000316000})
/-S/tasklet/experimental/internal/storage/ydb/client.go:116 +0x59
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.retryBackoff({0xecdff8, 0xc000392500}, {0xed3af8, 0xc000578370}, {0xebccc0, 0xc00003a390}, {0xebccc0, 0xc00003a3a8}, 0x0, 0xc0006e4c00, ...)
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/retry.go:183 +0x31a
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.(*client).Do(0xc000578370, {0xecdff8, 0xc000392500}, 0xc0006cfaf8, {0x0, 0x0, 0xbcdd80})
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/client.go:540 +0x1d4
github.com/ydb-platform/ydb-go-sdk/v3.(*lazyTable).Do(0xc000102b98, {0xecdff8, 0xc000392500}, 0x2, {0x0, 0x0, 0x0})
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/table.go:23 +0x73
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
related ticket: KIKIMR-14293
==================
WARNING: DATA RACE
Write at 0x00c0004c4170 by goroutine 216:
runtime.racewrite()
<autogenerated>:1 +0x24
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.(*client).Close()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/client.go:517 +0x991
github.com/ydb-platform/ydb-go-sdk/v3/internal/lazy.(*lazyTable).Close()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/lazy/table.go:56 +0x170
github.com/ydb-platform/ydb-go-sdk/v3.(*connection).Close()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/connection.go:98 +0x5c1
a.yandex-team.ru/cloud/disk_manager/internal/pkg/persistence.(*YDBClient).Close()
/-S/cloud/disk_manager/internal/pkg/persistence/ydb.go:468 +0x57
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks/tasks_tests_test.TestTasksRunningTwoConcurrentTasksOnOneRunner·dwrap·10()
/-S/cloud/disk_manager/internal/pkg/tasks/tasks_tests/tasks_test.go:779 +0x58
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks/tasks_tests_test.TestTasksRunningTwoConcurrentTasksOnOneRunner()
/-S/cloud/disk_manager/internal/pkg/tasks/tasks_tests/tasks_test.go:804 +0x731
testing.tRunner()
/-S/contrib/go/_std/src/testing/testing.go:1259 +0x22f
testing.(*T).Run·dwrap·21()
/-S/contrib/go/_std/src/testing/testing.go:1306 +0x47
Previous read at 0x00c0004c4170 by goroutine 170:
runtime.raceread()
<autogenerated>:1 +0x24
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.(*client).closeSession()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/client.go:842 +0x5c
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.(*client).Put()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/client.go:375 +0x5fb
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.retryBackoff.func1()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/retry.go:234 +0xd9
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.retryBackoff()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/retry.go:264 +0x635
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.do()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/retry.go:122 +0x356
github.com/ydb-platform/ydb-go-sdk/v3/internal/table.(*client).Do()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/table/client.go:531 +0x1bd
github.com/ydb-platform/ydb-go-sdk/v3/internal/lazy.(*lazyTable).Do()
/-S/vendor/github.com/ydb-platform/ydb-go-sdk/v3/internal/lazy/table.go:39 +0xb1
a.yandex-team.ru/cloud/disk_manager/internal/pkg/persistence.(*YDBClient).Execute()
/-S/cloud/disk_manager/internal/pkg/persistence/ydb.go:550 +0x138
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks/storage.(*storageYDB).UpdateTask()
/-S/cloud/disk_manager/internal/pkg/tasks/storage/storage_ydb.go:386 +0x269
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.(*executionContext).updateState()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:143 +0x364
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.(*executionContext).ping()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:231 +0xe4
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.taskPinger()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:626 +0xbc
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.tryExecutingTask·dwrap·7()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:711 +0x71
Goroutine 216 (running) created at:
testing.(*T).Run()
/-S/contrib/go/_std/src/testing/testing.go:1306 +0x726
testing.runTests.func1()
/-S/contrib/go/_std/src/testing/testing.go:1598 +0x99
testing.tRunner()
/-S/contrib/go/_std/src/testing/testing.go:1259 +0x22f
testing.runTests()
/-S/contrib/go/_std/src/testing/testing.go:1596 +0x7ca
testing.(*M).Run()
/-S/contrib/go/_std/src/testing/testing.go:1504 +0x9d1
main.main()
/-B/cloud/disk_manager/internal/pkg/tasks/tasks_tests/_test_main.go:36 +0x22b
Goroutine 170 (running) created at:
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.tryExecutingTask()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:711 +0x633
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.(*runnerForRun).tryExecutingTask()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:490 +0x155
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.runnerLoop()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:864 +0x143
a.yandex-team.ru/cloud/disk_manager/internal/pkg/tasks.startRunner·dwrap·9()
/-S/cloud/disk_manager/internal/pkg/tasks/runner.go:896 +0x99
==================
Current result contains sets field for non-stream result and ch for stream results
This implementation is too difficult for understanding and provide more bugs
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Describe the Feature Request
Consider typical transactions usage:
func foo(ctx context.Context, s table.Session) error {
txControl := table.TxControl(table.BeginTx(table.WithSerializableReadWrite()))
tx, res, err := s.Execute(ctx, txControl, "SELECT ... ", ...)
// ...
defer tx.Rollback(ctx)
_, err = tx.Execute(ctx, "UPDATE ...", ...)
_, err = tx.CommitTx(ctx, ...)
return err
}
In case of CommitTx()
success result of tx.Rollback()
is known before the call
It seems, that current implementaion still executes a request to YDB
tskv unixtime=1636635598.026496 levelname=WARN name=ydb.driver.core caller=trace/driver_gtrace.go:729 message=invoke failed version=3.3.3 latency=1.665293ms address=<address> dataCenter=true method=/Ydb.Table.V1.TableService/RollbackTransaction error=ydb: operation error: NOT_FOUND: [{#2015 Transaction not found: fe709ffb-4afe6090-274c38cd-e2b585da}]
Since it's quite common practice to execute rollback unconditionally in defer
we can skip extra call to database and return well-known result.
Another snippet from ((https://github.com/golang/go/blob/master/src/database/sql/sql.go#L2303-L2306 database/sql))
func (tx *Tx) rollback(discardConn bool) error {
if !atomic.CompareAndSwapInt32(&tx.done, 0, 1) {
return ErrTxDone
}
Describe Preferred Solution
Add atomic value to track transaction end of life (committed or rollbacked)
Describe Alternatives
leave as is
Additional Context
Users not check res.Err()
because golang
primarily helps to watch error from function calls and obliges process errors explicitly
res.Err()
is an analogue of rows.Err()
(see sql.Rows.Err() )
But checking res.Err()
- is not a golang style
Ydb-go-sdk stream calls (such as StreamReadTable
and ExecuteScanQuery
) primarily returns nil (as err result). But on read results stage may be happens an error.
If user dont check res.Err()
after reads - some business logic errors are happens
This issue about draft:
type Result interface {
...
--- NextResultSet() bool
+++ NextResultSet() error
...
}
With reflection and struct tags declared in deprecated tool ydbgen
YDB GO SDK version:
v3.2.7
Environment
Ubuntu 20.04 on WSL 2
go version go1.17.2 linux/amd64
Current behavior:
When running db.Close(), it freezes and doesn't continue the program.
Expected behavior:
It should close connection and continue program.
Steps to reproduce:
package main
import (
"context"
"log"
"os"
"time"
"github.com/ydb-platform/ydb-go-sdk/v3"
"github.com/ydb-platform/ydb-go-sdk/v3/config"
yc "github.com/ydb-platform/ydb-go-yc"
)
func main() {
ctx := context.Background()
db, err := ydb.New(
ctx,
ydb.WithConnectionString(os.Getenv("YDB_CONNECTION_STRING")),
ydb.WithDialTimeout(5*time.Second),
ydb.WithBalancingConfig(config.BalancerConfig{
Algorithm: config.BalancingAlgorithmRandomChoice,
PreferLocal: false,
}),
yc.WithServiceAccountKeyFileCredentials(
"sa.json",
yc.WithDefaultEndpoint(),
yc.WithInsecureSkipVerify(true),
),
ydb.WithSessionPoolSizeLimit(300),
ydb.WithSessionPoolIdleThreshold(time.Second*5),
ydb.WithGrpcConnectionTTL(5*time.Second),
)
if err != nil {
panic(err)
}
log.Printf("Database: %s", db.Name())
log.Println("1")
_ = db.Close(ctx)
log.Println("2")
}
Related code:
Output:
2021/11/03 16:25:49 Database: /ru-central1/**/**
2021/11/03 16:25:49 1
(it doesn't print 2 and doesn't exit)
Other information:
Stacktrace when pausing program in debug mode in GoLand:
runtime.gopark at proc.go:337
runtime.chanrecv at chan.go:576
runtime.chanrecv1 at chan.go:439
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver/cluster/repeater.(*repeater).Stop.func1 at repeat.go:60
sync.(*Once).doSlow at once.go:68
sync.(*Once).Do at once.go:59
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver/cluster/repeater.(*repeater).Stop at repeat.go:57
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver/cluster.(*cluster).Close at cluster.go:94
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver/cluster.Cluster.Close-fm at cluster.go:68
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver.(*driver).Close at driver_as_cluster.go:36
github.com/ydb-platform/ydb-go-sdk/v3.(*db).Close at connection.go:77
main.main at main.go:41
runtime.main at proc.go:225
runtime.goexit at asm_amd64.s:1371
- Async Stack Trace
runtime.rt0_go at asm_amd64.s:226
YDB GO SDK version:
v3.2.7
Environment
Windows 10
go version go1.17.2 windows/amd64
Current behavior:
When trying to create new connection with ydb.New(...), program freezes.
Expected behavior:
It should create new connection and continue program.
Steps to reproduce:
package main
import (
"context"
"log"
"os"
"time"
"github.com/ydb-platform/ydb-go-sdk/v3"
"github.com/ydb-platform/ydb-go-sdk/v3/config"
yc "github.com/ydb-platform/ydb-go-yc"
)
func main() {
ctx := context.Background()
log.Println("1")
db, err := ydb.New(
ctx,
ydb.WithConnectionString(os.Getenv("YDB_CONNECTION_STRING")),
ydb.WithDialTimeout(5*time.Second),
ydb.WithBalancingConfig(config.BalancerConfig{
Algorithm: config.BalancingAlgorithmRandomChoice,
PreferLocal: false,
}),
yc.WithServiceAccountKeyFileCredentials(
"sa.json",
yc.WithDefaultEndpoint(),
yc.WithInsecureSkipVerify(true),
),
ydb.WithSessionPoolSizeLimit(300),
ydb.WithSessionPoolIdleThreshold(time.Second*5),
ydb.WithGrpcConnectionTTL(5*time.Second),
)
log.Println("2")
if err != nil {
panic(err)
}
defer func() {
_ = db.Close(ctx)
}()
log.Printf("Database: %s", db.Name())
}
Related code:
Output:
2021/11/03 16:37:24 1
(it doesn't print 2 and doesn't return any error)
Other information:
Stacktrace when pausing program in debug mode in GoLand:
runtime.gopark at proc.go:337
runtime.selectgo at select.go:327
google.golang.org/grpc.(*ClientConn).WaitForStateChange at clientconn.go:508
google.golang.org/grpc.DialContext at clientconn.go:316
github.com/ydb-platform/ydb-go-sdk/v3/internal/dial.(*dialer).dial at dialer.go:78
github.com/ydb-platform/ydb-go-sdk/v3/internal/dial.(*dialer).dial-fm at dialer.go:77
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver/cluster/balancer/conn.(*conn).take at conn.go:88
github.com/ydb-platform/ydb-go-sdk/v3/internal/driver/cluster/balancer/conn.(*conn).Invoke at conn.go:217
github.com/ydb-platform/ydb-go-genproto/Ydb_Discovery_V1.(*discoveryServiceClient).ListEndpoints at ydb_discovery_v1_grpc.pb.go:36
github.com/ydb-platform/ydb-go-sdk/v3/internal/discovery.(*client).Discover at discovery.go:61
github.com/ydb-platform/ydb-go-sdk/v3/internal/dial.(*dialer).discover at discovery.go:18
github.com/ydb-platform/ydb-go-sdk/v3/internal/dial.(*dialer).connect at dialer.go:63
github.com/ydb-platform/ydb-go-sdk/v3/internal/dial.Dial at dialer.go:37
github.com/ydb-platform/ydb-go-sdk/v3.New at connection.go:124
main.main at main.go:17
runtime.main at proc.go:225
runtime.goexit at asm_amd64.s:1371
- Async Stack Trace
runtime.rt0_go at asm_amd64.s:226
Now subtests of TestTable has dependency and real used as logical code block, but not tests.
What about extract common part to top-level code and split logical block by comments instead of subtests.
Related Code
var (
single trace.Details
multi trace.Details
err error
)
single, err = trace.ParseDetails("DriverNetEvents")
if err != nil {
panic(err)
}
if single != trace.DriverNetEvents {
panic("failed parsed single")
}
multi, err = trace.ParseDetails("DriverNetEvents", "DriverCoreEvents")
if err != nil {
panic(err)
}
if multi != (trace.DriverNetEvents | trace.DriverCoreEvents) {
panic("failed parsed multi")
}
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
... instead NextResultSet(ctx, names...) + Scan(values...)
This feature also helps to scan mixed modes required/Optional/optional_with_default_value_type
Describe the Feature Request
Describe Preferred Solution
Describe Alternatives
Related Code
Additional Context
If the feature request is approved, would you be willing to submit a PR?
Yes / No (Help can be provided if you need assistance submitting a PR)
Add missing WithPartitioningSettings
method to provide clear way to customise partitioning options with existing settings modifiers when creating table with CreateTable
method.
Now you can modify these settings with options.WithPartitioningSettingsObject
like so:
s.CreateTable(
ctx,
"table",
options.WithPartitioningSettingsObject(options.PartitioningSettings{PartitioningByLoad: options.FeatureEnabled})
)
But this method is not aligned with other CreateTableOptions.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.