snowflakedb / gosnowflake Goto Github PK

Issue description

Clean up logging

Placeholder issue. Keypair auth would be nice to have. It looks like the python implementation is here https://github.com/snowflakedb/snowflake-connector-python/blob/master/auth_keypair.py.

Issue description

I expected the API to have a way to quote identifiers (like table names), but it does not appear to.

Example code

I want to do something like this:

table := os.Getenv("TABLE_NAME")
query := fmt.Sprintf("SELECT id FROM %s WHERE FOO=?", snowflake.QuoteIdentifier(table))

Issue description

converter.go, line 170 should store a nil back into the "dest" pointer parameter. Instead it sets the pointer itself to nil.
The end result is a column value from the prior row gets passed to caller's next row if the next row's column value is nil.

Example code

Just change line 170 from

dest = nil

to

*dest = nil

Error log

N/A

Configuration

Both 1.1.0 and 1.1.1, any and all versions of golang and client OS's.

Issue description

http://jmoiron.github.io/sqlx/

Find out what it is and can be integrated or incorporated with Go Snowflake driver.

As of commit 56b1dfc for some reason you guys decided to copy-paste glog into your repo instead of using vendoring. Can you please not copy-paste the glog code in your project? Not only it forces us downstream to vendor this "vendored" code, but also it removes our ability to override the glog implementation used.

It's not clear to me why you did this but if it's because you intend to change glog, then re-vendor it and add this to your Gopkg.toml:

[[override]]
  name = "github.com/golang/glog"
  # Don't use upstream glog, force our fork instead.
  source = "https://github.com/snowflakedb/glog.git"
  branch = "master"

And then make whatever changes you want to your fork of the upstream repo.

Issue description

Makefile will be used for CI and Deployment

Issue description

BINARY fetch and binding support is required.

Issue description

Golang driver should raise error if the specified db, schema or warehouse doesn't exist.
The role is already validated, but not others are. This is more or less historical reason. no-error-for-invalid db behavior has been around since there was no concept of default db, schema or warehouse. Well, even no warehouse exists 3.5 years ago.

Issue description

Need to investigate if DecimalSize applies to Timestamp types?
https://golang.org/pkg/database/sql/#ColumnType.DecimalSize

Currently DecimalSize supports only FIXED data type.

Issue description

In a long running query, the token must be refreshed periodically to keep the connection alive.

Issue description

Build started failing.

>go get -u github.com/golang/lint/golint
package golang.org/x/lint: unrecognized import path "golang.org/x/lint" (parse https://golang.org/x/lint?go-get=1: no go-import meta tags ())

Issue description

Purpose is detect the regression

Issue description

We may need to have Snowflake date, time and timestamp formatters. Currently all data are converted into native Golang types in fetch operations. But if the client application wants to honor timestamp formats TIMESTAMP_OUTPUT_FORMAT, etc, the driver needs to parse the date format string and format date time data.

Issue description

Add Authenticator interface so that both Snowflake and OKTA authentication are supported consistently and future extension.

Issue description

glide is good, but go tool is likely to have dep in the near future.

Issue description

Tell us what should happen and what happens instead

Issue description

Opening a database connection without specifying a database results in an error. This worked previously. I think this is an unintended side effect of #124

Example code

test.go

package main

import (
	"database/sql"
	"log"
	// Snowflake SQL DB
	_ "github.com/snowflakedb/gosnowflake"
)

func main() {
	db, err := sql.Open("snowflake", "user:pass@hostname/?account=account")
	if err != nil {
		log.Fatal(err)
	}
	err = db.Ping()
	if err != nil {
		log.Fatal(err)
	}
}

Error log

PBennes's Macbook Pro:snowflake pbennes$ go run test.go
2018/05/26 20:08:33 260009 (08006): specified object doesn't exists: [public]
exit status 1

Configuration

Driver version (or git SHA): 4f647d6
Go version: go1.10.2 darwin/amd64
Client OS: Mac OS X

Issue description

https://github.com/bradfitz/go-sql-test

Issue description

Transaction support

Issue description

If you run tests with -race flag, you'll get errors. We should fix this by prod.

Issue description

Running go get results in an error.

Example code

# github.com/snowflakedb/gosnowflake
../../snowflakedb/gosnowflake/auth.go:107:36: multiple-value uuid.NewV4() in single-value context
../../snowflakedb/gosnowflake/authokta.go:217:53: multiple-value uuid.NewV4() in single-value context
../../snowflakedb/gosnowflake/restful.go:116:25: multiple-value uuid.NewV4() in single-value context
../../snowflakedb/gosnowflake/restful.go:233:36: multiple-value uuid.NewV4() in single-value context
../../snowflakedb/gosnowflake/restful.go:288:36: multiple-value uuid.NewV4() in single-value context
../../snowflakedb/gosnowflake/restful.go:355:36: multiple-value uuid.NewV4() in single-value context

Configuration

Driver version (or git SHA): master

Go version: go version go1.10 darwin/amd64

Client OS: OSX

Thanks,

Ben

Issue description

Wrong usage of channels. If the number of chunks is more than the channel's size, it will hang.

Issue description

Beter to have Ping and PingContext

Issue description

This requires the server side change.

Issue description

We've been receiving the following error from Snowflake:

390114: Authentication token has expired.  The user must authenticate again.

It's not being caught and handled in the driver. restful.go has

sessionExpiredCode       = "390112"

Perhaps the error number has changed, or this is a new variant which needs to be handled the same way?

Error log

As above

Configuration

Long running kafka->snowflake process.

Issue description

We might want to run fuzzing tests to harden the code base.

Go fuzz package is available here:
https://github.com/dvyukov/go-fuzz

Issue description

Implement following interfaces with context arugment passing in. This is the interface not deprecated. Also it is easier implementing cancel function.

This is the context package: https://golang.org/pkg/context/

Interfaces to implement

type ConnPrepareContext interface {
        // PrepareContext returns a prepared statement, bound to this connection.
        // context is for the preparation of the statement,
        // it must not store the context within the statement itself.
        PrepareContext(ctx context.Context, query string) (Stmt, error)
}

type ExecerContext interface {
        ExecContext(ctx context.Context, query string, args []NamedValue) (Result, error)
}

type QueryerContext interface {
        QueryContext(ctx context.Context, query string, args []NamedValue) (Rows, error)
}

type StmtExecContext interface {
        // ExecContext executes a query that doesn't return rows, such
        // as an INSERT or UPDATE.
        //
        // ExecContext must honor the context timeout and return when it is canceled.
        ExecContext(ctx context.Context, args []NamedValue) (Result, error)
}

type StmtQueryContext interface {
        // QueryContext executes a query that may return rows, such as a
        // SELECT.
        //
        // QueryContext must honor the context timeout and return when it is canceled.
        QueryContext(ctx context.Context, args []NamedValue) (Rows, error)
}

Issue description

Need to run CI with TravisCI

Issue description

PUT and GET support. This requires AWS SDK to communicate with S3.

Issue description

We may want to have more examples in cmd/ directory.

Issue description

When CLIENT_SESSION_KEEP_ALIVE is enabled, snowflake go driver starts a heartbeat goroutine every time func (sc *snowflakeConn) exec(.....) is called. It only stops the heartbeat when Close() is called on the connection.

If you send multiple queries without closing the connection, snowflake go driver leaks goroutines.

In the example below the same connection is used to create and query a temporary table. This example starts 6 heartbeat goroutines but stops only one.

Is this the expected behavior?

Example code

func Test_KeepAlive(t *testing.T) {
	db, err := NewConnection(NewSnowflakeConfig())
	assertNoError(t, err)
	defer db.Close()

	db.Exec(`create temporary table my_temp_table (col1 number, col2 varchar(32))`)
	for i := 1; i <= 3; i++ {
		db.Exec(`insert into my_temp_table values (?, ?)`, i, fmt.Sprintf("test_%v", i))
	}
	rows, err := db.Query(`select col1, col2 from my_temp_table`)
	assertNoError(t, err)
	defer rows.Close()
	var (
		id   int
		name string
	)
	for rows.Next() {
		rows.Scan(&id, &name)
		fmt.Printf("%v - %v\n", id, name)
	}
}

func NewConnection(cfg *gosnowflake.Config) (*sql.DB, error) {
	dsn, err := gosnowflake.DSN(cfg)
	if err != nil {
		return nil, err
	}
	db, err := sql.Open("snowflake", dsn)
	return db, err
}

func NewSnowflakeConfig() *gosnowflake.Config {
	cfg := &gosnowflake.Config{
		Account:   "xxx",
		User:      "ilkin",
		Password:  "xxxx",
		Warehouse: "xxx",
		Role:      "xxxx",
		Params:    make(map[string]*string),
	}
	var keepAlive = "true"
	cfg.Params["client_session_keep_alive"] = &keepAlive
	return cfg
}

func assertNoError(t *testing.T, err error) {
	if err != nil {
		t.Fatalf("unexpected error %v", err)
	}
}

Configuration

driver version : v1.1.9

go version go1.10.2 darwin/amd64

Server version: E.g. 1.90.1

SELECT CURRENT_VERSION();
CURRENT_VERSION()
-----------------
2.53.1
1 Row(s) produced. Time Elapsed: 1.465s

Client OS: Mac OS (High Sierra)

Issue description

Code coverage by CodeCov

Issue description

Currently time.Local is used to translate TIMESTAMP_LTZ data. To be consistent with other drivers, we should honor TIMEZONE session parameter to translate TIMESTAMP_LTZ.

Issue description

For database drivers special characters in the password are normally url-encoded using percents. This lets you pass passwords that contain a '@' or ':' into the driver without causing issues. This driver just interprets whatever is between the '@' and ':' in the connection string as the password, which will cause failures if the user's password contains one of those characters.

Issue description

If you try to ping a non-existent Snowflake account, you just get an HTTP timeout after about 60s. This is pretty long, and there is Config.LoginTimeout which cancels the HTTP request after another duration.

However, if you use a Context that expires, this does not get respected by PingContext and the LoginTimeout (if specified) or the raw HTTP timeout gets triggered instead.

Example code

config := gosnowflake.Config{
  // include other auth params
  // use an "account" that does not exist to demonstrate this problem
  LoginTimeout: 30 * time.Second,
}

url, err := gosnowflake.DSN(&config)
if err != nil {
  return err
}

db, err := sql.Open("snowflake", url)
if err != nil {
  return err
}

ctx, cancel := context.WithCancel(context.Background())
cancel() // cancel immediately rather than using timeout to prove that context is not respected

if err := db.PingContext(ctx); err != nil {
  // this will fail after ~30s, but it should fail immediately
  return err
}

Configuration

Driver version (or git SHA): 1.1.2

Go version: go version go1.9.2 linux/amd64

Issue description

Scan when destination is interface should produce values with appropriate types
What I see: it doesn't matter the type a column has in the db, it produces a string
What I expect to see: get an int/string etc according to the data type in the db
Same test changing the driver from snowflake to postgres produces int64 in out1

Example code

const conn = "<conn string>"
const query = "select 15"
func main() {
	db, _ := sql.Open("snowflake", conn)

	row := db.QueryRow(query)
	var out1 interface{}
	row.Scan(&out1)
	fmt.Printf("%T %v\n", out1, out1) // type string. I would expect int64 or similar

	row = db.QueryRow(query)
	var out2 int
        row.Scan(&out2)
	fmt.Printf("%T %v\n", out2, out2) // type int

	row = db.QueryRow(query)
	var out3 string
	row.Scan(&out3)
	fmt.Printf("%T %v\n", out3, out3) // type string
}

Error log

No visible error (no idea about the glog option)

Configuration

Driver version (or git SHA): v1.1.13-3-g911918e (output of git describe --tags)

Go version: go1.11.1 linux/amd64

Server version: 3.1.0

Client OS: Archilux 4.18.16-arch1-1-ARCH (output of uname -a)

Issue description

TIMESTAMP_TZ and TIMESTAMP_LTZ supports are required for bind and fetch.

Issue description

Currently dep and glide are used to manage the dependency. As dep becomes the standard tool, we should drop glide configs.

Issue description

When i use flag.Parse() in my project, it conflicts with glog package.

Example code

package main

import (
	"flag"
	"fmt"
	"log"
	"os"

	"database/sql"
	_ "github.com/snowflakedb/gosnowflake"
)

const Version = "0.0.1"

func main() {

	var version = flag.Bool("v", false, "prints current program version")

	if *version {
		fmt.Println(Version)
		os.Exit(0)
	}

	db, err := sql.Open("snowflake", "test_user:test_password@test_account/test_db")
	if err != nil {
		log.Fatal(err)
	}

	err = db.Ping()
	if err != nil {
		log.Fatal(err)
	}

	log.Print("Snowflake connection is successfully established.")
}

Error log

./main flag redefined: v
panic: ./main flag redefined: v

Configuration

Driver version (or git SHA): 584b6e1

Go version: go1.8.1 darwin/amd64

Client OS: Darwin Kernel Version 16.6.0

Issue description

All HTTP communication needs proxy support

Issue description

There are two ways to use proxy server, one of which is by the parameters in connection and other of which is by the environment variables HTTP_PROXY, HTTPS_PROXY and NO_PROXY.

One missing feature by the parameters is NO_PROXY. It is possible to implement gosnowflake own no_proxy, but we want to deprecate the use of parameters as there is no much value to have connection specific proxy. Instead, we recommend to use the environment variables that apply all connections in a process.

A benefit is the environment variables can be shared not only with other Snowflake components (SnowSQL, Python Connector, etc) but also other common tools, e.g., curl.

glog must be flushed otherwise no log shows up.

Issue description

TLS connection must be validated with OCSP/CRL (unless Go's library does)

Issue description

As of today, versoin.go has Go Snowflake version. We may have better way to maintain the versioning.

Issue description

must retry on 5xx and 4xx

gosnowflake depends on the third party UUID library (satori and Google) for the request id, which is used to track the client request. It is pain when the UUID breaks the API compatibility like the event happened half year ago on https://github.com/satori/go.uuid, so it would be nice to have our own UUID Implementation based on https://tools.ietf.org/html/rfc4122.

The requirement is not performance and resource intensive. UUID v4 should be good enough.

Issue description

Snowflake supports array binding, and JDBC, ODBC and Python connector supports it. We want to have one for Gosnowflake.

Example code

Looks like lib/pq package for Postgresql supports it.
http://agniva.me/go/2018/03/11/lib-pq.html

Issue description

When hitting ctrl+C, the query must stop.

Looks like logging is currently used only for debugging. Also, libraries using their own logging mechanisms are mostly frowned upon, here are some references:

• https://www.0value.com/about-Go-logging
• https://peter.bourgon.org/go-best-practices-2016/#logging-and-instrumentation
• go-kit/kit#42

To remedy this, maybe we could create a compatible wrapper for glog and separate the actual glog import/usage from a no-op one using build tags. Say, a build tag named sfdebug would enable the actual glog wrapper and compile with glog, and without it would just use a null-logger (to satisfy the logging calls during compile) and not import glog.

This would also fix #106 since flag.Parse() is called by glog.

Would you consider such a solution? If so, we could send in a PR and go from there.