shogo82148 / go-sql-proxy Goto Github PK

a proxy package is a proxy driver for dabase/sql.

License: MIT License

Go 100.00%

go-sql-proxy's Introduction

go-sql-proxy

The proxy package is a proxy driver for the database/sql package. You can hook SQL executions. It supports Go 1.8 or laster.

SYNOPSIS

Use Ready‐Made SQL tracer

proxy.RegisterTracer is a shortcut for registering a SQL query tracer.

package main

import (
	"context"
	"database/sql"

	"github.com/shogo82148/go-sql-proxy"
)

func main() {
	proxy.RegisterTracer()

	db, _ := sql.Open("origin:trace", "data source")
	db.Exec("CREATE TABLE t1 (id INTEGER PRIMARY KEY)")
	// STDERR: main.go:14: Exec: CREATE TABLE t1 (id INTEGER PRIMARY KEY); args = [] (0s)
}

Use proxy.NewTraceProxy to change the log output destination.

logger := New(os.Stderr, "", LstdFlags)
tracer := NewTraceProxy(&another.Driver{}, logger)
sql.Register("origin:trace", tracer)
db, err := sql.Open("origin:tracer", "data source")

Use with the context package

From Go version 1.8 onward, database/sql supports the context package. You can register your hooks into the context.

package main

import (
	"context"
	"database/sql"

	"github.com/shogo82148/go-sql-proxy"
)

var tracer = proxy.NewTraceHooks(proxy.TracerOptions{})

func main() {
	proxy.RegisterProxy()

	db, _ := sql.Open("origin:proxy", "data source")

	// The tracer is enabled in this context.
	ctx := proxy.WithHooks(context.Background(), tracer)
	db.ExecContext(ctx, "CREATE TABLE t1 (id INTEGER PRIMARY KEY)")
}

Create your own hooks

First, register new proxy driver.

hooks := &proxy.HooksContext{
	// Hook functions here(Open, Exec, Query, etc.)
	// See godoc for more details
}
sql.Register("new-proxy-name", proxy.NewProxyContext(&origin.Driver{}, hooks))

And then, open new database handle with the registered proxy driver.

db, err := sql.Open("new-proxy-name", "data source")

EXAMPLES

EXAMPLE: SQL tracer

package main

import (
	"context"
	"database/sql"
	"database/sql/driver"
	"log"

	"github.com/mattn/go-sqlite3"
	"github.com/shogo82148/go-sql-proxy"
)

func main() {
	sql.Register("sqlite3-proxy", proxy.NewProxyContext(&sqlite3.SQLiteDriver{}, &proxy.HooksContext{
		Open: func(_ context.Context, _ interface{}, conn *proxy.Conn) error {
			log.Println("Open")
			return nil
		},
		Exec: func(_ context.Context, _ interface{}, stmt *proxy.Stmt, args []driver.NamedValue, result driver.Result) error {
			log.Printf("Exec: %s; args = %v\n", stmt.QueryString, args)
			return nil
		},
		Query: func(_ context.Context, _ interface{}, stmt *proxy.Stmt, args []driver.NamedValue, rows driver.Rows) error {
			log.Printf("Query: %s; args = %v\n", stmt.QueryString, args)
			return nil
		},
		Begin: func(_ context.Context, _ interface{}, conn *proxy.Conn) error {
			log.Println("Begin")
			return nil
		},
		Commit: func(_ context.Context, _ interface{}, tx *proxy.Tx) error {
			log.Println("Commit")
			return nil
		},
		Rollback: func(_ context.Context, _ interface{}, tx *proxy.Tx) error {
			log.Println("Rollback")
			return nil
		},

	}))

	db, err := sql.Open("sqlite3-proxy", ":memory:")
	if err != nil {
		log.Fatalf("Open filed: %v", err)
	}
	defer db.Close()

	_, err = db.Exec(
		"CREATE TABLE t1 (id INTEGER PRIMARY KEY)",
	)
	if err != nil {
		log.Fatal(err)
	}
}

EXAMPLE: elapsed time logger

package main

import (
	"context"
	"database/sql"
	"database/sql/driver"
	"log"
	"time"

	"github.com/mattn/go-sqlite3"
	"github.com/shogo82148/go-sql-proxy"
)

func main() {
	sql.Register("sqlite3-proxy", proxy.NewProxyContext(&sqlite3.SQLiteDriver{}, &proxy.HooksContext{
		PreExec: func(_ context.Context, _ *proxy.Stmt, _ []driver.NamedValue) (interface{}, error) {
			// The first return value(time.Now()) is passed to both `Hooks.Exec` and `Hook.ExecPost` callbacks.
			return time.Now(), nil
		},
		PostExec: func(_ context.Context, ctx interface{}, stmt *proxy.Stmt, args []driver.NamedValue, _ driver.Result, _ error) error {
			// The `ctx` parameter is the return value supplied from the `Hooks.PreExec` method, and may be nil.
			log.Printf("Query: %s; args = %v (%s)\n", stmt.QueryString, args, time.Since(ctx.(time.Time)))
			return nil
		},
	}))

	db, err := sql.Open("sqlite3-proxy", ":memory:")
	if err != nil {
		log.Fatalf("Open filed: %v", err)
	}
	defer db.Close()

	_, err = db.Exec(
		"CREATE TABLE t1 (id INTEGER PRIMARY KEY)",
	)
	if err != nil {
		log.Fatal(err)
	}
}

LICENSE

This software is released under the MIT License, see LICENSE file.

godoc

See godoc for more information.

go-sql-proxy's People

Contributors

Stargazers

Watchers

go-sql-proxy's Issues

proxy mysql.MySQLDriver locked

sql.Register("mysql-proxy", proxy.NewProxyContext(&mysql.MySQLDriver{}, &proxy.HooksContext{ Query: func(_ context.Context, _ interface{}, stmt *proxy.Stmt, args []driver.NamedValue, rows driver.Rows) error { queryString:=stmt.QueryString //access stmt be locked log.Println("Query: %s; args = %v\n", queryString, args) return nil },

Implement database/sql/driver.Validator interface

Issue description

Go 1.15 will introduce a new Validator interface, which may be implemented by Conn to allow drivers to signal if a connection is valid or if it should be discarded:

https://tip.golang.org/pkg/database/sql/driver/#Validator

ref. go-sql-driver/mysql#1106

proxy mysql.MySQLDriver locked

`conn.go`: Returned `err` is overwritten to `nil` by `PostXXX` hooks when trace hook is set

Hi @shogo82148, thank you for your long-time contribution to this project.

Recently, I've been experiencing some issues with this package due to the changes introduced in the latest release: v0.7.0

Background

In #84, specifically conn.go, the contributor introduced named return values for returning the error from the conn methods.

The problem happens when the same err value is also assigned to the PostXXX hooks (when the trace hook is set), which somehow overwrites the original err value when the error is not nil.

This issue has made the conn methods return nil as the error value, most, if not all the time when the trace hook is set even when there should be an error returned (example). This behaviour could be considered a bug.

Perhaps it could be something to do with the current trace hooks' behaviour and how they handle the err passed in?

I'd be thankful to get some input from your side regarding this, thank you so much.

mistake the usage of context.Context

I'm checking whether the context.Context has done by the following code.

	select {
	default:
	case <-ctx.Done():
		return nil, ctx.Err()
	}

but I mistook the place to check.

https://github.com/golang/go/blob/release-branch.go1.8/src/database/sql/ctxutil.go#L38-L42
https://github.com/shogo82148/go-sql-proxy/blob/master/conn.go#L195-L202

Iterating over PCs is deprecated

https://docs.google.com/presentation/d/1Wcblp3jpfeKwA0Y4FOmj63PW52M_qmNqlQkNaLj0P5o/edit#slide=id.g1b2157b5d1_3_229

pcs := make([]uintptr, 32)
n := runtime.Callers(skip, pcs)
for i := 0; i < n; i++ {
	fn := runtime.FuncForPC(pcs[i])
	log.Println(fn.Name())
}

runtime.FuncForPC will be deprecated. we should rewrite it.

pcs := make([]uintptr, 32)
runtime.Callers(skip, pcs)
frames := runtime.CallersFrames(pcs)

for {
	f, more := frames.Next()
	log.Println(f.Func)
	if !more {
		break
	}
}

Drop Go 1.7 Support

database/sql of Go 1.7 does not support the context.
I will drop old interface which does not support the context.

Go-sql-proxy changes behavior of some sql driver

proxy.Conn implements only driver.Conn interface.

But connection of some sql driver (e.g. go-sql-driver/mysql ) have Query(string, []driver.Value) (driver.Rows, error) method. It implements both driver.Conn and driver.Queryer .

When conn.Query() called without conn.Prepare(), it makes some behavior difference between driver itself and proxied one.

proxy.Conn should implements also driver.Queryer .

Support Go 1.10

https://tip.golang.org/doc/go1.10#database/sql/driver

Drivers that currently hold on to the destination buffer provided by driver.Rows.Next should ensure they no longer write to a buffer assigned to the destination array outside of that call. Drivers must be careful that underlying buffers are not modified when closing driver.Rows.

Drivers that want to construct a sql.DB for their clients can now implement the Connector interface and call the new sql.OpenDB function, instead of needing to encode all configuration into a string passed to sql.Open.

Drivers that want to parse the configuration string only once per sql.DB instead of once per sql.Conn, or that want access to each sql.Conn's underlying context, can make their Driver implementations also implement DriverContext's new OpenConnector method.

Drivers that implement ExecerContext no longer need to implement Execer; similarly, drivers that implement QueryerContext no longer need to implement Queryer. Previously, even if the context-based interfaces were implemented they were ignored unless the non-context-based interfaces were also implemented.

To allow drivers to better isolate different clients using a cached driver connection in succession, if a Conn implements the new SessionResetter interface, database/sql will now call ResetSession before reusing the Conn for a new client.

RFC: Implement Pre/Post hooks

Can you please check this?
https://github.com/lestrrat/go-sql-proxy/commit/a5e2df5cf337fd54fd16abad0474142b6ac26140

What is this?

I want to check if this sort of change is acceptable before going any further. I added some hooks that allows me to put timers.

Pseudocode:

   &Hooks{
        PreExec: func( .... ) (interface{}, error) {
             log.Printf("Executing  ....")
             return time.Now(), nil // this time.Now() is passed to Post* func
        },
        PostExec: func(ctx interface{}, ...) error {
             log.Printf("elapsed time: %s", time.Since(ctx.(time.Time)).String())
             return nil
        }
   }

The point being that I want this to ALWAYS execute, regardless of what the other hooks are doing.

Does this sound like something you can consider merging?