qustavo / sqlhooks Goto Github PK

when using wraper dierver, code like this:

sql exec err for table name has double quotes,

when not using wraper, code like this

sql exec success, table name change ok

so, what is the problem?

There's no way to react when Query|Exec fails. Hooks interface should define an Error handler.

I've got a panic when using your library (2.1.0) with github.com/microsoft/[email protected] sql driver.

Stack trace

/usr/local/go/src/runtime/panic.go:770 +0x124
github.com/qustavo/sqlhooks/v2.composed.OnError({0x14000a5a650?, 0x2?, 0x104ad1740?}, {0x1051a44d8, 0x140009884e0}, {0x10517a120, 0x14000acade0}, {0x1043e2b1a, 0x23}, {0x1400073c940, ...})
    /Users/vvinogradov/go/pkg/mod/github.com/qustavo/sqlhooks/[email protected]/compose.go:52 +0xfc
github.com/qustavo/sqlhooks/v2.handlerErr({0x1051a44d8?, 0x140009884e0?}, {0x1051851f0?, 0x14000915908?}, {0x10517a120, 0x14000acade0}, {0x1043e2b1a?, 0x23?}, {0x1400073c940?, 0x2?, ...})
    /Users/vvinogradov/go/pkg/mod/github.com/qustavo/sqlhooks/[email protected]/sqlhooks.go:32 +0x8c
github.com/qustavo/sqlhooks/v2.(*Stmt).QueryContext(0x140009884b0, {0x1051a4430, 0x106abf780}, {0x14000534a50, 0x2, 0x2})
    /Users/vvinogradov/go/pkg/mod/github.com/qustavo/sqlhooks/[email protected]/sqlhooks.go:312 +0x1c4
database/sql.ctxDriverStmtQuery({0x1051a4430, 0x106abf780}, {0x1051a56c8, 0x140009884b0}, {0x14000534a50, 0x2, 0x2?})
    /usr/local/go/src/database/sql/ctxutil.go:82 +0x9c
database/sql.rowsiFromStatement({0x1051a4430, 0x106abf780}, {0x1051935e0, 0x1400073c900}, 0x1400095d640, {0x14000a91828, 0x2, 0x2})
    /usr/local/go/src/database/sql/sql.go:2836 +0x118
database/sql.(*DB).queryDC(0x14000856a90?, {0x1051a4430, 0x106abf780}, {0x0, 0x0}, 0x14000d8abd0, 0x14000c71e40, {0x1043e2b1a, 0x23}, {0x14000a91828, ...})
    /usr/local/go/src/database/sql/sql.go:1806 +0x26c
database/sql.(*DB).query(0x14000856a90, {0x1051a4430, 0x106abf780}, {0x1043e2b1a, 0x23}, {0x14000a91828, 0x2, 0x2}, 0x0?)
    /usr/local/go/src/database/sql/sql.go:1754 +0xb4
database/sql.(*DB).QueryContext.func1(0xf8?)
    /usr/local/go/src/database/sql/sql.go:1732 +0x54
database/sql.(*DB).retry(0x58?, 0x14000a91630)
    /usr/local/go/src/database/sql/sql.go:1566 +0x4c
database/sql.(*DB).QueryContext(0x104e3c2e0?, {0x1051a4430?, 0x106abf780?}, {0x1043e2b1a?, 0x1400014af00?}, {0x14000a91828?, 0x14000a91718?, 0x104312878?})
    /usr/local/go/src/database/sql/sql.go:1731 +0x94
database/sql.(*DB).Query(...)
    /usr/local/go/src/database/sql/sql.go:1745

I can't start transaction with non default isolation level because connection returned from sqlhooks.Conn wrapper doesn't implement driver.ConnBeginTx interface https://github.com/golang/go/blob/master/src/database/sql/ctxutil.go#L100

If we don't implement QueryerContext or Queryer interface for connection, db.Query() will always do statement "prepare" and statement "exec". These extra roundtrips can add unneeded latency.

Run this test program and check in wireshark: https://play.golang.org/p/wvmsGliLPpz

A change like this fixes this: surki@98e9bd8

I could send a PR if you would like this fixed here

I would like to execute SQL commands inside the hook methods. These commands must be on the same transaction of the original command. I mean the one triggered by the programmer using the database/sql API. I have to do this transparently. Do you think it is feseable and straightforward to implement? I guess I would have to add extra parameters to the hook methods' signatures. However, I am wondering how I could append these extra comands to the ongoing transaction.

Benchmarks show a 2x and 3x overhead (mysql and psql). This should decrease.

I was hoping the after hook would include the resulting query, as it is actually passed to the database.

So instead of showing the original query like

'select id, first, last from users where id=?', '12'

It would include

'select id, first, last from users where id=12'

Useful for debugging edge cases, logging of all actual statements (to rebuild a database or specific situation) and other things.

When connecting over a high-latency network, it becomes visible in my tracing that Prepare takes a while to execute when using github.com/go-sql-driver/mysql.

Does it sound like it makes sense to add hooks around Prepare, too?

i add onError hook, add collect driver.ErrSkip error, how about add error check before call onError

It'd be easier to integrate with other wrappers if a WrapDriver function was available, similar to ExpansiveWorlds/instrumentedsql

Tracing hooks seems to be the perfect use case for sqlhooks

Go 1.8 has added a bunch of new interfaces that need to be wrapped, all the context related functions pretty much.

The latest version has implemented some new interfaces defined in Go 1.8. It would be useful if the Go version required is mentioned in the README.

https://github.com/avelino/awesome-go

There have been a few changes to sqlhooks. We should probably tag the latest commit. I noticed this while trying out the dep tool.

I'm using the hooks to create audit_logs for my project, I'm using the idea of this lib https://github.com/gmhafiz/audit but in this lib, the author uses a personalized version of sqlhooks (u can check this in the go.mod) that receive a driver.Result that has the LastInsertId() that he uses to populate the RowId in his audit struct.

My goal is to be able to use these Hooks to get the ID of the last record inserted on my Postgres database in a similar way.

I've recently tried out sqlhooks and encountered a problem:

(/Users/dobegor/go/src/smartback/v2/service/label_service.go:65)
[2018-05-23 14:51:15]  Error 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '"labels"  WHERE (user_id = 0 AND is_system = 1 AND label IN (?,?,?,?))' at line 1

(/Users/dobegor/go/src/smartback/v2/service/label_service.go:65)
[2018-05-23 14:51:15]  [2.00ms]  SELECT * FROM "labels"  WHERE (user_id = 0 AND is_system = 1 AND label IN ('Inbox','Archive','Trash','Snoozed'))

As you can see, for some reason requests went into mysql without substitution.
This only occurs if I wrap the driver with sqlhooks:

	sql.Register("mysqlWithHooks", sqlhooks.Wrap(&mysql.MySQLDriver{}, &Hooks{}))

	db, err := gorm.Open("mysqlWithHooks", config.DB.DSN())
	if err != nil {
		return stacktrace.Propagate(err, "Failed to connect to DB")
	}

I wonder if I'm doing something wrong here.

V0 has some benchmarks, it would be good to have them back on v1

Great work! I think this could be great for data change notification events to various parts of an application. Is it possible to make the hooks pre or post execution with the ability to cancel a pre-hook query? For example, a use case might be to stop any query that contains "CREATE TABLE" for users that don't have that permission.

The hooks get entered twice for each sql query because from my understanding, the first hit to the hook get cancelled because of driver.ErrSkip "driver: skip fast-path; continue as if unimplemented" It seems to happen with sqlx.

More details is in the discussion over at data-dog DataDog/dd-trace-go#270

Right now a quick fix would be checking for driver.ErrSkip and ignore it in every execs and query

results, err := conn.execContext(ctx, query, args)
- if err != nil {
+ if err != nil && err != driver.ErrSkip {
    return results, handlerErr(ctx, conn.hooks, err, query, list...)
}

The other fix (from the thread) is to set interpolateParams=true is the database dsn.

In order to combine multiple hooks (log+tracing) a new (compose) hook can be implemented. It should accept a list of Hooks and chain them together.

ATM no linters are set, anything will do for now. Code could be improved later