Git Product home page Git Product logo

gosql's Introduction

gosql

The package based on sqlx, It's simple and keep simple

Build Status codecov Go Report Card
GoDoc

⚠️ Because of some disruptive changes, The current major version is upgraded to V2,If you continue with V1, you can check out the v1 branches https://github.com/ilibs/gosql/tree/v1

V2 ChangeLog

  • Remove the second argument to the Model() and Table() functions and replace it with WithTx(tx)
  • Remove Model interface DbName() function,use the Use() function
  • Uniform API design specification, see APIDESIGN
  • Relation add connection:"db2" struct tag, Solve the cross-library connection problem caused by deleting DbName()
  • Discard the WithTx function

Usage

Connection database and use sqlx original function,See the https://github.com/jmoiron/sqlx

import (
    _ "github.com/go-sql-driver/mysql" //mysql driver
    "github.com/ilibs/gosql/v2"
)

func main(){
    configs := make(map[string]*gosql.Config)

    configs["default"] = &gosql.Config{
        Enable:  true,
        Driver:  "mysql",
        Dsn:     "root:123456@tcp(127.0.0.1:3306)/test?charset=utf8&parseTime=True&loc=Asia%2FShanghai",
        ShowSql: true,
    }

    //connection database
    gosql.Connect(configs)
    gosql.QueryRowx("select * from users where id = 1")
}

Use default database, So you can use wrapper function

//Exec
gosql.Exec("insert into users(name,email,created_at,updated_at) value(?,?,?,?)","test","[email protected]",time.Now(),time.Now())

//Queryx
rows,err := gosql.Queryx("select * from users")
for rows.Next() {
    user := &Users{}
    err = rows.StructScan(user)
}
rows.Close()

//QueryRowx
user := &Users{}
err := gosql.QueryRowx("select * from users where id = ?",1).StructScan(user)

//Get
user := &Users{}
err := gosql.Get(user,"select * from users where id = ?",1)

//Select
users := make([]Users)
err := gosql.Select(&users,"select * from users")

//Change database
db := gosql.Use("test")
db.Queryx("select * from tests")

You can also set the default database connection name

gosql.SetDefaultLink("log")
gosql.Connect(configs)

gosql.Get etc., will use the configuration with the connection name log

Using struct

type Users struct {
	Id        int       `db:"id"`
	Name      string    `db:"name"`
	Email     string    `db:"email"`
	Status    int       `db:"status"`
	CreatedAt time.Time `db:"created_at"`
	UpdatedAt time.Time `db:"updated_at"`
}

func (u *Users) TableName() string {
	return "users"
}

func (u *Users) PK() string {
	return "id"
}

//Get
user := &Users{}
gosql.Model(user).Where("id=?",1).Get()

//All
user := make([]Users,0)
gosql.Model(&user).All()

//Create and auto set CreatedAt
gosql.Model(&User{Name:"test",Email:"[email protected]"}).Create()

//Update
gosql.Model(&User{Name:"test2",Email:"[email protected]"}).Where("id=?",1).Update()
//If you need to update the zero value, you can do so
gosql.Model(&User{Status:0}).Where("id=?",1).Update("status")

//Delete
gosql.Model(&User{}).Where("id=?",1).Delete()

If you use struct to generate where conditions

//Get where id = 1 and name = "test1"
user := &Users{Id:1,Name:"test1"}
gosql.Model(&user).Get()

//Update default use primary key as the condition
gosql.Model(&User{Id:1,Name:"test2"}).Update()
//Use custom conditions
//Builder => UPDATE users SET `id`=?,`name`=?,`updated_at`=? WHERE (status = ?)
gosql.Model(&User{Id:1,Name:"test2"}).Where("status = ?",1).Update()

//Delete
gosql.Model(&User{Id:1}).Delete()

But the zero value is filtered by default, you can specify fields that are not filtered. For example

user := &Users{Id:1,Status:0}
gosql.Model(&user).Get("status")

You can use the genstruct tool to quickly generate database structs

Transaction

The Tx function has a callback function, if an error is returned, the transaction rollback

gosql.Tx(func(tx *gosql.DB) error {
    for id := 1; id < 10; id++ {
        user := &Users{
            Id:    id,
            Name:  "test" + strconv.Itoa(id),
            Email: "test" + strconv.Itoa(id) + "@test.com",
        }
		
		//v2 support, do some database operations in the transaction (use 'tx' from this point, not 'gosql')
        tx.Model(user).Create()

        if id == 8 {
            return errors.New("interrupt the transaction")
        }
    }

    //query with transaction
    var num int
    err := tx.QueryRowx("select count(*) from user_id = 1").Scan(&num)

    if err != nil {
        return err
    }

    return nil
})

If you need to invoke context, you can use gosql.Txx

Now support gosql.Begin() or gosql.Use("other").Begin() for example:

tx, err := gosql.Begin()
if err != nil {
    return err
}

for id := 1; id < 10; id++ {
    _, err := tx.Exec("INSERT INTO users(id,name,status,created_at,updated_at) VALUES(?,?,?,?,?)", id, "test"+strconv.Itoa(id), 1, time.Now(), time.Now())
    if err != nil {
        return tx.Rollback()
    }
}

return tx.Commit()

Automatic time

If your fields contain the following field names, they will be updated automatically

AUTO_CREATE_TIME_FIELDS = []string{
    "create_time",
    "create_at",
    "created_at",
    "update_time",
    "update_at",
    "updated_at",
}
AUTO_UPDATE_TIME_FIELDS = []string{
    "update_time",
    "update_at",
    "updated_at",
}

Using Map

Create Update Delete Count support map[string]interface,For example:

//Create
gosql.Table("users").Create(map[string]interface{}{
    "id":         1,
    "name":       "test",
    "email":      "[email protected]",
    "created_at": "2018-07-11 11:58:21",
    "updated_at": "2018-07-11 11:58:21",
})

//Update
gosql.Table("users").Where("id = ?", 1).Update(map[string]interface{}{
    "name":  "fifsky",
    "email": "[email protected]",
})

//Delete
gosql.Table("users").Where("id = ?", 1).Delete()

//Count
gosql.Table("users").Where("id = ?", 1).Count()

//Change database
gosql.Use("db2").Table("users").Where("id = ?", 1).Count()

//Transaction `tx`
tx.Table("users").Where("id = ?", 1}).Count()

sql.Null*

Now Model support sql.Null* field's, Note, however, that if sql.Null* is also filtered by zero values,For example

type Users struct {
	Id          int            `db:"id"`
	Name        string         `db:"name"`
	Email       string         `db:"email"`
	Status      int            `db:"status"`
	SuccessTime sql.NullString `db:"success_time" json:"success_time"`
	CreatedAt   time.Time      `db:"created_at" json:"created_at"`
	UpdatedAt   time.Time      `db:"updated_at" json:"updated_at"`
}

user := &Users{
    Id: 1,
    SuccessTime: sql.NullString{
        String: "2018-09-03 00:00:00",
        Valid:  false,
    }
}

err := gosql.Model(user).Get()

Builder SQL:

Query: SELECT * FROM users WHERE (id=?);
Args:  []interface {}{1}
Time:  0.00082s

If sql.NullString of Valid attribute is false, SQL builder will ignore this zero value

gosql.Expr

Reference GORM Expr, Resolve update field self-update problem

gosql.Table("users").Update(map[string]interface{}{
    "id":2,
    "count":gosql.Expr("count+?",1)
})
//Builder SQL
//UPDATE `users` SET `count`=count + ?,`id`=?; [1 2]

"In" Queries

Because database/sql does not inspect your query and it passes your arguments directly to the driver, it makes dealing with queries with IN clauses difficult:

SELECT * FROM users WHERE level IN (?);

sqlx.In is encapsulated In gosql and can be queried using the following schema

var levels = []int{4, 6, 7}
rows, err := gosql.Queryx("SELECT * FROM users WHERE level IN (?);", levels)

//or

user := make([]Users, 0)
err := gosql.Select(&user, "select * from users where id in(?)",[]int{1,2,3})

Relation

gosql used the golang structure to express the relationships between tables,You only need to use the relation Tag to specify the associated field, see example

⚠️ Since version v2, the relation query across library connections needs to be specified using connection tag

type MomentList struct {
	models.Moments
	User   *models.Users    `json:"user" db:"-" relation:"user_id,id"`         //one-to-one
	Photos []models.Photos `json:"photos" db:"-" relation:"id,moment_id" connection:"db2"`     //one-to-many
}

Get single result

moment := &MomentList{}
err := gosql.Model(moment).Where("status = 1 and id = ?",14).Get()
//output User and Photos and you get the result

SQL:

2018/12/06 13:27:54
	Query: SELECT * FROM `moments` WHERE (status = 1 and id = ?);
	Args:  []interface {}{14}
	Time:  0.00300s

2018/12/06 13:27:54
	Query: SELECT * FROM `moment_users` WHERE (id=?);
	Args:  []interface {}{5}
	Time:  0.00081s

2018/12/06 13:27:54
	Query: SELECT * FROM `photos` WHERE (moment_id=?);
	Args:  []interface {}{14}
	Time:  0.00093s

Get list result, many-to-many

var moments = make([]MomentList, 0)
err := gosql.Model(&moments).Where("status = 1").Limit(10).All()
//You get the total result  for *UserMoment slice

SQL:

2018/12/06 13:50:59
	Query: SELECT * FROM `moments` WHERE (status = 1) LIMIT 10;
	Time:  0.00319s

2018/12/06 13:50:59
	Query: SELECT * FROM `moment_users` WHERE (id in(?));
	Args:  []interface {}{[]interface {}{5}}
	Time:  0.00094s

2018/12/06 13:50:59
	Query: SELECT * FROM `photos` WHERE (moment_id in(?, ?, ?, ?, ?, ?, ?, ?, ?, ?));
	Args:  []interface {}{[]interface {}{1, 2, 3, 4, 5, 6, 7, 8, 9, 10}}
	Time:  0.00087s

Relation Where:

moment := &MomentList{}
err := gosql.Relation("User" , func(b *gosql.Builder) {
    //this is builder instance,
    b.Where("gender = 0")
}).Get(moment , "select * from moments")

Hooks

Hooks are functions that are called before or after creation/querying/updating/deletion.

If you have defiend specified methods for a model, it will be called automatically when creating, updating, querying, deleting, and if any callback returns an error, gosql will stop future operations and rollback current transaction.

// begin transaction
BeforeChange
BeforeCreate
// update timestamp `CreatedAt`, `UpdatedAt`
// save
AfterCreate
AfterChange
// commit or rollback transaction

Example:

func (u *Users) BeforeCreate(ctx context.Context) (err error) {
  if u.IsValid() {
    err = errors.New("can't save invalid data")
  }
  return
}

func (u *Users) AfterCreate(ctx context.Context, tx *gosql.DB) (err error) {
  if u.Id == 1 {
    u.Email = ctx.Value("email")
    tx.Model(u).Update()
  }
  return
}

BeforeChange and AfterChange only used in create/update/delete

All Hooks:

BeforeChange
AfterChange
BeforeCreate
AfterCreate
BeforeUpdate
AfterUpdate
BeforeDelete
AfterDelete
BeforeFind
AfterFind

Hook func type supports multiple ways:

func (u *Users) BeforeCreate()
func (u *Users) BeforeCreate() (err error)
func (u *Users) BeforeCreate(tx *gosql.DB)
func (u *Users) BeforeCreate(tx *gosql.DB) (err error)
func (u *Users) BeforeCreate(ctx context.Context)
func (u *Users) BeforeCreate(ctx context.Context) (err error)
func (u *Users) BeforeCreate(ctx context.Context, tx *rsql.DB)
func (u *Users) BeforeCreate(ctx context.Context, tx *rsql.DB) (err error)

If you want to use context feature, you need to use below function while start a sql, or the context in callback will be nil:

  1. gosql.WithContext(ctx).Model(...)
  2. gosql.Use("xxx").WithContext(ctx).Model(...)

Thanks

sqlx https://github.com/jmoiron/sqlx

gosql's People

Contributors

brucehuang2 avatar dependabot-preview[bot] avatar dependabot[bot] avatar fifsky avatar wanchaochao avatar wuerping avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gosql's Issues

Error: pq: syntax error at or near "`"

``
configs["default"] = &gosql.Config{
Enable: true,
Driver: "postgres",
Dsn: "...",
ShowSql: true,
}

err = gosql.Connect(configs)
_, err = gosql.Table("users").Create(map[string]interface{}{
	"username":      username,
	"status":        1,
})

the log
Query: INSERT INTO users (,status,username) VALUES(?,?);
Args: []interface {}{ 1, "test12345"}
Error: pq: syntax error at or near "`"
Time: 0.00698s
``

能不能提供一个对外设置showsql的方法?

是这样的,我现在线上有一个需求是,默认的情况下,不显示sql语句,但是比如线上出了问题,我需要在不重启服务的情况下修改我本地的配置文件,监听配置文件变化改变你的showsql为true,监听本地配置文件变化,并获取新的配置文件我用的viper 这个配置库,所以能不能对外提供一个主动设置showsql的方法?

Is this a BUG

result, err := b.db.Exec(b.updateString(m), b.args...)

err:
Query: UPDATE user SET updated_at=? WHERE (id in (?));
Args: []interface {}{1617872905, []int64{413}}
Error: sql: converting argument $2 type: unsupported type []int64, a slice of int64

Need visit DB.database member

I need to get DB.database member for set db config properties:
db.SetMaxOpenConns(10)
db.SetMaxIdleConns(4)
Can not do this now, add a Func to return internal member DB.database please.

偶尔会出现卡死

有时候跑着跑着, 查询数据库就卡死了,必须重启服务才可以

不知道是否是MySQL自动断开连接导致的

To much output

I think this library looks great, but unfortunately I can't use it because it has to much forced output.
I can't find any way to suppress the "[db] connect:default" to stderr, and I need it to be silent so I can handle logging my own way.
I would appreciate if if you didn't assuming I want logging to stderr and returned messages in a way that I could handle them the way I want instead.

support table lock

have a case like that

before insert into table A ,i need select count(x) as sum from tableA .

if sum < Y ,then insert .

so i want to have a table lock before count

invalid connection

你好,用你这个gosql 有时候回出现db 错误 错误是: invalid connection 好像是连接池的某个链接被关闭了

找到原因了:
在使用go-sql-driver过程中,有时候我发现会报invalid connection的错误,这个错误产生的原因设置了空闲连接不为0时,客户端拿到连接有可能是已经被服务端关闭了,此时就会发生invalid connection,网上有些解决方案是把空闲连接数设置0,不知道设置为0跟设置为50之类的性能有什么大的区别吗?跪求大神解答!
当然这是因为服务端主动关闭了连接,因为服务端对连接的保持超时时间到了也关闭的,所以可以设置客户端连接超时时间小于服务端DB.SetConnMaxLifetime(time.Second)

https://www.studygolang.com/topics/5576
你能不能框架 修复一下这个问题 默认配置一下

sess.SetMaxOpenConns(conf.MaxOpenConns)
sess.SetMaxIdleConns(conf.MaxIdleConns)
sess.SetConnMaxLifetime(conf.MaxLifetime)

type Config struct {
Enable bool json:"enable"
Driver string json:"driver"
Dsn string json:"dsn"
MaxOpenConns int toml:"max_open_conns" json:"max_open_conns"
MaxIdleConns int toml:"max_idle_conns" json:"max_idle_conns"
MaxLifetime int toml:"max_lifetime" json:"max_lifetime"
ShowSql bool toml:"show_sql" json:"show_sql"
}

有个低级错误

config.go里面

MaxLifetime int yml:"max_lefttime" toml:"max_lefttime" json:"max_lefttime"

这个英文你写错成lefttime了, 应该是lifetime

多数据库时update操作,无法找到数据库

在使用多库操作时,我使用下面操作管理端数据库,操作更新时,出现错误,无法找到对应的库名进行操作
model := &models.Report{}
data := make(map[string]interface{}, 0)
data["create_time"] = "2018-09-18 12:00:00"
gosql.Use("admin").Table(model.TableName()).Where("from_uid=? AND to_uid=? ", 1, 2).Update(data)

使用postgresql,条件查询时报错

错误信息:
Query: SELECT * FROM "pwdinfo" WHERE (password=?);
Args: []interface {}{"123"}
Error: pq: 操作符不存在: character varying =?
Time: 1.02744s
源码:
pwd:=c.QueryParam("pwd") pwdinfo:= &models.Pwdinfo{} err:= gosql.Model(pwdinfo).Where("password=?",pwd).Get() if err!=nil{ log.Println(err.Error()) }

模型批量查询能支持分表吗?

你好,这种查询能支持分表吗?
rows := make([]*models.TopicInfo, 0)
_ = gosql.Model(&rows).Where("uid=? ", uid).OrderBy("id DESC").Limit(20).All()

exec operation

Why only Exec don't call w.argsIn function to deal with "IN"

image

模型表字段为sql.nullstring时 插入掉sql出现多个字段

有个需求是,创建时间和成功时间,在创建的时候成功时间是可以为空的,设置表的模型为
由于我的时间类型是为字符串,所以我设置了 sql.NullString
SuccessTime sql.NullString db:"success_time" json:"success_time" // 成功时间
但是在插入的时候,你的sql语句出现了 多个字段 应该是解析的时候出了问题
success_time,success_time.String,success_time.Valid

log improvement

// Logger represents a logging collector. You can pass a logging collector to
// gosql.SetLogger(myCollector) to make it collect QueryStatus messages
// after executing a query.
type Logger interface {
	Printf(format string, v ...interface{})
}
  1. I use db.WithContext(ctx), set my [trace id] to ctx .
  2. When we print a sql log, i can not get the [trace id] in ctx, so this log cannot be concatenated with other logs through [trace id].

Maybe we shoud change this interface to:

type Logger interface {
	Printf(ctx context.Context, format string, v ...interface{})
}

can where method support In query?

I love this library so much, it's simple yet powerful.
It could even be better if it support "in query" in where method.

for example:

users := make([]*User, 0)
gosql.Model(&user).Where("id in (?)", []int{1, 2, 3}).All()

db反射生成go工具

我写了一个db反射的工具,已经适应gosql了,可以用上哈, 能支持postrges和mysql读取

github.com/athanxx/dbtag

已经适应gosql的IModel结构, 支持xorm,gorm和任何的tag生成

还能自动生成脚本,方便下次使用

created_at字段为什么是UTC时间

我今天用了,字段里面有created_at还有updated_at
在win下调试,然后插入进去的是UTC时间,比正常时间少8小时,这个怎么调整?

db tag lowercase?

资询个问题:
mysql 表字字段是大写, 比如 COL_NAME

结构定义为 
type def struct{
   ColName string `db:"col_name"` 
}
这样查询是空的,大小写不匹配, 有没有类似这样 `db:"col_name";lowercase:true`用法?

默认的gosql里面没有Commit ???

package main

import (
	"fmt"
	_ "github.com/go-sql-driver/mysql" //mysql driver
	"github.com/ilibs/gosql/v2"
	"os"
)

func main() {
	configs := make(map[string]*gosql.Config)
	configs["default"] = &gosql.Config{
		Enable:  true,
		Driver:  "mysql",
		Dsn:     "root:321@@tcp(127.0.0.1:3306)/soulma?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai",
		ShowSql: true,
	}


	configs["db2"] = &gosql.Config{
		Enable:  true,
		Driver:  "mysql",
		Dsn:     "root:123@@tcp(127.0.0.1:3306)/soulmb?charset=utf8mb4&parseTime=True&loc=Asia%2FShanghai",
		ShowSql: true,
	}
	//connection database
	if err := gosql.Connect(configs); err != nil {
		fmt.Println(err.Error())
		os.Exit(0)
	}

	type UserList struct {
		UID int `db:"Pid"`
	}
	user := UserList{}
	rows  := gosql.Use("db2").QueryRowx("SELECT * FROM user_list WHERE pid = 1333;")
	gosql.Begin()
	// `木有gosql.Commit这函数`
	gosql.Begin()
	gosql.Use("db2").Commit() // `这里有Commit函数`
	fmt.Println(rows.Err(),user)
}

钩子函数

注意到你新增加的钩子函数,为什么没有BeforeFind呀

报个bug, 当主键为uint64时候, 获取lastId出错

当主键维uint64时候, 原本代码在反射情况下会直接崩掉, 因为v.SetInt里面没有对Uint类型的支持, 需要修改为如下代码才行

在util.go下 107行开始

// fillPrimaryKey is created fill primary key
func fillPrimaryKey(v reflect.Value, value int64) {
	v = reflect.Indirect(v)
	if v.IsValid() {
		switch v.Kind() {
		case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
			v.SetInt(value)

		case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr:
			v.SetUint(uint64(value))
		}
	}
}

gosql.exec error

hi gosql.exec 不支持字段内容为空吗?

currTime := time.Now().Format("2006-01-02 15:04:05")
			_, err := gosql.Exec("update man_Task set tsk_state = %d, tsk_lasttime = '%s', tsk_nexttime = '%s' where tsk_Taskid= %d",
				TASKRUNING_STATE, currTime, "", 12)

image
把SQL 格式化后就可以正常执行

szSql := fmt.Sprintf("update man_Task set tsk_state = %d, tsk_lasttime = '%s', tsk_nexttime = '%s' where tsk_Taskid= %d",
				TASKRUNING_STATE, currTime, “”, 12)
		_, err := gosql.Exec(szSql)
		这里可以正常执行

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.