Git Product home page Git Product logo

vitess-sqlparser's Introduction

vitess-sqlparser

Simply SQL and DDL parser for Go (powered by vitess and TiDB ) this library inspired by https://github.com/xwb1989/sqlparser

(original source : https://github.com/youtube/vitess/tree/master/go/vt/sqlparser)

Why

xwb1989/sqlparser is famous sql parser in Go.
But it cannot parse some query (like offset or bulk insert...) because it customizes vitess's sql parser.

Also, some libraries use from vitess sql parser directly. But vitess's sql parser only partial supports DDL parsing.

We want to perfectly support parsing for SQL and DDL.
Therefore we use vitess sql parser directly and also use TiDB parser for DDL parsing.

Compare SQL parser libraries in Go

library supports offset (or other complexity) query supports DDL
xwb1989/sqlparser
zhenjl/sqlparser
knocknote/vitess-sqlparser

Installation

[NOTE] Required Go version more than 1.9

go get -u github.com/knocknote/vitess-sqlparser

Examples

package main

import (
 	"fmt"
	"github.com/blastrain/vitess-sqlparser/sqlparser"
)

func main() {
	stmt, err := sqlparser.Parse("select * from user_items where user_id=1 order by created_at limit 3 offset 10")
	if err != nil {
		panic(err)
	}
	fmt.Printf("stmt = %+v\n", stmt)
}

vitess-sqlparser's People

Contributors

goccy avatar kanataxa avatar kr-nakaya avatar ktsujichan avatar mpchadwick avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vitess-sqlparser's Issues

How can I get the binding query?

Hi, I have some question.

I want to collect and parse queries and make statistics by query pattern.
Query patterns should be collected in this form.

## AS-IS
select * from abc where x = 1

## TO-BE
select * from abc where x = ?

As a test, I tried the following code, but a different query came out.

func main() {
	stmt, err := sqlparser.Parse("select * from user_items where user_id=1 order by created_at limit 3 offset 10")
	if err != nil {
		panic(err)
	}
	q := sqlparser.GenerateParsedQuery(stmt).Query
	fmt.Println(q)
}

But, the result is as below.

select * from user_items where user_id = 1 order by created_at asc limit 10, 3

please reply.
Thanks.
Chan.

Compiling for 32bit architecture

When I try to compile for GOARCH=386, I get the compile errors below. For now I'll just use amd64 instead. Is it possible to also support 386?

# github.com/blastrain/vitess-sqlparser/tidbparser/dependency/mysql
vendor/github.com/blastrain/vitess-sqlparser/tidbparser/dependency/mysql/const.go:503:2: constant 2147483648 overflows SQLMode
vendor/github.com/blastrain/vitess-sqlparser/tidbparser/dependency/mysql/const.go:504:2: constant 4294967296 overflows SQLMode
vendor/github.com/blastrain/vitess-sqlparser/tidbparser/dependency/mysql/util.go:44:19: constant 4294967295 overflows int
vendor/github.com/blastrain/vitess-sqlparser/tidbparser/dependency/mysql/util.go:45:19: constant 4294967295 overflows int

Postgres `$` dollar placeholder support.

The library isn't able to parse postgres query with $ placeholders.

tree, err := sqlparser.Parse("INSERT INTO users (email, name) VALUES ($1, $2)")
if err != nil {
    fmt.Println(err)
    // err = syntax error at position 52 near '$'
}

I can see the variable yyDollar being used so not sure if this dollar placeholder is supported or not.

Writes from EXPLAIN ANALYZE are classified as OtherRead

In postgres and some others, EXPLAIN ANALYZE executes the query it's explaining, which might itself not be a read query. For example:

EXPLAIN ANALYZE DELETE FROM abc;

...will actually delete from the table, but sqlparser classifies it as OtherRead rather than Delete. It's possible this is intentional, as EXPLAIN ANALYZE results in an explanation meant to be read, but it's also unintuitive here, since the underlying query is not a read operation. In the above, not even walking the tree exposes the DELETE part.

I would like a way to verify read-only SELECTs, DESCRIBEs and non-mutating EXPLAINs, but I don't think I can do that simply with a type switch, as OtherRead won't capture the mutating EXPLAIN ANALYZE case.

Fix README Installation section

The Installation section of the README has an invalid (obsolete?) installation command:

go get -u github.com/knocknote/vitess-sqlparser

It should be changed to:

go get -u github.com/blastrain/vitess-sqlparser 

How do you add a new where clause to an existing sql statement?

I am unable to figure how to construct an Expr to pass to AddWhere method. Lets say I have a query like "SELECT * FROM product WHERE price < 500". Now how do I add "discount <= 10"? This is the closest I came to a solution:

	left := sqlparser.NewStrVal([]byte("price"))
	right := sqlparser.NewStrVal([]byte("?"))
	comp := &sqlparser.ComparisonExpr{
		Operator: sqlparser.LessEqualStr,
		Left:     left,
		Right:    right,
	}
	fmt.Printf("%v\n", sqlparser.String(comp))
	sel := stmt.(*sqlparser.Select)
	sel.AddWhere(comp)
	fmt.Printf("%v\n", sqlparser.String(sel))

But this seems to quote the left in single quotes, because I assume it is considering it as a value instead of a column name. So how do I add a column name expr?

example of parser details

Hi there,

From the example we just print the parse out. How can I get - that is like type (create or alter etc, DDL type, ) table name, columns etc, all details?

the thing I could see is using WalkSubtree and parse them out however it is not obvious to new users like me without reading code and finding types etc..

So a more example to show the detailed portion of the parsed stmt would be great.

Thanks

How to get 'table name'?

use demo

stmt, err := sqlparser.Parse
fmt.Printf("stmt = %+v\n", stmt)
The output "stmt = &{Cache: Comments:[] Distinct: Hints: SelectExprs:[0xc420088e10] From:[0xc420089110 0xc420089140] Where:0xc42000a5c0 GroupBy:[] Having: OrderBy:[] Limit: Lock:}"

How do I get the value in the From
please help me

Can't install

go get github.com/knocknote/vitess-sqlparser
can't load package: package github.com/knocknote/vitess-sqlparser: no Go files in /home/kh/go/src/github.com/knocknote/vitess-sqlparser

why "show databases" trigger [signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0xde0b22]

COM_QUERY: show databases
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x48 pc=0xde0b22]

goroutine 26 [running]:
github.com/blastrain/vitess-sqlparser/sqlparser.convertTiDBStmtToVitessShow(0xc0003280b0, 0x1, 0x1, 0xc0005a6100, 0x0, 0x0)
/root/software/sqlaudit/vendor/github.com/blastrain/vitess-sqlparser/sqlparser/type_converter.go:92 +0x62
github.com/blastrain/vitess-sqlparser/sqlparser.Parse(0xc000594490, 0xe, 0x4d35e6, 0x6139b690, 0x2c3ce2aa, 0x62d137f69c)
/root/software/sqlaudit/vendor/github.com/blastrain/vitess-sqlparser/sqlparser/ast.go:63 +0x3e7

New to sqlparser, how could I quickly understand AST

Hi All, I think you guys must be expert on sql parser, how could I understand AST as well as the whole design on this parser easily?
Even fetch the tableName looks pretty heavy, I am pretty confused somehow.

tableName := stmt.(*sqlparser.Select).From[0].(*sqlparser.AliasedTableExpr).Expr.(sqlparser.TableName).Name.String()

Support WITH clause

Hey, great project with a admirable aims.

Would be great to have the support for widely used WITH clause, i.e.

package main

import (
	"fmt"

	"github.com/blastrain/vitess-sqlparser/sqlparser"
)

func main() {
	stmt, err := sqlparser.Parse(stmt)
	if err != nil {
		panic(err)
	}
	fmt.Printf("stmt = %+v\n", stmt)
}

const stmt = `with alias as (select * from tab) select * from alias`

currently returns an error syntax error at position 5 near 'with'

Chinese are not supported

sql like "select 1 as 测试 from dual",
got error:
syntax error at position 14 near '�'

how can i fix this,thank you。

Support function?

I hope use custom function like min(1,2,3,4,5), or sort(1,2,3,4,5)... Can support this feature?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.