Git Product home page Git Product logo

dcparser's Introduction

DCParser - SQL Parser optimized based on PingCAP/parser

Language Go Report Card GoDoc Build Status codecov

Introduction

DCParser is a single project built for compatibility with TiDB for PostgreSQL. It is based on PingCAP/parser, but it is difficult to merge into the original parser repository because there are many changes. So a new project is created to provide parsing abilities for TiDB for PostgreSQL.

Development

The current development is driven by the implementation of TiDB for PostgreSQL. The implements of Schema structure, system functions, system views and other functions of TiDB for PostgreSQL depend on the parser modification. Currently, the DCParser can fully meet the functional requirements of the TiDB for PostgreSQL. If TiDB for PostgreSQL needs new system functions, system views and other functions, you also need to modify them in DCParser.

Quick start

DCParser project is the sql parser of TiDB for PostgreSQL. The most common way to use it is to import it into the TiDB for PostgreSQL project and play its role by parsing sql statements.

DCParser is written in Go language. SO to use DCParser, you need to ensure that the Go environment is installed.

In the TiDB for PostgreSQL project, it has been declared that DCParser needs to be pulled. You can use the vendor command to get the project before compiling the project.

In the GOPATH of the development environment, there will be the source files of the DCParser project that TiDB for PostgreSQL depends on. Modify the parser.y file, and then use the following command to compile the new parser.go file.

goyacc -o parser.go parser.y 

Contributing

We greatly welcome and appreciate developers to contribute to DCParser to realize other features of TiDB for PostgreSQL.

License

The license is pending.

dcparser's People

Contributors

tangenta avatar crazycs520 avatar spongedu avatar tiancaiamao avatar lysu avatar alivxxx avatar kennytm avatar ailinkid avatar exialin avatar orion7r avatar xuhuaiyu avatar pupillord avatar bb7133 avatar feloxx avatar leiysky avatar qw4990 avatar andrewdi avatar lzmhhh123 avatar jackysp avatar lauhg avatar foreyes avatar wjhuang2016 avatar deardrops avatar eurekaka avatar zhenghaoz avatar tony612 avatar cfzjywxk avatar gauss1314 avatar winkyao avatar lonng avatar

Stargazers

Oliver Yuan avatar  avatar  avatar 小猪快跑 avatar  avatar  avatar  avatar  avatar  avatar  avatar huangzhaohu avatar gpromote avatar ZhouWei avatar Michael avatar  avatar  avatar HOHO avatar  avatar

Watchers

James Cloos avatar HOHO avatar  avatar

dcparser's Issues

Support for Postgres transaction syntax in Parser

Development Task

Description

The SQL standard syntax for starting a transaction is START TRANSACTION, which both MySQL and PostgreSQL supports.
They also support syntax like START TRANSACTION + [modifier], where modifier can be READ WRITE, READ ONLY, etc.
For the ease of use, they also support use BEGIN instead of START TRANSACTION
However, they treat BEGIN + [modifier] differently. (like BEGIN READ WRITE)

In Postgres this is completely fine, BEGIN/BEGIN WORK/BEGIN TRANSACTION are all treated as aliases for START TRANSACTION

MySQL, on the other hand, does not allow modifier after BEGIN, a limitation which TiDB inherits.

In TiDB, the standard way to start new transaction via go code is

txn, err := db.Begin()

which calls underlying go-mysql-driver via go's database module, and return a transaction handle (the txn above).
After we switch the driver from lib/pq, because pq starts transaction by BEGIN READ WRITE when no modifier provided, it will cause parser to complain.
image

Previously in unit tests fixing, we have been solving the problem by skipping Begin()method and execute SQL statement START TRANSACTION; and execute COMMIT; at the end of query block. But since we are not using Begin(), we can't get txn handle. It will cause problem when multiple transactions are nested or intersected.

The ideal solution is for parser to support begin + modifier syntax.

References

PostgreSQL: Documentation: 9.1: BEGIN
MySQL :: MySQL 8.0 Reference Manual :: 13.3.1 START TRANSACTION, COMMIT, and ROLLBACK Statements
MySQL Difference between BEGIN and START TRANSACTION - Database Administrators Stack Exchange

Very import feature!

Many systems are built on postgres, mssql, and db2. Their all have database, schema, table lalyer.

It is hard to do migration many applications to TiDB for it lack of the schema layer.

Does this project has timeline that support main feature of postgres that build a release version?

Double, single and back quotation marks

Development Task

MySQL can use single quotes(') or double quotes(") to represent values, but PgSQL can only use single quotes(') to represent values. PgSQL's double quotes(") represent system identifiers, such as table names or field names . MySQL can use back quotes(`) to indicate system identifiers, such as table names and field names. PgSQL also does not support it.

We need to implement the use of these marks in PgSQL,replace their usage in MySQL.

  • Implement the use of double, single and back quotation marks in PgSQL

To support PG compliant SQL DML statement

Description

Since TiDB is Mysql compliant database, we need to adjust DCParser yacc paser files to remove mysql keywords and add pg keywords, and then implement the keywords logic functions in TiDB4PG project.

The things you need to do are :

  1. spot statements in parser.y file
  2. remove mysql keywords of the statement
  3. make sure build successfully
  4. correct related unit tests
  5. commit PR

Helpful reference:
Understand Delete Statement - Chinese

Task List

DCParser:

  • Delete Statement:
  1. remove mysql keywords
  2. correct unit tests
  • Select Statement Parser
  1. remove mysql keywords
  2. correct unit tests
  • Insert Statement Parser
  1. remove mysql keywords
  2. correct unit tests
  • Update Statement Parser
  1. remove mysql keywords
  2. correct unit tests

TiDB4PG:

Please check the issue
DigitalChinaOpenSource/TiDB-for-PostgreSQL#40

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.