Git Product home page Git Product logo

psqlparse's Introduction

psqlparse

Build Status

This project is not maintained anymore. If you would like to maintain it, send me a DM in twitter @alculquicondor.

This Python module uses the libpg_query to parse SQL queries and return the internal PostgreSQL parsetree.

Installation

pip install psqlparse

Usage

import psqlparse
statements = psqlparse.parse('SELECT * from mytable')
used_tables = statements[0].tables()  # ['my_table']

tables is only available from version 1.0rc1

Development

  1. Update dependencies
git submodule update --init
  1. Install requirements:
pip install -r requirements.txt
  1. Build Cython extension
USE_CYTHON=1 python setup.py build_ext --inplace
  1. Perform changes

  2. Run tests

pytest

Maintainers

psqlparse's People

Contributors

akeeman avatar alculquicondor avatar alexcodreanu86 avatar ehfeng avatar justinfay avatar kevinzg avatar lelit avatar lfittl avatar mracos avatar phillipknauss avatar terrorobe avatar wangym5106 avatar yiblet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

psqlparse's Issues

Release to pypi?

Could you please release the latest pre-release to pypi even if it's as a pre-release?

Error: 'Statement' object has no attribute 'tables'

Hi,

Trying to access the list of tables as in the example in the read.me,

`import psqlparse
statements = psqlparse.parse('SELECT * from mytable')
used_tables = statements[0].tables() # ['my_table']

I get an error:

Attribute error: 'Statement' object has no attribute 'tables'.

Also dir(statements[0]) gives:
['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_obj', 'from_clause', 'target_list', 'type', 'where_clause', 'with_clause']

How does the output look?

hey im new to this , i followed the development section and usage section .
how do i perfrom test
what command generates the parsed tree and how do i display it on console
please help.

installation error ubuntu

I am having this error when trying install psqlparse on ubuntu. I have flex and bison installed. What can cause this problem?

Collecting psqlparse
Using cached https://files.pythonhosted.org/packages/af/5a/a59c01632b5b9ade799cb0705de1a89d92021d554fa7f29fe27434db60be/psqlparse-0.2.5.tar.gz
Requirement already satisfied: six in /home/guliver/.virtualenvs/sql/lib/python3.6/site-packages (from psqlparse) (1.12.0)
Building wheels for collected packages: psqlparse
Running setup.py bdist_wheel for psqlparse ... error
Complete output from command /home/guliver/.virtualenvs/sql/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-a4rb5hkf/psqlparse/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-ixpq02gc --python-tag cp36:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psqlparse
copying psqlparse/init.py -> build/lib.linux-x86_64-3.6/psqlparse
copying psqlparse/exceptions.py -> build/lib.linux-x86_64-3.6/psqlparse
copying psqlparse/nodes.py -> build/lib.linux-x86_64-3.6/psqlparse
running build_ext
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 127 0 127 0 0 348 0 --:--:-- --:--:-- --:--:-- 349
100 1507k 0 1507k 0 0 767k 0 --:--:-- 0:00:01 --:--:-- 1032k
./build_libpg_query.sh: line 8: unzip: command not found

An error occurred during extension building.
Make sure you have bison and flex installed on your system.


Failed building wheel for psqlparse
Running setup.py clean for psqlparse
Failed to build psqlparse
Installing collected packages: psqlparse
Running setup.py install for psqlparse ... error
Complete output from command /home/guliver/.virtualenvs/sql/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-a4rb5hkf/psqlparse/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-omai97hh/install-record.txt --single-version-externally-managed --compile --install-headers /home/guliver/.virtualenvs/sql/include/site/python3.6/psqlparse:
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/psqlparse
copying psqlparse/init.py -> build/lib.linux-x86_64-3.6/psqlparse
copying psqlparse/exceptions.py -> build/lib.linux-x86_64-3.6/psqlparse
copying psqlparse/nodes.py -> build/lib.linux-x86_64-3.6/psqlparse
running build_ext
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 127 0 127 0 0 312 0 --:--:-- --:--:-- --:--:-- 312
100 1507k 0 1507k 0 0 808k 0 --:--:-- 0:00:01 --:--:-- 2542k
./build_libpg_query.sh: line 8: unzip: command not found

An error occurred during extension building.
Make sure you have bison and flex installed on your system.

----------------------------------------

Command "/home/guliver/.virtualenvs/sql/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-a4rb5hkf/psqlparse/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" install --record /tmp/pip-record-omai97hh/install-record.txt --single-version-externally-managed --compile --install-headers /home/guliver/.virtualenvs/sql/include/site/python3.6/psqlparse" failed with error code 127 in /tmp/pip-install-a4rb5hkf/psqlparse/

Installing on Windows 10?

Thanks for the previous reply!
I would like to install it on Windows.
Now I have downloaded the v1.0rc4 from pythons website

Index INCLUDE parse error

pgsqlparse version: 1.0rc7

Postgres v11 adds the INCLUDE keyword (https://www.postgresql.org/docs/11/sql-createindex.html) but it currently is not supported and parsing raises psqlparse.exceptions.PSqlParseError when included:

CREATE INDEX foo ON bar (baz) INCLUDE (qux)

Traceback:

Traceback (most recent call last):
  File "/Users/gavinr/Source/Applications/pglifecycle/tests/test_sql_parse_indexes.py", line 68, in test_include
    self.assertEqual(sql_parse.index(entry), expectation)
  File "/Users/gavinr/Source/Applications/pglifecycle/pglifecycle/sql_parse.py", line 103, in index
    parsed = parse(entry.defn)
  File "/Users/gavinr/Source/Applications/pglifecycle/pglifecycle/sql_parse.py", line 22, in parse
    return _parse(psqlparse.parse_dict(value))
  File "psqlparse/parser.pyx", line 27, in psqlparse.parser.parse_dict
psqlparse.exceptions.PSqlParseError: syntax error at or near "INCLUDE"

Expose elements of 'create' statements

I recently evaluated this module in the context of pre-flighting some SQL that creates schema, and discovered that statement nodes with types like CreateStmt, CreateSeqStmt, etc are not adorned with accessors for the various elements of those statements. However, if I look into the objects' internal state, I can see that it's all there:

To demonstrate, here's a script called schema-parse.py that prints all the variables (private and public) of the first three statements in my schema file:

#!/usr/bin/python

from pprint import pprint, pformat
import psqlparse

fh = open('my-best-schema.sql', 'r')
sql = fh.read()
fh.close()

p = psqlparse.parser.parse(sql)
for i in range(0,3):
    print pformat(vars(p[i])).replace("    ", " ")
    print
$ python schema-parse.py
{'_obj': {u'sequence': {u'RangeVar': {u'inhOpt': 2,
           u'location': 799,
           u'relname': u'resource_id_seq',
           u'relpersistence': u'p'}}},
 'from_clause': None,
 'target_list': None,
 'type': u'CreateSeqStmt',
 'where_clause': None,
 'with_clause': None}

{'_obj': {u'oncommit': 0,
    u'relation': {u'RangeVar': {u'inhOpt': 2,
           u'location': 971,
           u'relname': u'named_lock',
           u'relpersistence': u'p'}},
    u'tableElts': [{u'ColumnDef': {u'colname': u'lock_name',
           u'constraints': [{u'Constraint': {u'contype': 4,
                     u'location': 1011}}],
           u'is_local': True,
           u'location': 988,
           u'typeName': {u'TypeName': {u'location': 998,
                  u'names': [{u'String': {u'str': u'pg_catalog'}},
                    {u'String': {u'str': u'varchar'}}],
                  u'typemod': -1,
                  u'typmods': [{u'A_Const': {u'location': 1006,
                        u'val': {u'Integer': {u'ival': 255}}}}]}}}}]},
 'from_clause': None,
 'target_list': None,
 'type': u'CreateStmt',
 'where_clause': None,
 'with_clause': None}

{'_obj': {u'sequence': {u'RangeVar': {u'inhOpt': 2,
           u'location': 1108,
           u'relname': u'job_seq',
           u'relpersistence': u'p'}}},
 'from_clause': None,
 'target_list': None,
 'type': u'CreateSeqStmt',
 'where_clause': None,
 'with_clause': None}

This is a feature request for plumbing the elements of 'create' statements up into the public variables provided by psqlparse

OSS-Fuzz integration for continuous fuzzing

Hi,

I was wondering if you would like to integrate continuous fuzzing by way of OSS-Fuzz? Fuzzing is a way to automate test-case generation and has been heavily used for memory unsafe languages. Recently efforts have been put into fuzzing memory safe languages and Python is one of the languages where it would be great to use fuzzing.

In this PR I did an initial integration into OSS-Fuzz. Essentially, OSS-Fuzz is a free service run by Google that performs continuous fuzzing of important open source projects.

If you would like to integrate, the only thing I need is a list of email(s) that will get access to the data produced by OSS-Fuzz, such as bug reports, coverage reports and more stats. Notice the emails affiliated with the project will be public in the OSS-Fuzz repo, as they will be part of a configuration file.

Accessing nodes recursively

Hi.

I'm starting to use this library, and i was wondering if there's a way to go through the tree recursively.

a bug?

if my sql like this:
"select (b.script_name),'中' from (select * from temp.halfisolateworkjob20171221) a left join (select * from temp.script2table) b on a.schema_name||'.'||a.table_name=b.table_name where a.create_time <'20171201' and a.owner='app_vgop' and schema_name='SESSION' and process_flag=false and b.script_name is not null order by 1;;"

how can I get the relname of sql?

I look into the project "pg-materialize" but that project can't resolve this? if sql like this :
CREATE MATERIALIZED VIEW IF NOT EXISTS schema1.messages_mv
AS (
select (b.script_name),'中' from (select * from temp.halfisolateworkjob20171221) a left join (select * from temp.script2table) b on a.schema_name||'.'||a.table_name=b.table_name where a.create_time <'20171201' and a.owner='app_vgop' and schema_name='SESSION' and process_flag=false and b.script_name is not null order by 1
) WITH DATA;
the program is ok,but when I delete the create ,change sql like:"elect (b.script_name),'中' from (select * from temp.halfisolateworkjob20171221) a left join (select * from temp.script2table) b on a.schema_name||'.'||a.table_name=b.table_name where a.create_time <'20171201' and a.owner='app_vgop' and schema_name='SESSION' and process_flag=false and b.script_name is not null order by 1;"
I use a method like pg-materialize to test,failed,the result is:
[<psqlparse.nodes.parsenodes.SelectStmt object at 0x7fb9c7d22590>] <type 'list'> 0
[<psqlparse.nodes.parsenodes.SelectStmt object at 0x7fb9c7d22590>] <type 'list'> 0
SELECT <class 'psqlparse.nodes.parsenodes.SelectStmt'> 1
{'dependencies': set([]), 'views': set([])}

my code like this:#!coding:utf-8

def extract_nodes(content):
from psqlparse import parse
from psqlparse import nodes

class NS(object):
    pass
ns = NS()
ns.views = set()
ns.dependencies = set()

def inner(data, depth):
    print data,type(data),depth
    if isinstance(data, dict):
        
        for key, value in data.items():
            inner(value, depth)
    elif isinstance(data, list) or isinstance(data, tuple):
        print data,type(data),depth
        for item in data:
            inner(item, depth + 1)

parsed_content = parse(content)
inner(parsed_content, 0)
return {
'views': ns.views,
'dependencies': ns.dependencies - ns.views
}

query = """ select (b.script_name),'中' from (select * from temp.halfisolateworkjob20171221) a left join (select * from temp.script2table) b on a.schema_name||'.'||a.table_name=b.table_name where a.create_time <'20171201' and a.owner='app_vgop' and schema_name='SESSION' and process_flag=false and b.script_name is not null order by 1

    """

query2=r"insert into dis.td_bd_area_info_d SELECT A.DEAL_DATE,A.INT_ID,A.ZH_LABEL,A.COUNTY_ID, B.ZH_LABEL OUNTY_NAME,B.CITY_ID,case when B.city_id = '40' then '邢台市' when B.city_id = '33' then '秦皇岛市' when B.city_id = '41' then '邯郸市' when B.city_id = '34' then '沧州市' when B.city_id = '36' then '廊坊市' when B.city_id = '32' then '石家庄市' when B.city_id = '37' then '张家口市' when B.city_id = '38' then '保定市' when B.city_id = '42' then '唐山市' when B.city_id = '43' then '衡水市' when B.city_id = '39' then '承德市'ELSE '其他' END ,CASE WHEN A.CELL_SOURCE in ('铁通割接','无线宽带','新国标','自建','自建无线宽带') and A.COVER_TYPE IN ('0','1','2','3') THEN '自建有线' WHEN A.CELL_SOURCE in ('铁通割接','无线宽带','新国标','自建','自建无线宽带') and A.COVER_TYPE IN ('4') THEN '自建无线(WLAN)' WHEN A.CELL_SOURCE in ('铁通割接','无线宽带','新国标','自建','自建无线宽带') and A.COVER_TYPE IN ('6') THEN '自建无线(4G)' WHEN A.CELL_SOURCE in ('铁通割接','无线宽带','新国标','自建','自建无线宽带') and A.COVER_TYPE IN ('6') THEN '自建无线(4G)' WHEN A.CELL_SOURCE in ('第三方割接','第三方无线宽带') and A.COVER_TYPE IN ('0','1','2','3') THEN '三方有线' WHEN A.CELL_SOURCE in ('第三方割接','第三方无线宽带') and A.COVER_TYPE IN ('4') THEN '三方无线(WLAN)' else '其他' end,case when A.AREA_TYPE = '市区(含县城)' then '市区' when A.AREA_TYPE = '乡镇(含城乡结合部)' then '乡镇' when A.AREA_TYPE = '农村' then '农村' else '其他' end,A.CELL_SOURCE,A.COVER_TYPE,A.HOUSE_NUM,ROW_NUMBER() OVER (PARTITION BY A.INT_ID ORDER BY A.MODIFY_TIME DESC , B.MODIFY_TIME DESC ) RN FROM DW.TD_RMS_ADD_CELL_D A LEFT JOIN DW.TD_RMS_COUNTY_D B ON A.COUNTY_ID = B.INT_ID AND B.DEAL_DATE = 20170101 where A.DEAL_DATE = 20170101;"

query="""select (b.script_name),'中' from (select * from temp.halfisolateworkjob20171221) a left join (select * from temp.script2table) b on a.schema_name||'.'||a.table_name=b.table_name where a.create_time <'20171201' and a.owner='app_vgop' and schema_name='SESSION' and process_flag=false and b.script_name is not null order by 1;"""

print extract_nodes(query)

can you help me?thanks a lot

check style

maybe make travis check it? or a pre-commit hook?

How do I get a parse tree?

Hi @alculquicondor . Once again how do I get a PostgreSQL parse tree as an output.
There is no documentation provided to find out which columns are there nor where are the INSERT queries etc.
and what way is the parse tree saved and how I can further read it, for doing further computations.

used_tables = stmt[0].tables() like this what else functions are there and what function is used to get the entire parse tree.

pip install fails

Unable to install, even with upgraded Cython dependency: pip install -e git+https://github.com/ehfeng/psqlparse#egg=psqlparse results in error (below).

However, cloning this repo, USE_CYTHON=1 python setup.py build_ext --inplace, and then pip install -e ./path/to/local/repo works.

    creating build/temp.macosx-10.13-x86_64-3.7/psqlparse
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I./libpg_query-9.5-latest -I/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c psqlparse/parser.c -o build/temp.macosx-10.13-x86_64-3.7/psqlparse/parser.o
    clang: error: no such file or directory: 'psqlparse/parser.c'
    clang: error: no input files
    /Users/eric/.virtualenvs/laminate/lib/python3.7/site-packages/setuptools/dist.py:398: UserWarning: Normalizing '1.0-rc5' to '1.0rc5'
      normalized_version,
    error: command 'clang' failed with exit status 1

    ----------------------------------------
Command "/Users/eric/.virtualenvs/laminate/bin/python3.7 -c "import setuptools, tokenize;__file__='/Users/eric/.virtualenvs/laminate/src/psqlparse/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" develop --no-deps" failed with error code 1 in /Users/eric/.virtualenvs/laminate/src/psqlparse/

Error: Can't install the package with pip command

When I tried to install the package with the command pip install psqlparse. It did failed with the following output.
OS: WINDOWS 10.

Building wheels for collected packages: psqlparse
  Building wheel for psqlparse (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'E:\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\LE\\A
ppData\\Local\\TEMP_~1\\pip-install-a9q7ed_5\\psqlparse\\setup.py'"'"'; __file__='"'"'C:\\Users\\LE\\AppData\\Loca
l\\TEMP_~1\\pip-install-a9q7ed_5\\psqlparse\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.re
ad().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\U
sers\LE\AppData\Local\TEMP_~1\pip-wheel-2dszy1_1'
       cwd: C:\Users\LE\AppData\Local\TEMP_~1\pip-install-a9q7ed_5\psqlparse\
  Complete output (11 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.7
  creating build\lib.win-amd64-3.7\psqlparse
  copying psqlparse\exceptions.py -> build\lib.win-amd64-3.7\psqlparse
  copying psqlparse\nodes.py -> build\lib.win-amd64-3.7\psqlparse
  copying psqlparse\__init__.py -> build\lib.win-amd64-3.7\psqlparse
  running build_ext
  error: [WinError 193] %1 Not a valid Win32 application.
  ----------------------------------------
  ERROR: Failed building wheel for psqlparse
  Running setup.py clean for psqlparse
Failed to build psqlparse
Installing collected packages: psqlparse
    Running setup.py install for psqlparse ... error
    ERROR: Command errored out with exit status 1:
     command: 'E:\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\LE1\
\AppData\\Local\\TEMP_~1\\pip-install-a9q7ed_5\\psqlparse\\setup.py'"'"'; __file__='"'"'C:\\Users\\LE\\AppData\\Lo
cal\\TEMP_~1\\pip-install-a9q7ed_5\\psqlparse\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.
read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record '
C:\Users\LE\AppData\Local\TEMP_~1\pip-record-jmsarh_5\install-record.txt' --single-version-externally-managed --co
mpile --install-headers 'E:\Anaconda3\Include\psqlparse'
         cwd: C:\Users\LE\AppData\Local\TEMP_~1\pip-install-a9q7ed_5\psqlparse\
    Complete output (11 lines):
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    creating build\lib.win-amd64-3.7\psqlparse
    copying psqlparse\exceptions.py -> build\lib.win-amd64-3.7\psqlparse
    copying psqlparse\nodes.py -> build\lib.win-amd64-3.7\psqlparse
    copying psqlparse\__init__.py -> build\lib.win-amd64-3.7\psqlparse
    running build_ext
    error: [WinError 193] %1 Not a valid Win32 application.
    ----------------------------------------
ERROR: Command errored out with exit status 1: 'E:\Anaconda3\python.exe' -u -c 'import sys, setuptools, tokenize; sys.ar
gv[0] = '"'"'C:\\Users\\LE\\AppData\\Local\\TEMP_~1\\pip-install-a9q7ed_5\\psqlparse\\setup.py'"'"'; __file__='"'"
'C:\\Users\\LE\\AppData\\Local\\TEMP_~1\\pip-install-a9q7ed_5\\psqlparse\\setup.py'"'"';f=getattr(tokenize, '"'"'o
pen'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'
"'exec'"'"'))' install --record 'C:\Users\LE\AppData\Local\TEMP_~1\pip-record-jmsarh_5\install-record.txt' --singl
e-version-externally-managed --compile --install-headers 'E:\Anaconda3\Include\psqlparse' Check the logs for full comman
d output.

AttributeError: 'dict' object has no attribute 'tables'

There appears to be a bug with the tables method on certain queries.

This is reproducible using the following query

select * from foo where (a, b) in ((1,2), (3,4)) order by a, b

A full example from a python REPL

>>> import psqlparse
>>> statements = psqlparse.parse('select * from foo where (a, b) in ((1,2), (3,4)) order by a, b')
>>> statements[0].tables()
Traceback (most recent call last):
  File "<input>", line 1, in <module>
    statements[0].tables()
  File "/home/justin/workspace/dbmerge/env/lib/python3.6/site-packages/psqlparse-1.0rc5-py3.6-l
inux-x86_64.egg/psqlparse/nodes/parsenodes.py", line 49, in tables
    _tables |= self.where_clause.tables()
  File "/home/justin/workspace/dbmerge/env/lib/python3.6/site-packages/psqlparse-1.0rc5-py3.6-l
inux-x86_64.egg/psqlparse/nodes/parsenodes.py", line 272, in tables
    _tables |= item.tables()
AttributeError: 'dict' object has no attribute 'tables'

I am using python 3.6.5 and installed psqlparse from a git checkout at commit dc7117a

Can't pip install on MacOS M1 chip

The log reports a missing file as follows:

./src/postgres/include/port/atomics.h:63:10: fatal error: 'port/atomics/arch-arm.h' file not found
  #include "port/atomics/arch-arm.h"
           ^~~~~~~~~~~~~~~~~~~~~~~~~
  1 error generated.
  make: *** [src/pg_query_json_plpgsql.o] Error 1
  
  An error occurred during extension building.
  Make sure you have bison and flex installed on your system.
  ----------------------------------------
  ERROR: Failed building wheel for psqlparse

Create .travis.yml file

Basically, this should do a build of the module:

USE_CYTHON=1 python setup.py build_ext --inplace -f install

Once #5 is done, it should run those tests too, coverage included (see this)

Write tests

We should add a few tests. Basically: pass a query an verify that the proper objects with proper properties where instantiated. Also a wrong query should be pass and verify the Exception thrown.

Is there any interest in a "reformatter" facility?

Hi, I wonder if a SQL pretty printer (same goal as sqlparse.format()) would be considered within the scope of the project, or instead should be packaged in a 3rd party addon.

This is what I already coded:

from psqlparse import parse
from psqlparse.peace import PrettyPrinter

sql = """
SELECT pc.id as x, common.func(pc.name, ' '), 123 FROM ns.table;
SELECT 'accbf276-705b-11e7-b8e4-0242ac120002'::UUID as "X";
SELECT x.id, (select count(*) FROM sometable as y where y.id = x.id) count
from firsttable as x;
select id, count(*) FROM sometable GROUP BY id
order by id desc nulls last;
SELECT id, count(*) FROM sometable GROUP BY id having count(*) > 2
order by count(*) using @> nulls first;
SELECT DISTINCT ON (pc.id) pc.id as x, pc.foo, pc.bar, other.some
FROM ns.table AS pc, ns.other as other
WHERE pc.id < 10 and pc.foo = 'a'
  and (pc.foo = 'b' or pc.foo = 'c'
       and (x = 1 or x = 2));
"""

parsed = parse(sql)
pp = PrettyPrinter()
print(pp(parsed))

that emits:

SELECT pc.id AS x
     , common.func(  pc.name
                   , ' ')
     , 123
FROM ns.table;

SELECT 'accbf276-705b-11e7-b8e4-0242ac120002'::uuid AS "X";

SELECT x.id
     , (SELECT count(*)
        FROM sometable AS y
        WHERE y.id = x.id) AS count
FROM firsttable AS x;

SELECT id
     , count(*)
FROM sometable
GROUP BY id
ORDER BY id DESC NULLS LAST;

SELECT id
     , count(*)
FROM sometable
GROUP BY id
HAVING count(*) > 2
ORDER BY count(*) USING @> NULLS FIRST;

SELECT DISTINCT ON (pc.id) pc.id AS x
                         , pc.foo
                         , pc.bar
                         , other.some
FROM ns.table AS pc
   , ns.other AS other
WHERE pc.id < 10
  AND pc.foo = 'a'
  AND (   pc.foo = 'b'
       OR (    pc.foo = 'c'
           AND (   x = 1
                OR x = 2)))

What do you think?

parse errors when parse create table statement

it can not parse Postgres create table sql , with partition by, example:
sql = """create table aaa
(
id integer default nextval('aaa_id_seq'::regclass) not null,
name varchar
) partition by RANGE (name)"""
print(psqlparse.parse(sql))

throw an exception as:

Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py", line 1491, in _exec
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/Users/chuxiangfeng/PycharmProjects/pythonEffectiveS/tools/sqlparse.py", line 14, in
print(psqlparse.parse(sql))
File "psqlparse/parser.pyx", line 28, in psqlparse.parser.parse (psqlparse/parser.c:1042)
psqlparse.exceptions.PSqlParseError: syntax error at or near "partition"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.