siranthony / slpp Goto Github PK

View Code? Open in Web Editor NEW

118.0 10.0 52.0 71 KB

Simple lua-python parser

License: MIT License

Python 100.00%

slpp's Introduction

SLPP

SLPP is a simple lua-python data structures parser.

Lua data check:

data = '{ array = { 65, 23, 5 }, dict = { string = "value", array = { 3, 6, 4}, mixed = { 43, 54.3, false, string = "value", 9 } } }'
> data = assert(loadstring('return ' .. data))()
> for i,j in pairs(data['dict']) do print(i,j) end
mixed   table: 0x2014290
string  value
array   table: 0x2014200

Parse lua data:

>>> from slpp import slpp as lua
>>> data = lua.decode('{ array = { 65, 23, 5 }, dict = { string = "value", array = { 3, 6, 4}, mixed = { 43, 54.3, false, string = "value", 9 } } }')
>>> print data
{'array': [65, 23, 5], 'dict': {'mixed': {0: 43, 1: 54.33, 2: False, 4: 9, 'string': 'value'}, 'array': [3, 6, 4], 'string': 'value'}}

Dump python object:

>>> lua.encode(data)
'{array = {65,23,5},dict = {mixed = {43,54.33,false,9,string = "value"},array = {3,6,4},string = "value"}}'

Dump test:

> data = assert(loadstring('return ' .. '{array = {65,23,5,},dict = {mixed = {43,54.33,false,9,string = "value",},array = {3,6,4,},string = "value",},}'))()
> print(data['dict'])
table: 0x1b64ea0
> for i,j in pairs(data['dict']) do print(i,j) end
mixed   table: 0x880afe0
array   table: 0x880af60
string  value

slpp's People

Contributors

Stargazers

Watchers

slpp's Issues

use of slpp

I am using this code on a project. Since I am new to this, am I giving appropriate credit to you?
https://github.com/crash-horror/dcs-op-blue-flag
Thanks!

Some unexpect result

print(lua.decode("{[10] = 1}"))
[10
--should be {10:1}
print(lua.decode("{nil}"))
['nil']
--should be {None}
print(lua.decode("{'10'}"))
["'"]
--should be ["10"]

Installing from sdist is broken

README.markdown is not included in the source distribution tarball which setuptools attempts to read; this is when invoking python -m pip install --no-binary :all: slpp==1.2:

...
    ERROR: Command errored out with exit status 1:
     command: /nix/store/jwabyfhz3h2864i4mns3l64a71nc7rwl-python3-3.7.7-env/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/tmpwfrovct9/pip-download-4i8hnkw9/slpp/setup.py'"'"'; __file__='"'"'/tmp/tmpwfrovct9/pip-download-4i8hnkw9/slpp/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/tmpwfrovct9/pip-download-4i8hnkw9/slpp/pip-egg-info
         cwd: /tmp/tmpwfrovct9/pip-download-4i8hnkw9/slpp/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/tmpwfrovct9/pip-download-4i8hnkw9/slpp/setup.py", line 5, in <module>
        with open(path.join(cur_dir, 'README.markdown'), encoding='utf-8') as f:
    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmpwfrovct9/pip-download-4i8hnkw9/slpp/README.markdown'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Numbers in scientific format and float doubling

Numbers in scientific format not processed correctly.
Doubling the first digit after the decimal point.

python 2.4 adjustment diff

Here is the diff I made for fix __encode for python 2.4. I assume it will work for python on up:

def __encode(self, obj):
    s = ''
    tab = self.tab
    newline = self.newline
    tp = type(obj)
    if tp is str:
        s += '"%s"' % obj.replace(r'"', r'\"')
    elif tp in [int, float, long, complex]:
        s += str(obj)
    elif tp is bool:
        s += str(obj).lower()
    elif tp in [list, tuple, dict]:
        self.depth += 1
        if len(obj) == 0 or ( tp is not dict and len(filter(
                lambda x:  type(x) in (int,  float,  long) \
                or (type(x) is str and len(x) < 10),  obj
            )) == len(obj) ):
            newline = tab = ''
        dp = tab * self.depth
        s += "%s{%s" % (tab * (self.depth - 2), newline)
        if tp is dict:
            listtmp=[]
            for k,v in obj.iteritems():
                if type(k) is int:
                    self.__encode(v)
                else:
                    listtmp.append(dp + '%s = %s' % (k, self.__encode(v)))
            s += (',%s' % newline).join(listtmp)

        else:
            s += (',%s' % newline).join(
                [dp + self.__encode(el) for el in obj])
        self.depth -= 1
        s += "%s%s}" % (newline, tab * self.depth)
    return s

I also had to fix the exception handling on line 223

    except ParseError as e: 
          print e 
          return 0

should be changed to

 except ParseError :
      t, e = sys.exc_info()[:2]
      print(e)
      return 0

Numbers in hex format not processed correctly

In arrays such as;

{
    ID = 0x74fa4cae,
    Version = 0x07c2,
    Manufacturer = 0x21544948
};

The output is like this;

{
    1: 'x74fa4cae',
    3: 'x07c2',
    5: 'x21544948',
    'Version': 0,
    'ID': 0,
    'Manufacturer': 0
}

Python 3?

I'm planning to use SLPP in my project for parsing tables from Lua source files. Meantime, I'm preparing my project for transition to Python 3.

As I can see from #12, you still intentionally support ancient Python 2.4. Supporting both Py2.4 and Py3 in same codebase doesn't seems viable.

Whether Py2.4 support is the requirement for SLPP? Would you welcome if I will port SLPP to support Py2.7 and Py3 by the same code?

Of course, I could just copy slpp.py into my project and make required modifications. I'm just curious if your are interested in my pull requests for py3 support.

lua.decode('"--3"') fails

This looks like a bug in the string parsing to me - as if the parser tries to detect comments inside literal strings.

'--3' is a perfectly valid lua string, of course.

I'd like to use your code in my own projects, however I'm rather careful about licensing. I notice on the old google code page slpp is marked as MIT licensed, but I can't see any mention of the license in the GitHub version. I'd be really gratetful if you could explicitly add the license to the code?

No unicode encoding

While I can decode lua objects with unicode characters ok, when I try to encode a dictionary like this

{1: 'testing', 2: {'unicode value': 'κείμενο', 'int value': 456123}}

I get a UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 1: ordinal not in range(128) error
Any fix for this?

Decimal shorthand read as string

If decimals between 0 and 1 are written in shorthand, e.g., .8 rather than 0.8, then they are parsed as strings:

>>> lua.decode("""{ SizeZ = .8, }""")
{'SizeZ': '.8'}
>>> lua.decode("""{ SizeZ = 0.8, }""")
{'SizeZ': 0.8}

About mix

from the test:
s = '{ 43, 54.3, false, string = "value", 9 } }'
from slpp import slpp as lua
data = lua.decode(s)
print(data)

{0: 43, 1: 54.329999999999998, 2: False, 4: 9, 'string': 'value'}
why not
{0: 43, 1: 54.329999999999998, 2: False, 3: 9, 'string': 'value'}

Escaped quotes not accounted for

I have the following snippet from some LUA code:

    UnitName = '<LOC ope3001_name>Arnold\'s Black Box',

The ' appears to throw off the entire operation as the single quote still ends the string. I've hacked line 117 to the following to solve this for my purposes, however I'm not confident in my own knowledge of the entirety of slpp to know if this is best practice or buggy:

FROM

            if self.ch == end:

            if self.ch == end and s[-1] != '\\\':

Missing keys if integers

Encoding this dictionary

{1: 'testing', 2: {'text value': 'this is text', 'int value': 456123}}

returns this

{
"testing",
{
		int value = 456123,
		text value = "this is text"
	}
}

What do I miss here?

NameError: name 'long' is not defined

Tests fail in python 3.7:

======================================================================                         
ERROR: test_unicode (tests.TestSLPP)
----------------------------------------------------------------------                    
Traceback (most recent call last):                                                             
  File "/build/source/tests.py", line 147, in test_unicode
    self.assertEqual(slpp.encode({'s': u'Привет'}), '{\n\ts = "Привет"\n}')                    
  File "/build/source/slpp.py", line 58, in encode  
    return self.__encode(obj)                                                                  
  File "/build/source/slpp.py", line 87, in __encode                       
    key = '[%s]' if all(isinstance(k, (int, long)) for k in obj.keys()) else '%s'              
  File "/build/source/slpp.py", line 87, in <genexpr>
    key = '[%s]' if all(isinstance(k, (int, long)) for k in obj.keys()) else '%s'
NameError: name 'long' is not defined

I guess this is because python3 no longer has long type (https://www.python.org/dev/peps/pep-0237/)

Can't import.

My OS: Arch linux
Python version: 3.8.0
I installed slpp with the following command:
pip3 install git+https://github.com/SirAnthony/slpp
When I try to import a module, the following error appears:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/slpp.py", line 132
    print ERRORS['unexp_end_string']
          ^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(ERRORS['unexp_end_string'])?

Parsing lua strings enclosed with '[[' and ']]' that also contain ']' in between

Parsing this should bring the error to light.

{
    Base=
    {
        {
            Sub = [[ ("word") . [ ["word"] . ["word"] . ("word" | "word" | "word" | "word") . ["word"] ] ]]
        }
    }
}

During the decode process all closing ']' characters are omitted. Also, an additional "1" field is generated for some reason.

Splits strings containing dash (minus symbol)

For example, I think {name = first-name} is acceptable LUA data structure format.

>>> from slpp import slpp
>>> dump = '{name = first-name}'
>>> slpp.decode(dump)
Malformed number (no digits after initial minus).
{'name': 'first', 1: 'name'}

encode issue

from slpp import slpp as lua
print(lua.encode({3:'a'}))
{"a"}
should be {[3] = "a"}
print(lua.encode({'3':'a'}))
{3 = "a"}
should be {["3"] = "a"}

decode bug : missing last zero in lua array

from slpp import slpp as lua
print(lua.decode({0,1,0}))
set([0, 1])
should be set([0, 1, 0])

I comment line in slpp.py:154

if k:

the last zero is not missing now.

what's the purpose of this line?

siranthony / slpp Goto Github PK

slpp's Introduction

SLPP

slpp's People

Contributors

Stargazers

Watchers

Forkers

slpp's Issues

if k:

Recommend Projects

Recommend Topics

Recommend Org