jsommers / pytricia Goto Github PK

It is allowed to add invalid/illegal CIDR notation:

>>> import pytricia
>>> a = pytricia.PyTricia(128)
>>> a['192.168.1.23/24'] = 'Whatever'
>>> a.keys()
['192.168.1.23/24']

It seems to take the network address into account though:

>>> '192.168.1.2' in a
True

I would expect when adding a CIDR/IP address that only valid ones are allowed (subnet/bit-mask wise).

Thanks for Pytricia - I found the API to be super clean and straightforward and easy to use, much appreciated!

Unfortunately, there is a licensing issue with it: the code includes patricia.c, which is licensed under a 4-clause BSD License (i.e. with an advertising clause), and then used from LGPLv3 code.

Unfortunately, 4-clause BSD is not compatible with the (L)GPLs, and this makes the current combination undistributable :/

One option would be to relicense Pytricia to 4-clause BSD, but of course that would just perpetuate the licensing problems and prohibit others from using this software in a GPL program.

A better option would be to switch to a different trie implementation (Patricia or otherwise)... @vincentbernat has an excellent blog post about trie implementations, including pointers to Linux's (GPL) or Free/Open/NetBSD's (3-clause BSD) implementations.

Note that hadiasghari/pyasn#25 seems to be experiencing the same issue and reporters there thought that using Pytricia would help. This seems to be a misunderstanding, as the very same issue applies here too.

Sorry to the bearer of bad news & thanks again for all your efforts!

when i build project，and this error throw。anyone know what？

Workaround: just use dictionary-like assignment syntax for now.

Hi,
thanks for making pytricia available as Free Software!

Regarding the license in detail:

• Wouldn't make sense to have a library like this be available under
  a license with less 'freedom protection'? (The FSF for instance recommends the https://www.gnu.org/licenses/license-list.en.html#apache2 Apache 2.0 license if you want to do this.
  Another option would be the GNU Lesser GPL v>=2.1). Usually for a tiny library: less Freedom protection can result in higher usage. This is good for Free Software because for a tiny library people will often not consider changing the license and alternatively instead of using pytricia just pick a different library.
• Is the current license meant to be GNU GPL v==2 or GNU GPL v>=2? I suggest to add a sentence for this to the README.md. (Usually this is stated with the license file itself and GNU GPL says "or at your option later version" or so. This is good, because otherwise you'll stay incompatible with GNU GPL v>=3 or LGPL v>=3.)

To state my interest: Currently I am helping https://intelmq.org/ and they would rather replace py-radix within pyasn to get the license situation cleared.

Best Regards

I found another bug, related this time to pickling pytricia objects:

import pytricia
pyt = pytricia.PyTricia()
pyt["192.168.0.1"] = "foo"
pickle.dump(pyt, open("test.pkl", "wb"))
pyt2 = pickle.load(open("test.pkl", "rb"))
pyt2.keys()

gives a Segmentation fault (core dumped). I'm using Python 3.

Hello! Seem IPv6 is not supported. I catch segfault in case of IPv6 address.

>>> import pytricia
>>> pyt = pytricia.PyTricia(128)
>>> pyt.insert('32.0.0.0/8', "a")
>>> pyt.get_key('2009:b98:dc0:41:216:3eff:fe27:3d3f')
'32.0.0.0/8'
>>> pyt.get_key('1.2.3.4')
>>> 

For IPv4, this is the expected result (None) but for IPv6, it returns something???

This may be related to bug #19.

Hi,
could you please include COPYING.LESSER and test.py in release tarball?

>>> import pytricia as pyt
>>> p = pyt.PyTricia()
>>> p['205.56.0.0/19'] = 1
>>> p.get_key('205.56.0.0/13')
'205.56.0.0/19'

The less specific prefix appears like it is in the tree, while this should not happen

Please can you update the PyPI repository.

1.0.0 Oct 26, 2017
0.9.6 May 30, 2017
0.9.5 May 30, 2017
0.9.4 Feb 20, 2017
...

This misses the one memory leak in pytricia.c as done in 3f80e86.

Thanks in advance.

Hi!

I'am using a standard python 2.7 64bit environment, and when executing pip install pytricia I receive this error message.
Any suggestion? Should I use 32bit only distribution of python?

Thanks
Rob

Collecting pytricia Using cached pytricia-0.9.4.tar.gz Installing collected packages: pytricia Running setup.py install for pytricia ... error Complete output from command c:\python27\python.exe -u -c "import setuptools, tokenize;__file__='c:\\users\\robert~1 .gra\\appdata\\local\\temp\\pip-build-o0qlxf\\pytricia\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.rea d().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record c:\users\robert~1.gra\appdat a\local\temp\pip-xw9vem-record\install-record.txt --single-version-externally-managed --compile: running install running build running build_ext building 'pytricia' extension creating build creating build\temp.win-amd64-2.7 creating build\temp.win-amd64-2.7\Release C:\............i\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /no logo /Ox /MD /W3 /GS- /DNDEBUG -Ic:\python27\include -Ic:\python27\PC /Tcpytricia.c /Fobuild\temp.win-amd64-2.7\Release\ pytricia.obj pytricia.c pytricia.c(19) : fatal error C1083: Cannot open include file: 'arpa/inet.h': No such file or directory error: command 'C:\\...........\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\ VC\\Bin\\amd64\\cl.exe' failed with exit status 2

Can you add a max length property for the prefixes?

Something like
add "200.100.122.0/20" with max_length=22, so we wouldn't get "200.100.122.0/23" as found.

I'm experiencing a strange bug with IpV6 addresses:

pyt = pytricia.PyTricia()
pyt["2a03:2880:2130:cf05:face:b00c:0:1"] = "foo"
pyt["2a03:2880:f01c:2:face:b00c:0:1"] = "bar"
print(pyt.keys())

Gives the following (correct) answer: ['2a03:2880:f01c:2:face:b00c:0:1/128', '2a03:2880:2130:cf05:face:b00c:0:1/128']. But I get an error if I try to simply get the value of the one of these IPv6 addresses:

pyt["2a03:2880:f01c:2:face:b00c:0:1"]

which gives me: KeyError: 'Prefix not found.'

Any idea about what's happening?

I really like pytricia. But I do have a question
When calling pytricia from a python script I get an importerror.
Importing pytrica from interactive shell goes fine. Also running the script prepended with "python" goes fine. I figured out I have to manually set PYTHONPATH to directory where pytricia is installed (/usr/local/lib/python2.7/site-packages/)

Any idea what's going wrong? (I am using 0.9.4 version, installed via pip)

Below the first few lines of the script:

shell$ cat lookup-prefix.py
#!/usr/bin/python

import pytricia
import textfsm

route_table = pytricia.PyTricia()

+++++++++++++++++++++++
Running the script throws an error:

shell$ ./lookup-prefix.py
Traceback (most recent call last):
File "./lookup-prefix.py", line 3, in
import pytricia
ImportError: No module named pytricia

++++++++++++++++++++++++++++++++++
Importing pytricia via interactive shell goes fine:

shell$ python
Python 2.7.12 (default, Nov 24 2016, 23:19:15)
[GCC 4.8.4] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import pytricia
help(pytricia)

++++++++++++++++++++++++++++++++
Running the script with adding "python" upfront goes fine as well:

shell$ python lookup-prefix.py
Please enter prefix to lookup:

++++++++++++++++++++++++++++++++++
It works fine when manually specifying PYTHONPATH:

shell$ export PYTHONPATH=/usr/local/lib/python2.7/site-packages/
shell$ echo $PYTHONPATH
/usr/local/lib/python2.7/site-packages/
shell$ ./lookup-prefix.py
Please enter prefix to lookup:

The byte lookup is coded for Little Endian systems and breaks on Big Endian machines, yes S390 is alive and kicking.

 self = <test.PyTriciaTests testMethod=testNonStringKey>
 
    def testNonStringKey(self):
         pyt = pytricia.PyTricia()
    
        # insert as string
        pyt['10.1.2.3/24'] = 'abc'
     
        # lookup as string
        for i in range(256):
             self.assertEqual(pyt['10.1.2.{}'.format(i)], 'abc')
     
        # lookup as bytes (or, ugh, another str in python2)
        b = socket.inet_aton('10.1.2.3')
        self.assertEqual(pyt[b], 'abc')
     
        # bytes in py3k.  python2 stinks.
        if sys.version_info.major == 3 and sys.version_info.minor >= 4:
            i = b[0] * 2**24 + b[1] * 2**16 + b[2] * 2**8
            for j in range(256):
>               self.assertEqual(pyt[i+j], 'abc')
E               KeyError: 'Prefix not found.'

Hi, I am running into some weird things.
this is a set of cidr I am trying to add into the tree, and a cidr is lost or somehow deleted during that time.

['10.98.20.0/22', '10.98.20.0/24', '10.98.21.0/24', '10.98.22.0/24', '10.98.23.0/27', '10.98.23.32/27', '10.98.23.64/27', '10.98.24.0/22', '10.98.24.0/24', '10.98.25.0/24', '10.98.26.0/24', '10.98.27.0/24', '10.98.28.0/27', '10.98.29.0/24', '10.108.0.0/16', '10.108.0.0/24', '10.108.2.0/24', '10.108.17.0/24', '10.108.18.0/24', '10.108.19.0/24', '10.108.20.0/24', '10.108.21.0/24', '10.108.23.0/24', '10.108.25.0/24', '10.108.26.0/24', '10.108.27.0/24', '10.108.28.0/24', '10.108.29.0/24', '10.108.30.0/24', '10.108.31.0/24', '10.108.32.0/24', '10.108.33.0/24', '10.108.128.0/24', '10.108.130.0/24', '10.108.132.0/24', '10.108.135.0/24', '10.108.136.0/24', '10.108.137.0/24', '10.108.138.0/24', '10.108.139.0/24', '10.108.140.0/24', '10.108.141.0/24', '10.108.142.0/24', '10.108.143.0/24', '10.108.144.0/24', '10.108.145.0/24', '10.108.146.0/24', '10.108.147.0/24', '10.108.148.0/24', '10.108.149.0/24', '10.108.151.0/24', '10.108.153.0/24', '10.108.154.0/24', '10.108.155.0/24', '10.108.156.0/24', '10.108.166.0/24', '10.108.220.0/24', '10.108.222.0/24', '10.108.232.0/24', '10.108.233.0/24', '10.108.234.0/24', '10.108.236.0/24', '10.108.239.0/24', '10.108.240.0/24', '10.108.242.0/24', '10.108.244.0/24', '10.108.245.0/24', '10.108.246.0/24', '10.108.247.0/24', '10.108.248.0/24', '10.108.249.0/24', '10.108.250.0/24', '10.108.252.0/24', '10.108.254.0/24', '10.108.255.0/24', '10.112.0.0/16', '10.112.0.0/22', '10.112.4.0/22', '10.112.8.0/22', '10.112.12.0/22', '10.112.16.0/22', '10.112.20.0/22', '10.112.24.0/22', '10.112.28.0/22', '10.112.32.0/22', '10.112.36.0/22', '10.112.40.0/22', '10.112.44.0/22', '10.112.48.0/22', '10.112.52.0/22', '10.112.56.0/22', '10.112.60.0/22', '10.112.64.0/22', '10.112.68.0/22', '10.112.72.0/22', '10.112.76.0/22', '10.112.80.0/22', '10.112.84.0/22', '10.112.88.0/22', '10.112.92.0/22', '10.112.96.0/22', '10.112.100.0/22', '10.112.104.0/22', '10.112.108.0/22', '10.112.112.0/22', '10.112.116.0/22', '10.112.120.0/22', '10.112.124.0/22', '10.112.128.0/22', '10.112.132.0/22', '10.112.136.0/22', '10.112.140.0/22', '10.112.144.0/22', '10.112.148.0/22', '10.112.152.0/22', '10.112.156.0/22', '10.112.160.0/22', '10.112.164.0/22', '10.112.168.0/22', '10.112.172.0/22', '10.112.176.0/22', '10.112.180.0/22', '10.112.184.0/22', '10.112.188.0/22', '10.112.192.0/22', '10.112.196.0/22', '10.124.0.0/24', '10.124.1.0/24', '10.124.4.0/22', '10.124.4.0/24', '10.124.5.0/24', '10.124.6.0/24', '10.124.7.0/24', '10.124.8.0/22', '10.124.8.0/24', '10.124.9.0/24', '10.124.10.0/24', '10.124.11.0/24', '10.124.12.0/22', '10.124.12.0/24', '10.124.13.0/24', '10.124.14.0/24', '10.124.15.0/24', '10.124.16.0/22', '10.124.16.0/24', '10.124.17.0/24', '10.124.18.0/24', '10.124.19.0/24', '10.124.20.0/24', '10.124.24.0/22', '10.124.24.0/24', '10.124.25.0/24', '10.124.26.0/24', '10.124.27.0/24', '10.124.28.0/22', '10.124.28.0/24', '10.124.29.0/24', '10.124.30.0/24', '10.124.31.0/24', '10.124.32.0/22', '10.124.32.0/24', '10.124.48.0/22', '10.124.48.0/24', '10.124.49.0/24', '10.124.50.0/24', '10.124.51.0/24', '10.124.148.0/22', '10.124.148.0/24', '10.124.149.0/24', '10.124.150.0/24', '10.124.168.0/22', '10.124.168.0/24', '10.200.12.0/22', '10.200.16.0/22', '2403:28c0:30a:2:a111:cb5d:c7c7:a332/127', '2403:28c0:30a:2::/64', '2408:4002:1025:1008::/64', '2408:4002:1025:100c::/64', '2408:4003:10f3:d20c::/64', '2408:4003:10f3:d208::/64', '2408:4005:384:1208::/64', '2408:4005:384:120c::/64', '2408:400a:400::/56', '2408:400a:c0:e510::/64', '2408:400a:c0:e514::/64', '2408:400a:c0:e50c::/64', '2605:340:cdb1:504::/64', '2605:340:cdb1:500::/56', '2605:340:cdb1:506::/64', '2605:340:cdb1:703::/64', '2605:340:cdb1:b00::/64', '2605:340:cdb1:706::/64', '2605:340:cdb1:600::/64', '2605:340:cdb1:29b::/64', '2605:340:cdb1:2fe::/64', '100.118.58.0/24', '100.118.167.0/24', '100.118.170.0/24', '100.118.171.0/24', '100.118.172.0/24', '100.118.173.0/24', '100.118.173.128/25', '100.118.173.128/32', '100.118.173.128/26']

it seems that a previously inserted cidr : '2605:340:cdb1:b00::/64' is somehow deleted from the tree, when inserting other cidrs.

Hi,
I don't know if it is an expected behaviour, but it does not seem to be when reading the doc.

It seems that the prefix size is not taken into account when doing a lookup:

>>> tree = pytricia.PyTricia()
>>> tree["0.0.0.0/0"] = 1
>>> tree["1.0.0.0/16"] = 2
>>> tree["1.0.0.0/8"]   
2
>>> tree["1.1.0.0/8"]
1

I would expect the two last lines to return 1, as an 8 bits prefix cannot match a 16 bits prefix.

Confirming what @lithammer reported in #8 (comment)_ , the following exceptions are raised for the IPv6 addresses "::", "::1", "::2", ..., "::f", which should all be valid. For some reason, the errors do not occur with "::10".

>>> import pytricia
>>> t = pytricia.PyTricia(128)
>>> t['::'] = 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: error return without exception set
>>> t['::/64'] = 0
>>> t['::']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: Invalid prefix.

It is not clear to me at what point that this bug is triggered, perhaps somewhere in _key_object_to_prefix()?

There seems to be some error cases where methods exit with a NULL pointer, but no exception is defined. This results in a SystemError exception, which is surprising to the user:

>>> pyt['foo.bar']
SystemError: error return without exception set
>>> pyt['2001:db8:/32'] # Notice the missing column at the end
SystemError: error return without exception set

In this case, a KeyError exception would be expected.

This seems related to the prefix parsing code, but there may be other places as well.

It would be neat if prebuilt wheels could be provided for this package on pypi. Especially Windows wheels (possibly via appveyor) would be great, considering how hard compiling python modules is on windows.

Edit: it seems to compile/install on appveyor just fine, so I might a PR for this your way soon.

Windows is currently not supported. At least a couple issues have come up from users attempting a Windows install. Just adding the enhancement req here for future reference.

test.py fails for me with:

[    4s] ======================================================================
[    4s] ERROR: testRaw (test.PyTriciaTests)
[    4s] ----------------------------------------------------------------------
[    4s] Traceback (most recent call last):
[    4s]   File "/home/abuild/rpmbuild/BUILD/pytricia-1.0.0/test.py", line 415, in testRaw
[    4s]     pyt = pytricia.PyTricia(32, socket.AF_INET, True)
[    4s] ValueError: Error parsing prefix length or address family
[    4s] 
[    4s] ======================================================================
[    4s] ERROR: testRawIP6 (test.PyTriciaTests)
[    4s] ----------------------------------------------------------------------
[    4s] Traceback (most recent call last):
[    4s]   File "/home/abuild/rpmbuild/BUILD/pytricia-1.0.0/test.py", line 435, in testRawIP6
[    4s]     pyt = pytricia.PyTricia(128, socket.AF_INET6, True)
[    4s] ValueError: Error parsing prefix length or address family
[    4s] 
[    4s] ----------------------------------------------------------------------

This is with both python 2.7 and python 3.7.