coleifer / unqlite-python Goto Github PK

View Code? Open in Web Editor NEW

387.0 387.0 34.0 1.66 MB

Python bindings for the UnQLite embedded NoSQL database

Home Page: http://unqlite-python.readthedocs.org/

License: MIT License

Python 1.28% C 96.52% Cython 2.20%

embedded-database nosql python unqlite

unqlite-python's People

Contributors

Stargazers

Watchers

unqlite-python's Issues

pass primary key manually

Specifying __id manually is not supported in the python bindings.

    collection.store({
        '__id': 12345
    })

Then

    collection.all()

returns

[{'__id':0}]

It seems that the C++ interface supports a key argument (https://unqlite.org/c_api/unqlite_kv_store.html).

Collections store None instead of numpy numbers

Hi!
This problem arises because numpy numbers and python numbers have different datatypes. Specifically, I've noticed that unqlite collections work fine with values of type int, but they fail to store values of type numpy.uint8.

I can collection.store dictionaries that look like this:

{'VI': 2, 'V': 1, 'VII': 3, 'IV': 0}

The collections object stores the values:
- Correctly if the values are of typeint
- Not at all (all as None) if the values are of typenp.uint8
  - or any numpy type such as uint16, int64, float32, etc

A real `numpy.uint8` test case runs like this

import unqlite
import numpy as np
# Ignore this until I've made my point
from fancy_log_example import log

# Create in-memory database
mem_db = unqlite.UnQLite(':mem:')
collect = mem_db.collection('mem0')
collect.create()

# Store to mem_db collection
def store(keys, values):
    # Store keys and values as a dict
    entry = dict(zip(keys,values))
    print '    Storing:', entry
    # Store dict in your collection
    collect.store(entry)
    # defined below
    log(collect)

# Starwars keys and values
starwars_keys = ['IV','V','VI','VII']
starwars_values = [0, 1, 2, 3]

print 'Successfully store with int values:'
store(starwars_keys, starwars_values)

print 'Fail to store with numpy.uint8 values:'
store(starwars_keys, np.uint8(starwars_values))

The log shows the Main Issue and a potentially related Side Note:

Successfully store with int values:
    Storing: {'VI': 2, 'V': 1, 'VII': 3, 'IV': 0}
    Great values: [0, 1, 2, 3]
Fail to store with numpy.uint8 values:
    Storing: {'VI': 2, 'V': 1, 'VII': 3, 'IV': 0}
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
..............
AssertionError: 
    Main Issue: values [None, None, None, None] for last id 1
    Side Note: current id is somehow still 0

Finally, here's fancy_log_example.py so you can see
- The values from log only come from legitimate uses of your API
- I just assert expected behavior that any values are stored in the collection

def log(collect):
    # Get "last record" and "current record"
    last = collect.fetch(collect.last_record_id())
    current = collect.fetch_current()
    # Side Note: the current and last ids differ
    new_ids = last.pop('__id'), current.pop('__id')

    # Main Issue: ALL the added values have no value.
    added = sorted(last.values())
    assert any(added), '''
    Main Issue: values {} for last id {}
    Side Note: current id is somehow still {}
    '''.format(added, *new_ids)
    # Values are great for python int
    print '    Great values:', added

Please take time to respond to this. I can help you find out where to best fix this in your codebase if you only have time to supply a first impression of where Collection may be going wrong in storing numpy datatypes. Numpy ints or floats in particular are very useful for optimization as they are always a fixed number of bytes. You may find this particular method numpy.ndarray.tostring useful for writing these such values to the database.

Example:

print np.uint32(1).tobytes()
# prints '\x01\x00\x00\x00'

Thanks for your time!

Collection reverse iterator

I was wondering if you had any insight into how I can iterate through the values in a collection in reverse insertion order. I don't see anything in the jx9 docs that does it.

I did notice, however, that for each entry added to a collection, a new value is stored with the key of collectionname_ID. I was thinking a reverse iterator could be made by getting last_record_id() from the collection, creating a Cursor and seeking to collectionname_lastid and then calling previous_entry to seek backwards until it gets to the first value.

Thanks

After certain file size records not getting added

Hi, I am experiencing an issue where I reach a file size and records don't get added anymore. With a file size of 11.1 MB I am attaching the db:

cajeros.db.zip

>>> from unqlite import UnQLite
>>> db = UnQLite('cajeros.db', open_database=True)
>>> str(db.__len__())
'8034'
>>> str(db.__len__())
'7675'
>>> str(db.__len__())
'7675'

With the db I just attached ^, I do:

>>> from unqlite import UnQLite
>>> db = UnQLite('cajeros.db', open_database=True)
>>> str(db.__len__())
'8034'
>>> str(db.__len__())
'7675'
>>> db['04381'] = {'direccion': u'AVENIDA TECNOLOGICO  EDIFICIO DE D.S.P.Y TTO., SAN SALVADOR  TIZATLALI, METEPEC, ESTADO DE MEXICO', 'actualizacion': '2016-05-09 14:15:07.082634', 'lon': -99.57639, 'clave_institucion': 40072, 'nombre_institucion': 'BANORTE-IXE', 'horario': u'08:00-23:59', 'lat': 19.26695, 'cp': '52140', 'id': u'04381'}
>>> str(db.__len__())
'7676'
>>> db.close()
>>> quit()

and after:

>>> from unqlite import UnQLite
>>> db = UnQLite('cajeros.db', open_database=True)
>>> str(db.__len__())
'8034'
>>> str(db.__len__())
'7675'

As you can notice db size is still 7675 not 7676 even I call the db.close(), I tried with smaller file sizes and works correctly

Unqlite on python 3.10 fails with unqlite.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send

I can install it via wheel, but I get the above error when trying to import it.

Readme update

https://github.com/symisc/unqlite has had a release this year and some announcements over past few years. I guess the README could be update wrt. since 2014

Dictionaries can't be written to collections if keys are Unicode strings

coll = db.collection('things')
coll.create()
coll.store({u'dog':u'woof', u'cow':u'moo'})

produces the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 718, in store
    return self._simple_execute(script, record=record)
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 662, in _simple_execute
    with self._execute(script, **kwargs) as vm:
  File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 657, in _execute
    vm[key] = value
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 618, in __setitem__
    self.set_value(name, value)
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 586, in set_value
    ptr = self.create_value(value)
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 523, in create_value
    self._set_value(ptr, value)
  File "/work/virtualenvs/unql/lib/python2.7/site-packages/unqlite/core.py", line 507, in _set_value
    item_ptr))
ctypes.ArgumentError: argument 2: <type 'exceptions.AttributeError'>: 'unicode' object has no attribute '_as_parameter_'

This is especially a problem when dealing with things read in through Python's built-in JSON parser, as it outputs Unicode keys by default.

Installation

Got an Install error while using pip install :)

sudo pip install unqlite
[sudo] password for ****:
Downloading/unpacking unqlite
Downloading unqlite-0.1.3.tar.gz (524kB): 524kB downloaded
Running setup.py egg_info for package unqlite

warning: no files found matching 'LICENSE'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.o' found anywhere in distribution
warning: no previously-included files matching '*.so.0.1' found anywhere in distribution

Installing collected packages: unqlite
Running setup.py install for unqlite
error: Permission denied
Complete output from command /usr/bin/python -c "import setuptools;file='/tmp/pip_build_root/unqlite/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-jm2lD2-record/install-record.txt --single-version-externally-managed:
running install

running build

running build_py

error: Permission denied

Cleaning up...
Command /usr/bin/python -c "import setuptools;file='/tmp/pip_build_root/unqlite/setup.py';exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-jm2lD2-record/install-record.txt --single-version-externally-managed failed with error code 1 in /tmp/pip_build_root/unqlite
Storing complete log in /home/****/.pip/pip.log

python 3

How can we use Unqlite with python 3. ?? Any plan for porting soon?

Question about multithread

I install unqlite using "pip install unqlite" command. But such codes generate a file named "test.db_unqlite_journal" and data cannot be written into the file "test.db". If the line 'timer.start()' is deleted, then it works well. Cannot unqlite be used in multithread environment?

from threading import Thread
from unqlite import UnQLite
import time

class Timer(Thread):
    def run(self):
        while True:
            time.sleep(1)

db = UnQLite("test.db")
users = db.collection("users")
users.create()
users.store({'name':'Tom', 'age':23})

timer = Timer()
timer.start()

time.sleep(1000)

Flags unavailable via Python

I could not access the open flags via the Python implementation:

UNQLITE_OPEN_CREATE
UNQLITE_OPEN_READONLY
UNQLITE_OPEN_READWRITE
UNQLITE_OPEN_CREATE
UNQLITE_OPEN_EXCLUSIVE
UNQLITE_OPEN_TEMP_DB
UNQLITE_OPEN_NOMUTEX
UNQLITE_OPEN_OMIT_JOURNALING
UNQLITE_OPEN_IN_MEMORY
UNQLITE_OPEN_MMAP

They are defined as cdef in unqlite.pyx. Should they be defined as cpdef?

ValueError when inserting to nonexistent collection

db = UnQLite()
users = db.collection('users')
users.store({'a': 0})

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
unqlite.pyx in unqlite.Collection._simple_execute()

unqlite.pyx in unqlite.VM.__getitem__()

unqlite.pyx in unqlite.VM.get_value()

KeyError: 'ret'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
<ipython-input-36-2102913e9c8f> in <module>
----> 1 users.store({'a': 0}) # ValueError

unqlite.pyx in unqlite.Collection.store()

unqlite.pyx in unqlite.Collection._simple_execute()

unqlite.pyx in unqlite.Collection._simple_execute()

ValueError: Error fetching return value from script.

This error is not descriptive.

Iteration Run Time

the run time for iterating directly over the database is really slow. I suggest using a faster datatype like numpy arrays for unit storage at least for the in-memory, but the doc storage is just fine.

AsyncIO Support?

It would be quite handy if this supported async/AsyncIO in newer versions of python... assuming that the necessary features already exist in unqlite of course 😄

Python 3

Hello!

Problem

Unqlite-python doesn't seem to be python3-compatible. When trying to python setup.py build in a python3 virtualenv, I run into

running build
running build_py
Traceback (most recent call last):
  File "ctypesgen/ctypesgen.py", line 38, in <module>
    import ctypesgencore
  File "/home/sl/Code/Desktop/bising/unqlite-python/ctypesgen/ctypesgencore/__init__.py", line 56, in <module>
    import processor
ImportError: No module named 'processor'
Traceback (most recent call last):
  File "setup.py", line 61, in <module>
    cmdclass={'build_py': GenerateCtypesWrapper},
  File "/usr/lib/python3.4/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.4/distutils/command/build.py", line 126, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 28, in run
    wrapper])
  File "/usr/lib/python3.4/subprocess.py", line 561, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'ctypesgen/ctypesgen.py', 'unqlite/src/unqlite.h', '-L', './', '-l', 'unqlite', '-o', '/home/sl/Code/Desktop/bising/unqlite-python/unqlite/_unqlite.py']' returned non-zero exit status 1

which indeed comes from relative imports in ctypesgen/ctypesgencore/__init__.py (import parser only works as from . import parserin python3, etc.). FIxing all those, I then run into

running build
running build_py
Traceback (most recent call last):
  File "ctypesgen/ctypesgen.py", line 38, in <module>
    import ctypesgencore
  File "/home/sl/Code/Desktop/bising/unqlite-python/ctypesgen/ctypesgencore/__init__.py", line 55, in <module>
    from . import parser
  File "/home/sl/Code/Desktop/bising/unqlite-python/ctypesgen/ctypesgencore/parser/__init__.py", line 17, in <module>
    from .datacollectingparser import DataCollectingParser
  File "/home/sl/Code/Desktop/bising/unqlite-python/ctypesgen/ctypesgencore/parser/datacollectingparser.py", line 10, in <module>
    from . import ctypesparser
  File "/home/sl/Code/Desktop/bising/unqlite-python/ctypesgen/ctypesgencore/parser/ctypesparser.py", line 15, in <module>
    from .cparser import *
  File "/home/sl/Code/Desktop/bising/unqlite-python/ctypesgen/ctypesgencore/parser/cparser.py", line 199
    print '#define name=%r, value=%r' % (name, value)
                                    ^
SyntaxError: invalid syntax
Traceback (most recent call last):
  File "setup.py", line 61, in <module>
    cmdclass={'build_py': GenerateCtypesWrapper},
  File "/usr/lib/python3.4/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.4/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.4/distutils/command/build.py", line 126, in run
    self.run_command(cmd_name)
  File "/usr/lib/python3.4/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.4/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 28, in run
    wrapper])
  File "/usr/lib/python3.4/subprocess.py", line 561, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'ctypesgen/ctypesgen.py', 'unqlite/src/unqlite.h', '-L', './', '-l', 'unqlite', '-o', '/home/sl/Code/Desktop/bising/unqlite-python/unqlite/_unqlite.py']' returned non-zero exit status 1

before realizing this code isn't meant for python3 at all.

Question

Are there any plans for python3 support? An ETA?

Use in multi-process environment?

I have multiple processes reading/writing from a database and keep getting the following exception:

unqlite.UnQLiteError: b'Another process or thread hold the requested lock\nAnother process or thread have a reserved lock on this database\n'

Here it says that unqlite is thread-safe with the UNQLITE_ENABLE_THREADS flag turned on during compilation. Does this python package use a compiled version of unqlite with that flag enabled? If not, how would I create a version that does?

Slow performance when dropping collection

Python 3.6.3
unqlite 0.6.0

I've create a collection with 10,000 entries. When starting the application, I need to clear the collection and I'm using drop() to do so.

The db is about 25 Mib. But it takes forever to do the drop. I don't know exactly how many times because I always kill it before it reach the end but the last time I've kill it after 10 minutes.
During this time, one CPU is used at 100%. I'm unable to do a CTRL+C, I need the kill the process.
Inserting the records in a fresh db only took me seconds.
Why is it so slow?

with db.transaction():
        if __collPlanip.exists():
            __collPlanip.drop() # it blocks here
        __collPlanip.create()

error installing on Windows 10

Received the following error when attempting to install in Python34\Lib\ folder:

Directory of C:\Python34\Lib\unqlite-python-master

08/08/2015 01:44 PM

.
08/08/2015 01:44 PM ..
08/08/2015 01:44 PM docs
08/08/2015 01:44 PM 1,058 LICENSE
08/08/2015 01:44 PM 223 MANIFEST.in
08/08/2015 01:44 PM 7,412 README.md
08/08/2015 01:44 PM 762 setup.py
08/08/2015 01:44 PM src
08/08/2015 01:44 PM 15,364 tests.py
08/08/2015 01:44 PM 951,341 unqlite.c
08/08/2015 01:44 PM 39,819 unqlite.pyx
7 File(s) 1,015,979 bytes
4 Dir(s) 626,046,054,400 bytes free

C:\Python34\Lib\unqlite-python-master>pip install unqlite
You are using pip version 6.0.8, however version 7.1.0 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting unqlite
Using cached unqlite-0.4.1.tar.gz
C:\Users\Bob\AppData\Local\Temp\pip-build-__uoexdg\unqlite\setup.py:9: UserW
arning: Cython not installed, using pre-generated C source file.
warnings.warn('Cython not installed, using pre-generated C source file.')
Installing collected packages: unqlite
Running setup.py install for unqlite
building 'unqlite' extension
C:\Users\Bob\AppData\Local\Temp\pip-build-__uoexdg\unqlite\setup.py:9: UserW
arning: Cython not installed, using pre-generated C source file.
warnings.warn('Cython not installed, using pre-generated C source file.')
error: Microsoft Visual C++ 10.0 is required (Unable to find vcvarsall.bat).

Complete output from command C:\Python34\python.exe -c "import setuptools, t

okenize;file='C:\Users\Bob\AppData\Local\Temp\pip-build-uoexdg\unql
ite\setup.py';exec(compile(getattr(tokenize, 'open', open)(__file).read().rep
lace('\r\n', '\n'), file, 'exec'))" install --record C:\Users\Bob\AppData\Lo
cal\Temp\pip-zz_9rkjb-record\install-record.txt --single-version-externally-mana
ged --compile:
running install

running build

running build_ext

building 'unqlite' extension

C:\Users\Bob\AppData\Local\Temp\pip-build-__uoexdg\unqlite\setup.py:9: UserW

arning: Cython not installed, using pre-generated C source file.

  warnings.warn('Cython not installed, using pre-generated C source file.')

error: Microsoft Visual C++ 10.0 is required (Unable to find vcvarsall.bat).


----------------------------------------
Command "C:\Python34\python.exe -c "import setuptools, tokenize;__file__='C:

\Users\Bob\AppData\Local\Temp\pip-build-uoexdg\unqlite\setup.py';exec(
compile(getattr(tokenize, 'open', open)(__file).read().replace('\r\n', '\n'),
file, 'exec'))" install --record C:\Users\Bob\AppData\Local\Temp\pip-zz_9rkj
b-record\install-record.txt --single-version-externally-managed --compile" faile
d with error code 1 in C:\Users\Bob\AppData\Local\Temp\pip-build-__uoexdg\unqlit
e

C:\Python34\Lib\unqlite-python-master>

Wrong kwarg name in UnQLite() API docs.

reading the docs it says
UnQLite([database=':mem:' ... ])

and I think it should say
UnQLite([filename=':mem:' ... ])

Broken on Python3

I installed the bindings on Python 3.4 and got the following:

Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar  6 2015, 12:07:41)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import unqlite
>>> db = unqlite.UnQLite()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "unqlite.pyx", line 285, in unqlite.UnQLite.__init__ (unqlite.c:1888)
  File "unqlite.pyx", line 296, in unqlite.UnQLite.open (unqlite.c:2003)
TypeError: expected bytes, str found
>>>

Are the bindings not supported on Python 3 and I'm just missing it in the docs?

issues with exceptions

Two issues I noticed:

in your exception-raiser code you don't set the errno with the unqlite return code so an exception handler needs to examine the message - but this not easy to do because there is more than one message for a single unqlite error condition. You are forced to look through the .c file and extract strings, etc. Not much fun.
I have code that handles exceptions by doing a retry. If there are N exceptions on a particular operation, the exception is finally raised. There must be a message buffer that is appended to, but never cleared out, so I end up with exception messages that look like this:


IOError: Another process or thread hold the requested lock           
Another process or thread have a reserved lock on this database      
Another process or thread hold the requested lock                    
Another process or thread have a reserved lock on this database      
Another process or thread hold the requested lock                    
Another process or thread have a reserved lock on this database      
Another process or thread hold the requested lock                    
Another process or thread have a reserved lock on this database      
Another process or thread hold the requested lock

The above is just one message, with those repeats in it. That actually might kind of be useful, possibly, I guess. But not super useful. More of a pain. If I do more than about ten retries on a
store or commit, python usually crashes with a segfault. I suspect this is related.

I have seen an occasional file handle leak in my multi-process access exception testing. I am still working on coming up with a more consistently reproducible test case, but I thought I'd throw that out there as an FYI.

Thanks

segfault when trying to store into a collection

Python 2.7.3 (default, Mar 14 2014, 17:55:54)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>>
>>> import unqlite
>>> db = unqlite.UnQLite('db1.unqlite')
>>> c1 = db.collection('c1')
>>> c1.create()
>>> db.commit()
>>> c1.store({1:2})
Exception TypeError: 'expected string or Unicode object, int found' in 'unqlite.VM.create_value' ignored
[1]    18970 segmentation fault  python

Python 2.7.3 (default, Mar 14 2014, 17:55:54)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import unqlite
>>> db = unqlite.UnQLite('db1.unqlite')
>>> c2 = db.collection('c2')
>>> c2.store({'f':'f'})
[1]    19671 segmentation fault  python

# pip list|grep -i unq
unqlite (0.4.1)

Add Collection.update

UnQLite recently added db_update_record, so unqlite-python should add support for it as a method to the Collection object.

Errors on Install Example

I installed using "pip install Cython unqlite"

Then I run this:

python -c "import unqlite; unqlite.UnQLite()"

I get this:

Traceback (most recent call last):
File "<string>", line 1, in <module>
File "unqlite.pyx", line 285, in unqlite.UnQLite.__init__ (unqlite.c:1888)
File "unqlite.pyx", line 296, in unqlite.UnQLite.open (unqlite.c:2003)
TypeError: expected bytes, str found

My virtualenv is using Python version 3.4.3

empty_collection.last_record_id() is 0

db = UnQLite()
users = db.collection('users')
users.create()
users.last_record_id() # 0

It should be False.
https://unqlite.org/jx9_func/db.html#db_last_record_id

Retrieved values are byte strings (Python 3.7)

Any value I retrieve from the database is a bytes object:

Python 3.7.2 (default, Feb 12 2019, 08:16:38) 
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from unqlite import UnQLite
>>> db = UnQLite()
>>> db['a'] = 'hello'
>>> db['a']
b'hello'
>>> {**db} # note: keys are strings.
{'a': b'hello'}

This means that if I have deeply nested collections, I have to write some convoluted boilerplate code to convert bytes back to strings. I suspect this might be related to my version of Python, because all the examples in the documentation seem to yield different results.
Thanks.

Update db entry in multiple threads

Hello! Can anyone explain to me how to upgrade database entries in parallel? I have a db file opened in one process that needs to iterate over all entries while updating with a certain key (different values) if not already existent. I need a few hundred concurrent threads.

#!/usr/bin/python3

from unqlite import UnQLite
from concurrent.futures import ThreadPoolExecutor

db = UnQLite('tmp.db')
coll = db.collection('coll')
coll.create()

def fill_db():
    global coll
    for i in range(10):
        s = dict()
        s['id'] = i
        s['flag'] = True
        coll.store(s)

def show_db():
    global coll
    for i in range(len(coll)):
        s = coll.fetch(i)
        print('entry', i, '=>', s)

def upd(idx):
    global coll
    print('Working on', idx)
    c = coll.fetch(idx)
    c['other'] = 1
    if not c.update(idx, c):
        print('Error during update', idx)
    else:
        coll.commit()

def update_db():
    global coll

    with ThreadPoolExecutor(max_workers=2) as pool:
        for i in range(len(coll)):
            if 'other' in coll[i].keys():
                continue
            
            # update each item if key 'other' is not yet present
            pool.submit(upd, i)

        print('Done!')
        pool.shutdown(wait=True)

if __name__ == '__main__':
    fill_db()
    show_db()
    update_db()
    show_db()

    db.close()

Python 3.6.5
I can not find the __ version __ string for unqlite. But the filename is: unqlite.cpython-36m-i386-linux-gnu.so

Import error

unqlite.cpython-310-x86_64-linux-gnu.so: undefined symbol: _PyGen_Send

Python version: 3.10

Boolean functionality seems "off"

From the API docs you get the idea that unqlite can support booleans. However I'm getting this result

>>> from unqlite import UnQLite
>>> db = UnQLite()
>>> db.collection('test').create()
>>> db.collection('test').store({'booltest': False,'another': True})
>>> db.collection('test').all()
[{'booltest': 0, '__id': 0, 'another': 1}]
>>> db['anothertest'] = True
>>> db['anothertest']
'True'
>>> type(db['anothertest'])
<type 'str'>

Tested with unqlite (0.5.2) and Python 2.7.6

retrieving db keys causes some keys to become invisible to subsequent lookups

Using unqlite-python 0.5.3 with Python 2.7.11 on MacOS 10.11.6, the print statement after the second loop prints 488, not 500. This doesn't occur if I comment out keys = [k for k in db].

import unqlite
file_name = 'test.db'

db = unqlite.UnQLite(file_name)
for i in range(500):
    db['key_%s' % i] = 100*'x'
db.close()

db = unqlite.UnQLite(file_name)
keys = [k for k in db]
print len(keys)

count = 0
for k in db.keys():
    db[k]
    count += 1
print count
db.close()

Confusing documentation regarding accessing collection items

This is in the docs;

>>> users.fetch(0)  # Fetch the first record.

Does it fetch the first record, or the record with ID number 0? I believe it is the latter. Perhaps the wording could be improved.

Similarily with delete;

>>> users.delete(0)  # Delete the first record.

If I run that twice, would it delete record with ID number 0, then record with ID number 1, because that is what it currently implies?

Installation error: AttributeError: class Extension has no attribute 'mro'

Hi,

Gave me following error while installing on . I've tried with pip and manual installation from source but no luck.

From source:

python setup.py build
Traceback (most recent call last):
  File "setup.py", line 48, in <module>
    ext_modules=cythonize(unqlite_extension)
  File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 818, in cythonize
    aliases=aliases)
  File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 648, in create_extension_list
    elif isinstance(patterns, basestring) or not isinstance(patterns, collections.Iterable):
  File "/usr/lib/python2.7/abc.py", line 144, in __instancecheck__
    return cls.__subclasscheck__(subtype)
  File "/usr/lib/python2.7/abc.py", line 180, in __subclasscheck__
    if issubclass(subclass, scls):
  File "/usr/lib/python2.7/abc.py", line 180, in __subclasscheck__
    if issubclass(subclass, scls):
  File "/usr/lib/python2.7/abc.py", line 161, in __subclasscheck__
    ok = cls.__subclasshook__(subclass)
  File "/usr/lib/python2.7/dist-packages/backports_abc.py", line 66, in __subclasshook__
    mro = C.__mro__
AttributeError: class Extension has no attribute '__mro__'

With pip:

#pip install cython unqlite
Requirement already satisfied: cython in /usr/local/lib/python2.7/dist-packages
Collecting unqlite
  Using cached unqlite-0.6.0.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-H9POLw/unqlite/setup.py", line 48, in <module>
        ext_modules=cythonize(unqlite_extension)
      File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 818, in cythonize
        aliases=aliases)
      File "/usr/local/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 648, in create_extension_list
        elif isinstance(patterns, basestring) or not isinstance(patterns, collections.Iterable):
      File "/usr/lib/python2.7/abc.py", line 144, in __instancecheck__
        return cls.__subclasscheck__(subtype)
      File "/usr/lib/python2.7/abc.py", line 180, in __subclasscheck__
        if issubclass(subclass, scls):
      File "/usr/lib/python2.7/abc.py", line 180, in __subclasscheck__
        if issubclass(subclass, scls):
      File "/usr/lib/python2.7/abc.py", line 161, in __subclasscheck__
        ok = cls.__subclasshook__(subclass)
      File "/usr/lib/python2.7/dist-packages/backports_abc.py", line 66, in __subclasshook__
        mro = C.__mro__
    AttributeError: class Extension has no attribute '__mro__'

Install failed

I tried to do a sudo pip install unqlite however I get an error where gcc failed with exit status 1
here is the whole output:

Downloading/unpacking unqlite
  Running setup.py egg_info for package unqlite

Installing collected packages: unqlite
  Running setup.py install for unqlite
    building 'unqlite' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c unqlite.c -o build/temp.linux-armv7l-2.7/unqlite.o
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c src/unqlite.c -o build/temp.linux-armv7l-2.7/src/unqlite.o
    src/unqlite.c: In function ‘unqlite_value_string_format’:
    src/unqlite.c:4832:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘jx9_value_string_format’:
    src/unqlite.c:8487:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘jx9InputFormat’:
    src/unqlite.c:12182:8: warning: variable ‘zExtra’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘jx9CompileBreak’:
    src/unqlite.c:18336:8: warning: variable ‘nLine’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘GenStateCollectFuncArgs’:
    src/unqlite.c:19799:11: warning: variable ‘pCur’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘InternFormat’:
    src/unqlite.c:29077:9: warning: variable ‘zExtra’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘ParseEndOfCentralDirectory’:
    src/unqlite.c:29829:9: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
    src/unqlite.c: In function ‘SyHexStrToInt64.constprop.269’:
    src/unqlite.c:28363:1: error: unrecognizable insn:
    (insn 509 508 510 26 (set (subreg:SI (reg:DI 277 [ prephitmp.7687 ]) 0)
            (sign_extend:SI (mem/s/u/j:QI (plus:SI (reg:SI 329)
                        (const_int 1100 [0x44c])) [0 CSWTCH.1189 S1 A8]))) src/unqlite.c:28345 -1
         (nil))
    src/unqlite.c:28363:1: internal compiler error: in extract_insn, at recog.c:2109
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.
    Preprocessed source stored into /tmp/ccy394Q6.out file, please attach this to your bugreport.
    error: command 'gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools;__file__='/home/pi/pathfinderCollector/tcollector/build/unqlite/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-nZsmB3-record/install-record.txt:
    running install

running build

running build_ext

building 'unqlite' extension

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c unqlite.c -o build/temp.linux-armv7l-2.7/unqlite.o

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -c src/unqlite.c -o build/temp.linux-armv7l-2.7/src/unqlite.o

src/unqlite.c: In function ‘unqlite_value_string_format’:

src/unqlite.c:4832:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘jx9_value_string_format’:

src/unqlite.c:8487:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘jx9InputFormat’:

src/unqlite.c:12182:8: warning: variable ‘zExtra’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘jx9CompileBreak’:

src/unqlite.c:18336:8: warning: variable ‘nLine’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘GenStateCollectFuncArgs’:

src/unqlite.c:19799:11: warning: variable ‘pCur’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘InternFormat’:

src/unqlite.c:29077:9: warning: variable ‘zExtra’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘ParseEndOfCentralDirectory’:

src/unqlite.c:29829:9: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]

src/unqlite.c: In function ‘SyHexStrToInt64.constprop.269’:

src/unqlite.c:28363:1: error: unrecognizable insn:

(insn 509 508 510 26 (set (subreg:SI (reg:DI 277 [ prephitmp.7687 ]) 0)

        (sign_extend:SI (mem/s/u/j:QI (plus:SI (reg:SI 329)

                    (const_int 1100 [0x44c])) [0 CSWTCH.1189 S1 A8]))) src/unqlite.c:28345 -1

     (nil))

src/unqlite.c:28363:1: internal compiler error: in extract_insn, at recog.c:2109

Please submit a full bug report,

with preprocessed source if appropriate.

See <file:///usr/share/doc/gcc-4.6/README.Bugs> for instructions.

Preprocessed source stored into /tmp/ccy394Q6.out file, please attach this to your bugreport.

error: command 'gcc' failed with exit status 1

----------------------------------------
Command /usr/bin/python -c "import setuptools;__file__='/home/pi/pathfinderCollector/tcollector/build/unqlite/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --single-version-externally-managed --record /tmp/pip-nZsmB3-record/install-record.txt failed with error code 1 in /home/pi/pathfinderCollector/tcollector/build/unqlite
Storing complete log in /root/.pip/pip.log

subprocess.CalledProcessError: Command *** returned non-zero exit status 1

Hello,

when I try to install unqlite using pip, i get this error:

Downloading/unpacking unqlite
  Downloading unqlite-0.1.9.tar.gz (528kB): 528kB downloaded
  Running setup.py (path:/home/dan/prac/.virtualenvs/work/build/unqlite/setup.py) egg_info for package unqlite

    warning: no previously-included files found matching 'unqlite/_unqlite.py'
    warning: no previously-included files matching '*.pyc' found anywhere in distribution
    warning: no previously-included files matching '*.o' found anywhere in distribution
    warning: no previously-included files matching '*.so.0.1' found anywhere in distribution
Installing collected packages: unqlite
  Running setup.py install for unqlite
    Traceback (most recent call last):
      File "ctypesgen/ctypesgen.py", line 38, in <module>
        import ctypesgencore
      File "/home/dan/prac/.virtualenvs/work/build/unqlite/ctypesgen/ctypesgencore/__init__.py", line 56, in <module>
        import processor
    ImportError: No module named 'processor'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/home/dan/prac/.virtualenvs/work/build/unqlite/setup.py", line 61, in <module>
        cmdclass={'build_py': GenerateCtypesWrapper},
      File "/usr/lib64/python3.4/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib64/python3.4/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/dan/prac/.virtualenvs/work/lib/python3.4/site-packages/setuptools/command/install.py", line 54, in run
        return _install.run(self)
      File "/usr/lib64/python3.4/distutils/command/install.py", line 539, in run
        self.run_command('build')
      File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib64/python3.4/distutils/command/build.py", line 126, in run
        self.run_command(cmd_name)
      File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/dan/prac/.virtualenvs/work/build/unqlite/setup.py", line 28, in run
        wrapper])
      File "/usr/lib64/python3.4/subprocess.py", line 561, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['python', 'ctypesgen/ctypesgen.py', 'unqlite/src/unqlite.h', '-L', './', '-l', 'unqlite', '-o', '/home/dan/prac/.virtualenvs/work/build/unqlite/unqlite/_unqlite.py']' returned non-zero exit status 1
    Complete output from command /home/dan/prac/.virtualenvs/work/bin/python3 -c "import setuptools, tokenize;__file__='/home/dan/prac/.virtualenvs/work/build/unqlite/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-uz190krz-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/dan/prac/.virtualenvs/work/include/site/python3.4:
    running install

running build

running build_py

Traceback (most recent call last):

  File "ctypesgen/ctypesgen.py", line 38, in <module>

    import ctypesgencore

  File "/home/dan/prac/.virtualenvs/work/build/unqlite/ctypesgen/ctypesgencore/__init__.py", line 56, in <module>

    import processor

ImportError: No module named 'processor'

Traceback (most recent call last):

  File "<string>", line 1, in <module>

  File "/home/dan/prac/.virtualenvs/work/build/unqlite/setup.py", line 61, in <module>

    cmdclass={'build_py': GenerateCtypesWrapper},

  File "/usr/lib64/python3.4/distutils/core.py", line 148, in setup

    dist.run_commands()

  File "/usr/lib64/python3.4/distutils/dist.py", line 955, in run_commands

    self.run_command(cmd)

  File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()

  File "/home/dan/prac/.virtualenvs/work/lib/python3.4/site-packages/setuptools/command/install.py", line 54, in run

    return _install.run(self)

  File "/usr/lib64/python3.4/distutils/command/install.py", line 539, in run

    self.run_command('build')

  File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()

  File "/usr/lib64/python3.4/distutils/command/build.py", line 126, in run

    self.run_command(cmd_name)

  File "/usr/lib64/python3.4/distutils/cmd.py", line 313, in run_command

    self.distribution.run_command(command)

  File "/usr/lib64/python3.4/distutils/dist.py", line 974, in run_command

    cmd_obj.run()

  File "/home/dan/prac/.virtualenvs/work/build/unqlite/setup.py", line 28, in run

    wrapper])

  File "/usr/lib64/python3.4/subprocess.py", line 561, in check_call

    raise CalledProcessError(retcode, cmd)

subprocess.CalledProcessError: Command '['python', 'ctypesgen/ctypesgen.py', 'unqlite/src/unqlite.h', '-L', './', '-l', 'unqlite', '-o', '/home/dan/prac/.virtualenvs/work/build/unqlite/unqlite/_unqlite.py']' returned non-zero exit status 1

Python 3.4 and Arch Linux x64.

Possibly outdated warning

I had a look at the issue you referenced from your readme and it seems to be fixed. Perhaps you should re-evaluate your warning.

Disclaimer: I'm not affiliated with anyone; I'm just a developer who wants cool projects to have a fair chance. It would be a shame for a project like this to have support die away because of a misunderstanding.

Disclaimer for Disclaimer: I completely accept the fact that the project may still be dangerous to use.

Trying to create a "not in-memory db"

In [4]: unqlite
Out[4]: <module 'unqlite' from '/usr/local/lib/python2.7/dist-packages/unqlite/init.pyc'>

from unqlite import UnQLite
db = UnQLite("/media/HDD/UnQLite_DB_Storage/")

In [6]: db
Out[6]: <unqlite.core.UnQLite at 0x1b4ead0>

In [5]: db['foo'] = 'bar'
Exception Traceback (most recent call last)
in ()
----> 1 db['foo'] = 'bar'

/usr/local/lib/python2.7/dist-packages/unqlite/core.pyc in store(self, key, value)
153 -1,
154 value,
--> 155 len(value)))
156
157 def store_fmt(self, key, value, *params):

/usr/local/lib/python2.7/dist-packages/unqlite/core.pyc in handle_return_value(rc)
22 UNQLITE_NOTIMPLEMENTED: 'Not implemented',
23 UNQLITE_READ_ONLY: 'Database is in read-only mode',
---> 24 }.get(rc, 'Unknown exception: %s' % rc))
25 return True
26

Exception: OS error

It returning me an Error, I don't know if I'm doing it right but it seems to be correct referring to the documentation:

(Otherwise when I'm creating a db "in-memory" it's works correctly :)

Collection.store eventually fails after adding records inside with db.transaction()

I'm trying to convert a small project from an sqlite to unqlite database, and ran into an issue while trying to import the data from sqlite. After importing 409 events into a collection, Collection.store reliably fails with the error "ValueError: Error fetching return value from script"

Here's a minimal script that can reproduce the issue:

#!/usr/bin/env python

import unqlite

unq = unqlite.UnQLite('test.unq')

events = unq.collection('events')
events.create()
for i in xrange(10000):
    with unq.transaction():
        print events.fetch(events.store({'foo': 'bar'}))

it dies for me after adding 1016 events:

...
{'foo': 'bar', '__id': 1014}
{'foo': 'bar', '__id': 1015}
Traceback (most recent call last):
  File "./test.py", line 11, in <module>
    print events.fetch(events.store({'foo': 'bar'}))
  File "unqlite.pyx", line 1141, in unqlite.Collection.store (unqlite.c:18202)
  File "unqlite.pyx", line 1046, in unqlite.Collection._simple_execute (unqlite.c:16851)
  File "unqlite.pyx", line 1054, in unqlite.Collection._simple_execute (unqlite.c:16789)
ValueError: Error fetching return value from script.

API Documentation of collection.store()

API Documentation of collection.store() claims that a boolean value is returned, however i have seen that a record ID is returned instead.

Cython dependency fails pip install

Expectation:
Adding both cython and unqlite to a pip requirements.txt will install both libraries.

Issue:
If the package is installed with pip, the installation may fail unless Cython is explicitly installed first (ie. you first run pip install cython and only after that install the rest of the requirements). It appears that this dependency is not specified explicitly and thus pip can not reason about installation order.

Collection cursor only returns first record

Instead of fulling loading the entire collection into memory, I'd like to fetch one record at a time from a collection.
The API seems to indicate that the Collection object can behave like a cursor however it doesn't seem to work correctly.

Collection.current_record_id() seems to always return 0 no matter what. For example, I can store a record, delete it, and then store a new record. Record 0 shouldn't exist but then current_record_id() still reports 0.

Collection.fetch_current() always returns the first record in the collection. It behaves as if it doesn't advance the cursor. Looking at the implementation it looks like it just calls the jx9 db_fetch(). The unqlite documentation says that it should return the current document and then advance the cursor. I suspect what's happening is that it's a new cursor for each new VM instance and every call to Collection creates a new VM instance, compiles a new jx9 script, and runs it.

I tried this to test my theory and also see if I could work around the issue but I get a segfault trying to assign the collection variable on the VM instance:

import unqlite

db = unqlite.UnQLite()
c1 = db.collection('c1')
c1.create()
c1.store([{'a':1}, {'b':2}, {'c':3}, {'d':4}])

vm = db.vm('$ret = db_fetch($collection);')
vm['collection'] = 'c1'  # <-- This line segfaults
vm.execute()
print(vm['ret'])
vm.execute()
print(vm['ret'])

FR: Binary releases on PyPI

Hi. Could you please add binary wheels for pip, for widespread architectures and python versions?

Right now, installing unqlite via pip seems to require a C/C++ compiler, which can be problematic in some situations. I've stumbled upon this while trying to make a CI/CD chain for building a windows distributable of my application using a docker container with wine and pyinstaller. Stuffing MSVC inside is impossible, and stuffing MinGW requires a lot more that simple python knowledge.

Is it possible to use a samba shared database file?

Here is my code:

from unqlite import *

db = UnQLite("/mnt/smb/1.unq")
db['foo'] = 'bar'
db.close()

This will only produce a zero byte file without any error reporting if file is located in a samba shared folder.
Adding db.commit() before db.close() will get following trackback:

Traceback (most recent call last):
  File "/home/koi/tmp/1.py", line 6, in <module>
    db.commit()
  File "unqlite.pyx", line 527, in unqlite.UnQLite.commit
  File "unqlite.pyx", line 531, in unqlite.UnQLite.commit
  File "unqlite.pyx", line 490, in unqlite.UnQLite.check_call
unqlite.UnQLiteError: Another process or thread hold the requested lock
Another process or thread hold the requested lock
Another process or thread hold the requested lock
Cannot obtain an Exclusive lock on the target database

so, any solution? thanks!

collection.store returs int

db = UnQLite()
users = db.collection('users')
users.store([{'a':3}]) # 0

It's not a boolean, as described here.

Corrupt database when reading DB File from multiple processes

I tried to open two python consoles where one is writing, the other is reading. When the second read, exception is raised if changes have been committed but db file has not been reloaded cleanly:

Console 1:

>>> from unqlite import core
>>> db=core.UnQLite("test.db")
>>> db["xre"] = "test"
>>> "xre" in db
True

Console 2:

>>> from unqlite import core
>>> db=core.UnQLite("test.db")
>>> "xre" in db
False

Console 1:

>>> db.commit()
True
>>> "xre" in db
True

Console 2:

>>> print db["xre"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../unqlite/core.py", line 222, in fetch
    handle_return_value(rc)
  File ".../unqlite/core.py", line 24, in handle_return_value
    }.get(rc, 'Unknown exception: %s' % rc))
Exception: Corrupt database
>>> db=core.UnQLite("test.db")
>>> "xre" in db
True

Unqlite and numpy arrays

Is it possible to store numpy arrays (without creating a copy of the data for that) in unqlite?

Segmentation fault

I tried to add a collection of metrics to a collection but got a segmentation fault. I am using unqlite on a test.udb file and writing every result to that file if there is a connection error or exception.

Here is my code:

def AddToLocalDB(list):
    with UnQLite('/home/pi/test.udb') as db:
            collection = db.collection('pending')
            collection.create()
            collection.exists()
            collection.store(list)

I am passing this a list.

Integer keys-value pairs within a collection are being converted to lists

from unqlite import UnQLite
db = UnQLite('./test.udb')
t = db.collection('test')
t.create()
t.store({'count':{-1:100, 0:200, 1:100}}) #stores as 'count':[100,200,100]
t.store({'count':{'-1':100, '0':200, '1':100}}) #still stores as 'count':[100,200,100]

fetch() return a dict with binary values contrary to example

The example given returns binary values (for me):

>>> users.store({'name': 'Leslie', 'color': 'also green'})
0
>>> users.fetch(0)
{'name': b'Leslie', 'color': b'also green', '__id': 0}

collection fetch, anyway it will return a value

unqlite==0.7.0

Python 3.6.5 (default, Jun 17 2018, 12:13:06)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

>>> from unqlite import UnQLite
>>> db = UnQLite()
>>> demo = db.collection('demo')
>>> demo.create()
>>> demo.store({"one":1})
0
>>> demo.fetch(0)
{'one': 1, '__id': 0}
>>> demo.fetch('0')
{'one': 1, '__id': 0}
>>> demo.fetch('0xxx')
{'one': 1, '__id': 0}

Iteration Run Time

@coleifer To elaborate, numpy arrays are stored more efficientley in memory than regular Python tuples and list structures. I am merely suggesting you look into faster data structures and do more research into memory usage. ☺️. numpy can also store arbitrary length byte strings btw. also, my code is a for loop, what else lol

coleifer / unqlite-python Goto Github PK

unqlite-python's People

Contributors

Stargazers

Watchers

Forkers

unqlite-python's Issues

A real numpy.uint8 test case runs like this

The log shows the Main Issue and a potentially related Side Note:

Problem

Question

Python 3.6.3 unqlite 0.6.0

Recommend Projects

Recommend Topics

Recommend Org

A real `numpy.uint8` test case runs like this

Python 3.6.3
unqlite 0.6.0