Git Product home page Git Product logo

leveldb-py's Introduction

ctypes bindings for LevelDB

The existing three LevelDB interfaces (py-leveldb, cpy-leveldb, plyvel) use the Python C API and compile Python modules that work with LevelDB's C++ interface. This Python module simply uses the ctypes library to LevelDB's C interface - making it more portable across Python implementations and easier to install and distribute.

leveldb-py:

  • supports get/put/delete (with standard read/write options)
  • supports bloom filters
  • supports leveldb LRU cache
  • allows for manual or automatic database closing (compare with py-leveldb)
  • provides write batches
  • provides iterators (full control of leveldb iteration, with additional idiomatic python iterator support)
  • provides prefix-based iteration (returns iterators that work as if all keys with a shared prefix had the prefix stripped and were dumped into their own database)
  • provides scoped sub-databases (presents a new database wrapper backed by an existing database with all keys prefixed by some prefix)
  • provides range iterators (for idioms like give me all keys between start and end)
  • provides an in-memory db implementation (for faster unit tests)
  • supports snapshots
  • fits in one file
  • requires no compilation

Sample Usage

import leveldb

db = leveldb.DB("/path/to/db", create_if_missing=True)
db.put("key", "value")
print db.get("key")

for key, value in db:
  print key, value

leveldb-py's People

Contributors

jtolio avatar shanemhansen avatar smcquay avatar zeebo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

leveldb-py's Issues

DB() constructor doesn't handle str path on Python 3

The DB() constructor will error if it's passed a str object as the path on 
Python 3.

Seems like you should call path.encode("utf-8") before passing it to the 
underlying library -- this should work on Python 2 and Python 3.


Original issue reported on code.google.com by [email protected] on 16 Jan 2014 at 10:17

Some KeyboardInterrupts are swallowed

# python inserter_maker.py _dummydb _dummy-items < big-list
Loaded 71 new_encoded_urls from db
10000   read at 10170   URLs/sec, 0     inserted/queued, 10000  already in db
20000   read at 10042   URLs/sec, 0     inserted/queued, 10000  already in db
30000   read at 10309   URLs/sec, 0     inserted/queued, 10000  already in db
40000   read at 10070   URLs/sec, 0     inserted/queued, 10000  already in db
^CException KeyboardInterrupt in <bound method _PointerRef.close of 
<leveldb._PointerRef object at 0x7f1651bac190>> ignored
50000   read at 10143   URLs/sec, 0     inserted/queued, 10000  already in db
60000   read at 10260   URLs/sec, 0     inserted/queued, 10000  already in db

# python --version
Python 2.6.6

It was almost certainly running this when I hit Ctrl-C:

    packed_next_item_id = db.get("$next_item_id$", fill_cache=False)
    item_id, = unpack("<i", packed_next_item_id)

    batch = db.newBatch()
    for u in new_encoded_urls:
        db.putTo(batch, reversed_encoded_url(u), packed_next_item_id)
    db.putTo(batch, "$next_item_id$", pack("<i", item_id + 1))
    db.write(batch)

My latest code is available at 
https://github.com/ludios/greader-item-maker/blob/master/inserter_maker.py

Original issue reported on code.google.com by [email protected] on 10 Jun 2013 at 6:11

Please consider a different module name

Three of the four LevelDB bindings I know of use the same module name 
("leveldb"). This causes havoc when I'm trying to test the various bindings, 
even if I try to be careful to not have them installed at the same time.

I know it's not only your fault, but please consider choosing a distinguishable 
module name (e.g. "leveldb_py") so at least you won't be part of the problem.

Original issue reported on code.google.com by [email protected] on 16 Jan 2014 at 10:07

strange behavior when using unicode object as key or prefix


leveldb-py does not complain when a unicode object is used as a key (or when 
creating a scope) but produces some strange invalid behavior : 

* put still insert the value but the unicode key is not respected : the value 
is inserted with only the beginning of the key
* DBInterface created with scope(unicode) do not iterate correctly on the key 
corresponding to the given prefix.

If I use str(key) (where key is an unicode object) everything works as expected.

I think it would be nice to specify this in the documentation and maybe enforce 
it in the code by calling str(key) as it's rather easy to inadvertently use a 
unicode object thinking it's a string (I've been bitten by this while using the 
json module : I did not realize that the dict it was generating contained only 
unicode objects and not strings).


I can provide an example if needed.

Original issue reported on code.google.com by pierre.rust on 1 Aug 2013 at 2:58

Add iteration support to leveldb-py

Branch name:
iteration

Purpose of code changes on this branch:
Add iteration support to leveldb-py

When reviewing my code changes, please focus on:
General awesomeness.

After the review, I'll merge this branch into:
master


Original issue reported on code.google.com by shanemhansen on 11 Jul 2012 at 5:40

Segmentation fault on repeat large batch

What steps will reproduce the problem?

db = leveldb.DB('mydb')
batch = leveldb.WriteBatch()
for x in xrange(10000):
    batch.put(str(x), str(x))
db.write(batch)

What is the expected output? What do you see instead?
When repeating the above snippets in a short time, it results in a segmentation 
fault.

What version of the product are you using? On what operating system?

Working on Linux Mint, using the git repository master branch.

Original issue reported on code.google.com by [email protected] on 5 Oct 2012 at 10:29

Iteration doesn't work on Python 3

The "Iterator" object is not recognized as a valid iterator in Python 3. Trying 
to use it gives this error:

  File "/Users/matt/Dropbox/workspace/env/py33whoosh/lib/python3.3/site-packages/leveldb.py", line 361, in range
    for row in self:
TypeError: iter() returned non-iterator of type 'Iterator'

You can fix this by adding a __next__() method to the object, e.g.:

    def __iter__(self):
        return self

    def __next__(self):
        return self.next()

Original issue reported on code.google.com by [email protected] on 16 Jan 2014 at 10:03

Missing setup.py

leveldb-py has no setup.py, so I cannot install it with distutils (python 
setup.py install --user) or pip (pip install --user .)

Original issue reported on code.google.com by [email protected] on 10 Jun 2013 at 4:09

Can't run tests on Python 2.6

I couldn't run the tests on Python 2.6 (on my CentOS 6.4 machine):

# python --version
Python 2.6.6

# python test_leveldb.py
Traceback (most recent call last):
  File "test_leveldb.py", line 762, in <module>
    main()
  File "test_leveldb.py", line 758, in main
    unittest.main(argv=sys.argv[:1], exit=False)
TypeError: __init__() got an unexpected keyword argument 'exit'

After removing the exit= argument:

# python test_leveldb.py  
...........................E........................E..
======================================================================
ERROR: test__contains__ (__main__.LevelDBTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_leveldb.py", line 342, in test__contains__
    with self.assertRaises(KeyError):
TypeError: failUnlessRaises() takes at least 3 arguments (2 given)

======================================================================
ERROR: test__contains__ (__main__.MemLevelDBTestCases)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test_leveldb.py", line 342, in test__contains__
    with self.assertRaises(KeyError):
TypeError: failUnlessRaises() takes at least 3 arguments (2 given)

----------------------------------------------------------------------
Ran 55 tests in 75.682s

FAILED (errors=2)


Original issue reported on code.google.com by [email protected] on 9 Jun 2013 at 5:49

Being able to disable compression.

I would like to use lvldb instead of plyvel but it lacks the ability to disable compression. This is very important to be able to read the Bitcoin core database without corrupting it.

With plyvel I use:

plyvel.DB(db_dir, create_if_missing=False, compression=None)

pip install on Windows + Python 3.7?

On Windows and Python 3.7,

pip install leveldb-py

gives:

ERROR: Could not find a version that satisfies the requirement leveldb-py (from versions: none)
ERROR: No matching distribution found for leveldb-py

Would it be possible to release a version on PyPI @jtolio ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.