wgzhao / easybase Goto Github PK

View Code? Open in Web Editor NEW

18.0 3.0 4.0 231 KB

Developer-friendly Python library to interact with Apache HBase, supports time range scan and multi-versions

License: Other

Python 71.09% Thrift 27.63% Makefile 0.35% Shell 0.93%

hbase thriftpy2 easybase timerange happybase python

easybase's Introduction

EasyBase

EasyBase is a developer-friendly Python library to interact with Apache HBase . The original source code forked from HappyBase.

Feature highlight

easy using
support HBase Thrift 2 protocol(HBase Thrift 1 is NO longer supported)
using thriftpy2 instead of old thriftpy

Installation

pip install easybase

Usage

Connect

import easybase
host, port = 'localhost', 9000
tbl = 'test1'
conn = easybase.Connection(host=host, port=port)
table = conn.table(tbl)
rs = conn.scan(limit=10)
for row in rs:
  print(row)

Create Table

table_def = {'cf1':dict(),
             'cf2':{'max_versions':2000}}
conn.create_table('test1', table_def)

Write row to table

puts = {'cf1:c1': 'v1',
        'cf1:c2': 'v2'
       'cf2:c2': 'v3'}
tbl = conn.table('test1')
tbl.put(row='rk1', puts)

Get row from table

rk = 'rk1'
tbl = conn.table('test1')
rs = tbl.row(rk)

Scan rows

tbl = conn.table('test1')
scanner = tbl.scan(row_start='rk_0001', row_stop='rk_0100')
for row in scanner:
  print(row)

Get all namespace

for ns in conn.list_namespaces():
     print(ns)

You can get detail in DemoClient.py

License

MIT License http://www.opensource.org/licenses/MIT.

easybase's People

Contributors

Stargazers

Watchers

Forkers

niteshsinha ao-o woodslee rmekdma

easybase's Issues

连接建立后，查询一次后再次查询会报错

In [82]: with pool.connection() as con:
...: tbl = con.table('level1_md')
...: %time a=[i for i in tbl.scan(row_start='20200617090000000_sh_00', row_stop='20200617090000000_sh_01', limit=4000, columns = ['cf:symbol'])]

CPU times: user 0 ns, sys: 1 ms, total: 1 ms
Wall time: 7.59 ms

In [83]: with pool.connection() as con:
...: tbl = con.table('level1_md')
...: %time a=[i for i in tbl.scan(row_start='20200617090000000_sh_00', row_stop='20200617090000000_sh_01', limit=4000, columns = ['cf:symbol'])]
...:
...:
...:

TTransportException Traceback (most recent call last)
in ()

in (.0)

/home/XXXpython3.6/site-packages/easybase/table.py in scan(self, row_start, row_stop, row_prefix, columns, filter, timerange, include_timestamp, batch_size, scan_batching, limit, reversed, max_version)
344 )
345
--> 346 scan_id = self.connection.client.openScanner(self.name.encode(), tscan)
347
348 logger.debug("Opened scanner (id=%d) on '%s'", scan_id, self.name)

/home/XXXpython3.6/site-packages/thriftpy2/thrift.py in _req(self, _api, *args, **kwargs)
217 # wait result only if non-oneway
218 if not getattr(result_cls, "oneway"):
--> 219 return self._recv(_api)
220
221 def _send(self, _api, **kwargs):

/home/XXXpython3.6/site-packages/thriftpy2/thrift.py in _recv(self, _api)
229
230 def _recv(self, _api):
--> 231 fname, mtype, rseqid = self._iprot.read_message_begin()
232 if mtype == TMessageType.EXCEPTION:
233 x = TApplicationException()

/home/XXXpython3.6/site-packages/thriftpy2/protocol/cybin/cybin.pyx in cybin.TCyBinaryProtocol.read_message_begin()

/home/XXXpython3.6/site-packages/thriftpy2/protocol/cybin/cybin.pyx in cybin.read_i32()

/home/XXXpython3.6/site-packages/thriftpy2/transport/buffered/cybuffered.pyx in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.c_read()

/home/XXXpython3.6/site-packages/thriftpy2/transport/buffered/cybuffered.pyx in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.read_trans()

/home/XXXpython3.6/site-packages/thriftpy2/transport/cybase.pyx in thriftpy2.transport.cybase.TCyBuffer.read_trans()

/home/XXXpython3.6/site-packages/thriftpy2/transport/socket.py in read(self, sz)
130 if len(buff) == 0:
131 raise TTransportException(type=TTransportException.END_OF_FILE,
--> 132 message='TSocket read 0 bytes')
133 return buff
134

TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')

Support python3

Hi,
Do you have a plan to support python3?
I am using happybase, but it doesn't have time range scan. (sadbase 😄)
Supporting python3 will help me (and possibly others) a lot.

easybase==1.0.3无法正常import

easybase 1.0.3
thriftpy2 0.4.12

>>> import easybase
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/easybase/__init__.py", line 12, in <module>
    module_name='HBase_thrift'
  File "/usr/local/lib64/python3.6/site-packages/thriftpy2/parser/__init__.py", line 33, in load
    include_dir=include_dir)
  File "/usr/local/lib64/python3.6/site-packages/thriftpy2/parser/parser.py", line 576, in parse
    with open(path) as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/site-packages/easybase/HBase.thrift'

先导了happybase，再导入easybase会报错

Python 3.7.0 (default, Oct 9 2018, 10:31:47)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.

import happybase
import easybase
Traceback (most recent call last):
File "", line 1, in
File "/home/hadoop/miniconda3/envs/recommend/lib/python3.7/site-packages/easybase/init.py", line 17, in
from .connection import DEFAULT_HOST, DEFAULT_PORT, Connection
File "/home/hadoop/miniconda3/envs/recommend/lib/python3.7/site-packages/easybase/connection.py", line 17, in
from Hbase_thrift import TTableName, TTimeRange, TColumnFamilyDescriptor, TTableDescriptor
ImportError: cannot import name 'TTableName' from 'Hbase_thrift' (unknown location)

import error when i try easybase first time

In [1]: import easybase
File "/usr/local/lib/python2.7/site-packages/easybase/table.py", line 381
def delete(self, row, columns=None, timestamp=None, deletetype=1, attributes=None, durability=False):
^
IndentationError: expected an indented block

LOL