Developer-friendly Python library to interact with Apache HBase, supports time range scan and multi-versions
License: Other
I am wgzhao, a DevOps engineer working on Securities industry and currently learning Java and Python programming language.
Currently I focus on the development of my fork open source project Addax, which is a general-purpose ETL tools, it supports most of the relational databases on the market and the mainstream of NoSQL databases.
At the same time, I maintain the trino-clickhouse plugin in the Trino project and trino-admin
In [82]: with pool.connection() as con:
...: tbl = con.table('level1_md')
...: %time a=[i for i in tbl.scan(row_start='20200617090000000_sh_00', row_stop='20200617090000000_sh_01', limit=4000, columns = ['cf:symbol'])]
CPU times: user 0 ns, sys: 1 ms, total: 1 ms
Wall time: 7.59 ms
TTransportException Traceback (most recent call last)
in ()
in (.0)
/home/XXXpython3.6/site-packages/easybase/table.py in scan(self, row_start, row_stop, row_prefix, columns, filter, timerange, include_timestamp, batch_size, scan_batching, limit, reversed, max_version)
344 )
345
--> 346 scan_id = self.connection.client.openScanner(self.name.encode(), tscan)
347
348 logger.debug("Opened scanner (id=%d) on '%s'", scan_id, self.name)
/home/XXXpython3.6/site-packages/thriftpy2/thrift.py in _req(self, _api, *args, **kwargs)
217 # wait result only if non-oneway
218 if not getattr(result_cls, "oneway"):
--> 219 return self._recv(_api)
220
221 def _send(self, _api, **kwargs):
/home/XXXpython3.6/site-packages/thriftpy2/thrift.py in _recv(self, _api)
229
230 def _recv(self, _api):
--> 231 fname, mtype, rseqid = self._iprot.read_message_begin()
232 if mtype == TMessageType.EXCEPTION:
233 x = TApplicationException()
/home/XXXpython3.6/site-packages/thriftpy2/protocol/cybin/cybin.pyx in cybin.TCyBinaryProtocol.read_message_begin()
/home/XXXpython3.6/site-packages/thriftpy2/protocol/cybin/cybin.pyx in cybin.read_i32()
/home/XXXpython3.6/site-packages/thriftpy2/transport/buffered/cybuffered.pyx in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.c_read()
/home/XXXpython3.6/site-packages/thriftpy2/transport/buffered/cybuffered.pyx in thriftpy2.transport.buffered.cybuffered.TCyBufferedTransport.read_trans()
/home/XXXpython3.6/site-packages/thriftpy2/transport/cybase.pyx in thriftpy2.transport.cybase.TCyBuffer.read_trans()
/home/XXXpython3.6/site-packages/thriftpy2/transport/socket.py in read(self, sz)
130 if len(buff) == 0:
131 raise TTransportException(type=TTransportException.END_OF_FILE,
--> 132 message='TSocket read 0 bytes')
133 return buff
134
TTransportException: TTransportException(type=4, message='TSocket read 0 bytes')
Hi,
Do you have a plan to support python3?
I am using happybase, but it doesn't have time range scan. (sadbase 😄)
Supporting python3 will help me (and possibly others) a lot.
In [1]: import easybase
File "/usr/local/lib/python2.7/site-packages/easybase/table.py", line 381
def delete(self, row, columns=None, timestamp=None, deletetype=1, attributes=None, durability=False):
^
IndentationError: expected an indented block
LOL
Python 3.7.0 (default, Oct 9 2018, 10:31:47)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
import happybase
import easybase
Traceback (most recent call last):
File "", line 1, in
File "/home/hadoop/miniconda3/envs/recommend/lib/python3.7/site-packages/easybase/init.py", line 17, in
from .connection import DEFAULT_HOST, DEFAULT_PORT, Connection
File "/home/hadoop/miniconda3/envs/recommend/lib/python3.7/site-packages/easybase/connection.py", line 17, in
from Hbase_thrift import TTableName, TTimeRange, TColumnFamilyDescriptor, TTableDescriptor
ImportError: cannot import name 'TTableName' from 'Hbase_thrift' (unknown location)
easybase 1.0.3
thriftpy2 0.4.12
>>> import easybase
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.6/site-packages/easybase/__init__.py", line 12, in <module>
module_name='HBase_thrift'
File "/usr/local/lib64/python3.6/site-packages/thriftpy2/parser/__init__.py", line 33, in load
include_dir=include_dir)
File "/usr/local/lib64/python3.6/site-packages/thriftpy2/parser/parser.py", line 576, in parse
with open(path) as fh:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/lib/python3.6/site-packages/easybase/HBase.thrift'
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.