monetdbsolutions / monetdbe-python Goto Github PK
View Code? Open in Web Editor NEWEmbedded MonetDB with a Python frontend and fast Numpy/Pandas support
License: Mozilla Public License 2.0
monetdbe-python's Issues
Can't have multiple simultaneous in-memory MonetDB farms active
We solve this now by switching context, but we run into issues in case of multiple in memory farms, our context switcher will return the same instance.
support uri=True in monetdbe.connect
sqlite supports this:
https://docs.python.org/3/library/sqlite3.html#sqlite3.connect
If uri is true, database is interpreted as a URI. This allows you to specify options. For example, to open a database in read-only mode you can use:
db = sqlite3.connect('file:path/to/database?mode=ro', uri=True)
monetdb_query always sets affected_rows to 0
Compiling monetdb on centos7 fails, missing dl* symbols
[ 94%] Building C object sql/backends/monet5/UDF/udf/CMakeFiles/udf.dir/udf.c.o
[ 94%] Building C object sql/backends/monet5/generator/CMakeFiles/generator.dir/90_generator_hge.sql.c.o
[ 94%] Linking C shared library libembedded.so
[ 94%] Building C object sql/backends/monet5/UDF/udf/CMakeFiles/udf.dir/80_udf.sql.c.o
[ 94%] Building C object sql/backends/monet5/UDF/udf/CMakeFiles/udf.dir/80_udf_hge.sql.c.o
[ 94%] Built target embedded
Scanning dependencies of target example1
[ 94%] Building C object ctest/tools/embedded/CMakeFiles/example1.dir/example1.c.o
Scanning dependencies of target example2
[ 94%] Building C object ctest/tools/embedded/CMakeFiles/example2.dir/example2.c.o
[ 95%] Linking C executable example1
[ 96%] Linking C shared module lib_udf.so
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlsym'
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlerror'
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlopen'
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlclose'
collect2: error: ld returned 1 exit status
make[2]: *** [ctest/tools/embedded/CMakeFiles/example1.dir/build.make:85: ctest/tools/embedded/example1] Error 1
make[1]: *** [CMakeFiles/Makefile2:3409: ctest/tools/embedded/CMakeFiles/example1.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 96%] Linking C shared module lib_generator.so
[ 96%] Linking C executable example2
[ 96%] Built target udf
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlsym'
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlerror'
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlopen'
/usr/bin/ld: /build/MonetDB-default/build/monetdb5/tools/libmonetdb5.so.29: undefined reference to `dlclose'
collect2: error: ld returned 1 exit status
make[2]: *** [ctest/tools/embedded/CMakeFiles/example2.dir/build.make:85: ctest/tools/embedded/example2] Error 1
make[1]: *** [CMakeFiles/Makefile2:3436: ctest/tools/embedded/CMakeFiles/example2.dir/all] Error 2
[ 96%] Built target generator
make: *** [Makefile:163: all] Error 2
Make OS X binary wheel
monetdb_column.name is always empty
This should be set with the column names of the result set
Have a pandas DataFrame result cursor
pass the SQLite DBAPI test suite
18 out of 113 tests are still failing, mostly related to threading.
error: unknown type name ‘monetdb_table’;
$ python ./setup.py bdist_wheel -running bdist_wheel
running build
running build_py
running build_ext
generating cffi module 'build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c'
already up-to-date
building 'monetdbe._lowlevel' extension
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/opt/python/cp36-cp36m/include/python3.6m -c build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c -o build/temp.linux-x86_64-3.6/build/temp.linux-x86_64-3.6/monetdbe._lowlevel.o
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: In function ‘_cffi_d_monetdb_append’:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1055:37: warning: passing argument 4 of ‘monetdb_append’ from incompatible pointer type [-Wincompatible-pointer-types]
return monetdb_append(x0, x1, x2, x3, x4);
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:154:23: note: expected ‘bat *’ {aka ‘int *’} but argument is of type ‘monetdb_column **’ {aka ‘struct <anonymous> **’}
embedded_export char* monetdb_append(monetdb_connection conn, const char* schema, const char* table, bat *batids, size_t column_count);
^~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: In function ‘_cffi_f_monetdb_append’:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1121:41: warning: passing argument 4 of ‘monetdb_append’ from incompatible pointer type [-Wincompatible-pointer-types]
{ result = monetdb_append(x0, x1, x2, x3, x4); }
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:154:23: note: expected ‘bat *’ {aka ‘int *’} but argument is of type ‘monetdb_column **’ {aka ‘struct <anonymous> **’}
embedded_export char* monetdb_append(monetdb_connection conn, const char* schema, const char* table, bat *batids, size_t column_count);
^~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: At top level:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1461:52: error: unknown type name ‘monetdb_table’; did you mean ‘monetdb_types’?
static char * _cffi_d_monetdb_get_table(void * x0, monetdb_table * * x1, char const * x2, char const * x3)
^~~~~~~~~~~~~
monetdb_types
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: In function ‘_cffi_f_monetdb_get_table’:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1470:3: error: unknown type name ‘monetdb_table’; did you mean ‘monetdb_types’?
monetdb_table * * x1;
^~~~~~~~~~~~~
monetdb_types
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1497:39: error: ‘monetdb_table’ undeclared (first use in this function); did you mean ‘monetdb_time’?
x1 = ((size_t)datasize) <= 640 ? (monetdb_table * *)alloca((size_t)datasize) : NULL;
^~~~~~~~~~~~~
monetdb_time
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1497:39: note: each undeclared identifier is reported only once for each function it appears in
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1497:56: error: expected expression before ‘)’ token
x1 = ((size_t)datasize) <= 640 ? (monetdb_table * *)alloca((size_t)datasize) : NULL;
^
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1523:36: warning: passing argument 2 of ‘monetdb_get_table’ from incompatible pointer type [-Wincompatible-pointer-types]
{ result = monetdb_get_table(x0, x1, x2, x3); }
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:156:23: note: expected ‘sql_table **’ {aka ‘struct sql_table **’} but argument is of type ‘int **’
embedded_export char* monetdb_get_table(monetdb_connection conn, sql_table** table, const char* schema_name, const char* table_name);
^~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: In function ‘_cffi_d_monetdb_in_transaction’:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1538:10: warning: implicit declaration of function ‘monetdb_in_transaction’; did you mean ‘_cffi_d_monetdb_in_transaction’? [-Wimplicit-function-declaration]
return monetdb_in_transaction(x0);
^~~~~~~~~~~~~~~~~~~~~~
_cffi_d_monetdb_in_transaction
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: In function ‘_cffi_d_monetdb_result_fetch’:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1688:35: warning: passing argument 2 of ‘monetdb_result_fetch’ from incompatible pointer type [-Wincompatible-pointer-types]
return monetdb_result_fetch(x0, x1, x2, x3);
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:148:23: note: expected ‘monetdb_column **’ {aka ‘struct <anonymous> **’} but argument is of type ‘monetdb_result *’ {aka ‘struct <anonymous> *’}
embedded_export char* monetdb_result_fetch(monetdb_connection conn, monetdb_column** res, monetdb_result* mres, size_t column_index);
^~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1688:39: warning: passing argument 3 of ‘monetdb_result_fetch’ from incompatible pointer type [-Wincompatible-pointer-types]
return monetdb_result_fetch(x0, x1, x2, x3);
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:148:23: note: expected ‘monetdb_result *’ {aka ‘struct <anonymous> *’} but argument is of type ‘monetdb_column **’ {aka ‘struct <anonymous> **’}
embedded_export char* monetdb_result_fetch(monetdb_connection conn, monetdb_column** res, monetdb_result* mres, size_t column_index);
^~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: In function ‘_cffi_f_monetdb_result_fetch’:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1743:39: warning: passing argument 2 of ‘monetdb_result_fetch’ from incompatible pointer type [-Wincompatible-pointer-types]
{ result = monetdb_result_fetch(x0, x1, x2, x3); }
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:148:23: note: expected ‘monetdb_column **’ {aka ‘struct <anonymous> **’} but argument is of type ‘monetdb_result *’ {aka ‘struct <anonymous> *’}
embedded_export char* monetdb_result_fetch(monetdb_connection conn, monetdb_column** res, monetdb_result* mres, size_t column_index);
^~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1743:43: warning: passing argument 3 of ‘monetdb_result_fetch’ from incompatible pointer type [-Wincompatible-pointer-types]
{ result = monetdb_result_fetch(x0, x1, x2, x3); }
^~
In file included from build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:541:
/usr/local/include/monetdb/monetdb_embedded.h:148:23: note: expected ‘monetdb_result *’ {aka ‘struct <anonymous> *’} but argument is of type ‘monetdb_column **’ {aka ‘struct <anonymous> **’}
embedded_export char* monetdb_result_fetch(monetdb_connection conn, monetdb_column** res, monetdb_result* mres, size_t column_index);
^~~~~~~~~~~~~~~~~~~~
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c: At top level:
build/temp.linux-x86_64-3.6/monetdbe._lowlevel.c:1939:108: error: ‘_cffi_d_monetdb_get_table’ undeclared here (not in a function); did you mean ‘_cffi_f_monetdb_get_table’?
{ "monetdb_get_table", (void *)_cffi_f_monetdb_get_table, _CFFI_OP(_CFFI_OP_CPYTHON_BLTN_V, 52), (void *)_cffi_d_monetdb_get_table },
^~~~~~~~~~~~~~~~~~~~~~~~~
_cffi_f_monetdb_get_table
error: command 'gcc' failed with exit status 1
make: *** [Makefile:20: wheels] Error 1
Set up CI to push binary wheels for OSX for every release
properly handle all .fetch*() methods for cursor and connections
A string with zero byte not supported
sqlite3 supports this, probably escapes this.
Transactions tests failing
there are not transaction, commit and rollback methods exposed in monetdb_embedded.h
Have all SQLite regression tests pass
add support for complex and date/time types
currently, we only support string, float and int.
Compilation broken on OSX 10.15
something with md5
use monetdb_get_table and get_column to fetch metadata
The issue is that monetdb_table_t is not properly exposed yet. Once this is fixed we can optimize the fetching singled typed arrays.
Add bool type to monetdb_types
Out of scenario slots when running test suite
halfway running the test suite MonetDB starts to give errors:
#main thread: createExceptionInternal: !ERROR: MALException:sql.start:42000!out of scenario slots
#main thread: createExceptionInternal: !ERROR: SQLException:SQLinitClient:42000!Catalogue not available
#main thread: createExceptionInternal: !ERROR: SQLException:SQLexitClient:42000!Catalogue not available
SQLException:SQLinitClient:42000!Catalogue not available
This is due to not cleaning up properly, could be datasets or open connections.
Backup functionality not supported yet
SQLite supports backups, we don't (yet)
https://docs.python.org/3/library/sqlite3.html#sqlite3.Connection.backup
Have all SQLite type tests pass
Improve ? and : argument handling
I've made a very simple argument parsing implementation, but this doesn't probably support escaping. Should we do this server-side?
Leftover .gdk_lock and uuid
If you abort an open database connection in the Python application (or hangup) then
the database directory may still contain the .gdk_lock and UUID file
They should have been removed automatically with a signal handler
sqlite.intransaction
This property should be handled at the mapiclient library. It is necessary when
the autocommit mode is turned off.
A c-function is needed in the interface. Just assume it for now.
Iteration over a result set causes issues
Another one is taken from the Python SQL documentation. monetdbe 0.6
import monetdbe
conn = monetdbe.connect('example.db')
conn.execute('''CREATE TABLE stocks (date text, trans text, symbol text, qty real, price real)''')
c = conn.cursor()
purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),
('2006-04-06', 'SELL', 'IBM', 500, 53.00),
]
c.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)', purchases)
for row in c.execute('SELECT * FROM stocks ORDER BY price'):
print(row)
gives:
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/cursor.py", line 59, in __iter__
self._set_description(columns)
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/cursor.py", line 53, in _set_description
*list(zip(name, type_code, display_size, internal_size, precision, scale, null_ok)))
TypeError: _make() takes 2 positional arguments but 6 were given
monetdbe.complete_statement missing
From the Python SQLite documentation
3rd example to illustrate a simple frontend.
con = monetdbe.connect(":memory:")
con.isolation_level = None
cur = con.cursor()
buffer = ""
print("Enter your SQL commands to execute in monetdbe.")
print("Enter a blank line to exit.")
while True:
line = input()
if line == "":
break
buffer += line
if monetdbe.complete_statement(buffer):
try:
buffer = buffer.strip()
cur.execute(buffer)
if buffer.lstrip().upper().startswith("SELECT"):
print(cur.fetchall())
except monetdbe.Error as e:
print("An error occurred:", e.args[0])
buffer = ""
con.close()
Enter your SQL commands to execute in monetdbe.
Enter a blank line to exit.
select 1;
Traceback (most recent call last):
File "x.py", line 16, in <module>
if monetdbe.complete_statement(buffer):
AttributeError: module 'monetdbe' has no attribute 'complete_statement'
Fetching empy table: : Assertion `size != 0' failed.
from tempfile import TemporaryDirectory
from monetdbe import connect
cur = connect(TemporaryDirectory().name).cursor()
cur.execute("create table test(i int)")
cur.execute("select i from test")
res = cur.fetchall()
crashes with:
python: /home/gijs/Work/MonetDB/gdk/gdk_utils.c:1669: GDKmalloc_internal: Assertion `size != 0' failed.
on the current default branch
Make windows binary wheel
have all SQLlite factory tests pass
Nested exceptions
The following code:
import monetdbe
conn = monetdbe.connect('example.db')
conn.commit()
triggers a nested Exception handling message:
#main thread: createExceptionInternal: !ERROR: SQLException:sql.trans:2DM30!COMMIT: not allowed in auto commit mode
SQLException:sql.trans:2DM30!COMMIT: not allowed in auto commit mode
Traceback (most recent call last):
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/cursor.py", line 94, in execute
self.result, self.rowcount, self.prepare_id = self.connection.lowlevel.query(formatted, make_result=True)
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/_cffi.py", line 170, in query
check_error(lib.monetdb_query(_connection, query.encode(), p_result, affected_rows, prepare_id))
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/_cffi.py", line 48, in check_error
raise exceptions.DatabaseError(decoded)
monetdbe.exceptions.DatabaseError: SQLException:sql.trans:2DM30!COMMIT: not allowed in auto commit mode
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/connection.py", line 81, in commit
return self.execute("COMMIT")
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/connection.py", line 68, in execute
cur = Cursor(con=self).execute(query, args)
File "/ufs/mk/.virtualenvs/mk-OjYiXP4V/lib/python3.7/site-packages/monetdbe/cursor.py", line 96, in execute
raise OperationalError(e)
monetdbe.exceptions.OperationalError: SQLException:sql.trans:2DM30!COMMIT: not allowed in auto commit mode
Handle NULL values in numpy arrays
This test fixture inserts a NULL value in an int array:
https://github.com/MonetDBSolutions/MonetDBe-Python/blob/master/tests/test_lite/conftest.py#L37
If we retrieve this column as a numpy array the NULL value is garbage.
Crashing on MONETDB url (monetdb://localhost:5000/)
import monetdbe
remote = monetdbe.connect("monetdb://localhost:5000/sf1?user=monetdb&password=monetdb")
Gives:
monetdbe.exceptions.OperationalError: MALException:monetdbe.monetdbe_startup:Cannot add farm /home/gijs/Work/MonetDBe-Python/monetdb:/localhost:5000/sf1?user=monetdb&password=monetdb
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
Be monetdb-lite API compatible
Have the monetdblite test suite running
Unable to load the data
Running the taxi.py example, the system aborts on the copy into. The manual interaction with the server works great.
def loaddb():
clk = time.time()
cursor = conn.cursor()
cursor.execute("""
COPY OFFSET 2 INTO yellow_tripdata_2016_01 FROM '/ufs/mk/repository/monetdbe-examples/data/yellow_tripdata_2016-01.csv' delimiters ',','\n' best effort
""")
cursor.execute("SELECT count(*) FROM yellow_tripdata_2016_01")
print(cursor.fetchone())
cursor.close()
print("Load database %6.3f milliseconds" %(time.time() - clk))
(python) mk@toke::~/repository/monetdbe-examples/src/python> python taxi.py
Create database 0.001 milliseconds
#main thread: GDKfdlocate: !ERROR: no name specified
#main thread: GDKsave: !ERROR: failed name=, ext=tail, mode 0
!FATAL: COMMIT: transaction commit failed (perhaps your disk is full?) exiting (kernel error: (null))
double monetdb_cleanup_result call gives segfault
This accidentally happened in the test suite.
Building module gives compilation warnings
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:13: warning: "HAVE_SYS_TYPES_H" redefined
13 | #define HAVE_SYS_TYPES_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1194: note: this is the location of the previous definition
1194 | #define HAVE_SYS_TYPES_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:31: warning: "HAVE_DLFCN_H" redefined
31 | #define HAVE_DLFCN_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:274: note: this is the location of the previous definition
274 | #define HAVE_DLFCN_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:32: warning: "HAVE_FCNTL_H" redefined
32 | #define HAVE_FCNTL_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:337: note: this is the location of the previous definition
337 | #define HAVE_FCNTL_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:36: warning: "HAVE_LANGINFO_H" redefined
36 | #define HAVE_LANGINFO_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:573: note: this is the location of the previous definition
573 | #define HAVE_LANGINFO_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:38: warning: "HAVE_LIBINTL_H" redefined
38 | #define HAVE_LIBINTL_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:603: note: this is the location of the previous definition
603 | #define HAVE_LIBINTL_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:44: warning: "HAVE_POLL_H" redefined
44 | #define HAVE_POLL_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:750: note: this is the location of the previous definition
750 | #define HAVE_POLL_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:47: warning: "HAVE_STRINGS_H" redefined
47 | #define HAVE_STRINGS_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1039: note: this is the location of the previous definition
1039 | #define HAVE_STRINGS_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:49: warning: "HAVE_SYS_FILE_H" redefined
49 | #define HAVE_SYS_FILE_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1118: note: this is the location of the previous definition
1118 | #define HAVE_SYS_FILE_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:50: warning: "HAVE_SYS_IOCTL_H" redefined
50 | #define HAVE_SYS_IOCTL_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1121: note: this is the location of the previous definition
1121 | #define HAVE_SYS_IOCTL_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:52: warning: "HAVE_SYS_MMAN_H" redefined
52 | #define HAVE_SYS_MMAN_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1139: note: this is the location of the previous definition
1139 | #define HAVE_SYS_MMAN_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:53: warning: "HAVE_SYS_PARAM_H" redefined
53 | #define HAVE_SYS_PARAM_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1149: note: this is the location of the previous definition
1149 | #define HAVE_SYS_PARAM_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:54: warning: "HAVE_SYS_RESOURCE_H" redefined
54 | #define HAVE_SYS_RESOURCE_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1158: note: this is the location of the previous definition
1158 | #define HAVE_SYS_RESOURCE_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:56: warning: "HAVE_SYS_TIMES_H" redefined
56 | #define HAVE_SYS_TIMES_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1188: note: this is the location of the previous definition
1188 | #define HAVE_SYS_TIMES_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:57: warning: "HAVE_SYS_UIO_H" redefined
57 | #define HAVE_SYS_UIO_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1197: note: this is the location of the previous definition
1197 | #define HAVE_SYS_UIO_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:58: warning: "HAVE_SYS_UN_H" redefined
58 | #define HAVE_SYS_UN_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1200: note: this is the location of the previous definition
1200 | #define HAVE_SYS_UN_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:60: warning: "HAVE_SYS_WAIT_H" redefined
60 | #define HAVE_SYS_WAIT_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1206: note: this is the location of the previous definition
1206 | #define HAVE_SYS_WAIT_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:61: warning: "HAVE_TERMIOS_H" redefined
61 | #define HAVE_TERMIOS_H /usr/include/x86_64-linux-gnu
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1221: note: this is the location of the previous definition
1221 | #define HAVE_TERMIOS_H 1
|
In file included from /home/gijs/opt/monetdb/include/monetdb/monetdb_embedded.h:12,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:541:
/home/gijs/opt/monetdb/include/monetdb/monetdb_config.h:63: warning: "HAVE_UUID_UUID_H" redefined
63 | #define HAVE_UUID_UUID_H /usr/include
|
In file included from /usr/include/python3.8/pyconfig.h:3,
from build/temp.linux-x86_64-3.8/_monetdbe_cffi.c:22:
/usr/include/x86_64-linux-gnu/python3.8/pyconfig.h:1300: note: this is the location of the previous definition
1300 | #define HAVE_UUID_UUID_H 1
|
Write documentation
Have all SQLite hooks tests pass
INSERT OR REPLACE crashes monetdb embedded
create table test(id integer auto_increment primary key, name text, income float, unique_test text unique);
INSERT OR REPLACE INTO test(id, unique_test) VALUES (1, 'foo);Support UDFs
sqlite has connection.create_function()
Basis issue with fetchone()
Example taken from the Python SQLite documentation. No matter what, an empty result set should be properly handled
import monetdbe
conn = monetdbe.connect('example.db')
conn.execute('''CREATE TABLE stocks (date text, trans text, symbol text, qty real, price real)''')
c = conn.cursor()
symbol = 'RHAT'
c.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)
t = ('RHAT',)
c.execute('SELECT * FROM stocks WHERE symbol=?', t)
print(c.fetchone())
Fix readme, building with custom monetdb location
The readme is not complete, when talking about building monetdbe and MonetDB is not installed in the default location
Get MonetDBlite test suite working (tests/test_lite/test_pylite00.py)
for MonetDBlite these queries are all executed on the same database:
monetdbe.sql('CREATE TABLE pylite00 (i INTEGER)')
monetdbe.sql('INSERT INTO pylite00 VALUES (1), (2), (3), (4), (5)')
result = monetdbe.sql('SELECT * FROM pylite00')
because how we manage multiple connection with monetdbe, every statement is executed in a new in-memory database.
Have a basic binary wheel that can open a connection
I'm going to make a manylinux2014 binary wheel:
https://github.com/pypa/manylinux
for this I need to make a Docker container that builds MonetDB (mbeded branch) on CentOS 7.
Get multithreading sqlite test suite working (disabled for now)
And the threading test in the test suite fail.
Add support for fast inserting of numpy columns and/or dataframes
for example, monetdblite supports handing over a dict of arrays:
https://github.com/MonetDB/MonetDBLite-Python/blob/master/monetdblite/embeddedmonetdb.py#L184
We now iterate and execute SQL manually over an array, which is unnecessary slow.
Set up CI to push binary wheels for Linux for every release
Pass the SQLite test suite
have a numpy result cursor
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.