Git Product home page Git Product logo

python-arango's Introduction

Logo

CircleCI CodeQL Docs Coverage Status Last commit

PyPI version badge Python versions badge

License Code style: black Downloads

Python-Arango

Python driver for ArangoDB, a scalable multi-model database natively supporting documents, graphs and search.

Requirements

  • ArangoDB version 3.11+
  • Python version 3.8+

Installation

pip install python-arango --upgrade

Getting Started

Here is a simple usage example:

from arango import ArangoClient

# Initialize the client for ArangoDB.
client = ArangoClient(hosts="http://localhost:8529")

# Connect to "_system" database as root user.
sys_db = client.db("_system", username="root", password="passwd")

# Create a new database named "test".
sys_db.create_database("test")

# Connect to "test" database as root user.
db = client.db("test", username="root", password="passwd")

# Create a new collection named "students".
students = db.create_collection("students")

# Add a persistent index to the collection.
students.add_persistent_index(fields=["name"], unique=True)

# Insert new documents into the collection.
students.insert({"name": "jane", "age": 39})
students.insert({"name": "josh", "age": 18})
students.insert({"name": "judy", "age": 21})

# Execute an AQL query and iterate through the result cursor.
cursor = db.aql.execute("FOR doc IN students RETURN doc")
student_names = [document["name"] for document in cursor]

Another example with graphs:

from arango import ArangoClient

# Initialize the client for ArangoDB.
client = ArangoClient(hosts="http://localhost:8529")

# Connect to "test" database as root user.
db = client.db("test", username="root", password="passwd")

# Create a new graph named "school".
graph = db.create_graph("school")

# Create a new EnterpriseGraph [Enterprise Edition]
eegraph = db.create_graph(
    name="school",
    smart=True)

# Create vertex collections for the graph.
students = graph.create_vertex_collection("students")
lectures = graph.create_vertex_collection("lectures")

# Create an edge definition (relation) for the graph.
edges = graph.create_edge_definition(
    edge_collection="register",
    from_vertex_collections=["students"],
    to_vertex_collections=["lectures"]
)

# Insert vertex documents into "students" (from) vertex collection.
students.insert({"_key": "01", "full_name": "Anna Smith"})
students.insert({"_key": "02", "full_name": "Jake Clark"})
students.insert({"_key": "03", "full_name": "Lisa Jones"})

# Insert vertex documents into "lectures" (to) vertex collection.
lectures.insert({"_key": "MAT101", "title": "Calculus"})
lectures.insert({"_key": "STA101", "title": "Statistics"})
lectures.insert({"_key": "CSC101", "title": "Algorithms"})

# Insert edge documents into "register" edge collection.
edges.insert({"_from": "students/01", "_to": "lectures/MAT101"})
edges.insert({"_from": "students/01", "_to": "lectures/STA101"})
edges.insert({"_from": "students/01", "_to": "lectures/CSC101"})
edges.insert({"_from": "students/02", "_to": "lectures/MAT101"})
edges.insert({"_from": "students/02", "_to": "lectures/STA101"})
edges.insert({"_from": "students/03", "_to": "lectures/CSC101"})

# Traverse the graph in outbound direction, breath-first.
query = """
    FOR v, e, p IN 1..3 OUTBOUND 'students/01' GRAPH 'school'
    OPTIONS { bfs: true, uniqueVertices: 'global' }
    RETURN {vertex: v, edge: e, path: p}
    """
cursor = db.aql.execute(query)

Please see the documentation for more details.

python-arango's People

Contributors

153957 avatar alexbakharew avatar amahanna avatar apetenchea avatar aquamatthias avatar avinash240 avatar carlverge avatar coderjayo avatar cryptoninjageek avatar cw00dw0rd avatar darkheir avatar delirious-lettuce avatar dvzubarev avatar geenen124 avatar geomat0101 avatar gmacon avatar hkernbach avatar imetallica avatar jbylund avatar joerg84 avatar joowani avatar jsteemann avatar maxkernbach avatar mooncake4132 avatar moortiii avatar patricklx avatar philipmay avatar tjoubert avatar valentingregoire avatar wshayes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

python-arango's Issues

Support for Pregel API(s)

Hi

With ArangoDB 3.2, the functionality for using Pregel has been added. I had a discussion with the admin and they have mentioned about an HTTP endpoint or to have a look at js/client/modules/@arangodb/pregel.js but I am not sure how to use them.

Have you already added support for them to your driver. If not, is it possible to add it - if you could shed a bit light on how to use these resources I might be able to send a PR for this integration.

Also, are you there on slack? Tried to ping you there itself but couldn't find you. Thanks.

Question: Would be ORM set in project milestones?

This projects is very interesting and has apparently the best arrange of documentation, frequent updates and code style among other ArangoDB Python drivers.

In order to use this project it would be good having a sense whether is part of the scope adding some ORM classes to it, or if that should be part of another project, having this as a base for a lower level access to ArangoDB.

What are the new improvements?

I just got back to this package and i m loving it. I drifted back to sqlalchemy it is wasting all my lifetime, just to implement common models, but fast with arangodb. I will be glad to contribute as well,

Upsert command ?

Is there anyway to do an upsert without using AQL ?

Something like collection.upsert(data)

database object or not

I may have asked this before but I searched high and low and cannot find the solution anymore. The examples show how to create a database and do things with it.

What is wrong here, I can print the self.db containing the database connection but I cannot use the properties method of it..

import sys
import logging
import configparser
from arango import ArangoClient

logging.basicConfig(
    filename="test.log",
    level=logging.DEBUG,
    format = "%(asctime)s:%(levelname)s:%(message)s"
)

class dataBeest(object):
    def __init__(self):
        logging.debug("Initializing database object.")
        self.initialize_db()

    def read_db_config(self):
        logging.debug("Reading database configuration.")
        try:
            # Return db config from config file
            config = configparser.ConfigParser()
            config.read('arango-client.conf')
            return config['database']
        except IOError as e:
            print('Reading config failed: {}'.format(e))
        except:
            print('Unexpected error: {}'.format(sys.exc_info()[0]))
            raise

    def initialize_db(self):
        logging.debug("Initializing database connection.")
        try:
            databaseConfig = self.read_db_config()
            client = ArangoClient(
                protocol=databaseConfig['protocol'],
                host=databaseConfig['host'],
                port=databaseConfig['port'],
                username=databaseConfig['username'],
                password=databaseConfig['password'],
                enable_logging=databaseConfig['enable_logging']
            )

            self.db = client.database(databaseConfig['database'])

            # print(self.db.properties())

            self.collection = self.db.collection(databaseConfig['collection'])
            logging.debug('Database: {}'.format(self.db))
            logging.debug('Collection: {}'.format(self.collection))

        except:
            print('Unexpected error: {}'.format(sys.exc_info()[0]))
            raise

testen = dataBeest()

simple queries are slower than aql?

Hi,

I have an issue with the simple queries.

artist = db.collection(Artist.collection).document(key)

takes ~550 ms while

artist = list(db.execute_query("for a in prd_artist filter a._key == @val return a", bind_vars={'val': key}))[0]

only takes 110 ms

When I use the HTTP api of ArangoDB directly I see no difference.

Any chance to get this faster?

Thank you very much
Best Regards
Ludwig

Traverse N edges from start vertex

I have posted a question on the 'slack' community

I have a let's say 8 edges and i want to select 3 of them starting from the first vertex.
so based on the first vertex collection, i want to list all documents in the right intervals

Many functionalities missing for vertex/edge collections

There are many functionalities missing for the vertex/edge collections that are present for collections created by normal create_collection method of the database.
Some of them are to allow creating shards, allowing bulk_import.

No recognized create_edge_collection attribute for Graph object

Only a slight variation of the example provided in the documentation at: http://python-driver-for-arangodb.readthedocs.io/en/latest/graph.html#edge-collections

from arango import ArangoClient

client = ArangoClient(host="localhost", port=8529)
db = client.db('_system')
db.create_graph('schedule')
schedule = db.graph('schedule')

# Create some vertex collections
schedule.create_vertex_collection('profs')
schedule.create_vertex_collection('courses')

# Create a new edge definition (and a new edge collection)
schedule.create_edge_collection(
    name='teaches',
    from_collections=['profs'],
    to_collections=['courses']
)

Error returned:

Traceback (most recent call last):
  File "./test.py", line 15, in <module>
    schedule.create_edge_collection(
  File "/usr/lib/python3.5/site-packages/arango/api.py", line 13, in __getattribute__
    method = object.__getattribute__(self, attr)
AttributeError: 'Graph' object has no attribute 'create_edge_collection'

Versions: Python3, python-arango v3.0

cannot import name ArangoClient

Hello, I got an error message:

No handlers could be found for logger "arango.client"
Traceback (most recent call last):
  File "driver1.py", line 1, in <module>
    from arango import ArangoClient
ImportError: cannot import name ArangoClient

when I used from arango import ArangoClient. I know it's extremely strange, but I have totally no idea what happened. Any help will be thankful!

Extreme RAM usage

Loading a large set of data in batches results in 100% RAM utilization. The xmltodict in combination with Arango seems to be an issue. If I remove the Arango part it works, if I remove the xmltodict it also works.

Not sure if this an Arango, Python driver, xmltodict or my own code issue..

def load_datadb():
    sirp = db.arangodb().connectdb()
    batch = sirp.batch(return_result=False)

    mypath = "./data_xml/oct-datadb/november/"
    for root, dirs, filenames in os.walk(mypath):

        count = + 1
        if (count == 5):
            count = 0
            batch.commit()

        for f in filenames:
            print(f)
            file = open(os.path.join(root, f), 'rb')
            xml = xmltodict.parse(file)
            data = {
                "_key": hashlib.sha1(json.dumps(xml).encode()).hexdigest(),
                "id": hashlib.sha1(json.dumps(xml).encode()).hexdigest(),
                "type": "nieuw-data",
                "data": {
                    "meta": {
                        "source-time": str(datetime.datetime.utcnow()),
                        "source": f
                    },
                    "payload": xml
                }
            }

            file.close()
            batch.collection('source').insert(data)
            data = None

AQLQueryExecuteError: [HTTP 401] Unauthorized while running on web but have success on terminal

While using Flash, I tried to run AQL queries on the page, but i ended up with the above error.

I did run the same credential on a created database which works,

I have no clue of where the error is coming from

Here is an example of an error,

Traceback (most recent call last): File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1836, in __call__ return self.wsgi_app(environ, start_response) File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1820, in wsgi_app response = self.make_response(self.handle_exception(e)) File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1403, in handle_exception reraise(exc_type, exc_value, tb) File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/srv/makeups/env/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/home/apps/keyra/core/main/views.py", line 17, in index stocks = db.aql.execute("for a in keystock filter a.type == 'folder' limit 100 return a") File "/srv/makeups/env/lib/python2.7/site-packages/arango/api.py", line 22, in wrapped_method return conn.handle_request(request, handler) File "/srv/makeups/env/lib/python2.7/site-packages/arango/connection.py", line 143, in handle_request return handler(getattr(self, request.method)(**request.kwargs)) File "/srv/makeups/env/lib/python2.7/site-packages/arango/aql.py", line 160, in handler raise AQLQueryExecuteError(res) AQLQueryExecuteError: [HTTP 401] Unauthorized

ArangoDB 3.1 fails with collection.all

When running the simple querry all with the method collection.all using a batch size smaller than the actual collection (i.e. the cursor api has to be used to fetch more results), results in a requests.exceptions.ConnectionError and the ArangoDB 3.1 instance shuts down. With ArangoDB 3.0 there is no issue. Using the HTTP API of 3.1 with curl is also fine.

Here is my environment:

# installed via homebrew on OS X 10.10.5
ยป python --version                                                                                                                                                 
Python 3.5.2

ยป pip freeze
appnope==0.1.0
decorator==4.0.10
ipython==5.1.0
ipython-genutils==0.1.0
pexpect==4.2.1
pickleshare==0.7.4
prompt-toolkit==1.0.9
ptyprocess==0.5.1
Pygments==2.1.3
python-arango==3.4.0
requests==2.12.3
simplegeneric==0.8.1
six==1.10.0
traitlets==4.3.1
wcwidth==0.1.7

Here is a reproducer:
A modified script crash.py from your README.md:

from arango import ArangoClient

# Initialize the client for ArangoDB
client = ArangoClient(enable_logging=False)

# Create a new database named "my_database"
db = client.create_database('my_database')

# Create a new collection named "students"
students = db.create_collection('students')

# Add a hash index to the collection
students.add_hash_index(fields=['name'], unique=True)

# Insert new documents into the collection
students.insert({'name': 'jane', 'age': 19})
students.insert({'name': 'josh', 'age': 18})
students.insert({'name': 'jake', 'age': 21})

result = students.all(batch_size=1)
print([student['name'] for student in result])
# Starting ArangoDB
ยป docker run -p 8529:8529 --name arangodb -it -d -e ARANGO_NO_AUTH=1 arangodb:3.1

# After finished launching, run the script:
ยป python crash.py

Results in

Traceback (most recent call last):
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 391, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 387, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1197, in getresponse
    response.begin()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/adapters.py", line 423, in send
    timeout=timeout
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 643, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/util/retry.py", line 334, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 594, in urlopen
    chunked=chunked)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 391, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/packages/urllib3/connectionpool.py", line 387, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 1197, in getresponse
    response.begin()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 297, in begin
    version, status, reason = self._read_status()
  File "/usr/local/Cellar/python3/3.5.2_3/Frameworks/Python.framework/Versions/3.5/lib/python3.5/http/client.py", line 266, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
requests.packages.urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "crash.py", line 21, in <module>
    print([student['name'] for student in result])
  File "crash.py", line 21, in <listcomp>
    print([student['name'] for student in result])
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/arango/cursor.py", line 32, in __next__
    return self.next()
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/arango/cursor.py", line 120, in next
    res = self._conn.put("/_api/cursor/{}".format(self.id))
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/arango/connection.py", line 211, in put
    auth=(self._username, self._password)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/arango/http_clients/default.py", line 105, in put
    verify=self._check_cert
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/sessions.py", line 546, in put
    return self.request('PUT', url, data=data, **kwargs)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/Users/tammoippen/Desktop/venv/lib/python3.5/site-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))

And on the arango side (it is stopped):

ยป docker logs -f arangodb
Initializing database...Hang on...
Database initialized...Starting System...
2016-12-06T14:59:01Z [1] INFO ArangoDB 3.1.3 [linux] 64bit, using VPack 0.1.30, ICU 54.1, V8 5.0.71.39, OpenSSL 1.0.1t  3 May 2016
2016-12-06T14:59:01Z [1] INFO using SSL options: SSL_OP_CIPHER_SERVER_PREFERENCE, SSL_OP_TLS_ROLLBACK_BUG
2016-12-06T14:59:01Z [1] INFO file-descriptors (nofiles) hard limit is 1048576, soft limit is 1048576
2016-12-06T14:59:01Z [1] INFO JavaScript using startup '/usr/share/arangodb3/js', application '/var/lib/arangodb3-apps'
2016-12-06T14:59:02Z [1] INFO In database '_system': Found 14 defined task(s), 1 task(s) to run
2016-12-06T14:59:02Z [1] INFO In database '_system': state standalone/existing, tasks updateUserModels
2016-12-06T14:59:02Z [1] INFO In database '_system': existing cleanup successfully finished
2016-12-06T14:59:02Z [1] INFO using endpoint 'http+tcp://0.0.0.0:8529' for non-encrypted requests
2016-12-06T14:59:03Z [1] INFO ArangoDB (version 3.1.3 [linux]) is ready for business. Have fun!
2016-12-06T14:59:16Z [1] INFO created application directory '/var/lib/arangodb3-apps/_db/my_database' for database 'my_database'
2016-12-06T14:59:16Z [1] INFO In database 'my_database': No version information file found in database directory.
2016-12-06T14:59:16Z [1] INFO In database 'my_database': Found 14 defined task(s), 9 task(s) to run
2016-12-06T14:59:16Z [1] INFO In database 'my_database': state standalone/init, tasks setupGraphs, addDefaultUserOther, createModules, createRouting, insertRedirectionsAll, setupAqlFunctions, createFrontend, setupQueues, setupJobs
2016-12-06T14:59:17Z [1] INFO In database 'my_database': init successfully finished

This happens also with password for root on arangodb.

Issue witth query

HI @joowani can you please give a hand on the following aql query? i asked the arangodb community for the past many hours and got no reply yet,

based on the previous question i posted, i built a product taxonomy graph, but trying to traverse the edges, brings an error. can you please give a hint?

for t in inbound taxonomy catof
return t

[1577], 'collection 'taxonomy' used as expression operand'
[10], 'Invalid input for traversal: Only id strings or objects with _id are allowed'

any refined collection method?

with an example

articles = db.collection('articles')
i had to iterate through

for arts in articles:
  print arts['field_name']

is there way to get list of articles with custom refining methods such as limit, count? etc?

the all() methods in class definition fetch all, which will affect performance if we had to load all before filtering what we needed instead of filtering upfront and get what we needed.

connect to https server

I have arangodb running on ssl protected server. Admin interface is running ok.

When I try to connect using:

client = ArangoClient(protocol="https", host="www.test.com", port=12345, 
    username="root", password="password", enable_logging=True, verify=True
)

(python 3.5 on windows 7)

I got an SSL error:
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

complete traceback:

Traceback (most recent call last):
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 595, in urlopen
    chunked=chunked)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 352, in _make_request
    self._validate_conn(conn)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 831, in _validate_conn
    conn.connect()
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\packages\urllib3\connection.py", line 289, in connect
    ssl_version=resolved_ssl_version)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\packages\urllib3\util\ssl_.py", line 308, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "C:\Users\crecheverria\Apps\WinPython\64bit-3.5.1.1\python-3.5.1.amd64\lib\ssl.py", line 376, in wrap_socket
    _context=self)
  File "C:\Users\crecheverria\Apps\WinPython\64bit-3.5.1.1\python-3.5.1.amd64\lib\ssl.py", line 747, in __init__
    self.do_handshake()
  File "C:\Users\crecheverria\Apps\WinPython\64bit-3.5.1.1\python-3.5.1.amd64\lib\ssl.py", line 983, in do_handshake
    self._sslobj.do_handshake()
  File "C:\Users\crecheverria\Apps\WinPython\64bit-3.5.1.1\python-3.5.1.amd64\lib\ssl.py", line 628, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\adapters.py", line 423, in send
    timeout=timeout
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 621, in urlopen
    raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "c:\Users\crecheverria\Devel\SGOT\fallas\v_python\arangodb3\test_02.py", line 16, in <module>
    username="root", password="calipso01", enable_logging=True, verify=True
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\arango\client.py", line 65, in __init__
    self.verify()
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\arango\client.py", line 78, in verify
    res = self._conn.head('/_api/version')
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\arango\connection.py", line 162, in head
    auth=(self._username, self._password)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\arango\http_clients\default.py", line 40, in head
    auth=auth
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\sessions.py", line 510, in head
    return self.request('HEAD', url, **kwargs)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\sessions.py", line 596, in send
    r = adapter.send(request, **kwargs)
  File "C:\Users\crecheverria\Devel\z_venvs\py64-3.5.1-gen\lib\site-packages\requests\adapters.py", line 497, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

AQL query: count of document returned

Hi,

when I execute a query as:

cursor = my_db.execute_query(
  "FOR d IN my_col FILTER d.value == 'value' RETURN d"
)

How can I know if the resultset has values or is empty?
If I try to do:

for doc in cursor:
  print doc

I get this error:
'generator' object has no attribute 'current'

Thanks for your help.
M.

Collection.import_bulk onDuplicate option

Hello,

ArangoDB bulk import supports onDuplicate (error, update, replace, ignore) option.
Could you please implement this one?

Implementation Proposal

This gives an forward compatibility and opportunity to pass custom options to endpoint:

  • Add **kw to Collection.import_bulk signature.
  • Overwrite type, collection, complete keys with current Request params
  • Use resulting dict as Request params.

Example

891c891
<     def import_bulk(self, documents, halt_on_error=True, details=True):
---
>     def import_bulk(self, documents, halt_on_error=True, **kwargs):
910a911,914
>         kwargs['type'] = 'array'
>         kwargs['collection'] = self._name
>         kwargs['complete'] = halt_on_error
> 
915,920c919
<             params={
<                 'type': 'array',
<                 'collection': self._name,
<                 'complete': halt_on_error,
<                 'details': details
<             }
---
>             params=kwargs

Support for AND or OR in find()?

More a question than an issue:

is it possible to use the find() API e.g. for searching by

tags=foo OR tags=bar
tags=foo AND tags=bar

e.g. when the document contains tags=["foo", "bar"...]

or is this limited to AQL?

Unintentional loading of all collections

When using command: db.collections['collection_name']
the driver is collecting all properties of every collection. At the moment this is only possible if a collection is loaded. So the side effect is that every unloaded collection will get loaded. If someone has a very large database, this could take a while. I guess this behaviour is not wanted.

Collection.import_bulk : halt_on_error defaults to None, not True

This was a nasty gotcha: in the documentation for import_bulk it says "halt_on_error" defaults to True, in fact it defaults to None meaning that import_bulk by default silently ignores errors.

I believe it would be best if it actually defaults to True, but in any case the documentation should not be contradictory.

Connection lost on heavy load

Hello im playing around with a load testing tool, and trying to crash my application that is using this driver... i test 10k connection attempts with a vareyety of conncurent connections...while handeling 50-100 concurrent connections (based on the arangodb dashboard) without a problem, starting with < 100 concurrent connections is starting to raise "Error 54, in Connection reset by peer exceptions" for round about 20% of my connection attempts comming from the HttpClient of the driver... first i thought its a arangodb problem but based on my research arangodb should be able to handle 10k conccurent connections... (if no keep-alive header is send) is there something like a max concurrent connection thing in the driver and if so how to ajust it?

Error while looping over complete collection

Hi

I need to loop over all the documents of a collection of mine but I am unable to do that as I get errors. Please help me on how I can achieve this.

These are the two scenarios that I have tried -

>>> coll = acl.database('db_name').collection('coll_name')
>>> for i in coll:
...   print i
...   break
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/arango/collections/base.py", line 51, in __iter__
    raise DocumentGetError(res)
arango.exceptions.DocumentGetError: [HTTP 501][ERR 1470] '/_api/export' is not yet supported in a cluster
>>>

and

>>> for i in coll.find({}):
...   print i
...   break
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/arango/api.py", line 22, in wrapped_method
    return conn.handle_request(request, handler)
  File "/usr/local/lib/python2.7/dist-packages/arango/connection.py", line 165, in handle_request
    return handler(getattr(self, request.method)(**request.kwargs))
  File "/usr/local/lib/python2.7/dist-packages/arango/connection.py", line 233, in put
    auth=(self._username, self._password)
  File "/usr/local/lib/python2.7/dist-packages/arango/http_clients/default.py", line 105, in put
    verify=self._check_cert
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 546, in put
    return self.request('PUT', url, data=data, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))
>>>

The second scenario takes some time before throwing the error. The collection has around 26M documents if that matters.

Need more clarifications over traverse function

on arango.graph.Graph,

can you please explain the following and give examples of the following functions:
init_func
sort_func
visitor_func
expander_func
filter_function

i m still having difficulties traversing the graphs, when i added contents to the child graphs, the whole traversal system changed.

Cursor on cluster - ERROR 1600

I use a cursor to loop over all documents in a collection and this works fine for a single server setup but when I use a cluster I get ERROR 1600

1600 - ERROR_CURSOR_NOT_FOUND
Will be raised when a cursor is requested via its id but a cursor with that id cannot be found.

I am not sure how the cursor is handled when it comes to a cluster.

I use it like this:

inergy = db.arangodb().connectdb()

cursor = inergy.aql.execute(
    'FOR doc IN source '
    'FILTER doc.type == @type '
    'RETURN doc',
    bind_vars={'type': "nieuw-inergy"},
    batch_size=1,
    count=True
)

while cursor.has_more():
    doc = cursor.next()

verify gives error

Whenever I try to use .verify() it gives error. But, on the other hand, I can do queries and all other stuff!

ConnectionError: BadStatusLine() while invoking collection find() API

Hi,

I am using python-arango 3.8.0 to connect to ArangoDB 3.1.21 community edition server.

Recently, a ConnectionError was thrown while attempting to do a find() on a collection object (uuid_index) as shown below :

  cursr = self.uuid_index.find({"_key":obj_uuid})
  File "/usr/local/lib/python2.7/site-packages/arango/api.py", line 22, in wrapped_method
    return conn.handle_request(request, handler)
  File "/usr/local/lib/python2.7/site-packages/arango/connection.py", line 165, in handle_request
    return handler(getattr(self, request.method)(**request.kwargs))
  File "/usr/local/lib/python2.7/site-packages/arango/connection.py", line 233, in put
    auth=(self._username, self._password)
  File "/usr/local/lib/python2.7/site-packages/arango/http_clients/default.py", line 105, in put
    verify=self._check_cert
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 528, in put
    return self.request('PUT', url, data=data, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 453, in send
    raise ConnectionError(err, request=request)
ConnectionError: ('Connection aborted.', BadStatusLine("''",))

I did see earlier closed issues where similar error was reported

#32
#29
#30

The requests library used by python-arango defaults to no-retries, but does allow tuning it via the urllib3 Retry object as documented here at

https://github.com/requests/requests/blob/master/requests/adapters.py

class HTTPAdapter(BaseAdapter):
    """The built-in HTTP Adapter for urllib3.
    Provides a general-case interface for Requests sessions to contact HTTP and
    HTTPS urls by implementing the Transport Adapter interface. This class will
    usually be created by the :class:`Session <Session>` class under the
    covers.
    :param pool_connections: The number of urllib3 connection pools to cache.
    :param pool_maxsize: The maximum number of connections to save in the pool.
    :param max_retries: The maximum number of retries each connection
        should attempt. Note, this applies only to failed DNS lookups, socket
        connections and connection timeouts, never to requests where data has
        made it to the server. By default, Requests does not retry failed
        connections. If you need granular control over the conditions under
        which we retry a request, import urllib3's ``Retry`` class and pass
        that instead.
    :param pool_block: Whether the connection pool should block for connections.
    Usage::
      >>> import requests
      >>> s = requests.Session()
      >>> a = requests.adapters.HTTPAdapter(max_retries=3)
      >>> s.mount('http://', a)
    """
    __attrs__ = ['max_retries', 'config', '_pool_connections', '_pool_maxsize',
                 '_pool_block']

    def __init__(self, pool_connections=DEFAULT_POOLSIZE,
                 pool_maxsize=DEFAULT_POOLSIZE, max_retries=DEFAULT_RETRIES,
                 pool_block=DEFAULT_POOLBLOCK):

However, the session adapter creation is not exposed by python-arango.
Can you please support ways to implement something like the below inside python-arango ?

http://www.coglib.com/~icordasc/blog/2014/12/retries-in-requests.html
Code below is reproduced from above page

from requests.packages.urllib3.util import Retry
from requests.adapters import HTTPAdapter
from requests import Session, exceptions

s = Session()
s.mount('https://', HTTPAdapter(
    max_retries=Retry(total=5, status_forcelist=[500])
    )
)

s.get('https://httpbin.org/status/500')

KeyError: 'collections'

Hello,

everytime i try to use collection i get the error "KeyError: 'collections'"

i.e: delete a collection

Traceback (most recent call last):
File "B2ArangoDB.py", line 42, in bastDB.delete_collection("l")
File "C:\Python35\lib\site-packages\python_arango-2.2.0-py3.5.egg\arango\database.py", line 344, in delete_collection
File "C:\Python35\lib\site-packages\python_arango-2.2.0-py3.5.egg\arango\database.py", line 46, in _refresh_collection_cache
File "C:\Python35\lib\site-packages\python_arango-2.2.0-py3.5.egg\arango\database.py", line 234, in collections
KeyError: 'collections'

same error with create_colletion or retrieve collection information..

The actual action as create, delete etc is being executed but with the above error everytime.

Example

You can give a clear example of cache in documents?

๐Ÿ‘

Arango queries execute only once

Hi,

I noticed Arango queries executes only once, is there a way to make the queries persistent or save the queries to be reused?

Local connection is slow if you use localhost intead of 127.0.0.1

Just to comment something I detected using other python driver (pyArango).
On windows if you use host="localhost" instead of host="127.0.0.1", connection time is 1 seconds slower.
I don't know why, but this happends with both drivers on windows python-arango and pyArango.

Can't get document by id on database

The database object is missing a function to query a document by its _id.

Or I was just not able to find it but I scanned the class documentation for a couple of times.

Syntax error while installing python-arango

Python 2.7.10, virtualenv environment:

[ajung@dev1 arango]$ bin/pip install arango
Downloading/unpacking arango
Downloading arango-0.2.1.tar.gz (343kB): 343kB downloaded
Running setup.py (path:/home/ajung/src/arango/build/arango/setup.py) egg_info for package arango

Downloading/unpacking nose (from arango)
Downloading nose-1.3.7-py2-none-any.whl (154kB): 154kB downloaded
Downloading/unpacking mock (from arango)
Downloading mock-1.3.0-py2.py3-none-any.whl (56kB): 56kB downloaded
Downloading/unpacking funcsigs (from mock->arango)
Downloading funcsigs-0.4-py2.py3-none-any.whl
Downloading/unpacking six>=1.7 (from mock->arango)
Downloading six-1.9.0-py2.py3-none-any.whl
Downloading/unpacking pbr>=0.11 (from mock->arango)
Downloading pbr-1.6.0-py2.py3-none-any.whl (87kB): 87kB downloaded
Installing collected packages: arango, nose, mock, funcsigs, six, pbr
Running setup.py install for arango

  File "/home/ajung/src/arango/lib/python2.7/site-packages/arango/async.py", line 12
    return CommandsProxy(obj
                           ^
SyntaxError: invalid syntax

Successfully installed arango nose mock funcsigs six pbr
Cleaning up...

Client to an Arango cluster

Just wondering if there are already considered best practices connecting a client to an Arango cluster either through python-arango or other that python-arango adopts - I can't find any examples and the client API seems to assume you are only connecting to one end point.

When using a cluster is it more expected that the nodes will be behind a common load balancer for example and requests will be distributed around the cluster, or are people using multiple client objects for a cluster, and sharing the requests between them, within python code?

KeyError: u'doCompact' error

Hi, @joowani can you please look into this issue?

whenever i tried to find an edge property, i have the following error popping up.

igraph = mygraph.edge_collection('basicgraph')
igraph.properties()

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/afidegnum/mystore/local/lib/python2.7/site-packages/arango/api.py", line 22, in wrapped_method
    return conn.handle_request(request, handler)
  File "/home/afidegnum/mystore/local/lib/python2.7/site-packages/arango/connection.py", line 164, in handle_request
    return handler(getattr(self, request.method)(**request.kwargs))
  File "/home/afidegnum/mystore/local/lib/python2.7/site-packages/arango/collections/base.py", line 239, in handler
    'compact': res.body['doCompact'],
KeyError: u'doCompact'

Have issue fetching iD from tree parsing.

HI @joowani, i m in a Mess, i have search around but no answer, i m left with no choice to turn to you.

I m importing List of Google taxonomy product csv files, which have variable headings.

from the code below, the csv file fetch each line, break the lines delimited by '>' then check if fetch an id from an existing name or create a new name and fetch its id

from the loop below, the code seems not to detect existing name and keep duplicating names. I have tried the normal python code and even with the aql.

can you please give a hint on what i m doing wrong here? what is the correct code?

from arango import ArangoClient
client = ArangoClient(username='mystore', password='chou1979')



import config
app = Flask(__name__)

db = client.db('mystore')

storegraph = db.graph('storegraph')

@manager.command
def fetch_tree():
    with open('input.csv') as file:
        for row in file.readlines():
            items = row.split(' > ')

            for item in items:
                item = item.rstrip()
                item = item.decode('utf-8')
                storegraph = db.graph('storegraph')
                tax_item = storegraph.vertex_collection('taxonomy')
                tax = tax_item.find({'name': item})
                tx = db.aql.execute('FOR t IN taxonomy FILTER t.name == @value LIMIT 1 RETURN t', bind_vars={'value': item})
                x = [l for l in tx]


                if not x:
                    print item
                    tax_item.insert({'name': item})

                else:
                    mytax = tax_item.insert({'name':item})
                    catof = storegraph.edge_collection('catof')
                    catof.insert({'_from': x[0]['_id'], '_to': mytax['_id']})

AQL Caching returns an error

  File "/srv/makeups/env/local/lib/python2.7/site-packages/arango/api.py", line 22, in wrapped_method
    return conn.handle_request(request, handler)
  File "/srv/makeups/env/local/lib/python2.7/site-packages/arango/connection.py", line 143, in handle_request
    return handler(getattr(self, request.method)(**request.kwargs))
  File "/srv/makeups/env/local/lib/python2.7/site-packages/arango/aql.py", line 304, in handler
    raise AQLCacheConfigureError(res)
arango.exceptions.AQLCacheConfigureError: [HTTP 401] Unauthorized

This is what happens when i tried to configure caching. I have set up an object which supposed to run direct aql query, i first thought its an authentication issue, but when switched to root mode, i still face the same issue.

I am running this configuration on Flask.

KeyError: 'collections'

Hello, I use ArangoDB 3.0.0 [linux] 64bit. And it is first day.

When I tried it, I have a error 'KeyError'.

a = Arango(host="localhost", port=8529, username='root', password='password')
a.create_database("test")
my_db = a.db("test")
my_db.collections
/lib/python2.7/site-packages/arango/database.pyc in collections(self)
232 user_collections = []
233 system_collections = []
--> 234 for collection in res.body["collections"]:
235 if collection["isSystem"]:
236 system_collections.append(collection["name"])

KeyError: 'collections'

Do you have any idea?

Async support

Hi Joowani,

I am currently looking for a tornado arango async adapter and saw that you have async on your feature list. I might fork you repo and write that part myself. Please let me know if you have any thoughts about async support in your code already that I should consider.

So far I only briefly went over your code but it might be enough to write a async client class.
Cheers

ImportError: cannot import name 'ArangoClient'

When i import ArangoClient from arango, It raise

from arango import ArangoClient
Traceback (most recent call last):

  File "<ipython-input-30-fb5a35ea2f64>", line 1, in <module>
    from arango import ArangoClient

ImportError: cannot import name 'ArangoClient'

then dir(arango)

dir(arango)
Out[29]: 
['Connection',
 '__builtins__',
 '__cached__',
 '__doc__',
 '__file__',
 '__loader__',
 '__name__',
 '__package__',
 '__path__',
 '__spec__',
 'aql',
 'c',
 'clients',
 'collection',
 'core',
 'create',
 'cursor',
 'db',
 'document',
 'edge',
 'exceptions',
 'http_clients',
 'index',
 'mixins',
 'response',
 'utils']

[Question] Is it threadsafe?

Hi, thanks for the great library! I'm thinking about using ArangoDB on my next project with Flask, and I'm wondering if I can use it safely with eventlet? Or if I should create a new client everytime I receive a new request?

Thanks in advance,
I.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.