fabric8-analytics / fabric8-analytics-data-model Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
Start the import :
http://localhost:32000/api/v1/component-analyses/maven/org.apache.tomcat:tomcat-servlet-api/8.5.6
Data import into graph fails for a package: /maven/org.apache.tomcat:tomcat-servlet-api/8.5.6
data-model-importer_1 | --------------------------------------------------------------------------------
data-model-importer_1 | INFO in rest_api [/src/rest_api.py:94]:
data-model-importer_1 | Ingesting the given list of EPVs - [{"ecosystem": "maven", "version": "8.5.6", "name": "org.apache.tomcat:tomcat-servlet-api"}]
data-model-importer_1 | --------------------------------------------------------------------------------
data-model-importer_1 | ERROR:data_importer:import_epv() failed with error: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist
data-model-importer_1 | ERROR:data_importer:Traceback for latest failure in import call: Traceback (most recent call last):
data-model-importer_1 | File "/src/data_importer.py", line 164, in import_epv_http
data-model-importer_1 | prefix=pkg_key_prefix))
data-model-importer_1 | File "/src/data_source/s3_data_source.py", line 66, in list_files
data-model-importer_1 | for obj in bucket.objects.filter(Prefix=prefix):
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/boto3/resources/collection.py", line 83, in __iter__
data-model-importer_1 | for page in self.pages():
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/boto3/resources/collection.py", line 166, in pages
data-model-importer_1 | for page in pages:
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/botocore/paginate.py", line 255, in __iter__
data-model-importer_1 | response = self._make_request(current_kwargs)
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/botocore/paginate.py", line 332, in _make_request
data-model-importer_1 | return self._method(**current_kwargs)
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/botocore/client.py", line 317, in _api_call
data-model-importer_1 | return self._make_api_call(operation_name, kwargs)
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/botocore/client.py", line 615, in _make_api_call
data-model-importer_1 | raise error_class(parsed_response, operation_name)
data-model-importer_1 | NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist
data-model-importer_1 | [2018-01-31 11:29:48 +0000] [70] [ERROR] Error handling request /api/v1/ingest_to_graph
data-model-importer_1 | Traceback (most recent call last):
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle
data-model-importer_1 | self.handle_request(listener_name, req, client, addr)
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
data-model-importer_1 | super(GeventWorker, self).handle_request(*args)
data-model-importer_1 | File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request
data-model-importer_1 | respiter = self.wsgi(environ, resp.start_response)
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1997, in __call__
data-model-importer_1 | return self.wsgi_app(environ, start_response)
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
data-model-importer_1 | response = self.handle_exception(e)
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
data-model-importer_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
data-model-importer_1 | reraise(exc_type, exc_value, tb)
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
data-model-importer_1 | response = self.full_dispatch_request()
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
data-model-importer_1 | rv = self.handle_user_exception(e)
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
data-model-importer_1 | return cors_after_request(app.make_response(f(*args, **kwargs)))
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
data-model-importer_1 | reraise(exc_type, exc_value, tb)
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
data-model-importer_1 | rv = self.dispatch_request()
data-model-importer_1 | File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
data-model-importer_1 | return self.view_functions[rule.endpoint](**req.view_args)
data-model-importer_1 | File "/src/rest_api.py", line 102, in ingest_to_graph
data-model-importer_1 | report = data_importer.import_epv_from_s3_http(list_epv=input_json)
data-model-importer_1 | File "/src/data_importer.py", line 214, in import_epv_from_s3_http
data-model-importer_1 | list_epv, select_doc)
data-model-importer_1 | File "/src/data_importer.py", line 199, in import_epv_http
data-model-importer_1 | raise RuntimeError(msg)
data-model-importer_1 | RuntimeError: import_epv() failed with error: An error occurred (NoSuchBucket) when calling the ListObjects operation: The specified bucket does not exist
data-model-importer_1 | --------------------------------------------------------------------------------
data-model-importer_1 | INFO in rest_api [/src/rest_api.py:94]:
data-model-importer_1 | Ingesting the given list of EPVs - [{"ecosystem": "maven", "version": null, "name": "org.apache.tomcat:tomcat-servlet-api"}]
Originally found here: openshiftio/openshift.io#2048 (comment)
Some intermittent failures encountered in Graph layer. This seems to be happening very sparsely:
g.V().has('ecosystem','maven').has('name','org.apache.tomcat:tomcat-servlet-api').properties('tokens','libio_usedby').drop().iterate();pkg = g.V().has('ecosystem','maven').has('name', 'org.apache.tomcat:tomcat-servlet-api').tryNext().orElseGet{graph.addVertex('ecosystem', 'maven', 'name', 'org.apache.tomcat:tomcat-servlet-api', 'vertex_label', 'Package')};pkg.property('last_updated', 1517286212.43);pkg.property('tokens', 'org'); pkg.property('tokens', 'apache'); pkg.property('tokens', 'tomcat'); pkg.property('tokens', 'tomcat'); pkg.property('tokens', 'servlet'); pkg.property('tokens', 'api');pkg.property('latest_version', '9.0.0.M17');pkg.property('libio_latest_release', '1500854400.0');pkg.property('libio_usedby', 'keycloak/keycloak:1359');pkg.property('libio_usedby', 'SungardAS/enhanced-snapshots:35');pkg.property('libio_usedby', 'cf-unik/unik:1239');pkg.property('libio_usedby', 'entando/entando-components:15');pkg.property('libio_usedby', 'indeedeng/proctor:198');pkg.property('libio_usedby', 'magro/memcached-session-manager:552');pkg.property('libio_usedby', 'nysenate/OpenLegislation:163');pkg.property('libio_usedby', 'Red5/red5-server:1254');pkg.property('libio_usedby', 'aspose-words/Aspose.Words-for-Java:52');pkg.property('libio_usedby', 'google/identity-toolkit-java-client:32');pkg.property('libio_dependents_projects', '65');pkg.property('libio_dependents_repos', '2.05K');pkg.property('libio_total_releases', '124');pkg.property('libio_latest_version', '8.5.19');g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.19').property('gh_release_date', 1500854400.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M25').property('gh_release_date',1500854400.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M19').property('gh_release_date',1490572800.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.
5.16').property('gh_release_date',1498003200.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.14').property('gh_release_date',1492041600.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.15').property('gh_release_date',1493942400.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M21').property('gh_release_date',1493856000.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M20').property('gh_release_date',1491955200.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M22').property('gh_release_date',1498003200.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.13').property('gh_release_date',1490572800.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M1').properties('licenses','cve_ids','declared_licenses').drop().iterate();ver = g.V().has('pecosystem', 'maven').has('pname', 'org.apache.tomcat:tomcat-servlet-api').has('version', '9.0.0.M1').tryNext().orElseGet{graph.addVertex('pecosystem','maven', 'pname','org.apache.tomcat:tomcat-servlet-api', 'version', '9.0.0.M1', 'vertex_label', 'Version')};ver.property('last_updated',1517286212.43);ver.property('description','javax.servlet package');ver.property('cm_num_files',112);ver.property('cm_avg_cyclomatic_complexity', 1.23);ver.property('cm_loc',42622);ver.property('licenses', 'ASL 2.0'); ver.property('licenses', 'CDDL');ver.property('cve_ids', 'CVE-2017-6056:5.0'); ver.property('cve_ids', 'CVE-2016-8735:7.5'); ver.property('cve_ids', 'CVE-2016-6816:6.8'); ver.property('cve_ids', 'CVE-2016-6325:7.2'); ver.property('cve_ids', 'CVE-2016-5425:7.2'); ver.property('cve_ids', 'CVE-2016-3092:7.8'); ver.property('cve_ids', 'CVE-2016-0763:6.5'); ver.property('cve_ids', 'CVE-2016-0714:6.
5'); ver.property('cve_ids', 'CVE-2016-0706:4.0'); ver.property('cve_ids', 'CVE-2015-5351:6.8'); ver.property('cve_ids', 'CVE-2015-5346:6.8'); ver.property('cve_ids', 'CVE-2015-5345:5.0');ver.property('declared_licenses', 'Apache License'); ver.property('declared_licenses', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0');lic = g.V().has('lname', 'Apache License').tryNext().orElseGet{graph.addVertex('vertex_label', 'License', 'lname', 'Apache License', 'last_updated',1517286212.43)}; g.V(ver).out('has_declared_license').has('lname', 'Apache License').tryNext().orElseGet{ver.addEdge('has_declared_license', lic)};lic = g.V().has('lname', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0').tryNext().orElseGet{graph.addVertex('vertex_label', 'License', 'lname', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0', 'last_updated',1517286212.43)}; g.V(ver).out('has_declared_license').has('lname', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0').tryNext().orElseGet{ver.addEdge('has_declared_license', lic)};edge_c = g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M1').in('has_version').tryNext().orElseGet{pkg.addEdge('has_version', ver)};
ERROR:data_importer:The import failed: 'status'
ERROR:data_importer:Traceback for latest failure in import call: Traceback (most recent call last):
File "/src/data_importer.py", line 101, in _import_keys_from_s3_http
if resp['status']['code'] == 200:
KeyError: 'status'
g.V().has('ecosystem','maven').has('name','org.apache.tomcat:tomcat-servlet-api').properties('tokens','libio_usedby').drop().iterate();pkg = g.V().has('ecosystem','maven').has('name', 'org.apache.tomcat:tomcat-servlet-api').tryNext().orElseGet{graph.addVertex('ecosystem', 'maven', 'name', 'org.apache.tomcat:tomcat-servlet-api', 'vertex_label', 'Package')};pkg.property('last_updated', 1517286213.63);pkg.property('tokens', 'org'); pkg.property('tokens', 'apache'); pkg.property('tokens', 'tomcat'); pkg.property('tokens', 'tomcat'); pkg.property('tokens', 'servlet'); pkg.property('tokens', 'api');pkg.property('latest_version', '9.0.0.M17');pkg.property('libio_latest_release', '1500854400.0');pkg.property('libio_usedby', 'keycloak/keycloak:1359');pkg.property('libio_usedby', 'SungardAS/enhanced-snapshots:35');pkg.property('libio_usedby', 'cf-unik/unik:1239');pkg.property('libio_usedby', 'entando/entando-components:15');pkg.property('libio_usedby', 'indeedeng/proctor:198');pkg.property('libio_usedby', 'magro/memcached-session-manager:552');pkg.property('libio_usedby', 'nysenate/OpenLegislation:163');pkg.property('libio_usedby', 'Red5/red5-server:1254');pkg.property('libio_usedby', 'aspose-words/Aspose.Words-for-Java:52');pkg.property('libio_usedby', 'google/identity-toolkit-java-client:32');pkg.property('libio_dependents_projects', '65');pkg.property('libio_dependents_repos', '2.05K');pkg.property('libio_total_releases', '124');pkg.property('libio_latest_version', '8.5.19');g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.19').property('gh_release_date', 1500854400.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M25').property('gh_release_date',1500854400.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M19').property('gh_release_date',1490572800.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.
5.16').property('gh_release_date',1498003200.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.14').property('gh_release_date',1492041600.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.15').property('gh_release_date',1493942400.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M21').property('gh_release_date',1493856000.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M20').property('gh_release_date',1491955200.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M22').property('gh_release_date',1498003200.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.13').property('gh_release_date',1490572800.0);g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M11').properties('licenses','cve_ids','declared_licenses').drop().iterate();ver = g.V().has('pecosystem', 'maven').has('pname', 'org.apache.tomcat:tomcat-servlet-api').has('version', '9.0.0.M11').tryNext().orElseGet{graph.addVertex('pecosystem','maven', 'pname','org.apache.tomcat:tomcat-servlet-api', 'version', '9.0.0.M11', 'vertex_label', 'Version')};ver.property('last_updated',1517286213.63);ver.property('description','javax.servlet package');ver.property('cm_num_files',114);ver.property('cm_avg_cyclomatic_complexity', 1.23);ver.property('cm_loc',42800);ver.property('licenses', 'ASL 2.0'); ver.property('licenses', 'CDDL');ver.property('cve_ids', 'CVE-2017-6056:5.0'); ver.property('cve_ids', 'CVE-2016-8747:5.0'); ver.property('cve_ids', 'CVE-2016-8735:7.5'); ver.property('cve_ids', 'CVE-2016-6816:6.8'); ver.property('cve_ids', 'CVE-2016-6325:7.2'); ver.property('cve_ids', 'CVE-2016-5425:7.2');ver.property('declared_licenses', 'Apache License'); ver.property('declared_licenses'
, ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0');lic = g.V().has('lname', 'Apache License').tryNext().orElseGet{graph.addVertex('vertex_label', 'License', 'lname', 'Apache License', 'last_updated',1517286213.63)}; g.V(ver).out('has_declared_license').has('lname', 'Apache License').tryNext().orElseGet{ver.addEdge('has_declared_license', lic)};lic = g.V().has('lname', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0').tryNext().orElseGet{graph.addVertex('vertex_label', 'License', 'lname', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0', 'last_updated',1517286213.63)}; g.V(ver).out('has_declared_license').has('lname', ' Version 2.0 and
Common Development And Distribution License (CDDL) Version 1.0').tryNext().orElseGet{ver.addEdge('has_declared_license', lic)};edge_c = g.V().has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','9.0.0.M11').in('has_version').tryNext().orElseGet{pkg.addEdge('has_version', ver)};
ERROR:data_importer:The import failed: HTTPConnectionPool(host='172.30.80.86', port=8182): Read timed out. (read timeout=30)
ERROR:data_importer:Traceback for latest failure in import call: Traceback (most recent call last):
File "/src/data_importer.py", line 98, in _import_keys_from_s3_http
data=json.dumps(payload), timeout=30)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 112, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 521, in send
raise ReadTimeout(e, request=request)
ReadTimeout: HTTPConnectionPool(host='172.30.80.86', port=8182): Read timed out. (read timeout=30)
Figured out the cause of above error from gremlin server logs:
39989555 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
Script3023.groovy: 1: expecting ''', found '\n' @ line 1, column 4116.
d_licenses', ' Version 2.0 and
^
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script3023.groovy: 1: expecting ''', found '\n' @ line 1, column 4116.
d_licenses', ' Version 2.0 and
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:140)
at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Essentially the queries are not formed should factor in special characters like newlines correctly in existing code here:
drop_props.append('declared_licenses')
prp_version += " ".join(["ver.property('declared_licenses', '{}');".format
(dl) for dl in declared_licenses])
# Create License Node and edge from EPV
for lic in declared_licenses:
prp_version += "lic = g.V().has('lname', '{lic}').tryNext().orElseGet{{" \
"graph.addVertex('vertex_label', 'License', 'lname', '{lic}', " \
"'last_updated',{last_updated})}}; g.V(ver).out(" \
"'has_declared_license').has('lname', '{lic}').tryNext()." \
"orElseGet{{ver.addEdge('has_declared_license', lic)}};".format(
lic=lic, last_updated=str(time.time())
)
Happens for this package
has('pecosystem','maven').has('pname','org.apache.tomcat:tomcat-servlet-api').has('version','8.5.14')
metadata.json
for the above EPV contains
"declared_license": "Apache License, Version 2.0 and\n Common Development And Distribution License (CDDL) Version 1.0",
Notice the newline.
This is an error from logs of instance running in production:
22086511 [gremlin-server-exec-5] WARN org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 - Response [ResponseMessage{requestId=300a8de6-d7ea-4aa5-8216-826bb3a5b57d, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
--
| 22086511 [gremlin-server-exec-5] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Error during serialization for ResponseMessage{requestId=300a8de6-d7ea-4aa5-8216-826bb3a5b57d, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}
| org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
| at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:101)
| at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
| at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
| Caused by: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
| at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:339)
| at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:311)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:368)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:123)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
| at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
| ... 7 more
| Caused by: java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
| at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
| at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
| at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
| ... 17 more
| 22086513 [gremlin-server-exec-5] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Vertex with id 24985604280 was removed.
| java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
| at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
| at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
| at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
| at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
| at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
| at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
| 22087866 [gremlin-server-exec-3] WARN org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 - Response [ResponseMessage{requestId=a419bf20-2d55-4776-ba2c-f70ad93aeecb, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
| 22087866 [gremlin-server-exec-3] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Error during serialization for ResponseMessage{requestId=a419bf20-2d55-4776-ba2c-f70ad93aeecb, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}
| org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
| at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:101)
| at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
| at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
| Caused by: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
| at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:339)
| at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:311)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:368)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:123)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
| at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
| ... 7 more
| Caused by: java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
| at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
| at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
| at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
| ... 17 more
| 22087867 [gremlin-server-exec-3] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Vertex with id 24985604280 was removed.
| java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
| at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
| at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
| at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
| at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
| at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
| at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
| at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
| at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
| at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
| at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
| at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
| 22089179 [gremlin-server-exec-6] WARN org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 - Response [ResponseMessage{requestId=540ad248-41f5-4921-a45c-851965343c99, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
This code branch in function get_qstring_for_cve_node
needs to be covered by unit test
fabric8-analytics-data-model/src/cve.py
Line 95 in 7d93d83
As encountered in logs from production instance
44242668 [gremlin-server-exec-5] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
| Script4829.groovy: 1: expecting ')', found 'Reilly' @ line 1, column 10197.
| ; ver.property('licenses', 'O'Reilly Cod
| ^
|
| 1 error
|
| org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
| Script4829.groovy: 1: expecting ')', found 'Reilly' @ line 1, column 10197.
| ; ver.property('licenses', 'O'Reilly Cod
| ^
Not sure what is causing the following error in the logs, so adding it for tracking:
967823 [gremlin-server-exec-5] WARN org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 - Response [ResponseMessage{requestId=d67ac739-6ef4-4685-9cf5-5a9fa29762dd, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[33587556368]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
967824 [gremlin-server-exec-5] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Error during serialization for ResponseMessage{requestId=d67ac739-6ef4-4685-9cf5-5a9fa29762dd, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[33587556368]], meta={}}}
org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 33587556368 was removed. (through reference chain: java.util.ArrayList[0])
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:101)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 33587556368 was removed. (through reference chain: java.util.ArrayList[0])
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:339)
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:311)
at org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:368)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:123)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
... 7 more
Caused by: java.lang.IllegalStateException: Vertex with id 33587556368 was removed.
at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
... 17 more
967832 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Vertex with id 33587556368 was removed.
java.lang.IllegalStateException: Vertex with id 33587556368 was removed.
at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8592251 [gremlin-server-exec-8] WARN org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 - Response [ResponseMessage{requestId=214fb8d8-1ede-423f-ae88-8c66cd5a5a45, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
8592251 [gremlin-server-exec-8] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Error during serialization for ResponseMessage{requestId=214fb8d8-1ede-423f-ae88-8c66cd5a5a45, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}
org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:101)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:339)
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:311)
at org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:368)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:123)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
... 7 more
Caused by: java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
... 17 more
8592253 [gremlin-server-exec-8] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Vertex with id 24985604280 was removed.
java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
8593721 [gremlin-server-exec-2] WARN org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0 - Response [ResponseMessage{requestId=31f0bbca-2b40-4001-90a3-2a5df894b4b9, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}] could not be serialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0.
8593721 [gremlin-server-exec-2] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Error during serialization for ResponseMessage{requestId=31f0bbca-2b40-4001-90a3-2a5df894b4b9, status=ResponseStatus{code=SUCCESS, message='', attributes={}}, result=ResponseResult{data=[v[24985604280]], meta={}}}
org.apache.tinkerpop.gremlin.driver.ser.SerializationException: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:101)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException: Vertex with id 24985604280 was removed. (through reference chain: java.util.ArrayList[0])
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:339)
at org.apache.tinkerpop.shaded.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:311)
at org.apache.tinkerpop.shaded.jackson.databind.ser.std.StdSerializer.wrapAndThrow(StdSerializer.java:368)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:123)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
... 7 more
Caused by: java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
... 17 more
8593723 [gremlin-server-exec-2] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Vertex with id 24985604280 was removed.
java.lang.IllegalStateException: Vertex with id 24985604280 was removed.
at org.apache.tinkerpop.gremlin.structure.Element$Exceptions.elementAlreadyRemoved(Element.java:154)
at com.thinkaurelius.titan.core.InvalidElementException.removedException(InvalidElementException.java:44)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.it(AbstractVertex.java:40)
at com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.query(StandardTitanTx.java:1004)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.getVertexLabelInternal(AbstractVertex.java:108)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.vertexLabel(AbstractVertex.java:113)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.label(AbstractVertex.java:104)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.ser(GraphSONSerializersV1d0.java:198)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:182)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONSerializersV1d0$VertexJacksonSerializer.serialize(GraphSONSerializersV1d0.java:170)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serializeContents(IndexedListSerializer.java:119)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:79)
at org.apache.tinkerpop.shaded.jackson.databind.ser.impl.IndexedListSerializer.serialize(IndexedListSerializer.java:18)
at org.apache.tinkerpop.gremlin.structure.io.graphson.GraphSONUtil.writeWithType(GraphSONUtil.java:55)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.ser(AbstractGraphSONMessageSerializerV1d0.java:244)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:214)
at org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV1d0$ResponseMessageSerializer.serialize(AbstractGraphSONMessageSerializerV1d0.java:206)
at org.apache.tinkerpop.shaded.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:130)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper._configAndWriteValue(ObjectMapper.java:3613)
at org.apache.tinkerpop.shaded.jackson.databind.ObjectMapper.writeValueAsString(ObjectMapper.java:2980)
at org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV1d0.serializeResponseAsString(GraphSONMessageSerializerV1d0.java:98)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:248)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
From the following logs it appears that the query is failing due to single quote present in the input data:
8629825 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
Script936.groovy: 1: expecting ')', found 'The' @ line 1, column 1146.
operty('declared_licenses', ''The Apache
^
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script936.groovy: 1: expecting ')', found 'The' @ line 1, column 1146.
operty('declared_licenses', ''The Apache
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:145)
at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Encountered this error in the logs
5340448 [gremlin-server-exec-2] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
Script782.groovy: 1: unexpected char: '\' @ line 1, column 7263.
mber value using Number.equals\ boolean
^
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script782.groovy: 1: unexpected char: '\' @ line 1, column 7263.
mber value using Number.equals\ boolean
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:140)
at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
5341861 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
Script783.groovy: 1: unexpected char: '\' @ line 1, column 7284.
mber value using Number.equals\ boolean
It is likely that some string parameter is not being escaped properly.
We use Codecov.io to track code coverage changes. It would be good to document the usage of Codecov.io in the README.md as well.
INFO in rest_api [/src/rest_api.py:128]:
--
| Selective Ingestion with payload - {"select_ingest": ["keywords_tagging", "git_stats", "github_details", "libraries_io"], "package_list": [{"ecosystem": "maven", "version": null, "name": "com.yugabyte:cassandra-driver-extras"}]}
| --------------------------------------------------------------------------------
| ERROR:data_importer:The import failed: list index out of range
| ERROR:data_importer:Traceback for latest failure in import call: Traceback (most recent call last):
| File "/src/data_importer.py", line 87, in _import_keys_from_s3_http
| str_gremlin = GraphPopulator.create_query_string(obj)
| File "/src/graph_populator.py", line 361, in create_query_string
| str_package, prp_package = cls.construct_package_query(input_json)
| File "/src/graph_populator.py", line 272, in construct_package_query
| libio_latest = releases.get('recent', [{}])[-1] # last is latest
| IndexError: list index out of range
|
| ERROR:data_importer:
| Report from import_epv():
| The import failed: list index out of range
| Total number of EPVs imported: 0
| The last successfully imported EPV: None
|
| --------------------------------------------------------------------------------
| INFO in rest_api [/src/rest_api.py:136]:
| {'epv': {u'select_ingest': [u'keywords_tagging', u'git_stats', u'github_details', u'libraries_io'], u'package_list': [{u'ecosystem': u'maven', u'version': '', u'name': u'com.yugabyte:cassandra-driver-extras'}]}, 'message': 'The import failed: list index out of range', 'count_imported_EPVs': 0}
| --------------------------------------------------------------------------------
This is encountered in Production instance of data-model importer:
/usr/lib64/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>;.
--
| """)
18093838 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
--
| Script3654.groovy: 1: unexpected char: '\' @ line 1, column 589.
| rces using WebLab properties./\ At the t
| ^
|
| 1 error
|
| org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
| Script3654.groovy: 1: unexpected char: '\' @ line 1, column 589.
| rces using WebLab properties./\ At the t
| ^
|
| 1 error
|
| at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
| at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
| at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
| at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
| at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
| at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:140)
| at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
| at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
| at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
| at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
| at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
| at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
| at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
| at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
| at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
| at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
Seems like it is happening for description field: http://central.maven.org/maven2/org/ow2/weblab/core/annotator/1.2.4.1/annotator-1.2.4.1.pom
The method CVEPut.create_pv_nodes
from the script cve.py
needs to be covered by unit tests
fabric8-analytics-data-model/src/cve.py
Line 58 in 7d93d83
There are many logger.<level>()
other than from the REST API ( Flask ), which do not appear in the logs.
Over 100 instances of Timing out messages in logs are encountered as shown below
Timing out script - g.V().has('ecosystem',ecosystem).has('name',within(input_packages)).as('pkg').out('has_version').hasNot('cve_ids').as('ver').select('pkg','ver').by(valueMap()).dedup() - in thread [gremlin-server-worker-1]
45814468 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor - Timing out script - g.V().has('ecosystem',ecosystem).has('name',within(input_packages)).as('pkg').out('has_version').hasNot('cve_ids').as('ver').select('pkg','ver').by(valueMap()).dedup() - in thread [gremlin-server-worker-1]
45815205 [gremlin-server-exec-4] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Error encountered evaluating script: g.V().has('ecosystem',ecosystem).has('name',within(input_packages)).as('pkg').out('has_version').hasNot('cve_ids').as('ver').select('pkg','ver').by(valueMap()).dedup()
org.apache.tinkerpop.gremlin.process.traversal.util.TraversalInterruptedException
at com.thinkaurelius.titan.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:434)
at com.thinkaurelius.titan.diskstorage.BackendTransaction.edgeStoreQuery(BackendTransaction.java:254)
at com.thinkaurelius.titan.graphdb.database.StandardTitanGraph.edgeQuery(StandardTitanGraph.java:413)
at com.thinkaurelius.titan.graphdb.query.vertex.SimpleVertexQueryProcessor$2.lambda$get$0(SimpleVertexQueryProcessor.java:114)
at com.thinkaurelius.titan.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:84)
at com.thinkaurelius.titan.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:76)
at com.thinkaurelius.titan.graphdb.query.profile.QueryProfiler.profile(QueryProfiler.java:68)
at com.thinkaurelius.titan.graphdb.query.vertex.SimpleVertexQueryProcessor$2.get(SimpleVertexQueryProcessor.java:114)
at com.thinkaurelius.titan.graphdb.query.vertex.SimpleVertexQueryProcessor$2.get(SimpleVertexQueryProcessor.java:111)
at com.thinkaurelius.titan.graphdb.vertices.CacheVertex.loadRelations(CacheVertex.java:53)
at com.thinkaurelius.titan.graphdb.query.vertex.SimpleVertexQueryProcessor.getBasicIterator(SimpleVertexQueryProcessor.java:111)
at com.thinkaurelius.titan.graphdb.query.vertex.SimpleVertexQueryProcessor.iterator(SimpleVertexQueryProcessor.java:68)
at com.thinkaurelius.titan.graphdb.transaction.RelationConstructor$1$1.<init>(RelationConstructor.java:36)
at com.thinkaurelius.titan.graphdb.transaction.RelationConstructor$1.iterator(RelationConstructor.java:34)
at com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.properties(AbstractVertex.java:160)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertyMapStep.map(PropertyMapStep.java:67)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.PropertyMapStep.map(PropertyMapStep.java:46)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:37)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.next(DefaultTraversal.java:186)
at org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil.apply(TraversalUtil.java:93)
at org.apache.tinkerpop.gremlin.process.traversal.util.TraversalUtil.applyNullable(TraversalUtil.java:119)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectStep.map(SelectStep.java:69)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectStep.map(SelectStep.java:46)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:37)
at org.apache.tinkerpop.gremlin.process.traversal.step.map.SelectStep.processNextStart(SelectStep.java:158)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37)
at org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep.processNextStart(DedupGlobalStep.java:92)
at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:179)
at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.fill(IteratorUtils.java:62)
at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.list(IteratorUtils.java:85)
at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.asList(IteratorUtils.java:382)
at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:239)
at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36)
at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Some of the queries identified as timing out are:
g.V().has('ecosystem',ecosystem).has('name',within(input_packages)).as('pkg').out('has_version').hasNot('cve_ids').as('ver').select('pkg','ver').by(valueMap()).dedup() - in thread [gremlin-server-worker-1]
g.V().has('pecosystem', 'maven').has('pname', 'io.vertx:vertx-core').has('version', '3.4.1').as('version').in('has_version').as('package').select('version','package').by(valueMap()); - in thread [gremlin-server-worker-1]
g.V().has('pecosystem', 'maven').has('pname', 'io.vertx:vertx-web').has('version', '3.4.1').as('version').in('has_version').as('package').select('version','package').by(valueMap()); - in thread [gremlin-server-worker-1]
g.V().has('pecosystem', 'maven').has('pname', 'io.vertx:vertx-web-client').has('version', '3.4.1').as('version').in('has_version').as('package').select('version','package').by(valueMap()); - in thread [gremlin-server-worker-1]
When following local setup described in README.md, that is running
docker-compose -f docker-compose-server.yml up
Error message is produced:
ERROR: manifest for registry.centos.org/datamodel/gremlin:concaf not found
When going through https://registry.centos.org/containers/, there seems not to be such image.
[2018-02-22 17:49:32 +0000] [12] [ERROR] Error handling request /api/v1/sync_all?ecosystem=npm&package=08
--
| Traceback (most recent call last):
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle
| self.handle_request(listener_name, req, client, addr)
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
| super(GeventWorker, self).handle_request(*args)
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request
| respiter = self.wsgi(environ, resp.start_response)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1997, in __call__
| return self.wsgi_app(environ, start_response)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
| response = self.handle_exception(e)
| File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
| return cors_after_request(app.make_response(f(*args, **kwargs)))
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
| reraise(exc_type, exc_value, tb)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
| response = self.full_dispatch_request()
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
| rv = self.handle_user_exception(e)
| File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
| return cors_after_request(app.make_response(f(*args, **kwargs)))
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
| reraise(exc_type, exc_value, tb)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
| rv = self.dispatch_request()
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
| return self.view_functions[rule.endpoint](**req.view_args)
| File "/src/rest_api.py", line 73, in sync_all
| version=version_id)
| File "/src/data_importer.py", line 252, in fetch_pending_epvs
| items = self.rdb.execute(query, params)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1107, in execute
| bind, close_with_result=True).execute(clause, params or {})
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 971, in _connection_for_bind
| engine, execution_options)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind
| conn = bind.contextual_connect()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2106, in contextual_connect
| self._wrap_pool_connect(self.pool.connect, None),
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
| e, dialect, self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
| exc_info
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
| reraise(type(exception), exception, tb=exc_tb, cause=cause)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
| return fn()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect
| return _ConnectionFairy._checkout(self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout
| fairy = _ConnectionRecord.checkout(pool)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout
| rec = pool._do_get()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get
| self._dec_overflow()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
| compat.reraise(exc_type, exc_value, exc_tb)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get
| return self._create_connection()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection
| return _ConnectionRecord(self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__
| self.__connect(first_connect_check=True)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 661, in __connect
| exec_once(self.connection, self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 246, in exec_once
| self(*args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__
| fn(*args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 172, in on_connect
| do_on_connect(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 669, in on_connect
| fn(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 644, in on_connect
| hstore_oids = self._hstore_oids(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 775, in oneshot
| result = fn(self, *args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 678, in _hstore_oids
| oids = extras.HstoreAdapter.get_oids(conn)
| File "/usr/lib64/python2.7/site-packages/psycopg2/extras.py", line 917, in get_oids
| """ % typarray)
| DatabaseError: (psycopg2.DatabaseError) query_wait_timeout
| server closed the connection unexpectedly
| This probably means the server terminated abnormally
| before or while processing the request.
As observed in logs from prod instance:
--------------------------------------------------------------------------------
--
| [2018-02-20 12:28:51 +0000] [268] [ERROR] Error handling request /api/v1/sync_all?ecosystem=maven&package=com.github.hotchemi%3Aandroid-rate
| Traceback (most recent call last):
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle
| self.handle_request(listener_name, req, client, addr)
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
| super(GeventWorker, self).handle_request(*args)
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request
| respiter = self.wsgi(environ, resp.start_response)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1997, in __call__
| return self.wsgi_app(environ, start_response)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
| response = self.handle_exception(e)
| File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
| return cors_after_request(app.make_response(f(*args, **kwargs)))
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
| reraise(exc_type, exc_value, tb)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
| response = self.full_dispatch_request()
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
| rv = self.handle_user_exception(e)
| File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
| return cors_after_request(app.make_response(f(*args, **kwargs)))
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
| reraise(exc_type, exc_value, tb)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
| rv = self.dispatch_request()
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
| return self.view_functions[rule.endpoint](**req.view_args)
| File "/src/rest_api.py", line 73, in sync_all
| version=version_id)
| File "/src/data_importer.py", line 250, in fetch_pending_epvs
| items = self.rdb.execute(query, params)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1107, in execute
| bind, close_with_result=True).execute(clause, params or {})
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 971, in _connection_for_bind
| engine, execution_options)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind
| conn = bind.contextual_connect()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2106, in contextual_connect
| self._wrap_pool_connect(self.pool.connect, None),
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
| e, dialect, self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
| exc_info
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
| reraise(type(exception), exception, tb=exc_tb, cause=cause)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
| return fn()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect
| return _ConnectionFairy._checkout(self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout
| fairy = _ConnectionRecord.checkout(pool)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout
| rec = pool._do_get()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get
| self._dec_overflow()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
| compat.reraise(exc_type, exc_value, exc_tb)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get
| return self._create_connection()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection
| return _ConnectionRecord(self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__
| self.__connect(first_connect_check=True)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 661, in __connect
| exec_once(self.connection, self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 246, in exec_once
| self(*args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__
| fn(*args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 172, in on_connect
| do_on_connect(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 669, in on_connect
| fn(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 644, in on_connect
| hstore_oids = self._hstore_oids(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 775, in oneshot
| result = fn(self, *args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 678, in _hstore_oids
| oids = extras.HstoreAdapter.get_oids(conn)
| File "/usr/lib64/python2.7/site-packages/psycopg2/extras.py", line 917, in get_oids
| """ % typarray)
| DatabaseError: (psycopg2.DatabaseError) query_wait_timeout
| server closed the connection unexpectedly
| This probably means the server terminated abnormally
| before or while processing the request.
This is what I have noticed on production for few packages, for example org.wildfly.swarm:odata
:
All analyzed versions for this package (using bookkeeping api):
"analysed_versions": [
"2012.12.0",
"2017.12.0",
"2017.12.1",
"2018.1.0",
"2018.2.0",
"2018.2.0.Final"
]
Which of these versions are not yet synced to graph ? ( using graph_sync pending API):
{
"data": [
{
"ecosystem": "maven",
"name": "org.wildfly.swarm:odata",
"version": "2017.12.0"
}
]
}
Even after invoking the graph sync API, I still get the pending version as not synced to graph.
{
"data": {
"count_imported_EPVs": 0,
"epv": [
{
"ecosystem": "maven",
"name": "org.wildfly.swarm:odata",
"version": "2017.12.0"
}
],
"message": "Nothing to be synced to Graph!"
}
}
Notice the message above "count_imported_EPVs": 0,
. This clearly means that we have atleast one of three issues:
Current code coverage report is as below:
---------- coverage: platform linux2, python 2.7.14-final-0 ----------
Name Stmts Miss Cover Missing
-----------------------------------------------------------------------
src/__init__.py 2 0 100%
src/config.py 22 0 100%
src/data_importer.py 186 62 67% 69-70, 86, 97, 139, 158, 174, 189, 191-194, 213-215, 247-277, 294, 297-330, 333-356
src/data_source/__init__.py 0 0 100%
src/data_source/abstract_data_source.py 6 0 100%
src/data_source/s3_data_source.py 36 7 81% 29-31, 38, 43, 53, 63-64
src/graph_manager.py 38 13 66% 26-28, 35-39, 44-52, 57-68
src/graph_populator.py 178 24 87% 43, 61-63, 96-97, 116-117, 132, 142, 164, 167-169, 295-297, 307-309, 332-335, 352-360, 394
src/rest_api.py 97 65 33% 35-39, 45, 52, 58-71, 77-102, 108-126, 132-157, 161
src/sanitycheck.py 34 34 0% 3-59
src/set_logging.py 7 7 0% 3-12
-----------------------------------------------------------------------
TOTAL 606 212 65%
We don not use the last two files file:
src/set_logging.py
src/sanitycheck.py
We can remove them.
163856 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and Property value [1499082539.8] is of type class java.math.BigDecimal is not supported
java.lang.IllegalArgumentException: Property value [1499082539.8] is of type class java.math.BigDecimal is not supported
at org.apache.tinkerpop.gremlin.structure.Property$Exceptions.dataTypeOfPropertyValueNotSupported(Property.java:163)
Carried forward from another issue: openshiftio/openshift.io#2047 (comment)
package data (prod-bayesian-core-package-data) for a particular version is not present in S3 but package version does (prod-bayesian-core-data) have data in S3. For example there are 75 versions for arachne-ui but there is no package data present
I don't think that having no package-level data should be an issue for graph import. In case of arachne-ui
, it looks like an error in ingestion pipeline, but in general, there can be components for which we simply won't have any package-level data.
Could you please check in data-importer, how it behaves when there are no package-level data? And file an issue, if it fails :) Thanks 😉
Adding more issues to investigate further:
openshiftio/openshift.io#2047 (comment)
The graph sync ran for few days now, and we have 92695
NPM package versions in the pending list. There are no more package-versions going through the sync anymore.
There are known reasons due to which this is happening
prod-bayesian-core-package-data
) for a particular version is not present in S3 but package version does (prod-bayesian-core-data
) have data in S3. For example there are 75 versions for arachne-ui
but there is no package data present.Essentially for NPM, the graph-sync is done to the extent which it could be done in the current state.
ERROR:data_importer:The import failed: 'NoneType' object has no attribute 'split'
--
| ERROR:data_importer:Traceback for latest failure in import call: Traceback (most recent call last):
| File "/src/data_importer.py", line 87, in _import_keys_from_s3_http
| str_gremlin = GraphPopulator.create_query_string(obj)
| File "/src/graph_populator.py", line 371, in create_query_string
| str_package, prp_package = cls.construct_package_query(input_json)
| File "/src/graph_populator.py", line 204, in construct_package_query
| latest_version = cls._sanitize_text_for_query(input_json.get('latest_version')) or ''
| File "/src/graph_populator.py", line 18, in _sanitize_text_for_query
| text = " ".join([l.strip() for l in text.split("\n")])
| AttributeError: 'NoneType' object has no attribute 'split'
|
| ERROR:data_importer:
| Report from import_epv():
| The import failed: 'NoneType' object has no attribute 'split'
| Total number of EPVs imported: 0
| The last successfully imported EPV: None
This code branch in function get_qstring_for_cve_node
needs to be covered by unit test
fabric8-analytics-data-model/src/cve.py
Line 99 in 7d93d83
Reported by @geetikabatra
[2018-02-13 09:58:48 +0000] [13] [ERROR] Error handling request /api/v1/sync_all?package=org.webjars.npm%3Axml-writer&ecosystem=maven
| Traceback (most recent call last):
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle
| self.handle_request(listener_name, req, client, addr)
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
| super(GeventWorker, self).handle_request(*args)
| File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request
| respiter = self.wsgi(environ, resp.start_response)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1997, in __call__
| return self.wsgi_app(environ, start_response)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
| response = self.handle_exception(e)
| File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
| return cors_after_request(app.make_response(f(*args, **kwargs)))
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
| reraise(exc_type, exc_value, tb)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
| response = self.full_dispatch_request()
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
| rv = self.handle_user_exception(e)
| File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
| return cors_after_request(app.make_response(f(*args, **kwargs)))
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
| reraise(exc_type, exc_value, tb)
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
| rv = self.dispatch_request()
| File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
| return self.view_functions[rule.endpoint](**req.view_args)
| File "/src/rest_api.py", line 73, in sync_all
| version=version_id)
| File "/src/data_importer.py", line 250, in fetch_pending_epvs
| items = self.rdb.execute(query, params)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1107, in execute
| bind, close_with_result=True).execute(clause, params or {})
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 971, in _connection_for_bind
| engine, execution_options)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind
| conn = bind.contextual_connect()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2106, in contextual_connect
| self._wrap_pool_connect(self.pool.connect, None),
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
| e, dialect, self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
| exc_info
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
| reraise(type(exception), exception, tb=exc_tb, cause=cause)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
| return fn()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect
| return _ConnectionFairy._checkout(self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout
| fairy = _ConnectionRecord.checkout(pool)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout
| rec = pool._do_get()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get
| self._dec_overflow()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
| compat.reraise(exc_type, exc_value, exc_tb)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get
| return self._create_connection()
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection
| return _ConnectionRecord(self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__
| self.__connect(first_connect_check=True)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 661, in __connect
| exec_once(self.connection, self)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 246, in exec_once
| self(*args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__
| fn(*args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 172, in on_connect
| do_on_connect(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 669, in on_connect
| fn(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 644, in on_connect
| hstore_oids = self._hstore_oids(conn)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 775, in oneshot
| result = fn(self, *args, **kw)
| File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 678, in _hstore_oids
| oids = extras.HstoreAdapter.get_oids(conn)
| File "/usr/lib64/python2.7/site-packages/psycopg2/extras.py", line 904, in get_oids
| """ % typarray)
| DatabaseError: (psycopg2.DatabaseError) query_wait_timeout
| server closed the connection unexpectedly
| This probably means the server terminated abnormally
| before or while processing the request.
According to README.md a user should be running the following command to run data importer in a container:
sudo docker-compose -f docker-compose-importer.yml build
However, there seems to be incorrect yaml specification according to the error message produced:
yaml.scanner.ScannerError: while scanning a simple key
in "./docker-compose-importer.yml", line 27, column 7
could not find expected ':'
in "./docker-compose-importer.yml", line 28, column 7
The .DS_Store file needs to be ignored in .gitignore file
See:
https://ci.centos.org/job/devtools-fabric8-analytics-data-model-fabric8-analytics/420/console
test/test_data_importer.py::test_get_exception_msg Top level Sentry exception caught - failed creating log record
b'prefix: hello world!'
b'Traceback (most recent call last):\n File "/root/payload/venv/lib/python3.4/site-packages/raven/handlers/logging.py", line 91, in emit\n self.format(record)\n File "/usr/lib64/python3.4/logging/__init__.py", line 828, in format\n return fmt.format(record)\n File "/usr/lib64/python3.4/logging/__init__.py", line 573, in format\n record.exc_text = self.formatException(record.exc_info)\n File "/usr/lib64/python3.4/logging/__init__.py", line 523, in formatException\n traceback.print_exception(ei[0], ei[1], tb, None, sio)\n File "/usr/lib64/python3.4/traceback.py", line 169, in print_exception\n for line in _format_exception_iter(etype, value, tb, limit, chain):\n File "/usr/lib64/python3.4/traceback.py", line 146, in _format_exception_iter\n for value, tb in values:\n File "/usr/lib64/python3.4/traceback.py", line 125, in _iter_chain\n context = exc.__context__\nAttributeError: \'NoneType\' object has no attribute \'__context__\'\n'
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib64/python3.4/logging/__init__.py", line 978, in emit
msg = self.format(record)
File "/usr/lib64/python3.4/logging/__init__.py", line 828, in format
return fmt.format(record)
File "/usr/lib64/python3.4/logging/__init__.py", line 573, in format
record.exc_text = self.formatException(record.exc_info)
File "/usr/lib64/python3.4/logging/__init__.py", line 523, in formatException
traceback.print_exception(ei[0], ei[1], tb, None, sio)
File "/usr/lib64/python3.4/traceback.py", line 169, in print_exception
for line in _format_exception_iter(etype, value, tb, limit, chain):
File "/usr/lib64/python3.4/traceback.py", line 146, in _format_exception_iter
for value, tb in values:
File "/usr/lib64/python3.4/traceback.py", line 125, in _iter_chain
context = exc.__context__
AttributeError: 'NoneType' object has no attribute '__context__'
Call stack:
File "/root/payload/venv/bin/py.test", line 10, in <module>
sys.exit(main())
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/config/__init__.py", line 77, in main
return config.hook.pytest_cmdline_main(config=config)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 284, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 67, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 61, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/main.py", line 217, in pytest_cmdline_main
return wrap_session(config, _main)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/main.py", line 184, in wrap_session
session.exitstatus = doit(config, session) or 0
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/main.py", line 224, in _main
config.hook.pytest_runtestloop(session=session)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 284, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 67, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 61, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/main.py", line 245, in pytest_runtestloop
item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 284, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 67, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 61, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 77, in pytest_runtest_protocol
runtestprotocol(item, nextitem=nextitem)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 92, in runtestprotocol
reports.append(call_and_report(item, "call", log))
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 172, in call_and_report
call = call_runtest_hook(item, when, **kwds)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 196, in call_runtest_hook
treat_keyboard_interrupt_as_exception=item.config.getvalue("usepdb"),
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 212, in __init__
self.result = func()
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 194, in <lambda>
lambda: ihook(item=item, **kwds),
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 284, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 67, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 61, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/runner.py", line 122, in pytest_runtest_call
item.runtest()
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/python.py", line 1438, in runtest
self.ihook.pytest_pyfunc_call(pyfuncitem=self)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/hooks.py", line 284, in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 67, in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/manager.py", line 61, in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
File "/root/payload/venv/lib/python3.4/site-packages/pluggy/callers.py", line 187, in _multicall
res = hook_impl.function(*args)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/python.py", line 166, in pytest_pyfunc_call
testfunction(**testargs)
File "/root/payload/test/test_data_importer.py", line 68, in test_get_exception_msg
assert data_importer._get_exception_msg("prefix", e1) == "prefix: hello world!"
File "/root/payload/src/data_importer.py", line 55, in _get_exception_msg
tb = logging.exception(msg)
File "/usr/lib64/python3.4/logging/__init__.py", line 1814, in exception
error(msg, *args, **kwargs)
File "/usr/lib64/python3.4/logging/__init__.py", line 1805, in error
root.error(msg, *args, **kwargs)
File "/usr/lib64/python3.4/logging/__init__.py", line 1300, in error
Log 'msg % args' with severity 'ERROR'.
File "/usr/lib64/python3.4/logging/__init__.py", line 1308, in error
self._log(ERROR, msg, args, **kwargs)
File "/usr/lib64/python3.4/logging/__init__.py", line 1414, in _log
self.handle(record)
File "/usr/lib64/python3.4/logging/__init__.py", line 1424, in handle
self.callHandlers(record)
File "/usr/lib64/python3.4/logging/__init__.py", line 1486, in callHandlers
hdlr.handle(record)
File "/usr/lib64/python3.4/logging/__init__.py", line 853, in handle
self.emit(record)
File "/root/payload/venv/lib/python3.4/site-packages/_pytest/logging.py", line 206, in emit
logging.StreamHandler.emit(self, record)
Message: 'prefix: hello world!'
Arguments: ()
Top level Sentry exception caught - failed creating log record
b': hello world!'
b'Traceback (most recent call last):\n File "/root/payload/venv/lib/python3.4/site-packages/raven/handlers/logging.py", line 91, in emit\n self.format(record)\n File "/usr/lib64/python3.4/logging/__init__.py", line 828, in format\n return fmt.format(record)\n File "/usr/lib64/python3.4/logging/__init__.py", line 573, in format\n record.exc_text = self.formatException(record.exc_info)\n File "/usr/lib64/python3.4/logging/__init__.py", line 523, in formatException\n traceback.print_exception(ei[0], ei[1], tb, None, sio)\n File "/usr/lib64/python3.4/traceback.py", line 169, in print_exception\n for line in _format_exception_iter(etype, value, tb, limit, chain):\n File "/usr/lib64/python3.4/traceback.py", line 146, in _format_exception_iter\n for value, tb in values:\n File "/usr/lib64/python3.4/traceback.py", line 125, in _iter_chain\n context = exc.__context__\nAttributeError: \'NoneType\' object has no attribute \'__context__\'\n'
As per log from instance running on production:
85125493 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
--
| Script5383.groovy: 1: expecting ')', found '0.11' @ line 1, column 1088.
| al.jdk:core').has('version',''0.11.3.0''
| ^
|
| 1 error
|
| org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
| Script5383.groovy: 1: expecting ')', found '0.11' @ line 1, column 1088.
| al.jdk:core').has('version',''0.11.3.0''
| ^
The method CVEDelete.process
from the script cve.py
needs to be covered by unit tests:
fabric8-analytics-data-model/src/cve.py
Line 132 in 7d93d83
Please take a look and try to reduce cyclomatix complexity for the following functions:
src/graph_populator.py
M 106:4 GraphPopulator.construct_version_query - E (34)
M 272:4 GraphPopulator.construct_package_query - D (27)
2 blocks (classes, functions, methods) analyzed.
Average complexity: E (30.5)
1601524 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
--
| Script225.groovy: 1: expecting ')', found '0.11' @ line 1, column 1088.
| al.jdk:core').has('version',''0.11.3.0''
| ^
|
| 1 error
|
| org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
| Script225.groovy: 1: expecting ')', found '0.11' @ line 1, column 1088.
| al.jdk:core').has('version',''0.11.3.0''
| ^
|
| 1 error
|
| at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
| at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
| at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
| at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
| at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
| at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:145)
| at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
| at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
| at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
| at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
| at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
| at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
| at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
| at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
| at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
| at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
| 1603188 [gremlin-server-worker-1] WARN org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler - Invalid request - responding with 500 Internal Server Error and startup failed:
| Script226.groovy: 1: expecting ')', found '0.11' @ line 1, column 1088.
| al.jdk:core').has('version',''0.11.3.0''
| ^
|
| 1 error
|
| org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
| Script226.groovy: 1: expecting ')', found '0.11' @ line 1, column 1088.
| al.jdk:core').has('version',''0.11.3.0''
| ^
|
| 1 error
|
| at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
| at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
| at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
| at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
| at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:360)
| at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:145)
| at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:111)
| at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:237)
| at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:167)
| at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:931)
| at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:593)
| at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:569)
| at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:546)
| at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
| at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:211)
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.getScriptClass(GremlinGroovyScriptEngine.java:527)
| at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:446)
| at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
| at org.apache.tinkerpop.gremlin.groovy.engine.ScriptEngines.eval(ScriptEngines.java:119)
| at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$2(GremlinExecutor.java:287)
| at java.util.concurrent.FutureTask.run(FutureTask.java:266)
| at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
| at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
| at java.lang.Thread.run(Thread.java:748)
At present BayesianGraph.execute()
does not support queries with bounded parameters:
fabric8-analytics-data-model/src/graph_manager.py
Lines 16 to 30 in 13bd745
Support for bounded parameters will help us write better queries and avoid many bugs.
See for example https://ci.centos.org/job/devtools-fabric8-analytics-data-model-fabric8-analytics/14/consoleFull
The tests "pass", because the failure is silenced by try: except:, but it's there:
ERROR:test_insertion_from_minio:Traceback for latest failure in import call: Traceback (most recent call last):
File "/root/payload/test/test_insertion_from_minio.py", line 85, in test_insertion
assert report["epv"] == ["pypi:access_points:0.4.59"]
AssertionError: assert [{'pypi/acces...json'], ...}}] == ['pypi:access_points:0.4.59']
At index 0 diff: {'pypi/access_points/': {'version': '0.4.59', 'package': 'access_points', 'ecosystem': 'pypi', 'pkg_list_keys': ['pypi/access_points/github_details.json', 'pypi/access_points/libraries_io.json'], 'ver_key_prefix': 'pypi/access_points/0.4.59', 'pkg_key_prefix': 'pypi/access_points/', 'ver_list_keys': ['pypi/access_points/0.4.59/code_metrics.json', 'pypi/access_points/0.4.59/metadata.json', 'pypi/access_points/0.4.59/security_issues.json', 'pypi/access_points/0.4.59/source_licenses.json']}} != 'pypi:access_points:0.4.59'
Check if Python version installed on CI slaves are at least 3.6 or newer.
S3RegionRedirector.redirect_from_error of <botocore.utils.S3RegionRedirector object at 0x7fa6e6346950>>
DEBUG:botocore.hooks:Event after-call.s3.ListObjects: calling handler <function decode_list_object at 0x7fa6ea78b410>
DEBUG:boto3.resources.factory:Loading s3:ObjectSummary
EPEL repositories now contain proper Python 3.6 packages and at the same moment Python 3.4 is being deprecated [1] [2].
It means that we need to upgrade this repository to use Python 3.6 instead of Python 3.4.
What needs to be changed AND tested:
References:
[1] https://lists.fedoraproject.org/archives/list/[email protected]/thread/EGUMKAIMPK2UD5VSHXM53BH2MBDGDWMO/
[2] https://www.reddit.com/r/CentOS/comments/azetyy/python_34_to_be_deprecated_this_month/
The method CVEPut.process
from the script cve.py
needs to be covered by unit tests:
fabric8-analytics-data-model/src/cve.py
Line 46 in 7d93d83
Do we plan to remove all the Python 2-related code from this repository?
api_v1 = Blueprint('api_v1', __name__)
# Python2.x: Make default encoding as UTF-8
if sys.version_info.major == 2:
reload(sys)
sys.setdefaultencoding('UTF8')
IMHO it is not much relevant these days...
=================================== FAILURES ===================================
__________________________ test_ingest_to_graph_valid __________________________
client = <FlaskClient <Flask 'data_importer_app'>>
def test_ingest_to_graph_valid(client):
"""Add test for ingest to graph API when some key is missing."""
input_data = [
{
"ecosystem": "maven",
"name": "commons-collections:commons-collections",
"source_repo": "redhat_maven"
}
]
url = url_for('api_v1.ingest_to_graph')
response = client.post(url,
data=json.dumps(input_data),
headers={'Content-Type': 'application/json'})
# we expect that the HTTP code will be 400/Bad Request
assert response.status_code == 400
data = json.loads(response.get_data().decode('utf-8'))
epv_keys = input_data[0].keys()
> assert data['message'] == 'Invalid keys found in input: ' + ','.join(epv_keys)
E AssertionError: assert 'Invalid keys...m,source_repo' == 'Invalid keys ...cosystem,name'
E - Invalid keys found in input: name,ecosystem,source_repo
E ? ^^^ ^^^^^ -----
E + Invalid keys found in input: source_repo,ecosystem,name
E ? ^^^^^ +++++ ^^^
test/test_rest_api.py:122: AssertionError
Database fails to respond for some queries as shown in logs below. This however is not recurring, so adding here for tracking if it happens again:
--------------------------------------------------------------------------------
INFO in rest_api [/src/rest_api.py:28]:
Ready to serve requests
--------------------------------------------------------------------------------
[2018-02-06 06:04:44 +0000] [44] [ERROR] Error handling request /api/v1/sync_all?package=javax.xml%3Ajaxrpc-api-osgi&ecosystem=maven
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle
self.handle_request(listener_name, req, client, addr)
File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
super(GeventWorker, self).handle_request(*args)
File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/src/rest_api.py", line 73, in sync_all
version=version_id)
File "/src/data_importer.py", line 250, in fetch_pending_epvs
items = self.rdb.execute(query, params)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1107, in execute
bind, close_with_result=True).execute(clause, params or {})
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 971, in _connection_for_bind
engine, execution_options)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind
conn = bind.contextual_connect()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2106, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
e, dialect, self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
exc_info
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
return fn()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect
return _ConnectionFairy._checkout(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout
rec = pool._do_get()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get
self._dec_overflow()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get
return self._create_connection()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection
return _ConnectionRecord(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__
self.__connect(first_connect_check=True)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 661, in __connect
exec_once(self.connection, self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 246, in exec_once
self(*args, **kw)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__
fn(*args, **kw)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 172, in on_connect
do_on_connect(conn)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 669, in on_connect
fn(conn)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 644, in on_connect
hstore_oids = self._hstore_oids(conn)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 775, in oneshot
result = fn(self, *args, **kw)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 678, in _hstore_oids
oids = extras.HstoreAdapter.get_oids(conn)
File "/usr/lib64/python2.7/site-packages/psycopg2/extras.py", line 904, in get_oids
""" % typarray)
DatabaseError: (psycopg2.DatabaseError) query_wait_timeout
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
--------------------------------------------------------------------------------
INFO in rest_api [/src/rest_api.py:128]:
Selective Ingestion with payload - {"select_ingest": ["keywords_tagging", "git_stats", "libraries_io", "github_details"], "package_list": [{"ecosystem": "maven", "version": null, "name": "com.lihaoyi:mill-moduledefs_2.12"}]}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
INFO in rest_api [/src/rest_api.py:136]:
{'epv': {u'select_ingest': [u'keywords_tagging', u'git_stats', u'libraries_io', u'github_details'], u'package_list': [{u'ecosystem': u'maven', u'version': '', u'name': u'com.lihaoyi:mill-moduledefs_2.12'}]}, 'message': 'Nothing to be synced to Graph!', 'count_imported_EPVs': 0}
--------------------------------------------------------------------------------
[2018-02-06 06:17:59 +0000] [44] [ERROR] Error handling request /api/v1/sync_all?ecosystem=maven&package=net.artsy%3Aatomic-store_2.11
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 56, in handle
self.handle_request(listener_name, req, client, addr)
File "/usr/lib/python2.7/site-packages/gunicorn/workers/ggevent.py", line 152, in handle_request
super(GeventWorker, self).handle_request(*args)
File "/usr/lib/python2.7/site-packages/gunicorn/workers/async.py", line 107, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1997, in __call__
return self.wsgi_app(environ, start_response)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1985, in wsgi_app
response = self.handle_exception(e)
File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1540, in handle_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/lib64/python2.7/site-packages/flask_cors/extension.py", line 161, in wrapped_function
return cors_after_request(app.make_response(f(*args, **kwargs)))
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/lib64/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/src/rest_api.py", line 73, in sync_all
version=version_id)
File "/src/data_importer.py", line 250, in fetch_pending_epvs
items = self.rdb.execute(query, params)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 1107, in execute
bind, close_with_result=True).execute(clause, params or {})
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 971, in _connection_for_bind
engine, execution_options)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/orm/session.py", line 403, in _connection_for_bind
conn = bind.contextual_connect()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2106, in contextual_connect
self._wrap_pool_connect(self.pool.connect, None),
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2145, in _wrap_pool_connect
e, dialect, self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 1456, in _handle_dbapi_exception_noconnection
exc_info
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/compat.py", line 202, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/base.py", line 2141, in _wrap_pool_connect
return fn()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 387, in connect
return _ConnectionFairy._checkout(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 766, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 516, in checkout
rec = pool._do_get()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1138, in _do_get
self._dec_overflow()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 1135, in _do_get
return self._create_connection()
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 333, in _create_connection
return _ConnectionRecord(self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 461, in __init__
self.__connect(first_connect_check=True)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 661, in __connect
exec_once(self.connection, self)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 246, in exec_once
self(*args, **kw)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/event/attr.py", line 256, in __call__
fn(*args, **kw)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 172, in on_connect
do_on_connect(conn)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 669, in on_connect
fn(conn)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 644, in on_connect
hstore_oids = self._hstore_oids(conn)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/util/langhelpers.py", line 775, in oneshot
result = fn(self, *args, **kw)
File "/usr/lib64/python2.7/site-packages/sqlalchemy/dialects/postgresql/psycopg2.py", line 678, in _hstore_oids
oids = extras.HstoreAdapter.get_oids(conn)
File "/usr/lib64/python2.7/site-packages/psycopg2/extras.py", line 904, in get_oids
""" % typarray)
DatabaseError: (psycopg2.DatabaseError) query_wait_timeout
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
Following errors happen due to unexpected characters in the query strings:
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script782.groovy: 1: unexpected char: '\' @ line 1, column 7263.
mber value using Number.equals\ boolean
^
--
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script783.groovy: 1: unexpected char: '\' @ line 1, column 7284.
mber value using Number.equals\ boolean
^
--
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1123.groovy: 1: unexpected char: '\' @ line 1, column 11138.
\'/\' withBody name/\w/ \'harr
^
--
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1125.groovy: 1: unexpected char: '\' @ line 1, column 11138.
\'/\' withBody name/\w/ \'harr
^
1 error
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
Script1149.groovy: 1: unexpected char: '\' @ line 1, column 2867.
hodquotPOSTquot .bodyquot\quotname\
^
Following file contains full error logs:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.