Git Product home page Git Product logo

graphium-neo4j's Introduction

Graphium Neo4j

Graphium Neo4j is an extension of the project Graphium implemented as Neo4j Server plugin.

Graphium Neo4j is an extension of the project Graphium based on Neo4j. Neo4j is a famous graph database which comes with various graph algorithms. Neo4j's graph model is built for answering graph dependent questions in a more flexible and performant way than you can do with a relational database. As transport graphs can be modeled as graphs and stored in Neo4j, we benefit from its features.

Graphium Neo4j is built for those who want to manage graphs and graph versions within Neo4j and / or need routing or even a map matching API. Graphium Neo4j consists of modules which have to be deployed as Neo4j Server plugins (so called unmanaged extensions). Therefore Graphium Neo4j is not a standalone server but requires a Neo4j Server.

Features

Graphium Core

Graphium Neo4j include all Graphium core features. See Graphium.

Routing

Graphium Neo4j's routing API handles routing requests between two coordinates on a transport graph. The result will be returned in a JSON format.


Map: basemap.at

Map Matching

Graphium Neo4j's map matching API matches trajectories onto a transport graph. In contrast to Hidden Markov Model implementations, Graphium Neo4j's map matching core implementation is built for processing trajectories having a high sampling rate (<20 seconds). The main focus was to enable a high-performance processing of those high-sampled trajectories with a very low error ratio. Also low-sampled trajectories can be processed, but with an increasing error ratio. With Graphium's ability of working in distributed systems, you can improve the performance by horizontal scaling. Graphium Neo4j's map matching supports offline and online map matching.

Map Matching of a lower sampled track (for better visualization); blue point show GPS track points, red linestring represents the map matched path on graph (thin black linestrings):


Map: basemap.at

Map Matching of a track whose GPS track points partially could not have been matched onto graph. The map matcher detects non matchable parts of the track and splits the map matched path:


Map: basemap.at

Please note

For both routing and map matching a valid graph version (especially validity has to be defined correctly) has to be imported and activated (state is ACTIVE). If a graph version has been imported its state is INITIAL, which normally means someone has to verify the data first or this graph version is only some kind of test version and should not be taken into account for the production system. Only after verification and activation of a graph version it can be used for processing.

API

Routing

Defining road blocks

Map Matching

Quickstart

  1. Graphium Neo4j is an extension of Graphium. Therefore Graphium has to be checked out and built before.

  2. Build Graphium Neo4j via Maven mvn clean install

  3. The following plugins have to be deployed in the Neo4j's plugins directory: graphium-neo4j-server-integration-plugin-XXX.jar (Graphium's core functionality and integration into Neo4j) graphium-api-neo4j-plugin-XXX.jar (API) graphium-routing-neo4j-plugin-XXX.jar (routing functionality and API) graphium-mapmatching-neo4j-plugin-XXX.jar (map matching functionality and API)

  4. Copy property files to Neo4j's conf directory and configure:

    graphVersionCapacities.properties
    import.properties
    log4j.xml
    mapmatcher.properties
    neo4j_db.properties
    server.properties
    

    Default property files can be found in neo4j-server-integration/doc/neo4j-default/conf.

    The following properties within server.properties have to be adapted. Be careful at changing other property values: graphium.server.name graphium.server.uri

  5. Register Graphium at Neo4j: neo4j.conf in Neo4j's conf directory has to be extended by:

    dbms.unmanaged_extension_classes=at.srfg.graphium.neo4j.bootstrap=/graphium-neo4j/api
    dbms.jvm.additional=-Dgraphium.conf.path=file:conf/
    graphium.secured=true/false
    
  6. Start Neo4j

    neo4j.bat console
  7. Download and convert OSM File into Graphium's JSON format, import into Graphium central server *):

    java -jar /osm2graphium.one-jar.jar -i http://download.geofabrik.de/europe/andorra-latest.osm.pbf -o / -n osm_andorra -fd true -v 200603 -q 20000 -t 5 --highwayTypes "motorway, motorway_link, primary, primary_link" -u "http://localhost:7474/graphium/api/segments/graphs/osm_andorra/versions/200603?overrideIfExists=true"
  8. Activate imported graph version

    curl -X PUT "http://localhost:7474/graphium/api/metadata/graphs/osm_andorra/versions/200603/state/ACTIVE"
  9. Check server state

    curl -X GET "http://localhost:7474/graphium/api/status"

Docker

  1. Start Docker setup

    docker-compose up -d

Application and database logs can be obtained via docker-compose logs.

If any of the following steps crashes because of a Java heap exception you have to configure memory definition of Docker and configure Neo4j's heap memory by setting the environment variable NEO4J_dbms_memory_heap_max__size within docker-compose.yml.

  1. Download and convert OSM File into Graphium's JSON format, import into Graphium central server *):

    docker exec -it graphium-neo4j-server java -jar /osm2graphium.one-jar.jar -i http://download.geofabrik.de/europe/andorra-latest.osm.pbf -o / -n osm_andorra -fd true -v 200603 -q 20000 -t 5 --highwayTypes "motorway, motorway_link, primary, primary_link" -u "http://localhost:7474/graphium/api/segments/graphs/osm_andorra/versions/200603?overrideIfExists=true"
  2. Activate imported graph version

    curl -X PUT "http://localhost:7474/graphium/api/metadata/graphs/osm_andorra/versions/200603/state/ACTIVE"
  3. Check server state

    curl -X GET "http://localhost:7474/graphium/api/status"

*) Notice parameter highwayTypes: only high level streets will be imported; adapt if needed (see OSM highway types).

Routing Example

curl -X GET "http://localhost:7474/graphium/api/routing/graphs/osm_andorra/route.do?coords=1.614733,42.5720773;1.5937977,42.56574"

Map Matching Example

curl -H "Accept:application/json" -H "Content-Type:application/json" -X POST "http://localhost:7474/graphium/api/matching/graphs/osm_andorra/versions/current/matchtrack?outputVerbose=true&timeoutMs=60000" -d '{"id":1,"trackPoints":[{"id":0,"timestamp":1591776000000,"x":1.5512481,"y":42.512805,"z":0},{"id":0,"timestamp":1591776002000,"x":1.5515414,"y":42.5130289,"z":0},{"id":0,"timestamp":1591776006000,"x":1.5520432,"y":42.5132296,"z":0},{"id":0,"timestamp":1591776011000,"x":1.5527147,"y":42.5135306,"z":0},{"id":0,"timestamp":1591776017000,"x":1.5533361,"y":42.514206,"z":0},{"id":0,"timestamp":1591776025000,"x":1.5538764,"y":42.5151092,"z":0},{"id":0,"timestamp":1591776035000,"x":1.5542151,"y":42.5162136,"z":0}]}'

Plugins Development

You can build your own Neo4j plugins based on Graphium Neo4j. Graphium Neo4j plugins use Spring Framework, which defines configuration information within ApplicationContexts. For integration of custom Neo4j plugins into Graphium Neo4j only one ApplicationContext with the name pattern application-context-graphium-neo4j-plugin*.xml has to be provided. All Beans defined within this ApplicationContext will be loaded automatically by Graphium Neo4j.

Dependencies

graphium-neo4j's People

Contributors

dependabot[bot] avatar emplexed avatar jinnerbichler avatar micwimmer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

graphium-neo4j's Issues

NullPointerException when setting state to ACTIVE

I am trying to follow the configurations instructions but am getting stuck when trying to set the state of the imported map. The following request results in a null pointer exception.

curl -X PUT "http://localhost:7474/graphium/metadata/graphs/osm/versions/1/state/ACTIVE"
08:04:24,250 ERROR [GlobalExceptionController.handleInconsistentStateException()]  Error occured during request
java.lang.NullPointerException
	at at.srfg.graphium.neo4j.service.impl.STRTreeCacheManager.update(STRTreeCacheManager.java:199)
	at java.util.Observable.notifyObservers(Observable.java:159)
	at at.srfg.graphium.core.service.impl.GraphVersionMetadataServiceImpl.setGraphVersionState(GraphVersionMetadataServiceImpl.java:116)
	at at.srfg.graphium.api.service.impl.GraphVersionMetadataAPIServiceImpl.setGraphVersionState(GraphVersionMetadataAPIServiceImpl.java:156)
	at at.srfg.graphium.api.service.impl.GraphVersionMetadataAPIServiceImpl.changeState(GraphVersionMetadataAPIServiceImpl.java:216)
	at at.srfg.graphium.api.controller.GraphVersionMetadataApiController.setGraphMetadataFields(GraphVersionMetadataApiController.java:194)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:114)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
	at org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:808)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:221)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.eclipse.jetty.server.Server.handle(Server.java:497)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:748)

I am running neo4j in a docker container, and mounting the build plugins to the correct directory.

docker run -d -p 7474:7474 -p 7687:7687 -v ${PWD}/plugins:/var/lib/neo4j/plugins -v ${PWD}/conf:/var/lib/neo4j/conf --name neo4j neo4j:3.2.3

The issue seems to be with the metadata being passed the observable, but I can't say for sure.

Map Matching problem with tunnels

Tunnels are always a problem for GPS devices. Usually there is no GPS information (track points) about trajectory parts within tunnels available. The Map Matcher has to route in those cases from one tunnel's end to the other.
In some situations GPS track points at the end of a tunnel are too erroneous so routing fails and skips the way segments representing the tunnel, resulting in gaps of the map matched path.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.