zazuko / fuseki-geosparql Goto Github PK

View Code? Open in Web Editor NEW

7.0 6.0 7.0 242 KB

Non-official Fuseki Docker image with GeoSPARQL support

License: MIT License

Dockerfile 84.60% Shell 15.40%

fuseki geosparql docker docker-image triplestore

fuseki-geosparql's Introduction

Fuseki with GeoSPARQL support

Docker image

This Docker image is configured to have GeoSPARQL support.

You can run the Docker image using:

docker run --rm -p3030:3030 -it ghcr.io/zazuko/fuseki-geosparql

It is listening on the 3030 port, so you should be able to access the web interface using: http://localhost:3030.

Configuration

It is possible to use the following environment variables for configuration:

ADMIN_PASSWORD (default: admin), the password for the admin user
JAVA_OPTS (default: -Xmx2048m -Xms2048m), allocate more resources by changing this values
DISABLE_OTEL (default: false), disable OpenTelemetry support by setting this value to true

Feel free to edit the config/config.ttl file before building this image. For information, this file will be mounted at the following path in the container: /fuseki/config.ttl.

Routes

Here are some default routes, publicly available:

/$/status: get Fuseki's status
/$/server: get Fuseki's status
/$/ping: health check endpoint
/$/metrics: some Prometheus metrics

All other routes that have are prefixed with /$/ needs basic authentication:

username: admin
password: value of the ADMIN_PASSWORD environment variable

Some routes that are known to be used for write permissions are also protected; there are the ones ending with:

/data
/upload
/update

All other routes are publicly available.

If you want to change this behavior, you will need to change the config/shiro.ini file. It will be mounted at this location: /opt/fuseki/shiro.ini. When the container is starting, the value for ADMIN_PASSWORD will be set, and the final file would be created at /fuseki/shiro.ini.

Dataset

A dataset ds is already configured in the config/config.ttl file. It's a read-only dataset, with union graph and GeoSPARQL enabled. Feel free to update this file for your needs.

This dataset is stored at /fuseki/databases/ds. If you want to persist this dataset, you can mount /fuseki/databases/ as a volume.

If your dataset is huge, you may be interested in generating the spatial index file before starting the Fuseki instance. You can have a look at this tool: https://github.com/zazuko/spatial-indexer.

fuseki-geosparql's People

Stargazers

Watchers

Forkers

fils earthcube kurrawong brechtvdv mathiasvda kurrawong

fuseki-geosparql's Issues

Upgrade Jena to 4.10.0

A new version of Apache Jena is available.
We should upgrade to use it.

No access to database

Hi! I was trying your tool but I came across a problem. I think it might be caused by the configuration file but I am not sure.

I'd like to configure the triplestore to be able to both read/write so I added some lines to the configuration file. So I changed the config file as follows:

[] rdf:type fuseki:Server ;  
 .
<#service> rdf:type fuseki:Service;
  fuseki:name "ds";
  fuseki:endpoint [ fuseki:operation fuseki:query ; fuseki:name "sparql" ];
  fuseki:endpoint [ fuseki:operation fuseki:query ; fuseki:name "query" ];
  fuseki:endpoint [ fuseki:operation fuseki:gsp-r ; fuseki:name "get" ];
  fuseki:endpoint [ fuseki:operation fuseki:gsp-rw ; fuseki:name "data" ] ; 
  fuseki:endpoint [ fuseki:operation fuseki:update ; fuseki:name "update"] ;
  fuseki:endpoint [ fuseki:operation fuseki:patch ;  fuseki:name "patch" ] ;
  fuseki:dataset <#ds> ;
  .
<#ds> rdf:type geosparql:geosparqlDataset ;
  geosparql:spatialIndexFile "databases/ds/spatial.index";
  geosparql:inference            true ;
  geosparql:queryRewrite         true ;
  geosparql:indexEnabled         true ;
  geosparql:applyDefaultGeometry false ;
  geosparql:indexSizes           "-1,-1,-1" ;       # Default - unlimited.
  geosparql:indexExpires         "5000,5000,5000" ; # Default - time in milliseconds.
  geosparql:dataset <#baseDataset> ;
  .
<#baseDataset> rdf:type tdb2:DatasetTDB2 ;
  tdb2:location "databases/ds" ;
  tdb2:unionDefaultGraph true ;
  .

I built the docker folder as suggested and ran the repository from the new docker file.

docker run --rm -p 3090:3030 --volume /home/beyza/GeoFusekiData:/fuseki/databases/ds docker.io/library/fuseki_release
(also tried this: docker run --rm -p 3090:3030 -it docker.io/library/fuseki_release)

I tried to add data from the interface and I guess according to the logs everything seems fine.

[otel.javaagent 2024-02-05 16:26:28:354 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.32.0
[fuseki 2024-02-05 16:26:31:221 +0000] [main] INFO org.apache.jena.fuseki.Server {} - Apache Jena Fuseki 4.10.0
[fuseki 2024-02-05 16:26:31:427 +0000] [main] INFO org.apache.jena.fuseki.Config {} - FUSEKI_HOME=/opt/fuseki
[fuseki 2024-02-05 16:26:31:428 +0000] [main] INFO org.apache.jena.fuseki.Config {} - FUSEKI_BASE=/fuseki
[fuseki 2024-02-05 16:26:31:482 +0000] [main] INFO org.apache.jena.fuseki.Config {} - Shiro file: file:///fuseki/shiro.ini
[fuseki 2024-02-05 16:26:32:456 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - Applying GeoSPARQL Schema - Started
[fuseki 2024-02-05 16:26:55:617 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - GeoSPARQL schema applied to graph: default
[fuseki 2024-02-05 16:26:55:674 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - Applying GeoSPARQL Schema - Completed
[fuseki 2024-02-05 16:26:55:692 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - Find Mode SRS - Started
[fuseki 2024-02-05 16:26:55:696 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - Find Mode SRS - Completed
[fuseki 2024-02-05 16:26:55:696 +0000] [main] WARN org.apache.jena.geosparql.assembler.GeoAssembler {} - No SRS found. Check 'http://www.opengis.net/ont/geosparql#hasSerialization' or 'http://www.w3.org/2003/01/geo/wgs84_pos#lat'/'http://www.w3.org/2003/01/geo/wgs84_pos#lon' predicates are present in the source data. Hint: Inferencing with GeoSPARQL schema may be required.
[fuseki 2024-02-05 16:26:55:888 +0000] [main] INFO org.apache.jena.fuseki.Server {} - Configuration file: /fuseki/config.ttl
[fuseki 2024-02-05 16:26:55:891 +0000] [main] INFO org.apache.jena.fuseki.Server {} - Path = /ds
[fuseki 2024-02-05 16:26:55:893 +0000] [main] INFO org.apache.jena.fuseki.Server {} -   Memory: 7.8 GiB
[fuseki 2024-02-05 16:26:55:894 +0000] [main] INFO org.apache.jena.fuseki.Server {} -   Java:   16.0.2
[fuseki 2024-02-05 16:26:55:894 +0000] [main] INFO org.apache.jena.fuseki.Server {} -   OS:     Linux 5.4.0-150-generic amd64
[fuseki 2024-02-05 16:26:55:894 +0000] [main] INFO org.apache.jena.fuseki.Server {} -   PID:    1
[fuseki 2024-02-05 16:26:56:045 +0000] [main] INFO org.apache.jena.fuseki.Server {} - Started 2024/02/05 16:26:56 GMT on port 3030
[fuseki 2024-02-05 16:27:41:987 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=7af0c04255c9d11317f0104fdb16f407, span_id=572095e1dc30076e, trace_flags=01} - [4] POST http://localhost:3090/ds/data
[fuseki 2024-02-05 16:27:42:294 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=7af0c04255c9d11317f0104fdb16f407, span_id=572095e1dc30076e, trace_flags=01} - [4] Filename: townland-logainm.ttl, Content-Type=text/turtle, Charset=null => Turtle : Count=5970 Triples=5970 Quads=0
[fuseki 2024-02-05 16:27:42:390 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=7af0c04255c9d11317f0104fdb16f407, span_id=572095e1dc30076e, trace_flags=01} - [4] 200 OK (401 ms)
[fuseki 2024-02-05 16:27:49:278 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=d0ba0fc8f1909ba5c6cafd47c5c009fe, span_id=b7ecdc754c7f8b38, trace_flags=01} - [7] POST http://localhost:3090/ds/sparql
[fuseki 2024-02-05 16:27:49:281 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=d0ba0fc8f1909ba5c6cafd47c5c009fe, span_id=b7ecdc754c7f8b38, trace_flags=01} - [7] Query = PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT * WHERE {   ?sub ?pred ?obj . } LIMIT 10
[fuseki 2024-02-05 16:27:49:302 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=d0ba0fc8f1909ba5c6cafd47c5c009fe, span_id=b7ecdc754c7f8b38, trace_flags=01} - [7] 200 OK (23 ms)
[fuseki 2024-02-05 16:27:56:106 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=4003a510915e0f939d8d9e0cbc96bf84, span_id=92fd3c4d2d821295, trace_flags=01} - [8] POST http://localhost:3090/ds/data
[fuseki 2024-02-05 16:27:56:132 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=4003a510915e0f939d8d9e0cbc96bf84, span_id=92fd3c4d2d821295, trace_flags=01} - [8] Upload error: Unknown content type for triples: [application/x-www-form-urlencoded]
[fuseki 2024-02-05 16:27:56:134 +0000] [qtp1525943877-21] INFO org.apache.jena.fuseki.Fuseki {trace_id=4003a510915e0f939d8d9e0cbc96bf84, span_id=92fd3c4d2d821295, trace_flags=01} - [8] 400 Bad Request (27 ms)
[fuseki 2024-02-05 16:28:09:459 +0000] [qtp1525943877-25] INFO org.apache.jena.fuseki.Fuseki {trace_id=8924f6ea680735269506fd7e1b439979, span_id=a27b16a80cfae1e6, trace_flags=01} - [9] POST http://localhost:3090/ds/query
[fuseki 2024-02-05 16:28:09:462 +0000] [qtp1525943877-25] INFO org.apache.jena.fuseki.Fuseki {trace_id=8924f6ea680735269506fd7e1b439979, span_id=a27b16a80cfae1e6, trace_flags=01} - [9] Query = PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#> SELECT * WHERE {   ?sub ?pred ?obj . } LIMIT 10
[fuseki 2024-02-05 16:28:09:477 +0000] [qtp1525943877-25] INFO org.apache.jena.fuseki.Fuseki {trace_id=8924f6ea680735269506fd7e1b439979, span_id=a27b16a80cfae1e6, trace_flags=01} - [9] 200 OK (17 ms)

However, the problem is that I cannot insert or retrieve data even though the database is updated when I insert the data. I can see that when I bash into the container. The total number of triples seems to be 0 and nothing returns as a result of the query. Following is my configuration file:

Your help will be very much appreciated!

Add Apache SIS by default

Until now, users needed to add and configure Apache SIS by their own.
The goal would be to include it by default to provide a more out-of-the-box Docker image.

Windows line ending in entrypoint.sh script

Hello, the entrypoint script contains Windows line endings. When building and running the dockerfile, you will receive this error:

exec /opt/fuseki/entrypoint.sh: no such file or directory

Changing the line ending to UNIX resolves this

Add an option to remove OTEL

It could be great to have an option to remove OTEL support using an environment variable for people that don't want it.

request for advice on making databases active

I'm connecting with HTTP to set up a store, I POST an assembler, and there is a new folder/DB set up (I looked inside the docker instance) but this new store is not reported with a /$/datasets call, and I cannot activate it as expected with HTTP - I've done a POST of {URL}/$/datasets/{DB_NAME}?state=active but it isn't seeming to be activated. ( https://jena.apache.org/documentation/fuseki2/fuseki-server-protocol.html )
Advice appreciated please

Missing dot in config file

I think there is a missing dot at the end of this line:
https://github.com/zazuko/fuseki-geosparql/blob/a60b232a26bcc04d471f4306c057766931cbffb4/config/config.ttl#L6C1-L7C55

And the one after

Warning The “SIS_DATA” environment variable is not set / or value is not an existing directory

I have dowloaded and run successfully your docker but even after fresh builds with various changes in Dockerfile or added the option -e "SIS_DATA=/home/vidal/Logiciels/Apache/apache-sis-1.4/data/" I keep at best getting the warning :

[main] WARN org.apache.sis.system {} - The SIS_DATA environment variable is defined, but the given “/opt/fuseki/data” value is not an existing directory.

# even if 
ll /opt/fuseki/data
total 12K
drwxr-xr-x 3 root   root   4,0K  7 déc.  16:16 ./
drwxrwxr-x 6 fuseki fuseki 4,0K  7 déc.  16:15 ../
drwxr-xr-x 3 root   root   4,0K  7 déc.  16:16 Databases/

I am not flent with docker nor Java, could you please help me find howyo use SIS resource
Thanks for this very usefull work.

Enable tests back

I disabled tests with this commit: 0853d05

Tests were passing well on my ARM64 and AMD64 machines when building the Docker image.
I guess that the CI was failing because Fuseki was taking too much time to be instantiated (it was replying a 500 error code) during the tests, and this was making them fail.

This issue is here to remember why tests were disabled and to retry to enabled them back again for the next release.

Add multi-arch support to the Docker image

This will allow users to run the Docker image on more devices.

help getting started

First.. thanks so much for sharing your Dockerfile for building Jena with geosparql support.

I am able to both pull and run and also rebuild the image with podman on Fedora.

When running I get the following:

(base) ➜  fuseki-geosparql git:(main) podman run -i --privileged  -v /home/fils/Containers/dvols/jena:/data -p 3030:3030 --rm a9e5420065a9
[opentelemetry.auto.trace 2022-01-21 16:50:08:092 +0000] [main] INFO io.opentelemetry.javaagent.tooling.VersionLogger - opentelemetry-javaagent - version: 1.2.0
WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
[fuseki 2022-01-21 16:50:09:375 +0000] [main] INFO org.apache.jena.fuseki.geosparql.Main {} - Arguments Received: [--loopback, false, -t, /fuseki/databases/ds, -t2, --validate, --default_geometry]
[fuseki 2022-01-21 16:50:09:411 +0000] [main] INFO org.apache.jena.fuseki.geosparql.DatasetOperations {} - Server Configuration: port=3030, datsetName=ds, loopbackOnly=false, updateAllowed=false, inference=false, applyDefaultGeometry=true, validateGeometryLiteral=true, convertGeoPredicates=false, removeGeoPredicates=false, queryRewrite=true, tdbFile=/fuseki/databases/ds, fileGraphFormats=[], fileGraphDelimiters=[], indexEnabled=true, indexSizes=[-1, -1, -1], indexExpiries=[5000, 5000, 5000], spatialIndexFile=null, tdb2=true, help=false
[fuseki 2022-01-21 16:50:09:465 +0000] [main] INFO org.apache.jena.fuseki.geosparql.DatasetOperations {} - TDB Dataset: /fuseki/databases/ds, TDB2: true
[fuseki 2022-01-21 16:50:10:031 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - Applying hasDefaultGeometry - Started
[fuseki 2022-01-21 16:50:10:079 +0000] [main] INFO org.apache.jena.geosparql.configuration.GeoSPARQLOperations {} - Applying hasDefaultGeometry - Completed
[fuseki 2022-01-21 16:50:10:185 +0000] [main] WARN org.apache.sis.system {} - The “SIS_DATA” environment variable is not set.
[fuseki 2022-01-21 16:50:10:351 +0000] [main] WARN org.apache.jena.fuseki.geosparql.DatasetOperations {} - Datset empty. Spatial Index not constructed. Server will require restarting after adding data and any updates to build Spatial Index.
[fuseki 2022-01-21 16:50:10:761 +0000] [Thread-4] INFO org.apache.jena.fuseki.geosparql.GeosparqlServer {} - GeoSPARQL Server: Running - Port: 3030, Dataset: /ds, Loopback Only: false,  Allow Update: false
[fuseki 2022-01-21 16:50:10:841 +0000] [Thread-4] INFO org.apache.jena.fuseki.Server {} - Start Fuseki (http=3030)

I see the updates are set to false which I can confirm with

mc cat nas/issue31/summoned/invemarexpert//test.jsonld | jsonld format -q | curl -X PUT --header "Content-Type:application/n-quads" -d @- http://localhost:3030/ds/data
HTTP method not allowed: PUT : Read-only

So it seems the server is running and I can get the "read-only" warning when trying to load some data, which I expect.

hitting: http://localhost:3030/

results in Error 404: Not Found

I do have config files and approaches that work for me with core Jena 4 (no geosparql) but I am not exactly sure where to mount and leverage my config files.

Do you have some documentation on firing up the container with a mounted volume to allow setting up a read-write database that I could start to load in some geosparql enhanced RDF into?

Again. thanks so much for this. I was not able to get even close to this far with building in geosparql support!
Any help you can provide would be appreciated!