Git Product home page Git Product logo

litef-conductor's Introduction

About

Litef is a semantic indexer and the Data API service for Cendari project.

Building

You need CKAN and Virtuoso systems already installed.

In order to build and run litef-conductor, you need Java 1.7.x, Scala 2.10.4 and SBT.

In order to compile, you can use the sbt compile command, for starting the service, execute sbt reStart.

Environment requisites

If you have plans to use a non local CKAN/PostgreSQL and Virtuoso, you need to forward the necessary ports to your local system through an SSH tunnel.

You should copy the PROJECTDIR/src/main/resources/application.conf-example to PROJECTDIR/application.conf and adjust it accordingly. Do not commit the config file to the respository and do not package it in the assembly jar file.

When starting the application with reStart, it will start the server using that configuration file.

If you start the server manually from the assembly jar file, you need to pass the -Dconfig.file=PROJECTDIR/application.conf argument manually.

Data location

Indexer needs to be able to access CKAN files directly (litef.ckan.localStoragePrefix), so you need to have a local ckan installation, or to use sshfs. Also, it needs to be able to create and write into the directory specified in litef.indexer.localStoragePrefix.

You need to add the litef.indexer.localStoragePrefix path to the DirsAllowed variable in virtuoso.ini

In order not to have collisions between the main server and a local instance (in the case where you use the main server's databases and files), you need to create a local file /opt/litef/conductor:disable-document-processing before starting the local service.

Packaging

For the production environments, the sbt assembly command is provided. It builds a single jar file that contains all the libraries and litef.

For quicker deployment, but more involved maintainance, you can use the sbt package command and maintain the libraries manually. If you need to collect all the libraries, use sbt oneJar and extract the library jar files to the runtime classpath.

litef-conductor's People

Contributors

ivan-cukic avatar mknezevic avatar schildwaechter avatar

Watchers

 avatar  avatar  avatar  avatar

Forkers

cendari

litef-conductor's Issues

Fix encoding of NERD Uris

Check where encoding of files (and if) becomes broken.
On download it JSON file shows ISO-8859 character set, and it is further treated as such.
This produces very strange URIs and values afterwards in NERD.

https://int2.cendari.dariah.eu/ckan/dataset/9200316-bibliographicresource_3000092750591/resource/4a24a665-1bed-4a8c-bd97-77b67d8ac351

the RDF version of this resource is in UTF encoding, see
https://int2.cendari.dariah.eu/ckan/dataset/9200316-bibliographicresource_3000092750591/resource/f22c70aa-c640-4773-884d-076ac2f536c4

on download UTF-8 file, somewhere during processing (i could track to NERD2RDF processing) the encoding goes into ascii

nbulatovic@int2:/var/lib/litef/default/resources/f22/c70/aa-c640-4773-884d-076ac2f536c4$ file application:rdf+xml
application:rdf+xml: UTF-8 Unicode text, with very long lines

nbulatovic@int2:/var/lib/litef/default/resources/f22/c70/aa-c640-4773-884d-076ac2f536c4$ file application:x-nerd-output
application:x-nerd-output: UTF-8 Unicode text, with very long lines, with no line terminators

nbulatovic@int2:/var/lib/litef/default/resources/f22/c70/aa-c640-4773-884d-076ac2f536c4$ file "application:x-nerd-output -> application:rdf+xml"
application:x-nerd-output -> application:rdf+xml: ASCII text

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.