Node.js implementation to load compendia from third party repositories and handle direct user uploads for the o2r web api.

Currently, it implements the endpoint /api/v1/compendium.

• Sciebo (https://sciebo.de)
• Zenodo or Zenodo Sandbox (https://zenodo.org or https://sandbox.zenodo.org)

• Node.js >=8
• bagit-python (bagit.py)
• Docker socket access for running o2r-meta
• unzip
• wget

The configuration can be done via environment variables.

• LOADER_PORT Define on which port loader should listen. Defaults to 8088.
• LOADER_MONGODB Required Location for the mongo db. Defaults to mongodb://localhost/. You will very likely need to change this.
• LOADER_MONGODB_DATABASE Which database inside the mongo db should be used. Defaults to muncher.
• LOADER_BASEPATH The local path where compendia are stored. Defaults to /tmp/o2r/.
• LOADER_META_TOOL_CONTAINER Docker image name and tag for metadata tools, defaults to running latest o2r-meta in a container, i.e. o2rproject/o2r-meta:latest.
• SESSION_SECRET String used to sign the session ID cookie, must match other microservices.
• SLACK_BOT_TOKEN Authentication token for a bot app on Slack. See section Slack bot.
• SLACK_VERIFICATION_TOKEN Token provided by Slack for interative messages and events, to be used to verify that requests are actually coming from Slack.
• SLACK_CHANNEL_STATUS Channel to post status messages to, defaults to #monitoring.
• SLACK_CHANNEL_LOAD Channel to post messages related to (up)loading to, defaults to #monitoring.

Documentation of Slack API: https://api.slack.com/bot-users, especially interactive messages.

The bot needs the permissions to join channels and post to them. Add the following scopes to the app in the section "OAuth & Permissions" in the bot's apps page.

• channels:write
• chat:write:bot
• bot

While adding the app to your Slack organisation, make sure to allow the bot to post the the desired channel.

Start ngrok with ngrok http 8088 and enter the public endpoint pointing to your local server at https://api.slack.com/apps/A6J6CDLQK/interactive-messages. ngrok also has a useful web interface at http://127.0.0.1:4040/inspect/http on all incoming requests.

The upload process may fail if certain files with unsupported encoding are detected:

The encoding of text files analyzed by the o2r metadata extraction tool o2r-meta must be Unicode (UTF-8, UTF-16BE, ...) or Unicode compatible (e.g. ISO-8859-1). The supported encodings and the list of files checked can be configured in config.js.

mkdir /tmp/o2r-mongodb-data
mongod --dbpath /tmp/o2r-mongodb-data
# new terminal: start bouncer (default port 8081)
cd ../o2r-bouncer
DEBUG=* OAUTH_CLIENT_ID=<...> OAUTH_CLIENT_SECRET=<...> npm start

Then start the loader in your IDE or in a new terminal.

Tests can be started using mocha:

npm install
npm install -g mocha
mocha

# alternative:
npm test

# set test endpoint manually
TEST_HOST=http://localhost:80 npm test

For this, the loader has to run locally or as part of a docker-compose configuration.

The file Dockerfile describes the Docker image published at Docker Hub.

docker build --tag loader .

docker run --name mongodb -d mongo:3.4

docker run --name testloader -it -p 8888:8088 -v /var/run/docker.sock:/var/run/docker.sock --link mongodb:mongodb -e LOADER_MONGODB=mongodb://mongodb:27017 -e DEBUG=* loader

o2r-loader is licensed under Apache License, Version 2.0, see file LICENSE.

Copyright (C) 2017 - o2r project.