HUMAN is an annotation server that stands for...

• Hierarchical: Supports annotation of hierarchical data. This makes it easy to annotate instances (e.g. online comments) together with their context (e.g. the thread of comments a comment was posted in).
• Universal: Handles both textual data with and without context as well as PDFs and image annotation.
• Modular: Various question types (labeling questions, multiple-choice, yes-no, setting bounding boxes etc.) that are self-contained and can be arranged in any order needed. This also makes it easy to implement new custom question types and features.
• ANnotator: Comes with an easy to use GUI interface for your annotators and project manager.

The only requirement is a working version of python 3.9.x. Using anaconda or https://docs.conda.io/en/latest/miniconda.html for a python environment is highly recommended.

The setup consists of 5 parts:

1. Python environment
2. Database setup
3. Annotation protocol
4. Deploy server
5. Add data

First install the python environment.

With Conda:

conda env create

then activate the environment with

conda activate human

OR install from requirements.txt with pip or whatever you fancy.

pip install -r requirements.txt

First initialize the database with

flask init-db

Then add an admin account with

flask add-admin

Write your custom annotation protocol into protocol.yml. Refer to the wiki for documentation on how to do this and see our example protocols (under /examples) for inspiration.

Whenever you change any state names in protocol.yml, afterwards be sure to run

flask reset-annotations

This will reset the annotations table in the database and is necessary to properly save annotations after a change in the protocol.

To run a server you have 2 possibilities:

Locally you can start a server with

flask run

and visit http://127.0.0.1:5000 in your favorite browser and login with your admin account. However it is very ill advised to use this with an open port in a production environment.

When running HUMAN in a production environment on a server we recommend using gunicorn (included in the environment). First, you should set a secure SECRET_KEY in config.py. The script start_server.sh should take care of starting the server.

For docker refer to the https://github.com/uds-lsv/human/wiki/Docker/.

To add a file with data, start the server, log in with your admin account, and go to "Data Console". There you can upload the file. Be sure that it is a tab separated file with the three columns "content", "context" and "meta".

When you want to display files you can use "Upload Folder" in "Data Console" and then upload a file with <folder-name>/<file-name> in the content fields for every file in the folder.

After the setup you can try these examples.

First always reset the database with flask reset-database

1. Copy /example/protocol_image.yml to /protocol.yml
2. Run flask reset-annotations
3. Log in with your administrator account and go to the upload console
4. Upload folder /example/picture
5. Upload file /example/data_example_picture.csv
6. Go back to home page and start annotating.

1. Copy /example/protocol_text.yml to /protocol.yml
2. Run flask reset-annotations
3. Log in with your administrator account and go to the upload console
4. Upload file /example/data_example_text.csv
5. Go back to home page and start annotating.

To make it easier to install, update and remove node, especially if you plan to maybe use it in other projects we recommend using the node version manager https://github.com/nvm-sh/nvm.

When debugging and working on client side code it is very convenient to let webpack watch file changes and transpile your code automatically: npm run watch

Visit the wiki for full documentation.

https://github.com/RainbowRevenge

https://github.com/ruitedk6

https://github.com/GeetDsa

https://arxiv.org/abs/2010.01080

HUMAN is licensed under GPL-3.