Command-line client for Aleph. It can be used to bulk import document sets via the API, without direct access to the server. It requires an active API client to perform uploads.

Install using pip.

pip install alephclient

alephclient needs the url of an Aleph API instance and an API key for the said API. These can be provided by setting the environment variables ALEPH_HOST and ALEPH_API_KEY respectively; or by passing them in with --api-host and --api-key options.

The crawldir command crawls through a given directory recursively and uploads all the files and directories inside it to a collection. The foreign id of the collection needs to be passed to the command with --foreign-id option. The language used in the directory can optionally be specified with the --language option, which expects a 2-letter ISO 639 language code. It can be specified multiple times, for when the directory contains files in more than one language. The path argument needs to be a valid path to a directory

Example:

alephclient --api-base-url http://127.0.0.1:5000/api/2/ --api-key 2c0ae66024f0403bb751207e54c5eb5d crawldir --foreign-id wikileaks-cable --category leak /Users/sunu/data/cable

Load JSON-formatted entities formatted in the followthemoney structure into an aleph collection. This can be used in conjunction with the command-line tools for generating such data provided by followthemoney-util. Data that is loaded this way should be aggregated as much as possible, for example using the ftm aggregate command-line utility, or the balkhash database layer.

A typical use might look this:

ftm map my_mapping.yml | ftm aggregate | alephclient write-entities -f my_dataset

The inverse of write-entities, this will stream entities from the given aleph instance so that they can be written to a file.

Here's how you'd stream an aleph collection:

alephclient stream-entities -f my_dataset >my_dataset.json

The bulkload command executes an entity mapping in the system. Its only argument is a YAML mapping file.