The automated crawling and processing engine for the Delphi platform.

We are currently in pre-alpha state! There is no release and the code in this repository is purely experimental!

branch	status	codacy	snyk
master
develop

Delphi's crawler component is automatically scanning repositories of code for new identifiable artifacts to index. It the pushes these artifacts on its work queue and eventually processes them. During processing metrics for that artifact are collected and stored.

It searches new code artifact from Maven central, downloads necessary files, and pushes them through the Hermes metrics tool. The results will then be indexed into an elasticsearch database.

If you just wish to query the results, maybe the public instance at https://delphi.cs.uni-paderborn.de is the right choice for you.

If you want to create your own index, you can start the crawler directly using sbt in the project folder:

sbt run

It expects a running instance of elasticsearch on port 9200 on the same machine. You can override this default by setting the environment variable DELPHI_ELASTIC_URI to connect to your instance. The URI format is elasticsearch://<host>:<port>.

When using OpenJDK you have to additionally install openjfx as one of our dependencies currently requires JavaFX to be present. For many Linux distributions the two packages necessary are libopenjfx-java and openjfx. For Alpine there are custom packages available here and a prepared docker image available here.

Feel welcome to join our chatroom on Gitter:

Contributions are very welcome!

Before contributing, please read our Code of Conduct.

Refer to the Contribution Guide for details about the workflow. We use Pull Requests to collect contributions. Especially look out for "help wanted" issues , but feel free to work on other issues as well. You can ask for clarification in the issues directly, or use our Gitter chat for a more interactive experience.

The Delphi crawler is open source and available under Apache 2 License.