An ML API to compute the Jaccard similarity based on shingled subtrees of the dependency grammar. The API is programmed with the fastapi Python package. Dependency trees are extracted with a spacy model trained on HDT. The similarity scores are computed with the packages datasketch and treesimi. The deployment is configured for Docker Compose.

Call Docker Compose

export API_PORT=8084
docker-compose -f docker-compose.yml up --build

# or as oneliner:
API_PORT=8084 docker-compose -f docker-compose.yml up --build

(Start docker daemon before, e.g. open /Applications/Docker.app on MacOS).

Check

curl http://localhost:8084

Notes: Only main.py is used in Dockerfile.

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt --no-cache-dir
pip install -r requirements-dev.txt --no-cache-dir

(If your git repo is stored in a folder with whitespaces, then don't use the subfolder .venv. Use an absolute path without whitespaces.)

source .venv/bin/activate

source .venv/bin/activate
# uvicorn app.main:app --reload
gunicorn app.main:app --reload --bind=0.0.0.0:8084 \
    --worker-class=uvicorn.workers.UvicornH11Worker \
    --workers=1 --timeout=600

The following example should yield a high similarity score because both sentences exhibit an identical syntactic structure:

curl -X POST "http://localhost:8084/similarities/" \
    -H "accept: application/json" \
    -H "Content-Type: application/json" \
    -d '["Die Katze miaut.", "Der Hund bellte laut."]'

The example below should yield a lower similarity score because the two sentences differ in their syntactic structure:

curl -X POST "http://localhost:8084/similarities/" \
    -H "accept: application/json" \
    -H "Content-Type: application/json" \
    -d '["Leise rieselt der Schnee.", "Der Schnee rieselt leise."]'

The example below should yield a low similarity score because the two sentences differ a lot with regard to their syntactic structure.

curl -X POST "http://localhost:8084/similarities/"  \
   -H "accept: application/json"  \
   -H "Content-Type: application/json"   \
   -d '["Der Schneemann ist groß.", "Die Kinder spielen im Schnee."]'

• Check syntax: flake8 --ignore=F401 --exclude=$(grep -v '^#' .gitignore | xargs | sed -e 's/ /,/g')
• Run Unit Tests: PYTHONPATH=. pytest

• Show the docs: http://localhost:8084/docs
• Show Redoc: http://localhost:8084/redoc

find . -type f -name "*.pyc" | xargs rm
find . -type d -name "__pycache__" | xargs rm -r
rm -r .pytest_cache
rm -r .venv

@software{ulf_a_hamster_2022_7095998,
  author       = {Ulf A. Hamster and
                  Luise Köhler},
  title        = {{simiscore-syntax: ML API for syntactic 
                   similarities}},
  month        = sep,
  year         = 2022,
  publisher    = {Zenodo},
  version      = {0.1.0},
  doi          = {10.5281/zenodo.7095998},
  url          = {https://doi.org/10.5281/zenodo.7095998}
}

• Sebastián Ramírez, 2018, FastAPI, https://github.com/tiangolo/fastapi
• Ines Montani, Matthew Honnibal, Matthew Honnibal, Sofie Van Landeghem, Adriane Boyd, Henning Peters, Paul O'Leary McCann, Maxim Samsonov, Jim Geovedi, Jim O'Regan, Duygu Altinok, György Orosz, Søren Lind Kristiansen, Daniël de Kok, Lj Miranda, Roman, Explosion Bot, Leander Fiedler, Grégory Howard, … Björn Böing. (2022). explosion/spaCy: New Span Ruler component, JSON (de)serialization of Doc, span analyzer and more (v3.3.1). Zenodo. https://doi.org/10.5281/zenodo.6621076
• Rene Knaebel. (2022). reneknaebel/de_dep_hdt_dist (v0.1.0). Huggingface. https://huggingface.co/reneknaebel/de_dep_hdt_dist
• Eric Zhu, Vadim Markovtsev, aastafiev, Wojciech Łukasiewicz, ae-foster, Sinusoidal36, Ekevoo, Kevin Mann, Keyur Joshi, Peter Kubov, Qin TianHuan, Spandan Thakur, Stefano Ortolani, Titusz, Vojtech Letal, Zac Bentley, fpug, & oisincar. (2021). ekzhu/datasketch: v1.5.4 (v1.5.4). Zenodo. https://doi.org/10.5281/zenodo.5758425
• Ulf Hamster, & Luise Köhler. (2022). treesimi: Shingling for measuring tree similarity (0.1.6). Zenodo. https://doi.org/10.5281/zenodo.6501989

Please open an issue for support.

Please contribute using Github Flow. Create a branch, add commits, and open a pull request.

The "Evidence" project was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 433249742 (GU 798/27-1; GE 1119/11-1).

• till 31.Aug.2023 (v0.1.0) the code repository was maintained within the DFG project 433249742
• since 01.Sep.2023 (v0.2.0) the code repository is maintained by Ulf Hamster.