Decomposable Attention MOdel for Novelty (DAMON): A technique to identify redundant documents
This is the codebase used in the writing of the paper called: "Is Your Document Novel? Let Attention Guide You. An Attention-Based Model For Document Level Novelty Detection" which is published in Natural Langauge Engineering (NLE) journal by Cambridge University Press.
- Tensorflow
- Keras
- PyTorch
- Matplotlib
- GLoVe common crawl pre-trained word vectors (http://nlp.stanford.edu/data/glove.840B.300d.zip)
- TAP-DLND-1.0 (https://www.iitp.ac.in/~ai-nlp-ml/resources.html#)
- dlnd: Contains various scripts and the DLND dataset in .zip format
- scripts: Contains various scripts used for obtaining results in the paper
- apwsj: Contains .tar file containing judgements on APWSJ dataset and other .ttxt files derived from it
- scripts: Contains various scripts used for obtaining results in the paper
- sentence_encoder: Contains scripts from Infersent used for generating sentence embeddings