This repository contains the code of our winning solution for the Shared Task on Detecting Signs of Depression from Social Media Text at LT-EDI-ACL2022.

More information can be found in the following paper: OPI@LT-EDI-ACL2022: Detecting Signs of Depression from Social Media Text using RoBERTa Pre-trained Language Models.

If you use the code, models or datasets from this repository, please cite:

@inproceedings{poswiata-perelkiewicz-2022-opi,
    title = "{OPI}@{LT}-{EDI}-{ACL}2022: Detecting Signs of Depression from Social Media Text using {R}o{BERT}a Pre-trained Language Models",
    author = "Po{\'s}wiata, Rafa{\l} and Pere{\l}kiewicz, Micha{\l}",
    booktitle = "Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion",
    month = may,
    year = "2022",
    address = "Dublin, Ireland",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2022.ltedi-1.40",
    doi = "10.18653/v1/2022.ltedi-1.40",
    pages = "276--282",
}

The task was to create a system that, given social media posts in English, should detect the level of depression as ‘not depressed’, ‘moderately depressed’ or ‘severely depressed’.

1. python 3.8+
2. transformers 4.13.0
3. simpletransformers 0.63.7
4. pandas 1.2.5
5. scikit-learn 0.23.1
6. tqdm 4.62.3

We prepared two datasets. The first is a preprocessed dataset provided by the competition organizers. The second, Reddit Depression Corpora, was used to train the DepRoBERTa language model.

Dataset was prepared by removing duplicates and transfer some examples from the dev set to the train set. Files are available in the ./data/preprocessed_dataset folder.

We built a corpus based on the Reddit Mental Health Dataset (Low et al., 2020) and a dataset of 20,000 posts from r/depression and r/SuicideWatch subreddits. We filtered the data appropriately, leaving mainly those related to depression (31,2%), anxiety (20,5%) and suicide (18.1%), which resulted in a corpora consisting of 396,968 posts.

DepRoBERTa (RoBERTa for Depression Detection) - language model based on RoBERTa-large and further pre-trained on the Reddit Depression Corpora.

rafalposwiata/deproberta-large-v1

rafalposwiata/roberta-large-depression

rafalposwiata/deproberta-large-depression