ekaterinavoloshina / classification_of_dative_semantic_roles Goto Github PK

This reprository contains the code and the data, which are described in the term paper for HSE University «Semantic Role Labeling for Forms with a Dative Marker».

The concept of semantic roles is a model that unites similar participants of different situation on the basis of similarity of their meaning and their morphosyntactic behaviour.

The aim of the term paper is to prove that the classification models can classify semantic roles expressed by the same marker on the example of the Dative case marker in Russian. Our hypothesis is that the predictions of the models will be based on the semantic features, and the confusion matrix will reflect the structure of the category of the Dative case in Russian.

• Recipient (Я дала мальчику книгу)
• Beneficiary (Я купила сыну игрушку)
• Experiencer (Это кажется мне странным)
• External Possessor (Она подстригла мне ногти)
• Counteragent (Сотрудник подчинился начальству)
• Direction (Он поехал к бабушке)

The data was originally taken from pre-processed FrameBank and then manually annotated. The manually annotated dataset is available here.

The features were extracted automatically, see the code for feature extraction. The labeled dataset can be found here. The distribution by class is presented in the table:

Apart from that, FrameBank contains many unlabeled examples. 94864 examples with a Dative marker were taken from FrameBank, the unlabeled dataset can be found here

The features were extracted from RNC morphological and semantic annotation and SyntaxNet syntactic annotation presented in FrameBank.

In this research both supervised and semi-supervised methods are used.

The results of supervised methods models are presented in the table:

For training models based on semi-supervised methods the lemmas were excluded from the dataset. Self-training algorithm was used with two base estimators: Logistic Regression and XGBoost. The comparison of models based on the supervised methods and trained with lemmas and models based on semi-supervised methods and trained without lemmas is illustrated with the table, the metrics is weigthed F1-score:

The results for each semantic role can be found in the following table, the metrics is F1-score:

For further discussion see the handout.

To run the notebook with the preprocessing code, please download this file and clone this repository:

git clone https://github.com/EkaterinaVoloshina/classification_of_dative_semantic_roles

The notebooks with experiments can be opened in Google Colaboratory. Be aware that you need GPU to run the notebook with supervised methods models.