Uncertain Knowledge Graphs Embedding is implemented by means of BERT pretrained natural language model. Exploring approximate inference on the embedding space. Considering the commonsense reasoning based on these works.
- python 3.7
- pytorch 1.6.0+cpu
- transformers 4.0.0
- pretrained BERT models, such as 'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz", etc.
In data directory, train.tsv is the training data, test.tsv the testing data and val.tsv the validation data.
| dataset | entities | relations | facts | average of confidence | standard deviation | ratio of facts/relations |
|---|---|---|---|---|---|---|
| CN15k | 15,000 | 36 | 234,675 | 0.627 | 0.234 | 15.6 |
| PPI5k | 4,999 | 7 | 271,666 | 0.415 | 0.213 | 54.3 |
The unit of all values on MSE and MAE is 10$^{-2}$. "-" means the corresponding data did not report in literatures.
| dataset | CN15k | PPI5k | ||||
|---|---|---|---|---|---|---|
| metrics | MSE | MAE | epochs | MSE | MAE | epochs |
| URGE | 10.32 | 22.72 | - | 1.44 | 6.00 | - |
| UKGE$_{rect}$ | 8.61 | 19.90 | >100 | 0.95 | 3.79 | >100 |
| UKGE$_{logi}$ | 9.86 | 20.74 | >100 | 0.96 | 4.07 | >100 |
| UKG$_s$E | 7.71 | 21.34 | 19 | 0.98 | 5.98 | 80 |
| UKGE$_{bert}$ | 6.61 | 19.3 | 2 | 0.85 | 5.38 | 75 |
python.exe ./prediction.py
Using Theano backend.
Model: "bert_fine_tuning_UKGE"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
lstm_1 (LSTM) (None, 768) 4721664
_________________________________________________________________
dense_1 (Dense) (None, 1) 769
=================================================================
Total params: 4,722,433
Trainable params: 4,722,433
Non-trainable params: 0
_________________________________________________________________
...
Prediction for (HEAD, ?relation?, TAIL) RANKING:
Head = ' car '
Tail = ' people '
relation candidate 1 = ('hasa', 0.8294004)
relation candidate 2 = ('usedfor', 0.760266) *
relation candidate 3 = ('createdby', 0.65154946)
relation candidate 4 = ('definedas', 0.63723207)
relation candidate 5 = ('atlocation', 0.6002464)
The project is under working. We are very sorry for any trouble to you. You are welcome to put forward your valuable opinions by comments.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent