How to select relation in Attention while Testing? about nre HOT 13 OPEN

While testing, you need to calculate all P(r|s) and just use the query vector with relation r' when calculating P(r'|s).

from nre.

@Mrlyk423
Thanks for your reply. It is quite clear. That is to say, the formula (10)

It is just a stack of each relation's score
o_r = M_r s + d_r
which is calculated separately rather than an extra soft-max layer.
Is that right?

from nre.

Yes

from nre.

@Mrlyk423
Thanks, your reply helps me a lot in understanding the paper.
Best.

from nre.

@Mrlyk423 but in test phase how to calculate alpha(instance attention), and how to choose the y_pred, max(p(r|s))?

from nre.

@Mrlyk423 Sorry for another question. In the test and evaluation code, why do you only calculate the the top 2000 high probability items.

for (int i=0; i<2000; i++)
{
if (aa[i].second.first!=0)
	correct++;	
fprintf(f,"%lf\t%lf\t%lf\t%s\n",correct/(i+1), correct/tot,aa[i].second.second, aa[i].first.c_str());
}

It seems that it cannot cover all the test data?
looking forward to your reply.
Thanks.

from nre.

For relation extraction, we only focus on the top predict results. If you want to get the all predict results, just change 2000 to the number you need.

from nre.

@Mrlyk423 Thanks very much. By the way, do you have other versions of PCNN+ATT such as tensorflow or pytorch? I try to reproduce PCNN+ATT with pytorch but got a quite worse result comparing with your C++ version. But the pytorch version of PCNN+ONE got a similar result as yours and Zeng2015. This issue confused me for a long time, What may be the possible reason?
Thanks.

from nre.

Is vector r in equation (8) obtained from Matrix M mentioned in equation (10)? Or there are two separate trainable parameters for r in eq (8) and M in eq (10)?

from nre.

In my opinion, the vector r in equation (8) shares with Matrix M

from nre.

@Mrlyk423 Thanks very much. By the way, do you have other versions of PCNN+ATT such as tensorflow or pytorch? I try to reproduce PCNN+ATT with pytorch but got a quite worse result comparing with your C++ version. But the pytorch version of PCNN+ONE got a similar result as yours and Zeng2015. This issue confused me for a long time, What may be the possible reason?
Thanks.

I am also facing same issue with pytorch implementation. F1 score is very low when applying attention over PCNN features. Have you been able to fix it? Have you released your pytorch implementation?

from nre.

Yeah, My implementation is in https://github.com/ShomyLiu/pytorch-relation-extraction
The dataset is used in this repo, which is slightly different from the newer version dataset in OpenNRE.

from nre.

For relation extraction, we only focus on the top predict results. If you want to get the all predict results, just change 2000 to the number you need.

Is the Precision-Recall curve in the paper based on to 2000 predicted results?

from nre.