1-best is the same with 10-best?? about ncrfpp HOT 6 CLOSED

from ncrfpp.

Hi @azraelkuan , the reported value of the n-best is the same as the 1-best, i.e. the acc/p/r/f. They are all the performance of one best. The difference is the decoded text, you will see the decoded text includes 10 label sequences and their probabilities for each sentence.

from ncrfpp.

so, how can i use the decoded text with 10 label to get better acc/p/r/f?
what i do is that only choose the first label of 10 to do the evaluation, that is the same with 1 best..

from ncrfpp.

The n-best cannot give better performance in separate. It only provides more abundant predict results rather than the single one. For example, sometimes the gold label sequence is the second best rather than the one best.

The nbest decoding is normally used in reranking models to provide more available candidates. And the rerank system will learn how to choose the best sequence from the n-best candidate. The oracle results I reported is based on an assumption that the rerank system works perfectly to choose the best sequence from the n-best outputs. It is a standard metric to show the upbound of the rerank system. You may search the papers of reranking system for more details, or you can check our rerank paper: http://www.acl-bg.org/proceedings/2017/RANLP%202017/pdf/RANLP101.pdf.

In addition, the n-best decoding can also be used in other areas such as the search engine, which extracts all the recognized entities from the n-best outputs to give a better recommendation.

from ncrfpp.

thanks!

from ncrfpp.

hi jiesutd! What is the way you calculate the f1 and acc of n-best you reported in your paper?

from ncrfpp.