YAKE implementation error! about pke HOT 2 CLOSED

Hi @vitordouzi,

Sorry for the very late response, I finally have time to spend on checking YAKE implementation.

About (1), every model in pke uses the same default stopword list (the one provided by nltk) which can be changed using the stoplist parameter in candidate_selection(). My position here is that I do not want to have separate/different stopword lists for each model as this, as you know, greatly impacts performance.

About (2) and (3), I mainly relied on the model description in the ECIR 2018 paper to make this re-implementation and I may have missed some points. I will change my code according to the one you provided.

from pke.

So, these are the differences I found between ours and your implementations:

1. In candidate_selection(), you added a filter to remove candidates that contain a first/last word with less than 3 characters.

2. In context_building(), you changed how contexts are computed from the whole sentence to blocks. Blocks are sequences of in vocabulary words.

3. In _feature_extraction(), you changed how WRel is computed by removing the addition of PL and RL. This is different from the equation 4 in the ECIR 2018 paper.

4. In candidate_weighting(), you change how word weights are computed for stopwords. Here words that are less than 3 characters are also considered as stopwords.

from pke.