michmech / lemmatization-lists Goto Github PK

Hi,
you mention the Polish language in the readme. But the file for it is not present. Perhaps lost in conversion?
Thanks,
Jan

These two rows in https://github.com/michmech/lemmatization-lists/blob/master/lemmatization-sv.txt constitute a loop. You might want to remove loops from the lists.

"skön sköna"
"sköna skön"

Would it be possible to add more langauges by parsing the wiktionary dumps?

lemmatization-en.txt

Incorrect mapping:
spin-dry <- dry

The correct mapping:
spin-dry <- spin-dry

English dictionary:
In some cases a full-stop is has been included as the end of a word. Those lines should be removed. The incorrect word pairs are:

aggregate aggregated.
all-rounder all-rounders.
behind behinds.
beholder beholders.
coachman coachmen.
cockatoo cockatoos.

conjurer equates with also???

At the end of the Ukrainian lemma list, there is a redundant line which should be removed:
(193703 row(s) affected)

And in the Welsh lemma list, at line 257840, there are no token forms for lemma "n", so the line is useless.

Could you please provide some links to the tutorials or docs on how to create such a list for a new language (for example for a super small endangered language which doesn't have the model yet). Thanks in advance!

In line 12675 of the lemmatization-en.txt file we read:
erasure erasures
and in the next line:
erasures erasure
It seems like the second lemma is not correct.