skroutz / greek_stemmer Goto Github PK

View Code? Open in Web Editor NEW

54.0 54.0 12.0 154 KB

A simple Greek stemming library

License: MIT License

Ruby 100.00%

greek_stemmer's People

Contributors

Stargazers

Watchers

Forkers

zepolen pavlos256 dourvaris lazlyhu ptheof isaak4 dimitris-ap stjordanis dourosdimitris admariner aalepis iq-scm

greek_stemmer's Issues

"κολάν" has the same stem as "κολικός"

"μουσαμάδες" has wrong stem

GreekStemmer.stem("ΜΟΥΣΑΜΑΔΕΣ") # => "ΜΟΥΣΑΜΑΔ"

GreekStemmer.stem("ΜΟΥΣΑΜΑΣ") # => "ΜΟΥΣΑΜ"

"φανελακι" does not have the same stem as "φανελα"

"πληκτρολογιο" does not have the same stem as "πληκτρολογια"

"μπουφαν" has the same stem as "μπουφες"

ΑΔ-ΗΣ # ΑΔ-Ω

Probably, there is a problem here: giving the words ΑΔΩ and ΑΔΗΣ, the stemming process has as output the stem ΑΔ?

Encoding problems

Ειχαμε ενα προβλημα με τις κωδικοποιησεις:

Traceback (most recent call last):
File "E:/Python/classification/scripts/nsk_fit.py", line 31, in
stemmer = gr_stemm.GreekStemmer()
File "χ\Anaconda3\lib\site-packages\greek_stemmer_init_.py", line 35, in init
custom_rules = self.load_settings()
File "χ\Anaconda3\lib\site-packages\greek_stemmer_init_.py", line 340, in load_settings
custom_rules = yaml.load(f.read())
File "χ\Anaconda3\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 72: character maps to

Το προβλήμα λυνεται ως εξης ( στη περιπτωση μας):
γραμμη 339, file: init.py:
os.path.dirname(file), 'stemmer.yml'), 'r', encoding="utf8") as f:

Μονόφωτα vs Μονόφωτο

irb(main):023:0> "μονόφωτο".stem
=> "ΜΟΝΟΦΩΤ"
irb(main):024:0> "μονόφωτα".stem
=> "ΜΟΝΟΦΩ"

"*ινος" words have bad stems

GreekStemmer.stem("ΒΡΑΔΙΝΟΣ") # => "ΒΡΑΔΙΝ" instead of "ΒΡΑΔ"