Benchmarks for LaBSE about multilingual_similarity_compare HOT 4 OPEN

Hey @loretoparisi,

I have been off on vacation for a while. Will look into it as soon as I have a bit of time. Thanks for the suggestion! 😄

from multilingual_similarity_compare.

Hi @loretoparisi ,

Sorry for the late answer, it would be great to see the results on LaBSE. Unfortunately I do not have much capacity lately. Did you take a close look at it yourself? It should be fairly simple to replicate the propose work in this repo if LaBSE has been integrated to pytorch. Happy to connect on that. 😃

from multilingual_similarity_compare.

Thanks a lot, that would be very interesting also because recently a official blog post came out, pointing out a comparison among cross-lingual models (tatoeba dataset)

Model 14 Langs 36 Langs 82 Langs All Langs
m~USE* 93.9 — — —
LASER 95.3 84.4 75.9 65.5
LaBSE 95.3 95.0 87.3 83.7

The interesting part is about the support to "unsupported" languages

"...For one third of these languages the LaBSE accuracy is higher than 75% and only 8 have accuracy lower than 25%, indicating very strong transfer performance to languages without training data"

and - my opinion - minor/low resources languages.

If I can help, let me know.

Thank you!

from multilingual_similarity_compare.

@MastafaF thank you anyways, I will have a look.

from multilingual_similarity_compare.