Git Product home page Git Product logo

translate's Introduction

translatepy (originally: translate)

An aggregation of multiple translation API

Translate, transliterate, get the language of texts in no time with the help of multiple APIs!

PyPI version Downloads PyPI - Downloads PyPI - Python Version PyPI - Status GitHub - License GitHub top language CodeQL Checks Badge Pytest Code Size Repo Size Issues

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

You will need Python 3 to use this module

# vermin output
Minimum required versions: 3.2
Incompatible versions:     2

According to Vermin (--backport typing), Python 3.2 is needed for the backport of typing but some may say that it is available for python versions higher than 3.0

Always check if your Python version works with translatepy before using it in production

Installing

Option 1: From PyPI

pip install --upgrade translatepy

Option 2: From Git

pip install --upgrade git+https://github.com/Animenosekai/translate

You can check if you successfully installed it by printing out its version:

$ translatepy --version
# output:
translatepy v2.3

or just:

$ python -c "import translatepy; print(translatepy.__version__)"
# output:
translatepy v2.3

List of Built-in Services

... but plugins can be made and/or used. More on that in the plugins section.

All of the names belong to their respective rightholders.

Usage

Command line interface mode

Interactive Shell (REPL)

$ translatepy shell
## Choose the action
[?] What do you want to do?: Translate
 > Translate
   Transliterate
   Spellcheck
   Language
   Example
   Quit

## Choose the language to translate in (this step can be skipped by passing the `--dest-lang` argument when starting the program)
In what language do you want to translate in?
[?] (translatepy ~ Select Lang.) > : ...

## Translate
Enter '.quit' to stop translating
(translatepy ~ Translate) > ... # type in whatever you want to translate

In other applications/from the terminal

Select an action: {translate,transliterate,language,spellcheck}

and pass it as a command with the right arguments:

$ translatepy translate --dest-lang Français --text Hello
{
    "success": true,
    "service": "Google",
    "source": "Hello",
    "sourceLanguage": "eng",
    "destinationLanguage": "fra",
    "result": "Bonjour"
}

In Python script

The Translator Class

The translator lets you group and use multiple translators at the same time, to increase your chance on getting an answer.

It takes two optional arguments: the services_list argument, which is a list of Translator objects and the second one being the request argument which is the object which will be used to make requests.

It has all of the supported methods.

  • translate: To translate things
  • translate_html : To translate HTML snippets
  • transliterate: To transliterate things
  • spellcheck: To check the spelling of a text
  • language: To get the language of a text
  • example: To get a list of examples of a word
  • dictionary: To get a list of translations categorized into "featured" and "less common" by DeepL and Linguee
  • text_to_speech: To get an audio file containing the speech version of the given text

When something goes wrong or nothing got found, an exception will be raised. (this is in bold because it is one of the difference that comes with v2)

>>> from translatepy import Translator
>>> translator = Translator()
>>> translator.translate("Hello", "French")
TranslationResult(service=Yandex, source=Hello, source_language=auto, destination_language=French, result=Bonjour)
>>> translator.language("こんにちは")
LanguageResult(service=Yandex, source=こんにちは, result=Language(jpn))

Translators

You can use each translators separately by using them the same way as you would with translatepy.Translator (or translatepy.Translate)

>>> from translatepy.translators.google import GoogleTranslate
>>> gtranslate = GoogleTranslate()
>>> gtranslate.translate("Hello World", "Japanese")
TranslationResult(service=Google, source=Hello World, source_language=eng, destination_language=jpn, result=こんにちは世界)

And some translators have their own parameters:

>>> gtranslate_china = GoogleTranslate(service_url="translate.google.cn")
>>> gtranslate_china.translate("Hello World", "Japanese")
TranslationResult(service=Google, source=Hello World, source_language=eng, destination_language=jpn, result=こんにちは世界)

# it can even be used by translatepy.Translator
>>> from translatepy import Translator
>>> t = Translator([gtranslate_china])
>>> t.translate("Hello World", "Japanese")
TranslationResult(service=Google, source=Hello World, source_language=eng, destination_language=jpn, result=こんにちは世界)

The Language Class

The language class contains lots of information about a language.

You need to pass the language name or code to the class initialization:

>>> from translatepy import Language
>>> Language("French")
# Returns a Language class with the "fra" language
>>> Language("en")
# Returns a Language class with the "eng" language
>>> Language("eng")
# Returns a Language class with the "eng" language
>>> Language("日本語")
# Returns a Language class with the "jpn" language

The Language Class contains both the ISO 639-1 Alpha-2 language code and the ISO 639-2 Alpha-3 language code.

>>> Language("English").alpha2 # ISO 639-1 (alpha 2), nullable
'en'
>>> Language("English").alpha3 # ISO 639-3 (alpha 3)
'eng'
>>> Language("English").alpha3b # ISO 639-2B, nullable
'eng'
>>> Language("English").alpha3t # ISO 639-2T, nullable
'eng'

Each available language has its own ID, coming from the Alpha-3 Language Code most of the times (but which is also unique for languages such as the "Automatic" Language and the "Emoji" one)

>>> Language("French").id
'fra'
>>> Language("Emoji").id
'emj'
>>> Language("Automatic").id
'auto'

It also contains the language name for a lot of languages:

>>> Language("Français").in_foreign_languages.get("ja", None) # an alpha-2 code needs to be passed in, also make sure to have a fallback such as None here because not all of the languages had been translated.
'フランス語'

All of the languages which have an alpha2 code are assured to have at least their translation in all of the following languages:

to = ['af', 'am', 'ar', 'az', 'be', 'bg', 'bn', 'bs', 'ca', 'ceb', 'co', 'cs', 'cy', 'da', 'de', 'el', 'eo', 'es', 'et', 'eu', 'fa', 'fi', 'fr', 'fy', 'ga', 'gd', 'gl', 'gu', 'ha', 'haw', 'hi', 'hmn', 'hr', 'ht', 'hu', 'hy', 'id', 'ig', 'is', 'it', 'he', 'ja', 'jv', 'ka', 'kk', 'km', 'kn', 'ko', 'ku', 'ky', 'la', 'lb', 'lo', 'lt', 'lv', 'mg', 'mi', 'mk', 'ml', 'mn', 'mr', 'ms', 'mt', 'my', 'ne', 'nl', 'no', 'ny', 'or', 'pa', 'pl', 'ps', 'pt', 'ro', 'ru', 'sd', 'si', 'sk', 'sl', 'sm', 'sn', 'so', 'sq', 'sr', 'st', 'su', 'sv', 'sw', 'ta', 'te', 'tg', 'th', 'tl', 'tr', 'ug', 'uk', 'ur', 'uz', 'vi', 'xh', 'yi', 'yo', 'zh', 'zu']

The other ones may or may not have a translation in more or less languages.

The Language class also contains the "similarity" attribute which gives back a number between 0 and 100 which shows the similarity of the input language with what it found in the language code database:

>>> round(Language("French").similarity, 2)
100.0
>>> Language("Englesh").similarity
94.86832980505137
Note

Only the languages which have an alpha2 language code and are of type Living or Ancient are vectorized and will be used in the similarity search.


Each language also have 'extra' data: their type (nullable) and the scope (nullable).

>>> Language("French").extra
LanguageExtra(type=LanguageType(Living), scope=LanguageScope(Individual))
>>> Language("Latin").extra.type
LanguageType(Ancient)

A translatepy.exceptions.UnknownLanguage exception is raised if the given language is unknown.

This exception contains the most similar language along with its similarity:

>>> from translatepy import Language
>>> from translatepy.exceptions import UnknownLanguage
>>> try:
...     language = Language("**")
... except UnknownLanguage as error:
...     print("The similarity seemed to be too low for translatepy to accept it as a correct language name")
...     print("The language found is:", error.guessed_language)
...     print("Its similarity from the passed input is:", str(error.similarity))

If you find that the default threshold given to the language search is too low, you can always change it by passing the threshold parameter when initializing a Language:

>>> from translatepy import Language
>>> Language("国語")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/animenosekai/Documents/Coding/Projects/translate/translatepy/language.py", line 106, in __init__
    raise UnknownLanguage(_search_result, self.similarity, raising_message)
translatepy.exceptions.UnknownLanguage: Couldn't recognize the given language (**)
Did you mean: **語 (Similarity: 81.65%)?
>>> Language("**", threshold=80)
Language(zho)

Results

All of the methods should have its own result class (defined in translatepy/models.py) which all have at least the service, source, result attributes and a "as_json" method to convert everything into a JSON String.

Errors

All of the translatepy errors are inherited from translatepy.exceptions.TranslatepyException so that you can easily catch a translatepy error.

>>> from translatepy import Translator
>>> from translatepy.exceptions import TranslatepyException, UnknownLanguage
>>> t = Translator()
>>> def translate(text, dest):
...     try:
...         result = t.translate(text, destination_language=dest)
        except UnknownLanguage as err:
            print("An error occured while searching for the language you passed in")
            print("Similarity:", round(err.similarity), "%")
            return
        except TranslatepyException:
            print("An error occured while translating with translatepy")
            return
        except Exception:
            print("An unknown error occured")
            return
...     # do something with the result...
...     

Plugins

You can make your own Translator using the translatepy.translators.base.BaseTranslator class.

Make sure that you inherit from this class when creating your translator and to follow the instruction from plugin.md

Caching

All of the operations are cached to provide the best performances

You can empty the cache by calling the method "clean_cache"

Deployment

This module is currently in development and might contain bugs.

Feel free to use it in production if you feel like it is suitable for your production even if you may encounter issues.

Contributing

Pull requests are welcome. For major changes, please open an discussion first to discuss what you would like to change.

Please make sure to update the tests as appropriate.

Built With

Authors

Disclaimer

Please do not use this module in a commercial manner. Pay a proper API Key from one of the services to do so.

License

This project is licensed under the GNU Affero General Public License v3.0 License - see the LICENSE file for details

Dataset

The 'playground' folder contains a lot of our search and results for the language management on translatepy (this folder might be very messy because of all of our experiments in it)

The translatepy/utils/_language_cache.py file contains all of the data for the language searching used by translatepy

Please ask us if you want to use them in another project.

Most of the language data come from Google Translate, Yandex Translate and iso-639-3

Acknowledgments

  • Thanks to @spamz23 (Diogo Silva) for the development of the code refactoring used in v2 (tests and Translator) (check: spamz23/translate)
  • Thanks to @ZhymabekRoman (Zhymabek Roman) for working on making Yandex more stable and on the v2!
  • Inspired by py-googletrans (by @ssut) (especially the thread: Issue #268)

translate's People

Contributors

animenosekai avatar bitplus avatar bluemods avatar eggplants avatar hiteshbedre avatar saatvik-droid avatar zhymabekroman avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

translate's Issues

ServiceConnectionError Exception

Hi guys!

First of all, awesome project, many thanks for it!

Now to my issue:
I can not figure out a proper way of checking whether service_url is reachable by translatepy or not (some kind of ServiceConnectionError Exception).

If i just use this code when I'm offline:

from translatepy.translators.google import GoogleTranslate

server_url = 'translate.google.com'

translator = GoogleTranslate(service_url=server_url)
translator.translate('Текст 15', 'en').result

I get error:

translatepy\translators\base.py", line 110, in translate
    source_language, translation = self._translate(text, dest_code, source_code)
TypeError: cannot unpack non-iterable NoneType object

Which is not a clear reason of the translation failure.

I tried to check it myself prior to translation using requests:

import requests
from translatepy.translators.google import GoogleTranslate


def is_online(url):
    try:
        response = requests.head(url)
    except Exception as e:
        return f'{url} not reachable: {str(e)}'
    else:
        stat_code = response.status_code
        if stat_code == 200:
            return True
        else:
            return f'{url} not online. HTTP response code: {stat_code}'


server_url = 'translate.google.com'

print(is_online(f'https://{server_url}'))

translator = GoogleTranslate(service_url=server_url)
translator.clean_cache()
translator.translate('Текст 16', 'en').result

But it confuses me even more, as sometimes it gives opposite results.
Like, url is not reachable but translation is ok (or vice versa):

https://translate.google.com not reachable: HTTPSConnectionPool(host='translate.google.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x000001D488901930>, 'Connection to translate.google.com timed out. (connect timeout=None)'))
'Text 16'

This happens when I'm turning on/off VPN instead of shutting down the network.

What I need is a clear error that service's url could not be reached by translatepy.
Any suggestions on this issue?

Thanks in advance!

Bing Translator throwing error code 205

Made a simple script to reproduce the error using 2 different IP addresses but same country.

from translatepy.translators.bing import BingTranslate

Bing = BingTranslate()
Bing_result = Bing.translate(text="江戸時代初期に建てられた天守や櫓等の主要建築物が現存し、国宝や重要文化財に指定されている。", destination_language="en", source_language="ja").result

print(Bing_result)

Traceback log:

C:\Users\CoolBeans>"C:\Users\CoolBeans\Desktop\bing test.py"
Traceback (most recent call last):
  File "C:\Users\CoolBeans\Desktop\bing test.py", line 4, in <module>
    Bing_result = Bing.translate(text="江戸時代初期に建てられた天守や櫓等の主要建築物が現存し、 国宝や重要文化財に指定されている。", destination_language="en", source_language="ja").result
  File "C:\Users\CoolBeans\AppData\Local\Programs\Python\Python39\lib\site-packages\translatepy\translators\base.py", line 110, in translate
    source_language, translation = self._translate(text, dest_code, source_code)
  File "C:\Users\CoolBeans\AppData\Local\Programs\Python\Python39\lib\site-packages\translatepy\translators\bing.py", line 161, in _translate
    response = self.session_manager.send("https://www.bing.com/ttranslatev3", data={'text': text, 'fromLang': source_language, 'to': destination_language})
  File "C:\Users\CoolBeans\AppData\Local\Programs\Python\Python39\lib\site-packages\translatepy\translators\bing.py", line 140, in send
    raise BingTranslateException(status_code)
translatepy.translators.bing.BingTranslateException: 205 | Unknown error. Error code: 205

Perhaps something changed on their ends? It was fine when I last used Bing translators on 8th January 2023.
Many thanks for maintaining this awesome lib.

[translate_html] Thread pool with a different proxy ip in each thread?

I have a doubt related to \translatepy\translate.py in

with ThreadPool(int(threads_limit)) as pool:
            pool.map(_translate, nodes)

I just want to translate with deepL. Also, I know this method will throw by default 100 request at the same time. And I think, if I send 2,3 or 4 request in a little time to deepL it will launch a throttle and it will block my ip.

Did you find a way to execute multiple request to deepL without being blocked with this method?

Would be a good idea if translate_html execute each thread pool with a different proxy ip in each thread?

Thank you!

SSL verification warning when using Reverso dictionary

When using ReversoTranslate service dictionary() function, a SSL warning from urllib3 is thrown about making a nonverified HTTPS request to context.reverso.net:

Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from translatepy.translators.reverso import ReversoTranslate
>>> t = ReversoTranslate()
>>> t.dictionary("casa", "eng")
C:\Users\xxxx\AppData\Local\Programs\Python\Python39\lib\site-packages\urllib3\connectionpool.py:1013: InsecureRequestWarning: Unverified HTTPS request is being made to host 'context.reverso.net'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  warnings.warn(
DictionaryResult(service=Reverso, source=casa, source_language=spa, destination_language=eng, result=['house', 'home', 'place', 'cottage', 'household', 'villa', 'apartment', 'family', 'residence', 'flat', 'company', 'pad', 'homestead', 'firm', 'hearth', 'habitation', 'fireside', 'married', 'marries', 'gaff', 'Casa'])

When using the translate() function, this is not thrown:

Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from translatepy.translators.reverso import ReversoTranslate
>>> t = ReversoTranslate()
>>> t.translate("casa", "eng")
TranslationResult(service=Reverso, source=casa, source_language=spa, destination_language=eng, result=house)

Checking the urllib3 docs it seems that the certificate verification is now enabled by default starting from version 1.25. A worlaround would be to manually disable it:

import urllib3
urllib3.disable_warnings()

But maybe there is a nicer way of managing this.

There seems to be an issue with the Yandex translation module

I acquired a key from Yandex per the documentation, and added it to the self._id in yandex.py. I then tried the following test:

YandexTranslate().translate("Er ist klug.", 'en', source_language='de')

And received this response:

[2021-02-10 15:14:10] DEBUG [urllib3.connectionpool._new_conn:815] Starting new HTTPS connection (1): translate.yandex.com:443
[2021-02-10 15:14:11] DEBUG [urllib3.connectionpool._make_request:396] https://translate.yandex.com:443 "GET / HTTP/1.1" 302 None
[2021-02-10 15:14:11] DEBUG [urllib3.connectionpool._make_request:396] https://translate.yandex.com:443 "GET /showcaptcha?cc=1&retpath=https%3A//translate.yandex.com/%3F_d8b298bcbb08a8bc220250756257504f&t=0/1612995251/ed1baa2e6a042ec73b1baca9e7b53471&s=d8a68d97e1858e6fff641ad5605ad257 HTTP/1.1" 200 6231
[2021-02-10 15:14:11] DEBUG [urllib3.connectionpool._new_conn:815] Starting new HTTPS connection (1): translate.yandex.net:443
[2021-02-10 15:14:12] DEBUG [urllib3.connectionpool._make_request:396] https://translate.yandex.net:443 "GET /api/v1/tr.json/translate?id=1308a84a.6016deed.0c4881a2.74722d74657874-3-0&srv=tr-text&lang=en&reason=de&format=text HTTP/1.1" 403 44
[2021-02-10 15:14:12] DEBUG [translation_services.yandex.translate:85] status_code=403, json_code=406

When I googled for the error code from Yandex, I found this page: https://yandex.com/dev/translate/doc/dg/concepts/api-keys.html, which implies the free API was discontinued in May 2020 for non-residents of the Russian Federation.

What am I doing wrong, or has the api been discontinued?

Thanks!

Mark

Implementing proxy?

Is there any plan of implementing proxy into this library?

I can make a pull request if you wish althrough I'm not sure how much will that help (I mean implementing the proxies).

From what I saw with the other open source library googletrans, google seems still able to detect the user even after proxy usage.

Possible to use html mode?

It seems google has a bug with text mode which adds spaces on html tags. This also happens on google official api if I use the text mode (but it does not in html mode).

Does the endpoint this library uses have this kind of option?

MicrosoftTranslate.text_to_speech is not working

from translatepy.translators.microsoft import MicrosoftTranslate

t = MicrosoftTranslate()

result = t.text_to_speech("Hello", 65, "male", "eng")

Traceback (most recent call last):
File "D:\Project\Python\translate\123.py", line 3, in
t = MicrosoftTranslate()
File "D:\Project\Python\translate\translatepy\translators\microsoft.py", line 88, in init
self.session_manager = MicrosoftSessionManager(request)
File "D:\Project\Python\translate\translatepy\translators\microsoft.py", line 39, in init
self._parse_authorization_data()
File "D:\Project\Python\translate\translatepy\translators\microsoft.py", line 46, in parse_authorization_data
token_response = self.bing_session.send(url123, data={})
File "D:\Project\Python\translate\translatepy\translators\bing.py", line 106, in send
response = request.json()
File "D:\Project\Python\translate\translatepy\utils\request.py", line 80, in json
return loads(self.text, **kwargs)
File "C:\Users\Виктор\AppData\Local\Programs\Python\Python310\lib\json_init
.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\Виктор\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\Виктор\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Process finished with exit code 1

Bing & Deepl translators seem to be unable to get the correct JSON

Hi all,
I had tried to use the bing\deepl\google service, and only the google service can get the result, however the result text doesn't match the website shows... :(
Test code like this:

#from translatepy.translators.deepl import DeeplTranslate
from translatepy.translators.bing import BingTranslate
#from translatepy.translators.google import GoogleTranslate

#GoogleTranslate().translate('测试用例','en')
BingTranslate().translate('测试用例','en')
#DeeplTranslate().translate('测试用例','en')

And bing and deepl returned the same JSON error like this:

JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It seems pointing the requests post isn't working correctly...?

When install from git won't download some classes

Hello!!

I wanted to use translate_html method. As you told me yesterday I have to use the option 2: from git
pip install --upgrade git+https://github.com/Animenosekai/translate

First of all I did pip uninstall translatepy to ensure no conflicts with the old code. After this, I executed
pip install --upgrade git+https://github.com/Animenosekai/translate

Unfortunatelly, there are missing some classes inside translate.py file like
from translatepy.translators
from translatepy.utils.annotations import List
from translatepy.utils.queue import Queue
from translatepy.utils.request import Request
from translatepy.utils.sanitize import remove_spaces

I know I could copy and paste the missing code from github, but for me it would be a dirty way to solve it.

Is there a good way to solve it?

Thank you so much!

David

pip does not match github code?

Hello, I installed this package in my environment. I was trying to use translate_html but it does not exist in the current pip version. Is possible that needs to update the code in pip? Or am I doing something wrong?

I already executed pip install --upgrade translatepy and I have got translatepy v2.1. Unfortunately translate_html method is not inside the code..

Could you help me to solve that?

Thank you!!

ReversoTranslate TypeError

Good morning, today I've been testing a bit with the module with my application, and realized that some string sequences can lead to the translate function to output the following.

Traceback (most recent call last):
  File "c:\Users\Vioshim\Desktop\Parser\tests\test.py", line 7, in <module>
    print(translator.translate(text='', destination_language='Spanish'))
  File "C:\Users\Vioshim\Desktop\Parser\env\lib\site-packages\translatepy\translate.py", line 75, in 
translate
    lang, response = self.reverso_translate(text, destination_language, source_language)
TypeError: 'ReversoTranslate' object is not callable

This happens when attempting to translate a string that doesn't contain words, such as "" or "\n" rather than returning the string itself.

In my code, I handle it currently by catching the exception, however I decided to mention the inconvenient as I'm not sure if it is intended.

Example code:

from translatepy import Translator

translator = Translator()
print(translator.translate(text='    ', destination_language='Spanish'))

Have a good day

Consider adding DeepL.

This "issue" is a work-in-progress; feel free to contribute in the comments.

Strangely enough, I found this translation engine while watching a vTuber. This website is pretty complex, so strap in.

Wikipedia: https://en.wikipedia.org/wiki/DeepL_Translator
Translator: https://www.deepl.com/translator

Supported Languages

From

[
  ["Any language (detect)", "auto"], // It seems that language detection is done through a JSON-RPC request unrelated to translating, so this is kinda pointless, but I won't remove it.
  ["Chinese", "zh"],
  ["Dutch", "nl"],
  ["English", "en"],
  ["French", "fr"],
  ["German", "de"],
  ["Italian", "it"],
  ["Japanese", "ja"],
  ["Polish", "pl"],
  ["Portuguese", "pt"],
  ["Russian", "ru"],
  ["Spanish", "es"]
]

To

[
  ["English (American)", "en-US"],
  ["English (British)", "en-GB"],
  ["Chinese (simplified)", "zh-ZH"],
  ["Dutch", "nl-NL"], // Formality support
  ["French", "fr-FR"], // Formality support
  ["German", "de-DE"], // Formality support
  ["Italian", "it-IT"], // Formality support
  ["Japanese", "ja-JA"],
  ["Polish", "pl-PL"],  // Formality support
  ["Portuguese", "pt-PT"], // Formality support
  ["Portuguese (Brazilian)", "pt-BR"], // Formality support
  ["Russian", "ru-RU"], // Formality support
  ["Spanish", "es-ES"] // Formality support
]

Formality

This is where it starts to get interesting! Some languages that you can translate to can have differing levels of formality. All languages that do support it are labeled above.

[
  ["Formal tone", "formal"],
  ["Informal tone", "informal"],
  ["Automatic", "auto"]
]

Translating

This API seems to make use of JSON-RPC, which I'm not super familiar with, so assistance would be appreciated.
URL: https://www2.deepl.com/jsonrpc
HTTP Verb: POST

Example Request/Response

Here, we're translating hello in English to Japanese.

Request Payload (JSON)

{
  "jsonrpc": "2.0",
  "method": "LMT_handle_jobs",
  "params": {
    "jobs": [
      {
        "kind": "default",
        // Ignore use of en here, these same parameters are used for all languages
        "raw_en_sentence": "hello",
        "raw_en_context_before": [],
        "raw_en_context_after": [],
        "preferred_num_beams": 4
      }
    ],
    "lang": {
      "user_preferred_langs": [
        "NL",
        "DE",
        "IT",
        "PL",
        "PT",
        "RU",
        "ES",
        "ZH",
        "FR",
        "JA",
        "EN"
      ],
      // To and from
      "source_lang_computed": "EN",
      "target_lang": "JA"
    },
    "priority": 1,
    "commonJobParams": {},
    "timestamp": 1613084905408
  },
  // No idea what this means; random number?
  "id": 52850007
}

Response

{
  "jsonrpc": "2.0",
  "id": 52850007,
  "result": {
    "translations": [
      {
        "beams": [
          // Recommended translation
          {
            "postprocessed_sentence": "こんにちわ",
            "num_symbols": 6
          },
          // Alternatives
          {
            "postprocessed_sentence": "こんにちは",
            "num_symbols": 3
          },
          {
            "postprocessed_sentence": "ハロー",
            "num_symbols": 3
          },
          {
            "postprocessed_sentence": "もしもし",
            "num_symbols": 3
          }
        ],
        "quality": "normal"
      }
    ],
    "target_lang": "JA",
    "source_lang": "EN",
    "source_lang_is_confident": false,
    "detectedLanguages": {},
    "timestamp": 1613084907,
    "date": "20210211"
  }
}

Dictionary

Alongside translation, we also have access to a dictionary sent via an HTML fragment.

Example Request/Response

URL: https://dict.deepl.com/english-japanese/search?ajax=1&source=english&onlyDictEntries=1&translator=dnsof7h3k2lgh3gda&delay=300&jsStatus=0&kind=full&eventkind=langChange&forleftside=true
HTTP Verb: POST

Request Body (Form Data)

query=hello

Response

<div id='data' data-numberQueriesSoFar='0' data-lingueeEncoding='utf-8' data-sourceIsLang1='0' data-lang1='JA' data-lang2='EN' data-mainFlag='us' data-baseURL='/english-japanese' data-query='hello' data-correctSpellingOfQuery='hello' data-numberPhrases='0' data-numberSentences='0' data-sourceLang='EN' data-sourceFlag='us' data-targetLang='JA'></div>
<div class='innercontent'>
    <div id='dictionary'>
        <h1 class='dict_headline_for_0 bothsides wide_in_main'>
            <div class='openTriangle'> &#9662;</div>Dictionary English-Japanese</h1>
        <div class='isMainTerm' data-source-lang='EN'>
            <div class='exact'>
                <div class='lemma featured' wt='0'>
                    <div>
                        <h2 class='line lemma_desc' lid='EN:hello15448'><span class='tag_lemma'><a class='dictLink' rel='nofollow' href='/english-japanese/translation/hello.html'>hello</a> <a id='EN_US/5d/5d41402abc4b2a76b9719d911017c592-0' class='audio' onclick='playSound(this,"EN_US/5d/5d41402abc4b2a76b9719d911017c592-0","American English","EN_UK/5d/5d41402abc4b2a76b9719d911017c592-0","British English");'></a></span><span class='dash'>&mdash;</span></h2>
                        <div class='lemma_content'>
                            <div class='meaninggroup  sortablemg' gid='0'>
                                <div class='translation_lines'>
                                    <div class='translation sortablemg featured'>
                                        <h3 class='translation_desc'><span class='tag_trans' bid='10001294287' lid='JA:#########57716'><a id='dictEntry10001294287' href='/japanese-english/translation/%E3%83%8F%E3%83%AD%E3%83%BC.html' class='dictLink featured'>ハロー</a><a class='expand_i'></a></span>
                                            <!--tag_trans-->
                                        </h3>
                                        <!--translation_desc-->
                                        <!-- editorial isFeatured 1 inType 0 allowFeatured 1 available: 0 -->
                                    </div>
                                    <div class='translation sortablemg featured'>
                                        <h3 class='translation_desc'><span class='tag_trans' bid='10001347706' lid='JA:###############53649'><a id='dictEntry10001347706' href='/japanese-english/translation/%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF.html' class='dictLink featured'>こんにちは</a><a class='expand_i'></a></span>
                                            <!--tag_trans-->
                                        </h3>
                                        <!--translation_desc-->
                                        <!-- editorial isFeatured 1 inType 0 allowFeatured 1 available: 0 -->
                                    </div>
                                    <div class='translation_group'>
                                        <div class='line group_line translation_group_line'><span class='notascommon'>less common:</span>
                                            <div class='translation sortablemg translation_first'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001209742' lid='JA:#########20937'><a id='dictEntry10001209742' href='/japanese-english/translation/%E4%BB%8A%E6%97%A5%E3%81%AF.html' class='dictLink'>今日は</a><a class='expand_i'></a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001478146' lid='JA:#########18286'><a id='dictEntry10001478146' href='/japanese-english/translation/%E3%81%A9%E3%81%86%E3%82%82.html' class='dictLink'>どうも</a><a class='expand_i'></a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001347751' lid='JA:###############58474'><a id='dictEntry10001347751' href='/japanese-english/translation/%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%82%8F.html' class='dictLink'>こんにちわ</a><a class='expand_i'></a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001241561' lid='JA:############4113'><a id='dictEntry10001241561' href='/japanese-english/translation/%E3%81%93%E3%81%AB%E3%81%A1%E3%81%AF.html' class='dictLink'>こにちは</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001292425' lid='JA:############49954'><a id='dictEntry10001292425' href='/japanese-english/translation/%E3%81%BB%E3%81%84%E3%81%BB%E3%81%84.html' class='dictLink'>ほいほい</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001528104' lid='JA:#########31519'><a id='dictEntry10001528104' href='/japanese-english/translation/%E3%81%8A%E3%83%BC%E3%81%84.html' class='dictLink'>おーい</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001281333' lid='JA:########################9421'><a id='dictEntry10001281333' href='/japanese-english/translation/%E3%82%A2%E3%83%B3%E3%83%8B%E3%83%A7%E3%83%B3%E3%83%8F%E3%82%BB%E3%83%A8.html' class='dictLink'>アンニョンハセヨ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001371063' lid='JA:##################25995'><a id='dictEntry10001371063' href='/japanese-english/translation/%E3%82%A2%E3%83%8B%E3%83%A7%E3%83%8F%E3%82%BB%E3%83%A8.html' class='dictLink'>アニョハセヨ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001346566' lid='JA:############55814'><a id='dictEntry10001346566' href='/japanese-english/translation/%E3%81%93%E3%82%93%E3%81%A1%E3%81%AF.html' class='dictLink'>こんちは</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001320067' lid='JA:############36110'><a id='dictEntry10001320067' href='/japanese-english/translation/%E3%83%8F%E3%82%A4%E3%82%B5%E3%82%A4.html' class='dictLink'>ハイサイ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001241611' lid='JA:############34605'><a id='dictEntry10001241611' href='/japanese-english/translation/%E3%81%93%E3%81%AB%E3%81%A1%E3%82%8F.html' class='dictLink'>こにちわ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001336747' lid='JA:############24625'><a id='dictEntry10001336747' href='/japanese-english/translation/%E3%83%8B%E3%83%BC%E3%83%8F%E3%82%AA.html' class='dictLink'>ニーハオ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001295187' lid='JA:######56948'><a id='dictEntry10001295187' href='/japanese-english/translation/%E3%83%8F%E3%83%AD.html' class='dictLink'>ハロ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                                <span class='sep'>&middot;</span> </div>
                                            <div class='translation sortablemg'>
                                                <div class='translation_desc'><span class='tag_trans' bid='10001214671' lid='JA:############61778'><a id='dictEntry10001214671' href='/japanese-english/translation/%E3%83%9B%E3%82%A4%E3%83%9B%E3%82%A4.html' class='dictLink'>ホイホイ</a></span>
                                                    <!--tag_trans-->
                                                </div>
                                                <!--translation_desc-->
                                                <!-- editorial isFeatured 0 inType 0 allowFeatured 1 available: 0 -->
                                            </div>
                                        </div>
                                    </div>

                                </div>
                                <!--translation_lines-->
                            </div>
                            <!--meaninggroup-->
                        </div>
                        <!--lemma_content-->
                    </div>
                </div>
                <!--lemma-->
            </div>
            <!--exact-->
            <div class='copyrightLineOuter'>
                <div class='copyrightLine'>&copy; Linguee Dictionary, 2020</div>
            </div>
        </div>
    </div>
    <!--dictionary-->
</div>
<!--innercontent-->
<!-- 2021-Feb-11 23:21:03 -->

[SERVER]: cannot import name 'General' from 'nasse.config'

@ZhymabekRoman ➜ /workspaces/translate (cli-dev-1) $ translatepy server
Traceback (most recent call last):
  File "/home/codespace/.python/current/bin/translatepy", line 8, in <module>
    sys.exit(main())
  File "/usr/local/python/3.10.8/lib/python3.10/site-packages/translatepy/__main__.py", line 166, in main
    raise err
  File "/usr/local/python/3.10.8/lib/python3.10/site-packages/translatepy/__main__.py", line 153, in main
    from translatepy.server import translation
  File "/usr/local/python/3.10.8/lib/python3.10/site-packages/translatepy/server/translation.py", line 11, in <module>
    from translatepy.server.server import app
  File "/usr/local/python/3.10.8/lib/python3.10/site-packages/translatepy/server/server.py", line 2, in <module>
    from nasse.config import General
ImportError: cannot import name 'General' from 'nasse.config' (/usr/local/python/3.10.8/lib/python3.10/site-packages/nasse/config.py)

Missing spaces between tags when using translate_html

Missing spaces between tags when using translate_html

Code

from translatepy import Translator
print(Translator().translate_html("<p>I am a student and <strong>you are a teacher</strong></p>", "de"))

Current:

<p>Ich bin Student und<strong>du bist ein Lehrer</strong></p>

Expected:

<p>Ich bin Student und <strong>du bist ein Lehrer</strong></p>

GoogleTranslateV1 does not return the same results as translate.google.com (the website)

Hey guys, have you tested https://github.com/Animenosekai/translate/blob/main/translatepy/translators/google.py#L89 with the same text you want to translate to english, then tested it on the actual site, to see if it returns the same result?? Am I doing something wrong or does it do that for you too?

text:

كانت الحياة هناك ، على ما يبدو حلما ، على وشك أن تتحول إلى كابوس. "(وقفة درامية)" مرحبًا يا رفاق! لقد وجدت للتو طريقة جديدة رائعة لحلاقة شعر خصلاتي!

actual site:

Life there, apparently a dream, was about to turn into a nightmare. (dramatic pause) Hey guys! I just found a great new way to shave my tresses!

python module:

Life was there, apparently a dream, about to turn into a nightmare."(Dramatic pause)" Hello you guys!I just found a wonderful new way to shave my hair!

Originally posted by @NawtJ0sh in #21 (comment)

What is the right way to specify dest lang as Chinese?

from translatepy.utils.request import Request
from translatepy.translators.google import GoogleTranslateV1

proxy_request = Request(proxy_urls=["127.0.0.1:10809"])
translator = GoogleTranslateV1(request=proxy_request)
print(translator.translate(txt, Language("zh-cn")).result)

With above code, exception occurs:

translatepy.exceptions.UnknownLanguage: Couldn't recognize the given language (zh-cn)
Did you mean: czech (Similarity: 67.61%)?

I tried again with "zh-CN" or "Chinese", none of them worked!

example() not working

The example function is not working. First, I use the following code

from translatepy import Translator
translate = Translator()

print(translate.example("Hello", "German", "English"))

which gives the following error.

Traceback (most recent call last):
  File "C:\Users\femustafa\Desktop\Language app\main.py", line 81, in <module>
    print(translate.example("Hello", "German", "English"))
  File "C:\Users\femustafa\Desktop\Language app\.venv\lib\site-packages\translatepy\translate.py", line 363, in example
    raise NoResult("No service has returned a valid result")
translatepy.exceptions.NoResult: No service has returned a valid result

As example function is not implemented for all translators. I just use Bing translator

from translatepy.translators.bing import BingTranslate
translate = BingTranslate()

print(translate.example("Hello", "German", "English"))

which gives the following error.

Traceback (most recent call last):
  File "C:\Users\femustafa\Desktop\Language app\main.py", line 81, in <module>
    print(translate.translate("Hello", "German", "English"))
  File "C:\Users\femustafa\Desktop\Language app\.venv\lib\site-packages\translatepy\translators\base.py", line 110, in translate 
    source_language, translation = self._translate(text, dest_code, source_code)
  File "C:\Users\femustafa\Desktop\Language app\.venv\lib\site-packages\translatepy\translators\bing.py", line 156, in _translate    response = self.session_manager.send("https://www.bing.com/ttranslatev3", data={'text': text, 'fromLang': source_language, 'to': destination_language})
  File "C:\Users\femustafa\Desktop\Language app\.venv\lib\site-packages\translatepy\translators\bing.py", line 135, in send      
    raise BingTranslateException(status_code)
translatepy.translators.bing.BingTranslateException: 205 | Unknown error. Error code: 205

Am i doing something wrong? . Your help will be appreciated.

How reliable is this library?

First of all, nice work!
I would like to implement this in some of my projects. However I would like to know if this is reliable (you are using some API Keys), or you are using workarounds, which might stop working at any time.

PS: This is not really an issue, but rather a question.

How to set timeout for Translate requesting?

My server can't access Google, Bing, but Yandex is OK.

When I use this command It costs about 5 minutes to get result.

from translatepy import Translator
translator = Translator()
translator.translate("Hello", "French")

This works fast!

from translatepy.translators.yandex import YandexTranslate
YandexTranslate().translate('My name is John', 'French')

I know that translatepy is working with multiple engines. Does this code try all the engines one by one or at the same time with multi-threads?

I want to set a global request timeout to make it fail quickly, like this t = Translator(request=Request(timeout=10))

Any solution?

Language auto detecting is broken in MyMemory

➜  translate git:(main) ✗ ipython
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
Type 'copyright', 'credits' or 'license' for more information
IPython 8.8.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from translatepy.translators import MyMemoryTranslate
In [2]: dl = MyMemoryTranslate()
In [3]: dl.language("hello")

# server response
ic| request.json(): {'exception_code': None,
                     'matches': '',
                     'mtLangSupported': None,
                     'quotaFinished': None,
                     'responderId': None,
                     'responseData': {'translatedText': "'AUTODETECT' IS AN INVALID SOURCE LANGUAGE . "
                                                        'EXAMPLE: LANGPAIR=EN|IT USING 2 LETTER '
                                                        'ISO OR RFC3066 LIKE ZH-CN. ALMOST ALL '
                                                        'LANGUAGES SUPPORTED BUT SOME MAY HAVE NO '
                                                        'CONTENT'},
                     'responseDetails': "'AUTODETECT' IS AN INVALID SOURCE LANGUAGE . EXAMPLE: "
                                        'LANGPAIR=EN|IT USING 2 LETTER ISO OR RFC3066 LIKE ZH-CN. '
                                        'ALMOST ALL LANGUAGES SUPPORTED BUT SOME MAY HAVE NO '
                                        'CONTENT',
                     'responseStatus': '403'}

Same issues:
terryyin/translate-python#69
terryyin/translate-python#77
terryyin/translate-python#53

NoResult: No service has returned a valid result

Hello,

Get "No service has returned a valid result" from time to time. Sometimes it can be fixed by explicitly specifying the languages in "translate", but sometimes it just doesn't. It refuses to translate specific columns from a data frame while works well with others.

NoResult                                  Traceback (most recent call last)
Input In [14], in <cell line: 3>()
      1 #protest translator
      2 #df['Title EN'] = df['Titlle'].apply(str).apply(lambda x: translator.translate(x, 'en', 'ru'))
----> 3 df['short_description EN'] = df['short_description'].apply(str).apply(lambda x: translator.translate(x, 'en', 'ru'))

File /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/core/series.py:4138, in Series.apply(self, func, convert_dtype, args, **kwds)
   4136     else:
   4137         values = self.astype(object)._values
-> 4138         mapped = lib.map_infer(values, f, convert=convert_dtype)
   4140 if len(mapped) and isinstance(mapped[0], Series):
   4141     # GH 25959 use pd.array instead of tolist
   4142     # so extension arrays can be used
   4143     return self._constructor_expanddim(pd_array(mapped), index=self.index)

File pandas/_libs/lib.pyx:2467, in pandas._libs.lib.map_infer()

Input In [14], in <lambda>(x)
      1 #protest translator
      2 #df['Title EN'] = df['Title'].apply(str).apply(lambda x: translator.translate(x, 'en'))
----> 3 df['short_description EN'] = df['short_description'].apply(str).apply(lambda x: translator.translate(x, 'en', 'ru'))

File /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/translatepy/translate.py:135, in Translate.translate(self, text, destination_language, source_language)
    133         continue
    134 else:
--> 135     raise NoResult("No service has returned a valid result")

NoResult: No service has returned a valid result

Thanks

support for proxies

how to use proxies in the googletranslatev1() method ... ive tried in many ways but not working

DeepL translations not working

Today I installed a fresh translatepy. I tried to translate simple sentences with DeepL like: "Hola, ¿Cómo estás?" (means "Hello, how are you?") from spanish to english.

But DeepL never returns a response. It works properly with the other translators. As we know, DeepL works hard to prevent users use unofficial API.

Do you know if something changed in unofficial API?

Two months ago DeepL used to work. It did not work perfectly because when I sent more thant 3 or 4 request in little time it stopped working for a while. But at least it worked some times.

The thing is that now it is not going to work.

Please, could you check it out?

Thank you!

What translation is used by default

Hi there,

Thanks a lot for this great tool.

It is not really an issue with the code itself, but more with the documentation...

This sentence is a bit ambiguous to me:
"The translator lets you group and use multiple translators at the same time, to increase your chance on getting an answer"

so when you run something like

from translatepy import Translator
translator = Translator()
text = "bonjour"
print(translator.translate(text, "en"))

For example when you use from translatepy.translators.google import GoogleTranslate it is obvious here we're using google translate.
But in the generic case with Translator What service is actually used in the end? does it simply try a sequence of services until the first one that responds? if so, in what order?

Thanks in advance for a clarification :)

Not accurate source language autodetection

Hi!
First of all wanted to say that I love the project, have been using it for a while now.

I came across some bizarre behavior that maybe you could check or maybe explain to me (I tried checking the source code for the functions but did not see anything relevant that could be causing this).

In this case, it seems that the source language autodetection is a bit off when giving it short and single words. I reproduced it with Spanish, but I don't know if it does happen in other languages too.
In this case, if you give the words "casa" or "hola" for example, it will detect the source language as English instead of Spanish.

For example using the base translator:

Python 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import translatepy
>>> translatepy.Translator().language("casa")
LanguageResult(service=Google, source=casa, result=eng)

Then I tried using the translators explicitly, in this case Reverso and Google, then using the base translator again, and it worked correctly (I guess because of the cache, but I may be wrong):

Python 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import translatepy
>>> translatepy.translators.reverso.ReversoTranslate().language("casa")
LanguageResult(service=Reverso, source=casa, result=spa)
>>> translatepy.translators.google.GoogleTranslate().language("casa")
LanguageResult(service=Google, source=casa, result=spa)
>>> translatepy.Translate().language("casa")
LanguageResult(service=Google, source=casa, result=spa)

But interestingly enough, then, in the same session, using the base translator with the method translate(), the detection was off again:

>>> translatepy.Translate().translate("casa", "en")
TranslationResult(service=Google, source=casa, source_language=eng, destination_language=eng, result=casa)

Any ideas of why could be this happening? I guess the workaround by know would be to run the GoogleTranslate().language() method, and then the Translator().translate() method to get accurate results, like so:

>>> lang = translatepy.translators.google.GoogleTranslate().language("casa")
>>> translatepy.Translate().translate("casa", "en", lang.result)
TranslationResult(service=Google, source=casa, source_language=spa, destination_language=eng, result=house)

Anyway, wanted to ask about this and see if there is any reasoning behind it.
Sorry for the long message and thanks in adavance !

Is the DeepL split text correct?

I believe that deepl split text should splits the text into an array of text lines.

Currently it appears that the regex doesn't splits by line, and the regex usage doesn't change the text variable.

But I'm not sure if I'm confusing the calls to LMT_split_into_sentences vs LMT_split_text (I noticed that using deepl in the browser, it appears to use the later).

I made a test in the browser, translating "こんにちは世界" and "こんにちは" in two separated lines, and in case it helps, it made the calls:

An option and then a post request to https://www2.deepl.com/jsonrpc?method=LMT_split_text:

{
    "jsonrpc":"2.0",
    "method" : "LMT_split_text",
    "params":{"texts":["こんにちは世界","こんにちは"],
    "commonJobParams":{"mode":"translate"},
    "lang":{
        "lang_user_selected":"JA",
        "preference":{
            "weight":/*..*/,
            "default":"default"}
        }
    },
    "id":/*..*/
}

With the response:

{
    "jsonrpc":"2.0",
    "id":/*..*/,
    "result":{
        "lang":{"detected":"JA","isConfident":true,"detectedLanguages":{"JA":1.0}},
        "texts":[
            {"chunks":[{"sentences":[{"prefix":"","text":"\u3053\u3093\u306B\u3061\u306F\u4E16\u754C"}]}]},
            {"chunks":[{"sentences":[{"prefix":"","text":"\u3053\u3093\u306B\u3061\u306F"}]}]}
        ]
    }
}

Then an option and then a post request to https://www2.deepl.com/jsonrpc?method=LMT_handle_jobs:

{
    "jsonrpc":"2.0",
    "method": "LMT_handle_jobs",
    "params":{
        "jobs":[
            {
                "kind":"default",
                "sentences":[{"text":"こんにちは世界","id":0,"prefix":""}],
                "raw_en_context_before":[],
                "raw_en_context_after":["こんにちは"],
                "preferred_num_beams":1
            },
            {
                "kind":"default",
                "sentences":[ {"text":"こんにちは","id":1, "prefix":""}],
                "raw_en_context_before":["こんにちは世界"],
                "raw_en_context_after":[],
                "preferred_num_beams":1
            }
        ],
        "lang":{"preference":{"weight":{},"default":"default"},"source_lang_computed":"JA","target_lang":"EN"},
        "priority":1,
        "commonJobParams":{"regionalVariant":"en-US","mode":"translate","browserType":1},
        "timestamp":/*..*/
    },
    "id":/*..*/
}

With the response:

{
    "jsonrpc":"2.0","id":/*..*/,
    "result":{"translations":[
        {"beams":[{"sentences":[{"text":"Hello World","ids":[0]}],"num_symbols":3}],"quality":"normal"},
        {"beams":[{"sentences":[{"text":"Hello world","ids":[1]}],"num_symbols":3}],"quality":"normal"}],
    "target_lang":"EN","source_lang":"JA","source_lang_is_confident":false,"detectedLanguages":/*..*/}
}

(for some reason it incorrectly translated both lines to Hello World, but it gave a beam for each line)

I hope this helps in case there is something missing! (I would make more tests but I'm getting "Too Many Requests" error, so I guess my IP was banned for trying invalid requests).

Translations clipped

Hi,
You have something awesome going here :)
I have troubles with wrong spacing of translations, and others with googletrans and py-googletrans, so I gave yours a try.

The translations are clipped for some reason:
Source:

NYC Hot Sauce Expo NYC Hot Sauce Expo: Fiery Food & More Brooklyn Expo Center (Brooklyn, NY) This two-day tribute to all things hot and spicy blazes its way into the Brooklyn Event Center for the...
translatorpy ca: NYC Hot Sauce Expo NYC Hot Sauce Expo: menjar ardent
translatorpy es: NYC Hot Sauce Expo NYC Hot Sauce Expo: comida ardiente
translatorpy en: NYC Hot Sauce Expo NYC Hot Sauce Expo: Fiery Food
Google Translate API ca : NYC Hot Sauce Expo NYC Hot Sauce Expo: Fiery Food & More Brooklyn Expo Center (Brooklyn, NY) Aquest homenatge de dos dies a totes les coses picants i calentes s’obre al Centre d’esdeveniments de Brooklyn per ...
Google Translate API es : NYC Hot Sauce Expo NYC Hot Sauce Expo: Fiery Food & More Brooklyn Expo Center (Brooklyn, NY) Este homenaje de dos días a todo lo picante y picante se abre camino en el Brooklyn Event Center para el ...
Google Translate API en : NYC Hot Sauce Expo NYC Hot Sauce Expo: Fiery Food & More Brooklyn Expo Center (Brooklyn, NY) This two-day tribute to all things hot and spicy blazes its way into the Brooklyn Event Center for the...

DeeplTranslate is not working

⭕In translatepy 1.7:

from translatepy import Translator
t = Translator().deepl_translate
r = t.translate("こんにちは", "ru", "ja")
print(r)
('JA', 'Здравствуйте')

❌In translatepy 2.0:

from translatepy.translators.deepl import DeeplTranslate
t = DeeplTranslate()
r = t.translate("こんにちは", "ru", "ja")
print(r)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/site-packages/translatepy/translators/base.py", line 94, in translate
    source_language, translation = self._translate(text, dest_code, source_code)
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/site-packages/translatepy/translators/deepl.py", line 136, in _translate
    sentences, computed_lang = self._split_into_sentences(text, destination_language, source_language)
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/site-packages/translatepy/translators/deepl.py", line 127, in _split_into_sentences
    resp = self.jsonrpc.send_jsonrpc("LMT_split_into_sentences", params)
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/site-packages/translatepy/translators/deepl.py", line 95, in send_jsonrpc
    response = request.json()
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/site-packages/translatepy/utils/request.py", line 74, in json
    return loads(self.text, **kwargs)
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/home/eggplants/.pyenv/versions/3.9.2/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Expecting value: line 1 column 1 (char 0)

a suggest

I suggest that the author can package the project into a binary file and provide a network interface, like http, so that the other language can benefit from the project,

Proxy Support

Please Add proxy support so we can decrease the chance of getting banned when translating from Google for example.

Different Translation on Yandex

BTW Thanks for this library its such a nice library and suites to my on going project.

My question is I found out testing Yandex Translate on Live vs The Library Itself and it shows a different translation.

This is the text.

{
'original_text': 'And as surely as that, we will achieve the country all Filipinos deserve. God bless the Philippines, God bless our work. Maraming, maraming salamat po sa inyong lahat!',
'translated_text': 'Und so sicher werden wir das Land erreichen, das alle Filipinos verdienen. Gott segne die Philippinen, Gott segne unsere Arbeit. Marschieren, marschieren salamat po sa inyong lahat!'
}

image

But when I segment the full text into single sentences. It actually translate the same.

Bulk Translation support?

GoogleTrans supports bulk translation.
image
Can translatepy do the same? It could be improved from googletrans with parallelizing (googletrans does it sequentially I believe). Although implementing this outside the translatepy api is trivial, it would be nice and developer-friendly to allow this functionality by just passing a list to the translate method.

Next: 3.0


version: 3.0
status: ALPHA

Next

Provides informations about the next coming version

Design Banner

This version provides general improvements over the whole project, bringing exciting new features.

New

Fixes

Updates

Everything is detailed in the README.md file

A comparison from the previous release can be found here v2.3...v3.0

A comparison from the current branch can be found here v3.0...main

Note
Ideas with a 🧃 mark in front still need some consideration and might not be in the final product.

🍡 Animenosekai

Footnotes

  1. For single word translations ?

  2. Following #28

  3. Following https://github.com/Animenosekai/translate/issues/78#issuecomment-1443529618

  4. Following #54

  5. An actual public website was planned but due to the end of Heroku's free service and a lack of resource for now, we are going to stick with the local website (I think?). However the idea is not abandonned and a way of ranking the different translators based on real results is expected in the future.

  6. The branch needs to be changed to fit the main branch (local website) and to have a CLI access

  7. Following #66

  8. Following #65

  9. Following https://github.com/Animenosekai/translate/issues/28#issuecomment-918746811

  10. Refer to https://github.com/Animenosekai/translate/discussions/62 2

  11. Following #58

  12. We might use Textual for an interactive TUI. We should also default to the interactive TUI when running translatepy without any action specified.

  13. This provides a way of choosing the translators to use from the CLI (#53)
    But this also allows choosing the translator from the web server and thus the website.

  14. https://www.deepl.com/en/blog/deepl-welcomes-turkish-and-indonesian

None of the translators return the correct translation of the German sentence "Ich bin froh"

I tested bing, google, and yandex with the string "Ich bin froh" (German, translation is "I am happy"), and I had some issues. I used src_language = "auto".

  • Bing - returns "Englisch" as the translated text, although it did return the correct language code "de" when I tested the language detection option.
  • Google - returns "English" as the translated text
  • Yandex - returns status_code 400 and json_code 502.

Any thoughts on how to fix this?

Thanks!

Mark

Deepl does not seem to work

Or at least does not work from my location.

from translatepy.translators.deepl import DeepL
dl = DeepL()
dl.translate("test this", 'de', 'en')
# -> {"jsonrpc": "2.0","error":{"code":1042912,"message":"Too many requests."}}

pyppeteer is probably the only way to scrape deepl, but pyppeteer is quite slow.

Possible to specify translate.google.cn?

Hi. Thanks for the nice work.

I did some tests with translatepy with google (google_tr = Translator(use_google=True, use_yandex=False, use_bing=False, use_reverso=False, use_deepl=False).translate). It worked great.

However for users in some region that cannot access translate.google.com, they wont be able to use google in translatepy.

In googletrans you can do something like from googletrans import Translator; translator = Translator(service_urls=['translate.google.com', 'translate.google.co.kr',]). Is this possible with translatepy?

Thanks a lot.

Translation to Sindarin does not work

An error occurs in the Yandex Translate module when trying to translate text into Sindarin:

from translatepy.translators.yandex import YandexTranslate
from translatepy import Language
translate = YandexTranslate()
translate.translate("Hello World", "Sindarin")

gives an error:

Traceback (most recent call last):
  File "d:\Projects\translatepy\translate\translatepy\test_sjn.py", line 4, in <module>
    translate.translate("Hello World", "sjn")
  File "C:\Users\Potato\AppData\Roaming\Python\Python39\site-packages\translatepy\translators\base.py", line 93, in translate
    dest_code = self._detect_and_validate_lang(destination_language)
  File "C:\Users\Potato\AppData\Roaming\Python\Python39\site-packages\translatepy\translators\base.py", line 560, in _detect_and_validate_lang
    raise UnsupportedLanguage("The language {language_code} is not supported 
by {service}".format(language_code=language, service=str(self)))
translatepy.exceptions.UnsupportedLanguage: The language Sindarin is not supported by Yandex

Although Yandex is perfectly able to translate into Sindarin: see here

gtoken is broken

python
Python 3.9.6 (default, Aug 12 2021, 05:34:51)
[Clang 9.0.8 (https://android.googlesource.com/toolchain/llvm-project 98c855489 on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from translatepy.utils.gtoken import TokenAcquirer
>>> acquirer = TokenAcquirer()
>>> tk = acquirer.do(text)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'text' is not defined
>>> text = "Hello"
>>> tk = acquirer.do(text)
Traceback (most recent call last):
  File "/data/sdext2/data/com.termux/files/home/git/translate/translatepy/utils/gtoken.py", line 106, in _update
    code = self.RE_TKK.search(r.text).group(1).replace('var ', '')
AttributeError: 'NoneType' object has no attribute 'group'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/sdext2/data/com.termux/files/home/git/translate/translatepy/utils/gtoken.py", line 242, in do
    self._update()
  File "/data/sdext2/data/com.termux/files/home/git/translate/translatepy/utils/gtoken.py", line 110, in _update
    raise Exception('Could not find TKK token for this request.\nSee https://github.com/ssut/py-googletrans/issues/234 for more details.')
Exception: Could not find TKK token for this request.
See https://github.com/ssut/py-googletrans/issues/234 for more details.
>>>

Result mismatch Google Translate

Hi guys, hope you are doing well.
I know that you are pretty busy working on v3 and may are already aware of this already, but just in case I open this issue.

When using the GoogleTranslate service, the translation is different from what we get in the website/app:

  • Source: fr (auto)
  • Destination: en
  • Text: Sincèrement navré de ne pas avoir répondu à vos messages plus rapidement, lorsque le dossier est transmis aux développeurs, nous n'avons temporairement plus de visibilité dessus.

[1] Google translate result:
Sincerely sorry for not having replied to your messages more quickly, when the file is transmitted to the developers, we temporarily no longer have visibility on it.

[2] translatepy result:
Honestly sorry not to have answered your messages faster, when the file is transmitted to the developers, we do not temporarily have more visibility on it.

Any ideas of what/why could be this caused? I actually tried changing the session header as per #79 but it did not have any effect on the translation result.

[1]
image

[2]

Python 3.9.1 (tags/v3.9.1:1e5d33e, Dec  7 2020, 17:08:21) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import translatepy
>>> translatepy.Translator().translate("Sincèrement navré de ne pas avoir répondu à vos messages plus rapidement, lorsque le dossier est transmis aux développeurs, nous n'avons temporairement plus de visibilité dessus.", "en")
TranslationResult(service=Google, source=Sincèrement navré de ne pas avoir répondu à vos messages plus rapidement, lorsque le dossier est transmis aux développeurs, nous n'avons temporairement plus de visibilité dessus., source_language=fra, destination_language=eng, result=Honestly sorry not to have answered your messages faster, when the file is transmitted to the developers, we do not temporarily have more visibility on it.)
>>>

Consider adding the formality and glossary feature to DeepL

At first sight, it seems that it is only a matter of adding some formated fields to the JSONRPC request:

Capture d’écran 2022-07-21 172845

For formality:

  • Options available: "formal", "informal", null
  • Even though it is a "premium" feature, DeepL returns the correct tone if entered (output obtained without the tone: "Vous êtes".):
    Capture d’écran 2022-07-21 174721

For glossary (named dictionary in the json):

  • The only pairs in the glossary that are sent are those where the target text is in the target language that the user has chosen.
  • The glossary pairs are visibly sorted according to the order of the letters in the unicode table. (only the source text matters)
  • The same source text cannot have several translations (the previous pair is overwritten if this is the case)
  • A source text/target text pair is separated by '\t', two different pairs are separated by '\n'.
  • Backslash is escaped if used.

Notes:

  • browserType is optional.
  • There must be another requirement to generate a correct glossary, because after some tests I get the error -32600 which corresponds to :Invalid Request 'Invalid commonJobParams'

Error when translating long sentences

Good morning, well today I've been trying the module for my project and discovered a glitch of the application, which seems to happen in most strings.

Basically, It only translates the first line of a string.

I've tried to make workaround to it, by replacing the newlines with spaces, and even replacing the dots with commas, but it seems that it is still quite limited, in the sense that it doesn't translate text with more than 240+ ish.

This is a sample text

The Fusky entered into the library and breathed the tranquil atmosphere of the library, as well as the smell of old books.
"Nothing like a good old friend." He sighed and smiled. Even though he didn't particularly like the idea of being for hours and hours with his snout on some books... once again, but that was what he did, so he wouldn't stop now, would he?
When he reached the table he had left the books on some time ago, he blinked twice finding nothing there.        
"...That can be inconvenience."
He looked at his immediate surroundings and sighed in relief when he spotted those books on another table. When he reached this other table, he realized there was more books than those he had chosen before, and judging by the 
titles they were about Norwich's history itself.
He stood there with a wide and curious gaze, looking down at the books.
...Why would someone take my books when they were searching for the city's history...?
He shook his head and sat down on the table, placing the history books on the other side of it and focusing on the books he chose a while ago.
Now I have to organize them again...
He sighed and started working.

And it only translates this part

The Fusky entered into the library and breathed the tranquil atmosphere of the library, as well as the smell of old books.

This is how I use the translator

text = "" #Big Paragraph
translator = Translator()
print(translator.translate(text=text, destination_language="Spanish").result)

Anyways thanks for the time, this project is quite amazing.

Bing translator doesn't work

>>> from translatepy.translators.bing import BingTranslate
>>> dl = BingTranslate()
>>> dl.translate("hello", "ru")
Traceback (most recent call last):
  File "/home/ubuntu/translate/translatepy/translators/base.py", line 60, in translate
    translation = self._translate(text, dest_code, source_code)
  File "/home/ubuntu/translate/translatepy/translators/bing.py", line 84, in _translate
    return response[0]["translations"][0]["text"]
KeyError: 0

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/translate/translatepy/translators/base.py", line 75, in translate
    raise TranslationError from exc
translatepy.exceptions.TranslationError

Server response: {'statusCode': 400}

BingTranslate.text_to_speech is not working

In translatepy 2.0:

from translatepy.translators.bing import BingTranslate
t = BingTranslate()
r = t.text_to_speech("こんにちは")
print(r)
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
  File "/home/eggplants/prog/translate/translatepy/translators/base.py", line 442, in text_to_speech
    source_language, text_to_speech = self._text_to_speech(text, speed, gender, source_code)
  File "/home/eggplants/prog/translate/translatepy/translators/bing.py", line 196, in _text_to_speech
    source_language = self._language(text)
  File "/home/eggplants/prog/translate/translatepy/translators/bing.py", line 165, in _language
    response = self.session_manager.send("https://www.bing.com/ttranslatev3", data={'text': text, 'fromLang': "auto-detect", 'to': "en"})
  File "/home/eggplants/prog/translate/translatepy/translators/bing.py", line 123, in send
    raise BingTranslateException(status_code)
translatepy.translators.bing.BingTranslateException: 400 | Unknown error. Error code: 400

400 | Unknown error. Error code: 400

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.