403 on strings with unicode characters about google-translate-php HOT 10 CLOSED

Looks like the problem is that UTF-16 characters are handled very differently in PHP and JS.
This causes two problems in the PHP implementation of the token generation: a bug in string length, and a bug in charCodeAt. Here is my SO question:

http://stackoverflow.com/questions/40841149/unicode-charcodeat-equivalent-in-php/40853121

But to make this public, I will need as many of you as possible, to test, benchmark and check my code. I will make a pull request soon to make testing easier.

from google-translate-php.

@carlitosz Yes, I hopefully fixed this bug in my pull request referenced above. However the latest (v3.2.13) release is a bit out of date, and it does not include the latest few commits yet. So either you clone the current state of the "master" or install the package with composer, targeting the master branch.

Alternatively, wait for @Stichoza to close this issue, push a new release version and use that ;)

from google-translate-php.

I'm not sure if I understood the problem correctly, but first that comes to my mind is that some emojis are actually a combination of two unicode chars. Additionally, skin colored emojis might contain even more. I mean:

And after all, unicode characters (like Georgian letters) work perfectly as you can see in tests.

from google-translate-php.

Sorry, maybe I was not clear, but when testing this string I feel sad 😢, or this one This is a test 😍 , both containing an emoji, the request does not work, it returns a Forbidden error.

So this is a warning to users that might get the 403 error, and they don't know why, it might be because of some characters they have in their strings, which are not handled correctly by the charCodeAt function.

from google-translate-php.

Any solution to the problem? I am also facing it.

from google-translate-php.

The same problem. I currently trying to translate the The :attribute must be an image. with the same 403 forbidden error.

from google-translate-php.

As a temporary solution, it could be a great idea, to remove all emojis from a string, before translating it. Here is a PHP function, that should do the work:

function remove_emoji($string) {

    // Match Emoticons
    $regex_emoticons = '/[\x{1F600}-\x{1F64F}]/u';
    $clear_string = preg_replace($regex_emoticons, '', $string);

    // Match Miscellaneous Symbols and Pictographs
    $regex_symbols = '/[\x{1F300}-\x{1F5FF}]/u';
    $clear_string = preg_replace($regex_symbols, '', $clear_string);

    // Match Transport And Map Symbols
    $regex_transport = '/[\x{1F680}-\x{1F6FF}]/u';
    $clear_string = preg_replace($regex_transport, '', $clear_string);

    // Match Miscellaneous Symbols
    $regex_misc = '/[\x{2600}-\x{26FF}]/u';
    $clear_string = preg_replace($regex_misc, '', $clear_string);

    // Match Dingbats
    $regex_dingbats = '/[\x{2700}-\x{27BF}]/u';
    $clear_string = preg_replace($regex_dingbats, '', $clear_string);

    return $clear_string;
}

More information:
http://stackoverflow.com/q/12807176/2953830
https://en.wikipedia.org/wiki/Emoji

Update (2017.04.12)
Unfortunately, this will not remove all emojis, as more and more emojis are released. The only solution would be a fix in the token generation, but at least this removes the most popular (old) ones.

from google-translate-php.

Sorry for disappearing from this issue, now finally I had some time again to dig into this problem.
And I have good news: I think I got the solution!

The root of the problem was that JS works and counts with UTF-16 code units while in PHP we are mostly working with UTF-8. And as long as we have to work and count just like JavaScript, we have to convert strings to UTF-16 and extract informations from those variable. (For some reason it looks like we were good until we stayed in Unicode BMP, but not outside that region)

More details in PR #100.

from google-translate-php.

I encountered the same issue that has been talked about in this thread, where strings with emojis are causing a 403 error. Was this issue ever solved?

from google-translate-php.

Oh sorry friends, my bad. Pushed new v3.2.14 tag

from google-translate-php.