This is a header I recently had:

# Copyright (C) 2015 the WordPress team
# This file is distributed under the GNU General Public License v2 or later.
msgid ""
msgstr ""
"Project-Id-Version: Twenty Sixteen 0.1.20150828\n"
"Report-Msgid-Bugs-To: https://wordpress.org/support/theme/twentysixteen\n"
"POT-Creation-Date: 2015-11-20 12:58:54+00:00\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"PO-Revision-Date: 2015-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"

Parsing it, merging some data with MERGE_ADD and then storing the file again rearranges the header:

msgid ""
msgstr ""
"Project-Id-Version: Twenty Sixteen 0.1.20150828\n"
"Report-Msgid-Bugs-To: https://wordpress.org/support/theme/twentysixteen\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <[email protected]>\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"POT-Creation-Date: 2015-11-20 12:58:54+00:00\n"
"PO-Revision-Date: 2016-06-12T23:05:19+00:00\n"
"Language: \n"

In addition the comment and the Language field is gone.

This shouldn't be when you merge two translations without the MERGE_HEADER option.
It neither should be when you use the option.

@oscarotero Could you wait a few days before publishing a new release?
IMHO some of the plural rules that we took from http://localization-guide.readthedocs.org/en/latest/l10n/pluralforms.html are not correct.
I discovered two discrepancies but it seems that nobody is taking care of them, so IMHO that repository is not well maintained.
I'm going to write a script that takes the rules from CLDR and converts them to gettext.
It's not that easy, since CLDR rules refer to decimals and negative numbers, whereas gettext works with unsigned integers, but I'm working on it.

Currently we set the default headers are only set in the Po generator.
IMHO we should move them to the Translations constructor. This because the default headers should be included in the generated .mo files.
For instance, if we have:

$t = \Gettext\Translations::fromPhpCodeFile($file);
Generators\Mo::toFile($t, $file);

The resulting .mo file will be broken since it does not contain any header.

Initializing the default headers in the class constructor will simplify the whole process

Let's say we have a test.po file like this:

msgid ""
msgstr ""
"Project-Id-Version: \n"
"POT-Creation-Date: \n"
"PO-Revision-Date: \n"
"Language: it\n"
"Language-Team: \n"
"Last-Translator: \n"
"Plural-Forms: nplurals=2; plural=(n != 1);\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"

msgid "1 child"
msgstr ""

msgid "1 child"
msgid_plural "2 children"
msgstr[0] ""
msgstr[1] ""

If we compile it with msgfmt we have:

test.po:17: duplicate message definition...
test.po:15: ...this is the location of the first definition
msgfmt: found 1 fatal error

So, we may not have in .po files two strings with same context+msgid but different plurals.

I think that we should apply two changes:

1. Revert #46
2. Add an alternative method to Translations::find that does not take in account the plural form

Hey Oscar,

thanks for the awesome class! I'm using it to parse .po files (without headers). During developement I encountered a problem with the class: the first translation isn't parsed. I tracked down this line as the source of the problem:

for ($n = count($lines); $i < $n; $i++) {

https://github.com/oscarotero/Gettext/blob/master/src/Extractors/Po.php#L42

Why are you iterating over the lines like this?

I suggest to replace this line with

for ($i = 0; $i <= count($lines); $i++) {

or

foreach ($lines as $i => $line) {

If you like I can provide you a pull request.

Greets

When using xgettext to extract translatable strings from PHP files we have the option to extract comments (the -add-comments option):

For instance, let's say we have this test.php file

<?php

echo __(
    /* i18n: %1$s is the partial number, %2$s is the total number. Example: 2 of 3 */
    '%1$s of %2$s',
    $count,
    $total
);

If you run this command:

xgettext --add-comments=i18n --keyword=__:1 --add-location test.php

Here's the output:

#. i18n: %1$s is the partial number, %2$s is the total number. Example: 2 of 3
#: test.php:5
#, php-format
msgid "%1$s of %2$s"
msgstr ""

What about adding support to extract such comments?

I have a form that returns key => value translations (msgid => msgstr). Can I load this array into Extractors\PhpArray to store it with Generators\Mo?

my code:

<?php 

require __DIR__.'/vendor/autoload.php';


setlocale(LC_ALL, 'en-US');

$t = new \Gettext\GettextTranslator();

//Set the language and load the domain
$t->setLanguage('gl');
$t->loadDomain('messages', 'Locale');

And there are errors following...

Deprecated: setlocale(): Passing locale category name as string is deprecated. Use the LC_* -constants instead

Warning: setlocale(): Invalid locale category name LC_MESSAGES, must be one of LC_ALL, LC_COLLATE, LC_CTYPE, LC_MONETARY, LC_NUMERIC, or LC_TIME

Hi,
The procedure in Gettext\Extractors\Mo seems to extract basic string without split singular/plural strings delimited by NULL character.

From https://github.com/oscarotero/Gettext/blob/master/Gettext/Extractors/Mo.php#L52

        for ($i = 0; $i < $total; $i++) {
            $stream->seekto($table_originals[$i * 2 + 2]);
            $original = $stream->read($table_originals[$i * 2 + 1]);

            if ($original) {
                $stream->seekto($table_translations[$i * 2 + 2]);
                $entries->insert(null, $original)->setTranslation($stream->read($table_translations[$i * 2 + 1]));
            }
        }

This my workaround, while it looks a bit ugly and not very safe.

if ($original) {
    $stream->seekto($table_translations[$i * 2 + 2]);
    $original = explode("\000",$original);
    $translated = explode("\000",$stream->read($table_translations[$i * 2 + 1]));
    $singular = $original[0];
    $plural = isset($original[1]) ? $original[1] : '';
    $translation = $entries->insert(null, $original[0], $plural);
    $translation->setTranslation($translated[0]);
    if($plural) $translation->setPluralTranslation(isset($translated[1]) ? $translated[1] : $translated[0]);
}

Thanks in advance.

The merge flags passed to the Translations::mergeWith function do not have any effect. This is due to the following code staring on line 214:

$add = (boolean) $method & self::MERGE_ADD;

The intended result here is probably to cast the result of the bitwise &, however it's casting the $method param to bool, then performing the bitwise op. This basically sets the $method to self::MERGE_ADD making the other flags meaningless. The following code produces the expected results:

$add = (boolean) ($method & self::MERGE_ADD);
$references = (boolean) ($method & self::MERGE_REFERENCES);
$comments = (boolean) ($method & self::MERGE_COMMENTS);

Gettext: 3.6.0
laravel-framework: 5.2.41

After upgrading from laravel-framework 5.2.39 to 5.2.41 the Blade extractor caused an InvalidArgumentException related to the cache path of the BladeCompiler. The cause is that after this upgrade the base Compiler provided by Laravel checks whether a truthy cache path is provided as second argument. The Blade extractor creates a BladeCompiler with NULL as cache path.

I figure changing NULL into true would do the trick, as the Blade extractor does not appear to use cache anyway and there do not appear to be any negative consequences in the Compiler class.

Cheers,
Wouter van Dam

In your example you use
Gt::loadTranslations('locate.php');
to load a PHP array, but it seems that loading PO and MO files is not implemented yet ?

Extractors\PhpCode does not appear to be able to extract d__() and other domain-related gettext functions.

Hi,
Is there way to remove the tokens from the existing PO file.

Thanks,
Lionel

Hi,

After including the autoloader php file, I've got an error inside at line 2: syntax error, unexpected T_FUNCTION, expecting ')'

Thanks

I created a PHP file like this:

<p><?php __("Char x00: \x00"); ?></p>
<p><?php __("Char x01: \x01"); ?></p>
// omissis
<p><?php __("Char xfe: \xfe"); ?></p>
<p><?php __("Char xff: \xff"); ?></p>

And I run xgettext --from-code=utf-8 --language=PHP --keyword=__.

For the chars \x80 through 0xff it showed the following error:
invalid multibyte sequence
and the chars where passed as-is in the PO file

Only the following conversions occurred:

msgid "Char x00: " <- stripped out
msgid "Char x07: \a"
msgid "Char x08: \b"
msgid "Char x09: \t"
msgid "Char x0a: \n"
msgid "Char x0b: \v"
msgid "Char x0c: \f"
msgid "Char x0d: \r"
msgid "Char x22: \""
msgid "Char x5c: \\"

But for \x07 (\a), \x08 (\b), \x0b (\v), \x0c (\f), \x0d (\r), xgettext printed out error messages like this:

warning: internationalized messages should not contain the '\...' escape sequence

So, I think that when converting strings to PO we should:

• strip out \x00
• convert only \t, \n, " and \

But when converting strings from PO we should convert all the recognized sequences, I mean

• \a => "\x07"
• \b => "\x08"
• \t => "\x09"
• \n => "\x0a"
• \v => "\x0b"
• \f => "\x0c"
• \r => "\x0d"
• \" => "\x22"
• \\ => "\x5c"

If it's ok, I'll update the pull request #93

Hello. I've got .po file created with poEdit with translated plural form (http://pastebin.com/6YvDx6gY).

Using translation function "echo ('dragon');" works fine but plural translations like "echo n('%s apple', '%s apples', 1);" or "echo n__('%s apple', '%s apples', 2);" gives "%s jabłko" and "%s jabłek".

So translation is good but dynamic value %s is not set,

I cannot figure out what I'm doing wrong. Can you help me?

IMHO we should remove the $context and $language variables from the Translations class.
They should always be included only in the $headers variable.
If we do so, we can simplify the extractors.
For instance, we could remove these lines: https://github.com/oscarotero/Gettext/blob/c701efc6f88385471ad7d6f34ad11c5301cc01f5/src/Extractors/Po.php#L41-L48
Furthermore, if we do so, we prevent strange situations where we have a $language set to something and $header['Language'] set to something else.

What do you think?

Hello,

you might like to add this in the Gettext\Translations class for autocomplete:

* @method toMoFile(string $fileName)

(and others). Works well in phpStorm.

Cheers,

Mike

Would it be possible to add a merge option to use translations in translation2 to override the translations in translation1?
For instance, if you have 2 po files:

# default.po
msgid "unit"
msgstr "Unit"

# override.po
msgid "unit"
msgstr "Use this instead"

$translations = gt\Translations::fromPoFile($path . '/' . $lang . '/LC_MESSAGES/default.po');
$domainTranslation = gt\Translations::fromPoFile($path . '/' . $lang . '/LC_MESSAGES/override.po');
$translations->mergeWith($domainTranslation, gt\Translations::MERGE_OVERRIDE);

The idea here is that within each language, there would be a default translation and then another translation file that could override some of the msg strings.

I am using the following function to translate strings in an application I created. However, I am not sure if I am using it correctly because the strings are not translating. Below is my function:

function _t($msgid)
{
    $t = new Gettext\Translator();

    $t->loadTranslations(APP_PATH . 'lang' . DS . 'it_IT.php');

    $t->register();

    return __($msgid);
}

I exported the .po file into php arrays. But for some reason, the strings are not being translated into Italian. Am I instantiating the class incorrectly or missing a step?

We have a .po file like this:

#: attributes/address/composer.php:8 attributes/address/form.php:7
msgid   "Address 1"
msgstr  ""

Currently the po parser do a simple split with ':'.
In this case we'd have that we have that we call addReference with:

• $filename == attributes/address/composer.php
• $line == 8 attributes/address/form.php

Furthermore, under Windows, we may have references like this one:
C:/Users/Me/Documents/foo.php:1
The above split function will result in:

• $filename == C
• $line == /Users/Me/Documents/foo.php

We'd need a better reference parser...

can't install this package with composer require oscarotero/Gettext

could the problem be because of the upper G in project name ?

One remark on your example:
$translation = $translations->find(null, 'apples');

When you set $context as null and then you use $translation->is() function, it always returns false;

The README gives a few examples on how to use the library, but I could not find the method "Translations::fromPoFile". I installed the library through composer (gettext/gettext) v3.4.2.

Thanks!

When merging

$a->mergeWith($b, MERGE_REFERENCES);

Then B should overwrite the references of A.

Or as the references of A might not be up-to-date anymore there could be an additional option like OVERWRITE_* for each merge option (where it makes sense).

Hello. Congratulations to the Platinum Medal!

Here I am again with some nasty bug. Imagine this code:

<?php echo __( 'Something', TEXTDOMAIN_CONSTANT ); ?>

Your PhpFunctionsScanner does not recognize the second argument.
It just calls "stopArgument" on src/Utils/PhpFunctionsScanner.php:100

I would suggest this:

• Scan for the closing brackets to really stop arguments.
• If the argument is not T_CONSTANT_ENCAPSED_STRING but exists as a constant, then this is your text domain / real argument value.

So other developers need to introduce such constants before parsing code. This would be a workaround in your scanner after the T_STRING case:

case T_STRING:
                    //add an argument to the current function
                    if (defined($value[1]) && constant($value[1])) {
                        $bufferFunctions[0]->addArgumentChunk((string) constant($value[1]));
                        break;
                    }

There is more to do but I am not able to handle that at the moment.

Edit: Double checked. That's all for this issue.

Hello,
please wrap the __ function definition with

if(!function_exists('__')) {
 //... function
}

Because it is a popular name of the function, as well as t for translations. So personally in my system I use my own __ function and have fatal error after connect this composer package.

When generating PO files, this line always prepends an empty translation:

msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
...

Because of this, if I have an empty text in my gettext or __ function, e.g. __(''), it will be replaced by Project-Id-Version: Report-Msgid-Bugs-To: ... and the entire header.

What is the purpose of this empty translation? Can it be removed?

Hello!
I have a little problem when wordpress use multiline translations that i generated with your library.

Poedit soft parse php-file and generate po-file like this
http://picsee.net/upload/2014-10-28/8a1a2d4cafd9.png

If i parse php-file using your library it looks like this
http://picsee.net/upload/2014-10-28/69ca7c14834a.png

So Poedit makes a multiline msgid and your lib makes oneline msgid
And when wordpress uses msgstr from your lib translation it ouptuts the text than is placed before the first newline character.
http://picsee.net/upload/2014-10-28/2289ace38aab.png

With Poedit wordpress works good.
Hope you will help me =)
Sorry for my english. I hope you understand me =)

Could not extract from javascript n__ function

n__('%d plum','%d plums',5);

Why Extractors\PhpCode have not defined _ as a valid function in public static $functions?

I have edited your code to add support to default php gettext translations in Gettext\Extractors\PhpCode:

    public static $functions = array(
        '_' => '_',
        '__' => '__',
        '__e' => '__',
        'n__' => 'n__',
        'n__e' => 'n__',
        'p__' => 'p__',
        'p__e' => 'p__'
    );

Now I can load all strings from my code.

Hi guys.

I have one problem with your project and php ver. 5.5. When i used 5.3 or 5.4 - all works fine, but when i updated php to 5.5 - native translator (class GettextTranslator, vendor\gettext\gettext\src\GettextTranslator.php) stop working for me. If i use php Translator (vendor\gettext\gettext\src\Translator.php) - all is ok again, but i need exactly in native translator.

Test string:

Original string: Контакты
Translated string (en): Contacts
Translated string (de): Kontakte

Files structure:

index.php
data
  \- locales
     \- de
        \- LC_MESSAGES
           |- messages.mo
           \- messages.po
     \- en
        \- LC_MESSAGES
           |- messages.mo
           \- messages.po

Test code:

$translator = new \Gettext\GettextTranslator();
$translator->setLanguage('de')->loadDomain(
  'messages',
  'data/locales'
);
die($translator->gettext('Контакты'));

First test:

$ php -v
PHP 5.4.45 (cli) (built: Sep  2 2015 23:48:30)
Copyright (c) 1997-2014 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2014 Zend Technologies

Output:

Kontakte

All is ok. Second test:

$ php -v
PHP 5.5.33 (cli) (built: Mar  2 2016 15:19:21)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2015 Zend Technologies

Output:

Контакты

My string is not translated. Test code was not changed.

Gettext installed by composer and source has no any changes by me:

{
  "require": {
    "php": ">=5.5",
    "gettext/gettext": "3.*@dev",
  },
}

Server software:

• Apache 2.4
• PHP 5.3, 5.4, 5.5
• Windows

Can you fix this or say me how can i do it?

There is a problem extracting translation functions when they are used as params of other

foo(array('bar' => __('Array VALUE')));

Let's say we have a .po file like this:

msgid   "Address 1"
msgstr  ""

When we read this .po file, we split the 'msgid "Address 1"' string in two:

• 'msgid'
• ' "Address 1"'
  Since the second line starts with a space, the clean function behaves incorrectly.

Hi, I'm having trouble generating .mo files from an Entries-Object:

use Gettext\Extractors\Po as PoExtractor;
use Gettext\Generators\Po as PoGenerator;
use Gettext\Generators\Mo as MoGenerator;

$file = "/my/path/to/file.po";
$entries = PoExtractor::extract($file);
$entries->setHeader("Last-Translator", $user->getName() . " <" . $user->getEmail() . ">");
// editing translations
// ...

// create PO-File
$poLines = PoGenerator::generate($entries);
file_put_contents($file, $poLines);

// create MO-File
$moLines = MoGenerator::generate($entries);
file_put_contents(str_replace(".po", ".mo", $file), $moLines);

Everything works, except, that the used .mo file doesn't work in the application, it shows the msgid's and not the translated msgstr's :-(

Maybe I misunderstood something? Have you any suggestions?
Thanks in advance...

this entry will not be exported to PO files:

        $entry = $translations->find(null, "key") ? : $translations->insert(null, "key");
        $entry->setTranslation("");
        Gettext\Generators\Po::toFile($translations, $fn);

Po-Files can have multiline strings:

#. Description of the plugin/theme
msgid ""
"Twenty Sixteen is a modernized take on an ever-popular WordPress layout — "
"the horizontal masthead with an optional right sidebar that works perfectly "
"for blogs and websites. It has custom color options with beautiful default "
"color schemes, a harmonious fluid grid using a mobile-first approach, and "
"impeccable polish in every detail. Twenty Sixteen will make your WordPress "
"look beautiful everywhere."
msgstr ""

This makes it more readable in the IDE than a looooong single-line string.

The js style array declaration on line 78 of Translator.php is not compatible with php 5.3.

In Gettext\Translator.php Line 9

    private $context_glue = '\u0004';

It's not really escaped, tested as following:

var_dump('\u0004'); // string(6) "\u0004" 
var_dump("\04"); // string(1) "�"

I would propose a feature where $translations1 imports existing translations from $translations2, but doesn’t add new messages which are not present in $translations1.

For example, if $translations1 contains

{
     "dog" : "Hund",
     "cat" : ""
}

and $translations2 contains

{
     "cat" : "Katze",
     "mouse" : "Maus"
}

we would have $translations1->mergeWith($translations2, Merge::UPDATE) which would only import the translation for cat and ignore mouse.

In version 4.0.1, the generator does not use the passed $options array when calling toString:

https://github.com/oscarotero/Gettext/blob/master/src/Generators/Generator.php#L14

Obviously, this line should read

$content = static::toString($translations, $options);

The method “Gettext\Generators\Jed” does not appear to create a valid Jed Json structure. For non-plural translations it returns an extra empty string in the translation array:

{
  "messages": {
    "orig singular 1": [
      "",
      "trans singular 1"
    ],
    "orig singular 2": [
      "orig plural 2",
      "trans singular 2",
      "trans plural 2"
    ]
  }
}

To work properly with Jed apparently requires the following format:

{
  "messages": {
    "orig singular 1": [
      "trans singular 1"
    ],
    "orig singular 2": [
      "trans singular 2",
      "trans plural 2"
    ]
  }
}

This patch to src/Generators/PhpArray:toArray method seems to resolve the issue:

/*
        foreach ($translations as $translation) {
            $key = ($translation->hasContext() ? $translation->getContext().$context_glue : '').$translation->getOriginal();
            $entry = array($translation->getPlural(), $translation->getTranslation());

            if ($translation->hasPluralTranslation()) {
                $entry = array_merge($entry, $translation->getPluralTranslation());
            }

            $array[$key] = $entry;
        }
*/
        foreach ($translations as $translation) {
            $key = ($translation->hasContext() ? $translation->getContext().$context_glue : '').$translation->getOriginal();

            if ($translation->hasPluralTranslation()) {
                $array[$key] = array_merge(array($translation->getTranslation()), $translation->getPluralTranslation());
            } else {
                $array[$key] = array($translation->getTranslation());
            }
        }

Doing something wrong or is a patch needed?

Currently, while reading a .po file, the comments starting with

• # translator-comments
• #. extracted-comments
• #, flag

Are all handled as if they where translator-comments.
There should be a way to keep them separated...

When using function d__($domain, $original), I get the following error message:

[11-Nov-2015 00:11:16 America/New_York] PHP Fatal error:  Call to undefined method Gettext\BaseTranslator::dgettext() in /Applications/AMPPS/www/et/libs/gettext/src/translator_functions.php on line 78

Line 78 is:

$text = BaseTranslator::dgettext($domain, $original);

Hi oscarotero,

first of all thank you for great package. You've done great work :)

in class "Gettext\Generators\Po" you have "quote" method. In case of having tabs in original or translation string, method will not convert it to \t but editors do (same for "Gettext\Extractors\Po" ->clean).

Could you please add "\t" handling to this methods, or at least make "quote" and "clean" protected instead of private, so programmer can change this methods behavior.

Thank you.

Is it possible to use the twig i18n extension or has anyone managed to make a different extension that allows us to use gettext/gettext and have php arrays as translation?

{% trans %}
    Hey {{ name }}, I have one apple.
{% plural apple_count %}
    Hey {{ name }}, I have {{ count }} apples.
{% notes %}
    This is shown in the user menu. This string should be shorter than 30 chars
{% endtrans %}

If not, I will have to refactor my code a bit, but I especially liked using the twig variables inside the trans block (gets converted with strtr automatically).

The plural functions, such as ngettext, in the Translator class throw an exception when translations are not loaded. On line 247, instead of throwing an exception, it should probably just fallback on the default behavior, ie:

if (!isset($this->plurals[$domain])) {
    $this->plurals[$domain] = array('count' => 2, 'code' => ' return ($n != 1);;');
}`

I have updated from 23795c3 to 97b3d05 and I get this suggestion:

gettext/languages suggests installing php>=5.4 (In order to export with prettyjson you need at least PHP 5.4+)

But I'm using PHP 5.5.9-1ubuntu4.5

Hi! =)
Here https://github.com/oscarotero/Gettext/blob/master/tests/files/po.po
you show few examples, but there is no example with multiline translation such as

: /var/www/test/test.php:96

msgid "{test1}"
msgstr ""
"test1\n"
"

1. Why I say about that?
   Not so far I added an issue that i need support of \n in my strings and you did it.
   And your lib make multiline strings such as msgstr "test1\n
   \n test2\n
   \n test3"
   But it isn't valid for converting to mo.
   https://www.gnu.org/software/gettext/manual/html_node/PO-Files.html (see end of this page)
   Because I have multiline translation such as
   msgstr "test1\n
   \n test2\n
   \n test3"
   wordpress cuts this translation and echo only 'test1' to browser.
   Desktop program Poedit makes multiline msgstr as i show in the beginning of issue and makes a valid mo. If i try to convert it to mo with your lib it cuts only to 'test1' >:(

Help to solve this problem. Please tell me if I wrong =)

P.s. This editor broke my text) Here is screen of original
http://picsee.net/upload/2015-01-16/1d3b04ab37aa.png