Git Product home page Git Product logo

po2json's Introduction

po2json

Build Status Dependency Status devDependency Status

Convert PO files to Javascript objects or JSON strings. The result is Jed-compatible.

Getting Started

Install the module with: npm install po2json

As a library

var po2json = require('po2json');

As an executable

po2json translation.po translation.json

If you are using Jed >= 1.1.0, be sure to specify that format specifically.

po2json translation.po translation.json -f jed1.x

Documentation

Methods

po2json has 3 methods, all of which take exactly the same options. The main function is parse which actually does the parsing to JSON. The 2 others - parseFile and parseFileSync are convenience functions to directly read PO data from a file and convert it to JSON.

Parse a PO buffer to JSON

  • po2json.parse(buf[, options])
    • buf - a po file as a Buffer or an unicode string.
    • options - an optional object with the following possible parameters:
      • fuzzy Whether to include fuzzy translation in JSON or not. Should be either true or false. Defaults to false.
      • stringify If true, returns a JSON string. Otherwise returns a plain Javascript object. Defaults to false.
      • pretty If true, the resulting JSON string will be pretty-printed. Has no effect when stringify is false. Defaults to false
      • format Defaults to raw.
        • raw produces a "raw" JSON output
        • jed produces an output that is 100% compatible with Jed >= 1.1.0
        • jedold produces an output that is 100% compatible with Jed < 1.1.0
        • mf produces simple key:value output.
      • domain - the domain the messages will be wrapped inside. Only has effect if format: 'jed'.
      • fallback-to-msgid If true, for those entries that would be omitted (fuzzy entries without the fuzzy flag) and for those that are empty, the msgid will be used as translation in the json file. If the entry is plural, msgid_plural will be used for msgstr[1]. This means that this option makes sense only for those languages that have nplurals=2.

Parse a PO file to JSON

  • po2json.parseFile(fileName[,options], cb)
    • fileName - path to the po file
    • options - same as for po2json.parse
    • cb - a function that receives 2 arguments: err and jsonData

Parse a PO file to JSON (synchronous)

  • po2json.parseFileSync(fileName[, options])
    • fileName - path to the po file
    • options - same as for po2json.parse

Command Line Arguments

po2json in command-line parametrization support added to allow override default options.

  • --pretty, -p: same as pretty = true in function options
  • --fuzzy, -F: same as fuzzy = true in function options
  • --format, -f: Output format (raw, jed, jedold, or mf)
  • --full-mf, -M: return full messageformat output (instead of only translations)
  • --domain, -d: same as domain in function options
  • --fallback-to-msgid': 'use msgid if translation is missing (nplurals must match)

Note: 'format': 'mf' means the json format used by messageFormatter in github.com/SlexAxton/messageformat.js and jedold refers to Jed formats below 1.1.0

Examples

Basic usage with PO data as a buffer/string

var po2json = require('po2json'),
    fs = require('fs');
fs.readFile('messages.po', function (err, buffer) {
  var jsonData = po2json.parse(buffer);
  // do something interesting ...
});

Parse a PO file directly - Asynchronous Usage

var po2json = require('po2json');
po2json.parseFile('messages.po', function (err, jsonData) {
    // do something interesting ...
});

Parse a PO file directly - Synchronous Usage

var po2json = require('po2json');
var jsonData = '';
try {
    jsonData = po2json.parseFileSync('messages.po');
    // do something interesting ...
} catch (e) {}

Parse a PO file to messageformat format

var po2json = require('po2json'),
    MessageFormat = require('messageformat');

po2json.parseFile('es.po', { format: 'mf' }, function (err, translations) {
    var pFunc = function (n) {
      return (n==1 ? 'p0' : n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 'p1' : 'p2');
    };
    pFunc.cardinal = [ 'p0', 'p1', 'p2' ];
    var mf = new MessageFormat(
      {
        'es': pFunc
      }
    );
    var i18n = mf.compile( translations );
});

Parse a PO file to messageformat format using the full format

var po2json = require('po2json'),
    MessageFormat = require('messageformat');

po2json.parseFile('messages.po', { format: 'mf', fullMF: true }, function (err, jsonData) {
    var mf = new MessageFormat(
        { [jsonData.headers.language]: jsonData.pluralFunction }
    );
    var i18n = mf.compile( jsonData.translations );
});

Parse a PO file to Jed >= 1.1.0 format

var po2json = require('po2json'),
    Jed = require('jed');
po2json.parseFile('messages.po', { format: 'jed' }, function (err, jsonData) {
    var i18n = new Jed( jsonData );
});

Parse a PO file to Jed < 1.1.0 format

If you are using an older version of Jed, be sure to specify this format specifically.

var po2json = require('po2json'),
    Jed = require('jed');
po2json.parseFile('messages.po', { format: 'jedold' }, function (err, jsonData) {
    var i18n = new Jed( jsonData );
});

Running tests

npm test

Contributing

In lieu of a formal styleguide, take care to maintain the existing coding style. Add unit tests for any new or changed functionality. Lint and test your code using grunt.

Release History

1.0.0 / 2018-09-24

  • Updated dependencies.
  • Replaced nomnom with commander.
  • Switched formats values for Jed versions. jed now refers to versions >= 1.1.0.
  • Added tests for MessageFormat compilation.

0.4.6 / 2018-09-24

  • Add documentation and tests for different Jed versions.
  • Updated dependencies.

0.4.5 / 2016-10-13

  • Fixed issue with jed 1.x and ja-JP pluralization.

0.4.4 / 2016-08-29

  • Fix parsing of .po files with no headers.

0.4.3 / 2016-08-27

  • Better support for plurals in languages like Japanese.

0.4.2 / 2015-04-13

  • Updated documentation for Jed > 1.1.0
  • Use msgid_plural when there is no translation

0.4.1 / 2015-03-01

  • Updated Jed-format code and test to deal with the new plural form

0.4.0 / 2015-03-01

  • Added Jed > 1.1.0 compatible format (Evan Moses)

0.3.0 / 2014-07-16

  • Added command line flags for fuzzy, pretty, format, and domain (Szigetvári Áron)
  • Deals with fallback-to-msgid for fuzzy entries without the fuzzy flag (Szigetvári Áron)

0.2.4 / 2014-07-15

  • Fixed fuzzy flag (mahata)

0.2.3 / 2014-01-26

  • Raised minimum node version requirement to 0.8
  • Raised lodash version to ~2.4.1
  • Clean up documentations

0.2.0 / 2013-11-08

NB! This release is NOT backwards-compatible! It has the following breaking changes:

  • po2json.parse_po has been replaced with po2json.parse
  • po2json.parse has been replaced with po2json.parseFile
  • po2json.parseSync has been replaced with po2json.parseFileSync

Other changes in this release:

  • The library has been competely rewritten, it now uses the gettext-parser module to parse PO files. (Illimar Tambek)
  • Tests have been completely rewritten (Illimar Tambek)
  • Fixed issue with double-escaping quotes (Illimar Tambek)
  • Option to skip/include fuzzy translations (Illimar Tambek)

0.0.7 / 2012-10-26

  • Fixed linting bugs and added a better fr.po fixture (Mike Edwards)
  • Add tests for po2json.parse and po2json.parseSync (Dan MacTough)
  • updated README.md with version history (Mike Edwards)
  • updated history (Mike Edwards)

0.0.6 / 2012-10-22

  • Add AUTHORS to identify contributors (Dan MacTough)
  • Update README with revision history and basic examples (Dan MacTough)

0.0.5 / 2012-10-19

  • cut out fake README example from grunt boilerplate (Mike Edwards)
  • fixed README.md markdown (Mike Edwards)
  • fixes tests (Mike Edwards)
  • added first test for parse_po (Mike Edwards)
  • Added boilerplate using grunt init (Mike Edwards)
  • Changed exports.parse to use node's convetional error-first callback style. Added exports.parseSync for synchronous parsing. (Dan MacTough)

0.0.4 / 2012-09-18

  • Properly escape linebreaks (Zach Carter)
  • Update package.json (Mike Edwards)
  • package.json: define main module (Asbjørn Sloth Tønnesen)

0.0.2 / 2012-07-03

  • fix package, fix pretty print return, remove debug logs (gilles)
  • upped version (Mike Edwards)

0.0.1 / 2012-06-06

  • Added build status to README (Mike Edwards)
  • Removed built=ints from the dependencies (Mike Edwards)
  • Added a .travis file for continuous integration (Mike Edwards)
  • Added usage note to README.md (Mike Edwards)
  • First working script! (Mike Edwards)
  • Added new git repo (Mike Edwards)
  • initial commit (Mike Edwards)
  • Initial commit (Mike Edwards)

License

Copyright (c) 2012 Joshua I. Miller Licensed under the GNU, Library, General, Public, License licenses.

po2json's People

Contributors

abemedia avatar andrey-skl avatar andris9 avatar danmactough avatar dependabot[bot] avatar dmitriid avatar emosessfdc avatar gilles avatar granjow avatar guillaumepotier avatar invi avatar koorgoo avatar mikeedwards avatar mugami-ast avatar mwiencek avatar rafaltrojanowski avatar ragulka avatar tg-x avatar zaach avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

po2json's Issues

Cannot install

While trying to install with npm install po2json

npm WARN package.json [email protected] No README.md file found!
npm http GET https://registry.npmjs.org/po2json
npm http 304 https://registry.npmjs.org/po2json
npm ERR! Error: ENOENT, chmod '/Library/WebServer/Documents/slipfeed/node_modules/po2json/bin/handlebars-xgettext'
npm ERR! If you need help, you may report this log at:
npm ERR!     <http://github.com/isaacs/npm/issues>
npm ERR! or email it to:
npm ERR!     <[email protected]>

npm ERR! System Darwin 12.1.0
npm ERR! command "node" "/usr/local/bin/npm" "install" "po2json"
npm ERR! cwd /Library/WebServer/Documents/slipfeed
npm ERR! node -v v0.8.5
npm ERR! npm -v 1.1.46
npm ERR! path /Library/WebServer/Documents/slipfeed/node_modules/po2json/bin/handlebars-xgettext
npm ERR! code ENOENT
npm ERR! errno 34
npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     /Library/WebServer/Documents/slipfeed/npm-debug.log
npm ERR! not ok code 0

Using msgctxt

We currently have po2json in a system that we're adding msgctxt for each translation string.

the current output from po2json prepends the context to the id:

'label\u0004Open the QR code on your mobile device.': [ null, '' ],

Is this the expected output? As far as I can see, I would have to then split the id on \u0004 and test to see what the length of that was to see if there was a context in the output id but this seems like a bit of a kludge.

fuzzy and comments in po disappear on json.

Expected Behavior

Fuzzy and comments are added on generated json.

Actual Behavior

They are ignored

Steps to Reproduce

po2json a file with po comments and fuzzy flags.

Additional Data

Po file:

#: app\ui\apps\about\about.html
msgid "STR_test1"
msgstr "test1original"

#: app\ui\dashboard\app-area\app-area.html
msgid "STR_test2"
msgstr "test2original"

#: app\ui\apps\media-library\media-detail.html
#, fuzzy
msgid "test3"
msgstr "original"

#. This is a comment for translators.
#: test2
msgid "STR_JobStatus"
msgstr "Status"

Motivation: People may use JSON as middleware to convert to other formats, or build apps over json files, .. it's information on po file that's not being added.

Wrong output for Jed

I am using po2json to create JSON files for use with Jed. My po looks like this:

# French translations for PACKAGE package.
#
msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2016-01-05 16:44-0500\n"
"PO-Revision-Date: 2016-01-05 16:48-0500\n"
"Last-Translator: Patrick Fournier <...>\n"
"Language-Team: French\n"
"Language: fr\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=ASCII\n"
"Content-Transfer-Encoding: 8bit\n"
"Plural-Forms: nplurals=2; plural=(n > 1);\n"

#: ../app/routes/content.js:8
msgid "Content"
msgstr "Contenu"

#: ../app/routes/content.js:10
msgid "Potato"
msgid_plural "Potatoes"
msgstr[0] "Patate"
msgstr[1] "Patates"

The generated JSON file is:

{
   "domain": "messages",
   "locale_data": {
      "messages": {
         "": {
            "domain": "messages",
            "plural_forms": "nplurals=2; plural=(n > 1);",
            "lang": "fr"
         },
         "Content": [
            null,
            "Contenu"
         ],
         "Potato": [
            "Potatoes",
            "Patate",
            "Patates"
         ]
      }
   }
}

This does not work with Jed because there is an extra item at the beginning of each translation list, which correspond to the msgid_plural key. The expected JSON would be:

{
   "domain": "messages",
   "locale_data": {
      "messages": {
         "": {
            "domain": "messages",
            "plural_forms": "nplurals=2; plural=(n > 1);",
            "lang": "fr"
         },
         "Content": [
            "Contenu"
         ],
         "Potato": [
            "Patate",
            "Patates"
         ]
      }
   }
}

I traced the error to parse.js. I can create a pull request for this, but first, I would like to know if the current output is only an error for the Jed format, or if it is also an error for the RAW format?

Headers are output in inconsistent order

Hi,

I just started using po2json and I can see some behaviour where the headers are output in a different order on different runs (I'm not sure what the order depends on yet). This isn't really a problem but causes some unnecessarily noisy diffs when the JSON file is stored in version control.

https://github.com/mikeedwards/po2json/blob/v0.4.1/lib/parse.js#L62

Is this behaviour expected? Would you accept a pull request that returns the headers in a consistent order?

Thanks!


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

npm install fails

[root@ip-10-119-29-109] ~ $ npm install po2json
npm http GET https://registry.npmjs.org/po2json
npm http 304 https://registry.npmjs.org/po2json
npm http GET https://registry.npmjs.org/fs
npm http GET https://registry.npmjs.org/path
npm http 404 https://registry.npmjs.org/fs
npm ERR! 404 'fs' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it
npm ERR! 404
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, or http url, or git url.

npm ERR! System Linux 2.6.35-24-virtual
npm ERR! command "/usr/local/bin/node" "/usr/local/bin/npm" "install" "po2json"
npm ERR! cwd /root
npm ERR! node -v v0.8.0
npm ERR! npm -v 1.1.32
npm ERR! code E404
npm ERR! message 404 Not Found: fs
npm http 304 https://registry.npmjs.org/path
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /root/npm-debug.log
npm ERR! not ok code 0

Thanks for taking a look

import error po2json

I am using react and webpack.
I am trying to import po2json like:

var po2json = require('po2json')

but it is giving me error like:

ERROR in ./~/po2json/lib/parseFile.js
Module not found: Error: Cannot resolve module 'fs' in /home/anonymous/project/node_modules/po2json/lib

Why I am getting this error ?

But no error on importing Jed

var Jed = require('jed')

Fuzzy entries are ignored

Expected Behavior

Entries marked as fuzzy appear in the json output

Actual Behavior

Entries marked as fuzzy don't appear in the output

Steps to Reproduce

po2json test.po test.json
cat test.json
{"OK":"OK"}⏎  

Additional Data

test.po

#: somethng/something.js:96
msgid "OK"
msgstr "OK"

#: somethng/something.js:95
#, fuzzy
msgid "NOT OK"
msgstr "NOT OK"

The entries were marked as fuzzy using Weblate "Needs editing"

image

How do I actually save the output?

I get [object Object] no matter whether i parse the buffer or not. I see the content when I output the buffer to the console. But can't get it into a proper file.

po2json.parseFile('./source/da.po', function ( err, buffer ) {
	var jsonData = po2json.parse(buffer);

	fs.writeFile('translation.json', jsonData );

also tried:

`fs.writeFile('translation.json', JSON.stringify( jsonData ) );`

gives empty object.

Licensing issues

This library is GPL v2 licensed. When bundled in a front end application this will trigger clauses in the license which will force us to GPL license our frontend application, as we are effectively redistributing the source code by transferring it to the clients browser.

Is this intentional, or would it be possible to evaluate alternate licenses in able to let people use this library in front end applications?

Other than that, thanks for the great library!

Jed's changed its format, 'jed' format no longer compatible

Jed was updated so that it no longer has the first "null" in the translated string arrays, i.e., it used to look like

"key" : [null, "value", "values] //etc.

but now looks like

"key" : ["value", "values"]

Jed format needs to be updated to match, or possibly a new jed1.x format

strip slashes

A .po file generated with Virtaal add slashes:

test.po

msgid ""
"Hallo \\\"Welt\\\", das ist ein Test"
msgstr ""
"Hello \\\"World\\\", this is a test"

po2json add more slashes:

test.json

"Hallo \\\\\"Welt\\\\\", das ist ein Test": [
   null,
   "Hello \\\\\"World\\\\\", this is a test"
],

Unexpected output when parsing buffers

Steps to reproduce:

  1. Create the following de.po file:
#: app/scripts/templates/change_password.mustache:12
#: app/scripts/templates/delete_account.mustache:9
#: app/scripts/templates/pp.mustache:4
#: app/scripts/templates/reset_password.mustache:6
#: app/scripts/templates/reset_password.mustache:8
#: app/scripts/templates/tos.mustache:4
msgid "Back"
msgstr "Zurück"

#: app/scripts/templates/sign_up.mustache:10
msgid "1990 or earlier"
msgstr "1990 oder früher"
  1. Create the following Node.js test script:
// index.js
var po2json = require('po2json'),
    fs = require('fs'),
    jsonData = '';

// 1
fs.readFile('de.po', function (err, buffer) {
  var jsonData = po2json.parse(buffer);
  console.log("1. %j", jsonData);
});

// 2
po2json.parseFile('de.po', function (err, jsonData) {
    console.log("2. %j", jsonData);
});

// 3
try {
    jsonData = po2json.parseFileSync('de.po');
    console.log("3. %j", jsonData);
} catch (e) {
}
  1. Run the test script: $ node index

Actual results:

1. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}
2. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}
3. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}

Expected results:

1. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}
2. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}
3. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}

Workaround:

// 1
fs.readFile('de.po', function (err, buffer) {
  var jsonData = po2json.parse(buffer.toString());
  console.log("1. %j", jsonData);
});

Output:

1. {"Back":[null,"Zurück"],"1990 or earlier":[null,"1990 oder früher"]}

When calling toString() on the buffer, I get the expected output, but that doesn't work for the po2json.parseFile() or po2json.parseFileSync() methods.

Not sure if we can fix by just explicitly calling .toString() on the buffer in /lib/parse.js:

var parsed = require('gettext-parser').po.parse( buffer.toString() );

Version 1 alpha taged as latest in NPM breaks stuff

Hi !

To begin with, thanks for this new version !

This new alpha version has been tagged as the latest version in npm. Some libraries like po-loader that uses this library has a dependency to >= 0.4.0 so it uses the new alpha package.

As meant by the alpha tag, it probably has some bugs (see my other ticket) and it breaks po-loader.

Would you mind to tag the alpha version as next instead of latest in NPM ?

Expected Behavior

npm install po2json to not install an alpha version.

Actual Behavior

npm install po2json installs an alpha version.

Steps to Reproduce

Try to install po-loader and see which version of po2json is installed.

Jed converting error

If you use po2json with the jed format and po entries don't have a plural form, singular entries become plural entries. E.g. the po file

msgid ""
msgstr ""
"Plural-Forms: nplurals=2; plural=(n != 1);\n"

msgid "Let's make the web multilingual."
msgstr "Machen wir das Internet mehrsprachig."

msgid "We connect developers and translators around the globe "
"on Lingohub for a fantastic localization experience."
msgstr "Wir verbinden Entwickler mit Ãœbersetzern weltweit "
"auf Lingohub für ein fantastisches Lokalisierungs-Erlebnis."

msgid "%d page read."
msgid_plural "%d pages read."
msgstr[0] "Eine Seite gelesen wurde."
msgstr[1] "%d Seiten gelesen wurden."

will be translated to a json jsonData where jsonData.local_data.messages is

{ '':
   { domain: 'messages',
     plural_forms: 'nplurals=2; plural=(n != 1);',
     lang: undefined },
  'Let\'s make the web multilingual.': [ null, 'Machen wir das Internet mehrsprachig.' ],
  'We connect developers and translators around the globe on Lingohub for a fantastic localization experience.':
   [ null,
     'Wir verbinden Entwickler mit Übersetzern weltweit auf Lingohub für ein fantastisches Lokalisierungs-Erlebnis.' ],
  '%d page read.':
   [ '%d pages read.',
     'Eine Seite gelesen wurde.',
     '%d Seiten gelesen wurden.' ] }

So the first entry in the array becomes null and this is not correct.

Invalid output for jed 1.x and ja-JP pluralization

Using 0.4.4. Entry in the .po file

msgid ""
msgstr ""
"Project-Id-Version: \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2016-10-07 15:13-0700\n"
"PO-Revision-Date: 2016-10-07 14:35-0700\n"
"Last-Translator: \n"
"Language-Team: \n"
"Language: ja_JP\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: Poedit 1.8.9\n"
"X-Poedit-Basepath: .\n"
"Plural-Forms: nplurals=1; plural=0;\n"

#: dist/containers/Search.js:96
#, c-format
msgid "For %s person starting at %s"
msgid_plural "For %s people starting at %s"
msgstr[0] "For %1$s people starting at %2$s"

Json output:

        "For %s person starting at %s": [
            [
               "For %1$s people starting at %2$s"
            ]
         ],

French for example looks like this

        "For %s person starting at %s": [
            "For %1$s person starting at %2$s",
            "For %1$s people starting at %2$s"
         ],

About dependencies in package.json

Expected Behavior

What did you want to happen?

Actual Behavior

In package.json,have three dependencies:

  "dependencies": {
    "commander": "^2.18.0",
    "gettext-parser": "2.0.0",
    "gettext-to-messageformat": "^0.3.0"
  },
  "devDependencies": {
    "jed": "~1.1.1",
    "messageformat": "^2.0.4",
    "nodeunit": "*"
  },

Steps to Reproduce

When I use po2json on my project,it will install two dependencies which has two different version like "[email protected]" and "[email protected]", and will both build in my production Javascript files.

Additional Data

change to peerdependencies will fix this problem。

  "peerDependencies": {
    "commander": "^2.18.0",
    "gettext-parser": "2.0.0",
    "gettext-to-messageformat": "^0.3.0"
    "jed": "~1.1.1",
    "messageformat": "^2.0.4",
    "nodeunit": "*"
  },

Push latest to NPM

Could you push the latest version of this module to NPM please?

Commit 7dd200e updated the dependency on gettext-parser from 0.1.9 to 0.2.0, which actually turns out to be a big deal since 0.1.9 depends on a native module iconv whereas 0.2.0 does not.

`mf` format can't handle plurals

Expected Behavior

I have this entry in my po file -

#: test/registry.py:63
msgid "You only have %1 lesson left to finish %2."
msgid_plural "You only have %1 lessons left to finish %2."
msgstr[0] "asd %1 %2"
msgstr[1] "asd %1 %2"
msgstr[2] "asd %1 asd %2"

#: test/registry.py:68
msgid "An access token is required to access this resource"
msgstr ""

which should convert to a format that works fine with both gettext() and ngettext() in gettext.js.

Actual Behavior

If I use raw format, it generates -

{ 
	"You only have %1 lesson left to finish %2.": [
      "You only have %1 lessons left to finish %2.",
      "asd %1 %2",
      "asd %1 %2",
      "asd %1 asd %2"
   ],
   "An access token is required to access this resource": [
      null,
      ""
   ],
   ...
}

Which, when using gettext.js, doesn't work with gettext(), and only works with ngettext(%s, 1) even for the singular value as the second string.

If I use mf format, it generates -

{
   "You only have %1 lesson left to finish %2.": "asd %1 %2",
   "An access token is required to access this resource": "",
   ...
}

Which works fine with gettext() in gettext.js, but the plural values are ignored when converting to json.

Steps to Reproduce

after following recommended workflow.

Domain not included in Json file breaks Jed

I started testing Jed today for a project that needs translations both client side and server side, and I'm using po2json to convert from po files.

I've got some example .po files in the following locations (I'm presuming they are valid .po files)

http://svn.opensuse.org/svn/opensuse-i18n/trunk/
https://github.com/drupal/drupal/blob/7.x/modules/locale/tests/translations/test.xx.po

However, the compiled file doesn't work straight away, it needs to be amended with some extra domain info (as per the JED documentation) to make it work, even when passing the -d option.

I get the same problem regarding of converting in the command line or inside node.js.

Error converting some files

I am trying to convert all our old .po files to .json and it works fine for almost all the files but two. se_SV.po and nb_NO.po

All .po files are identical but the msgstr key

Script:

var path = require('path');
var fs = require('fs');
var shell = require('shelljs');

function fromDir(startPath, filter) {
  if (!fs.existsSync(startPath)) {
    console.log("no dir ", startPath);
    return;
  }

  var files = fs.readdirSync(startPath);
  for (var i = 0; i < files.length; i++) {
    var filename = path.join(startPath, files[i]);
    var stat = fs.lstatSync(filename);
    if (stat.isDirectory()) {
      fromDir(filename, filter); //recurse
    }
    else if (filename.indexOf(filter) >= 0) {
      const split = filename.split('/')[1].split('.')[0]
      //console.log(split)
      const string = './node_modules/.bin/po2json ' + filename + ' ./locale/json/'+split+'.json -f mf -p'
      console.log("Converting", filename, "to json format")
      shell.exec(string)
    }
  }
}

fromDir('locale', '.po');

Error:

/path/to/my/repo/node_modules/gettext-parser/lib/poparser.js:393
            table.headers = sharedFuncs.parseHeader(tokens[i].msgstr[0]);
                                                                    ^

TypeError: Cannot read property '0' of undefined
    at Parser._normalize (/path/to/my/repo/node_modules/gettext-parser/lib/poparser.js:393:69)
    at Parser._finalize (/path/to/my/repo/node_modules/gettext-parser/lib/poparser.js:414:17)
    at Parser.parse (/path/to/my/repo/node_modules/gettext-parser/lib/poparser.js:63:17)
    at Object.module.exports.parse (/path/to/my/repo/node_modules/gettext-parser/lib/poparser.js:17:19)
    at module.exports (/path/to/my/repo/node_modules/po2json/lib/parse.js:27:45)
    at Object.module.exports [as parseFileSync] (/path/to/my/repo//node_modules/po2json/lib/parseFileSync.js:14:28)
    at Object.<anonymous> (/path/to/my/repo//node_modules/po2json/bin/po2json:50:22)
    at Module._compile (module.js:569:30)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:503:32)

.po-file header

I've tries with the file header from #30 but still doesn't work

"Project-Id-Version: messages\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2015-08-03 16:00+0100\n"
"PO-Revision-Date: 2015-08-09 22:06+0100\n"
"Last-Translator: NAME <[email protected]>\n"
"Language-Team: NAME <[email protected]>\n"
"Language: sv\n"
"MIME-Version: 1.0\n"
"Content-Type: text/plain; charset=UTF-8\n"
"Content-Transfer-Encoding: 8bit\n"
"X-Generator: Poedit 1.8.1\n"

Invalid executable file `bin/po2json` which bundled npm package

Expected Behavior

node_modules/.bin/po2json is executable.

Actual Behavior

node_modules/.bin/po2json exits non-zero status and says:

$ ./po2json -h
env: node\r: No such file or directory

Steps to Reproduce

po2json package version: 1.0.0-alpha

$ uname -a
Darwin Ciconia.local 18.2.0 Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST 2018; root:xnu-4903.231.4~2/RELEASE_X86_64 x86_64

$ node -v
v11.8.0

$ npm -v
6.5.0

$ yarn -v
1.13.0

Additional Data

It seems to be a linebreak probrem.
If all \r\n replaced with \n, po2json works.
By the way, npx po2json works but I am not sure why.

A new stable version needed!

The "latest: 0.4.5" depends on "nomnom", which depends on a specific version of "underscore". At this version, "underscore" has a high severity vulnerability: https://npmjs.com/advisories/1674
A new stable version will make po2json depend on "commander" instead of "nomnom".

Different output in some languages when there is semicolon in header

There seems to be some inconsistency, about using semicolon at the end of plural-forms in .po headers. It's presence can break the format, that po2json produces. Let me explain with examples:

Let's have this file: (tmp.po)

  msgid ""                                                                        
  msgstr ""                                                                       
  "Project-Id-Version: PACKAGE VERSION\n"                                         
  "Language: ko\n"                                                                
  "MIME-Version: 1.0\n"                                                           
  "Content-Type: text/plain; charset=UTF-8\n"                                     
  "Content-Transfer-Encoding: 8bit\n"                                             
  "Plural-Forms: nplurals=1; plural=0\n"                                          
  "X-Generator: Weblate 3.10.1\n"                                                 
                                                                                  
  msgid "Combined usage of $0 CPU core"                                           
  msgid_plural "Combined usage of $0 CPU cores"                                   
  msgstr[0] "$0 CPU 코어의 총 사용량"

when I run ./node_modules/po2json/bin/po2json -p tmp.po tmp then tmp looks like this:

{                                                                               
   "": {                                                                        
      "project-id-version": "PACKAGE VERSION",                                  
      "language": "ko",                                                         
      "mime-version": "1.0",                                                    
      "content-type": "text/plain; charset=UTF-8",                              
      "content-transfer-encoding": "8bit",                                      
      "plural-forms": "nplurals=1; plural=0",                                   
      "x-generator": "Weblate 3.10.1"                                           
   },                                                                           
   "Combined usage of $0 CPU core": [                                           
      "Combined usage of $0 CPU cores",                                         
      "$0 CPU 코어의 총 사용량"                                                 
   ]                                                                            
}  

Which is correct. But let's now add semicolon to the "Plural-Forms: nplurals=1; plural=0\n" line.
Now the tmp.po file looks like this:

  msgid ""                                                                        
  msgstr ""                                                                       
  "Project-Id-Version: PACKAGE VERSION\n"                                         
  "Language: ko\n"                                                                
  "MIME-Version: 1.0\n"                                                           
  "Content-Type: text/plain; charset=UTF-8\n"                                     
  "Content-Transfer-Encoding: 8bit\n"                                             
  "Plural-Forms: nplurals=1; plural=0;\n"                                         
  "X-Generator: Weblate 3.10.1\n"                                                 
                                                                                                                                     
  msgid "Combined usage of $0 CPU core"                                           
  msgid_plural "Combined usage of $0 CPU cores"                                   
  msgstr[0] "$0 CPU 코어의 총 사용량" 

and when I run the same command, the tmp output is:

{                                                                               
   "": {                                                                        
      "project-id-version": "PACKAGE VERSION",                                  
      "language": "ko",                                                         
      "mime-version": "1.0",                                                    
      "content-type": "text/plain; charset=UTF-8",                              
      "content-transfer-encoding": "8bit",                                      
      "plural-forms": "nplurals=1; plural=0;",                                  
      "x-generator": "Weblate 3.10.1"                                           
   },                                                                           
   "Combined usage of $0 CPU core": [                                           
      "Combined usage of $0 CPU cores",                                         
      [                                                                         
         "$0 CPU 코어의 총 사용량"                                              
      ]                                                                         
   ]                                                                            
} 

So the translation for the string is not array of strings, but array of one string and one array.

Interestingly enough, if I have different file, like this:

  msgid ""                                                                        
  msgstr ""                                                                       
  "Project-Id-Version: PACKAGE VERSION\n"                                         
  "Language: cs\n"                                                                
  "MIME-Version: 1.0\n"                                                           
  "Content-Type: text/plain; charset=UTF-8\n"                                     
  "Content-Transfer-Encoding: 8bit\n"                                             
  "Plural-Forms: nplurals=3; plural=(n==1) ? 0 : (n>=2 && n<=4) ? 1 : 2\n"        
  "X-Generator: Weblate 3.10.1\n"                                                 
                                                                                  
  msgid "Combined usage of $0 CPU core"                                           
  msgid_plural "Combined usage of $0 CPU cores"                                   
  msgstr[0] "Kombinované využití $0 jádra procesoru"                              
  msgstr[1] "Kombinované využití $0 jader procesoru"                              
  msgstr[2] "Kombinované využití $0 jader procesoru"

The output is the same, no matter if there is semicolon or not on the Plural-Forms line.
From docs it seems there always should be semicolon (1, 2). This is likely problem in some library that po2json uses, but was not sure where it really comes from, so reporting here.
(side note: We had mix of some files having this semicolon and some don't for years and it seemed to work just fine. We were using Zanata to generate these files for us, now we migrated to Weblate and it adds this semicolon to some more languages (still not to all). So maybe this is known bug/documented somewhere)

[1-alpha] Over escaping braces

Expected Behavior

I'd like my msgstr containing braces not being escaped

msgid "test"
msgstr "Hi %{firstname}"
{ "test": "Hi %{firstname}" }

Actual Behavior

msgid "test"
msgstr "Hi %{firstname}"
{ "test": "Hi %\\{firstname\\}" }

Steps to Reproduce

The first example is taken from the 0.x version, the second one is after an (unwanted) update of the library (through po-loader) to the 1-alpha version.

Additional info

Here is my po-loader webpack loader configuration

          {
            loader: require.resolve('po-loader'),
            options: {
              format: 'mf', // Clean key value output
            }
          },

parsing commented out strings

Right now po2json extracts commented out strings and puts them in returned data without marking them in any way. I'm not sure if that's a bug or a feature, in my case marked strings are marked like this:

#~ "English"
#~ "Английский"

UTF-8 chars mangled

I have the following in my UTF-8 encoded .po file:

msgid "Start date"
msgstr "Data de início"

After running the CLI command, I get the following JSON (again UTF-8 encoded file)

"Start date":[null,"Data de início"]

Pretty sure this is something at my end, but any pointers would be great 😄

Invalid 'main' key

Expected Behavior

Simply run the program

Actual Behavior

Gor error for invalid 'main' key in package.json

Steps to Reproduce

po2json input.po outpuit.po

Additional Data

Can you please make a new release containing this latest fix?
51e2310

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.