Git Product home page Git Product logo

banana-i18n's Introduction

banana-i18n - Javascript Internationalization library

Build GitHub license npm version

banana-i18n is a javascript internationalization library that uses "banana" format - A JSON based localization file format.

Features

  • Simple file format - JSON. Easily readable for humans and machines.
  • Bindings and wrappers available for React.js and Vue.js. See frameworks sections.
  • Author and metadata information is not lost anywhere. There are other file formats using comments to store this.
  • Uses MediaWiki convention for placeholders. Easily readable and proven convention. Example: There are $1 cars
  • Supports plural conversion without using extra messages for all plural forms. Plural rule handling is done using CLDR. Covers a wide range of languages
  • Supports gender. By passing the gender value, you get correct sentences according to gender.
  • Supports grammar forms. banana-i18n has a basic but extensible grammar conversion support
  • Fallback chains for all languages.
  • Nestable grammar, plural, gender support. These constructs can be nested to any arbitrary level for supporting sophisticated message localization
  • Message documentation through special language code qqq
  • Extensible message parser to add or customize magic words in the messages. Example: {sitename} or [[link]]
  • Automatic message file linter using banana-checker
  • Tested in production - MediaWiki and and its extensions use this file format

Installation

npm i banana-i18n

Developer Documentation

Banana File format

The message files are json formatted. As a convention, you can have a folder named i18n inside your source code. For each language or locale, have a file named like languagecode.json.

Example:

App
    |--src
    |--doc
    |--i18n
        |--ar.json
        |--de.json
        |--en.json
        |--he.json
        |--hi.json
        |--fr.json
        |--qqq.json

A simple en.json file example is given below

{
    "@metadata": {
        "authors": [
            "Alice",
            "David",
            "Santhosh"
        ],
        "last-updated": "2012-09-21",
        "locale": "en",
        "message-documentation": "qqq",
        "AnotherMetadata": "AnotherMedatadataValue"
    },
    "appname-title": "Example Application",
    "appname-sub-title": "An example application with jquery.i18n",
    "appname-header-introduction": "Introduction",
    "appname-about": "About this application",
    "appname-footer": "Footer text"
}

The json file should be a valid json. The @metadata holds all kind of data that are not messages. You can store author information, copyright, updated date or anything there.

Messages are key-value pairs. It is a good convention to prefix your appname to message keys to make the messages unique. It acts as the namespace for the message keys. It is also a good convention to have the message keys with - separated words, all in lower case.

If you are curious to see some real jquery.i18n message file from other projects:

Loading the messages

The localized message should be loaded before using .i18n() method. This can be done as follows:

Using the constructor

const banana = new Banana('es',{
  messages: {
   'key-1': 'Localized message'
  }
})

Load the messages for a locale

After the initialization,

const messages = {
  'message-key-1': 'Localized message 1',
  // Rest of the messages
};
banana.load(messages, 'es' );

Load the messages for many locales at once

If you think it is convinient to load all messages for all locales at once, you can do this as follows. Here the messages are keyed by locale.

const messages = {
  'es': {
    'message-key-1': 'Localized message 1 for es',
    // Rest of the messages for es
  },
  'ru': {
    'message-key-1': 'Localized message 1 for ru',
    // Rest of the messages for ru
  }
};
banana.load(messages); // Note that the locale parameter is missing here

Depeding on your application, the messages can be fetched from a server or a file system. Here is an example that fetches the localized messages json file.

fetch('i18n/es.json').then((response) => response.json()).then((messages) => {
  banana.load(messages, 'es');
});

You may load the messages in parts too. That means, you can use the banana.load(message_set1, 'es') and later banana.load(message_set2, 'es'). Both of the messages will be merged to the locale. If message_2 has the same key of message_set1, the last message loaded wins.

Setting the locale

The constructor for Banana class accepts the locale

const banana = new Banana('es')

Once the banana i18n is initialized you can change the locale using setLocale method

banana.setLocale('es'); // Change to new locale

All .i18n() calls will set the message for the new locale from there onwards.

Fallback

If a particular message is not localized for locale, but localized for a fallback locale(defined in src/languages/fallbacks.json), the .i18n() method will return that. By default English is the final fallback language. But this configurable using finalFallback option. Example: new Banana('ru', {finalFallback:'es' })

Placeholders

Messages take parameters. They are represented by $1, $2, $3, … in the message texts, and replaced at run time. Typical parameter values are numbers (Example: "Delete 3 versions?"), or user names (Example: "Page last edited by $1"), page names, links, and so on, or sometimes other messages.

const message = "Welcome, $1";
banana.i18n(message, 'Alice'); // This gives "Welcome, Alice"

Plurals

To make the syntax of sentence correct, plural forms are required. jquery.i18n support plural forms in the message using the syntax {{PLURAL:$1|pluralform1|pluralform2|...}}

For example:

const message = "Found $1 {{PLURAL:$1|result|results}}";
banana.i18n(message, 1); // This gives "Found 1 result"
banana.i18n(message, 4); // This gives "Found 4 results"

Note that {{PLURAL:...}} is not case sensitive. It can be {{plural:...}} too.

In case of English, there are only 2 plural forms, but many languages use more than 2 plural forms. All the plural forms can be given in the above syntax, separated by pipe(|). The number of plural forms for each language is defined in CLDR. You need to provide all those plural forms for a language.

For example, English has 2 plural forms and the message format will look like {{PLURAL:$1|one|other}}. for Arabic there are 6 plural forms and format will look like {{PLURAL:$1|zero|one|two|few|many|other}}.

You cannot skip a plural form from the middle or beginning. However, you can skip from end. For example, in Arabic, if the message is like {{PLURAL:$1|A|B}}, for 0, A will be used, for numbers that fall under one, two, few, many, other categories B will be used.

If there is an explicit plural form to be given for a specific number, it is possible with the following syntax

const message = 'Box has {{PLURAL:$1|one egg|$1 eggs|12=a dozen eggs}}.';
banana.i18n(message, 4 ); // Gives "Box has 4 eggs."
banana.i18n(message, 12 ); // Gives "Box has a dozen eggs."

Gender

Similar to plural, depending on gender of placeholders, mostly user names, the syntax changes dynamically. An example in English is "Alice changed her profile picture" and "Bob changed his profile picture". To support this {{GENDER...}} syntax can be used as shown in example

const message = "$1 changed {{GENDER:$2|his|her}} profile picture";
banana.i18n(message, 'Alice', 'female' ); // This gives "Alice changed her profile picture"
banana.i18n(message, 'Bob', 'male' ); // This gives "Bob changed his profile picture"

Note that {{GENDER:...}} is not case sensitive. It can be {{gender:...}} too.

Grammar

const banana = new Banana( 'fi' );

const message = "{{grammar:genitive|$1}}";

banana.i18n(message, 'talo' ); // This gives "talon"

banana.locale = 'hy'; // Switch to locale Armenian
banana.i18n(message, 'Մաունա'); // This gives "Մաունայի"

Directionality-safe isolation

To avoid BIDI corruption that looks like "(Foo_(Bar", which happens when a string is inserted into a context with the reverse directionality, you can use {{bidi:…}}. Directionality-neutral characters at the edge of the string can get wrongly interpreted by the BIDI algorithm. This would let you embed your substituted string into a new BIDI context, //e.g.//:

"Shalom, {{bidi:$1}}, hi!"

The embedded context's directionality is determined by looking at the argument for $1, and then explicitly inserted into the Unicode text, ensuring correct rendering (because then the bidi algorithm "knows" the argument text is a separate context).

Wiki style links

The message can use MediaWiki link syntax. By default this is disabled. To enable support for this, pass wikilinks=true option to Banana constructor. Example:

new Banana('es', { wikilinks: true } )

The original wiki links markup is elaborate, but here we only support simple syntax.

  • Internal links: [[pageTitle]] or [[pageTitle|displayText]]. For example [[Apple]] gives <a href="./Apple" title="Apple">Apple</a>.
  • External links: [https://example.com] or [https://example.com display text]

Extending the parser

Following example illustrates extending the parser to support more parser plugins

const banana = new Banana('en');
banana.registerParserPlugin('sitename', () => {
  return 'Wikipedia';
});
banana.registerParserPlugin('link', (nodes) => {
  return '<a href="' + nodes[1] + '">' + nodes[0] + '</a>';
});

This will parse the message

banana.i18n('{{link:{{SITENAME}}|https://en.wikipedia.org}}');

to

<a href="https://en.wikipedia.org">Wikipedia</a>

Message documentation

The message keys and messages won't give a enough context about the message being translated to the translator. Whenever a developer adds a new message, it is a usual practice to document the message to a file named qqq.json with same message key.

Example qqq.json:

{
    "@metadata": {
        "authors": [
            "Developer Name"
        ]
    },
    "appname-title": "Application name. Transliteration is recommended",
    "appname-sub-title": "Brief explanation of the application",
    "appname-header-introduction": "Text for the introduction header",
    "appname-about": "About this application text",
    "appname-footer": "Footer text"
}

In MediaWiki and its hundreds of extensions, message documentation is a strictly followed practice. There is a grunt task to check whether all messages are documented or not. See https://www.npmjs.org/package/grunt-banana-checker

Translation

To translate the banana-i18n based application, depending on the expertise of the translator, there are multiple ways.

  • Editing the json files directly - Suitable for translators with technical background. Also suitable if your application is small and you want to work with only a small number of languages
  • Providing a translation interface along with your application: Suitable for proprietary or private applications with significant amount of translators
  • Using open source translation platforms like translatewiki.net. The MediaWiki and jquery.uls from previous examples use translatewiki.net for crowdsourced message translation. Translatewiki.net can update your code repo at regular intervals with updated translations. Highly recommended if your application is opensource and want it to be localized to as many as languages possible with maximum number of translators.

Frameworks and other programming languages

Thanks

This project is based on jquery.i18n library maintained by Wikimedia Foundation. Most of the internationalization related logic comes from that project. In Banana-i18n, jquery dependency was removed and the library was modernized to use in modern web applications. Contributors of jquery.i18n are greatly acknowledged and listed in AUTHORS.md

banana-i18n's People

Contributors

abijeet avatar amire80 avatar berndsi avatar bitpogo avatar davidbarratt avatar dependabot[bot] avatar feildmaster avatar hueitan avatar iiirxs avatar ricordisamoa avatar santhoshtr avatar siddharthvp avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

banana-i18n's Issues

Is `{{GENDER:||e}}` valid syntax?

This is the French translation of a message we've seen recently in the KaiOS app: Êtes-vous sûr{{GENDER:||e}} de vouloir quitter ? Tout votre avis sera perdu

While the gender of the user would be useful here it cannot be provided by the app. The GENDER function was not used in the English message and no parameter was provided or documented.

This caused a crash of the app in production. We've added a try/catch around our usage of the library to prevent that in the future.

Just thought we'd let you know in case you want to think about different ways to handle cases like that.

Provide a method to set the locale after construction

There is no way to change the locale after a Banana object has been constructed. One could change the local property, but they would also have to update the parser object as well. It would be helpful if a method was provided to do this so the whole thing doesn't have to be recreated to change the locale.

Support basic {{#if...}} constructs

Banana does not seem to support {{#if:$1|: \"$1\"}} and other simple template constructs, making it incompatible with the MW-based localization. Is that by default, or just hasn't been implemented yet?

Support loading messages to a given locale in parts

For a single locale, sometimes the messages need to be loaded in parts. For example, in an application, for locale fr, some messages are loaded on page load. After an event, a new UX is initiated and that require loading additional messages to the same locale - fr. Currently if you use the load method, it will overwrite the existing messages with the new messages. That is not nice.

The expected behavior is merging the new set of messages with the existing one. If there is a conflict in key, the last message loaded wins.

Should Banana return the language that was selected?

The locale that is set is not necessarily the language of the message that is returned. It might be helpful to return the language that was selected (maybe a new method that returns an object consisting of the message text and the language the message is in?). This way the HTML lang attribute can be set appropriately. What do you think?

Typescript support

Since some projects in the mw environment, which are using typescript, could profit from this lib, if it would provide type definitions. Is there any plan to do that, or a wish for contribution in this particular manor.

Rollup raises build size by about 20%

The current version of banana-i18n when built with rollup has a size of 25 KB. But if webpack is used (using the configuration that was removed in #45), it's only 21 KB.

Looks like a regression to me.

Crash with "zh-classical" messages

This code crashes because it calls new Intl.PluralRules("zh-classical"), which is an error in Chrome.

try {
  const bb = new Banana('zh-classical', { messages: {} });
  console.error(bb.i18n('zh-classical -- {{PLURAL:$1|$1 aaa|$1 bbb}}', 10))
} catch (err) {
  console.error(err)
}

Error:

index.js:1 RangeError: Incorrect locale information provided
    at new PluralRules (<anonymous>)
    at r.getPluralForm (language.js:51)
    at r.convertPlural (language.js:36)
    at c.plural (emitter.js:134)
    at c.emit (emitter.js:55)
    at emitter.js:54
    at Array.map (<anonymous>)
    at c.emit (emitter.js:54)
    at i.parse (parser.js:13)
    at o.i18n (index.js:28)

Error when having [] in value

The placeholder [[]] can then value into the link, but [] throws an error of

Error: Parse error at position x in input: xxx

Disable conversion of [[...]] into <a> tags

I use Banana library to create edit summary messages for a wiki. Those edit summaries can contain [[...]] links in wiki markup. Banana keeps converting them into HTML <a> tags. Currently I'm doing the reverse translation of back into [[...]] using regex replacement -- obviously not a very clean solution. How can I disable it in Banana?

Document how to load Banana with ES6 native modules

There doesn't appear to be any information. Doing something like:

import { Banana } from '../node_modules/banana-i18n/dist/banana-i18n.js';

... results in an error The requested module '/node_modules/banana-i18n/dist/banana-i18n.js' does not provide an export named 'Banana'...

Should locale parameter accept an optional array?

Sometimes the user provides (via the Accept-Language header) a list of languages that they know:

Accept-Language: en-US;q=0.8,es;q=0.5,ja;q=0.3

since the user is providing a list of languages, it might be helpful to accept that list as fallbacks.

For instance, based on that header, I would generate this list:

en-US
es
ja

and I would expect it to fallback like this:

en-US
en
es
ja

Perhaps the locale could accept a string or an array of strings? What do you think?

Loading messages is inconsistent

If you have a object of messages like:

const messages = {
  en: {
    test: 'TEST'
  }
};

And you do something like this:

new Banana( 'es', { messages }):

or even

const banana = new Banana( 'es' ):
banana.load(messages);

the messages will not be loaded correctly.

The only way to correctly load them is like this:

const banana = new Banana();
banana.load(messages)
banana.setLocale('es');

I think this is goofy. I'm not really sure how to fix it other than to not use the locale that has been set when loading messages, but then perhaps the constructor should not accept messages either.

Alternatively, it could just not accept multiple locals, or we could say that you must provide an object with the locals as the keys (and always accept multiple locales). (which would be my preference).

Regardless, I think the set locale should only apply to parsing, not loading.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.