Set up translation of additional strings with Weblate about gramps-web HOT 26 CLOSED

I think we can close this 🎉

from gramps-web.

Requested nb_NO.
Weblate is the best option, and formats with support for direct editing on WL, or in the very least editing without running scripts afterwards are preferred.
Going from GPLv2+ to MIT is a weaker license. AGPLv3+ has network protection.

from gramps-web.

Agreed, I set up an account on Weblate and plan to set it up in the next few days - it looks like JSON is the best option, it's directly supported by Weblate.

Concerning the license: Gramps.js is already AGPL, so you're right it might make more sense to match the license of the translations (if that's what you mean).

from gramps-web.

That would be fantastic, as I'm struggling to set it up. I'll let you know once I think it's in a format Weblate understands.

from gramps-web.

Ok it seems to me like a solution.

I've chosen https://github.com/swiing/rollup-plugin-import-assertions instead of https://www.npmjs.com/package/@rollup/plugin-json since we can use following syntax:

import en from './en_GB.json' assert {type: 'json'}
import ru from './ru.json' assert {type: 'json'}
import de from './de.json' assert {type: 'json'}

In this case its working by default if you develop in Chrome, you don't even need any further commands. It may also work on the latest version of Firefox I just don't have up to date version of Firefox locally.

Additionally I've installed as dev dependency @babel/plugin-syntax-import-assertions and added babel configuration in package.json otherwise rollup complains about assertion on the import.

I will create a PR so you can take a look what the changes are.

from gramps-web.

Thanks. That should work, but I forgot to add the web hook so it isn't in the Weblate repository yet.

We have had a request for a Portuguese (pt) translation already. Perhaps you could create an empty pt_PT.json file so that we can test the web hook.

I'll then set the base file to en.json.

from gramps-web.

@DavidMStraub We can use our existing Gramps project in Weblate, and just add another component.

Let me know the file format and location of the translation files.

from gramps-web.

I plan to work on this with https://lit.dev/docs/localization/overview/.

So that I don't do unnecessary work, could you please tell me what you think? Or do you prefer another library for the translation?

I installed Weblate locally with Docker and imported all *.po's by configuring it to use the Github repository. But I don't think it's possible to use the same *.po files from the backend in Grampsjs. Because these *.po files contain lines about the position of the translated string. And if you add another translation from Frontent that doesn't exist in the backend, the *.po file could get corrupted. I am not 100% sure about this.

But in my opinion it might be better to handle frontend translations separately. And it will less confuse future editors of the weblate project.

from gramps-web.

Great that you're taking this on! I will look into it and let you know later today.

from gramps-web.

Ok so lit-localize looks cool but the problem with this and any other existing JS i18n library is the peculiar way how tranlations are handled in Gramps.js right now: 90% of the strings are fetched from the Gramps Web API leveraging directly the translations in Gramps Python. Although that's a bit unusual, it greatly reduces the number of strings that need translation and brings all the languages in Gramps without extra effort and without a need to modify the code base when translations in Gramps are updated.

The GrampsTranslateMixin is used by almost all components and provides a method this._ which works like lit's msg or Python's gettext (abbreviated as _ in Gramps). The strings property is passed down from the GrampsJs component that fetches them from the API.

Due to this architecture, I would suggest keeping the additional strings in one of the JSON formats supported by Weblate and then implementing it such that it replaces the current additionalStrings object in strings.js, which I selfishly populated only with German translations.

Does it make sense to you?

from gramps-web.

It's always a challenge to handle such common things in a custom way, but it shouldn't be that much of a pain to handle it that way as well. I think the next step would be to override the export const additionalStrings = {} in strings.js so that it returns an object with all the strings that need to be fetched from somewhere (maybe pushing to the backend server). This parameter will look like this:

export const additionalStrings = _getStrings()

function _getStrings () {
  const languages = ['en', 'de']
  const frontEndStrings = {}
  const placeWhereToFetchTheJsonFiles = 'https://backendurl.com/translations/frontend'
  languages.forEach(lang => {
    fetch(`${placeWhereToFetchTheJsonFiles}/${lang}.json`).then(resp => {
      return resp.json()
    }).then(json => {
      frontEndStrings[lang] = json
    }).catch(error => {
      console.log(error)
    })
  })

  return frontEndStrings 
}

So, where would you store the *.json files? They could be saved in api-backend and we could use __APIHOST__ from api.js

from gramps-web.

@DavidMStraub Let me know if you would like me to add a component to our existing hosted Weblate.

from gramps-web.

So, where would you store the *.json files? They could be saved in api-backend and we could use APIHOST from api.js

I would leave the frontend translations with the frontend. I just recalled that I already created a directory for this here when I experimented with different JSON formats and Weblate last year, but didn't get it to work (I didn't try very hard...).

One minor challenge is how to ideally import the JSON files (like discussed in this StackOverflow). Probably there is a plugin for rollup that allows to import the JSON files as JS objects.

@Nick-Hall thanks, we'll let you know when we're ready.

from gramps-web.

Well keeping them in frontend repository is ok but we need a solution to host them somewhere maybe using github actions to host them in AWS S3.

You can't import json files (at the moment) I tried to do it as it suggested in tc39 proposal but it currently works only in Chrome. If you have any idea how this could be achieved please point me to it and I will implement it.

from gramps-web.

Well keeping them in frontend repository is ok but we need a solution to host them somewhere maybe using github actions to host them in AWS S3.

I don't understand. I thought Weblate is connected to Github and pushes/pulls translations directly from the repository? Isn't that the way it works with Gramps desktop?

You can't import json files (at the moment) I tried to do it as it suggested in tc39 proposal but it currently works only in Chrome. If you have any idea how this could be achieved please point me to it and I will implement it.

I think the key is to convert them to JS modules on build. Probably this
https://www.npmjs.com/package/@rollup/plugin-json
would work, the problem is that I'm not sure if it can be made to work also with the live environment used for development (Web Dev Server) and it would be annoying not to have the translations during development...

from gramps-web.

I don't understand. I thought Weblate is connected to Github and pushes/pulls translations directly from the repository? Isn't that the way it works with Gramps desktop?

Yes, that is correct. Sorry for the confusion, but I was talking about the problem of not being able to import the json files and proposing to host them to fetch() them as I have suggested here: #41 (comment)

I think the key is to convert them to JS modules on build. Probably this
https://www.npmjs.com/package/@rollup/plugin-json

In my opinion, this goes further in the complicated direction of involving third party libraries, which makes it harder to contribute. I would solve this problem by using your suggestion (plugin-json) for the reason that it could be done in a short time, but only as a temporary solution.

I would rather convert all existing *.po files from https://github.com/gramps-project/gramps/tree/master/po in to json files.

So my next step would be:

1. Solve this issue by using plugin-json
2. Work on an separate branch to convert all *.po's and replace everything in Grampsjs with https://lit.dev/docs/localization/overview/

What do you think?

from gramps-web.

In my opinion, this goes further in the complicated direction of involving third party libraries, which makes it harder to contribute.

If this is set up correctly in rollup.json, it should not have any impact on future development, I think it's actually the simplest way as we can work with simple JSON files. Or am I misunderstanding your point?

Concerning my previous concerning about Web Dev Server, since strings don't change too often (presumably), I think it could be acceptable to have a dedicated command (sth like npm run build-translations) to build only the translation files (JSON -> JS) during development.

Work on an separate branch to convert all *.po's and replace everything in Grampsjs with https://lit.dev/docs/localization/overview/

I would really like to avoid this as I see two big disadvantages:

1. More than 90% of the strings in Gramps's po files are not needed
2. This would completely clutter our commit history with Weblate commits (just look at https://github.com/gramps-project/gramps/commits/master), even if most of them are not even needed

So I don't really see the need for this major change.

from gramps-web.

Would storing all translations in JSON files be of any help? The translation memory feature of Weblate should enable strings already used in core Gramps to be translated automatically. We would need to test this.

from gramps-web.

@DavidMStraub Let me know if you would like me to add a component to our existing hosted Weblate.

I think you could set up new component and point it to this folder: https://github.com/gramps-project/Gramps.js/tree/main/src/lang

from gramps-web.

I think you could set up new component and point it to this folder: https://github.com/gramps-project/Gramps.js/tree/main/src/lang

@Nick-Hall is this sufficient information for you to set up a new component in Weblate?

from gramps-web.

I've created a new component called Web. There are two issues at the moment: there are only 4 translations and the Norwegian translation is empty.

At the moment I'm using a bilingual configuration. I'll investigate specifying a monolingual base file. Hopefully this will allow us to add new strings to the base file only.

from gramps-web.

OK. I've now specified a monolingual base file.

This makes the Norwegian translation available starting from an empty file. If we go down this route then you need to create a en.json base file with US spellings (key = translation). The key field could then be changed to any convenient string.

When you decide on the best approach we also need to create files for the other currently translated languages.

from gramps-web.

Great, thanks @Nick-Hall! I just added an en.json copying the existing en_GB.json. Is this sufficient or does it have to be a different format?

from gramps-web.

I added an empty JSON for each of the missing languages now.

from gramps-web.

Everything is looking good.

Weblate has created a pull request with updates to the Norwegian and English (GB) translations. You will need to merge these on a regular basis.

There are two new questions relating to the "Home Person" and "Global" strings. Do you have a Weblate account yet? I may need to give you extra permissions if you want to do things like adding screenshots.

There are also some warnings because "Home Person" is also a string in the "gramps" component and the shared translation memory points out inconsistent translations. In this case it is just that the string is translated in one component and not the other.

from gramps-web.

Great! The Portuguese and Norwegian are already working 🚀

Yes, I have a Weblate account, I will try to sort out the remaining issues later today.

from gramps-web.