Improve translation pipeline

[gyorokpeter]

Currently the build pipeline seems to work like this:

1. Pull strings from crowdin
2. Build the game
3. Publish new version to steam
4. Push new strings to crowdin

There is a glaringly obvious flaw in this process. Whenever there are new strings, translators don't get a chance to translate them until the build that actually uses them goes out. This means there will almost never be a build with a 100% complete translation - the only situation where that can happen is if there is a release with no new strings and the translators were fast enough to finish translating the ones from the previous versions. This is exacerbated by the fact that sometimes new strings are introduced that are just a few characters away from an old one, and the change is specific to English (e.g. using singular vs plural, using synonyms for certain words, changing articles) that has no impact on how the string should be translated, and the TM feature of crowdin usually deals with these fine - but still a human must go and push the button, and the string goes back into untranslated in the build.

This was especially visible with the 1.0 release where the spidertron-related strings were untranslated, exposing this problem to the wide world who normally don't see it because they don't play experimental or don't play early access games. And this persisted for 3 months since there was that much time between 1.0 and the first 1.1 release.

My suggestion is to decouple translations from the core game and introduce the concept of language packs that should work similarly to mods but with a very precise version dependency (e.g. for version 1.1.6 you need a language pack that is compatible with 1.1.6). Additionally introduce a daily (or even more frequent, depending on the availability of compute power) job that pulls all the strings from crowdin and creates a language pack for the latest experimental version. The regular build process could call this job to generate the language pack for the new release when one is made, and so the very first time the new version is launched it might contain untranslated strings, but after translators go in and translate them, the game should notice that there is a new language pack available and fetch the update.

Of course other solutions are possible such as making new releases every X days even if there is zero code change in them just to pick up any new translated strings. But the point is that the time between strings are translated and the translation appears in game should be reduced.

[gyorokpeter]

As a workaround I have come up with a script that downloads all the strings from Crowdin using the Crowdin API.

https://github.com/gyorokpeter/factorio ... ownload.py

[ssilk]

I point you to viewforum.php?f=135 Developing Tools
where you might also post your tool. And see others like
viewtopic.php?f=135&t=90093