How to use rosetta in the Cloud

[batiste]

Hi Marco! How are you doing?

Imagine I want to run Rosetta in the cloud, e.g. on kubernetes. The files will only be modified on one pod/machine, and as soon as it restart the modifications are lost I imagine?

Did you ever you consider an database plugin of some sort so the translation work could be done, and saved in the DB on a live platform? I imagine you would still need to download and commit the translation files to the source code.

Cheers,
Batiste

[mbi]

Hiya @batiste !

Imagine I want to run Rosetta in the cloud, e.g. on kubernetes. The files will only be modified on one pod/machine, and as soon as it restart the modifications are lost I imagine?

That's right, Rosetta only deals in gettext catalog files, and as such is entirely file based.

What you suggest has been discussed before: #265 and #246 but at the moment I think it's beyond Rosett's scope to go that way. Even if we decided to store the translations in a db, we'd still have to sync these translations back to and from gettext, with all the complexities this entails.

As a workaround, you can always download the PO catalog file using the link at the top-right

[batiste]

Interesting, I read the 2 tickets.

My idea might be a little different, though. I believe translation files should still probably be committed to an application's repository, so the proposals involving persistent and mounted file systems seem a bit off. I'm not sure having files both in the codebase and in a storage bucket makes any sense.

This is how I imagine the workflow in a k8s setup:

1. The application(s) receive new translations strings
2. The new application is released in some kind of environment (let's say staging).
3. Translators do their job on staging
4. Before release the new translations file are downloaded and added to the code base.

The only issues with this workflow are:

1. Since pods are transient by nature, if a pod restarts, the work is lost.
2. If multiple pods exist, a filesystem-based approach won't work.

How I could imagine to fix this in a quick and dirty way:

1. Every time Rosetta saves a file to disk, a copy is also saved in a database table with a timestamp.
2. Every time Rosetta reads a file from disk, it checks the timestamps. If the database version is newer, it overwrites the file on the filesystem beforehand.

Does that sound crazy to you?
It also seems that is something I could easily implement using the signals? (Maybe I would need a read signal as well?)

[batiste]

Correct me if I am wrong, but I think I can get what I want just by choosing a persistent cache for ROSETTA_STORAGE_CLASS. Either DB, Redis or equivalent would do the job no?

[mbi]

Correct me if I am wrong, but I think I can get what I want just by choosing a persistent cache for ROSETTA_STORAGE_CLASS. Either DB, Redis or equivalent would do the job no?

Well I'm not sure, but ROSETTA_STORAGE_CLASS only defines a cache-like storage to persist translations (as a pickled polib object IIRC). Not sure if you can extract any valid PO catalog from that. 🤔

[batiste]

I think I could get there with just one small tweak to the code:

1. A new setting, maybe called FORCE_CACHE here that would force usage of the cache at all times
2. Setup a cache that is persistent (DB)

But now I see a bit the issue, is that the files are never read again... Ok that is not that simple

[mbi]

Okay sounds like a plan! Please open a PR if you're happy with the result!