gh-156360: Improve turtle translation support

[StanFromIreland]

• Issue: Set up a coordinated system for the translation of turtle's docstrings #156360

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34243108 | 📁 Comparing 906cf7c against main (9036982)

  🔍 Preview build  

2 files changed

± library/turtle.html
± whatsnew/changelog.html

[m-aciek]

Could the new PyPI integration use non-executable catalogues, such as gettext .mo files, instead of importing Python modules? Importing turtle_translations.<lang> executes both the package and the language module, while the existing eval(key) additionally permits arbitrary expressions in dictionary keys.

Since this package is intended to provide translation data, we could use GNUTranslations.pgettext(), with the method name as the context and the current English docstring as the message ID, and iterate over _tg_screen_functions and _tg_turtle_functions. This way, CPython – rather than the catalogue – controls which objects may have their docstrings replaced.

The existing top-level Python-module format could remain as a legacy compatibility path.

[StanFromIreland]

That's possible, but I'm also going to defer that discussion for now. I'd like to have turtle-translations in existence before we start considering larger changes like this. Once we have translations it'll be relatively easy to convert from one format (dict) to another (PO).

[m-aciek]

Converting the translation data may be straightforward, but I think the compatibility cost lies in changing the consumer contract after a CPython release has shipped. Once a released loader imports turtle_translations.<lang> and expects docsdict, future package releases cannot drop those modules in favour of .mo files without breaking that Python version. A Requires-Python split would leave older Pythons on stale translations, while an in-place migration would require shipping both formats for their supported lifetime.

Also, converting the current dictionary to a message gettext catalogue would require the original English docstrings, which the dictionary itself does not preserve. Would it make sense to use PO/MO as the source format from the start, while generating docsdict modules for the initial loader? That would allow the package to exist now without making the legacy runtime format the canonical translation format.

[StanFromIreland]

after a CPython release has shipped.

Thankfully we still have 8 months till the feature freeze to figure this out.

Also, converting the current dictionary to a message gettext catalogue would require the original English docstrings, which the dictionary itself does not preserve. Would it make sense to use PO/MO as the source format from the start, while generating docsdict modules for the initial loader? That would allow the package to exist now without making the legacy runtime format the canonical translation format.

Let's discuss once we have a repository to work in.

[m-aciek]

Since this is a PYTHON_* environment variable, it should respect -E and -I. Currently, PYTHON_TURTLE_LANG=ga python -E -c "import turtle" still attempts to load the ga translation because the value is read directly from os.environ. Could this lookup be guarded by not sys.flags.ignore_environment and covered by a subprocess regression test using -E? Alternatively, given that turtle.cfg already provides an explicit override, perhaps the new variable could be deferred until the language-selection semantics are settled.

[m-aciek]

Since the Discourse discussion has not yet reached consensus on bundling, could we avoid the categorical statement “The translations are not part of Python”? Perhaps this could simply say: “Translation catalogues can be installed from PyPI using the turtle-translations package.” This would document the mechanism introduced by this PR without deciding whether some catalogues may also be bundled with Python in the future.

[StanFromIreland]

This is still a draft, I'll update once the discussion has settled.