Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 41 additions & 17 deletions Doc/library/turtle.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2675,12 +2675,47 @@ These modified docstrings are created automatically together with the function
definitions that are derived from the methods at import time.


.. _turtle-docstring-translation:

Translation of docstrings into different languages
--------------------------------------------------

There is a utility to create a dictionary the keys of which are the method names
and the values of which are the docstrings of the public methods of the classes
Screen and Turtle.
The docstrings of the public methods of the Screen and Turtle classes, and of
the functions derived from them, can be replaced by translations, so that
:func:`help` and IDE tooltips are shown in another language.

The translations are not part of Python. They are distributed on PyPI in the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the Discourse discussion has not yet reached consensus on bundling, could we avoid the categorical statement “The translations are not part of Python”? Perhaps this could simply say: “Translation catalogues can be installed from PyPI using the turtle-translations package.” This would document the mechanism introduced by this PR without deciding whether some catalogues may also be bundled with Python in the future.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still a draft, I'll update once the discussion has settled.

:pypi:`turtle-translations` package, which holds the docstring dictionaries of
all available languages. To use it, you must first install it with :program:`pip`::

python -m pip install turtle-translations
Comment thread
StanFromIreland marked this conversation as resolved.

The language is taken from the :envvar:`PYTHON_TURTLE_LANG` environment
variable, or, if that is unset, from the ``language`` entry of the
:file:`turtle.cfg` file. If no docstring dictionary is found for it, the
English docstrings are kept.

.. envvar:: PYTHON_TURTLE_LANG

The name of the language to read the docstring dictionary for.

.. versionadded:: 3.16

A docstring dictionary is a module defining a dictionary named ``docsdict``,
the keys of which are method names such as ``Turtle.forward`` and the values of
which are the translated docstrings. It is looked up on :data:`sys.path`, first
as the submodule of that name of a package named :mod:`!turtle_translations`,
then as a top-level module named :samp:`turtle_docstringdict_{language}.py`,
and is read in at import time. Entries naming a method which does not exist in
the running version are ignored.

.. versionchanged:: 3.16
The docstring dictionary may also be provided as a submodule of a
:mod:`!turtle_translations` package, and entries naming an unknown method
are ignored instead of reported.

To translate the docstrings into a language which is not available yet, write
out a template with :func:`write_docstringdict` and translate its values.

.. function:: write_docstringdict(filename="turtle_docstringdict")

Expand All @@ -2692,17 +2727,6 @@ Screen and Turtle.
Python script :file:`{filename}.py`. It is intended to serve as a template
for translation of the docstrings into different languages.

If you (or your students) want to use :mod:`!turtle` with online help in your
native language, you have to translate the docstrings and save the resulting
file as e.g. :file:`turtle_docstringdict_german.py`.

If you have an appropriate entry in your :file:`turtle.cfg` file this dictionary
will be read in at import time and will replace the original English docstrings.

At the time of this writing there are docstring dictionaries in German and in
Italian. (Requests please to glingl@aon.at.)



How to configure Screen and Turtles
-----------------------------------
Expand Down Expand Up @@ -2753,9 +2777,9 @@ Short explanation of selected entries:
the cfg file).
- If you want to reflect the turtle its state, you have to use ``resizemode =
auto``.
- If you set e.g. ``language = italian`` the docstringdict
:file:`turtle_docstringdict_italian.py` will be loaded at import time (if
present on the import path, e.g. in the same directory as :mod:`!turtle`).
- The *language* entry selects the language of the docstrings, unless the
:envvar:`PYTHON_TURTLE_LANG` environment variable is set. See
:ref:`turtle-docstring-translation` for more information.
- The entries *exampleturtle* and *examplescreen* define the names of these
objects as they occur in the docstrings. The transformation of
method-docstrings to function-docstrings will delete these names from the
Expand Down
44 changes: 44 additions & 0 deletions Lib/test/test_turtle.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
from test import support
from test.support import import_helper
from test.support import os_helper
from test.support.script_helper import assert_python_ok


turtle = import_helper.import_module('turtle')
Expand Down Expand Up @@ -713,5 +714,48 @@ def test_all_signatures(self):
self.assertEqual(str(sig), known_signatures[name])


class TurtleDocstringTranslationTest(unittest.TestCase):

def _make_translation(self, dirname, filename, docstring):
with open(os.path.join(dirname, filename), 'w') as f:
f.write('docsdict = {"Turtle.forward": %r}\n' % docstring)

def _make_package(self, dirname):
pkgdir = os.path.join(dirname, 'turtle_translations')
os.mkdir(pkgdir)
with open(os.path.join(pkgdir, '__init__.py'), 'w'):
pass
return pkgdir

def _get_forward_docstring(self, dirname, lang):
rc, out, err = assert_python_ok(
'-c', 'import turtle; print(turtle.forward.__doc__)',
PYTHONPATH=dirname, PYTHON_TURTLE_LANG=lang)
return out.decode()

def test_translation_from_package(self):
with os_helper.temp_dir() as dirname:
pkgdir = self._make_package(dirname)
self._make_translation(pkgdir, 'ga.py', 'chun tosaigh')

out = self._get_forward_docstring(dirname, 'ga')
self.assertIn('chun tosaigh', out)

def test_translation_from_top_level_dict(self):
with os_helper.temp_dir() as dirname:
self._make_translation(dirname, 'turtle_docstringdict_ga.py',
'chun tosaigh')

out = self._get_forward_docstring(dirname, 'ga')
self.assertIn('chun tosaigh', out)

def test_unknown_language(self):
with os_helper.temp_dir() as dirname:
out = self._get_forward_docstring(dirname, 'ga')

self.assertIn('Cannot find docsdict for ga', out)
self.assertIn('Move the turtle forward', out)


if __name__ == '__main__':
unittest.main()
20 changes: 17 additions & 3 deletions Lib/turtle.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@
import inspect
import sys

from os import environ
from os.path import isfile, split, join
from pathlib import Path
from contextlib import contextmanager
Expand Down Expand Up @@ -4017,18 +4018,31 @@ def read_docstrings(lang):
Transfer docstrings, translated to lang, from a dictionary-file
to the methods of classes Screen and Turtle and - in revised form -
to the corresponding functions.

The dictionary is looked up as the submodule lang of the package
turtle_translations, then as the top-level module
turtle_docstringdict_lang.

Entries naming a method which does not exist in this version are
ignored.
"""
modname = "turtle_docstringdict_%(language)s" % {'language':lang.lower()}
module = __import__(modname)
import importlib
lang = lang.lower()
try:
module = importlib.import_module("turtle_translations.%s" % lang)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could the new PyPI integration use non-executable catalogues, such as gettext .mo files, instead of importing Python modules? Importing turtle_translations.<lang> executes both the package and the language module, while the existing eval(key) additionally permits arbitrary expressions in dictionary keys.

Since this package is intended to provide translation data, we could use GNUTranslations.pgettext(), with the method name as the context and the current English docstring as the message ID, and iterate over _tg_screen_functions and _tg_turtle_functions. This way, CPython – rather than the catalogue – controls which objects may have their docstrings replaced.

The existing top-level Python-module format could remain as a legacy compatibility path.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's possible, but I'm also going to defer that discussion for now. I'd like to have turtle-translations in existence before we start considering larger changes like this. Once we have translations it'll be relatively easy to convert from one format (dict) to another (PO).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Converting the translation data may be straightforward, but I think the compatibility cost lies in changing the consumer contract after a CPython release has shipped. Once a released loader imports turtle_translations.<lang> and expects docsdict, future package releases cannot drop those modules in favour of .mo files without breaking that Python version. A Requires-Python split would leave older Pythons on stale translations, while an in-place migration would require shipping both formats for their supported lifetime.

Also, converting the current dictionary to a message gettext catalogue would require the original English docstrings, which the dictionary itself does not preserve. Would it make sense to use PO/MO as the source format from the start, while generating docsdict modules for the initial loader? That would allow the package to exist now without making the legacy runtime format the canonical translation format.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

after a CPython release has shipped.

Thankfully we still have 8 months till the feature freeze to figure this out.

Also, converting the current dictionary to a message gettext catalogue would require the original English docstrings, which the dictionary itself does not preserve. Would it make sense to use PO/MO as the source format from the start, while generating docsdict modules for the initial loader? That would allow the package to exist now without making the legacy runtime format the canonical translation format.

Let's discuss once we have a repository to work in.

except ModuleNotFoundError:
module = importlib.import_module("turtle_docstringdict_%s" % lang)
docsdict = module.docsdict
for key in docsdict:
try:
# eval(key).im_func.__doc__ = docsdict[key]
eval(key).__doc__ = docsdict[key]
except AttributeError:
pass
except Exception:
print("Bad docstring-entry: %s" % key)

_LANGUAGE = _CFG["language"]
_LANGUAGE = environ.get("PYTHON_TURTLE_LANG") or _CFG["language"]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is a PYTHON_* environment variable, it should respect -E and -I. Currently, PYTHON_TURTLE_LANG=ga python -E -c "import turtle" still attempts to load the ga translation because the value is read directly from os.environ. Could this lookup be guarded by not sys.flags.ignore_environment and covered by a subprocess regression test using -E? Alternatively, given that turtle.cfg already provides an explicit override, perhaps the new variable could be deferred until the language-selection semantics are settled.


try:
if _LANGUAGE != "english":
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Add the :envvar:`PYTHON_TURTLE_LANG` environment variable to select the
language of :mod:`turtle` docstrings, and allow the docstring dictionary to
be provided as a submodule of the :pypi:`turtle-translations` package.
Entries naming a method which does not exist in the running version are now
ignored.
Loading