Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions Doc/builtins/functions.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. XXX document all delegations to __special__ methods
.. _built-in-funcs:

Built-in Functions
Built-in functions
==================

The Python interpreter has a number of functions and types built into it that
Expand Down Expand Up @@ -1459,7 +1459,8 @@ are always available. They are listed here in alphabetical order.
already exists), ``'x'`` for exclusive creation, and ``'a'`` for appending
(which on *some* Unix systems, means that *all* writes append to the end of
the file regardless of the current seek position). In text mode, if
*encoding* is not specified the encoding used is platform-dependent:
*encoding* is not specified, UTF-8 is used by default; if
:ref:`Python UTF-8 Mode <utf8-mode>` is disabled,
:func:`locale.getencoding` is called to get the current locale encoding.
(For reading and writing raw bytes use binary mode and leave
*encoding* unspecified.) The available modes are:
Expand Down Expand Up @@ -1490,7 +1491,7 @@ are always available. They are listed here in alphabetical order.
argument) return contents as :class:`bytes` objects without any decoding. In
text mode (the default, or when ``'t'`` is included in the *mode* argument),
the contents of the file are returned as :class:`str`, the bytes having been
first decoded using a platform-dependent encoding or using the specified
first decoded using the default encoding or using the specified
*encoding* if given.

.. note::
Expand Down Expand Up @@ -1519,9 +1520,11 @@ are always available. They are listed here in alphabetical order.
described above for binary files.

*encoding* is the name of the encoding used to decode or encode the file.
This should only be used in text mode. The default encoding is platform
dependent (whatever :func:`locale.getencoding` returns), but any
:term:`text encoding` supported by Python can be used.
This should only be used in text mode. The default encoding is UTF-8;
if :ref:`Python UTF-8 Mode <utf8-mode>` is disabled, the default is
platform-dependent (whatever :func:`locale.getencoding` returns).
Any :term:`text encoding` supported by Python can be used, and
``encoding="locale"`` specifies the current locale encoding explicitly.
See the :mod:`codecs` module for the list of supported encodings.

*errors* is an optional string that specifies how encoding and decoding
Expand Down Expand Up @@ -1638,6 +1641,10 @@ are always available. They are listed here in alphabetical order.
.. versionchanged:: 3.11
The ``'U'`` mode has been removed.

.. versionchanged:: 3.15
UTF-8 is now the default encoding, instead of the
platform-dependent locale encoding (:pep:`686`).

.. function:: ord(character, /)

Return the ordinal value of a character.
Expand Down
6 changes: 6 additions & 0 deletions Doc/c-api/bytearray.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@ Byte Array Objects

This subtype of :c:type:`PyObject` represents a Python bytearray object.

.. impl-detail::

The internal buffer of :c:type:`PyByteArrayObject` always includes an
extra trailing null byte for compatibility with null terminated C
strings. This extra byte is not counted in :c:func:`PyByteArray_Size`
nor in the *len* arguments of the functions below.

.. c:var:: PyTypeObject PyByteArray_Type

Expand Down
7 changes: 7 additions & 0 deletions Doc/c-api/bytes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ Bytes Objects
These functions raise :exc:`TypeError` when expecting a bytes parameter and
called with a non-bytes parameter.

.. impl-detail::

The internal buffer of :c:type:`PyBytesObject` always includes an extra
trailing null byte for compatibility with null terminated C strings.
This extra byte is not counted in :c:func:`PyBytes_Size` nor in the
various *length* and *size* arguments of the functions below.

.. index:: pair: object; bytes


Expand Down
9 changes: 6 additions & 3 deletions Doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@
# and replace the values accordingly.
# See Doc/tools/extensions/patchlevel.py
version, release = get_version_info()
v = get_header_version_info()
branch = "main" if v.releaselevel == "alpha" else f"{v.major}.{v.minor}"

rst_epilog = f"""
.. |python_version_literal| replace:: ``Python {version}``
Expand Down Expand Up @@ -298,6 +300,7 @@
"repository_url": repository_url or None,
"pr_id": os.getenv("READTHEDOCS_VERSION"),
"enable_analytics": os.getenv("PYTHON_DOCS_ENABLE_ANALYTICS"),
"source_branch": branch,
}

# This 'Last updated on:' timestamp is inserted at the bottom of every page.
Expand All @@ -307,6 +310,9 @@
# Path to find HTML templates to override theme
templates_path = ['tools/templates']

# We link to sources on GitHub, so don't copy them into the HTML output.
html_copy_source = False

# Custom sidebar templates, filenames relative to this file.
html_sidebars = {
# Defaults taken from https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_sidebars
Expand Down Expand Up @@ -571,9 +577,6 @@
# Options for sphinx.ext.extlinks
# -------------------------------

v = get_header_version_info()
branch = "main" if v.releaselevel == "alpha" else f"{v.major}.{v.minor}"

# This config is a dictionary of external sites,
# mapping unique short aliases to a base URL and a prefix.
# https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
Expand Down
14 changes: 14 additions & 0 deletions Doc/library/codecs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1395,6 +1395,15 @@ encodings.
| | | :mod:`encodings.idna`. |
| | | Only ``errors='strict'`` |
| | | is supported. |
| | | |
| | | .. warning:: |
| | | |
| | | This codec builds on |
| | | ``punycode``, whose |
| | | algorithms scale |
| | | poorly, so limit the |
| | | length of untrusted |
| | | input. |
+--------------------+---------+---------------------------+
| mbcs | ansi, | Windows only: Encode the |
| | dbcs | operand according to the |
Expand Down Expand Up @@ -1646,6 +1655,11 @@ Applications) and :rfc:`3492` (Nameprep: A Stringprep Profile for
Internationalized Domain Names (IDN)). It builds upon the ``punycode`` encoding
and :mod:`stringprep`.

.. warning::

This module builds on ``punycode``, whose algorithms scale poorly, so limit
the length of untrusted input.

If you need the IDNA 2008 standard from :rfc:`5891` and :rfc:`5895`, use the
third-party :pypi:`idna` module.

Expand Down
17 changes: 8 additions & 9 deletions Doc/library/csv.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
:mod:`!csv` --- CSV File Reading and Writing
:mod:`!csv` --- CSV file reading and writing
============================================

.. module:: csv
Expand Down Expand Up @@ -42,7 +42,7 @@ using the :class:`DictReader` and :class:`DictWriter` classes.

.. _csv-contents:

Module Contents
Module contents
---------------

The :mod:`!csv` module defines the following functions:
Expand Down Expand Up @@ -451,7 +451,7 @@ The :mod:`!csv` module defines the following exception:

.. _csv-fmt-params:

Dialects and Formatting Parameters
Dialects and formatting parameters
----------------------------------

To make it easier to specify the format of input and output records, specific
Expand Down Expand Up @@ -557,7 +557,7 @@ with the specified formatting parameters replaced.

.. _reader-objects:

Reader Objects
Reader objects
--------------

Reader objects (:class:`DictReader` instances and objects returned by the
Expand Down Expand Up @@ -594,7 +594,7 @@ DictReader objects have the following public attribute:



Writer Objects
Writer objects
--------------

:class:`writer` objects (:class:`DictWriter` instances and objects returned by
Expand Down Expand Up @@ -673,17 +673,16 @@ The corresponding simplest possible writing example is::
writer.writerows(someiterable)

Since :func:`open` is used to open a CSV file for reading, the file
will by default be decoded into unicode using the system default
encoding (see :func:`locale.getencoding`). To decode a file
will by default be decoded into Unicode using UTF-8. To decode a file
using a different encoding, use the ``encoding`` argument of open::

import csv
with open('some.csv', newline='', encoding='utf-8') as f:
with open('some.csv', newline='', encoding='latin-1') as f:
reader = csv.reader(f)
for row in reader:
print(row)

The same applies to writing in something other than the system default
The same applies to writing in something other than the default
encoding: specify the encoding argument when opening the output file.

Registering a new dialect::
Expand Down
43 changes: 15 additions & 28 deletions Doc/library/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -115,34 +115,21 @@ The raw stream API is described in detail in the docs of :class:`RawIOBase`.

.. _io-text-encoding:

Text Encoding
Text encoding
-------------

The default encoding of :class:`TextIOWrapper` and :func:`open` is
locale-specific (:func:`locale.getencoding`).

However, many developers forget to specify the encoding when opening text files
encoded in UTF-8 (e.g. JSON, TOML, Markdown, etc...) since most Unix
platforms use UTF-8 locale by default. This causes bugs because the locale
encoding is not UTF-8 for most Windows users. For example::

# May not work on Windows when non-ASCII characters in the file.
with open("README.md") as f:
long_description = f.read()

Accordingly, it is highly recommended that you specify the encoding
explicitly when opening text files. If you want to use UTF-8, pass
``encoding="utf-8"``. To use the current locale encoding,
``encoding="locale"`` is supported since Python 3.10.
The default encoding of :class:`TextIOWrapper` and :func:`open` is UTF-8.
If :ref:`Python UTF-8 Mode <utf8-mode>` is disabled, the default encoding
is locale-specific (:func:`locale.getencoding`).

.. seealso::

:ref:`utf8-mode`
Python UTF-8 Mode can be used to change the default encoding to
UTF-8 from locale-specific encoding.
Python UTF-8 Mode ignores the locale encoding and forces the use
of UTF-8.

:pep:`686`
Python 3.15 will make :ref:`utf8-mode` default.
Python 3.15 made :ref:`utf8-mode` the default.

.. _io-encoding-warning:

Expand All @@ -152,7 +139,7 @@ Opt-in EncodingWarning
.. versionadded:: 3.10
See :pep:`597` for more details.

To find where the default locale encoding is used, you can enable
To find where the default encoding is used, you can enable
the :option:`-X warn_default_encoding <-X>` command line option or set the
:envvar:`PYTHONWARNDEFAULTENCODING` environment variable, which will
emit an :exc:`EncodingWarning` when the default encoding is used.
Expand All @@ -165,7 +152,7 @@ please consider using UTF-8 by default (i.e. ``encoding="utf-8"``) for
new APIs.


High-level Module Interface
High-level module interface
---------------------------

.. data:: DEFAULT_BUFFER_SIZE
Expand Down Expand Up @@ -315,7 +302,7 @@ ABC Inherits Stub Methods Mixin M
========================= ================== ======================== ==================================================


I/O Base Classes
I/O base classes
^^^^^^^^^^^^^^^^

.. class:: IOBase
Expand Down Expand Up @@ -660,7 +647,7 @@ I/O Base Classes
so the implementation should only access *b* during the method call.


Raw File I/O
Raw file I/O
^^^^^^^^^^^^

.. class:: FileIO(name, mode='r', closefd=True, opener=None)
Expand Down Expand Up @@ -728,7 +715,7 @@ Raw File I/O
given in the constructor.


Buffered Streams
Buffered streams
^^^^^^^^^^^^^^^^

Buffered I/O streams provide a higher-level interface to an I/O device
Expand Down Expand Up @@ -1004,8 +991,8 @@ Text I/O
:class:`TextIOBase`.

*encoding* gives the name of the encoding that the stream will be decoded or
encoded with. In :ref:`UTF-8 Mode <utf8-mode>`, this defaults to UTF-8.
Otherwise, it defaults to :func:`locale.getencoding`.
encoded with. This defaults to UTF-8; if :ref:`UTF-8 Mode <utf8-mode>` is
disabled, it defaults to :func:`locale.getencoding`.
``encoding="locale"`` can be used to specify the current locale's encoding
explicitly. See :ref:`io-text-encoding` for more information.

Expand Down Expand Up @@ -1187,7 +1174,7 @@ Text I/O
It inherits from :class:`codecs.IncrementalDecoder`.


Static Typing
Static typing
-------------

The following protocols can be used for annotating function and method
Expand Down
8 changes: 4 additions & 4 deletions Doc/tools/templates/customsourcelink.html
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{%- if show_source and has_source and sourcename %}
{%- if page_source_suffix is defined %}
<script>
document.addEventListener('DOMContentLoaded', () => {
const title = document.querySelector('meta[property="og:title"]').content;
Expand All @@ -19,13 +19,13 @@ <h3>{{ _('This page') }}</h3>
<li><a href="{{ pathto('bugs') }}">{% trans %}Report a bug{% endtrans %}</a></li>
<li><a class="improvepage" href="{{ pathto('improve-page-nojs') }}">{% trans %}Improve this page{% endtrans %}</a></li>
<li>
<a href="https://github.com/python/cpython/blob/main/Doc/{{ pagename }}.rst?plain=1"
<a href="https://github.com/python/cpython/blob/{{ source_branch }}/Doc/{{ pagename }}.rst?plain=1"
rel="nofollow">{{ _('Show source') }}
</a>
</li>
{% if language != "en" %}
{% if language != "en" and source_branch != "main" %}
<li>
<a href="https://github.com/python/python-docs-{{ language }}/blob/{{ version }}/{{ pagename }}.po?plain=1"
<a href="https://github.com/python/python-docs-{{ language | replace('_', '-') | lower }}/blob/{{ source_branch }}/{{ pagename }}.po?plain=1"
rel="nofollow">{{ _('Show translation source') }}</a>
</li>
{% endif %}
Expand Down
Loading
Loading