[Feature Request]: Add Support for Python 3.14 free threading

[tkinz27]

What would you like to happen?

Python 3.14 was released and this is the first release that the free threading mode is officially supported.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[jrmccluskey]

Hi there, this is on my radar for deeper investigation in the next few weeks. Long story short, this is going to require a lot of things to move both within and beyond Beam Python to work properly. Beam carries a lot of dependencies that will require free threading support, most importantly gRPC (a tracking bug for this feature in gRPC is at grpc/grpc#38762.) Then there's the separate problem of updating the SDK code itself to work in free-threading. The biggest hurdles here are A) having sufficient testing for race conditions and B) updating extension code that hasn't been changed in a very long time for gil-less execution. Hopefully I'll have a more thorough doc for this in the coming weeks, along with Beam removing 3.9 support and starting to add 3.14 support.

[reneleonhardt]

Shouldn't this be split into

• Remove Python 3.9 support
• Support Python 3.14
• Support Python 3.14 free-threaded

tensorflow doesn't support Python 3.14 yet.
tensorflow/tensorflow#102890
conda-forge/tensorflow-feedstock#452

[jrmccluskey]

Removing 3.9 support is not really a prerequisite for supporting 3.14/3.14t, it's just normal housekeeping with 3.9 reaching EOL and dependencies starting to drop support themselves. That effort is also done at this point, the Beam 2.70 release will not support Python 3.9.

As for mainline 3.14 support, extras like tensorflow supporting 3.14 are not necessarily a problem for us as far as getting Beam itself to support the version. The only real hurdle is having core Beam and its dependencies supporting the version, with common extras largely just preventing us from rolling out expansive testing or publishing containers. This happened this year with apitools and Python 3.13, Beam itself supported the version for months (and users could work with Python 3.13 locally pretty easily) but GCP dependencies tied to apitools didn't work, so we couldn't run any common postcommit integration tests with Dataflow. We only "fully" supported 3.13 once apitools cut a release.

Separately, the work to support free-threading within beam can be based on 3.13t (which has the same dependency issues as 3.14t) so we don't necessarily need 3.14 support anyway.

[reneleonhardt]

Glad to hear 3.14 support is being tackled ❤️

When I see the workarounds for Python 2 and the deprecated apitools in this code base, I hope more functionality could be migrated.
https://github.com/google/apitools#alternatives-to-apitools

[reneleonhardt]

After Cloudflare took the internet down today, GitHub is returning 500 Internal Server errors too 😞
https://github.com/apache/beam/actions/runs/19480578457/job/55751320574#step:2:58

remote: Internal Server Error
Error: fatal: unable to access 'https://github.com/apache/beam/': The requested URL returned error: 500

[Tunneller]

I see a wheel for Linux 3.14t, which is great, but I'm having trouble on Windows so I am building "by hand" from setup.py. One confusion on the compilation flag. For a Python wheel should I be using /MD or /MT?

Also, do you see any downside with compiling with flags -DPy_GIL_DISABLED=1 and -DCYTHON_FREE_THREADED_SUPPORT=1

[github-actions]

This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.