Fix cross-platform builds, PyPI upload, and Beam 2.53+ compatibility

[czgdp1807]

Problem

Building and publishing tfx-bsl fails on both Linux and macOS due to multiple issues:

1. macOS linker error: ThinLTO assumes Apple's libLTO.dylib, causing failures with conda-provided compilers
2. zlib compilation: Macro conflicts between zlib 1.2.11 and modern macOS SDK headers
3. Module symbols: Python 2-era extension symbols cause linker failures on Python 3
4. HDF5 dependency: Missing system library for h5py (transitive dependency) on Python 3.10+ and 3.11+
5. Test failures: Apache Beam 2.53+ metrics API changes break telemetry tests on Python 3.11+
6. Missing dependencies: dill not explicitly listed despite being required
7. C++ compilation: Missing absl/strings/str_cat.h header causes build failure
8. PyPI upload failure: Incorrect repository URL (pypi.org/legacy/ instead of upload.pypi.org/legacy/) causes 404 errors

Solution

This PR addresses all build and test failures across platforms:

Changes

.bazelrc

• Disable ThinLTO on macOS/macOS arm64 to avoid linker errors with conda compilers
• Define HAVE_UNISTD_H=1 for zlib compatibility with modern macOS headers

.github/workflows/build.yml

• Add workflow_dispatch trigger to enable manual PyPI uploads
• Fix PyPI upload repository URL from https://pypi.org/legacy/ to https://upload.pypi.org/legacy/
• Resolves 404 errors when publishing packages to PyPI
• Successfully tested: Package published to PyPI (https://pypi.org/project/tfx-bsl-czgdp1807/1.18.0.dev0/)
• Verified in fork CI run: https://github.com/czgdp1807/tfx-bsl/actions/runs/21641775927/job/62386135354#logs

WORKSPACE

• Update zlib to version 1.3.1 (from implicit 1.2.11)
• Define explicit http_archive for zlib before tensorflow dependency loads it
• Prevents fdopen macro conflicts with macOS SDK _stdio.h

pyproject.toml

• Linux: Upgrade from manylinux2014 to manylinux_2_28 for HDF5 >= 1.10.7
• Linux: Add hdf5-devel installation in before-build step
• macOS: Add brew install hdf5 in before-build step
• Fixes h5py build failures for Python 3.10+ on Linux and Python 3.11+ on all platforms

setup.py

• Add explicit dill>=0.3.1,<1.0.0 dependency (required by Apache Beam but not declared)
• Pin tensorflow-serving-api to >=2.17.1,<3 (was >=2.13.0,<3)
• Updated all selectors: default, and nightly
• Ensures consistent protobuf/grpc versions across all build configurations

tfx_bsl/build_macros.bzl

• Remove Python 2 module initialization symbols (init_*, init*)
• Keep only PyInit_<module> for Python 3 compatibility
• Fixes undefined symbol errors during extension module linking

tfx_bsl/cc/sketches/misragries_sketch.h

• Add missing #include "absl/strings/str_cat.h" header
• Fixes compilation error: no member named 'StrCat' in namespace 'absl'

tfx_bsl/telemetry/collection_test.py

• Update tests for Apache Beam 2.53+ metrics API compatibility
• Remove with context manager pattern and explicitly call pipeline_result.wait_until_finish()
• Ensures metrics are committed before assertions run

tfx_bsl/coders/csv_decoder_test.py

• Update exception handling for Apache Beam 2.53+ compatibility
• Accept both ValueError and RuntimeError (newer Beam wraps exceptions)
• Add explicit pipeline_result.wait_until_finish() call

Testing

Local testing: macOS arm64, Python 3.12, Bazel 6.5.0

• Build completes without compiler or linker errors
• All previously failing tests now pass
• Package installs via pip install -e ".[test]"

CI validation: All checks passing on fork

• Fix macOS arm64 build compatibility with Xcode compiler toolchain clang 17 czgdp1807/tfx-bsl#1
• Tested on: Python 3.9, 3.10, 3.11, 3.12 on both ubuntu-latest and macos-latest
• Previously failing test suite (refer - https://github.com/czgdp1807/tfx-bsl/actions/runs/21594219988) now passes completely

PyPI Upload verification: Tested and working

• Successfully published package to PyPI
• Fork CI run: https://github.com/czgdp1807/tfx-bsl/actions/runs/21641775927/job/62386135354#logs
• Published package: https://pypi.org/project/tfx-bsl-czgdp1807/1.18.0.dev0/
• The workflow_dispatch trigger enables manual PyPI uploads for future releases

Impact

• Enables macOS arm64 builds with conda environments
• Fixes CI failures on Python 3.10+ (Linux) and 3.11+ (all platforms)
• Resolves HDF5 dependency issues for h5py transitive dependency
• Updates tests for Apache Beam 2.53+ compatibility
• Maintains Python 3.9+ compatibility
• Drops Python 2 support (already EOL)

Root Cause Analysis

The CI failures emerged because,

• 6 months ago: older package versions didn't require h5py or use Beam 2.53+
• Today: same commit resolves to newer package versions that:
  • Pull in h5py as transitive dependency (requires HDF5 C library)
  • Use Apache Beam 2.53+ with changed metrics API behavior

This is why Python 3.9 still passes (uses older dependency ranges) while 3.10+ fails.

[czgdp1807]

@vkarampudi @aktech This is ready to go. Please feel free to merge or let me know any changes needed in this PR.

[czgdp1807]

ModuleNotFoundError: No module named 'pkg_resources'

Minor error I guess. Will fix it after I am free from tfx.

[kmontemayor2-sc]

Adding a downstream user data point to support this PR.

The TFT/Beam combination this PR unblocks is already declared compatible upstream, but the wheel build breaks the contract. Per TFT
1.17.0's Compatible versions table:

tensorflow-transform	apache-beam[gcp]	pyarrow	tensorflow	tensorflow-metadata	tfx-bsl
1.17.0	2.65.0	10.0.1	2.17	1.17.1	1.17.1

So TFT 1.17.0 + Beam 2.65.0 + tfx-bsl 1.17.1 is explicitly the supported combination. In practice it doesn't work end-to-end in any
Beam-recommended Python runtime, because:

• The published tfx-bsl==1.17.1 wheel is tagged manylinux_2_39_x86_64.
• apache/beam_python3.11_sdk:2.65.0 is built FROM python:3.11-bookworm (glibc 2.36) and is the standard
  runtime base for Dataflow workers per Google's
  build-container-image guide.
• Result: pip install tfx-bsl==1.17.1 fails inside that image, and any TFT-on-Dataflow user is blocked at TFT 1.16 / apache-beam==2.60.0
  (or stuck on the previous compat row).

We hit this concretely in GiGL (Snap's open-source GNN training/inference library), where TFT and tfx-bsl
run inside Apache Beam pipelines on Dataflow — see
containers/Dockerfile.dataflow.base.

This PR's manylinux2014 → manylinux_2_28 change in pyproject.toml is the fix.
manylinux_2_28 installs cleanly on Debian 12 (glibc 2.36) and Ubuntu 24.04 (2.39), restoring the documented support contract. Looking at the
v1.17.1 build workflow and
pyproject.toml, the manylinux_2_39 tag appears unintentional — there's
no manylinux-x86_64-image setting and the build runs on ubuntu-latest, so auditwheel embeds whichever host glibc the runner happens to have
(currently 2.39 on ubuntu-24.04).

A v1.17.2 (or v1.18.0) cut with this PR + the dill / tensorflow-serving-api / Beam 2.53+ test fixes would unblock us and, we suspect, every
other TFT-on-Dataflow user at the documented compat row.

(Note: #100 makes the same manylinux_2_28 change as part of a larger TF 2.21 + Python
3.12/3.13 push. Either landing fixes our specific blocker; this PR's narrower scope seems easier to ship first.)

Happy to dogfood a release candidate against our Dataflow integration suite if that helps land it.