[MT255026]

Summary

Adds a Dagster I/O manager for Teradata to dagster-teradata, with type handlers for pandas, polars and PySpark DataFrames. An asset that returns a DataFrame is stored as a Teradata table and loaded back for downstream assets, with no hand-written SQL or CREATE TABLE.

Only libraries/dagster-teradata is changed (plus one CI line, see below).

What's new

I/O manager core (io_manager.py)

• TeradataIOManager, TeradataDbIOManager, TeradataDbClient and build_teradata_io_manager, built on Dagster's DbIOManager.
• Table naming from asset keys, partition support (slice cleanup before write), and a drift check that compares the frame against the existing table.

Type handlers

Handler	Extra	Notes
TeradataPandasIOManager / TeradataPandasTypeHandler	pandas	Maps pandas dtypes to Teradata types, NaN to NULL
TeradataPolarsIOManager / TeradataPolarsTypeHandler	polars	Timezone-aware timestamps, NaN to NULL
TeradataPySparkIOManager / TeradataPySparkTypeHandler	pyspark	JDBC write/read, PySpark 3.4-3.5, classic sessions only (Spark Connect is rejected with a clear error)

• Shared helpers: _catalog.py (DBC catalog reflection, character-type parsing, drift check) and _type_handler_base.py.
• Handlers are exported lazily from dagster_teradata, so importing the package does not require pandas, polars or pyspark. A missing or too-old dependency gives an actionable error.
• column_types lets callers override the Teradata type per column.

Safety checks before any destructive step. The previous slice is deleted before the write is committed, so the handlers validate first:

• character length/charset and type-drift checks, and overrides that can't be sized are rejected;
• constraint overrides (NOT NULL, PRIMARY KEY, ...) are rejected;
• PySpark: NaN is written as NULL, executor time zones must be UTC, and an unreadable catalog fails closed.

Packaging and docs

• New extras: pandas, polars, pyspark; requires-python is now >=3.10; pandas added to dev deps; uv.lock regenerated.
• README.md updated, new user guide docs/io-manager.md, a runnable examples/kitchen_sink/ project, and PLANS.md (design and decision log).

CI

• quality-check-dagster-teradata.yml: the type-check step ran uv run pyright, which isn't a dependency. It now runs uv run ty check ., matching the Makefile.

Tests

• Unit tests for the I/O manager and each handler (pandas, polars, PySpark, shared datatype matrix).
• Live Teradata tests under dagster_teradata_tests/functional/ (not run in CI; they need a database): 16/16 passed, including the PySpark JDBC path.
• Last full local unit run: 567 passed. Two test_tpt_utils permission tests fail on Windows only, regardless of this change.
• CI (ruff, ruff format, ty, pytest) passes on this PR.

Notes for reviewers

• Existing resources, TeradataResource, BTEQ and TPT code are untouched.
• Known follow-ups, not in this PR: staging-table writes (so a failed write cannot lose the previous slice) and per-type range validation for numeric column_types overrides.