Repository navigation
feat(harness): python node write modes (append, upsert) - #80
Merged
Merged
Conversation
carolsimone
force-pushed
the
feat/write-modes-trino
branch
from
October 11, 2026 11:31
8d7772b to
e335444
Compare
A pure comparison: names case-insensitively (exact spelling paired first), nullability exactly, order and types ignored. ShapeDiff lists missing and unexpected columns and nullability mismatches, and renders one message naming all of them and the remedy. Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw
_WRITERS maps replace, append and upsert to the capability they need and the write call. The capability is checked right after the config tripwire, so an engine that cannot append fails before the script runs. The upsert key goes to ensure_table and upsert as written; DuplicateKeyError becomes ConformError, any other write failure stays LoadError. Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw
After ensure_table, an append or upsert compares the target table's columns with output_columns (names case-insensitive, nullability exact, order and types ignored) and fails with one LoadError naming every difference. It inspects TARGET_SCHEMA.TABLE_NAME, the table actually written. Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw
Two append runs keep both runs' rows; two upsert runs overwrite by (rate_date, base) and leave the PRIMARY KEY in place; an append into a table missing a declared column fails with LoadError naming it and writes nothing. Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw
Two append runs keep both runs' rows; two upsert runs overwrite by (rate_date, base) on the DuckLake stack. Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw
Two append runs keep both runs' rows; two upsert runs overwrite by (rate_date, base) on the Trino plus Iceberg stack. Signed-off-by: Simone Carolini <simonecarolini.sc@gmail.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw
carolsimone
force-pushed
the
feat/write-modes-harness
branch
from
October 11, 2026 12:30
c43b433 to
f37b28c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sixth runtime PR of the write-modes series (spec R6). Depends on R1–R5; stacked on #78 (Trino) locally — base is
feat/write-modes-trino; retarget tomainas the stack merges._WRITERSbeside_PRODUCERS: replace ->load, append ->AppendCapable.append, upsert ->UpsertCapable.upsert. The capability is checked right after the config tripwire, soLoadError: <Adapter> does not support <mode>comes before the script runs or the warehouse is touched. (This replaces the interim guard added to feat(contract): mode_type and primary_key contract fields #74 after review — see that thread.)ensure_table(primary_key=...)andupsertas written (the loader requires each key name to match an output column exactly).continuo_python_runtime/shape.py:shape_diff(names case-insensitive, nullability exact, order and types ignored). Before an append or upsert the liveTARGET_SCHEMA.TABLE_NAMEis compared and any difference is oneLoadErrornaming all of them.DuplicateKeyError->ConformError(the script's rows are at fault), caught before the generic wrapper; every other write failure staysLoadError.PRIMARY KEY); a drifted Postgres table is refused.The version bumps and the tag follow in a separate
chore(release)PR after this merges.🤖 Generated with Claude Code
https://claude.ai/code/session_0137E9XmyUXhwPTRrtNfivDw