refactor phase 3a: break the core knot — the import graph is now a DAG - #298
Merged
Conversation
hussainsultan
force-pushed
the
refactor/phase2-primitives-and-errors
branch
from
August 18, 2026 23:15
7b90d39 to
cc417f7
Compare
hussainsultan
force-pushed
the
refactor/phase3-core-knot
branch
from
August 18, 2026 23:15
c80ade6 to
ecd25f5
Compare
hussainsultan
force-pushed
the
refactor/phase2-primitives-and-errors
branch
from
August 18, 2026 23:26
cc417f7 to
67b9ee8
Compare
hussainsultan
force-pushed
the
refactor/phase3-core-knot
branch
from
August 18, 2026 23:26
ecd25f5 to
e9d512b
Compare
Ten representative query shapes (flat agg, filters+order+limit, derived time dim, join_one, join_many fan-out pre-agg, mean decomposition, calc percent-of-total, three-way join, filtered join, JSON query path) compile to golden SQL files. Any compilation-output drift during the restructure fails loudly; BSL_UPDATE_SNAPSHOTS=1 regenerates after an intentional change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… leaves the SCC
- The 12 lazy 'from .expr import' sites (fluent wrapper construction on
SemanticJoinOp/SemanticIndexOp, _semantic_table) and the build_query
site now go through documented call-time module accessors — the same
upward-dispatch pattern as .chart(); follow-up moves those methods
onto the wrappers entirely
- expr.SemanticJoin.group_by/filter construct their wrappers directly
instead of ping-ponging through the op layer
- _collect_struct (flavor-aware struct collection — nest lowering)
moves from expr.py down into ops.py where it's consumed
- The SemanticUnnest isinstance check duck-types on SemanticUnnestOp
- ops imports fmt from ibis directly (same singledispatch registry the
format module registers handlers on) instead of importing format
- The SCC scanner now skips TYPE_CHECKING blocks: type-only imports are
not runtime edges
SQL snapshots byte-identical; SCC: 10 -> 5 modules (expr/query/api +
serialization/{__init__,reconstruct}).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
expr's .query()/.compare_periods() and to_tagged() convenience methods now dispatch upward at call time (the .chart() pattern) instead of importing the query and serialization layers. That was the last cycle: Tarjan over the full graph (lazy imports included) finds zero non-trivial SCCs. The ratchet now enforces permanent acyclicity. Full suite green (1703 passed, exit 0); SQL snapshots byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
hussainsultan
force-pushed
the
refactor/phase3-core-knot
branch
from
August 18, 2026 23:37
e9d512b to
016622a
Compare
hussainsultan
changed the base branch from
refactor/phase2-primitives-and-errors
to
main
August 18, 2026 23:37
hussainsultan
marked this pull request as ready for review
August 18, 2026 23:42
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Phase 3a of the SCC-driven refactor, stacked on #297. This PR takes the residual 10-module strongly connected component to zero: the intra-package import graph (function-local imports included) is now acyclic, and the ratchet test enforces that permanently. Full suite green (1703 passed, exit 0).
Guard first: SQL-snapshot corpus
Before touching the compiler's structure, ten representative query shapes (flat aggregate, filter+order+limit, derived time dimension, join_one, join_many fan-out pre-agg, mean decomposition, calc percent-of-total, three-way join, filtered join, JSON query path) now compile to golden SQL files in
tests/snapshots/. Every subsequent commit in this PR — and everything in phase 3b — must keep them byte-identical;BSL_UPDATE_SNAPSHOTS=1regenerates after an intentional change. All snapshots are unchanged by this PR.The cuts
opsno longer importsexpr,query, orformat— the compiler leaves the SCC:from .expr importsites (fluent wrapper construction onSemanticJoinOp/SemanticIndexOp,_semantic_table) and thebuild_querysite go through documented call-time module accessors — the same upward-dispatch pattern.chart()established. A follow-up moves those convenience methods onto the wrappers entirely; the accessors mark the debt explicitly.expr.SemanticJoin.group_by/filterused to delegate to op methods that imported expr back — a ping-pong cycle. They now construct their wrappers directly._collect_struct(flavor-aware struct collection, i.e. nest lowering) moves fromexpr.pydown intoops.pywhere it is consumed.SemanticUnnestisinstance check duck-types onSemanticUnnestOpinstead.opsimportsfmtfrom ibis directly — the same singledispatch registryformat.pyregisters the semantic handlers on — instead of importing the format module.exprno longer importsqueryorserialization:.query()/.compare_periods()andto_tagged()dispatch upward at call time. These are genuine layer-above conveniences on the expression API (like pandas'.plot), now resolved without an import-time edge.Scanner refinement:
if TYPE_CHECKING:blocks are skipped — type-only imports are not runtime edges.Result
Tarjan over 60+ modules finds zero non-trivial SCCs.
KNOWN_SCC_MEMBERSis the empty set; any future cycle fails CI. The target layering is now realizable:_xorq → primitives → ops → expr → api/query → serialization → extras → __init__, with three documented call-time upward dispatches at user-API boundaries (chart,query,serialization).What phase 3b covers (next PR)
The physical restructure now that edges can't regress: split
ops.py(~7,600 lines) into anops/package along the audited zone map; extract the five_to_untagged_with_*compile strategies into acompile/subpackage; decompose the 1,089-line_to_untagged_with_preaggalong its own# --- N.phase markers with an explicitPreaggContext; move pruning/lineage intocompile/pruning.py. The snapshot corpus and soundness suites from this stack are the gates.Verification
python3 -m pytest src/boring_semantic_layer: 1703 passed, exit 0ruff check/format --checkclean🤖 Generated with Claude Code