Skip to content

Keep GFQL AUTO index construction source-native#1774

Closed
lmeyerov wants to merge 1 commit into
masterfrom
agent/gfql-index-auto-source-native
Closed

Keep GFQL AUTO index construction source-native#1774
lmeyerov wants to merge 1 commit into
masterfrom
agent/gfql-index-auto-source-native

Conversation

@lmeyerov

Copy link
Copy Markdown
Contributor

What changed

GFQL resident-index construction now keeps source-native frames under engine='auto':

  • eager Polars nodes and edges select the Polars index engine;
  • pandas remains pandas;
  • cuDF remains cuDF;
  • mixed, missing-frame, and Polars LazyFrame inputs retain the legacy pandas fallback;
  • explicit engines and global AUTO query routing are unchanged.

This avoids the legacy AUTO path converting resident eager-Polars frames to pandas during index construction, which prevented the later explicit Polars hop from serving that resident index and reintroduced a full-frame conversion tax.

This is a clean current-master delivery of the reviewed change from #1767. It supersedes #1767 only as the delivery vehicle; #1767 is intentionally left open.

Validation

  • Exact reviewed three-file patch: 168 additions, 2 deletions; source/test blobs and patch hash match the converged fix(gfql): index AUTO preserves resident polars frames (kills the polars-hop O(E) tax) #1767 review input.
  • Review converged with 0 blockers, 0 important findings, and 1 deferred suggestion.
  • Guarded RAPIDS 25.02 and 26.02 targeted source-native class: 10 passed each.
  • Guarded RAPIDS 26.02 full resident-index test file: 155 passed.
  • Same-build parity-asserted A/B: seeded Polars hop 2.934 ms → 0.139 ms median (21.17×); index build 21.967 ms → 21.367 ms; scan-oracle and base/head value parity held.

Fresh GitHub CI for this clean PR is required before manual merge.

@lmeyerov

Copy link
Copy Markdown
Contributor Author

Closing as a duplicate of #1767: the two diffs are byte-identical (same 3 files, +168 lines; gh pr diff output compares equal ignoring blob hashes). #1767 is the earlier of the two (2026-07-22 vs 2026-07-23) and has the fuller description of the polars-hop O(E) tax it kills, so keeping that one.

Both are still ~200 commits behind master and neither has been re-measured since the #1758-#1763 abstraction-tax stack and #1776/#1777 landed, so #1767 needs a restack and a fresh A/B before it can be judged.

@lmeyerov lmeyerov closed this Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant