WIP: joins, column transforms, JSON array support (#4, #11, #5) - #13
Draft
fahadsiddiqui wants to merge 2 commits into
Draft
fahadsiddiqui wants to merge 2 commits into
fahadsiddiqui wants to merge 2 commits into
Conversation
Implements #4 (Join/JoinOn, all four join types, composite keys, deterministic ordering, benchmarked) and #11 (Apply/Map/FillNA/DropNA) in full, with tests and README/CHANGELOG updates. Also fixes a real bug found while testing panic paths: SafeOperation never actually recovered panics because recover() was called from a helper invoked by the deferred closure, where Go treats it as a no-op. #5 (JSON array read/write) is mid-flight: json_test.go is written and jsonl.go has been refactored to expose shared, format-neutral decode/build helpers, but json.go itself isn't written yet, so the package does not currently build. See docs/PROGRESS.md for the full status and next steps across all in-scope issues (#4-#12).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Working through the open feature/test issues (#4-#12). This PR is a WIP snapshot, not ready to merge.
Join/JoinOn, all four join types, composite keys, int/float key coercion, deterministic ordering, column-collision suffixing, benchmarked (~3ms for a 100k×10k inner join).Apply/Map/FillNA/DropNAwith derived-column type inference and panic-safe user callbacks.[{...}]) read/write support #5 JSON array read/write — in progress. Tests written,jsonl.gorefactored to share its decode/build logic, butjson.goitself isn't written yet — the package does not currently build on this branch.SafeOperationnever actually recovered panics (recover()was called from a helper, not directly by the deferred func, where Go treats it as a no-op). Fixed and tested.Full status, what's left, and how to resume is in
docs/PROGRESS.md.Test plan
go test ./...green as of the feat: Join operations (inner, left, right, outer) #4/feat: column derivation & transforms (Apply, Map, FillNA, DropNA) #11 work (prior to the feat: JSON array ([{...}]) read/write support #5 tests being added)json.goimplemented before tests/CI can run again