Repository navigation
feat: unified chunk relations, importance, and entities-first metadata - #462
Merged
Merged
Conversation
- Removed the `chunk_stats.json` reference from the documentation as it is no longer used. - Updated the `importance` field in the metadata schema to improve clarity on its usage. - Refactored the asset connection logic to streamline the addition of `embedded_by` relationships for images and tables. - Introduced a new `asset_region` metadata structure to encapsulate asset positioning and dimensions. - Adjusted tests to validate the new asset metadata and connection behaviors, ensuring consistency across chunk and zip formats.
Wire retrieval importance into the public result projection, migrate metadata consumers to entities with legacy fallbacks, and stop emitting redundant metadata.tokens from chunk/ZIP payloads. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
embeds/embedded_by/related), filter hydration to forward relations only, and align ZIP imageconnect_towith API/DB.importanceon public results; preferentities(with legacykeywords/tokensfallback) in search, asset tools, and graph keyword extraction.metadata.tokensfrom chunk/ZIP payloads; keepkeywords/lengthfor SDK and offline-table compatibility.Test plan
pytestcontract suite for relations / page-memory / table embedsmake check(ruff + pyright)importanceand asset chunks carryembedded_byMade with Cursor