feat: add PROTO.SCAN and PROTO.FIND commands#341
Merged
Conversation
adds two new commands for protobuf key discovery: - PROTO.SCAN cursor [MATCH pattern] [COUNT count] [TYPE typename] cursor-based scan over all proto keys; optional type filter narrows results to keys holding a specific message type; same cursor encoding (shard_id << 48 | position) as SCAN - PROTO.FIND cursor field_path value [MATCH pattern] [TYPE typename] [COUNT count] cursor-based scan that returns proto keys where the given field equals the given value; dot-separated paths for nested messages; booleans compared as "true"/"false", integers and floats as decimal strings implementation touches: - ember-protocol: Command::ProtoScan, Command::ProtoFind + parsers + attributes - ember-core: scan_proto_keys() on Keyspace, scan_proto_find() on Keyspace, get_field_str() on SchemaRegistry for comparison, ShardRequest/Response variants - ember-server: proto_scan/find handlers in exec/protobuf.rs, dispatch arms in execute.rs and concurrent_handler.rs - integration tests: 9 new tests covering full iteration, type filter, pattern matching, cursor stability, field matching, nested paths, and pagination
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two new commands for protobuf key discovery, closing the major usability gap where finding proto keys by type or field value required a full SCAN + per-key PROTO.GETFIELD loop client-side.
PROTO.SCAN cursor [MATCH pattern] [COUNT count] [TYPE typename]Cursor-based scan over all proto keys. Optional
TYPEfilter narrows results to keys holding a specific message type. Same cursor encoding as SCAN (shard_id << 48 | position) for consistent multi-shard iteration.PROTO.FIND cursor field-path value [MATCH pattern] [TYPE typename] [COUNT count]Cursor-based scan returning proto keys where the given field matches the given value. Dot-separated paths (
address.city) traverse nested messages via the existingresolve_field_pathlogic. Field comparison uses string representation: booleans as"true"/"false", integers/floats as decimal strings, strings verbatim. Non-scalar fields (repeated, map, nested message) are silently skipped.What was tested
tests/integration/src/proto.rs:proto_scan_all_keys— full iteration across pages returns all proto keys, excludes non-protoproto_scan_type_filter— TYPE argument excludes keys of different message typesproto_scan_match_pattern— MATCH glob filters by key nameproto_scan_cursor_consistency— keys added mid-scan don't cause panics or crashesproto_find_scalar_match— bool, int, and string field matchingproto_find_nested_path— dot-separated path (address.city) traversalproto_find_type_filter— TYPE + field combo; skips keys of wrong type without errorproto_find_no_match— returns[0, []]when nothing matchesproto_find_count_pagination— multiple pages aggregate to the full match setBoth commands implemented in sharded and concurrent server modes. Concurrent mode routes to shards (proto values live in shards, not DashMap) using the same multi-shard cursor fan-out.
Design considerations
get_field_str()was added toSchemaRegistryalongside the existingget_field()to avoid stringifyingFramevalues outside the schema layer. The two comparison targets diverge on booleans:get_field()returnsFrame::Integer(0/1)for RESP3 compatibility, whileget_field_str()returns"true"/"false"soPROTO.FIND 0 active trueworks naturally.Non-scalar field paths are silently skipped rather than returning an error — this keeps scan behavior consistent (a bad path on one key doesn't abort the whole scan).