refactor(annotations): role-in-key-name scheme + modular annotation processors#524
Draft
ElmoPA wants to merge 3 commits into
Draft
refactor(annotations): role-in-key-name scheme + modular annotation processors#524ElmoPA wants to merge 3 commits into
ElmoPA wants to merge 3 commits into
Conversation
…rocessors
Annotation role is now encoded in the zarr KEY NAME (annotations_task /
annotations_subtask) instead of a 'level' field inside each entry — entries
are plain {text, start_idx, end_idx} spans.
- ZarrDataset fetches ALL annotations* keys per episode (each under its own
name; keymap-declared aliases take precedence and now honor their declared
zarr_key instead of a hardcoded 'annotations' read). Per-key cache.
- annotation_collate unions list keys across the batch and fills [] for
items missing a key (annotation keys are per-episode: only sort episodes
carry annotations_subtask).
- New egomimic/rldb/annotation_processing.py: AnnotationProcessor (default:
sample one string from one key, first/random) and
SubtaskAnnotationProcessor (task + subtask roles from role-named keys; no
cross-role fallback, so a missing task never leaks the subtask target into
the prompt; tie_identical keeps eva's task==subtask draws identical).
- PI + HPT: prompt sampling delegated to the processor; annotation_key +
sampling_mode still accepted and build the default processor, so every
existing config behaves identically.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TYmG3nhxt7LYiaaSJPsKEV
Contributor
Author
This stack of pull requests is managed by Graphite. Learn more about stacking. |
…d discovery, subclass cache, features flush - norm-stats inference: internal DataLoader gets _tensor_only_collate (drops the variable-length annotation lists the dataset now always emits) — default_collate crashed on ragged/per-episode-differing list keys. - annotation discovery + reads go through the actual zarr STORE, not the features attrs metadata: post-hoc injected annotation arrays never updated features, so metadata gating missed them (incl. existing injected zarrs). - ZarrWriter.append_annotations merges the new key into store.attrs features so metadata listings see future injections. - ZarrActionExpertDataset keeps its span entries on _ann_entries instead of clobbering the base per-key cache dict with a list (TypeError under the new cache + fetch-all). - ZarrAnnotationCutoffDataset unions the EOS clamp over ALL annotation keys (role-keyed episodes have no plain 'annotations'; clamp silently no-oped). - HPT honors an explicitly-provided annotation_processor when annotation_key is None (gate matched PI's behavior). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TYmG3nhxt7LYiaaSJPsKEV
The base class no longer commits to one-string-per-key sampling — that
policy moves to DefaultAnnotationProcessor. The ABC only defines
__call__(batch, batch_size) -> {role: per-item outputs}, with the entry
type processor-defined (a subclass may emit candidate lists, structured
targets, ...). SubtaskAnnotationProcessor now extends the Default; PI/HPT
back-compat shims construct DefaultAnnotationProcessor. Behavior unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TYmG3nhxt7LYiaaSJPsKEV
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Annotation role is now encoded in the zarr KEY NAME (annotations_task /
annotations_subtask) instead of a 'level' field inside each entry — entries
are plain {text, start_idx, end_idx} spans.
name; keymap-declared aliases take precedence and now honor their declared
zarr_key instead of a hardcoded 'annotations' read). Per-key cache.
items missing a key (annotation keys are per-episode: only sort episodes
carry annotations_subtask).
sample one string from one key, first/random) and
SubtaskAnnotationProcessor (task + subtask roles from role-named keys; no
cross-role fallback, so a missing task never leaks the subtask target into
the prompt; tie_identical keeps eva's task==subtask draws identical).
sampling_mode still accepted and build the default processor, so every
existing config behaves identically.
Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01TYmG3nhxt7LYiaaSJPsKEV