Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,11 +466,22 @@ runtime uses. It returns a signed `RewardEvidenceReceiptV1`: the terminal
effect landed, or it didn't, or the store couldn't be read and the episode is
unscored. Unscored is never 0.

The worker is not in a published release yet, and the release carrying it has
no date. These two commands are what will work once it lands:

```bash
pip install 'openadapt-flow[reward]'
openadapt-flow serve-reward --seed-mockmed --port 8788
```

To run it today, install from the repository head:

```bash
git clone https://github.com/OpenAdaptAI/openadapt-flow
cd openadapt-flow && pip install -e '.[reward]'
python -m openadapt_flow serve-reward --seed-mockmed --port 8788
```

A reward receipt isn't an Execute Seal. A model rollout isn't a qualified
program, so it never gets one, and the receipt never says Flow governed the
policy. The adapters for TRL's `GRPOTrainer` and verl's reward manager live in
Expand Down
35 changes: 35 additions & 0 deletions docs/EFFECT_KIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,7 @@ effects:
patient_id: {param: patient_id} # binds to THIS episode's patient
type: Triage
expected_count: 1
count_new_only: true # and to THIS episode's write
```

Wrong, and `RewardBundle.load` refuses it:
Expand All @@ -88,6 +89,40 @@ patient whose record was never read. The load-time guard closes that. A bundle
whose required effects select no record by a declared identity key fails to
load, and the error names the missing keys and the selector to add.

## The claim names the change

A required effect that names its subject can still be satisfied by a row that
was already in the store. `record_written` and `field_equals` are statements
about the store's current contents, so a rollout that did nothing collects
the full reward whenever the subject already has a matching row.

`count_new_only` and `exact_new_set` are the two kinds the judge settles
against the pre-episode baseline, so a reward contract's required effects
must include at least one of them. `RewardBundle.load` refuses a contract
where none does. The required effects are judged as a conjunction, so one
change claim is enough to make `verified` mean the episode added something:
pair a `count_new_only` write with as many `field_equals` read-backs as the
contract needs.

The baseline comes from `begin_episode`, which the environment calls before
the rollout runs. Without it the change claim is INDETERMINATE and the
episode is unscored, never zero.

## The tier comes from the mechanism, not the recipe name

`json_file` and `screen_dump` build the same reader over a JSON document on
the worker's own disk, so they read through the same channel, `ocr`, at tier
0. A contract that declares `file` for a `json_file` recipe fails to load.
Nothing in a JSON document separates a system-of-record dump from a screen
scrape, and letting the recipe kind decide would let the same bytes earn tier
0 under one name and tier 2 under the other.

`sqlite` earns tier 2 because the worker opens a real database read-only and
runs one SELECT through the engine, and it checks the file header, so a
screen dump renamed `store.db` is refused. `rest` and `fhir` earn it on the
network read the worker performs; whether the endpoint is the customer's
system of record is the bundle author's claim and nobody checks it here.

`idempotency_key` counts as a selector too, because it filters the matched set
by `key_field`. `value` does not. On a `field_equals` contract `match` chooses
the record and `value` asserts its content, so a subject id in `value`
Expand Down
224 changes: 177 additions & 47 deletions docs/REWARD_WORKER.md

Large diffs are not rendered by default.

13 changes: 11 additions & 2 deletions openadapt_flow/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7794,8 +7794,9 @@ def _repair_store_flag(rp: argparse.ArgumentParser) -> None:
"--seed-mockmed",
action="store_true",
help=(
"Write the synthetic MockMed reward bundles (tier-2 file oracle "
"with a calibrated synthetic certificate; tier-0 screen dump)"
"Write the synthetic MockMed reward bundles (tier-2 read-only "
"SQLite oracle with a calibrated synthetic certificate; tier-0 "
"screen dump)"
),
)
p.set_defaults(func=_cmd_serve_reward)
Expand Down Expand Up @@ -8150,6 +8151,14 @@ def _cmd_serve_reward(args: argparse.Namespace) -> int:
print(f" {REWARD_NOTICE}")
for label, path in seeded_paths.items():
print(f" seeded {label:<6} {path}")
if seeded_paths:
print(f" store {data_dir / 'mockmed' / 'records.db'}")
print(f" screen {data_dir / 'mockmed' / 'screen.json'}")
print(
" Register each episode with POST /v1/episodes BEFORE the "
"rollout runs. A required effect asserts a change, so the worker "
"needs the pre-episode baseline and fixes the subject in advance."
)
serve(worker, host=args.host, port=args.port)
return 0

Expand Down
Loading