Repository navigation
Port research controls, session metadata and better extraction to main - #12
Merged
Merged
Conversation
These came from research-controls (d022fe3), whose PR #2 was stacked onto fix/steering-review and merged after that branch had already merged, so none of it reached main. Ported onto the renamed modulo package and the features added since: - Controls: SteeringAgent(control="real"|"sham"|"random"). Sham applies nothing; random applies a fixed random direction of the same magnitude. Everything the model sees is identical across conditions; with disclose_controls (any --control) the prompt says, truthfully in every condition, that some sessions are controls. --control assign --blind, replay.py --blind. - Reports record the session's control condition, so sham "it worked" reports can be filtered out (optional field; older reports still valid). - Session logs start with a header (argv, git commit, versions, control, RNG seed, generation settings, rails, vector sha256s); turns record the rendered system prompt and the unsteered NLL of steered outputs. --rng-seed, --temperature, --top-p, --max-new-tokens. - Extraction averages over response tokens by default (split-half cosine on Qwen2.5-7B: sycophancy 0.33 -> 0.83, refusal 0.10 -> 0.68); --read last keeps the old readout. Sycophancy, refusal and creativity get their own question sets; the other starters use a better general mix. - Optional --nll-threshold incoherence check, off by default (NLL doesn't separate word salad from coherent heavy role-play). - CPU tests run in GitHub Actions. Not ported: making auto_extract opt-in. Main has since built on it being on by default (11 starters, only-missing extraction). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The condition is always in the log's session header; unblind with replay.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports the work stranded on
research-controls(the PR #2 merge never reachedmain) onto currentmain.What's in it
SteeringAgent(control="real"|"sham"|"random", disclose_controls=True), and in the loop--control real|sham|random|assignplus--blind. If a control is set, the system prompt truthfully says some sessions are controls, and it's identical in every condition. Tool results, decay, cooldown and the safety rails behave the same in every condition.randomuses a fixed random direction of the same magnitude for each concept.control, so sham "it worked" reports can be told apart from real ones. The field is optional, so older reports still validate.sessionrecord: argv, git commit and dirty flag, versions, the control condition, RNG seed, generation settings, rail constants, and each vector's registry entry with its sha256. New flags:--rng-seed,--temperature,--top-p,--max-new-tokens.unsteered_nllis logged on steered turns, with an optional--nll-thresholdincoherence check, off by default (see the AGENTS.md table for why no single threshold works).replay.pytakes multiple logs and has a--blindmode that hides the condition and the NLL.--read mean|last). There are dedicated question sets for sycophancy, refusal and creativity.Decisions worth a second look
auto_extractkeeps its default ofTrue. It was not made opt-in.--blindnever prints the condition, not even when the session ends. To unblind, read the log or runreplay.pywithout--blind.Verified on an A100 80GB (Qwen2.5-7B-Instruct)
pytest: 141 passed, including all 5 GPU steering tests (sham output is identical to unsteered, random differs from real, NLL is logged only on steered turns).Split-half reliability of extracted vectors at layer 14 (cosine between vectors from even and odd questions):
A 3-turn
--control assign --blindsession: the header is complete;replay.py --blindshowscontrol=(hidden)and no NLL; the unblinded replay showscontrol=real.🤖 Generated with Claude Code