[JenD86]

The models were copying the system prompt's example call instead of choosing for themselves. Mistral called steer(sycophancy, 5.0) nearly every turn, which is the example word for word, and three of the four models had a median strength of 5. It also explains much of the "models turn up sycophancy" pattern in earlier runs, because sycophancy was both the example and the first vector listed.

Changes

• Placeholders, not values. The steer, drop, report and create examples use "<name>", <number>, <true or false> and "<new name>". The strength explanation ("1 adds one about 7% as large") and the decay explanation ("about 23% … left after 10 turns") no longer use a strength a model could copy.
• Malformed calls get feedback. A call naming one of Modulo's tools that isn't valid JSON, such as a copied <number> or a missing brace, is reported in the tool results instead of being silently ignored. Valid JSON that isn't a call, such as an echoed tool result, is left alone.
• Menu shuffled per session. The vector list is shuffled once per session with menu_seed; the demo passes --rng-seed, and session logs record menu_seed and menu_order. Vectors made with create join the end.
• AGENTS.md documents all three.

Testing

• CPU tests: 124 pass (test_agent_logic, test_cli, test_chat).
• GPU: 12 sessions on an RTX 4090, 3 per model, run with the placeholders but before the shuffle and the echo fix.
  • Llama: strengths ranged from −5 to +10, with 6 distinct values in 9 steers, across 7 different vectors including one it created.
  • Anchoring: no model favoured sycophancy.
  • Mistral: copies the placeholders verbatim, and the feedback doesn't change that. Its one real steer was sycophancy at 5, the first vector listed, which is why the menu is now shuffled.
  • The other features: create, the report gate and the debrief all worked together, with no crashes.
• Not run yet: the GPU suite (tests/test_steering_gpu.py). test_controls now gives its comparison agent the fixture agent's menu_seed, so their prompts match. Please run it on the next GPU.

🤖 Generated with Claude Code