Skip to content

ask's options sent as a JSON string still gets refused: unwrapEncoded does not fire when the list carries blocks #1007

Description

@santoshkumarradha

What happened

On 2026-09-12, TestQuestionsE2E/ACardWithABlockUnderEachAnswerArrivesWhole on deepseek/deepseek-v4-flash refused the model's ask call and made it write the question three times:

questions_e2e_test.go:505: the first ask did not arrive whole: 3 ask calls (1 turned away by the question gate),
  1 argument refusals, blocks mentioned N times in the journal; want one call the decoder read,
  carrying blocks, and none refused.
Refused with:
  Invalid arguments: options takes a list; it arrived as text, \"[{\\\"blocks\\\": [{\\\"body\\… — send the value itself, not a string holding it
questions_e2e_test.go:514: the page lists the answers: the screen never said "HRV".

The argument arrived as a JSON string holding a well-formed list — "[{\"blocks\": [{\"body\"… — which is the exact shape unwrapEncoded was written for after the 2026-09-10 measurement (#797). It did not fire. The difference from the shapes that decode today is what is inside the list: each option carries a blocks array, so the held string is deeply nested and several levels of escaping deep.

The scenario is the one written for precisely this, and its header says why it is not a needle problem:

On 2026-09-10 deepseek-v4-flash was given the message below — a person's own words, on the Spark, in a real terminal — and three calls running it sent the answers list as a JSON STRING holding the list; every one was refused with options takes a list, the turn ended on the loop guard and the person never saw a question.

That is what happened again. The person's own words, and no question on the screen.

Replication

Deterministic (no model). A internal/session test over toolargs.go: take an ask argument object whose options is a JSON string holding a list of options that each carry a blocks array, and decode it. It is refused with options takes a list; the same string with the blocks removed unwraps. The bytes to build the fixture from are written by the field run below — keepJournals in internal/e2e/questions_e2e_test.go copies every transcript of a failing run to $TMPDIR/aforge-e2e-journals/ and the failure message names the files, so the refused call is on whoever ran it, not only on the box that found it. That is the fixture this issue is owed and could not be pasted here: the log shows only arrivedAsText's twenty-four-rune glimpse.

Field (real models).

make build
go test -tags e2e -run 'TestQuestionsE2E/ACardWithABlockUnderEachAnswerArrivesWhole' -count=1 -timeout 15m -v ./internal/e2e/

Needs OPENROUTER_API_KEY (via liveKey) and tmux; about thirty seconds and a fraction of a cent. It is model-dependent and not every-run: three consecutive runs on 2026-09-12 from fix/e2e-question-needles went fail (2 ask calls, 1 refusal), pass (1 call, none refused), fail (3 calls, 1 refusal) — so roughly two runs in three. What makes it fire is the model choosing to wrap options in quotes on a call whose options carry blocks; the steer is a person's verbatim message on purpose and is not what varies.

Where

  • unwrapEncoded, unswallowTail, heldText and listRepair/arrivedAsText in internal/session/toolargs.go.
  • The tool that refuses: internal/session/tools_ask.go.
  • The scenario and its journal reader: questionsBlocksUnderAnswers and keepJournals in internal/e2e/questions_e2e_test.go.

The fix

unwrapEncoded already states the law — a value that has closed is whole, and what follows the first complete value is not part of it. What it does not yet survive is the nesting: an options list whose members carry their own arrays comes back through json.Unmarshal into a string and then has to parse as JSON on its own, and something in that round trip (an escape heldText does not repair, or a truncation) leaves it undecodable. Whichever it is, the answer is the same shape as the existing one: read it where it can be read, and refuse only what genuinely cannot be.

A person's question may not be lost to a model's quoting. The loop guard already carries the correction back, and it still cost three calls.

Acceptance

  • e2e: TestQuestionsE2E/ACardWithABlockUnderEachAnswerArrivesWhole passes on deepseek/deepseek-v4-flash over five consecutive runs: one ask call the decoder read, carrying blocks, and no argument refusal at all — which is what the scenario already asserts off the journal.
  • Unit: a internal/session case built from the refused bytes above, decoding to the whole question with its blocks — beside the existing unwrapEncoded cases, so the flat and the nested shapes are pinned together.

Found on 2026-09-12 while un-rotting the questions e2e needles (#998). Not caused by that work: the assertion is read off the journal's tool calls, which no needle touches.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    good first issueSmall and isolated, the acceptance already written, no ruling owed; a safe first pull request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions