Skip to content

Let --data read the body: @<path> from a file, @- from stdin - #4

Merged
mattpodwysocki merged 1 commit into
mainfrom
data-from-file
Sep 14, 2026
Merged

mattpodwysocki merged 1 commit into
mainfrom
data-from-file

Conversation

@mattpodwysocki

Copy link
Copy Markdown
Contributor

Ported from mapbox-cli-private#136, which can no longer merge there now that oss/ is a submodule (#132). CI was green there; re-verified here.

--data can now read the request body instead of carrying it:

mapbox styles create --data @style.json
jq '.name = "Renamed"' style.json | mapbox styles update STYLE_ID --data @-

Why

--data "$(cat style.json)" works on macOS and Linux and nowhere else this CLI ships to. There is no cmd.exe equivalent of $(cat …) on a platform with its own installer and CI leg; the body lands in the process table where ps shows it to other users; and it breaks at a size that fails in the shell, before mapbox runs, so no remedy string could ever be seen.

Exactly five operations take --data, so the blast radius is knowable: geocoder batch-geocode, styles create, styles update, styles draft update, sprites delete-batch.

The part that would have been silently wrong

Payload::Bounded's doc comment read:

A --data body was typed on a command line, and every operating system caps how much one of those can hold — two megabytes at the outside, which goes out inside the ordinary budget with room to spare.

Correct until this change, and exactly what @<path> breaks: BodySource::Json looks identical whether it was typed or read from a 200 MB file, and the second would have been handed a 60-second budget instead of the 900 seconds --file gets. Nothing would have failed in CI — a large body would just have timed out with an error blaming the network. payload_of now takes the provenance rather than inferring it from a body that no longer carries the distinction.

Three decisions, each with a test

  • Read as text, not bytes. A JSON body has to be UTF-8, so a file that isn't says so rather than being lossily converted into a body the API rejects for reasons naming nothing the caller did.
  • Sent byte for byte. The trailing newline an editor leaves is not stripped; trimming a supplied body would be the CLI editing what it was asked to send.
  • An empty source names itself. Otherwise cat missing.json | mapbox … fails as EOF while parsing a value, which describes the parser rather than the pipe whose own error scrolled past.

One interaction documented rather than changed

Of the five, only sprites delete-batch is a DELETE, and a confirmation needs stdin to be a terminal — which a pipe is not, so confirm::decide returns Proceed. Piping a body therefore sends it unasked, exactly as < file always did. @<path> leaves stdin alone and keeps the prompt.

This was the one thing I'd flagged as needing a decision — whether @- must imply --yes or error. It doesn't: the thing that makes @- possible is the same thing that already suppresses the prompt.

The @ ambiguity

Only the first character decides, so --data '{"contact":"a@b.example"}' is still the body it looks like. A body whose first character is a literal @ can't be passed this way — curl has the same limitation — and it costs nothing because all five operations send JSON and @ is not valid JSON. --data-raw is the escape hatch if a text body that could start with one is ever wired up; noted in the comment rather than built for a case that doesn't exist.

Verified

548 tests, fmt and clippy clean. Before the port, against the live API: batch-geocode --data @queries.json and the same piped through @- both returned Helsinki and Tampere. Under --dry-run: the 242 KB style from the issue at 241,946 bytes, a missing file, a bare @, a directory, an empty pipe, and a non-UTF-8 file each reporting the actual mistake.

@zmofei zmofei left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — verified diff, timeout budget threading, edge-case tests, and CI (all green).

`--data` could only carry a body typed on the command line, so sending a
style from a file meant `--data "$(cat style.json)"`. That works on macOS and
Linux and nowhere else this CLI ships to: there is no `cmd.exe` equivalent of
`$(cat …)`, the body lands in the process table where `ps` shows it, and it
breaks at a size that fails in the shell *before* `mapbox` runs — so no
remedy string can explain it.

`@<path>` and `@-` are curl's spelling. Only the first character decides, so
`--data '{"contact":"a@b.example"}'` is still the body it looks like; a body
whose first character is a literal `@` cannot be passed this way, which costs
nothing because all five operations that take `--data` send JSON and `@` is
not valid JSON.

Resolved before the dry-run branch, so `--dry-run` validates the file too.

**The timeout had to move with it.** `Payload::Bounded`'s reasoning was "a
`--data` body was typed on a command line, and every operating system caps
how much one of those can hold" — true until this change, and
`BodySource::Json` looks identical whether it was typed or read from a 200 MB
file. A read body now gets the 900-second transfer budget, so `payload_of`
takes the provenance rather than inferring it from a body that no longer
carries the distinction.

Three smaller decisions, each with a test: read as text, because a JSON body
has to be UTF-8 and a lossy conversion would be rejected for reasons naming
nothing the caller did; sent byte for byte, because trimming a supplied body
would be the CLI editing what it was asked to send; and an empty source names
itself, because `cat missing.json | mapbox …` otherwise fails as "EOF while
parsing a value", which describes the parser rather than the pipe.

Documented one interaction rather than changing it: of the five operations
only `sprites delete-batch` is a DELETE, and a confirmation needs stdin to be
a terminal — which a pipe is not. So `@-` sends it unasked, exactly as
`< file` always did. `@<path>` keeps the prompt.

Verified live against the API through both forms, not only under `--dry-run`.

Ported from mapbox/mapbox-cli-private#136, which cannot merge there now that
`oss/` is a submodule (mapbox/mapbox-cli-private#132).
@mattpodwysocki
mattpodwysocki merged commit 6b2f8de into main Sep 14, 2026
8 checks passed
@mattpodwysocki
mattpodwysocki deleted the data-from-file branch September 14, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants