Repository navigation
Exporter 1.2.0: scriptable, retries a typo, and says what its logs expose - #98
Merged
Merged
Conversation
Two things issue #89 exposed, and the second is the one that matters. **The one error that most needs a traceback printed none.** `UnhandledProtocolError` means Apple sent a shape the library does not model - not the user's mistake, nothing for them to correct - and the message names a symptom rather than a place. "Truncated DER: element claims 109 bytes, 61 remain" is identical from a dozen call sites, so a report carrying only that line cannot be acted on. It now prints the stack under -v, and without it says to re-run. **And then telling people to attach that output was advice we had not read.** `-vv` turns on findmy.keychain, which describes escrow records the way a person recognises them: device name, model, serial. It prints keychain item attributes verbatim, `acct` and `labl` included, whatever Apple happens to store there. It names every peer in the trust circle. The docs said "this is what to attach to a bug report" and said nothing about any of that. Not solved by scrubbing, and deliberately not attempted: the text comes from a library reading Apple's structures, so no allow-list here can promise a field is safe on every account. What it can do is say so, specifically enough to act on - a serial and an `acct` field are things somebody can find and delete, where "contains personal information" is not. So the warning prints from the CLI itself whenever logging is turned on, in red, before the output it is about rather than after it has scrolled. Colour is skipped when stderr is not a terminal, because the next step is usually piping to a file to attach, and escape codes in that file are noise in the issue. NO_COLOR is honoured. The same warning is in docs/how-to-export-with-the-cli.md and the wiki page.
The CLI can warn somebody before they turn logging on. The wizard cannot: it logs at INFO unconditionally, because a windowed build has no console and a log that must be enabled is a log nobody has when it is needed. So the file already exists on every machine that has run this, holding what INFO holds - escrow records described by device name, model and serial, keychain item attributes as Apple stores them, an identifier per peer in the trust circle. The warning goes in the file rather than only in the window, because the file is what travels. Somebody asked to attach exporter.log sends it and never sees the window again. At both ends, for different reasons. The end is where a reader starts, since that is where the failure they came for is, and a banner only at the top is buried under the run's own output within seconds. But the end can only be written on the way out, and the interesting runs are the ones that die - so the head copy is the one that is always there and the tail copy is the one that gets read. The tail is guarded against writing twice, which --self-test would otherwise do.
Same mistake the wizard's log banner was written to avoid, not carried back to the CLI. The copy printed when logging is turned on is thousands of lines up by the time the run finishes, and the end is both where somebody is looking and where they start selecting from - so in practice the warning was not given. In a finally, so it fires on a successful export too. That is the common case: the run works, the output is interesting, and it gets pasted.
Both entry points made exactly one attempt, and a rejection propagated out of the whole export - so one wrong character in a hidden field threw away the sign-in, the verification code and everything after it, and the user began again from their Apple ID. That is the wrong shape for this failure in particular. FindMy.py's own advice on a rejection leads with trying again *with the same passcode*, because the call has been observed to fail intermittently and then succeed; its second point is that a wrong passcode and a mis-parameterised exchange are indistinguishable here, so a rejection is not proof the passcode was wrong. Both say retry, and only a loop makes retrying cheap. Bounded at three, and nothing retries on its own: the rejection callback is asked each time and a no stops. Attempts are probably a limited resource - the library says Apple's escrow services generally cap them and that this one's allowance is not established - which is not a thing to discover on somebody's real account, so spending another is the user's decision. The loop lives in icloud.py because the wizard needs it as much as the CLI; each supplies its own way of asking, the way the sign-in already takes `get_code`. Also stops main reporting this as "Apple returned something unexpected" over a traceback. RecoveryError is an UnhandledProtocolError by inheritance but it is not an unmodelled shape - it is a documented rejection carrying three ordered pieces of advice, and the wrapper was both wrong and covering the part worth reading. Tests use a fake client and fake errors, including the intermittent case where the identical passcode is refused and then accepted. Checked by mutation: five of them fail if the cap goes back to one.
The passcode loop landed and the two stages before it still ended the run on a typo, which is the worse half: a rejected password or code throws away everything already entered, and the verification code cannot simply be re-entered once it is gone. Each stage is bounded for its own reason, and none retries on its own. **Password.** Capped at three and always asked, because Apple locks an account after enough failed sign-ins and how many is not a thing to establish by experiment on somebody's real account. Both fields are offered again rather than only the password - the Apple ID may be the one that is wrong, and a retry that cannot correct it is a dead end with a text box in it. **Verification code.** Re-typing is the default and sending a new code is a separate choice, because requesting delivery again invalidates the code Apple already sent (findmy-export 01-authentication §5). Getting that backwards would take away the code sitting on the user's phone as part of recovering from a mistyped one. A test asserts the retry path requests nothing. Both loops take a callback, so the CLI asks in a terminal and the wizard in a dialog, matching how sign-in already takes get_code. Passing no callback keeps the old one-attempt behaviour, which is what the tests for everything else rely on. test_unlock_retry.py becomes test_retries.py, since it now covers all three. 20 tests, fakes throughout - no account is touched.
Headless and scriptable are different things: the CLI could already run with no window, and still stopped to ask five questions. Answers can now be given up front, and --non-interactive fails instead of asking rather than hanging a cron job on a keystroke that never comes. **There is deliberately no --password flag, and the tests say so.** A command line is readable by every user on the machine through `ps` and lands in shell history, and offering the flag guarantees it gets used. The ways in, worst last: a file whose permissions are checked (or `-` for standard input), then an environment variable, then being asked. argparse made that harder than it looks. Prefix matching is on by default, so `--password hunter2` was quietly accepted as `--password-file hunter2` - the exact spelling somebody reaches for, taken without complaint, failing later on a missing file called `hunter2` with the password already in `ps` and history. `allow_abbrev` is off, and `--out` is spelled out as an alias so the useful half of abbreviation survives. A file that other users can read is refused rather than warned about: permissions are the only reason to prefer a file over an environment variable, so one without them is worse than what it replaced. Nothing is trimmed either - a password may legitimately end in a space, and removing it produces a rejected sign-in indistinguishable from a wrong password. --all-tags leaves out your own iPhones and Macs unless --include-my-devices says otherwise. "All my tags" and "all my tags plus the laptop I am sitting at" are different requests, and only one is what a cron job meant; the interactive path asks again before including one, and a scripted run has nobody to ask. The prompt guard lives in prompts rather than at each call site, and a test walks the module asserting every coroutine there is guarded - so a prompt added next year fails that instead of quietly hanging somebody's schedule. 35 new tests. Nothing touches an account.
Says the thing that is easy to get wrong and expensive to discover: sign in by hand once first. That run registers this machine on the Apple account and remembers it, so later runs usually skip the verification code - and until it has happened no combination of flags makes the run unattended, because Apple sends a code to a phone and something has to type it. The password advice is ranked rather than stated, because "use an environment variable" is common advice and is only second best. A file with permissions beats it: a variable is readable through /proc by the same user's processes, is inherited by every child, and setting it inline on a command puts it in history regardless. Both beat a command-line flag, which is why there is not one. Also written down: --all-tags omits your own devices, and exit code 1 can accompany a bundle that was written, so "a file exists" is not success.
…n the account list Two things, both found by running the wizard. **A rejected password reached the "please report this" dialog.** InvalidCredentialsError fell through to the generic handler, so entering the wrong password - then choosing Cancel rather than retrying - ended in a window naming an exception, quoting a log path, and asking the user to open a GitHub issue about their own typo. It now gets a plain message saying nothing was changed and the button is still there. RecoveryError gets the same treatment, since three rejected passcodes ended the same way. **And Cancel meant the wrong thing.** Returning None from the retry callback re-raises Apple's rejection, so a person whose last action was pressing Cancel was told "Password authentication failed". Cancelling now raises its own error saying signing in was stopped. **The device is also named now.** FindMy.py grew `device_name=` and `announce_device()` this morning - the ask in docs/findmy-py-device-name-request.md - so the entry a user sees in their Apple device list reads "OpenTagViewer Exporter" rather than a bare "MacBook Pro" sitting among their real Macs with a Remove from Account button beside it. Pin moved dbed340 -> 534867d. Announced on the first run only. Whether an identity is already stored is exactly whether this installation has registered before, so it is also the test for whether there is anything new to name - and announcing writes to the account, so doing it per export would be a write per run to say something already true. A failure to name is logged, never fatal: what is lost is a label, and failing a sign-in that already succeeded over one would be the wrong trade. CLOUDKIT_DEVICE_NAME becomes DEVICE_NAME, since two places now read it and rule 11 says they must agree. The old spelling stays as an alias.
The naming shipped in a form that could not work for anybody who already used the exporter, which is everybody it was for. It announced only when no identity was stored, on the reasoning that a stored identity meant the device had registered before. True, and the wrong question: every existing user has an identity and no name, so the check skipped precisely them. Confirmed on a real account - the device list still read "MacBookPro", with the serial correct beside it, because the file was written yesterday. Whether this installation has been *named* is now a field rather than something inferred from a file existing. Announcing renames the registration that signing in already made; it does not add a second one, so an existing install has its entry corrected on the next run and never again. A failed announce is recorded as not done, so it is retried rather than left wrong forever with no way back short of deleting the file. 1.1.0 is published and is the current release, so this cannot ship under that number - two builds stamping `OpenTagViewer.wizard:1.1.0` into exports is the drift rule 9 exists to stop. 1.2.0 rather than 1.1.1 because since 1.1.0 there is a whole scripting surface, retries at all three prompts, and a change to what appears in the user's Apple device list. Only the wizard dialog fix is patch-shaped. `scripts/release_version.py --kind exporter --tag exporter-v1.2.0` passes. 447 tests.
Naming the device fails with HTTP 401 on a live account. Not a missing token - the library raises InvalidStateError for that and did not - so a well-formed request was rejected. Two headers diverge from Stage 2 §7, and the specification states both explicitly, so this is a report rather than a question and the spec needs no further work. X-MMe-Client-Info sends the AOSKit/Xcode variant where §7 asks for mmeClientInfoAkd, so the request tells a Grand Slam endpoint that Xcode is speaking while its User-Agent says akd. A consumer cannot correct it: the provider exposes 'client' and no akd form. And the hardcoded User-Agent names Darwin 18.7.0, which is macOS 10.14, while the client info claims 13.4.1 - Darwin 22. Stage 1 §2.2 requires those to describe one real release. Neither is proven to be the cause and the report says so, along with the two candidates that could not be tested from here.
534867d -> ab18447. The announce now sends client_akd and an akd user agent whose CFNetwork and Darwin versions describe the same release its client info claims. `client` is unchanged, which matters: §2.2 is explicit that altering an established identity invalidates sessions and adds a device-list entry rather than renaming one. Does not claim to fix the 401 - neither divergence was ever traced to it. 447 tests still pass; whether Apple accepts the request is a question only a live account answers.
FindMy.py now reports a refused announce with its body and headers, and the full reply at DEBUG. None of it would have reached the log: configure_logging enumerated findmy.cloudkit, findmy.keychain and findmy.icloud - every part anybody had needed so far, and therefore wrong the first time a new one mattered. announce_device logs under findmy.reports, so the diagnostic written specifically to explain an HTTP 401 was discarded by the flag turned on to read it. Now the package, not a list of its subpackages. The reason it was a list is still real - the root logger at DEBUG carries aiohttp and asyncio, which bury everything about Apple in socket chatter - so `findmy` is the level that excludes those without predicting which of its modules matters next. Two tests: that findmy.reports.account is covered, and that aiohttp and asyncio still are not. Pin acd17c50.
…e to send ec -800012, 'Push token is invalid.' Neither header divergence caused it; both were real and both are fixed. ptkn is omitted deliberately - Stage 2 §7 says to omit the key entirely, and Stage 1 §13 argues why: a push token is the likeliest reason a registered device becomes trusted for verification codes, and this client must never be a second factor for somebody's Apple ID. Apple appears to read absent as invalid, which is a reading of one error string rather than a proven mechanism, but it fits an endpoint whose purpose is announcing a live client. Making it succeed means registering with APNs and becoming push-addressable, which is the thing §13 forbids, and is not a small change either. The safety property currently holds and the user can see it: the device list says the entry cannot receive verification codes. That is worth more than a better name. So this consumer is not asking for ptkn support and would not use it. The open question is for the specification: whether §7's naming is reachable at all by a client that declines to be push-addressable, or whether it should say that it is not.
…t do it
Removed rather than left failing. `announce_device` is refused with
HTTP 401, {'ec': -800012, 'em': 'Push token is invalid.'}
and `ptkn` is the one field the flow omits on purpose. A push token is what makes a registered
device trusted for verification codes, and 01-authentication §13 calls that "a boundary to hold
rather than a gap to close" - an exporter that became a second factor for somebody's Apple ID
would be a far larger thing to have compromised. Apple appears to read absent as invalid, which
fits an endpoint whose job is announcing a client as live, and live means reachable by push.
So the call cannot succeed while that boundary holds, and keeping it meant every fresh install
making a doomed request and logging a warning about it for good.
**Nothing is lost that was working.** Signing in registers the device by itself (§13) and always
did; the announce only ever set the name. The entry, its model, its version and its serial are
all unaffected - and §13's own answer to this is the serial, the one field sign-in controls that
a person can read, which is why `0PENTAGXPORT` is legible rather than plausible. The row also
still says the device cannot receive verification codes, which is the boundary being visible.
DEVICE_NAME stays: CloudKit is still told it, and that call works. The `announced` field goes
with the code that wrote it; a stale one in an existing file is ignored on load.
saamdotexe
pushed a commit
to saamdotexe/OpenTagViewer
that referenced
this pull request
Oct 22, 2026
…-1.2.0 Exporter 1.2.0: scriptable, retries a typo, and says what its logs expose
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything in exporter
1.2.0. Five themes, in the order they were found — each one came out of running the previous.15 files, +1,852 / −38. 441 tests pass; nothing here touches a real account.
1. Verbose output says what it exposes, before it prints it
The docs told people
-vvoutput "is what to attach to a bug report" and said nothing about what that contains. It contains escrow records described the way a person recognises them — device name, model and serial — keychain item attributes verbatim, and an identifier for every peer in the account's trust circle.Not solved by scrubbing, and deliberately not attempted: the text comes from a library reading Apple's structures, so no allow-list can promise a field is safe on every account. What it can do is say so specifically enough to act on — a serial and an
acctfield are things somebody can find and delete, where "contains personal information" is not.NO_COLORhonoured.Also:
UnhandledProtocolErrornow prints its traceback under-v. It was the one error where the stack is the whole diagnosis — the message names a symptom, and a dozen call sites produce it identically.2. A typo no longer ends the run
All three prompts made exactly one attempt, and a rejection unwound the entire export — so one wrong character threw away everything before it, and the verification code cannot simply be re-entered once it is gone.
Nothing retries on its own. Every attempt is offered, because the caps are Apple's and undocumented — not a thing to establish by experiment on somebody's real account.
The loops live in
icloud.pyso the wizard gets them too; each entry point supplies its own way of asking, matching how sign-in already takesget_code.3. Scriptable, without inviting a leaked password
Headless and scriptable are different things. The CLI could already run without a window and still stopped to ask five questions.
--apple-id,--password-file,--passcode-file,--device,--all-tags,--include-my-devices,--non-interactive.There is deliberately no
--passwordflag, and a test says so. A command line is readable by every user on the machine throughpsand lands in shell history. The ways in, worst last: a file whose permissions are checked (or-for stdin), then an environment variable, then being asked.argparse made that harder than it looks — prefix matching quietly accepted
--password hunter2as--password-file hunter2, failing later on a missing file calledhunter2with the password already inps.allow_abbrevis off;--outis spelled out as an alias so the useful half survives.Two more choices worth reviewing:
--all-tagsexcludes your own iPhones and Macs unless--include-my-devicesis given. "All my tags" and "all my tags plus the laptop I am sitting at" are different requests, and only one is what a cron job meant.The prompt guard lives in
promptsrather than at each call site, and a test walks the module asserting every coroutine there is guarded — so a prompt added next year fails that instead of quietly hanging somebody's schedule.4. The wizard stops calling a wrong password a bug
Entering the wrong password and then choosing Cancel ended in a dialog naming an exception, quoting a log path, and asking the user to open a GitHub issue about their own typo.
InvalidCredentialsErrorandRecoveryErrornow get plain messages.Cancel also meant the wrong thing: returning
Nonefrom a retry callback re-raises Apple's rejection, so somebody whose last action was pressing Cancel was told "Password authentication failed". It now says signing in was stopped.5. Device naming: tried, refused, removed
Attempted and then taken back out, which is worth reading as a whole rather than as three commits.
FindMy.py gained
device_name=andannounce_device(), so the entry in the user's Apple device list could read "OpenTagViewer Exporter" rather than a bareMacBookPro. Apple refuses it:ptknis the one field the flow omits on purpose. A push token is what makes a registered device trusted for verification codes, and01-authentication§13 calls that "a boundary to hold rather than a gap to close" — an exporter that became a second factor for somebody's Apple ID is a far larger thing to have compromised.So the call cannot succeed while that boundary holds, and keeping it meant every fresh install making a doomed request forever. Removed.
Nothing that was working is lost. Signing in registers the device by itself and always did; the announce only ever set the name. §13's own answer is the serial — the one field sign-in controls that a person can read — which is why
0PENTAGXPORTis legible rather than plausible. The row still says the device cannot receive verification codes, which is the boundary being visible.Two things came out of it and are kept:
User-Agentnaming Darwin 18 (macOS 10.14) beside a client info claiming 13.4.1. Neither caused the 401; both were wrong.-vvwas discarding its own diagnostics.configure_loggingenumeratedfindmy.cloudkit,findmy.keychain,findmy.icloud— every part that had mattered so far, and therefore wrong the first time a new one did.announce_devicelogs underfindmy.reports. Now the package, withaiohttpandasynciostill excluded, and two tests holding it.docs/findmy-py-announce-401-report.mdrecords the whole thing for the FindMy.py and specification sessions. The open question is for the spec: §7 documents a call that §13's boundary makes unreachable, and it should say so next to the call — otherwise the next implementer rediscovers this from a 401.Version
VERSION = "1.2.0".1.1.0is published and current, so this cannot ship under that number — two builds stampingOpenTagViewer.wizard:1.1.0into exports is exactly the drift rule 9 exists to stop.1.2.0rather than1.1.1because of §2 and §3; only §4 is patch-shaped.scripts/release_version.py --kind exporter --tag exporter-v1.2.0passes.Verification
app/.get_key_fallbackstill needs macOS 14 or older and was not run.PR description summarised by Claude Code.