Summary
A token_cache.json entry can be written with expires_at: None. Because gws can then never conclude the entry is stale, it never refreshes it and keeps re-sending the same access token indefinitely. Once that token's grant ages out or is superseded, every call needing that scope fails permanently with:
401 UNAUTHENTICATED
"Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential."
details[].reason = ACCESS_TOKEN_TYPE_UNSUPPORTED
It does not self-recover, and gws auth login does not clear the cache, so re-authenticating does not fix it either. In my case gws gmail +watch was broken for ~5 days across two successful re-auths, failing at its Pub/Sub topic-creation step, while every Gmail call kept working normally.
Version / environment
gws 0.22.5 (current latest release)- Linux (Debian 12) in Docker, headless,
keyring_backend: keyring
- OAuth user credentials (
credentials.enc), 27 scopes granted including pubsub and cloud-platform
Evidence
I preserved the bad token_cache.json and decrypted it (AES-256-GCM, 12-byte nonce prefix, key from .encryption_key). It decrypts to a dict keyed by scope URL, each value {access_token, refresh_token, expires_at, id_token}.
Every entry had a real expires_at except the one for the failing scope:
key access_token expires_at
----------------------------- --------------- ------------------------------
.../auth/pubsub ya29… (254 ch) None <-- never refreshed
.../auth/gmail.modify ya29… (254 ch) [2026, 233, 6, 29, 26, …]
https://mail.google.com/ ya29… (254 ch) [2026, 233, 6, 34, 39, …]
.../auth/chat.bot ya29… (253 ch) [2026, 232, 16, 14, 46, …]
.../auth/chat.admin.spaces ya29… (253 ch) [2026, 232, 16, 34, 22, …]
The cached access_token was a well-formed ya29. value, not corrupt data — just long dead. ACCESS_TOKEN_TYPE_UNSUPPORTED appears to be what the API frontend returns for a bearer token it cannot resolve at all; I confirmed that a garbage string, a truncated ya29.…, and a 1//…-shaped value all produce that same reason, whereas an unauthenticated call gives 403 PERMISSION_DENIED instead. So the reason code is misleading here — it reads like "wrong credential type" but the real condition is "unresolvable token".
Deleting token_cache.json fixes it immediately and permanently.
Why this is easy to misdiagnose
Two properties made this cost several days of investigation:
- The cache is per-scope, so the failure is per-API. Gmail calls kept succeeding off their own valid entry, so
gws auth status was clean, gws gmail users messages list worked, and auth looked entirely healthy. Only the one scope was dead.
- The error is byte-identical across credential changes, because the credential is never consulted — the cache short-circuits it. I replaced the credential twice and re-ran
gws auth login twice with identical output each time, which strongly (and wrongly) suggested the problem was outside gws.
There was also a red herring worth mentioning: the same command worked on a desktop machine with the same account, binary and project, and failed on the server, which looked like an environment difference. It wasn't. Per-scope entries are created lazily on first use, so the desktop run minted a fresh entry and used it within the hour; the server had one cached from days earlier that could never refresh. Same latent bug on both machines — it only bites on the second and later runs.
Reproduction
Non-destructive A/B, using GOOGLE_WORKSPACE_CLI_CONFIG_DIR against a copy of the config dir:
mkdir -p /tmp/gwscfg && chmod 700 /tmp/gwscfg
cp -a ~/.config/gws/client_secret.json ~/.config/gws/credentials.enc \
~/.config/gws/.encryption_key /tmp/gwscfg/
# no token_cache.json present -> succeeds
GOOGLE_WORKSPACE_CLI_CONFIG_DIR=/tmp/gwscfg gws gmail +watch --project <PROJECT> \
--label-ids INBOX --once --cleanup --output-dir inbox
# same dir, with the bad token_cache.json copied in -> 401 ACCESS_TOKEN_TYPE_UNSUPPORTED
cp -a ~/.config/gws/token_cache.json /tmp/gwscfg/
GOOGLE_WORKSPACE_CLI_CONFIG_DIR=/tmp/gwscfg gws gmail +watch --project <PROJECT> \
--label-ids INBOX --once --cleanup --output-dir inboxI don't have a deterministic reproducer for how the null expires_at gets written in the first place — that's the part I can't pin down without the auth internals. The entry was created by gmail +watch's own per-scope token acquisition (gmail and pubsub scopes are fetched separately there). It may be relevant that this happened on a machine where the credential had previously gone bad with invalid_grant: Token has been expired or revoked before being re-authed.
Expected behaviour
Any one of these would have prevented a multi-day outage:
- Never persist a cache entry with an unknown expiry. If the token response has no usable expiry, either don't cache it or treat a missing
expires_at as "already expired" so the next call refreshes. A null expiry currently means "valid forever", which is the opposite of the safe default.
- Invalidate the cache entry on a 401 from the API. There is currently no negative feedback: a token that the server rejects stays cached and is resent forever. One retry-after-refresh on
UNAUTHENTICATED would make this self-healing.
- Clear (or scope-invalidate)
token_cache.json on gws auth login. After a successful login the cached tokens belong to a superseded grant — and the new login may carry a different scope set or even a different account. On my machine credentials.enc was rewritten at 08:22 while token_cache.json remained untouched from 08:05.
(3) is independently worth fixing: as it stands, "re-authenticate" is not a reliable remedy for any token-level problem, which is counter-intuitive for anyone debugging auth.
Possibly related: #205
#205 ("Apps Script API returns 401 … while Drive API works with same credentials") looks like the same bug observed from the outside: same 401 message, one API dead while another works off the same credential — which is the per-scope cache signature. Notably, a commenter there reports that deleting credentials*.enc and token_cache*.json and re-authorising resolves it, which matches what I found. That issue is on 0.4.4 and covers a couple of other problems too, so I've filed this separately with the mechanism and the decrypted evidence rather than adding to it — but they may well share a cause, and if so fixing the expiry/invalidation handling would close both.
Workaround
rm ~/.config/gws/token_cache.json — gws regenerates it from the stored credential on the next call, and nothing is lost. Worth noting in the docs, since re-auth alone does not do it.
Summary
A
token_cache.jsonentry can be written withexpires_at: None. Because gws can then never conclude the entry is stale, it never refreshes it and keeps re-sending the same access token indefinitely. Once that token's grant ages out or is superseded, every call needing that scope fails permanently with:It does not self-recover, and
gws auth logindoes not clear the cache, so re-authenticating does not fix it either. In my casegws gmail +watchwas broken for ~5 days across two successful re-auths, failing at its Pub/Sub topic-creation step, while every Gmail call kept working normally.Version / environment
gws 0.22.5(current latest release)keyring_backend: keyringcredentials.enc), 27 scopes granted includingpubsubandcloud-platformEvidence
I preserved the bad
token_cache.jsonand decrypted it (AES-256-GCM, 12-byte nonce prefix, key from.encryption_key). It decrypts to a dict keyed by scope URL, each value{access_token, refresh_token, expires_at, id_token}.Every entry had a real
expires_atexcept the one for the failing scope:The cached
access_tokenwas a well-formedya29.value, not corrupt data — just long dead.ACCESS_TOKEN_TYPE_UNSUPPORTEDappears to be what the API frontend returns for a bearer token it cannot resolve at all; I confirmed that a garbage string, a truncatedya29.…, and a1//…-shaped value all produce that samereason, whereas an unauthenticated call gives403 PERMISSION_DENIEDinstead. So the reason code is misleading here — it reads like "wrong credential type" but the real condition is "unresolvable token".Deleting
token_cache.jsonfixes it immediately and permanently.Why this is easy to misdiagnose
Two properties made this cost several days of investigation:
gws auth statuswas clean,gws gmail users messages listworked, and auth looked entirely healthy. Only the one scope was dead.gws auth logintwice with identical output each time, which strongly (and wrongly) suggested the problem was outside gws.There was also a red herring worth mentioning: the same command worked on a desktop machine with the same account, binary and project, and failed on the server, which looked like an environment difference. It wasn't. Per-scope entries are created lazily on first use, so the desktop run minted a fresh entry and used it within the hour; the server had one cached from days earlier that could never refresh. Same latent bug on both machines — it only bites on the second and later runs.
Reproduction
Non-destructive A/B, using
GOOGLE_WORKSPACE_CLI_CONFIG_DIRagainst a copy of the config dir:I don't have a deterministic reproducer for how the null
expires_atgets written in the first place — that's the part I can't pin down without the auth internals. The entry was created bygmail +watch's own per-scope token acquisition (gmailandpubsubscopes are fetched separately there). It may be relevant that this happened on a machine where the credential had previously gone bad withinvalid_grant: Token has been expired or revokedbefore being re-authed.Expected behaviour
Any one of these would have prevented a multi-day outage:
expires_atas "already expired" so the next call refreshes. A null expiry currently means "valid forever", which is the opposite of the safe default.UNAUTHENTICATEDwould make this self-healing.token_cache.jsonongws auth login. After a successful login the cached tokens belong to a superseded grant — and the new login may carry a different scope set or even a different account. On my machinecredentials.encwas rewritten at 08:22 whiletoken_cache.jsonremained untouched from 08:05.(3) is independently worth fixing: as it stands, "re-authenticate" is not a reliable remedy for any token-level problem, which is counter-intuitive for anyone debugging auth.
Possibly related: #205
#205 ("Apps Script API returns 401 … while Drive API works with same credentials") looks like the same bug observed from the outside: same 401 message, one API dead while another works off the same credential — which is the per-scope cache signature. Notably, a commenter there reports that deleting
credentials*.encandtoken_cache*.jsonand re-authorising resolves it, which matches what I found. That issue is on 0.4.4 and covers a couple of other problems too, so I've filed this separately with the mechanism and the decrypted evidence rather than adding to it — but they may well share a cause, and if so fixing the expiry/invalidation handling would close both.Workaround
rm ~/.config/gws/token_cache.json— gws regenerates it from the stored credential on the next call, and nothing is lost. Worth noting in the docs, since re-auth alone does not do it.