Skip to content

[Plugin] Fix intermittent re-auth from cross-process refresh-token rotation#44

Open
pragati-agrawal-glean wants to merge 1 commit into
mainfrom
pragati/fix-plugin-token-rotation-reauth
Open

[Plugin] Fix intermittent re-auth from cross-process refresh-token rotation#44
pragati-agrawal-glean wants to merge 1 commit into
mainfrom
pragati/fix-plugin-token-rotation-reauth

Conversation

@pragati-agrawal-glean

Copy link
Copy Markdown
Contributor

Problem

The plugin intermittently asks the user to authenticate again — find_skills (and run_tool) return [SETUP_REQUIRED] — even though a valid grant exists. This is not a token-TTL change (the 7-day access-token / 180-day refresh TTLs in scio config/default.ini [client_api] have been unchanged since Feb 2026, per Mohit's check).

Root cause: per-session in-memory snapshot + refresh-token rotation

MCP servers are spawned per session, so several plugin processes can be alive at once. The OAuth provider reads credentials from disk once at construction and then serves everything from memory:

constructor() { const stored = loadCredentials(); this._tokens = stored?.tokens } // read ONCE
tokens()      { return this._tokens }                                            // never re-reads
saveTokens(t) { this._tokens = t; saveCredentials(t) }                           // last-writer-wins

Refresh is reactive: only on a 401 does the SDK call provider.tokens() (auth.js:272) and refresh with that token. Combined with Ory single-use refresh-token rotation (ory_rt_… are consumed on use; a refresh returns a new one and invalidates the old):

When process A refreshes, it persists a new refresh token to disk and invalidates the old one. Any other live process still holds the old, now-dead refresh token in memory. Its next 401 → refresh → invalid_grant → full re-auth → [SETUP_REQUIRED].

This also explains the observed anomaly of an older token being "last used" after a newer token was minted: a long-running session kept refreshing the token it snapshotted at startup, oblivious to the newer grant on disk.

Fixes (both independent of the PLUGIN_DATA_DIR store split)

1. tokens() re-reads the store when it changed on disk (auth-provider.ts, token-store.ts)
mtime-guarded, so the steady state is a single stat(), not a re-parse. This sits exactly where the SDK reads tokens at the start of its auth flow, so a process picks up a sibling's freshly-rotated grant before attempting a refresh. Conservative on removal: a missing file or a tokens-less rewrite does not evict the in-memory token (a transient stat failure or another process's logout shouldn't drop a token that still works for us).

2. createRemoteClient retries connect once on a sibling refresh (remote-client.ts)
If auth fails and a newer access token has since appeared on disk, rebuild the transport and retry once — turning the rotation race into a silent reconnect. Bounded to a single retry so it can't spin.

Tests

  • tokens() adopts a sibling's newer token; keeps its token when the file vanishes; ignores a tokens-less rewrite.
  • credentialsMtimeMs probe (undefined when absent, advances on rewrite).
  • connect retry fires only when the on-disk token changed; otherwise surfaces AuthRequiredError.

Full suite: 217 passing, typecheck clean.

Deliberately out of scope (follow-ups)

  • Collapse the store split. Credentials resolve to process.env.PLUGIN_DATA_DIR || ~/.glean; the host sets PLUGIN_DATA_DIR on some launches and not others, producing two stores with two different DCR clients that never converge (confirmed on a single machine). Fixes above heal processes sharing one store; crossing surfaces (e.g. terminal ↔ VS Code) still forces one re-auth. Pin to a single canonical path (or read-merge both known locations) + one-time migration.
  • Reuse the DCR client across re-auths instead of registering a fresh one each time, to stop the orphaned-token pile-up.

🤖 Generated with Claude Code

…tation

MCP servers are spawned per session, and the OAuth provider reads
credentials from disk once at startup, then serves tokens from that
in-memory snapshot. With Ory single-use refresh-token rotation, when one
process refreshes it persists a new refresh token and invalidates the old
one that every other live process still holds in memory. The next process
to hit a 401 refreshes with its now-dead token -> invalid_grant -> full
re-auth -> [SETUP_REQUIRED]. This is the intermittent "why is it asking me
to auth again", not the (intentional, unchanged) 7-day access-token TTL.

Two fixes, both independent of the PLUGIN_DATA_DIR store split:

1. tokens() re-reads the credentials file when its mtime advances, so a
   process picks up a sibling's freshly-rotated grant before the SDK's
   auth flow reads tokens and attempts a refresh. mtime-guarded so the
   steady state is a single stat(). Conservative on removal: a missing
   file or a tokens-less rewrite does not evict the in-memory token.

2. createRemoteClient retries connect once when, after an auth failure, a
   newer access token has appeared on disk (sibling refresh) -- turning
   the rotation race into a silent reconnect instead of a re-auth. Bounded
   to a single retry.

Tests: provider adopts a sibling's newer token, keeps its token when the
file vanishes, ignores a tokens-less rewrite; credentialsMtimeMs probe;
connect retry fires only when the on-disk token changed.

Follow-ups (not in this PR): collapse the PLUGIN_DATA_DIR / ~/.glean store
split to one canonical path so surfaces share one DCR client; reuse the
DCR client across re-auths to stop orphaned-token pile-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant