Skip to content

feat(cli-generator): add named profiles for multi-tenant CLIs - #17654

Open
rishabh-fern wants to merge 1 commit into
mainfrom
rishabh/cli-profile-management
Open

feat(cli-generator): add named profiles for multi-tenant CLIs#17654
rishabh-fern wants to merge 1 commit into
mainfrom
rishabh/cli-profile-management

Conversation

@rishabh-fern

@rishabh-fern rishabh-fern commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Description

Linear ticket: Refs

Named profiles for generated CLIs. A profile is a named bundle of request context — a credential slot, parameter defaults, server variables, an optional base URL and output format — resolved once per invocation, so a tenant identifier stops being typed on every command.

Framework feature in generators/cli/sdk, not a per-customer custom command: it is equally correct for a CLI with no subaccounts, no regions, and a single bearer token.

twilio profiles create prod --set AccountSid=AC11… --with-token
twilio profiles create acme --parent prod --set AccountSid=AC99…   # subaccount, same credential
twilio profiles use acme

twilio core messages list            # no --account-sid, no exported env vars
twilio core messages list -p prod    # one command against another tenant
twilio profiles list                 # "which account am I about to hit?"

Not new transport. A profile is a source of defaults for mechanisms that already accept defaults — the keyring account an AuthCredentialSource::Keyring reads, a clap::Arg's default_value, servers[].variables substitution, resolve_base_url_override, OutputPipeline. Nothing about auth, HTTP, retries, or command construction changes. That is what keeps it tractable and generic.

Ships off unless config.profiles.enabled is set. A CLI generated without the block is byte-identical to one generated before this change.

Precedence, and the reasoning behind it, is in a new ADR: 0011-profile-resolution-precedence.md. ADR-0008 now points to it.

Changes Made

  • New generators/cli/sdk/src/profiles/store.rs (format-preserving TOML via toml_edit, single-level parent inheritance, cycle/depth rejection), selection.rs (pre-clap -p scan → <BIN>_PROFILEactive → none, plus the process-global slot), commands.rs (create | list | use | remove | current), mod.rs (the read-side helpers every consumer goes through).
  • Precedence is flag → env → profile → spec default at all eight resolution points. Env sits above profile so a CI pipeline that exports credentials is never silently overridden by a developer's stored profile; profile sits above spec defaults so it can change a parameter the spec defaults.
  • Credentials get a selector, not a rung. The profile only chooses which keyring account ADR-0008's priority-3 rung reads — <scheme> unprofiled, <scheme>#<credential> with a profile — so two tenants hold separate credentials for one scheme and existing keychain entries keep resolving after upgrade.
  • Fixes a latent OAuth token-cache bug. TokenCache was keyed by token_url alone, so two profiles authenticating against the same client-credentials endpoint would have clobbered each other's access and refresh tokens. Now <token_url>#<credential> when profiled, byte-identical when not; profiles remove purges that profile's entries and only those.
  • No secrets in profiles.toml. The file names a keychain account; --with-token / --from-env write the credential to the OS keychain under a profile-scoped account. oauth_client_id is in the file because a client id is public by construction (RFC 6749 §2.2) and profiles list should show it without unlocking the keychain.
  • --set is validated at write time, both halves. parameters has to be a free-form map — the framework cannot know which parameter is the tenant key — but an unvalidated key makes a typo a silent no-op, and an out-of-enum value makes every command carrying that parameter fail on a flag the caller never passed. Accepted values are derived from the same PossibleValues clap builds its parser from, so the check cannot drift from the command itself.
  • Built-ins now appear in --schema under a new builtinCommands array (auth, profiles, completion, man). They were absent entirely, so an agent reading --schema could not discover the CLI had any way to authenticate or switch tenant.
  • Generator wiringconfig.profiles.{enabled,commandName} with boundary validation, renderMainRs emits .profiles(...) only when enabled, changelog entry, docs/customize.md section, new cli-profiles seed fixture.
  • Updated README.md generator (if applicable) — n/a, README emission unchanged

Bugs found while building that the design didn't anticipate

  1. profiles create --region panicked every invocation. Spec server-variable flags are global(true); clap propagates a global into a subcommand only when the subcommand has no arg with the same id, so a create-local --region under id server-var:region was two args sharing one long name → clap rejects the whole tree at startup. Only reproduced on the seed fixture, whose spec declares a server variable. create now reads the propagated global, counting only CommandLine so the spec's default isn't frozen into every profile.
  2. Profile defaults were silently discarded on any parameter with x-fern-default. collect_params_from_flags substitutes the typed spec default whenever clap reports DefaultValue — which a profile value also is.
  3. profiles list never marked the active profile, because the group runs unprofiled by design (so a stale active can be repaired).
  4. A child with --parent and no explicit credential got its own empty slot instead of the parent's — breaking the subaccount case the feature exists for.

Also renamed create's --format to --default-format: a same-id arg would have shadowed the global output flag, making profiles create p --format json quietly store a format while looking like a request for JSON output.

Testing

  • Unit tests added/updated

  • Manual testing completed

  • 2108 SDK tests pass (2051 unit + 57 integration). New: 58 unit tests, and three integration files — tests/profiles.rs (34, through the compiled fixture binary), tests/profiles_server_vars.rs (8, the regression above), tests/profiles_command_name.rs (5, renamed group + a spec that owns the profiles noun).

  • 452 generator tests pass, including new copySpecs / customConfig coverage for the config block and its injection guards.

  • All 157 CLI seed cases regenerated and passing. cli-profiles snapshots the emitted main.rs; every other cli-* fixture is the negative control — profiles must be entirely absent from their output.

  • The generated binary was driven by hand to confirm the whole flow in real output: inherited tenant path parameter, inherited credential, child-overridden region, -p switching tenant for one invocation without changing the active profile. cargo build --locked --all-features --tests passes in the generated fixture.

  • Regression guards specifically for the compatibility claim: a profile-less CLI's --help, auth status, credential resolution, and x-fern-default behaviour are unchanged, and -p nope errors with the known-profile list rather than silently falling back to env credentials.

Deliberately not in this PR

  • --revoke / revokeOperation. The design names an operation but nothing specifies how its parameters are supplied from a profile, and there is no Binding seam to invoke an operation without synthetic ArgMatches. Left out rather than shipping a flag that only prints a hint.
  • The generator migration. Per the breaking-changes policy the migration lands when the default flips, not now — pinning enabled: false for a default that is already false would be misleading. Noted in the ADR.

Note for the reviewer

sdk/Cargo.lock gains toml_edit + 3 transitives, so per generators/cli/CLAUDE.md the seed image needs rebuilding before the seed scripts will pass:

docker build --no-cache -f docker/seed/Dockerfile.cli -t fernapi/cli-seed:latest .

I verified cargo build --locked --all-features --tests in the generated fixture natively but did not run the docker build.

Generated with Claude Code


Devin Review

A profile is a named bundle of request context — a credential slot,
parameter defaults, server variables, an optional base URL and output
format — resolved once per invocation. Enabling it adds a
`profiles create | list | use | remove | current` group and a global
`--profile` / `-p` flag, so a tenant identifier stops being typed on
every command.

Not new transport. A profile is a *source of defaults* for mechanisms
that already accept defaults: the keyring account an
`AuthCredentialSource::Keyring` reads, a `clap::Arg`'s `default_value`,
`servers[].variables` substitution, `resolve_base_url_override`,
`OutputPipeline`. That is what keeps it generic — nothing about auth,
HTTP, retries, or command construction changes.

Precedence per value is `flag -> env var -> profile -> spec default`.
Env sits above profile so a CI pipeline that exports credentials is
never silently overridden by a developer's stored profile; profile sits
above spec defaults so it can change a parameter the spec defaults. For
credentials the profile adds no rung to ADR-0008's chain — it only
selects which keyring account the existing `Keyring` rung reads
(`<scheme>` unprofiled, `<scheme>#<credential>` with a profile), so two
tenants can hold separate credentials for one scheme.

Also fixes a latent correctness bug in the OAuth token cache, which was
keyed by `token_url` alone: two profiles authenticating against the same
client-credentials endpoint would have clobbered each other's access and
refresh tokens. The key is now `<token_url>#<credential>` when a profile
is selected and byte-identical to before when none is, so existing
caches keep resolving and no one is logged out by the upgrade.

Storage is `~/.config/<bin>/profiles.toml`, beside the credential store.
No secrets ever reach it: the file names a keychain account, and
`profiles create --with-token` / `--from-env` write the credential to the
OS keychain under an account scoped to that profile. Written through
`toml_edit`, so a user's comments and any field a newer binary wrote
survive a round-trip.

`parameters` is a free-form map — the framework cannot know which
parameter is the tenant key — but `--set` validates both halves at write
time against the parsed operation table. An unvalidated key would make a
typo a silent no-op, and an out-of-enum value would make *every* command
carrying that parameter fail on a flag the caller never passed.

Built-in commands (`auth`, `profiles`, `completion`, `man`) now appear in
`--schema` under a new `builtinCommands` array. They were absent
entirely, so an agent reading `--schema` could not discover that the CLI
had any way to authenticate or switch tenant.

Ships off unless `config.profiles.enabled` is set. A CLI generated
without the block is byte-identical to one generated before this change;
`tests/profiles.rs::with_no_profile_configured_nothing_changes` pins
that. Flipping the default is a separate decision needing a generator
major bump plus a migration, per the breaking-changes policy.

See generators/cli/sdk/docs/adr/0011-profile-resolution-precedence.md.

Co-Authored-By: Claude <noreply@anthropic.com>
@rishabh-fern rishabh-fern self-assigned this Sep 4, 2026

@nitpickybot nitpickybot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Review Summary

Large feature PR adding named profiles to the generated CLI SDK, plus regenerated seed fixtures. The diff is heavily truncated (only the generator/SDK core is visible), so review focused on app.rs, the changelog/docs, and the Cargo/lock changes. Main concerns: the --profile flag is registered on clap but never read from ArgMatches (pre-clap scan only), and resolve_and_install_profile silently drops non-UTF-8 argv, which can misresolve the profile.

  • 🟡 1 warning(s)
  • 🔵 4 suggestion(s)

To request another review, comment /ai-review on this pull request.

Comment on lines +689 to +692
let str_args: Vec<String> = args
.iter()
.filter_map(|a| a.to_str().map(String::from))
.collect();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 warning

filter_map(|a| a.to_str()) silently drops non-UTF-8 args rather than replacing them, which shifts positions in str_args. A non-UTF-8 value immediately before -p (or a non-UTF-8 argv[1]) changes what the pre-clap scanner sees — e.g. first_positional_is could match a token that is actually the value of a preceding option. Prefer to_string_lossy().into_owned() so the vector stays positionally faithful to argv.

Comment on lines +1122 to +1136
if let Some(ref config) = self.profiles {
cli = cli.arg(
clap::Arg::new(crate::profiles::selection::PROFILE_FLAG)
.long(crate::profiles::selection::PROFILE_FLAG)
.short(crate::profiles::selection::PROFILE_SHORT)
.value_name("NAME")
.help(format!(
"Run this command under a named profile (see `{} {}`). \
Overrides {} and the active profile.",
self.name,
config.command_name,
crate::profiles::selection::profile_env_var(&self.name),
))
.global(true),
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 suggestion

--profile is registered here purely for help/acceptance, but nothing ever reads it out of ArgMatches; resolution happens in the pre-clap scan. Worth asserting the two agree — the scanner and clap can disagree on forms clap accepts (--profile=x, -pfoo, -xp foo bundling, -- terminator). If the scanner misses a form clap accepts, the flag is silently ignored while the command still parses, which is the worst failure mode for a tenant selector. A test matrix over those spellings (or re-reading matches.get_one(PROFILE_FLAG) and erroring on mismatch) would close it.

Comment on lines +1155 to +1159
let spec_owns_profiles = self
.profiles
.as_ref()
.is_some_and(|config| merged_subtree.find_subcommand(&config.command_name).is_some());
let profiles_vocabulary = self.profiles_vocabulary();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 suggestion

self.profiles_vocabulary() is computed here and again inside builtin_commands(...) at the --schema branch (line 938), each walking every binding's parameter specs. Minor, but for large specs this doubles a non-trivial traversal on --schema; consider computing once earlier in dispatch_pipeline and passing it down.

Comment on lines +1840 to +1846
"description": format!(
"Run this command under a named profile (see `{}`), overriding \
the <NAME>_PROFILE env var and the active profile",
config.command_name,
),
}));
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 suggestion

The description hardcodes the literal <NAME>_PROFILE while the clap help (line 1133) uses the actual profile_env_var(&self.name). Agents reading --schema get a placeholder they can't act on. Pass the CLI name in and use profile_env_var here too.

Comment on lines +1910 to +1913
None => entry.insert(
"argument".into(),
arg.get_id().as_str().to_string().into(),
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 suggestion

For positionals you emit the arg id, but the doc comment says "name it by its value placeholder". get_value_names() is what the user actually sees in help; the id may differ (e.g. snake_case internal ids). Consider arg.get_value_names().and_then(|v| v.first()) with the id as fallback.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Docs Generation Benchmark Results

Comparing PR branch against median of 5 nightly run(s) on main (latest: 2026-09-04T04:06:24Z).

Fixture main PR Delta
docs 272.5s (n=5) 270.1s (35 versions) -2.4s (-0.9%)

Docs generation runs fern generate --docs --preview end-to-end against the benchmark fixture with 35 API versions (each version: markdown processing + OpenAPI-to-IR + FDR upload).
Delta is computed against the nightly baseline on main.
Baseline from nightly run(s) on main (latest: 2026-09-04T04:06:24Z). Trigger benchmark-baseline to refresh.
Last updated: 2026-09-04 16:17 UTC

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

SDK Generation Benchmark Results

Comparing PR branch against median of 5 nightly run(s) on main (latest: 2026-09-04T04:06:24Z).

Full benchmark table (click to expand)
Generator Spec main (generator) main (E2E) PR (generator) Delta
csharp-sdk square 99s (n=5) 120s (n=5) 71s -28s (-28.3%)
go-sdk square 149s (n=5) 311s (n=5) 135s -14s (-9.4%)
java-sdk square 238s (n=5) 283s (n=5) 204s -34s (-14.3%)
php-sdk square 77s (n=5) N/A 59s -18s (-23.4%)
python-sdk square 152s (n=5) 254s (n=5) 145s -7s (-4.6%)
ruby-sdk-v2 square 96s (n=5) 154s (n=5) 90s -6s (-6.2%)
rust-sdk square 208s (n=5) 228s (n=5) 188s -20s (-9.6%)
swift-sdk square 85s (n=5) 464s (n=5) 59s -26s (-30.6%)
ts-sdk square 178s (n=5) 184s (n=5) 138s -40s (-22.5%)

main (generator): generator-only time via --skip-scripts (includes Docker image build, container startup, IR parsing, and code generation — this is the same Docker-based flow customers use via fern generate). main (E2E): full customer-observable time including build/test scripts (nightly baseline, informational). Delta is computed against generator-only baseline.
⚠️ = generation exited with a non-zero exit code (timing may not reflect a successful run).
Baseline from nightly runs on main (latest: 2026-09-04T04:06:24Z). Trigger benchmark-baseline to refresh.
Last updated: 2026-09-04 16:18 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant