[APPS-2792] Add: local-file resolution for Custom Credentials - #512
Conversation
a4eef2b to
dba2c98
Compare
dba2c98 to
a2cdb20
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f5d489b49
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
cd0824a to
cb89f55
Compare
There was a problem hiding this comment.
🔵 Needs a closer look
Secret handling spans filesystem access, process environment isolation, Vite resolution, and production packaging, warranting final human security review.
Review details
- Files reviewed: 10/10 changed files
- Comments generated: 0 new
- Review effort level: Balanced
4d38d9a to
d5611d6
Compare
d5611d6 to
0e3665e
Compare
0e3665e to
2d41ef8
Compare
No server-side resolution endpoint exists for Custom Credentials, so local execution reads them from a datadog-app.local.json file the developer maintains themselves in the project root, mirroring Rapid's own config/dev.json convention for local secrets. runScriptLocally resolves the file alongside the existing network/env guards and feeds the result into buildScopedEnv. The dev-server priming load (loadCustomerModuleEntry) is left unresolved on purpose: it has no projectRoot and already runs outside the guarded scope.
…ages Node's JSON.parse error can embed a raw slice of the source text when the malformed token looks like an unquoted value (e.g. `..."API_KEY": sk_live_ab"...`), which flowed through to the dev server's debug log and its HTTP response body. Also fixes a credential literally named "__proto__" being silently dropped instead of resolved, by building the result on a null-prototype object.
Vite's dev server serves any project-root file over HTTP unless it is on server.fs.deny; the default list only covers .env*/certs/.git, not datadog-app.local.json, so a browser could fetch the secrets file directly (e.g. GET /datadog-app.local.json). Also addresses two smaller review findings on the same file: reads now go through @dd/core/helpers/fs's readFile instead of node:fs/promises directly, matching this package's existing convention, and a test's Error narrowing uses an instanceof guard instead of an `as` cast. Documents in the README that Custom Credentials are only available inside a function body, not during a module's top-level evaluation.
…pped app options.include is user-configured with no gitignore-awareness, so a broad pattern (e.g. "**/*.json") would otherwise zip datadog-app.local.json's real secret values straight into the published app package. Also swaps a fragile error cast for a real type guard, and trims two comments to their single load-bearing WHY, and removes a test made fully redundant by an existing toEqual assertion one test above it.
The packaging-exclusion filter only ever inspects the original, unbundled file — importing it directly (or via import.meta.glob) lets Vite inline its real secret values into a generated chunk instead, bypassing that filter entirely. Rejected at resolveId time, for both dev and build, client and SSR. Also removes the remaining as-casts on the plugin's config() hook in tests, narrowing it through a runtime-checked helper instead, matching the pattern already used for the other hooks in this file.
…s filename A query-suffixed specifier (`?raw`, `?url`) bypassed the resolveId guard's basename comparison, letting Vite inline the real secret file's content into a built chunk.
…tom Credentials guard The direct-import basename check only stripped a query suffix, letting a hash-suffixed import (`#fragment`) through as Vite itself strips both. The packaging exclusion filter compared only a discovered asset's own basename or, for a symlinked asset, its target's basename — missing both the inverse case where the credentials file itself is a symlink to a separately-matched asset, and a hardlink under another name, since realpath resolves symlink components but not hardlink identity. It now compares each candidate's (device, inode) identity against the credentials file's, which catches a symlink on either side and a hardlink uniformly, treating any non-ENOENT stat failure as unsafe to package rather than silently letting it through.
a39fca1 to
afaae70
Compare
…path loadCustomerModuleEntry's priming load passed an empty Custom Credentials object even though projectRoot was available at both call sites, so module-top-level SDK initialization (e.g. new Stripe(process.env.X)) always saw undefined despite the same read working inside the invoked function body.
afaae70 to
c73b87d
Compare
Motivation
undefined, so a backend function that reads a secret directly (e.g. to call a third-party SDK) fails undernpm run deveven though the same code works in production.customcredentials.yaml,dequeue_task.go,js-function-with-actions.ts), not a rare edge case.datadog-app.local.json, a file the developer maintains in their project root — no network call, no new auth model — mirroring Rapid's ownconfig/dev.jsonconvention for local secrets.Architecture
loadCustomerModuleEntry(the dev-server's priming load for a customer module's top-level code) callsbuildScopedEnv({})on purpose — it has noprojectRootin its signature and already runs outsidenetwork-guard.ts'srunBlockedscope, an accepted gap predating this PR.Changes
13 changes across custom-credentials-resolver.ts, custom-credentials-resolver.test.ts, local-execution.ts, local-execution.test.ts, env-guard.ts, index.ts, index.test.ts, build-package.ts, index.test.ts (root), README.md
resolveCustomCredentials()reads and validatesdatadog-app.local.json: missing file resolves to{}, malformed JSON/non-object/non-string values throw via anisErrnoExceptiontype guard (not anas-cast); the JSON-parse-error path never interpolates V8's own error message, which could otherwise embed a raw secret value into the dev server's debug log and HTTP response; usesObject.create(null)as the target object so a credential literally named__proto__round-trips as a real own property instead of silently no-op'ingcustom-credentials-resolver.ts__proto__-named credentialcustom-credentials-resolver.test.tsrunScriptLocallyresolves real credentials viaPromise.alland passes them intobuildScopedEnv;loadCustomerModuleEntrydocumented as intentionally unresolvedlocal-execution.tsresolveCustomCredentialsmocked inbeforeEach(real fs I/O races the suite's fake-timer tests); added an integration test proving a real local file's value reachesprocess.envin the executed functionlocal-execution.test.tscustomCredentialsis always{}, now that a real value can be resolvedenv-guard.tsdatadog-app.local.jsonto the Vite plugin'sserver.fs.deny, spread alongside Vite's own default deny patterns (.env,.env.*, certs,.git) rather than replacing them — Vite replaces its whole default list when a plugin sets this field, so a naive single-entry list would silently drop existing.env/cert/.gitprotection while only adding coverage for the new filenameindex.tsconfig()hook returns theserver.fs.denyentry, and that it still contains Vite's own default patterns alongside itindex.test.tsdatadog-app.local.json, the need to gitignore it, and that values are only available inside a function body (not a module's top-level evaluation)README.mdbuildAppPackageexcludesdatadog-app.local.jsonfrom the shipped archive by comparing each candidate asset's(device, inode)identity (viafs.stat, which follows symlinks) against the credentials file's own — this catches a lowercased-basename match, a symlink in either direction (an asset symlinking to the credentials file, or the credentials file itself symlinking to a separately-matched asset), and a hardlink under another name, none of which a basename or one-directional realpath check alone can see —options.includeis user-configured and has no gitignore-awareness, so a broad pattern like**/*.jsonwould otherwise bundle real secret values into the production packagebuild-package.tsoptions.includematching the local file can't override the exclusion; removed a test fully redundant with an existingserver.fs.denyassertionindex.test.tsindex.test.ts(root)resolveIdhook rejects any import (orimport.meta.globmatch) ofdatadog-app.local.json, for dev and build, client and SSR, stripping a resource query or hash fragment (e.g.?raw,?url,#fragment) before comparing basenames, matching Vite's own postfix-stripping during resolution — the packaging-exclusion filter above only ever inspects the original, unbundled file, so a direct (possibly suffixed) import would otherwise let Vite inline the real secret values into a generated chunk insteadindex.tsplugin!.config as (...)casts with agetConfigHandlerruntime-narrowing helper, matching this file's existinggetConfigureServer/getResolveIdHandlerpatternindex.test.tsQA Instructions
CI run: https://github.com/DataDog/build-plugins/actions/runs/34528100647
yarn workspace @dd/apps-plugin typecheck # Expected: exit 0, no output ✅ VERIFIEDyarn eslint packages/plugins/apps/src/vite/{custom-credentials-resolver,custom-credentials-resolver.test,build-package,local-execution,local-execution.test,env-guard,index,index.test}.ts packages/plugins/apps/src/index.test.ts # Expected: 0 errors, 8 pre-existing func-names warnings on untouched lines ✅ VERIFIEDManual QA: packaging fix — broad options.include can't ship the local credentials file
Scaffolded a real Vite app, configured
apps.include: ['**/*.json'](deliberately broad, to exercise the exact leak scenario), built a production package, and inspected the resulting archive.Real captured output from running this script standalone, from a clean checkout (archive truncated to the relevant rows — the full listing includes 83 files, since
**/*.jsonalso matchednode_modules/**/*.json, which is exactly the kind of over-broad match this fix guards against):unzip -l dist/datadog-app-assets.zip | grep -i local→ no output — neitherdatadog-app.local.jsonnor its case-variantDatadog-App.Local.Jsonappears anywhere in the archive.✅ VERIFIED —
app-config.json(the legitimate config) still ships; both the exact-case and case-variant credentials files are excluded even though the same broad**/*.jsonpattern matched all three.Manual QA: symlink in either direction, a hardlink, and a combined query+hash-suffixed import are all still caught
The basename/one-directional-realpath check above still missed three cases: the credentials file itself being a symlink to a separately-matched asset, a hardlink under another name (no symlink involved at all, so
fs.realpathcan't see it), and a direct import suffixed with both a query and a hash fragment together (?raw#fragment). Reproduced each against the real production code with real filesystem objects (fs.symlink,fs.link) and a real archive read-back, not a synthetic assertion.yarn workspace @dd/tests test:unit packages/plugins/apps/src/index.test.ts -t "never packages"yarn workspace @dd/tests test:unit packages/plugins/apps/src/vite/index.test.ts -t "Should reject a direct import"✅ VERIFIED — the credentials file's own
(device, inode)identity check catches the inverse-symlink and hardlink cases the basename/realpath-only version missed, and the combined?raw#fragmentspecifier is rejected by the same postfix-stripping that already handled?raw/#fragmentindividually.Manual QA: a direct import of the credentials file fails the build instead of bundling the secret
Confirmed the packaging-exclusion filter above only ever inspects the copied static asset, not a module the build actually bundles — an app importing
datadog-app.local.jsondirectly (e.g.import creds from './datadog-app.local.json') goes through a different code path (Vite's module graph) that the filter never sees. Reproduced against the real published plugin: built once against the pre-fix commit to confirm the leak, rebuilt against the fix, and re-ran the identical build.Real captured output, against the pre-fix commit (
cb89f55d):grep -o "sk_test[a-zA-Z_]*" dist/vite.js→sk_test_should_never_ship— the real secret was bundled straight into the shipped chunk.Real captured output, with the fix (
32a1766b):✅ VERIFIED — the build now fails and produces no
distoutput at all, so the secret can never reach a shipped package this way. The surfaced error text comes from an unrelated internal plugin's own "no output" check racing ahead of this fix's own thrown error (both cause failure; whichever the runtime settles on first wins the final rejection message) — a purely cosmetic gap in an internal plugin outside this PR's scope, not a functional one: no case exists where the build succeeds with the secret still bundled.Manual QA: a query-suffixed import (`?raw`, `?url`) of the credentials file is rejected too
The direct-import rejection above compared
path.basename(source)against the credentials filename, but a resource-query specifier like./datadog-app.local.json?rawkeeps the query in that basename and never matches — Vite still inlines the real file content via its?raw/?urlloaders. Reproduced with a real scaffolded app against the published plugin, for a plain import and both query-suffixed forms.Real captured output, against the fix (
70fc2db1):✅ VERIFIED — all three specifier forms are rejected and produce no
distoutput.Manual QA: server.fs.deny merges with (not replaces) Vite's default deny patterns
Started a real dev server and requested
.env, a certificate,.git/HEAD, and the credentials file directly over HTTP — all four must 403. Then reproduced the regression by temporarily reverting the fix (deny: [CUSTOM_CREDENTIALS_LOCAL_FILENAME]instead of spreading in Vite's defaults) to confirm it actually causes.env/cert/.gitto become servable, before restoring the real fix and re-verifying.Real captured output, from a clean checkout:
Then, with
packages/plugins/apps/src/vite/index.ts'sdenytemporarily reverted to[CUSTOM_CREDENTIALS_LOCAL_FILENAME](no spread) and the plugin rebuilt, the identical requests against a fresh dev server instance returned:✅ VERIFIED — the regression is real (reverting the spread exposes
.env/certs/.gitwhile only the new filename stays protected), and the actual fix in this PR closes it: all four paths 403 with the real code, and a legitimate source file still serves normally.Manual QA: real file → real env var, end to end (beyond the mocked unit tests)
local-execution.test.ts's integration test already covers this path, but it goes through Jest's module system. This script bundles the real source files with esbuild and runs them under plain Node against a real temp-directory file, to rule out any test-harness-only behavior.✅ VERIFIED
Manual QA: real end-to-end dev server (Vite plugin, dev server, executeAction), not just the isolated resolver logic above
The script above proves
resolveCustomCredentials/buildScopedEnvin isolation via a throwaway esbuild bundle under plain Node — real file I/O, real env var, but bypassing the actual Vite plugin, dev server, andrunScriptLocallywiring. This script instead drives the real path end to end: a freshly scaffolded Vite app's real dev server, through the real@datadog/vite-pluginbuild, calling a real backend function that reads a Custom Credential sourced from a realdatadog-app.local.json. Copy-paste and run directly; it is idempotent (safe to re-run from a clean checkout).Real captured output from running this script standalone, from a clean checkout:
✅ VERIFIED — real file → dev server → Vite plugin → resolveCustomCredentials → buildScopedEnv → process.env, fully wired, with no leak of the real shell env var when the local file is absent. The secrets file itself now 403s instead of being servable. Confirmed the 403 comes from the new
server.fs.denyentry, not an unrelated allow-list effect, by rebuilding with that entry temporarily removed and re-issuing the same request against an identical scaffold — it returned HTTP 200 with the raw JSON body. Re-run from clean state with identical output.Blast Radius
undefined, matching today's behavior.buildScopedEnvcall sites (loadCustomerModuleEntry) is deliberately left unresolved — no behavior change there.datadog-app.local.jsonis now always excluded from a built app package by filename, regardless ofoptions.include— strictly closes a pre-existing secret-leak path, not a functional regression for any legitimate use (the file is never meant to ship).Out of Scope / Follow-ups
6 items deferred
loadCustomerModuleEntry's priming load callsbuildScopedEnv({}), so a backend function reading a Custom Credential at module top level (outside any function body — e.g.const stripe = new Stripe(process.env.KEY)) seesundefinedlocally, even though the same read inside the function body now worksnetwork-guard.ts's protected scope — trades a functional limitation for a real security regression, so no fix planned. Confined to local dev only:npm run dev:verifyand production both resolve real credentials via the cloud path, unaffected. Documented inREADME.mddatadog-app.local.jsonisn't yet added to the.gitignoretemplatecreate-appsscaffolds for new projectscreate-appssidedatadog-app.local.jsonisn't actually gitignored — protection today is documentation-onlycreate-appstemplate gap above; consider a startup warning if the file is present and unignoredVITE_DEFAULT_SERVER_FS_DENYinindex.tsis a hand-typed copy of Vite's internalserver.fs.denydefault (Vite doesn't export it) — a future Vite upgrade that changes its own default silently isn't reflected hereresolveId's direct-import guard compares the raw specifier's basename before resolution, so a symlink under a different name or aresolve.aliasentry pointing atdatadog-app.local.jsonstill reaches Vite's bundlervite.config.ts/filesystem control — that attacker has easier exfiltration paths this check can't close either way, so the added resolution-pass cost across every import in the app isn't worth it. No fix plannedDocumentation
Confluence
GitHub