Skip to content

Add errors to TomlFile and Project for TOML parse failure tracking#7130

Merged
ryancbahan merged 6 commits intomainfrom
ryan/surface-discovery-errors
Mar 31, 2026
Merged

Add errors to TomlFile and Project for TOML parse failure tracking#7130
ryancbahan merged 6 commits intomainfrom
ryan/surface-discovery-errors

Conversation

@ryancbahan
Copy link
Copy Markdown
Contributor

@ryancbahan ryancbahan commented Mar 30, 2026

What

Unify TOML error types, add errors to TomlFile and Project, and surface TOML parse failures in app config validate --json.

How

Unified error type: TomlFileNotFoundError and TomlParseError replaced with single TomlFileError class. Parse errors produce clean one-line messages ("Invalid boolean, expected true or false at row 6, col 13") instead of multi-line ASCII art.

TomlFile.errors: New field on TomlFile. read() still throws. Callers that catch create a TomlFile with the path and error populated — the file exists in the model even when malformed.

Project.errors: Aggregated from all discovery passes. Malformed files are included in appConfigFiles/extensionConfigFiles/webConfigFiles (with errors and empty content) instead of silently dropped.

Validate command: Checks project.errors before attempting to link. Reports TOML parse errors as {valid: false, issues: [{file, message}]} in JSON mode.

Depends on #7128, #7126.

Tophatting

valid config
Screen Recording 2026-03-30 at 4.20.35 PM.mov (uploaded via Graphite)

invalid config -- shopify domain
Screen Recording 2026-03-30 at 4.21.52 PM.mov (uploaded via Graphite)

invalid config -- invalid toml syntax
Screen Recording 2026-03-30 at 4.22.47 PM.mov (uploaded via Graphite)

@github-actions
Copy link
Copy Markdown
Contributor

We detected some changes at packages/*/src and there are no updates in the .changeset.
If the changes are user-facing, run pnpm changeset add to track your changes and include them in the next release CHANGELOG.

Caution

DO NOT create changesets for features which you do not wish to be included in the public changelog of the next CLI release.

Copy link
Copy Markdown
Contributor Author

ryancbahan commented Mar 30, 2026

@ryancbahan ryancbahan changed the title Add errors field to TomlFile and Project for TOML parse failure tracking Add errors to TomlFile and Project for TOML parse failure tracking Mar 30, 2026
@ryancbahan ryancbahan force-pushed the ryan/surface-discovery-errors branch from 4f4200b to 50263be Compare March 30, 2026 21:48
@ryancbahan ryancbahan force-pushed the ryan/validate-json-contract branch from 14a803b to aacac17 Compare March 30, 2026 21:48
@ryancbahan ryancbahan requested a review from dmerand March 30, 2026 22:24
Copy link
Copy Markdown
Contributor

dmerand commented Mar 31, 2026

I ran some inactive-sibling tophat for this PR, and it looks like the current behavior is:

  • a malformed inactive shopify.app.*.toml fails app config validate --json
  • a malformed extension TOML referenced only by an inactive sibling config also fails it
  • a malformed web TOML referenced only by an inactive sibling config also fails it

In other words, this now behaves like project-wide discovery validation, not just validation of the selected/active config.

That might be the right contract, but it’s worth calling out because this was a known concern in the earlier Project rollout: eager parsing of all discovered sibling configs can let an unrelated inactive file poison the active validation path.

So I think we should make one of these explicit before merge:

  1. Intentional broadening: add/keep regression coverage proving inactive sibling parse failures are supposed to surface in app config validate --json, and describe the command as project-wide discovery validation.

  2. Preserve active-config semantics: change the flow so malformed inactive sibling app/extension/web TOMLs don’t fail validation for the selected config.

I'm leaning toward (1). It's not horrible UX for folks to rename TOMLs that are intentionally invalid to e.g. have some other file extension so they can be ignored. AFAICT the overall behavior is coherent, but the scope decision is important enough that it shouldn’t stay implicit.

Copy link
Copy Markdown
Contributor

dmerand commented Mar 31, 2026

I tophatted this; there's an inconsistency between linked and local command paths for a malformed active app TOML.

Using the same temp app fixture with shopify.app.toml replaced by invalid TOML (123):

linked path

shadowenv exec -- node packages/cli/bin/dev.js app info --path /tmp/dlm-local-vs-linked-jdSGP9/app

This fails with the actual TOML parse error:

Invalid character, expected "=" at row 1, col 5, pos 4: at row 0, col 4

local path

shadowenv exec -- node packages/cli/bin/dev.js app build --path /tmp/dlm-local-vs-linked-jdSGP9/app --skip-dependencies-installation

This fails later with schema errors against empty content:

[client_id]: Required
[auth]: Required
[webhooks]: Required
[application_url]: Valid URL is required
[embedded]: Boolean is required

From the code, that seems to be because Project.load() now preserves malformed app TOMLs as TomlFiles with file.errors and empty content, and linkedAppContext() explicitly guards that case:

if (activeConfig.file.errors.length > 0) {
  throw new AbortError(activeConfig.file.errors.map((err) => err.message).join('\n'))
}

but localAppContext() doesn’t do the same check and goes straight into loadAppFromContext().

So the same malformed selected config behaves differently depending on which context loader the command uses. Would it make sense to reject activeConfig.file.errors in localAppContext() too, or otherwise centralize that invariant so malformed active configs fail the same way in both entrypoints?

ryancbahan and others added 5 commits March 31, 2026 08:11
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lidate --json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lidate --json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lidate --json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…lidate --json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ryancbahan ryancbahan force-pushed the ryan/validate-json-contract branch from aacac17 to 6c8d2ed Compare March 31, 2026 14:14
@ryancbahan ryancbahan force-pushed the ryan/surface-discovery-errors branch from eebfb20 to 9262a31 Compare March 31, 2026 14:14
Comment on lines +69 to +72
const message = err instanceof AbortError ? unstyled(stringifyMessage(err.message)).trim() : ''
const isValidationError = message.startsWith('Validation errors in ')
if (isValidationError && flags.json) {
outputResult(JSON.stringify({valid: false, issues: [{message}]}, null, 2))
Copy link
Copy Markdown
Contributor

@isaacroldan isaacroldan Mar 31, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be throwing a ValidationError (new type extending from AbortError) instead? that'd be easier to detect, rather than comparing unstyled strings :/

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I want to address this in a followup though, since the blast radius is contained to just this command and i think it warrants more thinking.

…lidate --json

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@ryancbahan ryancbahan requested a review from isaacroldan March 31, 2026 19:59
Copy link
Copy Markdown
Contributor Author

@dmerand addressed

@github-actions
Copy link
Copy Markdown
Contributor

Differences in type declarations

We detected differences in the type declarations generated by Typescript for this branch compared to the baseline ('main' branch). Please, review them to ensure they are backward-compatible. Here are some important things to keep in mind:

  • Some seemingly private modules might be re-exported through public modules.
  • If the branch is behind main you might see odd diffs, rebase main into this branch.

New type declarations

We found no new type declarations in this PR

Existing type declarations

packages/cli-kit/dist/public/node/toml/toml-file.d.ts
 import { JsonMapType } from './codec.js';
 /**
- * Thrown when a TOML file does not exist at the expected path.
+ * An error on a TOML file — missing or malformed.
+ * Extends Error so it can be thrown. Carries path and a clean message suitable for JSON output.
  */
-export declare class TomlFileNotFoundError extends Error {
+export declare class TomlFileError extends Error {
     readonly path: string;
-    constructor(path: string);
+    constructor(path: string, message: string);
 }
 /**
- * Thrown when a TOML file cannot be parsed. Includes the file path for context.
- */
-export declare class TomlParseError extends Error {
-    readonly path: string;
-    constructor(path: string, cause: Error);
-}
-/**
  * General-purpose TOML file abstraction.
  *
  * Provides a unified interface for reading, patching, removing keys from, and replacing
  * the content of TOML files on disk.
  *
  * - `read` populates content from disk
  * - `patch` does surgical WASM-based edits (preserves comments and formatting)
  * - `remove` deletes a key by dotted path (preserves comments and formatting)
  * - `replace` does a full re-serialization (comments and formatting are NOT preserved).
  * - `transformRaw` applies a function to the raw TOML string on disk.
  */
 export declare class TomlFile {
     /**
-     * Read and parse a TOML file from disk. Throws if the file doesn't exist or contains invalid TOML.
-     * Parse errors are wrapped in {@link TomlParseError} with the file path for context.
+     * Read and parse a TOML file from disk. Throws {@link TomlFileError} if the file
+     * doesn't exist or contains invalid TOML.
      *
      * @param path - Absolute path to the TOML file.
      * @returns A TomlFile instance with parsed content.
      */
     static read(path: string): Promise<TomlFile>;
     readonly path: string;
     content: JsonMapType;
+    readonly errors: TomlFileError[];
     constructor(path: string, content: JsonMapType);
     /**
      * Surgically patch values in the TOML file, preserving comments and formatting.
      *
      * Accepts a nested object whose leaf values are set in the TOML. Intermediate tables are
      * created automatically. Setting a leaf to `undefined` removes it (use `remove()` for a
      * clearer API when deleting keys).
      *
      * @example
      * ```ts
      * await file.patch({build: {dev_store_url: 'my-store.myshopify.com'}})
      * await file.patch({application_url: 'https://example.com', auth: {redirect_urls: ['...']}})
      * ```
      */
     patch(changes: {
         [key: string]: unknown;
     }): Promise<void>;
     /**
      * Remove a key from the TOML file by dotted path, preserving comments and formatting.
      *
      * @param keyPath - Dotted key path to remove (e.g. 'build.include_config_on_deploy').
      * @example
      * ```ts
      * await file.remove('build.include_config_on_deploy')
      * ```
      */
     remove(keyPath: string): Promise<void>;
     /**
      * Replace the entire file content. The file is fully re-serialized — comments and formatting
      * are NOT preserved.
      *
      * @param content - The new content to write.
      * @example
      * ```ts
      * await file.replace({client_id: 'abc', name: 'My App'})
      * ```
      */
     replace(content: JsonMapType): Promise<void>;
     /**
      * Transform the raw TOML string on disk. Reads the file, applies the transform function
      * to the raw text, writes back, and re-parses to keep `content` in sync.
      *
      * Use this for text-level operations that can't be expressed as structured edits —
      * e.g. Injecting comments or positional insertion of keys in arrays-of-tables.
      * Subsequent `patch()` calls will preserve any comments added this way.
      *
      * @param transform - A function that receives the raw TOML string and returns the modified string.
      * @example
      * ```ts
      * await file.transformRaw((raw) => `# Header comment\n${raw}`)
      * ```
      */
     transformRaw(transform: (raw: string) => string): Promise<void>;
     private decode;
 }

Base automatically changed from ryan/validate-json-contract to main March 31, 2026 21:34
@ryancbahan ryancbahan added this pull request to the merge queue Mar 31, 2026
Merged via the queue into main with commit d3dea9e Mar 31, 2026
24 checks passed
@ryancbahan ryancbahan deleted the ryan/surface-discovery-errors branch March 31, 2026 21:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants