fix(typescript): avoid quadratic ts-morph re-parsing with consolidateTypeFiles - #17633
fix(typescript): avoid quadratic ts-morph re-parsing with consolidateTypeFiles#17633devin-ai-integration[bot] wants to merge 1 commit into
Conversation
…TypeFiles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
There was a problem hiding this comment.
AI Review Summary
Solid perf fix: batching statements into a single addStatements call removes the quadratic re-parse. Main concern is a behavioral difference between writeToFile and generateStatements (side effects / import registration) and the empty-statements edge case.
- 🟡 2 warning(s)
To request another review, comment /ai-review on this pull request.
| statements.push(generated); | ||
| } | ||
| } | ||
| context.sourceFile.addStatements(statements); |
There was a problem hiding this comment.
🟡 warning
addStatements with an empty array still triggers a manipulation in some ts-morph versions; more importantly, guard against no-op work when typeDeclarations is empty:
| context.sourceFile.addStatements(statements); | |
| if (statements.length > 0) { | |
| context.sourceFile.addStatements(statements); | |
| } |
| statements.push((writer) => writer.newLine()); | ||
| const generated = context.type | ||
| .getGeneratedType(typeDeclaration.name) | ||
| .generateStatements(context); |
There was a problem hiding this comment.
🟡 warning
Worth double-checking that writeToFile did nothing beyond sourceFile.addStatements(generateStatements(context)) for every GeneratedType implementation (alias, enum, union, undiscriminated union, object). If any override does extra work (e.g. adding imports, exports, or module declarations), that behavior is now silently dropped. The exhaustive fixture snapshot passing is good evidence but may not cover all variants (e.g. noOptionalProperties, retainOriginalCasing paths).
SDK Generation Benchmark ResultsComparing PR branch against median of 5 nightly run(s) on Full benchmark table (click to expand)
main (generator): generator-only time via --skip-scripts (includes Docker image build, container startup, IR parsing, and code generation — this is the same Docker-based flow customers use via |
Description
Linear ticket: N/A
Auth0's
node-auth0regeneration (3,758 IR types,consolidateTypeFiles: true) has hit the 10‑minute remote task cap on every run since 2026-09-01. Production generator stdout stops atGenerating /src/management/api/types/types.ts(2 MB / 40k lines) and emits nothing for the remaining ~9m54s; fiddle coordinator logs show run duration climbing with IR size (median 63% of the cap in June → 85% July → 92% Aug → 96% on the last success, Aug 27) at a fixed generator version, then a +15% IR jump tipped it over.Root cause:
generateConsolidatedTypeDeclarationsdid, per type,Every ts-morph manipulation re-parses the whole source file, so building one file from N types is O(N²) in file size. With thousands of types in a single file this dominates generation time.
Changes Made
generateStatements(context)for every type in the consolidated file (with the same blank-line separators) into one array and callsourceFile.addStatements(...)once.generators/typescript/sdk/changes/unreleased/.Testing
pnpm seed test --generator ts-sdk --fixture exhaustive --outputFolder consolidate-type-files --skip-scriptspasses; snapshot output unchanged.seed run --generator ts-sdk --local --skip-scriptswithconsolidateTypeFiles: true,mainvs this branch.diff -rof the full generated output (924 files) is empty in every case:maingenerators.yml(3,758 IR types) — seed "Generation Time", same machineWith the fix,
consolidateTypeFiles: trueon the Auth0 spec is faster than theconsolidateTypeFiles: falseworkaround (~47s).Link to Devin session: https://app.devin.ai/sessions/22b0c13b5d0948a593e3b0e5444a48b7
Open in Devin Desktop: https://app.devin.ai/desktop/session/22b0c13b5d0948a593e3b0e5444a48b7?variant=devin