Skip to content

Add a default PGO profile for the compiler - #64986

Open
Jake Bailey (jakebailey) wants to merge 3 commits into
microsoft:mainfrom
jakebailey:pgo
Open

Jake Bailey (jakebailey) wants to merge 3 commits into
microsoft:mainfrom
jakebailey:pgo

Conversation

@jakebailey

@jakebailey Jake Bailey (jakebailey) commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Let's give this a try again.

Generated using the repos from the blog post

Optimize the compiler for real-world checking and emit workloads by
making a representative CPU profile available to Go's default PGO build.

The training workload is one emit-enabled compilation each of VS Code,
Sentry, Bluesky, t3code server, t3code web, and Material UI docs from
jakebailey/TypeScript-blog-benchmarks, using the default four checkers.
Profiles were collected at 4163ca1 with Go 1.27.3 on linux/amd64.

Following golang/go's src/cmd/compile/profile.sh, merge the six CPU
profiles with go tool pprof -proto, retaining their CPU-sample weights.
Copilot AI balanced review requested due to automatic review settings October 27, 2026 21:46
@github-project-automation github-project-automation Bot moved this to Not started in PR Backlog Oct 27, 2026
@typescript-automation typescript-automation Bot added Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug labels Oct 27, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review any files in this pull request. Check if the Files changed in this pull request are included in default exclusions.

@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started 👀 Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey)
The results of the perf run you requested are in!

Here they are:

Provisional hosted-runner results: these VMs are not the dedicated benchmark machines. Results may be noisier and are not comparable to the historical series. For comparisons, both revisions of each scenario ran on the same VM.

tsc

Comparison Report - baseline..pr
Metric baseline pr Delta Best Worst p-value
Compiler-Unions - native
Errors 41 41 ~ ~ ~ p=1.000 n=12
Symbols 88,466 88,466 ~ ~ ~ p=1.000 n=12
Types 79,607 79,607 ~ ~ ~ p=1.000 n=12
Memory Used 104,188k (± 0.02%) 104,126k (± 0.03%) -63k (- 0.06%) 104,041k 104,214k p=0.004 n=12
Memory Allocs 2,200,666 (± 0.00%) 2,029,582 (± 0.00%) 🟩-171,084 (- 7.77%) 2,029,351 2,029,830 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.032s (± 3.56%) 0.032s (± 4.16%) ~ 0.028s 0.034s p=0.677 n=12
Bind Time 0.012s (± 4.64%) 0.012s (± 9.27%) ~ 0.010s 0.016s p=0.878 n=12
Check Time 0.340s (± 0.70%) 0.341s (± 1.37%) ~ 0.331s 0.352s p=0.830 n=12
Emit Time 0.211s (± 1.62%) 0.208s (± 2.50%) ~ 0.194s 0.220s p=0.433 n=12
Total Time 0.600s (± 0.70%) 0.598s (± 1.63%) ~ 0.581s 0.620s p=0.640 n=12
angular-1 - native
Errors 3 3 ~ ~ ~ p=1.000 n=12
Symbols 586,513 (± 0.02%) 586,570 (± 0.01%) ~ 586,389 586,750 p=0.298 n=12
Types 241,693 (± 0.00%) 241,691 (± 0.00%) ~ 241,690 241,693 p=0.055 n=12
Memory Used 517,800k (± 0.02%) 517,908k (± 0.02%) ~ 517,589k 518,278k p=0.219 n=12
Memory Allocs 13,117,484 (± 0.07%) 12,267,305 (± 0.07%) 🟩-850,179 (- 6.48%) 12,258,369 12,307,842 p=0.000 n=12
Config Time 0.017s (± 1.07%) 0.017s ~ ~ ~ p=1.000 n=12
Parse Time 0.188s (± 2.62%) 0.183s (± 3.24%) ~ 0.172s 0.209s p=0.053 n=12
Bind Time 0.054s (±11.73%) 0.052s (± 7.39%) ~ 0.050s 0.072s p=0.255 n=12
Check Time 0s 0s ~ ~ ~ p=1.000 n=12
Emit Time 1.130s (± 1.27%) 1.130s (± 1.56%) ~ 1.081s 1.177s p=0.989 n=12
Total Time 1.403s (± 0.68%) 1.394s (± 1.34%) ~ 1.338s 1.447s p=0.385 n=12
arktype - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 685,943 685,943 ~ ~ ~ p=1.000 n=12
Types 939,490 939,490 ~ ~ ~ p=1.000 n=12
Memory Used 497,263k (± 0.01%) 497,236k (± 0.02%) ~ 497,056k 497,411k p=0.713 n=12
Memory Allocs 12,803,992 (± 0.02%) 12,735,560 (± 0.01%) -68,432 (- 0.53%) 12,732,810 12,740,481 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.058s (± 3.26%) 0.057s (± 2.35%) ~ 0.054s 0.062s p=0.352 n=12
Bind Time 0.017s (±11.48%) 0.019s (± 9.88%) 🔻+0.002s (+12.20%) 0.015s 0.025s p=0.046 n=12
Check Time 1.371s (± 0.68%) 1.396s (± 0.52%) +0.024s (+ 1.76%) 1.378s 1.412s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 1.488s (± 0.70%) 1.516s (± 0.70%) +0.028s (+ 1.90%) 1.493s 1.541s p=0.000 n=12
mui-docs - native
Errors 11,405 11,405 ~ ~ ~ p=1.000 n=12
Symbols 3,784,027 3,784,027 ~ ~ ~ p=1.000 n=12
Types 1,355,982 1,355,982 ~ ~ ~ p=1.000 n=12
Memory Used 1,831,344k (± 0.10%) 1,830,307k (± 0.09%) ~ 1,826,188k 1,835,497k p=0.378 n=12
Memory Allocs 26,706,258 (± 0.07%) 25,560,195 (± 0.09%) 🟩-1,146,063 (- 4.29%) 25,513,915 25,605,192 p=0.000 n=12
Config Time 0.014s (± 2.00%) 0.014s (± 2.00%) ~ 0.014s 0.015s p=1.000 n=12
Parse Time 0.488s (± 2.88%) 0.496s (± 3.03%) ~ 0.449s 0.541s p=0.599 n=12
Bind Time 0.001s (±23.23%) 0.001s (±16.76%) ~ 0.001s 0.002s p=0.317 n=12
Check Time 4.507s (± 0.39%) 4.354s (± 0.47%) 🟩-0.152s (- 3.38%) 4.304s 4.440s p=0.000 n=12
Emit Time 0.419s (± 5.24%) 0.389s (± 7.90%) 🟩-0.030s (- 7.17%) 0.329s 0.467s p=0.046 n=12
Total Time 6.218s (± 0.59%) 6.094s (± 0.78%) -0.123s (- 1.98%) 5.985s 6.202s p=0.000 n=12
sentry - native
Errors 2 2 ~ ~ ~ p=1.000 n=12
Symbols 4,793,249 4,793,249 ~ ~ ~ p=1.000 n=12
Types 2,607,051 2,607,051 ~ ~ ~ p=1.000 n=12
Memory Used 2,879,163k (± 0.06%) 2,878,144k (± 0.03%) ~ 2,876,566k 2,880,230k p=0.767 n=12
Memory Allocs 75,984,467 (± 0.08%) 75,168,151 (± 0.19%) -816,317 (- 1.07%) 74,876,845 75,648,528 p=0.000 n=12
Config Time 0.044s (± 0.70%) 0.045s (± 0.81%) ~ 0.044s 0.046s p=0.073 n=12
Parse Time 0.704s (± 3.69%) 0.694s (± 3.13%) ~ 0.642s 0.745s p=0.744 n=12
Bind Time 0.185s (±14.05%) 0.214s (±15.03%) 🔻+0.029s (+15.71%) 0.172s 0.298s p=0.003 n=12
Check Time 9.119s (± 0.38%) 9.139s (± 0.41%) ~ 9.074s 9.277s p=0.478 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 10.107s (± 0.48%) 10.147s (± 0.40%) ~ 10.067s 10.292s p=0.285 n=12
social-app - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,883,075 1,883,075 ~ ~ ~ p=1.000 n=12
Types 1,158,171 1,158,171 ~ ~ ~ p=1.000 n=12
Memory Used 971,652k (± 0.02%) 971,738k (± 0.03%) ~ 971,107k 972,461k p=0.932 n=12
Memory Allocs 13,713,815 (± 0.17%) 13,573,054 (± 0.19%) -140,761 (- 1.03%) 13,523,148 13,667,521 p=0.000 n=12
Config Time 0.015s (± 1.93%) 0.015s (± 1.65%) ~ 0.014s 0.015s p=1.000 n=12
Parse Time 0.246s (± 3.56%) 0.242s (± 2.88%) ~ 0.227s 0.269s p=0.599 n=12
Bind Time 0.052s (±13.19%) 0.054s (± 9.65%) 🔻+0.002s (+ 4.65%) 0.048s 0.080s p=0.043 n=12
Check Time 2.119s (± 0.44%) 2.106s (± 0.66%) ~ 2.071s 2.158s p=0.097 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.452s (± 0.42%) 2.435s (± 0.62%) ~ 2.397s 2.478s p=0.086 n=12
strada-build-src - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 1,184,976 1,184,976 ~ ~ ~ p=1.000 n=12
Types 438,967 438,967 ~ ~ ~ p=1.000 n=12
Memory Used 1,168,665k (± 0.38%) 1,166,772k (± 0.29%) ~ 1,159,022k 1,175,972k p=0.410 n=12
Memory Allocs 77,390,458 (± 0.14%) 76,201,933 (± 0.15%) -1,188,526 (- 1.54%) 75,969,201 76,601,679 p=0.000 n=12
Config Time 0.004s (±17.94%) 0.004s (±16.90%) ~ 0.002s 0.006s p=0.733 n=12
Parse Time 0.188s (± 2.75%) 0.181s (± 1.92%) 🟩-0.007s (- 3.90%) 0.172s 0.188s p=0.043 n=12
Bind Time 0.000s 0.000s (±217.90%) ~ 0.000s 0.002s p=1.000 n=12
Check Time 1.183s (± 0.92%) 1.180s (± 0.47%) ~ 1.162s 1.191s p=1.000 n=12
Emit Time 0.217s (± 1.58%) 0.210s (± 2.03%) 🟩-0.007s (- 3.38%) 0.197s 0.223s p=0.010 n=12
Total Time 14.075s (± 0.97%) 13.828s (± 0.55%) -0.247s (- 1.76%) 13.605s 14.000s p=0.004 n=12
strada-compiler - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 263,513 263,513 ~ ~ ~ p=1.000 n=12
Types 196,118 196,118 ~ ~ ~ p=1.000 n=12
Memory Used 224,587k (± 0.03%) 224,573k (± 0.02%) ~ 224,468k 224,701k p=0.977 n=12
Memory Allocs 4,303,416 (± 0.01%) 4,239,811 (± 0.01%) -63,605 (- 1.48%) 4,238,838 4,240,521 p=0.000 n=12
Config Time 0.001s (±19.81%) 0.001s ~ ~ ~ p=1.000 n=12
Parse Time 0.071s (± 3.95%) 0.068s (± 4.08%) ~ 0.061s 0.075s p=0.153 n=12
Bind Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Check Time 0.595s (± 0.69%) 0.590s (± 0.62%) ~ 0.582s 0.598s p=0.173 n=12
Emit Time 0.092s (± 4.13%) 0.092s (± 2.20%) ~ 0.087s 0.097s p=0.765 n=12
Total Time 0.793s (± 1.03%) 0.786s (± 0.73%) ~ 0.773s 0.804s p=0.247 n=12
t3code-server - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 4,350,705 4,350,705 ~ ~ ~ p=1.000 n=12
Types 3,040,691 3,040,691 ~ ~ ~ p=1.000 n=12
Memory Used 2,432,850k (± 0.02%) 2,433,078k (± 0.03%) ~ 2,431,597k 2,434,940k p=0.630 n=12
Memory Allocs 37,892,311 (± 0.04%) 37,513,057 (± 0.11%) -379,254 (- 1.00%) 37,443,245 37,663,958 p=0.000 n=12
Config Time 0.005s 0.005s ~ ~ ~ p=1.000 n=12
Parse Time 0.361s (± 2.77%) 0.361s (± 1.87%) ~ 0.348s 0.382s p=0.660 n=12
Bind Time 0.118s (±13.06%) 0.108s (± 7.45%) ~ 0.099s 0.147s p=0.416 n=12
Check Time 10.979s (± 0.89%) 10.758s (± 0.45%) -0.222s (- 2.02%) 10.654s 10.918s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 11.477s (± 0.86%) 11.245s (± 0.43%) -0.231s (- 2.02%) 11.163s 11.404s p=0.000 n=12
ts-pre-modules - native
Errors 87 87 ~ ~ ~ p=1.000 n=12
Symbols 231,617 231,617 ~ ~ ~ p=1.000 n=12
Types 181,196 181,196 ~ ~ ~ p=1.000 n=12
Memory Used 193,541k (± 0.02%) 193,504k (± 0.03%) ~ 193,369k 193,630k p=0.198 n=12
Memory Allocs 1,372,719 (± 0.02%) 1,317,818 (± 0.02%) 🟩-54,901 (- 4.00%) 1,317,068 1,318,469 p=0.000 n=12
Config Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Parse Time 0.061s (± 3.96%) 0.059s (± 4.35%) ~ 0.053s 0.066s p=0.268 n=12
Bind Time 0.026s (± 5.17%) 0.027s (± 7.10%) ~ 0.022s 0.032s p=0.554 n=12
Check Time 0.459s (± 0.80%) 0.451s (± 0.83%) -0.007s (- 1.60%) 0.442s 0.460s p=0.009 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.556s (± 0.56%) 0.546s (± 0.79%) -0.010s (- 1.81%) 0.535s 0.554s p=0.001 n=12
vscode - native
Errors 10 10 ~ ~ ~ p=1.000 n=12
Symbols 7,304,211 7,304,211 ~ ~ ~ p=1.000 n=12
Types 4,304,434 4,304,434 ~ ~ ~ p=1.000 n=12
Memory Used 5,086,949k (± 0.02%) 5,086,496k (± 0.01%) ~ 5,085,076k 5,088,316k p=0.478 n=12
Memory Allocs 64,311,898 (± 0.01%) 56,094,174 (± 0.03%) 🟩-8,217,725 (-12.78%) 56,062,698 56,134,425 p=0.000 n=12
Config Time 0.070s (± 0.75%) 0.071s (± 1.41%) ~ 0.069s 0.075s p=0.181 n=12
Parse Time 1.352s (± 2.00%) 1.322s (± 2.19%) -0.030s (- 2.22%) 1.251s 1.404s p=0.050 n=12
Bind Time 0.417s (±10.47%) 0.406s (± 3.16%) ~ 0.378s 0.448s p=0.541 n=12
Check Time 10.724s (± 0.67%) 10.843s (± 0.68%) +0.118s (+ 1.10%) 10.661s 11.035s p=0.025 n=12
Emit Time 3.868s (± 4.95%) 3.442s (± 4.00%) 🟩-0.426s (-11.01%) 3.304s 4.114s p=0.000 n=12
Total Time 16.504s (± 1.09%) 16.156s (± 1.05%) -0.349s (- 2.11%) 15.926s 16.934s p=0.001 n=12
webpack - native
Errors 619 619 ~ ~ ~ p=1.000 n=12
Symbols 918,308 918,308 ~ ~ ~ p=1.000 n=12
Types 638,248 638,248 ~ ~ ~ p=1.000 n=12
Memory Used 691,232k (± 0.02%) 691,082k (± 0.02%) ~ 690,813k 691,488k p=0.178 n=12
Memory Allocs 6,612,636 (± 0.01%) 6,500,597 (± 0.05%) -112,039 (- 1.69%) 6,496,104 6,514,896 p=0.000 n=12
Config Time 0.007s (± 4.22%) 0.008s (± 4.38%) ~ 0.007s 0.008s p=0.680 n=12
Parse Time 0.207s (± 5.61%) 0.208s (± 4.22%) ~ 0.172s 0.227s p=0.787 n=12
Bind Time 0.063s (±11.60%) 0.061s (±12.37%) ~ 0.053s 0.090s p=0.466 n=12
Check Time 1.731s (± 0.29%) 1.681s (± 0.42%) -0.050s (- 2.87%) 1.662s 1.707s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 2.043s (± 0.52%) 1.993s (± 0.91%) -0.050s (- 2.44%) 1.942s 2.049s p=0.000 n=12
xstate-main - native
Errors 0 0 ~ ~ ~ p=1.000 n=12
Symbols 640,494 640,494 ~ ~ ~ p=1.000 n=12
Types 387,992 387,992 ~ ~ ~ p=1.000 n=12
Memory Used 350,846k (± 0.03%) 350,779k (± 0.04%) ~ 350,498k 351,187k p=0.319 n=12
Memory Allocs 4,067,693 (± 0.05%) 4,009,858 (± 0.03%) -57,835 (- 1.42%) 4,007,247 4,015,155 p=0.000 n=12
Config Time 0.002s 0.002s ~ ~ ~ p=1.000 n=12
Parse Time 0.089s (± 2.66%) 0.087s (± 2.36%) ~ 0.083s 0.095s p=0.243 n=12
Bind Time 0.024s (± 8.00%) 0.025s (± 9.10%) ~ 0.022s 0.033s p=0.155 n=12
Check Time 0.607s (± 0.73%) 0.629s (± 0.75%) +0.022s (+ 3.65%) 0.615s 0.642s p=0.000 n=12
Emit Time 0.000s 0.000s ~ ~ ~ p=1.000 n=12
Total Time 0.727s (± 0.76%) 0.749s (± 0.66%) +0.022s (+ 3.04%) 0.732s 0.761s p=0.000 n=12
System info unknown
Hosts
  • native
Scenarios
  • Compiler-Unions - native
  • angular-1 - native
  • arktype - native
  • mui-docs - native
  • sentry - native
  • social-app - native
  • strada-build-src - native
  • strada-compiler - native
  • t3code-server - native
  • ts-pre-modules - native
  • vscode - native
  • webpack - native
  • xstate-main - native
Benchmark Name Iterations
Current pr 12
Baseline baseline 12

Developer Information:

Download Benchmarks

The initial profile mostly represents large TypeScript applications.
Broaden it to include checker-heavy library code, declaration emit,
JavaScript checking, and project-reference builds.

Add one emit-enabled CPU profile each from ArkType, Webpack, and the
TypeScript strada source build, using the same non-PGO seed compiler.
Retain the original six profiles and merge all nine with pprof -proto,
without reweighting their CPU samples. XState is excluded from training
and reserved as an independent evaluation workload.

Additional input revisions:
ArkType: c34bf69b3371f63ae383ac533e63ad9311452a8b
Webpack: 573ed5e52662f6c82ad0b6025ea13ccc2e6ba3a9
TypeScript strada: b465fdb

ArkType emits JavaScript and declarations; Webpack checks and emits
JavaScript with its existing diagnostics; the strada build uses
-b src --force and emits declarations for the project-reference graph.
@jakebailey

Copy link
Copy Markdown
Member Author

TypeScript Bot (@typescript-bot) perf test this faster

@typescript-automation

typescript-automation Bot commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Starting jobs; this comment will be updated as builds start and complete.

Command Status Results
perf test this faster ✅ Started ❌ Results

@typescript-automation

Copy link
Copy Markdown
Contributor

Jake Bailey (@jakebailey), the perf run you requested failed. You can check the log here.

Coverage builds with PGO exhaust the default temporary filesystem quota
on the Linux ARM64 runner. Use the runner's temporary storage so tests
and benchmarks can retain PGO without running out of temporary space.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug

Projects

Status: Not started

Development

Successfully merging this pull request may close these issues.

2 participants