perf: custom loader (up to 70x+ improvements)

[cmilesio]

This PR overhauls the loader process for Wire which has significant performance issues under various circumstances. Most of which were tied to go/packages semantics and go/types type checking work. With larger codebases, you incur a growing cost of both as a baseline. For example, in a project that stock Google Wire would take 1.3s to run, 400-600ms of that was go list deps and the rest was type checking all external and internal packages recursively in order to do what Wire needs to do. This PR replaces the loader process, caches the heavy work and invalidates it when necessary.

Safety

Safety, accuracy is important above all else. Existing Wire tests pass along with a plethora of newly added scenario based testing.

Test it Yourself

You can test the new loader yourself by installing this branch:

go install github.com/goforj/wire/cmd/wire@8a3ed6d

Revert back to goforj

go install github.com/goforj/wire/cmd/wire@latest

Or to stock google wire

go install github.com/google/wire/cmd/wire@latest

Custom Loader

This custom loader implementation is far more consistent than previous cache attempts (on goforj/wire) where those cache attempts were tackling the issue further down the line when they needed to be addressed more upstream in the loader. While this was helpful, you'd lose your compile speed under certain edit types. This implementation provides consistent lightning fast compile times across the board in many scenarios. See table below for measurements.

The table below illustrates up to 70x+ speed improvements, but these improvements scale with codebase size and complexity so you can maintain a great development experience even under massive repositories.

Benchmarks

+---------------+-------+--------+----------+----------------------+----------+----------+---------+
|       profile | local | stdlib | external | change type          | stock    | current  | speedup |
+---------------+-------+--------+----------+----------------------+----------+----------+---------+
|         local | 41    | 191    | 1        | cold run             | 350.6ms  | 2976.0ms | 0.12x   |
|         local | 41    | 191    | 1        | unchanged rerun      | 339.6ms  | 9.4ms    | 36.18x  |
|         local | 41    | 191    | 1        | body-only local edit | 328.5ms  | 27.2ms   | 12.09x  |
|         local | 41    | 191    | 1        | shape change         | 325.9ms  | 146.7ms  | 2.22x   |
|         local | 41    | 191    | 1        | import change        | 327.6ms  | 147.6ms  | 2.22x   |
|         local | 41    | 191    | 1        | known import toggle  | 327.9ms  | 145.2ms  | 2.26x   |
|    local-high | 1016  | 191    | 1        | cold run             | 759.4ms  | 5421.0ms | 0.14x   |
|    local-high | 1016  | 191    | 1        | unchanged rerun      | 605.5ms  | 78.6ms   | 7.71x   |
|    local-high | 1016  | 191    | 1        | body-only local edit | 604.8ms  | 150.4ms  | 4.02x   |
|    local-high | 1016  | 191    | 1        | shape change         | 601.8ms  | 674.6ms  | 0.89x   |
|    local-high | 1016  | 191    | 1        | import change        | 602.4ms  | 688.7ms  | 0.87x   |
|    local-high | 1016  | 191    | 1        | known import toggle  | 601.7ms  | 675.6ms  | 0.89x   |
|  external-low | 42    | 243    | 342      | cold run             | 1490.8ms | 7499.4ms | 0.20x   |
|  external-low | 42    | 243    | 342      | unchanged rerun      | 1198.1ms | 16.2ms   | 73.98x  |
|  external-low | 42    | 243    | 342      | body-only local edit | 1113.6ms | 81.4ms   | 13.68x  |
|  external-low | 42    | 243    | 342      | shape change         | 1208.2ms | 405.4ms  | 2.98x   |
|  external-low | 42    | 243    | 342      | import change        | 1186.3ms | 421.1ms  | 2.82x   |
|  external-low | 42    | 243    | 342      | known import toggle  | 1056.9ms | 431.2ms  | 2.45x   |
| external-high | 117   | 243    | 342      | cold run             | 1448.5ms | 7643.8ms | 0.19x   |
| external-high | 117   | 243    | 342      | unchanged rerun      | 1132.0ms | 22.5ms   | 50.37x  |
| external-high | 117   | 243    | 342      | body-only local edit | 1167.2ms | 91.6ms   | 12.75x  |
| external-high | 117   | 243    | 342      | shape change         | 1224.0ms | 483.1ms  | 2.53x   |
| external-high | 117   | 243    | 342      | import change        | 1286.2ms | 467.1ms  | 2.75x   |
| external-high | 117   | 243    | 342      | known import toggle  | 1268.8ms | 468.5ms  | 2.71x   |
+---------------+-------+--------+----------+----------------------+----------+----------+---------+

• Stock: is google/wire, not goforj/wire
• cold run: first wire gen
• unchanged rerun: run wire gen again without changing any files.
• body-only local edit: change only function body/content in a local Go file, without changing imports, types, or constructor signatures.
• shape change: change local type/provider shape, like constructor params, fields, or return shape, while staying within the same general dependency graph.
• import change: add or remove an import in a local package, which can change the discovered package graph and cached shape.
• known import toggle: switch back to a previously seen import/shape state in the same repo, so the loader can potentially reuse an already-known cached graph.

Implementation Details

This PR replaces the primary go/packages path with a custom loader that is much more intentional about what work it does and when it does it.

• Discover the package graph for the requested roots
• Load typed package state from that discovered graph
• Cache both discovery state and typed artifacts aggressively
• Fall back cleanly when the custom path cannot safely satisfy the request

There are two different kinds of work:

• Discovery work tells us what packages, files, and imports are involved
• Typed work is the expensive parse, typecheck, and package stitching work

By separating those layers we can skip far more repeated work than before.

Loader

Most of the new implementation lives in internal/loader.

• custom.go is the main custom loading backend
• discovery.go and discovery_cache.go build and reuse the discovered graph
• artifact_cache.go stores typed package artifacts for reuse between runs
• fallback.go preserves a safe fallback path when needed
• timing.go exposes timings so we can see where time is actually going

Caching

Caching is now a first class part of generation instead of a thin optimization layered on afterward.

• Discovery cache stores the discovered package graph
• Loader artifacts store typed package summaries for local and external packages
• Cache continues to support parser level reuse
• Output cache remains available for generated output reuse

This is what enables the very fast unchanged rerun and body-only local edit paths in the benchmark table.

• External packages are heavily reused after the first run
• Local packages only pay for the work required by the current edit shape
• Previously seen states can often reuse old cache state instead of rediscovering everything

Parser / Wire Integration

The parser and provider set handling needed to be updated to operate cleanly with cached semantic state.

• internal/wire/parse.go now reconstructs provider information from semantic artifacts when safe
• internal/wire/output_cache.go was updated around the new loader flow
• internal/wire/wire.go now drives generation through the new loading backend
• internal/wire/load_debug.go and internal/wire/loader_timing_bridge.go make the new path observable via -timings flag

One important detail here is that the system still prefers falling back cleanly over trusting cached reconstruction when it is not safe to do so.

CLI

The command surface was updated to route through the same generation model instead of splitting behavior between commands.

• wire gen, wire check, wire diff, and wire show now run through the new backend
• wire watch stays on the same core generation path rather than inventing a separate implementation
• wire cache was expanded so cache inspection and clearing are easier
• cmd/wire/main.go now wires cache and loader behavior more explicitly

Colorization There are now new colorization for errors (red) and success (green) to make things a little easier to read in between watcher tooling spam. Multiline errors are now presented more user friendly.

Compatibility / Safety

This is meant to be a conservative loader change, not a semantic rewrite of Wire. Everything works the same outside of some of the tweaks goforj/wire has made by introducing wire cache clear wire serve

• Existing generation entrypoints stay intact
• Fallback behavior exists when the custom path cannot safely proceed
• Test coverage was added heavily around loader behavior, parser behavior, and command integration

That shows up in:

• internal/loader/loader_test.go
• internal/wire/wire_test.go
• internal/wire/parse_coverage_test.go
• cmd/wire/main_test.go

Benchmarks

The benchmark harness was also expanded so it measures concrete developer workflows instead of only raw repo scale. (Seen in the table above)

• scripts/import-benchmarks.sh now prints both scale and scenario tables
• internal/wire/import_bench_test.go now measures edit types like unchanged reruns, body edits, shape edits, and import toggles
• scenario runs distinguish cold runs from warmed runs
• benchmark output now reports real graph composition:
  • local packages
  • stdlib packages
  • external packages

There are now a few benchmark profiles:

• local for a modest local graph
• local-high for a very large local graph
• external for a graph with a much heavier external dependency surface

[zzzz465]

I tried this version and it works fine in local environment, decreased wiring time from 20s to roughly 1s.
however, there's some issue that block using this from CI environment.

1. mtime is changed when cache is stored/loaded in CI environment, cache is always considered stale.
2. discovery-cache is hardcoded to UserCacheDir(). this can be resolved by caching whole cache dir, but it is good to have a granular control over cache directory.

[cmilesio]

I tried this version and it works fine in local environment, decreased wiring time from 20s to roughly 1s. however, there's some issue that block using this from CI environment.

1. mtime is changed when cache is stored/loaded in CI environment, cache is always considered stale.
2. discovery-cache is hardcoded to UserCacheDir(). this can be resolved by caching whole cache dir, but it is good to have a granular control over cache directory.

Thank you! Merged your changes in and unified the cache directory settings.

You can now point all Wire-managed caches at one base directory with WIRE_CACHE_DIR.

WIRE_CACHE_DIR=/tmp/wire-cache wire gen

That makes Wire use:

/tmp/wire-cache/loader-artifacts
/tmp/wire-cache/discovery-cache
/tmp/wire-cache/output-cache

The existing per-cache env vars still work and take precedence if set:

• WIRE_LOADER_ARTIFACT_DIR
• WIRE_DISCOVERY_CACHE_DIR
• WIRE_OUTPUT_CACHE_DIR

To test

go install github.com/goforj/wire/cmd/wire@8a3ed6d

Going back to your quote

I tried this version and it works fine in local environment, decreased wiring time from 20s to roughly 1s.
however, there's some issue that block using this from CI environment.

Is this 20s to 1s on a pubg service ?

[cmilesio]

@zzzz465 I'm sure you're busy - have you had a chance to try with these changes?

Thank you

[zzzz465]

I benchmarked the version you suggested with stock, here's the result

Version	Cold	Warm	Warm2	Warm3
google/wire v0.7.0	10,031ms	11,755ms	5,404ms	4,042ms
goforj/wire v1.1.0	25,183ms	577ms	572ms	578ms
8a3ed6d	49,688ms	3,264ms	3,268ms	3,349ms

• go 1.26.1
• cold: running wire without any wire cache
• warm: running wire after running wire first, delete all wire generated files, then run wire again
• have 10 targets to wire (10 wire.Build(...))

I also ran wire@8a3ed6d with -timings to see which one is causing bottleneck

loader.custom.root.discovery              = 122ms

# External artifact cache — works well
loader.custom.typed.artifact_prefetch     = 251ms
loader.custom.typed.artifact_hits         = 2292
loader.custom.typed.artifact_misses       = 2        # net, os/user only
loader.custom.typed.artifact_writes       = 0

# Local packages — no artifact cache read, full re-typecheck every time
loader.custom.typed.parse_files.local     = 503ms
loader.custom.typed.typecheck.local       = 10,399ms  # <- bottleneck?
loader.custom.typed.typecheck.external    = 3ms

# Other
loader.custom.typed.artifact_import_link  = 3,244ms
loader.custom.typed.root_load.wall        = 2,596ms   # per-root, parallel

generate.output_cache = miss??

I did some more digging, I've found that the cache is actually not being hit, because artifact_cache.go hits but output_cache.go also has an mtime issue, which I addressed in #7

it might be possible to use git for lookup hash for each file, which would completely eliminate file I/O, but I'm not sure this is a good idea, since it would depends on external library.
I think this is as far as we can get with hashing mechanism. if this PR targets local environments only, you should rollback my previous #5 PR and go back to mtime based caching.

[cmilesio]

I benchmarked the version you suggested with stock, here's the result

Version Cold Warm Warm2 Warm3
google/wire v0.7.0 10,031ms 11,755ms 5,404ms 4,042ms
goforj/wire v1.1.0 25,183ms 577ms 572ms 578ms
8a3ed6d 49,688ms 3,264ms 3,268ms 3,349ms

• go 1.26.1
• cold: running wire without any wire cache
• warm: running wire after running wire first, delete all wire generated files, then run wire again
• have 10 targets to wire (10 wire.Build(...))

I also ran wire@8a3ed6d with -timings to see which one is causing bottleneck

loader.custom.root.discovery              = 122ms

# External artifact cache — works well
loader.custom.typed.artifact_prefetch     = 251ms
loader.custom.typed.artifact_hits         = 2292
loader.custom.typed.artifact_misses       = 2        # net, os/user only
loader.custom.typed.artifact_writes       = 0

# Local packages — no artifact cache read, full re-typecheck every time
loader.custom.typed.parse_files.local     = 503ms
loader.custom.typed.typecheck.local       = 10,399ms  # <- bottleneck?
loader.custom.typed.typecheck.external    = 3ms

# Other
loader.custom.typed.artifact_import_link  = 3,244ms
loader.custom.typed.root_load.wall        = 2,596ms   # per-root, parallel

generate.output_cache = miss??

I did some more digging, I've found that the cache is actually not being hit, because artifact_cache.go hits but output_cache.go also has an mtime issue, which I addressed in #7

it might be possible to use git for lookup hash for each file, which would completely eliminate file I/O, but I'm not sure this is a good idea, since it would depends on external library. I think this is as far as we can get with hashing mechanism. if this PR targets local environments only, you should rollback my previous #5 PR and go back to mtime based caching.

I'm assuming this feedback is anchored to you running this in CI and trying to restore cache artifacts right? Not pertained to actual local development, correct?

[zzzz465]

@cmilesio yes, that benchmark result was run in CI. for the local environment result, check #7 (I see you already reviewed that, though)

[cmilesio]

@cmilesio yes, that benchmark result was run in CI. for the local environment result, check #7 (I see you already reviewed that, though)

Sounds good. I'll take a look soon. Had a longer weekend away and I know I'm going to need likely a few hours for this one.

It's only in CI you need this content based caching right? You mentioned it would suffice if that was environment variable / feature flag toggle-able ?

Let me know, thanks!

[cmilesio]

@cmilesio yes, that benchmark result was run in CI. for the local environment result, check #7 (I see you already reviewed that, though)

I added a mode just for your CI use case, where you restore S3 files and modtime gets trashed. You can flip on a environment variable to use file content caching for all 3 caching layers.

• Discovery cache
• Loader artifact cache
• Generated output cache

Environment Variable

• WIRE_CACHE_MODE=mtime as the default
• WIRE_CACHE_MODE=content to switch to content-based identity

Benchmark Results

Not a huge hit to general performance relative to mtime but it is there

WIRE_CACHE_MODE=content scripts/import-benchmarks.sh scenarios

+---------------+-------+--------+----------+----------------------+----------+----------+---------+
|       profile | local | stdlib | external | change type          | stock    | current  | speedup |
+---------------+-------+--------+----------+----------------------+----------+----------+---------+
|         local | 41    | 191    | 1        | cold run             | 375.1ms  | 2996.9ms | 0.13x   |
|         local | 41    | 191    | 1        | unchanged rerun      | 341.2ms  | 9.3ms    | 36.62x  |
|         local | 41    | 191    | 1        | body-only local edit | 334.4ms  | 9.6ms    | 34.76x  |
|         local | 41    | 191    | 1        | shape change         | 339.7ms  | 124.5ms  | 2.73x   |
|         local | 41    | 191    | 1        | import change        | 329.5ms  | 126.1ms  | 2.61x   |
|         local | 41    | 191    | 1        | known import toggle  | 342.3ms  | 126.2ms  | 2.71x   |
|    local-high | 1016  | 191    | 1        | cold run             | 746.4ms  | 5252.6ms | 0.14x   |
|    local-high | 1016  | 191    | 1        | unchanged rerun      | 605.8ms  | 94.5ms   | 6.41x   |
|    local-high | 1016  | 191    | 1        | body-only local edit | 603.7ms  | 88.6ms   | 6.82x   |
|    local-high | 1016  | 191    | 1        | shape change         | 593.6ms  | 620.5ms  | 0.96x   |
|    local-high | 1016  | 191    | 1        | import change        | 596.9ms  | 614.4ms  | 0.97x   |
|    local-high | 1016  | 191    | 1        | known import toggle  | 615.1ms  | 609.1ms  | 1.01x   |
|  external-low | 42    | 243    | 342      | cold run             | 1545.8ms | 7761.5ms | 0.20x   |
|  external-low | 42    | 243    | 342      | unchanged rerun      | 1072.6ms | 18.0ms   | 59.58x  |
|  external-low | 42    | 243    | 342      | body-only local edit | 1129.0ms | 16.8ms   | 67.16x  |
|  external-low | 42    | 243    | 342      | shape change         | 1086.7ms | 352.2ms  | 3.09x   |
|  external-low | 42    | 243    | 342      | import change        | 1128.5ms | 346.9ms  | 3.25x   |
|  external-low | 42    | 243    | 342      | known import toggle  | 1112.8ms | 339.6ms  | 3.28x   |
| external-high | 117   | 243    | 342      | cold run             | 1473.7ms | 7785.3ms | 0.19x   |
| external-high | 117   | 243    | 342      | unchanged rerun      | 1167.7ms | 22.6ms   | 51.68x  |
| external-high | 117   | 243    | 342      | body-only local edit | 1109.0ms | 23.3ms   | 47.65x  |
| external-high | 117   | 243    | 342      | shape change         | 1117.6ms | 386.6ms  | 2.89x   |
| external-high | 117   | 243    | 342      | import change        | 1158.9ms | 384.6ms  | 3.01x   |
| external-high | 117   | 243    | 342      | known import toggle  | 1207.6ms | 377.5ms  | 3.20x   |
+---------------+-------+--------+----------+----------------------+----------+----------+---------+

Can test with

go install github.com/goforj/wire/cmd/wire@d591989

cc @zzzz465

[zzzz465]

@cmilesio I tested your new version and it works great!
now in CI environment cache is properly loaded

[cmilesio]

@cmilesio I tested your new version and it works great! now in CI environment cache is properly loaded

That’s good news!

[jostreff]

I still don't understand - is it official that version 1.2.0 is out, or it's still not production ready?

On first page it is shown only version 1.1.0, but this one exists as new version tag...

[cmilesio]

I still don't understand - is it official that version 1.2.0 is out, or it's still not production ready?

On first page it is shown only version 1.1.0, but this one exists as new version tag...

Hello! It's out and is available under latest tag 1.2.0 - it doesn't need to be "Released" for the go versioning system to pick it up. So @latest during instal should grab it.

It's been stable and been using it for nearly 2 months and hasn't had any issues through manual testing.

[jostreff]

I still don't understand - is it official that version 1.2.0 is out, or it's still not production ready?
On first page it is shown only version 1.1.0, but this one exists as new version tag...

Hello! It's out and is available under latest tag 1.2.0 - it doesn't need to be "Released" for the go versioning system to pick it up. So @latest during instal should grab it.

It's been stable and been using it for nearly 2 months and hasn't had any issues through manual testing.

Thanks!

updating freebsd port devel/go-wire to latest version then.