perf(aws): replace channel-based last-used fetch with atomic worker pool#5
Open
watany-dev wants to merge 1 commit into
Open
perf(aws): replace channel-based last-used fetch with atomic worker pool#5watany-dev wants to merge 1 commit into
watany-dev wants to merge 1 commit into
Conversation
ListRoles hydrated each role's last-used timestamp through a worker pool built on two len(roles)-sized channels plus a dedicated collector goroutine, and always spawned 55 goroutines regardless of role count. Extract the concurrency into hydrateLastUsed, which hands out work via an atomic index counter and lets each worker write straight into its own slice index. This removes both channels and the collector goroutine, caps worker count at min(concurrency, len(roles)), and short-circuits empty input before building the progress bar. Benchmarks (pkg/aws, in-memory fetch): - orchestration overhead: 3.3-4.5x faster, ~2.4x less memory - under simulated API latency: equal wall-clock, ~2.2x less memory Adds correctness tests and head-to-head benchmarks against the previous channel-based implementation so the production code path is measurable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ListRoles hydrated each role's last-used timestamp through a worker pool
built on two len(roles)-sized channels plus a dedicated collector
goroutine, and always spawned 55 goroutines regardless of role count.
Extract the concurrency into hydrateLastUsed, which hands out work via an
atomic index counter and lets each worker write straight into its own
slice index. This removes both channels and the collector goroutine,
caps worker count at min(concurrency, len(roles)), and short-circuits
empty input before building the progress bar.
Benchmarks (pkg/aws, in-memory fetch):
Adds correctness tests and head-to-head benchmarks against the previous
channel-based implementation so the production code path is measurable.