Skip to content

fix(aws/recommendations): paginate CE recommendation + RDS engine-version describe (single-page truncation) #692

Description

@cristim

Symptom

Three AWS describe/recommendation calls in CUDly read only the first page of their result set and silently discard everything after. Each output struct has a NextPageToken (or NextToken) that we never follow. For customers with large payer accounts (i.e. the entire reason CUDly exists), Cost Explorer can return recommendations across multiple pages; we are emitting only a fraction of them and the user has no signal that it is partial.

This is a correctness bug, not a Lambda-budget hazard like #691: the values we emit are wrong, not "potentially slow". A customer can act on a partial RI/SP purchase recommendation thinking it is the complete picture.

Affected call sites (verified on feat/multicloud-web-frontend + PR #690 tip)

High-impact -- silently truncates customer-facing recommendations

  1. providers/aws/recommendations/client.go:93 -- GetReservationPurchaseRecommendation is called inside a rate-limiter retry loop but the loop only retries on retryable errors. result.NextPageToken is never read. Every page-2+ RI recommendation is silently dropped. This is the path that produces every AWS RI recommendation the UI shows.

  2. providers/aws/recommendations/parser_sp.go:63 -- GetSavingsPlansPurchaseRecommendation -- same shape (rate-limiter retry around a single SDK call, result.NextPageToken ignored). Drops every page-2+ Savings Plans recommendation.

Lower-impact -- truncates RDS engine-version metadata

  1. cmd/multi_service_engine_versions.go:198 -- DescribeDBMajorEngineVersions is called once per engine in a for _, engine := range engines loop but output.NextToken is never read. If AWS spans the major-version list across pages for any engine, the RDS extended-support filter (which uses this output) silently misses versions, causing either over-recommendation or under-recommendation of RDS RIs.

Fix shape

For each call site, wrap the SDK call in a token-driven pagination loop:

var nextPageToken *string
for {
    if err := ctx.Err(); err != nil {
        return nil, err
    }
    input.NextPageToken = nextPageToken  // or input.NextToken for the SDK calls that use it
    result, err := <existing rate-limiter loop, but now per-page>
    if err != nil { ... }
    // accumulate result.<items>
    if result.NextPageToken == nil || aws.ToString(result.NextPageToken) == "" {
        break
    }
    nextPageToken = result.NextPageToken
}

Same maxPages cap (e.g. 20) + ctx.Err() between pages from the PR #690 / #691 pattern; same nil-or-empty terminator from CR's category-A finding on PR #690 (see commit f2becf790).

Regression test per call site

  • Mock the SDK to return N pages with NextPageToken set, plus a final page with NextPageToken == nil. Assert the parser sees ALL recommendations across all pages, not just page 1.
  • Add a separate mock that returns empty-string NextPageToken. Assert the loop terminates without an extra call (parity with the PR fix(purchases): narrow Describe*Offerings + cap pagination (#688) #690 fix).

Acceptance criteria

  • All 3 call sites paginate via NextPageToken / NextToken
  • Each loop has ctx.Err() between pages and a maxPages cap with diagnostic error
  • Each loop terminates on nil OR empty-string token (PR fix(purchases): narrow Describe*Offerings + cap pagination (#688) #690 parity)
  • Per-call-site regression test asserting the multi-page accumulation
  • No regression in existing RI/SP rec tests
  • Manual verification against a large payer account: cudly recommendations refresh for a 100+ account org returns more recs than before the fix

Cross-references

Activity

  1. added a commit that references this issue on May 22, 2026
    423e87c
  2. cristim commented on May 28, 2026

    @cristim
    MemberAuthor

    Verified shipped on feat/multicloud-web-frontend via PR #693 (merged 2026-05-22). GetReservationPurchaseRecommendation, GetSavingsPlansPurchaseRecommendation, and DescribeDBMajorEngineVersions now loop over all pages using NextPageToken/NextToken. Large payer accounts no longer silently receive only page-1 results.

  3. added a commit that references this issue on Sep 27, 2026
    02722a0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions