From be478b03ebb56280a0a9870f6fe8498bc07728bd Mon Sep 17 00:00:00 2001 From: raytoo Date: Tue, 8 Sep 2026 10:24:27 +0800 Subject: [PATCH 1/3] feat(commitments): risk-adjusted SP/RI sizing, expiry renewals, purchasable specs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds discounted-commitment analysis in two forms: a portable markdown-only skill that drives the AWS CLI, and a `commitments` Lambda MCP tool that runs the same method in Python behind the gateway. The method never presents the AWS best case as achievable. It classifies the spend profile from the observed hourly floor, sizes the commitment down accordingly, scales savings linearly, and blocks on existing under-utilization before quoting any figure — including the case the raw API will endorse, where break-even lands beyond the term. Expiry and renewal (`get_commitment_expiry`, opt-in via `regions` on the analysis tool): inventories Savings Plans and per-service reservations, derives the end date where AWS returns none (`StartTime + Duration`), buckets at 30/60/90 days, and gives a renew / renew-smaller / let-lapse / review verdict — expiry being the one moment a commitment can be resized at zero switching cost. Savings Plans are fetched once account-wide, never per region. Recommendations now carry what is actually purchasable. `RecommendationSummary` sums every specification in `details[]`, so one RDS finding can span `db.r6g.large Multi-AZ` and `db.t4g.medium Single-AZ`; a reservation only discounts usage matching its exact spec. Each finding therefore breaks down into line items naming the instance type, deployment option, engine and AZ, with size-flexible and previous-generation flags, and the family total is quoted as a budget rather than an order. Read-only throughout: no purchase, renewal, modification or cancellation API is ever called. `queries_run` counts billable Cost Explorer requests only, so the free `Describe*` inventory does not appear as spend. Reservation unit counts are never converted to money, and account-level utilization is never attributed to an individual commitment — both would need data this feature does not query. Known blind spot, declared in the report: DynamoDB reserved capacity has no describe API in any SDK. Also fixes gateway schema sync dropping the newest target. `list_gateway_targets` was called without following `nextToken`, so boto3 returned one bounded page — at 11 targets that silently omitted the 11th, and a missing target reads as "not found in gateway", failing the sync for a target that is present. Surfaced while deploying this change: `lambda-runtime` had a gateway target carrying zero tool schemas, leaving its 8 tools unreachable, and every tool added past the page boundary would have hit the same wall. Tests: 274 commitments tests, 1006 unit tests passing. Deployed and verified in dev: 34 in-place changes, all 7 new `Describe*` permissions authorized on a live invoke, and both gateway targets now carrying their full schemas. --- README.md | 16 +- docs/agents/cost-operations.md | 21 +- docs/development.md | 2 +- docs/skills/discounted-commitments.md | 1039 +++++++++++++++++ scripts/lib/sync.sh | 20 +- skills/discounted-commitments/SKILL.md | 332 ++++++ .../reference/method.md | 254 ++++ .../reference/output-template.md | 252 ++++ src/agents/hierarchy.json | 13 +- .../discounted_commitments.json | 12 + .../discounted_commitments.json | 12 + .../mcp/commitments/commitments/__init__.py | 1 + .../mcp/commitments/commitments/analyze.py | 816 +++++++++++++ src/lambda/mcp/commitments/commitments/api.py | 865 ++++++++++++++ .../mcp/commitments/commitments/collect.py | 581 +++++++++ .../mcp/commitments/commitments/report.py | 645 ++++++++++ src/lambda/mcp/commitments/handler.py | 603 ++++++++++ src/lambda/mcp/commitments/requirements.txt | 8 + src/lambda/mcp/tools.json | 222 ++++ tests/unit/conftest.py | 48 +- tests/unit/test_commitments_analyze.py | 511 ++++++++ tests/unit/test_commitments_api.py | 319 +++++ tests/unit/test_commitments_collect.py | 371 ++++++ tests/unit/test_commitments_expiry.py | 791 +++++++++++++ tests/unit/test_commitments_spec.py | 809 +++++++++++++ tests/unit/test_commitments_tool.py | 771 ++++++++++++ 26 files changed, 9309 insertions(+), 25 deletions(-) create mode 100644 docs/skills/discounted-commitments.md create mode 100644 skills/discounted-commitments/SKILL.md create mode 100644 skills/discounted-commitments/reference/method.md create mode 100644 skills/discounted-commitments/reference/output-template.md create mode 100644 src/agents/shared/report_templates/discounted_commitments.json create mode 100644 src/lambda/frontend/core-api/report_templates/discounted_commitments.json create mode 100644 src/lambda/mcp/commitments/commitments/__init__.py create mode 100644 src/lambda/mcp/commitments/commitments/analyze.py create mode 100644 src/lambda/mcp/commitments/commitments/api.py create mode 100644 src/lambda/mcp/commitments/commitments/collect.py create mode 100644 src/lambda/mcp/commitments/commitments/report.py create mode 100644 src/lambda/mcp/commitments/handler.py create mode 100644 src/lambda/mcp/commitments/requirements.txt create mode 100644 tests/unit/test_commitments_analyze.py create mode 100644 tests/unit/test_commitments_api.py create mode 100644 tests/unit/test_commitments_collect.py create mode 100644 tests/unit/test_commitments_expiry.py create mode 100644 tests/unit/test_commitments_spec.py create mode 100644 tests/unit/test_commitments_tool.py diff --git a/README.md b/README.md index 1d05be8..e3bdf05 100644 --- a/README.md +++ b/README.md @@ -247,8 +247,6 @@ scripts/ docs/ architecture.md # system internals -skills/ - developer-guide/SKILL.md # Adding agents / tools / collectors (interactive) agents/ # Per-leaf-agent references (deploy modes, data model, gotchas) README.md # When to add a file + section template cost-operations.md # Cost Explorer / CUR / COH reference @@ -257,8 +255,14 @@ skills/ network-resiliency.md # Direct Connect topology + resilience rules tag-governance.md # Tag governance feature reference lambda-upgrade.md # Lambda runtime upgrade discovery + migration + skills/ # Per-skill references (method, API surface, portability) + discounted-commitments.md # SP/RI risk-adjusted sizing + commitments MCP tool observability-tuning.md # X-Ray + Transaction Search knobs +skills/ # Portable agent skills (invocable as /skill-name) + developer-guide/SKILL.md # Adding agents / tools / collectors (interactive) + discounted-commitments/ # SP/RI sizing — 3 markdown files, no code + tests/unit/ # pytest + moto ``` @@ -339,6 +343,13 @@ See `.env.example` for the canonical identity-only `.env` template and setup, tag-policy bring-up commands, read-only-by-design rationale. - [docs/agents/lambda-upgrade.md](docs/agents/lambda-upgrade.md) — Lambda runtime discovery, code analysis, and migration guidance. +- [docs/skills/](docs/skills/) — per-skill reference files for skills + with a non-trivial method or a platform counterpart. + - [docs/skills/discounted-commitments.md](docs/skills/discounted-commitments.md) + — Savings Plan / Reserved Instance risk-adjusted sizing: the + volatility bands and break-even guards, the Cost Explorer API + surface and its per-request cost, the `commitments` MCP tool, and + how the portable skill and the deployed Lambda stay in agreement. - [docs/observability-tuning.md](docs/observability-tuning.md) — X-Ray sampling and Transaction Search indexing knobs. @@ -356,6 +367,7 @@ Skills provide the system's analytical capabilities as portable workflows that w | `/health-events-digest` | "any critical health events?" | Health event digest with risk scoring | | `/tag-governance-assessment` | "how's my tag compliance?" | Tag compliance scoring + remediation links | | `/lambda-runtime-upgrade` | "find deprecated lambda functions" | Multi-region deprecated-runtime discovery + migration report | +| `/discounted-commitments` | "what savings plans should we buy?" | Risk-adjusted SP/RI sizing — achievable vs AWS best case ([docs](docs/skills/discounted-commitments.md)) | ### How to use skills diff --git a/docs/agents/cost-operations.md b/docs/agents/cost-operations.md index c705289..27063cc 100644 --- a/docs/agents/cost-operations.md +++ b/docs/agents/cost-operations.md @@ -7,7 +7,7 @@ FinOps domain under `finops-agent` (peer of `pricing-agent`). ## 1. What the feature does -Answers spending questions using three complementary surfaces, from +Answers spending questions using four complementary surfaces, from narrowest to broadest: - **Cost Explorer API** — monthly / daily cost and usage, optional @@ -19,6 +19,12 @@ narrowest to broadest: - **Cost Optimization Hub** — savings recommendations (right-sizing, idle resources, Savings Plans, Reserved Instances) aggregated across the org. +- **Commitments** — risk-adjusted Savings Plan / Reserved Instance + purchase sizing. The only surface that reaches + `GetSavingsPlansPurchaseRecommendation` and + `GetReservationPurchaseRecommendation`, so it is the only one that can + size a commitment rather than just report existing coverage. Full + reference: [`docs/skills/discounted-commitments.md`](../skills/discounted-commitments.md). Representative prompts: @@ -182,6 +188,17 @@ Lambda hardcodes `region_name="us-east-1"`. Same pattern as } ``` +### `generate_commitment_analysis` (commitments) + +Returns a `report_markdown` field holding the complete pre-formatted +report, alongside the structured envelope (`recommendations`, `count`, +`total_estimated_monthly_savings`, `aws_best_case_monthly_savings`, +`reconciliation`, `existing_commitment_posture`, `blockers`). The agent +prompt requires emitting `report_markdown` verbatim rather than rebuilding +the table from the structured fields. Response shape and the other three +commitment tools: +[`docs/skills/discounted-commitments.md`](../skills/discounted-commitments.md). + ### `start_query_execution` (CUR/Athena) Synchronous — the Lambda waits for the Athena query to complete (up @@ -209,3 +226,5 @@ savings, anomalies, and forecast sections. See | Cost Explorer shows different totals than the bill | Metric mismatch (UnblendedCost vs AmortizedCost vs NetAmortizedCost) + Credits/Refunds exclusion | Specify the metric explicitly; match what the finance team uses for reconciliation | | Forecast returns an error for start_date in the past | `get_cost_forecast` requires future start_date | Use `get_cost_and_usage` for historical; forecast is future-only | | Model called `group_by=["SERVICE"]` when the user only asked "how much" | Model embellishing beyond the ask | The worker prompt already gates this; if it recurs, tighten the "ONE call, no group_by unless asked" rule | +| Model answered a "should we buy an SP?" question from `cost-optimization-hub` alone | COH publishes commitment recommendations but cannot size one | The worker prompt routes all SP/RI purchase questions to the `commitments` tools; see [`docs/skills/discounted-commitments.md`](../skills/discounted-commitments.md) | +| Unexpected Cost Explorer charges after a commitment question | CE bills $0.01 per recommendation request; a full default sweep is 48 | Narrow `savings_plan_types` / `ri_services` on `generate_commitment_analysis` | diff --git a/docs/development.md b/docs/development.md index 5c9c3d6..8a5c3f6 100644 --- a/docs/development.md +++ b/docs/development.md @@ -4,7 +4,7 @@ This file documents project-specific conventions, architecture, and gotchas for ## Project -CloudOps Multi-Agent System — a hierarchical multi-agent system for AWS cloud operations built on Amazon Bedrock AgentCore with Strands Agents SDK and an AG-UI streaming Next.js frontend. See `README.md` for architecture diagrams and `skills/developer-guide/SKILL.md` for step-by-step how-tos. `docs/agents/` has one reference file per leaf agent (deploy modes, data model, gotchas) — `health-events.md` and `tag-governance.md` today; add a new file here for any new leaf with non-trivial deploy or operational surface. +CloudOps Multi-Agent System — a hierarchical multi-agent system for AWS cloud operations built on Amazon Bedrock AgentCore with Strands Agents SDK and an AG-UI streaming Next.js frontend. See `README.md` for architecture diagrams and `skills/developer-guide/SKILL.md` for step-by-step how-tos. `docs/agents/` has one reference file per leaf agent (deploy modes, data model, gotchas); add a new file there for any new leaf with a non-trivial deploy or operational surface. `docs/skills/` does the same for skills whose method needs documenting beyond `SKILL.md` — typically because a platform Lambda implements the same logic and the two copies have to stay in agreement. The richest source of project-specific conventions and gotchas is this file plus the `docs/` directory — treat them as authoritative. `docs/architecture.md` covers agent topology decisions. diff --git a/docs/skills/discounted-commitments.md b/docs/skills/discounted-commitments.md new file mode 100644 index 0000000..5f423ea --- /dev/null +++ b/docs/skills/discounted-commitments.md @@ -0,0 +1,1039 @@ +# Discounted Commitments — skill and MCP tool reference + +End-to-end reference for the `discounted-commitments` skill +([`skills/discounted-commitments/`](../../skills/discounted-commitments/)) +and the `commitments` Lambda MCP tool +([`src/lambda/mcp/commitments/`](../../src/lambda/mcp/commitments/)) that +implements the same method inside the platform. + +Part of the FinOps domain: the MCP tool binds to `cost-operations-agent` +under `finops-agent`. See +[`docs/agents/cost-operations.md`](../agents/cost-operations.md) for that +agent's other three tool surfaces. + +--- + +## 1. What the feature does + +Sizes AWS discounted commitments — Savings Plans and Reserved Instances — +to what a workload can actually sustain, then reports that instead of the +number AWS returns. + +The gap it exists to close: `GetSavingsPlansPurchaseRecommendation` +assumes the lookback window repeats forever and returns the commitment +that maximizes savings under that assumption. On a spiky workload that +means committing above the trough, and every quiet hour strands the +difference. The same API response also carries the **minimum** and +**average** hourly on-demand spend, which is enough to size against the +quietest hour instead — and to quantify what the AWS figure would have +wasted. + +Representative prompts: + +It also answers the other half of the commitment question — **what is already +committed, when it lapses, and what to renew.** A commitment that expires +unnoticed returns its covered spend to on-demand rates silently, and expiry is +the one moment a commitment can be resized at zero switching cost. + +Representative prompts: + +- `"What Savings Plans should we buy?"` +- `"Is a 3-year all-upfront worth it?"` +- `"Are our existing RIs being wasted?"` +- `"How much could we save with a Compute Savings Plan?"` +- `"Size an RDS reservation for us."` +- `"What commitments expire in the next 90 days?"` +- `"Which RIs should we renew?"` + +Pipeline: + +``` +"What Savings Plans should we buy?" + → supervisor → finops-agent → cost-operations-agent + → commitments___generate_commitment_analysis() (one call) + ├─ sweep 16 SP + 32 RI recommendation permutations (6 threads) + ├─ posture SP/RI coverage + utilization, COH enrollment, spend + ├─ expiry active commitment inventory per region (opt-in, free) + ├─ risk-adjust every recommendation down to its measured floor + ├─ reconcile CE total vs Cost Optimization Hub total + └─ render complete markdown report + → emit report_markdown verbatim +``` + +Two properties are load-bearing and worth stating up front: + +- **Read-only, always.** Every call is a `Get*` / `List*` / `Describe*`. + Nothing is purchased, renewed or cancelled, and no billable analysis job is + started. The output is input to a purchase decision, never authorization + for one. +- **Blockers precede savings.** Section order in the report is + deliberate: a reader cannot reach the savings total without passing + the existing-commitment health check. Buying on top of an + under-utilized commitment compounds waste rather than reducing it. + +--- + +## 2. Two artifacts, one method + +This feature ships twice, on purpose, and the two copies are not +interchangeable. + +| | Skill | MCP tool | +|---|---|---| +| Path | `skills/discounted-commitments/` | `src/lambda/mcp/commitments/` | +| Form | 3 markdown files, 838 lines, **zero code** | 5 Python modules, 3,510 lines | +| Dependency | AWS CLI v2 + read-only credentials | Lambda runtime, boto3, `shared.cross_account` | +| Who executes | The host agent, via Bash | The Lambda, called through the gateway | +| Arithmetic by | The agent, following `reference/method.md` | `commitments/analyze.py` | +| Portable | Yes — copy the directory anywhere | No — platform-coupled | +| Tests | None (no code to test) | 274 tests, `tests/unit/test_commitments_*.py` | + +The skill is the portable expression: it must run in any coding agent +with a shell, with nothing installed, so it drives the AWS CLI and states +the arithmetic as instructions. The MCP tool is the fast, deterministic +expression: the same method in Python, so the model makes one call +instead of orchestrating dozens of Cost Explorer requests and doing +floating-point work in its head. + +Consequence to keep in mind when editing: **the band constants and +formulas exist in both places.** `reference/method.md` and +`commitments/analyze.py` must agree. A change to one is a change to both, +and only the Python side has tests to catch drift. + +--- + +## 3. Execution paths — how the skill routes + +`SKILL.md` opens with this tree, evaluated top-down: + +``` +Does your host expose this analysis as a tool? +├── Yes → call it, emit its report verbatim. Nothing else needed. +└── No + ├── Can you run `aws` with read-only credentials? → query it + └── No + ├── Is a coding agent with a shell available? → delegate + └── No → stop; say sizing needs Cost Explorer access you lack +``` + +**Generic cost tools are not a substitute.** Cost Explorer and Cost +Optimization Hub wrappers expose spend, coverage, and COH's own +recommendations — but neither exposes +`GetSavingsPlansPurchaseRecommendation` or +`GetReservationPurchaseRecommendation`. Without those two APIs nothing +can size a commitment. In that situation the skill still reports eligible +spend and existing coverage, and is required to say plainly: *"Sizing and +risk adjustment are unavailable here — these figures are AWS best case, +unadjusted."* + +Out of scope by design: rightsizing, idle resources, and Graviton +migration (Cost Optimization Hub covers those), and general cost +breakdowns, trends, forecasts, and anomalies (use `/finops-analysis`). + +--- + +## 4. Prerequisites and permissions + +### Skill path (AWS CLI) + +- `aws --version` → **2.x**. v1 has no `cost-optimization-hub` command at + all. +- `aws sts get-caller-identity` must succeed. If it fails the skill stops + and produces no report rather than guessing. +- Cost Explorer enabled (on by default; takes ~24h to populate on a new + account). +- Cost Optimization Hub enrollment is **optional** — without it the skill + reports that reconciliation was unavailable and continues. + +### IAM — read-only is sufficient + +``` +ce:GetSavingsPlansPurchaseRecommendation +ce:GetReservationPurchaseRecommendation +ce:GetSavingsPlansCoverage +ce:GetSavingsPlansUtilization +ce:GetReservationCoverage +ce:GetReservationUtilization +ce:GetCostAndUsage +cost-optimization-hub:ListEnrollmentStatuses +cost-optimization-hub:ListRecommendations +savingsplans:DescribeSavingsPlans +ec2:DescribeReservedInstances +rds:DescribeReservedDBInstances +elasticache:DescribeReservedCacheNodes +redshift:DescribeReservedNodes +es:DescribeReservedInstances +memorydb:DescribeReservedNodes +sts:GetCallerIdentity +``` + +This is exactly the `iam_actions` list on the `commitments` entry in +[`src/lambda/mcp/tools.json`](../../src/lambda/mcp/tools.json) — the two +paths are permission-identical, which is what makes a skill run a valid +rehearsal for the deployed tool. + +The seven `Describe*` actions serve the expiry inventory only. Sizing works +without them: omit `regions` and the tool makes zero inventory calls and +returns `expiry: null`. Note that **OpenSearch's IAM prefix is `es:`**, not +`opensearch:` — the API is `opensearch describe-reserved-instances` but the +permission is `es:DescribeReservedInstances`. + +### Region is pinned to us-east-1 — except reservations + +Cost Explorer purchase recommendations and Cost Optimization Hub resolve +**only** in us-east-1, regardless of where the resources live or where +the stack is deployed. Savings Plans use a global endpoint that also resolves +in us-east-1. Every such CLI command in `SKILL.md` carries +`--region us-east-1`; the Lambda hardcodes `CE_REGION = COH_REGION = +"us-east-1"` in `commitments/api.py`. Same pattern as `pricing-agent`. + +**Reservations are the exception.** They are regional resources and only appear +in the region they were purchased in, so the expiry inventory takes an explicit +`regions` list and sweeps one call per (family, region). A us-east-1-only sweep +of an ap-northeast-1 fleet finds nothing and would otherwise report "nothing +expiring" — which is why the report always names the regions it swept. Savings +Plans are account-level and are fetched **once**, not per region; multiplying +them would return the same plans N times and inflate every total by N. + +### Cost + +Cost Explorer bills **$0.01 per recommendation request**. Coverage, +utilization, cost-and-usage, and every commitment-inventory `Describe*` call are +not billed at that rate. That distinction is enforced in code, not just +documented: `collect_all` snapshots `sweep_errors` before the expiry inventory +runs, and `queries_run` is computed from that snapshot — folding a failed free +`Describe*` into the counter would overstate the bill. + +| Run | Recommendation requests | Cost | +|---|---:|---:| +| Skill default (`COMPUTE_SP` × 2 terms × 2 payments) | 4 | $0.04 | +| MCP tool default (4 SP types + 8 RI services, × 2 terms × 2 payments) | 48 | $0.48 | +| Adding `PARTIAL_UPFRONT` to the tool default | 72 | $0.72 | + +The skill sweeps narrow by default and widens only on request. The tool +sweeps wide because it parallelizes and returns a full report in one +call — narrow it with `savings_plan_types` / `ri_services` when cost per +invocation matters. + +--- + +## 5. The AWS API surface + +### Verified enums + +Getting these wrong is the most common failure, so they are stated +literally in `SKILL.md` and in `commitments/api.py`. + +| Parameter | Values | +|---|---| +| `--savings-plans-type` | `COMPUTE_SP`, `EC2_INSTANCE_SP`, `SAGEMAKER_SP`, `DATABASE_SP` | +| `--term-in-years` | `ONE_YEAR`, `THREE_YEARS` | +| `--payment-option` | `NO_UPFRONT`, `PARTIAL_UPFRONT`, `ALL_UPFRONT` | +| `--lookback-period-in-days` | `SEVEN_DAYS`, `THIRTY_DAYS`, `SIXTY_DAYS` | +| `--account-scope` | `PAYER` (whole org), `LINKED` (this account) | + +`MACHINE_LEARNING_SP` **does not exist**. The fourth Savings Plan type is +`DATABASE_SP`. + +RI `--service` takes exactly these eight strings and no others. Note the +inconsistent ` Service` suffix — it is the API's, not a typo: + +| Cost Explorer `Service` | Short label | +|---|---| +| `Amazon Elastic Compute Cloud - Compute` | EC2 | +| `Amazon Relational Database Service` | RDS | +| `Amazon Redshift` | Redshift | +| `Amazon ElastiCache` | ElastiCache | +| `Amazon Elasticsearch Service` | Elasticsearch (legacy) | +| `Amazon OpenSearch Service` | OpenSearch | +| `Amazon MemoryDB Service` | MemoryDB | +| `Amazon DynamoDB Service` | DynamoDB | + +SageMaker, Lambda, Fargate, Aurora, CloudFront, S3, Neptune, DocumentDB, +MSK, Kinesis and Timestream are all rejected. Do not guess a service +name — copy the `Supported value(s)` list out of the API's own +`ValidationException`. + +### The six query groups + +`SKILL.md` organizes the CLI work into six numbered groups. Each carries +a `--query` (JMESPath) expression that trims the response to the fields +the method consumes — a bare recommendation response runs to tens of KB +of per-instance detail. + +1. **Savings Plans sizing** — `ce get-savings-plans-purchase-recommendation`, + one call per (type, term, payment). Projects `commit_hr`, `savings_mo`, + `savings_pct`, `ondemand_mo`, and per-detail `floor_hr`, `avg_hr`, + `upfront`, `est_util`, `commit_hr`, `spec` (`SavingsPlansDetails`). +2. **Reserved Instance sizing** — `ce get-reservation-purchase-recommendation`. + Projects `buy_units`, `floor_units`, `avg_units`, `upfront`, `monthly`, + `break_even_mo`, `util`, `savings_mo`, plus `spec` (`InstanceDetails`) and + `capacity` (`ReservedCapacityDetails`) — see §5b, without which a + recommendation is a unit count nobody can purchase. + `--service-specification OfferingClass=STANDARD` + is **EC2-only**; sending it to RDS or Redshift is a `ValidationException`. +3. **Existing commitment posture** — four calls + (`get-savings-plans-coverage`, `get-savings-plans-utilization`, + `get-reservation-coverage`, `get-reservation-utilization`). Run before + quoting any savings figure. +4. **Eligible spend** — `ce get-cost-and-usage` grouped by SERVICE. Run + first when the user has no specific target: an account whose bill is + serverless, storage and support has nothing to commit against, and + saying so costs $0. +5. **Reconciliation** — `cost-optimization-hub list-enrollment-statuses` + then `list-recommendations` filtered to + `PurchaseSavingsPlans` / `PurchaseReservedInstances`. +6. **Expiry inventory** — `savingsplans describe-savings-plans` (once, + us-east-1) plus one `describe-*` per (reservation family, region) across + EC2, RDS, ElastiCache, Redshift, OpenSearch and MemoryDB. Free. Filtered to + active states, then each end date is derived and bucketed. See §5a. + +### 5a. Deriving an end date — the field names differ everywhere + +Only Savings Plans and EC2 return an explicit end. The other five families +return `StartTime` plus `Duration` in **seconds**, so the end must be computed +as `StartTime + Duration` (31536000s = 1 year, 94608000s = 3 years). Every +family also names its id, count and type fields differently, which is why the +Lambda encodes them as a data table (`api.RESERVATION_INVENTORY`, a tuple of +frozen `InventorySpec` records) rather than six near-identical functions: + +| Family | Call | ID field | Count field | Type field | End | Match attributes | +|---|---|---|---|---|---|---| +| Savings Plans | `savingsplans describe-savings-plans` | `savingsPlanId` | `commitment` (USD/hr) | `savingsPlanType` | `end` | `ec2InstanceFamily` (EC2Instance plans only) | +| EC2 | `ec2 describe-reserved-instances` | `ReservedInstancesId` | `InstanceCount` | `InstanceType` | `End` | `Scope`, `AvailabilityZone`, `ProductDescription`, `OfferingClass`, `InstanceTenancy` | +| RDS | `rds describe-reserved-db-instances` | `ReservedDBInstanceId` | `DBInstanceCount` | `DBInstanceClass` | derived | `MultiAZ` (**bool**), `ProductDescription` | +| ElastiCache | `elasticache describe-reserved-cache-nodes` | `ReservedCacheNodeId` | `CacheNodeCount` | `CacheNodeType` | derived | `ProductDescription` | +| Redshift | `redshift describe-reserved-nodes` | `ReservedNodeId` | `NodeCount` | `NodeType` | derived | `ReservedNodeOfferingType` | +| OpenSearch | `opensearch describe-reserved-instances` | `ReservedInstanceId` | `InstanceCount` | `InstanceType` | derived | none — the API exposes no engine field | +| MemoryDB | `memorydb describe-reserved-nodes` | `ReservationId` | `NodeCount` | `NodeType` | derived | none | + +A field-name typo here fails silently — you get rows with blank IDs and no +expiry rather than an error — so `tests/unit/test_commitments_expiry.py` +asserts the derived end date per family rather than only for EC2. + +`InventorySpec.attribute_fields` holds the last column. These are what a +**renewal has to match**: an RDS reservation covers one deployment option and one +engine, and an EC2 reservation is pinned to one Availability Zone when `Scope` is +zonal. Renew against a different value and the discount silently does not apply. +They are normalized onto each inventory row as `attributes` (a label→value map) +and joined into a display `spec`. Two traps: + +- **`MultiAZ` is a boolean**, so it is tested against `None`, not truthiness — + `False` is the meaningful value `Single-AZ`, and dropping it as falsy would + leave a reader assuming Multi-AZ. `INVENTORY_ATTRIBUTE_VALUES` maps + `True`/`False` to `Multi-AZ`/`Single-AZ`; an absent field yields no key at all, + because absent is not the same claim as Single-AZ. +- **An empty `spec` on a Compute Savings Plan is correct**, not missing data — a + Compute plan commits to dollars and nothing else. Only an EC2 Instance plan is + family- and region-pinned. + +**Two blind spots, both declared rather than hidden:** + +- **DynamoDB reserved capacity has no describe API** in any SDK. It is listed + in `api.INVENTORY_BLIND_SPOTS` and printed in the report, because silently + omitting it would render as "nothing expiring" for a reservation that does. +- **Utilization is account-level only.** AWS publishes no per-commitment + utilization API, so every Savings Plan in the account shares one figure and + every reservation shares another. The verdicts are directional and the report + says so inline. + +### 5b. The purchasable spec inside a recommendation + +`GetReservationPurchaseRecommendation` returns a count and a savings figure per +line item, but **the thing you buy is in a service-specific sub-structure** — +and `RecommendationSummary` sums across all of them. One RDS recommendation +routinely spans `db.r6g.large Multi-AZ` and `db.t4g.medium Single-AZ`; since a +reservation only discounts usage matching its exact specification, that total is +a budget, not an order. + +`api.RECOMMENDATION_SPECS` is the data table for this (a tuple of frozen +`SpecShape` records), read by `api.describe_recommendation_spec(detail)`: + +| Service | Sub-object | Container | Size fields | Attributes | +|---|---|---|---|---| +| EC2 | `EC2InstanceDetails` | `InstanceDetails` | `InstanceType` | `AvailabilityZone`, `Platform`, `Tenancy` | +| RDS / Aurora | `RDSInstanceDetails` | `InstanceDetails` | `InstanceType` | `DeploymentOption`, `DatabaseEngine`, `DatabaseEdition`, `LicenseModel`, `DeploymentModel` | +| ElastiCache | `ElastiCacheInstanceDetails` | `InstanceDetails` | `NodeType` | `ProductDescription` | +| Redshift | `RedshiftInstanceDetails` | `InstanceDetails` | `NodeType` | none | +| MemoryDB | `MemoryDBInstanceDetails` | `InstanceDetails` | `NodeType` | none | +| OpenSearch / Elasticsearch | `ESInstanceDetails` | `InstanceDetails` | `InstanceClass` **+** `InstanceSize` | none | +| DynamoDB | `DynamoDBCapacityDetails` | `ReservedCapacityDetails` | none | `CapacityUnits` | + +Two services break the pattern, which is why this is a table and not one reader +per service: **OpenSearch has no `InstanceType`** (its type is split across +`InstanceClass` and `InstanceSize`, joined with a `.`) and no `Family`, and +**DynamoDB is not under `InstanceDetails` at all** — it has capacity units and a +region, no instance. `describe_recommendation_spec` returns `{}` for an unknown +shape so a caller degrades to the family-level figure instead of raising. + +`analyze.LineItem` carries the result per line: `spec`, `region`, its own `floor` +(`MinimumNumberOfInstancesUsedPerHour` — the count that line never dropped below, +so the part with no unused-commitment risk), `achievable`, `monthly_savings`, +`utilization_pct`, plus `size_flex_eligible` and `current_generation`. The last +two are opposite signals worth reading per line: size flexibility means the +recommended size is not binding, while a previous-generation instance means a +three-year commitment locks the account out of the cheaper current generation. + +Per-line `achievable` is `int(recommended × scale)` — reservations sell whole — so +the lines can sum **below** the family achievable total. The family math is left +untouched and the report discloses the shortfall, naming the line that should +absorb it, rather than padding a line or restating the headline savings. A line +whose sub-structure is absent renders as `analyze.SPEC_UNAVAILABLE` +(`(specification not returned)`), not as a blank. + +### JMESPath: `to_number()` is mandatory on money + +Cost Explorer returns money and percentages as JSON **strings**. A bare +ordering comparison does not silently return empty — it raises: + +``` +Groups[?Metrics.UnblendedCost.Amount>`1000`] + → TypeError: '>' not supported between instances of 'str' and 'int' + +Groups[?to_number(Metrics.UnblendedCost.Amount)>`1000`] + → [[['EC2', '5000.00']]] +``` + +Absent values arrive as `""`. Treat `""` and `null` as *no data*, never as +`0` — a `""` floor is an unmeasured floor, not a floor of zero. + +--- + +## 6. The method — risk adjustment + +Full statement in +[`skills/discounted-commitments/reference/method.md`](../../skills/discounted-commitments/reference/method.md); +implementation in +[`commitments/analyze.py`](../../src/lambda/mcp/commitments/commitments/analyze.py). +Applying it is **required**, not optional — emitting the AWS figure as +achievable is the exact error the feature exists to prevent. + +Constants: `HOURS_PER_MONTH = 730.0`; `TERM_MONTHS = {ONE_YEAR: 12, +THREE_YEARS: 36}`. + +### Savings Plans + +Sum across all `detail[]` entries: + +``` +floor_hr = Σ CurrentMinimumHourlyOnDemandSpend +avg_hr = Σ CurrentAverageHourlyOnDemandSpend (fallback: ondemand_mo ÷ 730) +``` + +Classify on `ratio = floor_hr ÷ avg_hr`: + +| ratio | Profile | Commitment to report | Confidence | +|---|---|---|---| +| ≥ 0.80 | stable | `commit_hr` — take the AWS figure as-is | High | +| ≥ 0.50 | moderate | `floor_hr + (commit_hr − floor_hr) × 0.5` | Medium | +| < 0.50 | spiky | `min(commit_hr, floor_hr)` — clamp to the floor | Low | +| `avg_hr` ≤ 0 | unknown | `commit_hr`, marked explicitly unvalidated | Low | + +Thresholds are `STABLE_FLOOR_RATIO = 0.80` and +`MODERATE_FLOOR_RATIO = 0.50`. The moderate case also caps at `commit_hr` +— never report a commitment above what AWS recommended. + +Savings scale linearly with commitment size, because the discount rate is +fixed per plan: + +``` +scale = safe_commit_hr ÷ commit_hr +safe_mo = savings_mo × scale ← the recommendation +waste_mo = max(0, commit_hr − floor_hr) × 730 ← stranded at the AWS figure +break_even_months = upfront ÷ safe_mo ← null, not 0, when upfront = 0 +``` + +Break-even uses the **adjusted** `safe_mo`. Paying back against savings +you will not achieve is the error being guarded against. With no upfront +cost break-even is `null` — reporting `0` reads as "pays back instantly". + +### Reserved Instances + +Same shape in whole units instead of dollars per hour, with three +differences: + +- **Round down.** Rounding up lands the commitment above the level just + judged safe. +- **Break-even comes from the API** (average the positive + `EstimatedBreakEvenInMonths` across `detail[]`), not from your own + division, so the figure matches the console. +- **Waste is a unit count**, so convert: + `waste_mo = (Σ monthly ÷ recommended) × (recommended − floor_units)`. + +If `recommended` comes out 0, re-read using the capacity-unit field names +(`RecommendedNumberOfCapacityUnitsToPurchase`, etc.) — that is how +DynamoDB reports. + +### Two guards that override the numbers + +- **Break-even beyond the term.** If `break_even_months` exceeds 12 or 36 + as applicable, the purchase expires before it pays back. Force + confidence to **Low** and lead the rationale with **"Do not buy"**. + This is the one case where the raw AWS API will endorse a purchase that + loses money outright, so it is checked every time. +- **Posture gate.** Existing utilization below `UTILIZATION_WARN_PCT = + 95.0`, or coverage above `COVERAGE_SATURATED_PCT = 90.0`, is a blocker, + reported *before* any savings figure. + +### Reconciliation + +Compare the **unadjusted** Cost Explorer total (Σ `savings_mo`) against +the COH total (Σ `estimatedMonthlySavings`). Like-for-like: COH also +publishes a best case, so comparing the adjusted figure would manufacture +a variance that is really just this method's own adjustment. + +``` +delta_pct = |ce_total − coh_total| ÷ max(|ce_total|, |coh_total|) × 100 +``` + +| `delta_pct` | Verdict | +|---|---| +| both totals 0 | agree-zero | +| ≤ 10% | reconciled | +| ≤ 30% | minor variance | +| > 30% | material variance | + +A material variance is **flagged, never averaged away** — a blended +number is defensible to nobody. Usual causes: differing account scope +(`PAYER` vs `LINKED`), or COH lagging a recent usage change. + +### Choosing what to report + +Keep **one finding per (family, label)**: highest `safe_mo` wins; break +ties toward the **shorter term**, then the **less upfront cash**, since +that is the lower-risk purchase. Sort the report by `safe_mo` descending. + +### Renewals — a different decision from a new purchase + +Implemented in `analyze.analyze_expiry`, which takes `as_of` as a parameter +rather than calling `date.today()` so it stays a pure function a test can pin. + +The premise: **a lapsing commitment has zero switching cost.** Mid-term, resizing +means buying out of an obligation; at expiry it is free. So expiry is the one +moment the size can change without penalty, and "let it lapse" is a legitimate +outcome rather than a failure to act. + +Buckets, on `days_remaining = end − as_of`: + +| `days_remaining` | Bucket | Constant | +|---|---|---| +| < 0 | `expired` — already back at on-demand rates, reported separately | — | +| ≤ 30 | `urgent` | `EXPIRY_URGENT_DAYS` | +| ≤ 60 | `soon` | `EXPIRY_SOON_DAYS` | +| ≤ 90 | `upcoming` | `EXPIRY_HORIZON_DAYS` | +| > horizon | counted in `total_active`, kept out of the table | — | + +Verdicts, on the family's utilization figure: + +| Utilization | Action | Reasoning | +|---|---|---| +| ≥ `UTILIZATION_WARN_PCT` (95%) | `renew` | Being consumed; lapsing returns that spend to on-demand | +| ≥ `RENEW_LAPSE_PCT` (50%) | `renew-smaller` | Partly wasted — renew at the consumed portion, using the free resize point | +| < 50% | `let-lapse` | Over half unused; re-size from current usage instead of renewing the mistake | +| `None` | `review` | Stated plainly. A verdict with no utilization behind it is a guess. | + +Exposure is quantified in the unit the data is actually in: + +``` +hourly_commitment_expiring = Σ commitment (Savings Plans, USD/hour) +monthly_committed_spend_expiring = hourly × 730 +reserved_units_expiring = Σ instance/node count (reservations, units) +``` + +Reservation unit counts are **deliberately not converted to dollars** — that +needs per-instance pricing this module never queries, and an invented rate would +be a fabricated figure. The two totals are separate keys and must never be +summed. An unparseable end date lands in `undated` and is reported as needing +manual checking rather than dropped. + +**Precedence:** an urgent expiry outranks every new-purchase recommendation. The +report's Bottom line surfaces the urgent count above the savings table, because a +renewal deadline is fixed and a purchase is optional. + +### Worked example + +`commit_hr = 10.00`, `savings_mo = 1000.00`, `floor_hr = 2.00`, +`avg_hr = 10.00`, `upfront = 0`: + +``` +ratio = 2.00 ÷ 10.00 = 0.20 → below 0.50 → spiky, Low confidence +safe = min(10.00, 2.00) = $2.00/hr +scale = 2.00 ÷ 10.00 = 0.20 +safe_mo = 1000.00 × 0.20 = $200.00/mo ← the recommendation +waste_mo = (10.00 − 2.00) × 730 = $5,840/mo stranded at the AWS figure +break_even = null (no upfront) +``` + +Reported as **$200/mo achievable** against an AWS ceiling of $1,000/mo, +Low confidence, because the quietest hour runs at 20% of average. + +--- + +## 7. Platform deployment — the `commitments` MCP tool + +### Where it sits + +``` +cost-operations-agent (leaf, under finops-agent) + tools: cost-explorer, cur-athena, cost-optimization-hub, commitments +``` + +Registered in +[`src/agents/hierarchy.json`](../../src/agents/hierarchy.json); tool +definitions in +[`src/lambda/mcp/tools.json`](../../src/lambda/mcp/tools.json). Gateway +tool names are target-prefixed — +`commitments___generate_commitment_analysis` and its three siblings. + +Deployed as `${PROJECT_TAG}-commitments-tool`: `python3.12`, **300 s** +timeout, **1024 MB**, both cross-account role aliases wired +(`CROSS_ACCOUNT_ROLE_ARN` for Cost Explorer, +`CROSS_ACCOUNT_ROLE_ARN_COH` for Cost Optimization Hub — same split as +the `cost-optimization-hub` tool, because COH can be enabled on a +delegated admin account separate from the payer). + +### The five tools + +| Tool | Use when | Returns | +|---|---|---| +| `generate_commitment_analysis` | Default. Any purchase question. | `report_markdown` (complete report) + structured envelope + posture + blockers + reconciliation + `expiry` when `regions` is passed | +| `size_savings_plans` | The question is narrowed to Savings Plans | Risk-adjusted SP findings, **no posture gate** | +| `size_reservations` | The question is narrowed to reservations | Risk-adjusted RI findings in whole units | +| `get_commitment_posture` | Only existing coverage/utilization is asked about | Coverage, utilization, blockers, `safe_to_buy_more`, COH enrollment, eligible spend | +| `get_commitment_expiry` | "What's expiring?", "What should we renew?" | Active commitment inventory bucketed urgent/soon/upcoming with a renew / renew-smaller / let-lapse / review verdict each, plus exposure totals and declared blind spots | + +`generate_commitment_analysis` is the preferred path and the agent prompt +says so. It is one call, and it is the only one that includes both the +posture gate and the reconciliation the numbers are defensible with. It +takes up to ~90 s for a full sweep. + +If a sizing tool is called directly, `get_commitment_posture` **must** +also be called — `size_savings_plans` and `size_reservations` +deliberately skip the gate, and their responses carry a `note` saying so. + +### Parameters (all five tools) + +| Parameter | Values | Default | +|---|---|---| +| `lookback` | `SEVEN_DAYS` / `THIRTY_DAYS` / `SIXTY_DAYS` | `THIRTY_DAYS` | +| `terms` | `ONE_YEAR`, `THREE_YEARS` | both | +| `payment_options` | `NO_UPFRONT`, `PARTIAL_UPFRONT`, `ALL_UPFRONT` | `NO_UPFRONT` + `ALL_UPFRONT` | +| `account_scope` | `PAYER` / `LINKED` | `PAYER` | +| `families` | `sp`, `ri` | both | +| `savings_plan_types` | the 4 SP types, or `all` | `all` | +| `ri_services` | short labels, exact CE names, or `all` | `all` | +| `posture_days` | integer ≥ 1 | `30` | +| `spend_days` | integer ≥ 1 | `60` | +| `regions` | AWS region names | `generate_commitment_analysis`: none (expiry skipped); `get_commitment_expiry`: the Lambda's own region | +| `services` / `inventory_services` | `ec2`, `rds`, `elasticache`, `redshift`, `opensearch`, `memorydb` | all | +| `horizon_days` / `expiry_horizon_days` | integer ≥ 1 | `90` | + +Every value arrives as caller-controlled JSON and is validated in +`handler.py` before any AWS call; a bad value returns +`{"error": "Invalid term: ... Choose from ..."}` rather than a 500. +Lists accept either a JSON array or a comma-separated string. + +`regions` gets stricter validation than the rest: region names are interpolated +into SDK endpoints, so `collect.resolve_regions` shape-checks each one against +`^[a-z]{2}(-[a-z]+)+-\d$`, lowercases, and de-duplicates (a repeated region would +otherwise double every total). On `generate_commitment_analysis` it is **opt-in**: +omit it and no inventory call is made at all, so a deployment granted only +`ce:Get*` keeps working and simply gets `expiry: null`. + +### Module layout + +``` +src/lambda/mcp/commitments/ +├── handler.py 603 lines — event → params → clients → collect +└── commitments/ + ├── api.py 661 — the read-only AWS calls, enums, labels, + │ InventorySpec table, inventory getters + ├── analyze.py 694 — bands, sizing, break-even, posture, + │ reconcile, expiry buckets + verdicts + ├── collect.py 552 — permutation sweep, thread pool, region + │ validation, expiry sweep, envelope + └── report.py 556 — markdown rendering +``` + +The `commitments/` subpackage stays boto3-free even though the inventory needs +regional clients. `api.Clients` carries a `make_client: Callable[[str, str], Any]` +factory as a **defaulted trailing field**, so `collect.py` and `analyze.py` never +import boto3 and every pre-existing `Clients(...)` construction still works +positionally. When the factory is absent the inventory getters return an +explained error record rather than raising — a host that granted only `ce:Get*` +gets a warning, not a crash. + +Two design rules hold this together: + +- **`handler.py` owns only what is specific to being a gateway tool** — + turning a caller event into validated parameters, and building AWS + clients from `shared.cross_account`. Everything between comes from + `commitments.collect`. Logic added to the handler instead is logic the + unit tests do not cover, and `TestHandlerDiscipline` fails the build + over it. +- **The `commitments/` subpackage carries no platform imports**, so it + stays independently testable and never opens a `boto3.Session` of its + own. + +Enum and default constants are **aliased** into `handler.py`, never +restated. A local copy would let the gateway tool accept a term the +shared pipeline rejects, and nothing would catch it. + +Fan-out is a `ThreadPoolExecutor` with `MAX_WORKERS = 6`. A failed job +lands as an error record in its own slot, so a partial sweep still +produces a report with a `collection_warnings` table and totals +explicitly labelled a lower bound. + +--- + +## 8. Data model + +### `generate_commitment_analysis` response + +``` +{ + "report_markdown": "# AWS Discounted Commitments Report\n...", + "account_id": "123456789012", + "generated_at": "2026-09-03T06:11:20Z", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + + "recommendations": [ ... ], + "count": 3, + "total_estimated_monthly_savings": 1840.22, + "aws_best_case_monthly_savings": 3011.75, + + "reconciliation": {"status": "reconciled", "delta_pct": 4.1, ...}, + "existing_commitment_posture": { ... }, + "blockers": ["Savings Plans utilization 82.0% is below the 95% floor"], + "expiry": null, // populated only when `regions` was passed + "queries_run": 48, + "collection_warnings": [], + "data_source": "live" +} +``` + +`queries_run` counts **billable** Cost Explorer recommendation requests only. It +is derived from `payload["sweep_errors"]`, a snapshot taken before the free +expiry `Describe*` calls run, so a region with no reservations does not appear +as $0.07 of nonexistent spend. + +### `get_commitment_expiry` response + +``` +{ + "account_id": "123456789012", + "as_of": "2026-09-04", + "horizon_days": 90, + "regions": ["ap-northeast-1", "us-east-1"], + "total_active": 11, + "expiring": [ ... ], // sorted by days_remaining, then label + "expired": [ ... ], // end date passed but still listed active + "undated": [ ... ], // no parseable end date — needs manual check + "counts": {"urgent": 1, "soon": 2, "upcoming": 0}, + "actions": {"renew": 2, "renew-smaller": 1, "let-lapse": 0, "review": 0}, + "hourly_commitment_expiring": 5.5, + "monthly_committed_spend_expiring": 4015.0, + "reserved_units_expiring": 4.0, + "blind_spots": ["DynamoDB reserved capacity (no describe API exists)"], + "renewal_actions": { ... }, + "collection_warnings": [], + "note": "...account-level utilization, read-only...", + "data_source": "live" +} +``` + +Per commitment: `family` (`savings-plan` | `reservation`), `service`, `label`, +`commitment_id`, `arn`, `instance_type`, `attributes`, `spec`, `quantity`, +`unit`, `region`, `state`, `payment_option`, `start`, `end`, `term_months`, +`days_remaining`, `urgency` (`urgent` | `soon` | `upcoming` | `expired`), +`utilization_pct`, `action` (`renew` | `renew-smaller` | `let-lapse` | +`review`), `rationale`. + +`utilization_pct` is `null` when unmeasured — `null` means unmeasured, never +zero, and `review` is the verdict that goes with it. + +`attributes` is the map of dimensions a renewal has to match (RDS: +`deployment`, `engine`; EC2: `scope`, `AZ`, `platform`, `class`, `tenancy`), and +`spec` is those joined onto `instance_type` for display — +`db.r6g.large · Multi-AZ · postgresql`. "Renew" means renew *the same thing*, so +a row without its spec is not actionable. A key absent from `attributes` means +AWS did not return the field, which is not the same claim as `Single-AZ`; `spec` +is empty for a Compute Savings Plan by design. + +### Per-recommendation shape + +``` +{ + "commitment_family": "sp", + "commitment_type": "Compute Savings Plan", + "term": "ONE_YEAR", + "payment_option": "NO_UPFRONT", + "aws_recommended_commitment": 10.0, + "achievable_commitment": 2.0, + "commitment_unit": "USD/hour", + "estimated_monthly_savings": 200.0, + "aws_best_case_monthly_savings": 1000.0, + "estimated_savings_percentage": 21.4, + "upfront_cost": 0.0, + "break_even_months": null, + "waste_exposure_monthly": 5840.0, + "confidence": "Low", + "spend_profile": "spiky", + "implementation_effort": "Low", + "rationale": ["..."], + "line_items": [ ... ] +} +``` + +`commitment_unit` is `USD/hour` for Savings Plans and `units` for +reservations. Reading an hourly-dollar commitment as a unit count +misreads it by roughly 1000×, so the unit travels with the number +everywhere it goes. + +**`line_items` is the purchasable part; the item-level total is not.** The total +sums every specification the service returned, and a reservation only discounts +usage matching its exact `spec` — so quote the item as a budget and its line +items as the order. Per entry: `spec`, `region`, `commitment_unit`, +`aws_recommended_commitment`, `achievable_commitment`, `minimum_observed_units`, +`average_observed_units`, `estimated_monthly_savings`, `upfront_cost`, +`monthly_on_demand_cost`, `estimated_utilization_percentage` (`null` when AWS did +not return it), `size_flexible`, `current_generation`, `account_id`. See §5b for +where each field comes from. + +Key names deliberately mirror a Cost Optimization Hub recommendation +list, so a host that already renders those needs no translation layer. +`reconciliation` is omitted when there was nothing to reconcile, so a +sizing-only response does not carry an empty key. + +### Report structure + +Eight sections, in this order, emitted by `report.py:render()` and +mirrored in +[`reference/output-template.md`](../../skills/discounted-commitments/reference/output-template.md): + +1. **Bottom line** — AWS best case vs risk-adjusted achievable vs + high-confidence-only, monthly and annual; leads with the urgent-expiry + count when there is one +2. **Reconciliation against AWS native tools** +3. **Existing commitment health** — blockers first +4. **Commitment expiry and renewal** — regions swept, exposure, the + expiring table (8 columns: ends, days, commitment, **spec**, size, region, + utilization, action), the account-level-utilization caveat, per-row + reasoning, already-ended commitments with their spec, declared blind + spots. Omitted entirely when the inventory did not run. +5. **Recommended commitments** — summary table, then per-finding detail + ending in a **line items** table (`Buy | Region | AWS units | Floor | + Achievable | Utilization | Savings/mo`) that names the instance type and + deployment option for each purchase, flags size-flexible and + previous-generation lines, and discloses any whole-unit rounding shortfall. + The table is omitted when no line has anything to buy. +6. **Eligible spend** +7. **Method** — bands restated in-report so figures are auditable +8. **Collection warnings** — present only when a query failed + +Emit only the sections you have data for, and name the ones you dropped. + +Section 4 sits between existing-commitment health and the new-purchase +recommendations on purpose: a commitment lapsing in three weeks is a decision +with a deadline, and it belongs ahead of an optional purchase. `render()` reads +it with `data.get("expiry")`, so a payload produced before this section existed +still renders — the section is simply absent. + +--- + +## 9. Report template + +[`discounted_commitments.json`](../../src/agents/shared/report_templates/discounted_commitments.json) +(mirrored at `src/lambda/frontend/core-api/report_templates/`) — one +section, `full_commitment_analysis`, which: + +1. Calls `generate_commitment_analysis` with **no arguments**. +2. Emits `report_markdown` **verbatim** — no summarizing, truncating, + re-ordering, or dropping table rows and rationale bullets. +3. Appends one extra section, **Purchase sequence**: clear every blocker + first; then High-confidence rows in descending achievable-savings + order; treat Medium as a smaller first tranche and re-measure after 30 + days; never buy a row whose break-even exceeds its own term. + +The template explicitly forbids rebuilding the recommendation table from +the structured fields — that is what guarantees the risk-adjusted +figures, the non-cancellable disclaimer, and every rationale bullet +survive to the reader. + +--- + +## 10. Portability — using the skill elsewhere + +The skill directory is three markdown files and nothing else: + +``` +discounted-commitments/ +├── SKILL.md # routing, queries, enums, constraints +└── reference/ + ├── method.md # required: risk adjustment + guards + └── output-template.md # report structure and field names +``` + +Copy the directory into another repo or another agent's skills folder and +it works unchanged. There is nothing to install, no `requirements.txt`, +no bundled scripts, and no reference to this platform anywhere in the +three files. The only runtime dependency is the AWS CLI, which the host +either has or does not — and the routing tree handles the case where it +does not. + +To delegate rather than run it locally, `SKILL.md` supplies a brief for +handing to a coding agent with a shell. If the delegate returns an error +(missing credentials, `AccessDenied`, expired SSO), pass it through with +the fix from the failure-modes table rather than retrying in your own +sandbox. + +--- + +## 11. Why "what skills do you have" never names this + +Worth stating explicitly, because it surprises people. + +**The gateway has no concept of skills.** Nothing in `src/`, `scripts/`, +or `terraform/` reads `skills/`. `sync_gateway_tools` uploads only tool +name, description, and `inputSchema`; the packaging step globs +`src/lambda/mcp/*/` and never sees the skill directory. + +When an agent is asked what it can do, it answers from the authoritative +inventory that `_inject_tool_inventory()` appends to its system prompt +([`src/agents/shared/agent_base.py`](../../src/agents/shared/agent_base.py)), +and which list that is depends on the agent's tier: + +- **Mid-level agents** list *child agents* from the + `cloudops-agent-registry` DynamoDB table, filtered on `parent_agent` + and `enabled`. The supervisor therefore names three children and + nothing deeper — which makes each child's `description` in + `hierarchy.json` the entire basis for what the supervisor believes it + can do. +- **Leaf agents** list *gateway MCP tools*, filtered to the `tools` + allowlist by `___` prefix. + +So the deepest name reachable is +`commitments___generate_commitment_analysis` — a tool name, not a skill +name. The skill reaches production only because its method was ported +into the Lambda. If commitment sizing needs to be discoverable from the +supervisor, the lever is `finops-agent.description`, not skill +registration. + +--- + +## 12. Testing + +```bash +.venv/bin/python -m pytest tests/unit/test_commitments_*.py -q +# 274 passed +``` + +| File | Covers | +|---|---| +| `test_commitments_analyze.py` | Bands at their boundaries, sizing, break-even, posture, reconciliation, selection | +| `test_commitments_api.py` | The Cost Explorer / COH calls, enum validation, response parsing | +| `test_commitments_collect.py` | Service/type resolvers, permutation sweep, partial-failure handling, envelope key names and units, plus `test_module_builds_no_clients_and_touches_no_files` — the guard that keeps `collect.py` free of clients and filesystem access | +| `test_commitments_expiry.py` | Per-family field mapping and derived end dates, date coercion across SDK/CLI shapes, expiry buckets and renewal verdicts at their boundaries, unit separation, region validation, expiry rendering, and that the Savings Plans job is **not** multiplied per region | +| `test_commitments_spec.py` | The per-service spec shapes (all seven, including the OpenSearch two-field type and the DynamoDB container), `MultiAZ` as a boolean, per-line breakdown and ranking, whole-unit rounding and its disclosure, the report's line-items and expiry `Spec` columns, and `line_items` in the envelope | +| `test_commitments_tool.py` | Handler parameter validation, error normalization, `TestHandlerDiscipline`, and `tools.json` wiring | + +The band tests pin the risk-adjustment thresholds *at* their boundaries +(0.80, 0.50, 95%, 90%, and the 30/60/90-day expiry cutoffs), which is what +makes a constant change visible rather than silent. Three expiry tests exist +specifically because their failure modes are silent rather than loud: + +- **Derived end dates, per family.** A wrong field name yields rows with blank + IDs and no expiry instead of an error, so every family is asserted, not just + EC2. +- **One Savings Plans call regardless of region count.** Sweeping SPs per region + would return the same plans N times and inflate every total by N. +- **`queries_run` excludes free `Describe*` failures.** Asserted directly, + because folding them in overstates a real dollar figure. + +The spec tests exist for the same reason: a mistyped field name in +`RECOMMENDATION_SPECS` yields a line with no instance type rather than an +exception, and writing them surfaced a real defect — `str(None)` rendering as the +literal `"None"`, which would have printed a fabricated specification into a +customer-facing report. + +`test_tools_json_declares_every_dispatched_tool` reads the dispatcher table out +of `handler.py` rather than restating it, so a tool added to one and not the +other fails the build in both directions. The skill's markdown copy of the +constants has no test — verify it by hand against `analyze.py` after any change. + +--- + +## 13. Known gotchas + +| Symptom | Likely cause | Fix | +|---|---|---| +| `ValidationException` on `--savings-plans-type MACHINE_LEARNING_SP` | That value does not exist | The fourth type is `DATABASE_SP` | +| `ValidationException` on `--service` | Not one of the eight exact strings | Copy the API's own `Supported value(s)` list from the error; do not guess | +| `ValidationException` mentioning `OfferingClass` | `--service-specification` sent to a non-EC2 service | EC2-only; drop it elsewhere | +| `TypeError: '>' not supported between instances of 'str' and 'int'` | JMESPath ordering comparison on a money string | Wrap in `to_number()` | +| `DataUnavailableException` with a blank `Message` | No existing commitment of that kind | Report "no existing commitment of this type", not a blank error | +| All posture metrics zero | **No commitment exists** — not 0% utilization on a broken one | Say which it is; they are very different findings | +| A `$0` / no-recommendation result | A real result, not a failure | Check eligible spend and explain there is nothing to commit against | +| Recommendation figure is not purchasable by the account | `PAYER` scope aggregates the whole org | Re-run with `--account-scope LINKED` | +| Endpoint resolution failure | Called outside us-east-1 | CE purchase recommendations and COH are us-east-1-only | +| COH empty or `NOT_ENROLLED` | Not enrolled | Continue; state that figures rest on Cost Explorer alone | +| Response blows up the context window | `--query` omitted | Always send it — bare responses run to tens of KB of per-instance detail | +| Unexpectedly large Cost Explorer bill | Wide sweep, repeated | $0.01 per recommendation request; note results as you go and never re-query a number you already have | +| Tool times out | Full 48-permutation sweep on a slow account | 300 s Lambda timeout; narrow `savings_plan_types` / `ri_services` | +| Agent reformats or summarizes the report | Model ignoring the verbatim rule | The template and agent prompt both mandate verbatim `report_markdown`; tighten if it recurs | +| Skill and Lambda disagree on a figure | Band constants drifted between `method.md` and `analyze.py` | They are two copies of one method — reconcile both | +| Expiry inventory finds nothing on an account that has reservations | Swept the wrong region — reservations only exist in the region they were bought in | Pass every region the account runs in; the report names the regions swept for exactly this reason | +| `AccessDenied` on `es:DescribeReservedInstances` | Granted `opensearch:` — OpenSearch's IAM prefix is `es:` | The API is `opensearch describe-reserved-instances`; the permission is `es:DescribeReservedInstances` | +| A commitment shows a blank expiry | Looked for `End`; only EC2 and Savings Plans return one | Derive it: `StartTime + Duration` seconds | +| Savings Plan exposure looks ~1000× too small | `commitment` read as a unit count | It is USD/hour — multiply by 730 for monthly | +| `expiry` is `null` on a `generate_commitment_analysis` response | `regions` was not passed — the inventory is opt-in | Pass `regions`, or call `get_commitment_expiry` | +| DynamoDB reserved capacity missing from the inventory | No describe API exists for it, in any SDK | Permanent blind spot, declared in `INVENTORY_BLIND_SPOTS` and printed in the report | +| Every expiring row shows the same utilization | Correct — AWS publishes utilization account-wide, never per commitment | Treat it as the portfolio signal it is; the report states this inline | +| A recommendation total cannot be purchased as one reservation | `RecommendationSummary` sums every specification in `details[]` — a single RDS finding routinely spans Multi-AZ and Single-AZ | Quote the total as a budget and buy from `line_items`; each line is one purchasable spec | +| OpenSearch line has no instance type | `ESInstanceDetails` has no `InstanceType` field | Join `InstanceClass` + `InstanceSize` (`r6g` + `large` → `r6g.large`) | +| DynamoDB recommendation has an empty spec | Its spec is not under `InstanceDetails` | Read `ReservedCapacityDetails.DynamoDBCapacityDetails` — capacity units and a region, no instance | +| A Single-AZ reservation renders as Multi-AZ-unknown, or vice versa | `MultiAZ` is a **boolean**, so a truthiness test collapses `False` and absent | Test against `None`: `False` means Single-AZ, absent means AWS did not say | +| Line items sum below the finding's achievable total | Each line is floored to whole reservations | Expected; the report names the shortfall and the line that should absorb it — do not pad a line or restate the headline | +| A line reads `(specification not returned)` | AWS gave a count without the service sub-structure | Treat the line as unpurchasable until confirmed in the console; it is not a spec of nothing | +| A Compute Savings Plan line shows `any instance family` | Correct — only `EC2_INSTANCE_SP` is pinned to a family and region | A Compute plan commits to dollars, not to a family; an empty spec is not missing data | + +--- + +## 14. Constraints — do not violate + +Restated from `SKILL.md` because they are the feature's contract, not +style preferences: + +- **Read-only.** Never `CreateSavingsPlan`, + `PurchaseReservedInstances*`, `StartCommitmentPurchaseAnalysis`, + `ModifyReservedInstances`, `ReturnSavingsPlan`, or any other mutating or + billable API. This feature *sizes* and *reports on* commitments; it never + buys, renews, modifies or cancels one. +- **Never present the AWS best case as achievable.** Quote the + risk-adjusted figure as the recommendation and the AWS figure as the + ceiling, in that order. +- **Never fabricate figures.** Every number traces to a documented call + or to arithmetic that is shown. A query you did not run is not a number + you have. +- **Lead with blockers.** Under-utilized existing commitments come before + the savings figure, always. A commitment expiring inside 30 days outranks + any new purchase — it is a deadline, not an option. +- **Never imply per-commitment utilization.** AWS publishes it account-wide + only. Attributing the account figure to one commitment is a fabricated + attribution, even though the number itself is real. +- **Never convert reservation unit counts to money.** That needs per-instance + pricing this feature does not query. +- **Never recommend a reservation without its specification.** Instance type, + deployment option (Multi-AZ vs Single-AZ), engine and Availability Zone are + what a reservation matches on, so "buy 4 RDS units" is not an actionable + recommendation. And never present a multi-specification total as a single + purchase — the total is a budget, the line items are the order. +- **Do not average Cost Explorer and COH when they disagree materially.** + Identify the cause instead. +- **No credential exfiltration.** Never emit access keys, session tokens, + or SSO refresh tokens. Account ID and profile name are fine. +- **Report, don't authorize.** Close every report by stating that + commitments are non-cancellable for their full term, that figures + should be verified in the console, and that the workload must not be + scheduled for migration or decommissioning within the term. diff --git a/scripts/lib/sync.sh b/scripts/lib/sync.sh index 803dd21..acfcaaf 100755 --- a/scripts/lib/sync.sh +++ b/scripts/lib/sync.sh @@ -552,12 +552,24 @@ def sanitize_schema(schema): with open('src/lambda/mcp/tools.json') as f: tools_config = json.load(f) -# Get existing targets +# Get existing targets. This has to page: the API returns a bounded page and +# boto3 does not follow nextToken on its own, so a single call silently drops +# the newest targets once the gateway holds more than one page of them — and a +# dropped target reads as 'not found', which fails the sync for a target that +# is actually present. existing = {} try: - resp = client.list_gateway_targets(gatewayIdentifier=gateway_id) - for t in resp.get('items', []): - existing[t['name']] = t['targetId'] + next_token = None + while True: + kwargs = {'gatewayIdentifier': gateway_id, 'maxResults': 100} + if next_token: + kwargs['nextToken'] = next_token + resp = client.list_gateway_targets(**kwargs) + for t in resp.get('items', []): + existing[t['name']] = t['targetId'] + next_token = resp.get('nextToken') + if not next_token: + break except Exception as e: print(f'Failed to list targets: {e}') sys.exit(1) diff --git a/skills/discounted-commitments/SKILL.md b/skills/discounted-commitments/SKILL.md new file mode 100644 index 0000000..c108966 --- /dev/null +++ b/skills/discounted-commitments/SKILL.md @@ -0,0 +1,332 @@ +--- +name: discounted-commitments +description: "AWS discounted commitment sizing and renewal — Savings Plans and Reserved Instances sized to what a workload can actually sustain, not the AWS best case, plus which existing commitments expire soon and what to renew. Drives the AWS CLI's Cost Explorer, Cost Optimization Hub and reservation-inventory read-only APIs, then risk-adjusts the result into a report. Use when the user asks what Savings Plans or RIs to buy, how much a commitment could save, whether a commitment is worth it, whether existing commitments are being wasted, what is expiring or needs renewing, or to size/validate an RI/SP purchase across compute, database, and analytics services." +argument-hint: "[what do you want to know? e.g. 'what savings plans should we buy', 'is a 3-year all-upfront worth it', 'are our existing RIs wasted', 'what expires in the next 90 days']" +user-invokable: true +--- + +# AWS Discounted Commitments + +Size achievable AWS commitment purchases — Savings Plans and Reserved Instances — and report what a workload can actually sustain rather than what the AWS API recommends in its best case. + +Needs only the **AWS CLI v2** and read-only credentials. Nothing to install, nothing bundled. + +The distinction this skill exists for: `GetSavingsPlansPurchaseRecommendation` assumes the lookback window repeats forever and recommends the commitment that maximizes savings under that assumption. On a spiky workload that means committing above the trough, which strands spend in quiet hours. The same response also carries the *minimum* and *average* hourly on-demand spend — so you can size to the quietest hour instead, and report what the over-commitment would have cost. + +## Routing — read first + +``` +Does your host expose this analysis as a tool +(a "commitment analysis" / "size savings plans" / "commitment expiry" tool +returning a report)? +├── Yes → Call it. Emit its report verbatim. Nothing below is needed. +└── No + ├── Can you run `aws` with read-only credentials? → Query it (below) + └── No + ├── Is a coding agent with a shell available? → Delegate (below) + └── No → Stop. Say sizing needs Cost Explorer access you do not have. +``` + +Generic cost tools are **not** a substitute. Cost Explorer and Cost Optimization Hub tool wrappers expose spend, coverage, and COH's own recommendations, but neither exposes `GetSavingsPlansPurchaseRecommendation` / `GetReservationPurchaseRecommendation`. Without those APIs nothing can size a commitment — you can still report eligible spend and existing coverage, and you must then say plainly: *"Sizing and risk adjustment are unavailable here — these figures are AWS best case, unadjusted."* + +Do **not** use this skill for rightsizing, idle-resource, or Graviton-migration findings — Cost Optimization Hub covers those. For general cost breakdowns, trends, forecasts, and anomalies, use a general FinOps skill. + +## Prerequisites + +- `aws --version` → 2.x. Confirm identity with `aws sts get-caller-identity`. +- Permissions: `ce:Get*`, `cost-optimization-hub:ListRecommendations`, `cost-optimization-hub:ListEnrollmentStatuses`, `sts:GetCallerIdentity`. Read-only suffices. +- For the expiry inventory (step 6) only, also: `savingsplans:DescribeSavingsPlans`, `ec2:DescribeReservedInstances`, `rds:DescribeReservedDBInstances`, `elasticache:DescribeReservedCacheNodes`, `redshift:DescribeReservedNodes`, `es:DescribeReservedInstances` (OpenSearch's IAM prefix is `es:`, not `opensearch:`), `memorydb:DescribeReservedNodes`. Sizing works without these — skip step 6 and say so. +- Cost Explorer enabled (default; ~24h to populate on a new account). +- Cost Optimization Hub enrollment is optional — without it, say the figures could not be reconciled. + +**Every call in steps 1–5 needs `--region us-east-1`.** Cost Explorer purchase recommendations and Cost Optimization Hub are us-east-1-only regardless of where the resources live. The one exception is the reservation inventory in step 6: reservations are regional and must be queried in the region they were bought in. Add `--profile ` to every command if the user named a profile. + +**Cost:** Cost Explorer bills **$0.01 per recommendation request**. Sweep narrow by default; widen only when asked. The step 3 posture calls and the step 6 `Describe*` calls are free — never quote them in a cost estimate. + +**Production caution:** use the least-privileged read-only profile available. Commitment purchases are non-cancellable financial obligations — this report is input to a decision, never authorization for one. + +## Query it + +Enums, verified against the CLI: `--savings-plans-type` is `COMPUTE_SP` | `EC2_INSTANCE_SP` | `SAGEMAKER_SP` | `DATABASE_SP`; `--term-in-years` is `ONE_YEAR` | `THREE_YEARS`; `--payment-option` is `NO_UPFRONT` | `PARTIAL_UPFRONT` | `ALL_UPFRONT`; `--lookback-period-in-days` is `SEVEN_DAYS` | `THIRTY_DAYS` | `SIXTY_DAYS`; `--account-scope` is `PAYER` (whole org) | `LINKED` (this account). + +### 1. Savings Plans sizing + +One call per (type, term, payment). `--query` trims the response to the eight fields the method needs — send it, or you will pull tens of KB of instance detail into context. + +```bash +aws ce get-savings-plans-purchase-recommendation --region us-east-1 \ + --savings-plans-type COMPUTE_SP --term-in-years ONE_YEAR \ + --payment-option NO_UPFRONT --lookback-period-in-days THIRTY_DAYS \ + --account-scope PAYER \ + --query 'SavingsPlansPurchaseRecommendation.{ + commit_hr: SavingsPlansPurchaseRecommendationSummary.HourlyCommitmentToPurchase, + savings_mo: SavingsPlansPurchaseRecommendationSummary.EstimatedMonthlySavingsAmount, + savings_pct: SavingsPlansPurchaseRecommendationSummary.EstimatedSavingsPercentage, + ondemand_mo: SavingsPlansPurchaseRecommendationSummary.CurrentOnDemandSpend, + detail: SavingsPlansPurchaseRecommendationDetails[].{ + floor_hr: CurrentMinimumHourlyOnDemandSpend, + avg_hr: CurrentAverageHourlyOnDemandSpend, + upfront: UpfrontCost, est_util: EstimatedAverageUtilization, + commit_hr: HourlyCommitmentToPurchase, + spec: SavingsPlansDetails}}' +``` + +`SavingsPlansDetails` carries `InstanceFamily` and `Region` for an +`EC2_INSTANCE_SP` — the two things that plan is pinned to, and therefore the two +things a purchase has to name. For a `COMPUTE_SP` it comes back empty, which is +the plan being flexible by design, not data going missing: report it as *any +instance family*. + +Sweep several permutations in **one** Bash call rather than one call each — same billing, far fewer round-trips: + +```bash +for term in ONE_YEAR THREE_YEARS; do for pay in NO_UPFRONT ALL_UPFRONT; do + echo "== COMPUTE_SP $term $pay" + aws ce get-savings-plans-purchase-recommendation --region us-east-1 \ + --savings-plans-type COMPUTE_SP --term-in-years "$term" --payment-option "$pay" \ + --lookback-period-in-days THIRTY_DAYS --account-scope PAYER \ + --query 'SavingsPlansPurchaseRecommendation.SavingsPlansPurchaseRecommendationSummary.[HourlyCommitmentToPurchase,EstimatedMonthlySavingsAmount,EstimatedSavingsPercentage]' \ + --output text +done; done +``` + +**Default scope: `COMPUTE_SP` across both terms × `NO_UPFRONT`/`ALL_UPFRONT` — 4 calls, $0.04.** Compute SPs are the flexible instrument that fits most accounts. Add `EC2_INSTANCE_SP` when the fleet is stable and single-family, `SAGEMAKER_SP`/`DATABASE_SP` only when that spend exists. Add `PARTIAL_UPFRONT` only on request — it rarely wins and doubles the sweep. + +### 2. Reserved Instance sizing + +`--service` takes exactly these 8 values, no others: + +`Amazon Elastic Compute Cloud - Compute`, `Amazon Relational Database Service`, `Amazon Redshift`, `Amazon ElastiCache`, `Amazon Elasticsearch Service`, `Amazon OpenSearch Service`, `Amazon MemoryDB Service`, `Amazon DynamoDB Service` + +```bash +aws ce get-reservation-purchase-recommendation --region us-east-1 \ + --service "Amazon Relational Database Service" \ + --term-in-years ONE_YEAR --payment-option NO_UPFRONT \ + --lookback-period-in-days THIRTY_DAYS --account-scope PAYER \ + --query 'Recommendations[0].{ + savings_mo: RecommendationSummary.TotalEstimatedMonthlySavingsAmount, + savings_pct: RecommendationSummary.TotalEstimatedMonthlySavingsPercentage, + detail: RecommendationDetails[].{ + buy_units: RecommendedNumberOfInstancesToPurchase, + floor_units: MinimumNumberOfInstancesUsedPerHour, + avg_units: AverageNumberOfInstancesUsedPerHour, + upfront: UpfrontCost, monthly: RecurringStandardMonthlyCost, + break_even_mo: EstimatedBreakEvenInMonths, util: AverageUtilization, + savings_mo: EstimatedMonthlySavingsAmount, + spec: InstanceDetails, capacity: ReservedCapacityDetails}}' +``` + +**`spec` is not optional.** `RecommendationSummary` gives one total across every +line item, and a reservation only discounts usage matching its *exact* +specification — so "buy 4 RDS reservations" is a budget, not an order. Report one +row per `RecommendationDetails` entry, each naming what to buy, and say the total +spans several specifications when it does. + +`InstanceDetails` holds exactly one service-specific sub-object, and each names +its fields differently: + +| Service | Sub-object | Read | +|---|---|---| +| EC2 | `EC2InstanceDetails` | `InstanceType`, `AvailabilityZone`, `Platform`, `Tenancy` | +| RDS / Aurora | `RDSInstanceDetails` | `InstanceType`, **`DeploymentOption`** (`Multi-AZ` / `Single-AZ`), `DatabaseEngine`, `DatabaseEdition`, `LicenseModel` | +| ElastiCache | `ElastiCacheInstanceDetails` | `NodeType`, `ProductDescription` | +| Redshift | `RedshiftInstanceDetails` | `NodeType` | +| MemoryDB | `MemoryDBInstanceDetails` | `NodeType` | +| OpenSearch / Elasticsearch | `ESInstanceDetails` | `InstanceClass` **+** `InstanceSize` — there is no `InstanceType` field; join them | +| DynamoDB | `DynamoDBCapacityDetails`, under `ReservedCapacityDetails` (**not** `InstanceDetails`) | `CapacityUnits`, `Region` | + +Every sub-object except `ESInstanceDetails` and `DynamoDBCapacityDetails` also +carries `Family`, `Region`, `CurrentGeneration` and `SizeFlexEligible`. Report the +last two per line: `SizeFlexEligible: true` means the recommended size is not +binding (the discount follows any size in the family), and +`CurrentGeneration: false` means a three-year commitment locks the account out of +the cheaper current generation for the whole term. + +For **EC2 only**, add `--service-specification OfferingClass=STANDARD`; RDS, Redshift and the rest reject it. For **DynamoDB**, the unit fields are named `RecommendedNumberOfCapacityUnitsToPurchase` / `MinimumNumberOfCapacityUnitsUsedPerHour` / `AverageNumberOfCapacityUnitsUsedPerHour` instead. Query only services the account actually uses — check step 4 first. + +### 3. Existing commitment posture — run this before quoting any savings + +Four calls, not billed as recommendations. Substitute real dates (30 days back → today): + +```bash +aws ce get-savings-plans-coverage --region us-east-1 \ + --time-period Start=2026-08-04,End=2026-09-03 --granularity MONTHLY \ + --query 'SavingsPlansCoverages[-1].Coverage.[CoveragePercentage,OnDemandCost]' --output text + +aws ce get-savings-plans-utilization --region us-east-1 \ + --time-period Start=2026-08-04,End=2026-09-03 --granularity MONTHLY \ + --query 'Total.Utilization.[UtilizationPercentage,UnusedCommitment]' --output text + +aws ce get-reservation-coverage --region us-east-1 \ + --time-period Start=2026-08-04,End=2026-09-03 --granularity MONTHLY \ + --query 'Total.CoverageHours.[CoverageHoursPercentage,OnDemandHours]' --output text + +aws ce get-reservation-utilization --region us-east-1 \ + --time-period Start=2026-08-04,End=2026-09-03 --granularity MONTHLY \ + --query 'Total.[UtilizationPercentage,UnusedHours,RealizedSavings]' --output text +``` + +All-zero results mean **no commitment exists**, not 0% utilization on a broken one. Say which it is. + +### 4. Eligible spend — what is even commitable + +```bash +aws ce get-cost-and-usage --region us-east-1 \ + --time-period Start=2026-07-05,End=2026-09-03 --granularity MONTHLY \ + --metrics UnblendedCost --group-by Type=DIMENSION,Key=SERVICE \ + --query 'ResultsByTime[].Groups[?to_number(Metrics.UnblendedCost.Amount)>`1000`].[Keys[0],Metrics.UnblendedCost.Amount]' \ + --output text +``` + +`to_number()` is required, not decoration: `Amount` is a JSON string, and comparing a string to a number raises `TypeError: '>' not supported` instead of filtering. + +Run this first when the user has no specific target: an account whose bill is serverless, storage and support has nothing to commit against, and you can say so for $0. + +### 5. Reconciliation (optional second opinion) + +```bash +aws cost-optimization-hub list-enrollment-statuses --region us-east-1 \ + --query 'items[0].status' --output text + +aws cost-optimization-hub list-recommendations --region us-east-1 \ + --filter '{"actionTypes":["PurchaseSavingsPlans","PurchaseReservedInstances"]}' \ + --query 'items[].[currentResourceType,estimatedMonthlySavings]' --output text +``` + +`NOT_ENROLLED` or an empty list is fine — report that reconciliation was unavailable. + +### 6. Expiry inventory — what is already committed, and when it lapses + +Run this whenever the user asks what is expiring or what to renew, and before recommending any purchase: a commitment that lapses next month returns that spend to on-demand rates, and expiry is the one moment resizing costs nothing. Free — these are `Describe*` calls, not billed recommendation requests. + +**Savings Plans are account-level and live on the global us-east-1 endpoint. Reservations are regional and only appear in the region they were bought in** — a us-east-1 sweep finds nothing for an ap-northeast-1 fleet. Query each region the account actually runs in, and say which regions you swept. + +Savings Plans — one call, `end` comes back directly: + +```bash +aws savingsplans describe-savings-plans --region us-east-1 \ + --states active payment-pending \ + --query 'savingsPlans[].[savingsPlanId,savingsPlanType,commitment,end,state,paymentOption,ec2InstanceFamily,region]' \ + --output text +``` + +`ec2InstanceFamily` and `region` are populated only for an `EC2Instance` plan; +blank on a `Compute` plan is correct. + +Reservations — one call per (service, region). Only EC2 returns an end date; the rest return `StartTime` plus `Duration` in **seconds**, so derive `end = StartTime + Duration` yourself (31536000s = 1 year, 94608000s = 3 years): + +```bash +REGION=ap-northeast-1 + +aws ec2 describe-reserved-instances --region $REGION \ + --query 'ReservedInstances[?State==`active`].[ReservedInstancesId,InstanceType,InstanceCount,End,OfferingType,Scope,AvailabilityZone,ProductDescription,OfferingClass,InstanceTenancy]' \ + --output text + +aws rds describe-reserved-db-instances --region $REGION \ + --query 'ReservedDBInstances[?State==`active`].[ReservedDBInstanceId,DBInstanceClass,DBInstanceCount,StartTime,Duration,MultiAZ,ProductDescription]' \ + --output text + +aws elasticache describe-reserved-cache-nodes --region $REGION \ + --query 'ReservedCacheNodes[?State==`active`].[ReservedCacheNodeId,CacheNodeType,CacheNodeCount,StartTime,Duration,ProductDescription]' \ + --output text + +aws redshift describe-reserved-nodes --region $REGION \ + --query 'ReservedNodes[?State==`active`].[ReservedNodeId,NodeType,NodeCount,StartTime,Duration,ReservedNodeOfferingType]' \ + --output text + +aws opensearch describe-reserved-instances --region $REGION \ + --query 'ReservedInstances[?State==`active`].[ReservedInstanceId,InstanceType,InstanceCount,StartTime,Duration,PaymentOption]' \ + --output text + +aws memorydb describe-reserved-nodes --region $REGION \ + --query 'ReservedNodes[?State==`active`].[ReservationId,NodeType,NodeCount,StartTime,Duration]' \ + --output text +``` + +The id, count and type fields are named differently in every one of these — copy them as written rather than reusing EC2's. A typo yields rows with blank ids and no expiry rather than an error. + +The trailing fields are what a **renewal has to match**, and they are the whole +point of listing them: an RDS reservation covers one deployment option and one +engine, and an EC2 reservation is pinned to a single Availability Zone when +`Scope` is `Availability Zone`. Renew against a different value and the discount +silently does not apply. `MultiAZ` comes back as a JSON **boolean**: `true` is +Multi-AZ, `false` is Single-AZ, and an absent field means AWS did not report it — +do not read absence as Single-AZ. OpenSearch and MemoryDB reservations expose no +engine or product field at all, so their spec is the node type alone. + +**DynamoDB reserved capacity cannot be inventoried at all** — no describe API exists for it, in any SDK. If the account uses it, say so explicitly rather than reporting "nothing expiring". + +Then read the renewal rules in [`reference/method.md`](reference/method.md) to turn each expiring commitment into renew / renew-smaller / let-lapse. Utilization is published only account-wide, never per commitment, so every verdict is directional — state that once, in the section, and do not imply otherwise per row. + +## Then risk-adjust + +Read [`reference/method.md`](reference/method.md) and apply it. **It is required, not optional** — the AWS numbers are unadjusted best case, and emitting them as achievable is the exact error this skill exists to prevent. It gives you the volatility bands, the commitment formula, break-even and waste-exposure arithmetic, the posture gate, and the reconciliation thresholds. + +Then format per [`reference/output-template.md`](reference/output-template.md). + +Show your arithmetic for each finding — the trough÷average ratio, the band it lands in, and the resulting commitment. A reviewer must be able to recheck a figure without re-querying AWS. + +## Delegate + +Hand the coding agent this brief: + +> "Use the `discounted-commitments` skill to size AWS commitments for {{ACCOUNT_OR_PROFILE}} and report what expires within {{HORIZON_DAYS|90}} days. Read its `SKILL.md` and `reference/method.md`, run the Cost Explorer queries with `--region us-east-1` and read-only credentials, and run the step 6 reservation inventory in {{REGIONS}}. Return the finished markdown report. Read-only: no purchases, no renewals, no `StartCommitmentPurchaseAnalysis`. Report existing-commitment blockers and expiring commitments before any savings figure, name the regions you swept, and list any query that failed." + +Interpret what they return; if it carries an error (no credentials, `AccessDenied`, expired SSO), pass it through with the fix from the Failure modes table rather than retrying in your own sandbox. + +## Constraints — do not violate + +- **Read-only.** Only `ce:Get*`, `cost-optimization-hub:List*`, the `Describe*` reads in step 6, and `sts:GetCallerIdentity`. Never `CreateSavingsPlan`, `PurchaseReservedInstances*`, `StartCommitmentPurchaseAnalysis`, `ModifyReservedInstances`, `DeleteQueuedSavingsPlan`, `ReturnSavingsPlan`, or any other mutating or billable API. This skill *sizes* and *reports on* commitments; it never buys, renews, modifies or cancels one. +- **Never present the AWS best case as achievable.** Quote the risk-adjusted figure as the recommendation and the AWS figure as the ceiling, in that order. +- **Never fabricate figures.** Every number traces to a command above or to arithmetic you show. A query you did not run is not a number you have. +- **Lead with blockers.** Under-utilized existing commitments come before the savings figure, always. A commitment expiring inside 30 days outranks any new purchase — it is a deadline, not an option. +- **Never recommend a reservation without its specification.** Instance type, and the deployment option for RDS/Aurora (Multi-AZ vs Single-AZ) plus the Availability Zone for a zonal EC2 reservation, decide whether the discount applies at all. A unit count with no spec is not a recommendation anyone can act on — and never present a total that spans several specifications as a single purchase. +- **Never imply per-commitment utilization.** AWS publishes it account-wide only. Quoting one commitment as "98% utilized" when that is the account figure is a fabricated attribution. +- **Do not average Cost Explorer and Cost Optimization Hub when they disagree materially.** Identify the cause. A blended number is defensible to nobody. +- **Region is pinned** to us-east-1 for every Cost Explorer, Cost Optimization Hub and Savings Plans command. Only the step 6 reservation inventory varies by region, and it must name the regions swept. +- **No credential exfiltration.** Never emit access keys, session tokens, or SSO refresh tokens. Account ID and profile name are fine. +- **Report, don't authorize.** Close with: commitments are non-cancellable, verify in the console, and confirm the workload is not scheduled for migration or decommissioning within the term. + +## Failure modes — handle explicitly + +| Situation | Behavior | +|-----------|----------| +| `aws sts get-caller-identity` fails | Stop. Tell the user to authenticate (`aws sso login` or set credentials). Produce no report. | +| `aws` missing or 1.x | Stop. `aws --version` must show 2.x; v1 lacks `cost-optimization-hub` entirely. | +| `DataUnavailableException`, blank message | Cost Explorer returns it with an empty `Message` when the account holds no commitment of that kind. Report "no existing commitment of this type", not a blank error. | +| `ValidationException` on `--service` | The name is not one of the 8 above. Do not guess — copy the API's own "Supported value(s)" list from the error. | +| COH not enrolled | Continue. State that figures rest on Cost Explorer alone. | +| Throttling / `AccessDenied` on some queries | Continue with what succeeded. List every failed query and present totals explicitly as a lower bound. | +| A `$0` / no-recommendation result | A real result. Check eligible spend (step 4) and say why there is nothing to commit against. | +| Posture queries all fail | Say posture could not be measured and that recommendations are unvalidated against existing commitments. | +| Member account, `PAYER` scope | The figure is org-aggregated and not purchasable by that account. Re-run with `--account-scope LINKED`. | +| Expiry inventory returns nothing | Distinguish "no commitment exists" from "wrong region" — reservations only appear in the region they were bought in. Name the regions you swept before concluding nothing expires. | +| `AccessDenied` on a `Describe*` | Report the expiry section as partial, naming the families you could not read. Do not report the surviving families as the complete picture. | +| An active commitment whose end date has already passed | Real and worth flagging on its own: that spend is already back at on-demand rates. Do not fold it in with future expiries. | + +## Common mistakes + +- **The 4th Savings Plan type is `DATABASE_SP`.** Not `MACHINE_LEARNING_SP` — that value does not exist and the call fails. +- **Do not guess RI service names.** They are inconsistent: `Amazon MemoryDB Service` and `Amazon DynamoDB Service` carry a ` Service` suffix that `Amazon Redshift` does not. SageMaker, Lambda, Fargate, Aurora, CloudFront, S3, Neptune, DocumentDB, MSK, Kinesis and Timestream are all rejected. +- **Cost Explorer returns money and percentages as JSON strings**, and `""` for absent values. Treat empty string as "no data", not zero. +- **`OfferingClass` is EC2-only.** Sending it elsewhere is a `ValidationException`. +- **`PAYER` aggregates the organization.** A member account cannot purchase a payer-scoped recommendation. +- **Don't drop `--query`.** A bare recommendation response runs to tens of KB of per-instance detail. +- **Don't re-query for a number you already have.** Each recommendation request costs $0.01; note results as you go. +- **Only EC2 reservations return an end date.** Everywhere else it is `StartTime` + `Duration` seconds. Reporting a blank expiry because you looked for `End` is how a lapsing commitment gets missed. +- **`RecommendationSummary` is a sum, not a purchase.** One RDS recommendation routinely spans `db.r6g.large Multi-AZ` and `db.t4g.medium Single-AZ`. Reporting only the total hides which to buy, and the two are not interchangeable. +- **OpenSearch has no `InstanceType` in its recommendation detail** — `ESInstanceDetails` splits it across `InstanceClass` and `InstanceSize`. Looking for `InstanceType` there returns nothing and reads as "no spec available". +- **DynamoDB's spec is not under `InstanceDetails`** — it is `ReservedCapacityDetails.DynamoDBCapacityDetails`, and it has capacity units rather than an instance. +- **A Savings Plan's `commitment` is dollars per hour, not a unit count.** Multiply by 730 for a monthly figure; reading `5.5` as "5.5 units" understates the exposure by roughly 1000x. Reservation counts are the opposite — they are units, and converting them to money needs per-instance pricing you have not queried. + +## Layout + +``` +discounted-commitments/ +├── SKILL.md # this file — routing, queries, constraints +└── reference/ + ├── method.md # required: risk adjustment + guards + └── output-template.md # report structure and field names +``` + +Three markdown files, no code and no dependencies beyond the AWS CLI. Copy the directory anywhere — into another repo, another agent's skill folder — and it works unchanged. diff --git a/skills/discounted-commitments/reference/method.md b/skills/discounted-commitments/reference/method.md new file mode 100644 index 0000000..febcfe6 --- /dev/null +++ b/skills/discounted-commitments/reference/method.md @@ -0,0 +1,254 @@ +# How the risk adjustment works + +Apply this to every recommendation before reporting it. The AWS APIs return a +best case; these steps turn it into a figure a workload can actually sustain, +and into the two warnings the raw API will never give you. + +Every input is a field from the responses in `SKILL.md`. Nothing here is +estimated, and no step needs data you did not query. + +**Reading rules for Cost Explorer values.** Money and percentages arrive as JSON +*strings*, and absent values arrive as `""` — treat `""` and `null` as "no +data", never as `0`. A `""` floor is not a floor of zero; it means the floor +could not be measured, which is the `unknown` band below. + +## Savings Plans + +Constants: **730 hours per month**. Term length: `ONE_YEAR` = 12 months, +`THREE_YEARS` = 36 months. + +**1. Measure the hourly envelope.** Sum across all entries of `detail[]`: + +``` +floor_hr = Σ CurrentMinimumHourlyOnDemandSpend +avg_hr = Σ CurrentAverageHourlyOnDemandSpend +``` + +If `avg_hr` is 0 or absent, fall back to `avg_hr = ondemand_mo ÷ 730`. If +`commit_hr` and `savings_mo` are both 0, there is no recommendation — drop it. + +**2. Classify the workload** on `ratio = floor_hr ÷ avg_hr`: + +| ratio | Profile | Commitment to report | Confidence | +|---|---|---|---| +| ≥ 0.80 | stable | `commit_hr` — take the AWS figure as-is | High | +| ≥ 0.50 | moderate | `floor_hr + (commit_hr − floor_hr) × 0.5` | Medium | +| < 0.50 | spiky | `min(commit_hr, floor_hr)` — clamp to the floor | Low | +| `avg_hr` ≤ 0 | unknown | `commit_hr`, marked explicitly unvalidated | Low | + +The moderate case also caps at `commit_hr` — never report a commitment above +what AWS recommended. + +**3. Scale the savings.** The discount rate is fixed per plan, so savings move +linearly with commitment size: + +``` +scale = safe_commit_hr ÷ commit_hr +safe_mo = savings_mo × scale +``` + +Report `safe_mo` as the recommendation and `savings_mo` as the AWS ceiling. + +**4. Waste exposure** — what committing at the AWS figure costs per month if +usage sits at the trough. Only meaningful when a floor was measured: + +``` +waste_mo = max(0, commit_hr − floor_hr) × 730 [only if floor_hr > 0] +``` + +**5. Break-even**, when `upfront = Σ UpfrontCost` is above 0: + +``` +break_even_months = upfront ÷ safe_mo +``` + +Use the **adjusted** `safe_mo`, not `savings_mo` — paying back against savings +you will not achieve is the error being guarded against. With no upfront cost, +break-even is **null**, not 0: reporting 0 reads as "pays back instantly". + +## Reserved Instances + +Identical shape, in whole units instead of dollars per hour. + +``` +recommended = Σ RecommendedNumberOfInstancesToPurchase +floor_units = Σ MinimumNumberOfInstancesUsedPerHour +avg_units = Σ AverageNumberOfInstancesUsedPerHour +``` + +If `recommended` comes out 0, re-read using the capacity-unit field names +(`RecommendedNumberOfCapacityUnitsToPurchase`, +`MinimumNumberOfCapacityUnitsUsedPerHour`, +`AverageNumberOfCapacityUnitsUsedPerHour`) — that is how DynamoDB reports. + +Same bands on `floor_units ÷ avg_units`, then **round the result down to a whole +number**. Rounding up would land the commitment above the level just judged +safe. Savings scale the same way: `safe_mo = savings_mo × (safe_units ÷ recommended)`. + +Two RI-specific differences: + +- **Break-even comes from the API**, not from your division: average the + positive `break_even_mo` values across `detail[]`. This keeps the figure tied + to what the console shows. +- **Waste is a unit count**, so convert to money: + `waste_mo = (Σ monthly ÷ recommended) × (recommended − floor_units)`. + +### Then break the total back down — the sum is not purchasable + +The bands above are applied to the family total so it can be ranked against +other services. But a reservation discounts only usage matching its **exact** +specification, so a total spanning `db.r6g.large Multi-AZ` and +`db.t4g.medium Single-AZ` is a budget, not an order. Apply the same `scale` to +each `detail[]` entry and report one line per specification: + +``` +line_safe_units = floor(line_recommended × scale) [whole reservations] +``` + +Each line carries its own `MinimumNumberOfInstancesUsedPerHour` — the count that +line never dropped below, and therefore the part of it that carries no +unused-commitment risk — plus its own `EstimatedMonthlySavingsAmount` and +`AverageUtilization`. Rank the lines by savings so the largest decision is first. + +Flooring each line can leave the lines summing **below** the family total. Say by +how much and which line should absorb the remainder (the one with the highest +floor); do not pad a line silently, and do not restate the headline figure to +match — the family total is what AWS costed. + +Read the spec fields per the table in `SKILL.md` step 2. A line whose +sub-structure is missing entirely is reported as *specification not returned* and +treated as unpurchasable until confirmed in the console — not as a spec of +nothing. + +For Savings Plans the same breakdown applies without the rounding: dollar +commitments are divisible, and only an `EC2_INSTANCE_SP` line has a family and +region to name at all. + +## Two guards that override the numbers + +- **Break-even beyond the term.** If `break_even_months` exceeds the term (12 or + 36), the purchase expires before it pays back. Force confidence to **Low** and + lead the rationale with **"Do not buy"** — recommend the no-upfront option, a + shorter term, or nothing. This is the one case where the raw AWS API will + endorse a purchase that loses money outright, so check it every time. +- **Posture gate.** From the step-3 posture queries: existing utilization below + **95%**, or coverage above **90%**, is a blocker. Report blockers *before* any + savings figure. Buying on top of an under-used commitment compounds the waste + rather than reducing it. + +## Renewals — deciding what to do with an expiring commitment + +This is a different question from sizing a new purchase, and the difference is +the whole point: **a lapsing commitment has zero switching cost.** Expiry is the +one moment the size can change without buying out of anything, so the bands here +are allowed to be less conservative than a net-new buy — and equally, "let it +lapse" is a real answer rather than a failure. + +**1. Derive the end date.** Only EC2 returns `End`. Everywhere else: + +``` +end = StartTime + Duration seconds (31536000 = 1 year, 94608000 = 3 years) +``` + +An unparseable or absent date makes the commitment **undated** — list it as +needing manual checking. Do not silently drop it, and do not guess. + +**2. Bucket by `days_remaining = end − today`:** + +| days_remaining | Bucket | Why it matters | +|---|---|---| +| < 0 | **expired** | Already back at on-demand rates. Report separately and first — this is a live cost, not a deadline. | +| ≤ 30 | urgent | Too close to run a full sizing exercise before it lapses. | +| ≤ 60 | soon | Enough time to size a replacement properly. | +| ≤ horizon (default 90) | upcoming | Note it; no action this month. | +| > horizon | not listed | Counted in the total, kept out of the table. | + +**3. Verdict, from utilization:** + +| Utilization | Action | Reasoning to report | +|---|---|---| +| ≥ 95% | **renew** | The commitment is being consumed; letting it lapse returns that spend to on-demand rates. | +| 50–95% | **renew smaller** | Partly wasted. Renew at roughly the consumed portion — expiry is a zero-cost resize point, which mid-term it never is. | +| < 50% | **let lapse** | More than half the commitment is unused. Re-size from current usage instead of renewing the mistake. | +| not measured | **review** | State that plainly. A verdict with no utilization behind it is a guess wearing a recommendation's clothes. | + +Use the **Savings Plans** utilization figure for Savings Plans and the +**reservation** figure for reservations — they are separate metrics and crossing +them produces a confident wrong answer. + +**4. Quantify the exposure, in the right unit.** + +``` +hourly_expiring = Σ commitment (Savings Plans only, USD/hour) +monthly_expiring = hourly_expiring × 730 +units_expiring = Σ instance/node count (reservations only) +``` + +Do **not** convert reservation unit counts to dollars. That needs per-instance +pricing this method never queries, and an invented rate is a fabricated figure. +Report units as units. + +**5. Carry the specification into the verdict.** "Renew" means renew *the same +thing*: the same instance class, the same deployment option (Multi-AZ vs +Single-AZ), the same engine, and the same Availability Zone when the reservation +is zonal. A renewal that changes any of those is a new purchase and needs the +sizing method above, not a renewal verdict. So state the spec on every row — +`db.r6g.large · Multi-AZ · postgresql`, not "2 RDS units" — and read `MultiAZ` as +the boolean it is: `false` means Single-AZ, absent means AWS did not say. + +**6. State the limitation once.** AWS publishes SP and RI utilization at the +**account level only** — there is no per-commitment utilization API. So every +row in a family shares one figure, and the verdicts are directional. Say this in +the section; never attribute the account figure to an individual commitment. + +**Precedence.** An urgent expiry outranks every new-purchase recommendation in +the report. A renewal deadline is fixed; a purchase is optional. + +## Reconciliation + +Compare the **unadjusted** Cost Explorer total (Σ `savings_mo`) against the +Cost Optimization Hub total (Σ `estimatedMonthlySavings`). Like-for-like: COH +also publishes a best case, so comparing your adjusted figure would manufacture +a variance that is really just this method's own adjustment. + +``` +delta_pct = |ce_total − coh_total| ÷ max(|ce_total|, |coh_total|) × 100 +``` + +| delta_pct | Verdict | +|---|---| +| both totals 0 | agree-zero | +| ≤ 10% | reconciled | +| ≤ 30% | minor variance | +| > 30% | material variance | + +A material variance is **flagged, never averaged away**. Usual causes: a +differing account scope (`PAYER` vs `LINKED`), or COH lagging a recent usage +change. + +## Choosing what to report + +Every (type, term, payment) permutation is a separate query and the winner is +not knowable in advance — a 3-year all-upfront plan can lose to a 1-year +no-upfront one once break-even is accounted for. Keep **one finding per +(family, label)**: highest `safe_mo` wins; break ties toward the **shorter term**, +then the **less upfront cash**, since that is the lower-risk purchase. Sort the +report by `safe_mo` descending. + +## Worked example + +`commit_hr = 10.00`, `savings_mo = 1000.00`, `floor_hr = 2.00`, +`avg_hr = 10.00`, `upfront = 0`: + +``` +ratio = 2.00 ÷ 10.00 = 0.20 → below 0.50 → spiky, Low confidence +safe = min(10.00, 2.00) = $2.00/hr +scale = 2.00 ÷ 10.00 = 0.20 +safe_mo = 1000.00 × 0.20 = $200.00/mo ← the recommendation +waste_mo = (10.00 − 2.00) × 730 = $5,840/mo stranded if committed at the AWS figure +break_even = null (no upfront) +``` + +Reported as: **$200/mo achievable** (AWS ceiling $1,000/mo), Low confidence, +because the quietest hour runs at 20% of average and committing to the AWS +figure would strand $5,840/mo in quiet hours. diff --git a/skills/discounted-commitments/reference/output-template.md b/skills/discounted-commitments/reference/output-template.md new file mode 100644 index 0000000..7c40ba9 --- /dev/null +++ b/skills/discounted-commitments/reference/output-template.md @@ -0,0 +1,252 @@ +# Output template + +Emit this structure. A host tool that returns its own rendered report already +follows it — pass that through verbatim rather than reformatting. + +Order is deliberate: a reader cannot reach the savings number without passing +the blockers. Preserve it. Emit only the sections you have data for, and name +the ones you dropped. + +```markdown +# AWS Discounted Commitments Report + +**Account:** [account ID] +**Profile:** `[profile]` +**Generated:** [timestamp] +**Lookback:** [7/30/60 days] | **Account scope:** [PAYER/LINKED] +**Source APIs:** Cost Explorer (purchase recommendations, coverage + utilization), +Cost Optimization Hub (ListRecommendations), Savings Plans and per-service +reservation inventory (`Describe*`) + +All data is read-only. This report does not purchase anything. + +## Bottom line + +| Measure | Monthly | Annual | +|---|---:|---:| +| AWS best-case savings (as the console shows) | $[X,XXX.XX] | $[XX,XXX.XX] | +| **Risk-adjusted achievable savings** | **$[X,XXX.XX]** | **$[XX,XXX.XX]** | +| High-confidence subset only | $[X,XXX.XX] | $[XX,XXX.XX] | + +The risk-adjusted figure is [X]% below the AWS best case. [+ "Do not act on +these numbers yet" when blockers exist] + +[**[n] existing commitment(s) expire within 30 days.** That deadline comes before +any new purchase — see *Commitment expiry and renewal*. — present only when the +urgent count is above zero] + +_(or, with no findings: "**No commitment opportunity found.**" — a real result, +not a failure; Eligible spend below shows why)_ + +## Reconciliation against AWS native tools + +[verdict: reconciled / minor variance / MATERIAL VARIANCE / not reconciled] + +| Pipeline | Recommended monthly savings | +|---|---:| +| Cost Explorer purchase recommendations | $[X,XXX] | +| Cost Optimization Hub ([n] commitment recs) | $[X,XXX] | +| Delta | $[X,XXX] ([X]%) | + +## Existing commitment health + +### Blockers +- **[blocker — e.g. Savings Plans utilization 82%, below the 95% floor]** + +| Metric | Value | +|---|---:| +| Savings Plans coverage / utilization | [X]% / [X]% | +| Unused SP commitment | $[X] | +| Reservation coverage / utilization | [X]% / [X]% | +| Unused reservation hours | [n] | + +_(or prose when no commitments exist — all-zero metrics mean nothing to +measure, not a wasted commitment)_ + +## Commitment expiry and renewal + +Inventory taken [date] over a [90]-day horizon. Reservation regions swept: +[regions]. Savings Plans are account-level and are listed once regardless of +region. + +**[n] commitment(s) expire within [90] days** — [n] within 30 days, [n] within +60, [n] within 90. + +Exposure: $[X,XXX]/mo of committed Savings Plan spend ($[X.XXXX]/hr) reverts to +on-demand rates if not renewed; [n] reserved unit(s) lose their discount. The +dollar value of that is not stated because it needs per-instance pricing this +analysis does not query. + +| Ends | Days | Commitment | Spec | Size | Region | Utilization | Action | +|---|---:|---|---|---:|---|---:|---| +| [YYYY-MM-DD] | [n] | [label] `[id]` | [m5 or —] | $[X.XXXX]/hr | [region] | [X]% | **renew** | +| [YYYY-MM-DD] | [n] | RDS `[id]` | db.r6g.large · Multi-AZ · postgresql | [n] unit(s) | [region] | [X]% | let lapse | + +> Utilization is the account-level figure from Cost Explorer, not +> per-commitment — there is no API that reports utilization for an individual +> Savings Plan or reservation. Treat it as the portfolio signal it is, and +> confirm a specific commitment in the console before acting. + +> *Spec* is what a renewal has to match. A reservation bought against a +> different instance class, deployment option (Single-AZ vs Multi-AZ) or engine +> does not cover the same usage, so a renewal that changes any of these is a new +> purchase and needs fresh sizing. An empty spec on a Compute Savings Plan is +> correct — it commits to dollars, not to a family. + +### Why + +- `[id]` — [rationale citing the utilization figure it relied on] + +### Already ended but still listed as active + +- `[id]` ([label] [spec]) — ended [n] days ago; that spend is already at on-demand rates + +[n] commitment(s) returned no usable end date and could not be assessed: `[id]`. + +Not covered by this inventory: DynamoDB reserved capacity (no describe API +exists). + +_(or, when nothing is expiring: "**No commitment expires within [90] days.** [n] +active commitment(s) were inventoried." — omit the table entirely. Omit the whole +section when the expiry inventory was not run, and say in Method that it was +skipped.)_ + +## Recommended commitments + +| # | Commitment | Term | Payment | AWS commitment | Achievable commitment | Achievable savings/mo | Discount | Confidence | +|---|---|---|---|---:|---:|---:|---:|---| +| 1 | [label] | 1-year | No upfront | $[X.XXXX]/hr | $[X.XXXX]/hr | $[X,XXX] | [X]% | High | + +### Detail + +#### 1. [label] — [term], [payment] + +- **Confidence:** [High/Medium/Low] (spend profile: [stable/moderate/spiky]) +- **Commitment:** AWS recommends [X]; this report recommends [Y] +- **Savings:** $[X]/mo achievable ($[Y]/mo at the AWS best case) +- **Upfront cost:** $[X] +- **Break-even:** [X] months [+ "longer than the [n]-month term, so this + purchase cannot pay back"] +- **Waste exposure at the AWS figure:** up to $[X]/mo unused if usage falls to + its observed floor +- **Line items analyzed:** [n] + +Line items — what to buy: + +| Buy | Region | AWS units | Floor | Achievable | Utilization | Savings/mo | +|---|---|---:|---:|---:|---:|---:| +| db.r6g.large · Multi-AZ · Aurora PostgreSQL (size-flexible) | [region] | 4 | 3 | 3 | [X]% | $[X,XXX] | +| db.t4g.medium · Single-AZ · PostgreSQL (**previous generation**) | [region] | 2 | 2 | 2 | [X]% | $[XXX] | + +[Rounding each line down to whole reservations leaves [n] unit(s) unallocated +against the [n] achievable total — add them to the line with the highest floor. +— reservations only] + +*Floor* is the count that line never dropped below during the lookback, so it is +the part of the recommendation that carries no unused-commitment risk. + +_(Omit this table when no line item has anything to buy. A Savings Plan line +shows `any instance family` unless it is an EC2 Instance plan, which is pinned to +one family and region. `(specification not returned)` means AWS gave a count +without the sub-structure that names the instance — treat that line as +unpurchasable until confirmed in the console.)_ + +Why: + +- [rationale lines] + +## Eligible spend + +| Period | Total unblended spend | +|---|---:| +| [start] → [end] | $[X,XXX] | + +Top services, [start] → [end]: + +| Service | Spend | +|---|---:| +| [service] | $[X,XXX] | + +## Method + +[risk-adjustment bands, reconciliation, and posture gate, restated in the +report so the numbers are auditable] + +## Collection warnings + +| Query | Error | Message | +|---|---|---| +| [query] | [code] | [message] | + +_(present only when a query failed — the totals are then a lower bound)_ +``` + +## JSON envelope + +Use this shape when the caller asks for machine-readable output instead of (or +alongside) the markdown. It is also what a host tool returns, so the same key +names travel either route. + +Top level: `recommendations`, `count`, `total_estimated_monthly_savings`, +`aws_best_case_monthly_savings`, and `reconciliation` (omitted when the caller +had nothing to reconcile, so a sizing-only response does not carry an empty key). + +Per item: `commitment_family`, `commitment_type`, `term`, `payment_option`, +`aws_recommended_commitment`, `achievable_commitment`, `commitment_unit`, +`estimated_monthly_savings`, `aws_best_case_monthly_savings`, +`estimated_savings_percentage`, `upfront_cost`, `break_even_months`, +`waste_exposure_monthly`, `confidence`, `spend_profile`, +`implementation_effort`, `rationale`, `line_items`. + +`commitment_unit` is `USD/hour` for Savings Plans and `units` for reservations. +Reading an hourly-dollar commitment as a unit count misreads it by roughly +1000x, so carry the unit wherever the number goes. + +Per `line_items` entry: `spec`, `region`, `commitment_unit`, +`aws_recommended_commitment`, `achievable_commitment`, `minimum_observed_units`, +`average_observed_units`, `estimated_monthly_savings`, `upfront_cost`, +`monthly_on_demand_cost`, `estimated_utilization_percentage`, `size_flexible`, +`current_generation`, `account_id`. + +A line item is the purchasable unit; the item-level figure is not. The +recommendation total sums every specification the service returned, and a +reservation only discounts usage matching its exact `spec`, so quote the total as +a budget and the line items as the order. `estimated_utilization_percentage` is +`null` when AWS did not return it. `size_flexible` means the recommended size is +not binding (the discount follows any size in the family); +`current_generation: false` means a long-term commitment locks the account out of +the cheaper current generation. + +The key names deliberately mirror what an AWS Cost Optimization Hub style +recommendation list looks like, so a host that already renders those needs no +translation layer. + +### Expiry envelope + +Present under `expiry` when the inventory ran, and **absent (not empty) when it +did not** — an empty expiry block reads as "nothing expires", which is a +different and possibly wrong claim. + +Top level: `as_of`, `horizon_days`, `regions`, `total_active`, `expiring`, +`expired`, `undated`, `counts` (`urgent` / `soon` / `upcoming`), `actions` +(counts per verdict), `hourly_commitment_expiring`, +`monthly_committed_spend_expiring`, `reserved_units_expiring`, `blind_spots`. + +Per commitment: `family` (`savings-plan` | `reservation`), `service`, `label`, +`commitment_id`, `arn`, `instance_type`, `attributes`, `spec`, `quantity`, +`unit`, `region`, `state`, `payment_option`, `start`, `end`, `term_months`, +`days_remaining`, `urgency`, `utilization_pct`, `action`, `rationale`. + +`attributes` is a small map of the dimensions a renewal must match — RDS carries +`deployment` (`Multi-AZ` / `Single-AZ`) and `engine`, EC2 carries `scope`, `AZ`, +`platform`, `class` and `tenancy` — and `spec` is those joined onto +`instance_type` for display. A missing key means AWS did not return the field: +absent is not the same as `Single-AZ`, so do not infer one from the other. `spec` +is empty for a Compute Savings Plan by design. + +`action` is `renew` | `renew-smaller` | `let-lapse` | `review`; `urgency` is +`urgent` | `soon` | `upcoming` | `expired`. `unit` follows the same rule as +above — `USD/hour` for Savings Plans, `units` for reservations — which is why +`hourly_commitment_expiring` and `reserved_units_expiring` are separate totals +and must never be summed together. `utilization_pct` is `null` when it could not +be measured; `null` means unmeasured, not zero. diff --git a/src/agents/hierarchy.json b/src/agents/hierarchy.json index de098af..8805bf3 100644 --- a/src/agents/hierarchy.json +++ b/src/agents/hierarchy.json @@ -8,7 +8,7 @@ "memory": true, "suggestions": true, "reports": true, - "prompt": "You are the Supervisor Agent for the CloudOps Multi-Agent System.\n\nYou receive user requests and delegate to the appropriate domain agent.\n\n{agent_listing}\n\nDomain routing rules:\n- CloudWatch alarms, alarm inventory, alarm coverage, monitoring alerts,\n alarm recommendations, and threshold tuning MUST be delegated to\n ops-excellence-agent. CloudWatch is an Ops Excellence capability even if\n a shorter agent description omits it.\n- Cost, billing, pricing, forecasts, and savings belong to finops-agent.\n- Tag policy and tag compliance belong to governance-agent.\n\nDelegation rules:\n1. Identify which ONE domain agent handles the request.\n2. Before delegating, resolve any ambiguous references (\"this\", \"that\",\n \"it\", \"last result\", \"compare to\") using conversation history.\n Sub-agents are STATELESS — they cannot see prior turns. If the user\n says \"compare this to January\", you must send something like\n \"Compare total AWS spending for February 2026 vs January 2026\".\n3. Keep the delegation prompt concise but self-contained. Do NOT add\n extra analysis requests the user didn't ask for.\n4. If a request spans multiple domains, delegate to each relevant agent.\n5. If an agent returns an error, report it clearly.\n6. Present the agent's response to the user. Add brief context if needed.\n\nCRITICAL:\n- NEVER generate filler text like \"I'll check that for you\" while waiting.\n- ONLY include actual data returned by tool calls.\n- NEVER fabricate cost figures, service names, or any data.\n- Do NOT ask multiple agents when one will suffice.\n- When a sub-agent reports that a CloudFormation template artifact was generated, present its compact summary to the user. Never emit YAML, `` markup, or a `` marker yourself; the platform persists the typed artifact and injects the valid report marker.\n", + "prompt": "You are the Supervisor Agent for the CloudOps Multi-Agent System.\n\nYou receive user requests and delegate to the appropriate domain agent.\n\n{agent_listing}\n\nDomain routing rules:\n- CloudWatch alarms, alarm inventory, alarm coverage, monitoring alerts,\n alarm recommendations, and threshold tuning MUST be delegated to\n ops-excellence-agent. CloudWatch is an Ops Excellence capability even if\n a shorter agent description omits it.\n- Cost, billing, pricing, forecasts, and savings belong to finops-agent.\n- Savings Plans, Reserved Instances, and any commitment question — should we\n buy, how much, which term, is our coverage or utilization healthy — MUST be\n delegated to finops-agent. Commitment sizing is a FinOps capability even if\n a shorter agent description omits it.\n- Tag policy and tag compliance belong to governance-agent.\n\nDelegation rules:\n1. Identify which ONE domain agent handles the request.\n2. Before delegating, resolve any ambiguous references (\"this\", \"that\",\n \"it\", \"last result\", \"compare to\") using conversation history.\n Sub-agents are STATELESS — they cannot see prior turns. If the user\n says \"compare this to January\", you must send something like\n \"Compare total AWS spending for February 2026 vs January 2026\".\n3. Keep the delegation prompt concise but self-contained. Do NOT add\n extra analysis requests the user didn't ask for.\n4. If a request spans multiple domains, delegate to each relevant agent.\n5. If an agent returns an error, report it clearly.\n6. Present the agent's response to the user. Add brief context if needed.\n\nCRITICAL:\n- NEVER generate filler text like \"I'll check that for you\" while waiting.\n- ONLY include actual data returned by tool calls.\n- NEVER fabricate cost figures, service names, or any data.\n- Do NOT ask multiple agents when one will suffice.\n- When a sub-agent reports that a CloudFormation template artifact was generated, present its compact summary to the user. Never emit YAML, `` markup, or a `` marker yourself; the platform persists the typed artifact and injects the valid report marker.\n", "children": [ "finops-agent", "governance-agent", @@ -19,9 +19,9 @@ "type": "orchestrator", "dir": "agents/orchestrator", "protocol": "http", - "description": "FinOps domain — cost analysis, CUR queries, billing, savings recommendations", + "description": "FinOps domain — cost analysis, CUR queries, billing, savings recommendations; risk-adjusted Savings Plan and Reserved Instance commitment sizing and coverage health; pricing catalog, estimates, anomalies, budgets", "model": "global.anthropic.claude-sonnet-4-6", - "prompt": "You are the FinOps Mid-Level Agent for the CloudOps Multi-Agent System.\n\nYour role is to ROUTE financial operations requests to exactly ONE specialized\nleaf agent. You do NOT access AWS services directly.\n\n{agent_listing}\n\nRouting rules — pick ONE agent per request:\n- cost-operations-agent: actual spending, cost breakdowns, cost trends,\n forecasts, CUR queries, savings recommendations, comparisons.\n Trigger words: 'spend', 'cost', 'bill', 'usage', 'forecast', 'save', 'compare'\n- pricing-agent: AWS service pricing catalog, cost estimates for new\n workloads, cost anomalies, budget alerts, account info.\n Trigger words: 'price', 'pricing', 'estimate', 'anomaly', 'budget'\n\nCRITICAL: Default to cost-operations-agent for ANY spending question.\nOnly call pricing-agent when the user explicitly asks about pricing\ncatalog, anomalies, or budgets. Do NOT call both agents unless the\nuser explicitly asks for both spend data AND pricing in the same request.\n\nNEVER ask clarifying questions. If the request is ambiguous, make a\nreasonable assumption and delegate immediately. You are a router, not\na conversationalist.\n\nPass through the user's question directly to the child agent.\n", + "prompt": "You are the FinOps Mid-Level Agent for the CloudOps Multi-Agent System.\n\nYour role is to ROUTE financial operations requests to exactly ONE specialized\nleaf agent. You do NOT access AWS services directly.\n\n{agent_listing}\n\nRouting rules — pick ONE agent per request:\n- cost-operations-agent: actual spending, cost breakdowns, cost trends,\n forecasts, CUR queries, savings recommendations, comparisons, and ALL\n Savings Plan / Reserved Instance commitment sizing, coverage, and\n utilization questions.\n Trigger words: 'spend', 'cost', 'bill', 'usage', 'forecast', 'save', 'compare',\n 'savings plan', 'reserved instance', 'RI', 'SP', 'commitment', 'commit',\n 'coverage', 'utilization', 'buy', 'purchase', 'term', 'upfront'\n- pricing-agent: AWS service pricing catalog, cost estimates for new\n workloads, cost anomalies, budget alerts, account info.\n Trigger words: 'price', 'pricing', 'estimate', 'anomaly', 'budget'\n\nCRITICAL: Default to cost-operations-agent for ANY spending question.\nOnly call pricing-agent when the user explicitly asks about pricing\ncatalog, anomalies, or budgets. Do NOT call both agents unless the\nuser explicitly asks for both spend data AND pricing in the same request.\n\nNEVER ask clarifying questions. If the request is ambiguous, make a\nreasonable assumption and delegate immediately. You are a router, not\na conversationalist.\n\nPass through the user's question directly to the child agent.\n", "children": [ "cost-operations-agent", "pricing-agent" @@ -56,14 +56,15 @@ "type": "worker", "dir": "agents/worker", "protocol": "http", - "description": "Cost analysis via CUR/Athena queries, Cost Explorer API, and Cost Optimization Hub", + "description": "Cost analysis via CUR/Athena queries, Cost Explorer API, Cost Optimization Hub, and risk-adjusted Savings Plan / Reserved Instance commitment sizing", "model": "global.anthropic.claude-sonnet-4-6", "tools": [ "cost-explorer", "cur-athena", - "cost-optimization-hub" + "cost-optimization-hub", + "commitments" ], - "prompt": "You are the Cost Operations Leaf Agent for the CloudOps Multi-Agent System.\n\nYou answer questions about AWS spending using gateway tools.\n\nTool selection — use the MINIMUM number of calls needed:\n- For simple spend questions ('how much did I spend last month'), make ONE\n call to get_cost_and_usage with monthly granularity. That's it.\n- Only add group_by (SERVICE, REGION) if the user asks for a breakdown.\n- Only use daily granularity if the user asks about trends or specific days.\n- Only call get_cost_and_usage_comparisons if the user asks to compare periods.\n- Only call get_cost_forecast if the user asks about future costs.\n- Only use cur-athena for detailed line-item queries the Cost Explorer API\n can't answer (e.g., specific resource IDs, usage types, custom SQL).\n- Only use cost-optimization-hub when the user asks about savings or\n optimization. Always call get_enrollment_status first.\n\nDo NOT proactively run extra queries the user didn't ask for. Answer the\nquestion asked, then offer to drill deeper if relevant.\n\nWhen presenting cost data:\n- Include the time period.\n- Format amounts as USD with two decimal places.\n- Highlight top cost drivers only when showing breakdowns.\n\nWhen: Need specific breakdowns or resource-level detail\nDecision:\n- Cost Explorer with filters: Standard dimensions (service, region, account, tags), fast results\n- Custom CUR queries (CUR MCP): Resource IDs, usage types, complex filtering\n\nCUR Query Pattern:\n1. Always use date filtering (IMPORTANT: date columns are TIMESTAMP type, not strings):\n - Monthly granularity: WHERE date_format(bill_billing_period_start_date, '%Y-%m') = '2026-01'\n - Daily granularity: WHERE line_item_usage_start_date >= TIMESTAMP '2026-01-15 00:00:00' AND line_item_usage_start_date < TIMESTAMP '2026-01-16 00:00:00'\n - Always combine with partition filter for performance: AND billing_period = '2026-01'\n - billing_period is the ONLY partition column (format YYYY-MM, string type)\n2. Use fully qualified table name from the tool description (e.g., finopsagent_cur_db.cur2)\n3. NEVER use LIMIT in CUR SQL queries — the tool has its own max_results parameter to control result size\n\nKey CUR 2 Fields:\n- bill_billing_period_start_date — Billing period start (TIMESTAMP type, use date_format for comparison)\n- line_item_usage_start_date — Usage start datetime (TIMESTAMP type)\n- bill_billing_entity — 'AWS' or 'AWS Marketplace'\n- line_item_line_item_type — Charge type: Usage, DiscountedUsage, SavingsPlanCoveredUsage, SavingsPlanRecurringFee, SavingsPlanNegation, SavingsPlanUpfrontFee, RIFee, Fee, Credit, Refund, Tax, BundledDiscount, Discount, FlatRateSubscription\n- line_item_unblended_cost — Unblended cost\n- line_item_resource_id — Resource ID or ARN\n- line_item_usage_account_id — Account ID\n- line_item_usage_account_name — Account name\n- line_item_usage_amount — Usage quantity\n- line_item_operation — AWS operation (e.g., RunInstances, PutObject)\n- line_item_line_item_description — Most granular cost description\n- line_item_product_code — AWS service code\n- pricing_purchase_option — Purchase option\n\nCost Allocation Tags:\n- Query by tag: element_at(resource_tags, 'user_') as \n- Tag key convention: prefix with user_, replace - with _\n- Example: tag \"project-group\" → element_at(resource_tags, 'user_project_group') as project_group\n\nAmortized Cost SQL:\nsum(CASE\n WHEN (line_item_line_item_type = 'SavingsPlanCoveredUsage') THEN savings_plan_savings_plan_effective_cost\n WHEN (line_item_line_item_type = 'SavingsPlanRecurringFee') THEN (savings_plan_total_commitment_to_date - savings_plan_used_commitment)\n WHEN (line_item_line_item_type = 'SavingsPlanNegation') THEN 0\n WHEN (line_item_line_item_type = 'SavingsPlanUpfrontFee') THEN 0\n WHEN (line_item_line_item_type = 'DiscountedUsage') THEN reservation_effective_cost\n WHEN (line_item_line_item_type = 'RIFee') THEN (reservation_unused_amortized_upfront_fee_for_billing_period + reservation_unused_recurring_fee)\n WHEN ((line_item_line_item_type = 'Fee') AND (reservation_reservation_a_r_n <> '')) THEN 0\n WHEN ((line_item_line_item_type = 'Refund') AND (line_item_product_code = 'ComputeSavingsPlans')) THEN 0\n ELSE line_item_unblended_cost\nEND) as amortized_cost\n\nPurchase Option Classification:\n(CASE\n WHEN (savings_plan_savings_plan_a_r_n <> '') THEN 'SavingsPlan'\n WHEN (reservation_reservation_a_r_n <> '') THEN 'Reserved'\n WHEN (line_item_usage_type LIKE '%Spot%') THEN 'Spot'\n ELSE 'OnDemand'\nEND) as purchase_option\n\nCommon Exclusion Filters:\n- Exclude Marketplace: AND bill_billing_entity = 'AWS'\n- Exclude non-usage: AND line_item_line_item_type NOT IN ('Tax', 'Credit', 'Refund')\n" + "prompt": "You are the Cost Operations Leaf Agent for the CloudOps Multi-Agent System.\n\nYou answer questions about AWS spending using gateway tools.\n\nTool selection — use the MINIMUM number of calls needed:\n- For simple spend questions ('how much did I spend last month'), make ONE\n call to get_cost_and_usage with monthly granularity. That's it.\n- Only add group_by (SERVICE, REGION) if the user asks for a breakdown.\n- Only use daily granularity if the user asks about trends or specific days.\n- Only call get_cost_and_usage_comparisons if the user asks to compare periods.\n- Only call get_cost_forecast if the user asks about future costs.\n- Only use cur-athena for detailed line-item queries the Cost Explorer API\n can't answer (e.g., specific resource IDs, usage types, custom SQL).\n- Only use cost-optimization-hub when the user asks about savings or\n optimization. Always call get_enrollment_status first.\n- For ANY question about Savings Plans or Reserved Instances — should we buy,\n how much, which term, is our coverage right — use the commitments tools, NOT\n cost-optimization-hub alone. Prefer ONE call to generate_commitment_analysis\n with no arguments: it sweeps both terms and payment options, risk-adjusts the\n sizing against the workload's quietest hour, checks existing commitment\n health, reconciles against Cost Optimization Hub, and returns a complete\n pre-formatted report in 'report_markdown' which you should emit verbatim.\n- Use size_savings_plans or size_reservations instead only when the user\n narrowed the question to one commitment family or one specific term/payment\n option. Use get_commitment_posture alone when they ask only about existing\n coverage and utilization, with no purchase question.\n- NEVER recommend a purchase without the posture data: unused existing\n commitments must be surfaced first, because buying on top of an\n under-utilized commitment compounds waste. generate_commitment_analysis\n already includes this; if you called a sizing tool directly, call\n get_commitment_posture too.\n- Report the risk-adjusted (achievable) savings as the headline number, never\n the AWS best-case figure, and always state that commitments are\n non-cancellable for their full term.\n\nDo NOT proactively run extra queries the user didn't ask for. Answer the\nquestion asked, then offer to drill deeper if relevant.\n\nWhen presenting cost data:\n- Include the time period.\n- Format amounts as USD with two decimal places.\n- Highlight top cost drivers only when showing breakdowns.\n\nWhen: Need specific breakdowns or resource-level detail\nDecision:\n- Cost Explorer with filters: Standard dimensions (service, region, account, tags), fast results\n- Custom CUR queries (CUR MCP): Resource IDs, usage types, complex filtering\n\nCUR Query Pattern:\n1. Always use date filtering (IMPORTANT: date columns are TIMESTAMP type, not strings):\n - Monthly granularity: WHERE date_format(bill_billing_period_start_date, '%Y-%m') = '2026-01'\n - Daily granularity: WHERE line_item_usage_start_date >= TIMESTAMP '2026-01-15 00:00:00' AND line_item_usage_start_date < TIMESTAMP '2026-01-16 00:00:00'\n - Always combine with partition filter for performance: AND billing_period = '2026-01'\n - billing_period is the ONLY partition column (format YYYY-MM, string type)\n2. Use fully qualified table name from the tool description (e.g., finopsagent_cur_db.cur2)\n3. NEVER use LIMIT in CUR SQL queries — the tool has its own max_results parameter to control result size\n\nKey CUR 2 Fields:\n- bill_billing_period_start_date — Billing period start (TIMESTAMP type, use date_format for comparison)\n- line_item_usage_start_date — Usage start datetime (TIMESTAMP type)\n- bill_billing_entity — 'AWS' or 'AWS Marketplace'\n- line_item_line_item_type — Charge type: Usage, DiscountedUsage, SavingsPlanCoveredUsage, SavingsPlanRecurringFee, SavingsPlanNegation, SavingsPlanUpfrontFee, RIFee, Fee, Credit, Refund, Tax, BundledDiscount, Discount, FlatRateSubscription\n- line_item_unblended_cost — Unblended cost\n- line_item_resource_id — Resource ID or ARN\n- line_item_usage_account_id — Account ID\n- line_item_usage_account_name — Account name\n- line_item_usage_amount — Usage quantity\n- line_item_operation — AWS operation (e.g., RunInstances, PutObject)\n- line_item_line_item_description — Most granular cost description\n- line_item_product_code — AWS service code\n- pricing_purchase_option — Purchase option\n\nCost Allocation Tags:\n- Query by tag: element_at(resource_tags, 'user_') as \n- Tag key convention: prefix with user_, replace - with _\n- Example: tag \"project-group\" → element_at(resource_tags, 'user_project_group') as project_group\n\nAmortized Cost SQL:\nsum(CASE\n WHEN (line_item_line_item_type = 'SavingsPlanCoveredUsage') THEN savings_plan_savings_plan_effective_cost\n WHEN (line_item_line_item_type = 'SavingsPlanRecurringFee') THEN (savings_plan_total_commitment_to_date - savings_plan_used_commitment)\n WHEN (line_item_line_item_type = 'SavingsPlanNegation') THEN 0\n WHEN (line_item_line_item_type = 'SavingsPlanUpfrontFee') THEN 0\n WHEN (line_item_line_item_type = 'DiscountedUsage') THEN reservation_effective_cost\n WHEN (line_item_line_item_type = 'RIFee') THEN (reservation_unused_amortized_upfront_fee_for_billing_period + reservation_unused_recurring_fee)\n WHEN ((line_item_line_item_type = 'Fee') AND (reservation_reservation_a_r_n <> '')) THEN 0\n WHEN ((line_item_line_item_type = 'Refund') AND (line_item_product_code = 'ComputeSavingsPlans')) THEN 0\n ELSE line_item_unblended_cost\nEND) as amortized_cost\n\nPurchase Option Classification:\n(CASE\n WHEN (savings_plan_savings_plan_a_r_n <> '') THEN 'SavingsPlan'\n WHEN (reservation_reservation_a_r_n <> '') THEN 'Reserved'\n WHEN (line_item_usage_type LIKE '%Spot%') THEN 'Spot'\n ELSE 'OnDemand'\nEND) as purchase_option\n\nCommon Exclusion Filters:\n- Exclude Marketplace: AND bill_billing_entity = 'AWS'\n- Exclude non-usage: AND line_item_line_item_type NOT IN ('Tax', 'Credit', 'Refund')\n" }, "pricing-agent": { "type": "worker", diff --git a/src/agents/shared/report_templates/discounted_commitments.json b/src/agents/shared/report_templates/discounted_commitments.json new file mode 100644 index 0000000..08d38ba --- /dev/null +++ b/src/agents/shared/report_templates/discounted_commitments.json @@ -0,0 +1,12 @@ +{ + "name": "Discounted Commitments Report", + "description": "Risk-adjusted AWS Savings Plan and Reserved Instance purchase plan: what AWS recommends, what the workload can actually sustain on its quietest hour, break-even and waste exposure per option, health blockers on existing commitments, and reconciliation against Cost Optimization Hub. Routes to the FinOps domain (cost-operations worker agent).", + "sections": [ + { + "id": "full_commitment_analysis", + "title": "AWS Discounted Commitments Report", + "prompt": "Produce the AWS discounted commitments report.\n\nSTEP 1: Call generate_commitment_analysis with no arguments (the defaults sweep both terms, no-upfront and all-upfront, all four Savings Plan types, and all RI-eligible services over a 30-day lookback). This one call returns a field named 'report_markdown' that already contains the COMPLETE, pre-formatted report: bottom line, reconciliation against Cost Optimization Hub, existing commitment health with blockers, every recommended commitment WITH its risk rationale, eligible spend, and method.\n\nSTEP 2: Output the value of 'report_markdown' EXACTLY as returned — verbatim, in full. Do not summarize, truncate, re-order, or drop any table rows or rationale bullets. It is already correct and complete, and the section ordering is deliberate: posture blockers come BEFORE the savings number so a reader cannot skim the total without seeing that existing commitments are under-utilized.\n\nSTEP 3: After the report_markdown content, append this section only:\n\n## Purchase sequence\n- Buy nothing until every blocker listed under *Existing commitment health* is cleared — buying on top of an under-utilized commitment compounds waste.\n- Then take the High-confidence rows first, in descending achievable-savings order.\n- Treat Medium confidence as a smaller first tranche; re-measure after 30 days and top up rather than committing the full amount now.\n- Do not buy any row whose break-even exceeds its own term — that purchase cannot pay back.\n\nRULES: The body of the report MUST be the report_markdown field verbatim — that is what guarantees the risk-adjusted figures, the non-cancellable disclaimer, and every rationale bullet are present. Do NOT rebuild the recommendation table from the structured fields yourself. Do NOT quote the AWS best-case savings figure as if it were achievable; the achievable (risk-adjusted) figure is the headline. If 'reconciliation' reports status 'material-variance', do NOT present a single savings number as settled — the report already says why. If the report says no commitment opportunity was found, that is a REAL result, not a failure: report it as-is and point at the eligible-spend section, do not retry with different parameters and do not invent recommendations. This tool returns LIVE data from the real AWS account (data_source='live') — do NOT add any demo/mock/sample-data disclaimer. Do NOT ask the user anything. Do NOT add follow-up questions. Execute immediately." + } + ], + "dependencies": {} +} diff --git a/src/lambda/frontend/core-api/report_templates/discounted_commitments.json b/src/lambda/frontend/core-api/report_templates/discounted_commitments.json new file mode 100644 index 0000000..08d38ba --- /dev/null +++ b/src/lambda/frontend/core-api/report_templates/discounted_commitments.json @@ -0,0 +1,12 @@ +{ + "name": "Discounted Commitments Report", + "description": "Risk-adjusted AWS Savings Plan and Reserved Instance purchase plan: what AWS recommends, what the workload can actually sustain on its quietest hour, break-even and waste exposure per option, health blockers on existing commitments, and reconciliation against Cost Optimization Hub. Routes to the FinOps domain (cost-operations worker agent).", + "sections": [ + { + "id": "full_commitment_analysis", + "title": "AWS Discounted Commitments Report", + "prompt": "Produce the AWS discounted commitments report.\n\nSTEP 1: Call generate_commitment_analysis with no arguments (the defaults sweep both terms, no-upfront and all-upfront, all four Savings Plan types, and all RI-eligible services over a 30-day lookback). This one call returns a field named 'report_markdown' that already contains the COMPLETE, pre-formatted report: bottom line, reconciliation against Cost Optimization Hub, existing commitment health with blockers, every recommended commitment WITH its risk rationale, eligible spend, and method.\n\nSTEP 2: Output the value of 'report_markdown' EXACTLY as returned — verbatim, in full. Do not summarize, truncate, re-order, or drop any table rows or rationale bullets. It is already correct and complete, and the section ordering is deliberate: posture blockers come BEFORE the savings number so a reader cannot skim the total without seeing that existing commitments are under-utilized.\n\nSTEP 3: After the report_markdown content, append this section only:\n\n## Purchase sequence\n- Buy nothing until every blocker listed under *Existing commitment health* is cleared — buying on top of an under-utilized commitment compounds waste.\n- Then take the High-confidence rows first, in descending achievable-savings order.\n- Treat Medium confidence as a smaller first tranche; re-measure after 30 days and top up rather than committing the full amount now.\n- Do not buy any row whose break-even exceeds its own term — that purchase cannot pay back.\n\nRULES: The body of the report MUST be the report_markdown field verbatim — that is what guarantees the risk-adjusted figures, the non-cancellable disclaimer, and every rationale bullet are present. Do NOT rebuild the recommendation table from the structured fields yourself. Do NOT quote the AWS best-case savings figure as if it were achievable; the achievable (risk-adjusted) figure is the headline. If 'reconciliation' reports status 'material-variance', do NOT present a single savings number as settled — the report already says why. If the report says no commitment opportunity was found, that is a REAL result, not a failure: report it as-is and point at the eligible-spend section, do not retry with different parameters and do not invent recommendations. This tool returns LIVE data from the real AWS account (data_source='live') — do NOT add any demo/mock/sample-data disclaimer. Do NOT ask the user anything. Do NOT add follow-up questions. Execute immediately." + } + ], + "dependencies": {} +} diff --git a/src/lambda/mcp/commitments/commitments/__init__.py b/src/lambda/mcp/commitments/commitments/__init__.py new file mode 100644 index 0000000..73523ab --- /dev/null +++ b/src/lambda/mcp/commitments/commitments/__init__.py @@ -0,0 +1 @@ +"""Read-only AWS commitment (RI/Savings Plan) analysis.""" diff --git a/src/lambda/mcp/commitments/commitments/analyze.py b/src/lambda/mcp/commitments/commitments/analyze.py new file mode 100644 index 0000000..71a9c70 --- /dev/null +++ b/src/lambda/mcp/commitments/commitments/analyze.py @@ -0,0 +1,816 @@ +"""Turn raw recommendation data into a measured, achievable commitment plan. + +The AWS recommendation APIs return a best case: the commitment level that +maximizes savings assuming the lookback window repeats. This module adds the +parts a purchase decision actually needs — + + * whether spend is stable enough for the recommendation to hold (volatility) + * a floor-based commitment the workload sustains even on its quietest hour + * break-even and waste exposure if usage drops + * reconciliation against Cost Optimization Hub's independent pipeline + +Everything is a pure function over collected data; nothing calls AWS. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from datetime import date +from typing import Any + +from .api import HOURS_PER_MONTH, SP_TYPE_LABELS, describe_recommendation_spec + +# Volatility bands for the ratio of trough to average hourly on-demand spend. +# A workload whose quietest hour is close to its average is safe to commit near +# the API recommendation; a spiky one is not. +STABLE_FLOOR_RATIO = 0.80 +MODERATE_FLOOR_RATIO = 0.50 + +# Utilization below this on an existing commitment means money is already being +# wasted, and is a reason to hold off buying more. +UTILIZATION_WARN_PCT = 95.0 + +# Coverage above this means there is little on-demand left to convert; further +# purchase risks over-committing. +COVERAGE_SATURATED_PCT = 90.0 + +CONFIDENCE_HIGH = "High" +CONFIDENCE_MEDIUM = "Medium" +CONFIDENCE_LOW = "Low" + +# Commitment terms in months, for checking whether an upfront payment can even +# pay back before the commitment expires. +TERM_MONTHS = {"ONE_YEAR": 12, "THREE_YEARS": 36} + +# Expiry urgency bands, in days remaining. 30 days is roughly the shortest +# notice on which a renewal can clear finance approval, so it is the point at +# which a lapse becomes a scheduling problem rather than a planning one. +EXPIRY_URGENT_DAYS = 30 +EXPIRY_SOON_DAYS = 60 +EXPIRY_HORIZON_DAYS = 90 + +# Renewal verdicts. Deliberately less conservative than the initial-purchase +# bands above: a lapsing commitment has zero switching cost, so expiry is the +# one free resize point in a commitment's life. Renewing at the same size is +# the risk; renewing smaller is not. +RENEW = "renew" +RENEW_SMALLER = "renew-smaller" +LET_LAPSE = "let-lapse" +REVIEW = "review" + +# Utilization below this means the commitment is more waste than saving, so +# re-buying it at the same size compounds a mistake rather than protecting a +# discount. +RENEW_LAPSE_PCT = 50.0 + + +@dataclass +class LineItem: + """One purchasable line of a recommendation: what to buy, and how much. + + A Finding aggregates every line item AWS returned for a service so the + savings can be ranked against other services, but the aggregate is not + purchasable — "4 RDS reservations" is not an order. Each line item carries + its own specification (`db.r6g.xlarge · Multi-AZ · Aurora PostgreSQL`) and + its own measured floor, which is what someone takes to the console. + """ + + spec: str + unit: str + recommended: float + floor: float + average: float + achievable: float + monthly_savings: float + upfront_cost: float + utilization_pct: float | None = None + monthly_on_demand: float = 0.0 + size_flex_eligible: bool = False + current_generation: bool = True + region: str = "" + account_id: str = "" + + +@dataclass +class Finding: + """One actionable commitment opportunity, risk-adjusted.""" + + family: str + label: str + term: str + payment: str + api_hourly_commitment: float + safe_hourly_commitment: float + api_monthly_savings: float + safe_monthly_savings: float + savings_percentage: float + upfront_cost: float + confidence: str + volatility: str + rationale: list[str] = field(default_factory=list) + break_even_months: float | None = None + waste_exposure_monthly: float = 0.0 + source: str = "cost-explorer" + detail_count: int = 0 + line_items: list[LineItem] = field(default_factory=list) + + +def _f(value: Any, default: float = 0.0) -> float: + """Coerce an API numeric-string to float. + + Cost Explorer returns money and percentages as STRINGS, and returns "" for + absent values — float("") raises, so every read goes through here. + """ + if value is None or value == "": + return default + try: + return float(value) + except (TypeError, ValueError): + return default + + +def classify_volatility(floor: float, average: float) -> tuple[str, float]: + """Rate spend stability from the trough-to-average ratio. + + Returns (label, ratio). A ratio near 1.0 means a flat workload. + """ + if average <= 0: + return "unknown", 0.0 + ratio = floor / average + if ratio >= STABLE_FLOOR_RATIO: + return "stable", ratio + if ratio >= MODERATE_FLOOR_RATIO: + return "moderate", ratio + return "spiky", ratio + + +def _safe_commitment( + api_hourly: float, + floor_hourly: float, + avg_hourly: float, + unit: str = "$/hr", +) -> tuple[float, str, str, list[str]]: + """Derive a commitment level the workload sustains, plus confidence. + + The API optimizes for total savings and will happily recommend committing + above the trough, which produces unused-commitment waste in quiet hours. + For spiky workloads this clamps the commitment to the measured floor. + + `unit` selects both the wording and the granularity: Savings Plans commit in + dollars per hour, reservations in whole instance or capacity units. Rounding + happens here rather than in the caller so the figure quoted in the notes is + the same one the report tabulates. + """ + volatility, ratio = classify_volatility(floor_hourly, avg_hourly) + notes: list[str] = [] + whole_units = unit != "$/hr" + + def quantize(value: float) -> float: + # Reservations are sold in whole units; round down so the commitment + # never lands above the level that was judged safe. + return float(int(value)) if whole_units else value + + def fmt(value: float) -> str: + return f"{value:,.0f} unit(s)" if whole_units else f"${value:,.4f}/hr" + + if api_hourly <= 0: + return 0.0, CONFIDENCE_LOW, volatility, ["API recommended no commitment."] + + if volatility == "stable": + safe = quantize(api_hourly) + confidence = CONFIDENCE_HIGH + notes.append( + f"Trough hour is {ratio:.0%} of average — flat workload, " + "API recommendation is safe to take as-is." + ) + elif volatility == "moderate": + # Commit to the floor plus half the gap to the API figure: captures + # most of the savings while staying clear of the trough. + safe = quantize( + min(api_hourly, floor_hourly + (api_hourly - floor_hourly) * 0.5) + ) + confidence = CONFIDENCE_MEDIUM + notes.append( + f"Trough hour is {ratio:.0%} of average — moderately variable. " + f"Commitment trimmed to {fmt(safe)} (midpoint of floor and API " + "recommendation) to limit unused-commitment risk." + ) + else: + # Never commit above what the quietest hour consumes. + safe = quantize(min(api_hourly, floor_hourly)) + confidence = CONFIDENCE_LOW + notes.append( + f"Trough hour is only {ratio:.0%} of average — spiky workload. " + f"Commitment clamped to the measured floor ({fmt(safe)}); " + "committing to the API figure would strand spend in quiet hours." + ) + + if volatility == "unknown": + notes.append( + "No hourly on-demand spend data returned, so the floor could not be " + "measured — treat the API figure as unvalidated." + ) + confidence = CONFIDENCE_LOW + + return max(safe, 0.0), confidence, volatility, notes + + +def _check_break_even_against_term( + break_even: float | None, term: str, notes: list[str], confidence: str +) -> str: + """Flag an upfront purchase that cannot pay back within its own term. + + A commitment is non-cancellable and expires at the end of the term, so a + break-even beyond the term means the purchase loses money outright — the + single most expensive mistake this report exists to prevent. Returns the + confidence to use, downgraded to Low when the purchase cannot pay back. + """ + term_months = TERM_MONTHS.get(term) + if break_even is None or term_months is None or break_even <= term_months: + return confidence + notes.insert( + 0, + f"**Do not buy.** Break-even is {break_even:.1f} months but the term " + f"ends at {term_months}. At the achievable commitment level this " + "purchase never pays back — take the no-upfront option, a shorter " + "term, or nothing.", + ) + return CONFIDENCE_LOW + + +SPEC_UNAVAILABLE = "(specification not returned)" + + +def _ri_line_items(details: list[dict[str, Any]], scale: float) -> list[LineItem]: + """Break an RI recommendation into the individual purchases it implies. + + `scale` is the family-level risk adjustment, applied per line and rounded + **down** because reservations are sold whole. Rounding down can leave the + line items summing slightly below the family total; the report says so + rather than quietly padding a line. + """ + items: list[LineItem] = [] + for d in details: + # The instance fields come back empty for capacity-unit services + # (DynamoDB), which report the same three measures under different + # names. `or` picks up the fallback because a genuine 0 needs it too. + recommended = _f(d.get("RecommendedNumberOfInstancesToPurchase")) or _f( + d.get("RecommendedNumberOfCapacityUnitsToPurchase") + ) + floor = _f(d.get("MinimumNumberOfInstancesUsedPerHour")) or _f( + d.get("MinimumNumberOfCapacityUnitsUsedPerHour") + ) + average = _f(d.get("AverageNumberOfInstancesUsedPerHour")) or _f( + d.get("AverageNumberOfCapacityUnitsUsedPerHour") + ) + spec = describe_recommendation_spec(d) + utilization = _f(d.get("AverageUtilization"), -1.0) + items.append( + LineItem( + spec=spec.get("label") or SPEC_UNAVAILABLE, + unit="units", + recommended=recommended, + floor=floor, + average=average, + achievable=float(int(recommended * scale)), + monthly_savings=_f(d.get("EstimatedMonthlySavingsAmount")), + upfront_cost=_f(d.get("UpfrontCost")), + utilization_pct=utilization if utilization >= 0 else None, + monthly_on_demand=_f(d.get("EstimatedMonthlyOnDemandCost")), + size_flex_eligible=bool(spec.get("size_flex_eligible")), + current_generation=bool(spec.get("current_generation", True)), + region=spec.get("region", ""), + account_id=str(d.get("AccountId") or ""), + ) + ) + return sorted(items, key=lambda i: -i.monthly_savings) + + +def _sp_line_items(details: list[dict[str, Any]], scale: float) -> list[LineItem]: + """Break a Savings Plan recommendation into its per-family commitments. + + Only an EC2 Instance Savings Plan is scoped to a family and region, so for a + Compute plan these fields come back empty — which is the plan being flexible + by design, not data going missing. + """ + items: list[LineItem] = [] + for d in details: + sp = d.get("SavingsPlansDetails") or {} + region = str(sp.get("Region") or "") + family = str(sp.get("InstanceFamily") or "") + hourly = _f(d.get("HourlyCommitmentToPurchase")) + utilization = _f(d.get("EstimatedAverageUtilization"), -1.0) + items.append( + LineItem( + spec=family or "any instance family", + unit="USD/hour", + recommended=hourly, + floor=_f(d.get("CurrentMinimumHourlyOnDemandSpend")), + average=_f(d.get("CurrentAverageHourlyOnDemandSpend")), + achievable=hourly * scale, + monthly_savings=_f(d.get("EstimatedMonthlySavingsAmount")), + upfront_cost=_f(d.get("UpfrontCost")), + utilization_pct=utilization if utilization >= 0 else None, + monthly_on_demand=_f(d.get("EstimatedOnDemandCost")), + region=region, + account_id=str(d.get("AccountId") or ""), + ) + ) + return sorted(items, key=lambda i: -i.monthly_savings) + + +def analyze_sp_recommendation(rec: dict[str, Any]) -> Finding | None: + """Convert one SP recommendation permutation into a risk-adjusted Finding.""" + if rec.get("error"): + return None + summary = rec.get("summary") or {} + details = rec.get("details") or [] + + api_hourly = _f(summary.get("HourlyCommitmentToPurchase")) + api_monthly = _f(summary.get("EstimatedMonthlySavingsAmount")) + if api_hourly <= 0 and api_monthly <= 0: + return None + + # Aggregate the per-detail hourly spend envelope. The summary omits it, so + # the floor has to come from the details. + floor = sum(_f(d.get("CurrentMinimumHourlyOnDemandSpend")) for d in details) + average = sum(_f(d.get("CurrentAverageHourlyOnDemandSpend")) for d in details) + if average <= 0: + average = _f(summary.get("CurrentOnDemandSpend")) / HOURS_PER_MONTH + + safe_hourly, confidence, volatility, notes = _safe_commitment( + api_hourly, floor, average + ) + + # Savings scale with the commitment, since the discount rate is fixed per + # plan. Scaling down the commitment scales down the savings proportionally. + scale = (safe_hourly / api_hourly) if api_hourly > 0 else 0.0 + safe_monthly = api_monthly * scale + + # Waste exposure: what an unused commitment costs per month if usage falls + # to the trough while committed at the recommended level. + waste = max(0.0, (api_hourly - floor)) * HOURS_PER_MONTH if floor > 0 else 0.0 + + upfront = sum(_f(d.get("UpfrontCost")) for d in details) + if upfront > 0 and safe_monthly > 0: + break_even = upfront / safe_monthly + notes.append( + f"${upfront:,.2f} upfront pays back in {break_even:.1f} months at the " + "adjusted savings rate." + ) + else: + break_even = None + + confidence = _check_break_even_against_term( + break_even, rec["term"], notes, confidence + ) + + est_util = [_f(d.get("EstimatedAverageUtilization")) for d in details] + est_util = [u for u in est_util if u > 0] + if est_util: + mean_util = sum(est_util) / len(est_util) + notes.append( + f"AWS projects {mean_util:.1f}% average utilization on the " + "recommended commitment." + ) + + return Finding( + family="savings-plan", + label=SP_TYPE_LABELS.get(rec["sp_type"], rec["sp_type"]), + term=rec["term"], + payment=rec["payment"], + api_hourly_commitment=api_hourly, + safe_hourly_commitment=safe_hourly, + api_monthly_savings=api_monthly, + safe_monthly_savings=safe_monthly, + savings_percentage=_f(summary.get("EstimatedSavingsPercentage")), + upfront_cost=upfront, + confidence=confidence, + volatility=volatility, + rationale=notes, + break_even_months=break_even, + waste_exposure_monthly=waste, + detail_count=len(details), + line_items=_sp_line_items(details, scale), + ) + + +def analyze_ri_recommendation(rec: dict[str, Any]) -> Finding | None: + """Convert one RI recommendation permutation into a risk-adjusted Finding. + + RIs commit to instance counts rather than dollars per hour, so the floor is + measured in instances: MinimumNumberOfInstancesUsedPerHour is the count the + workload never drops below. + """ + if rec.get("error"): + return None + summary = rec.get("summary") or {} + details = rec.get("details") or [] + + api_monthly = _f(summary.get("TotalEstimatedMonthlySavingsAmount")) + if api_monthly <= 0 and not details: + return None + + recommended_units = sum( + _f(d.get("RecommendedNumberOfInstancesToPurchase")) for d in details + ) + floor_units = sum( + _f(d.get("MinimumNumberOfInstancesUsedPerHour")) for d in details + ) + avg_units = sum( + _f(d.get("AverageNumberOfInstancesUsedPerHour")) for d in details + ) + + # DynamoDB and other capacity-unit services report capacity units instead + # of instance counts. + if recommended_units <= 0: + recommended_units = sum( + _f(d.get("RecommendedNumberOfCapacityUnitsToPurchase")) for d in details + ) + floor_units = sum( + _f(d.get("MinimumNumberOfCapacityUnitsUsedPerHour")) for d in details + ) + avg_units = sum( + _f(d.get("AverageNumberOfCapacityUnitsUsedPerHour")) for d in details + ) + + safe_units, confidence, volatility, notes = _safe_commitment( + recommended_units, floor_units, avg_units, unit="units" + ) + + scale = (safe_units / recommended_units) if recommended_units > 0 else 0.0 + safe_monthly = api_monthly * scale + + upfront = sum(_f(d.get("UpfrontCost")) for d in details) + monthly_recurring = sum( + _f(d.get("RecurringStandardMonthlyCost")) for d in details + ) + + # AWS computes break-even per detail; average it rather than recomputing, + # so the figure ties out to the console. + be = [_f(d.get("EstimatedBreakEvenInMonths")) for d in details] + be = [b for b in be if b > 0] + break_even = (sum(be) / len(be)) if be else None + + line_items = _ri_line_items(details, scale) + if recommended_units > 0: + notes.insert( + 0, + f"API recommends {recommended_units:.0f} unit(s); workload floor is " + f"{floor_units:.0f}, so {safe_units:.0f} is defensible.", + ) + # A single-line recommendation needs no allocation guidance; a multi-line one + # does, because the aggregate above spans several instance specifications and + # a reservation can only be bought against one of them. + if len(line_items) > 1: + notes.append( + f"This total spans {len(line_items)} distinct instance " + "specifications — see the line items for what to buy against each. " + "Reservations only apply to usage matching their exact " + "specification, so the aggregate is a budget, not an order." + ) + confidence = _check_break_even_against_term( + break_even, rec["term"], notes, confidence + ) + waste = max(0.0, recommended_units - floor_units) + waste_cost = ( + (monthly_recurring / recommended_units * waste) + if recommended_units > 0 + else 0.0 + ) + + util = [_f(d.get("AverageUtilization")) for d in details] + util = [u for u in util if u > 0] + if util: + notes.append( + f"Observed average utilization across recommended families: " + f"{sum(util) / len(util):.1f}%." + ) + + return Finding( + family="reserved-instance", + label=rec.get("label", rec.get("service", "")), + term=rec["term"], + payment=rec["payment"], + api_hourly_commitment=recommended_units, + safe_hourly_commitment=safe_units, + api_monthly_savings=api_monthly, + safe_monthly_savings=safe_monthly, + savings_percentage=_f(summary.get("TotalEstimatedMonthlySavingsPercentage")), + upfront_cost=upfront, + confidence=confidence, + volatility=volatility, + rationale=notes, + break_even_months=break_even, + waste_exposure_monthly=waste_cost, + detail_count=len(details), + line_items=line_items, + ) + + +def assess_existing_posture( + sp_coverage: dict[str, Any], + sp_utilization: dict[str, Any], + ri_coverage: dict[str, Any], + ri_utilization: dict[str, Any], +) -> dict[str, Any]: + """Summarize whether existing commitments are healthy enough to add more. + + Buying on top of an under-utilized commitment compounds waste, so this + produces explicit blockers the report surfaces before any recommendation. + """ + posture: dict[str, Any] = {"blockers": [], "notes": []} + + # --- Savings Plans coverage ------------------------------------------- + periods = sp_coverage.get("periods") or [] + if periods: + latest = periods[-1].get("Coverage", {}) + pct = _f(latest.get("CoveragePercentage")) + posture["sp_coverage_pct"] = pct + posture["sp_on_demand_cost"] = _f(latest.get("OnDemandCost")) + if pct >= COVERAGE_SATURATED_PCT: + posture["blockers"].append( + f"Savings Plans coverage is already {pct:.1f}% — little " + "on-demand spend left to convert. Verify headroom before buying." + ) + elif sp_coverage.get("error"): + posture["notes"].append(f"SP coverage unavailable: {sp_coverage['error']}") + + # --- Savings Plans utilization ---------------------------------------- + sp_total = sp_utilization.get("total") or {} + if sp_total: + util = sp_total.get("Utilization", {}) + pct = _f(util.get("UtilizationPercentage")) + posture["sp_utilization_pct"] = pct + posture["sp_unused_commitment"] = _f(util.get("UnusedCommitment")) + if pct and pct < UTILIZATION_WARN_PCT: + posture["blockers"].append( + f"Existing Savings Plans are only {pct:.1f}% utilized " + f"(${_f(util.get('UnusedCommitment')):,.2f} unused). Fix this " + "before adding commitment." + ) + elif sp_utilization.get("error"): + posture["notes"].append( + f"SP utilization unavailable: {sp_utilization['error']}" + ) + + # --- Reservations ------------------------------------------------------ + ri_total = ri_coverage.get("total") or {} + if ri_total: + hours = ri_total.get("CoverageHours", {}) + pct = _f(hours.get("CoverageHoursPercentage")) + posture["ri_coverage_pct"] = pct + posture["ri_on_demand_hours"] = _f(hours.get("OnDemandHours")) + elif ri_coverage.get("error"): + posture["notes"].append(f"RI coverage unavailable: {ri_coverage['error']}") + + ri_util_total = ri_utilization.get("total") or {} + if ri_util_total: + pct = _f(ri_util_total.get("UtilizationPercentage")) + posture["ri_utilization_pct"] = pct + posture["ri_unused_hours"] = _f(ri_util_total.get("UnusedHours")) + posture["ri_realized_savings"] = _f(ri_util_total.get("RealizedSavings")) + if pct and pct < UTILIZATION_WARN_PCT: + posture["blockers"].append( + f"Existing reservations are only {pct:.1f}% utilized " + f"({_f(ri_util_total.get('UnusedHours')):,.0f} unused hours). " + "Reconcile before buying more." + ) + elif ri_utilization.get("error"): + posture["notes"].append( + f"RI utilization unavailable: {ri_utilization['error']}" + ) + + return posture + + +def reconcile_with_coh( + findings: list[Finding], coh: dict[str, Any] +) -> dict[str, Any]: + """Compare Cost Explorer totals against Cost Optimization Hub's. + + The two run independent pipelines over the same billing data, so a material + gap means one of them is looking at a different lookback or scope. Surfacing + the delta is what makes the report defensible rather than just plausible. + """ + if coh.get("error"): + return {"status": "unavailable", "reason": coh["error"]} + + coh_recs = coh.get("recommendations", []) + coh_total = round(sum(r["estimated_monthly_savings"] for r in coh_recs), 2) + # Compare against the API's own figures, not the risk-adjusted ones — COH + # publishes an unadjusted best case too, so that is the like-for-like axis. + ce_total = round(sum(f.api_monthly_savings for f in findings), 2) + + delta = round(ce_total - coh_total, 2) + larger = max(abs(ce_total), abs(coh_total)) + delta_pct = (abs(delta) / larger * 100) if larger > 0 else 0.0 + + if coh_total == 0 and ce_total == 0: + status = "agree-zero" + elif delta_pct <= 10: + status = "reconciled" + elif delta_pct <= 30: + status = "minor-variance" + else: + status = "material-variance" + + by_type: dict[str, float] = {} + for r in coh_recs: + key = r["recommended_resource_type"] or r["current_resource_type"] + by_type[key] = round( + by_type.get(key, 0.0) + r["estimated_monthly_savings"], 2 + ) + + return { + "status": status, + "ce_monthly_savings": ce_total, + "coh_monthly_savings": coh_total, + "delta": delta, + "delta_pct": round(delta_pct, 1), + "coh_count": len(coh_recs), + "coh_by_resource_type": dict( + sorted(by_type.items(), key=lambda kv: -kv[1]) + ), + } + + +def select_best_findings(findings: list[Finding]) -> list[Finding]: + """Pick the strongest permutation per commitment family. + + Every (term, payment) combination is fetched, but a report that lists all of + them buries the decision. Rank by risk-adjusted savings, breaking ties + toward the shorter term and less upfront cash — the lower-risk purchase. + """ + TERM_RANK = {"ONE_YEAR": 0, "THREE_YEARS": 1} + PAYMENT_RANK = {"NO_UPFRONT": 0, "PARTIAL_UPFRONT": 1, "ALL_UPFRONT": 2} + + best: dict[str, Finding] = {} + for f in findings: + key = f"{f.family}:{f.label}" + current = best.get(key) + if current is None: + best[key] = f + continue + if ( + -f.safe_monthly_savings, + TERM_RANK.get(f.term, 9), + PAYMENT_RANK.get(f.payment, 9), + ) < ( + -current.safe_monthly_savings, + TERM_RANK.get(current.term, 9), + PAYMENT_RANK.get(current.payment, 9), + ): + best[key] = f + + return sorted(best.values(), key=lambda f: -f.safe_monthly_savings) + + +# --------------------------------------------------------------------------- +# Expiry and renewal +# --------------------------------------------------------------------------- + + +def _urgency(days_remaining: int) -> str: + if days_remaining <= EXPIRY_URGENT_DAYS: + return "urgent" + if days_remaining <= EXPIRY_SOON_DAYS: + return "soon" + return "upcoming" + + +def _renewal_verdict( + item: dict[str, Any], utilization_pct: float | None +) -> tuple[str, str]: + """Decide what to do with one lapsing commitment, and say why. + + Utilization is the whole basis of the call: a commitment running at 99% is + load-bearing and lapsing it raises the bill, while one running at 30% is + already waste that renewal would lock in for another term. When + utilization could not be measured the honest answer is "review", not a + guess — the figure is what makes this decision defensible. + """ + label = item.get("label") or item.get("service") or "commitment" + if utilization_pct is None: + return ( + REVIEW, + f"Utilization for {label} could not be measured, so renewal size " + "cannot be justified from data. Check the console before the term " + "ends.", + ) + if utilization_pct >= UTILIZATION_WARN_PCT: + return ( + RENEW, + f"{label} is running at {utilization_pct:.1f}% utilization — fully " + "consumed. Letting it lapse moves this usage back to on-demand " + "rates.", + ) + if utilization_pct >= RENEW_LAPSE_PCT: + return ( + RENEW_SMALLER, + f"{label} is at {utilization_pct:.1f}% utilization, so part of the " + "commitment is unused. Expiry is a zero-cost resize point: re-buy " + "at roughly the utilized share, not the current size.", + ) + return ( + LET_LAPSE, + f"{label} is only at {utilization_pct:.1f}% utilization — more waste " + "than saving. Let it lapse and re-buy only what fresh sizing supports.", + ) + + +def analyze_expiry( + items: list[dict[str, Any]], + as_of: date, + horizon_days: int = EXPIRY_HORIZON_DAYS, + sp_utilization_pct: float | None = None, + ri_utilization_pct: float | None = None, +) -> dict[str, Any]: + """Sort a commitment inventory into what expires when, and what to do. + + `as_of` is a parameter rather than `date.today()` so this stays a pure + function that a test can pin to a fixed day. + + Utilization arrives per family from `assess_existing_posture` because Cost + Explorer only reports it in aggregate — there is no per-commitment + utilization API, so every Savings Plan in the account shares the account's + SP utilization figure. That is a real limitation of the data and is stated + in the report rather than papered over. + + Anything already past its end date but still listed as active is reported + separately: the commitment is gone and the spend it covered is already back + at on-demand rates, which is a different (and more urgent) conversation + than a renewal. + """ + expiring: list[dict[str, Any]] = [] + expired: list[dict[str, Any]] = [] + undated: list[dict[str, Any]] = [] + + for item in items: + end_raw = item.get("end") or "" + try: + end = date.fromisoformat(end_raw) + except ValueError: + undated.append(dict(item)) + continue + + days_remaining = (end - as_of).days + utilization = ( + sp_utilization_pct + if item.get("family") == "savings-plan" + else ri_utilization_pct + ) + action, rationale = _renewal_verdict(item, utilization) + entry = { + **item, + "days_remaining": days_remaining, + "utilization_pct": utilization, + "action": action, + "rationale": rationale, + } + if days_remaining < 0: + entry["urgency"] = "expired" + entry["rationale"] = ( + f"Already ended {abs(days_remaining)} days ago on {end_raw} but " + "still listed as active. Confirm whether the covered usage is " + "now billing at on-demand rates." + ) + expired.append(entry) + elif days_remaining <= horizon_days: + entry["urgency"] = _urgency(days_remaining) + expiring.append(entry) + + expiring.sort(key=lambda e: (e["days_remaining"], e.get("label", ""))) + expired.sort(key=lambda e: e["days_remaining"]) + + counts = {band: 0 for band in ("urgent", "soon", "upcoming")} + actions = {verdict: 0 for verdict in (RENEW, RENEW_SMALLER, LET_LAPSE, REVIEW)} + hourly_commitment = 0.0 + reserved_units = 0.0 + for entry in expiring: + counts[entry["urgency"]] = counts.get(entry["urgency"], 0) + 1 + actions[entry["action"]] = actions.get(entry["action"], 0) + 1 + if entry.get("family") == "savings-plan": + hourly_commitment += entry.get("quantity", 0.0) + else: + reserved_units += entry.get("quantity", 0.0) + + return { + "as_of": as_of.isoformat(), + "horizon_days": horizon_days, + "total_active": len(items), + "expiring": expiring, + "expired": expired, + "undated": undated, + "counts": counts, + "actions": actions, + # Savings Plan commitment is USD/hour, so it converts to money. RI + # quantities are unit counts and deliberately are NOT converted — + # turning units into dollars needs pricing data this module does not + # have, and inventing a rate would be a fabricated figure. + "hourly_commitment_expiring": round(hourly_commitment, 4), + "monthly_committed_spend_expiring": round( + hourly_commitment * HOURS_PER_MONTH, 2 + ), + "reserved_units_expiring": round(reserved_units, 2), + } diff --git a/src/lambda/mcp/commitments/commitments/api.py b/src/lambda/mcp/commitments/commitments/api.py new file mode 100644 index 0000000..0a2d6b9 --- /dev/null +++ b/src/lambda/mcp/commitments/commitments/api.py @@ -0,0 +1,865 @@ +"""Read-only AWS API wrappers for commitment (RI/SP) analysis. + +Every call in this module is a Get*/List*/Describe* operation. Nothing here +mutates state, purchases a commitment, or starts a billable analysis. + +Both Cost Explorer and Cost Optimization Hub are us-east-1-only APIs regardless +of where your resources live, so the clients are pinned there. +""" + +from __future__ import annotations + +from collections.abc import Callable +from dataclasses import dataclass +from datetime import date, datetime, timedelta, timezone +from typing import Any + +import boto3 +from botocore.exceptions import ClientError + +# Cost Explorer and Cost Optimization Hub are global services fronted only by +# us-east-1. Calling them in another region fails to resolve an endpoint. +CE_REGION = "us-east-1" +COH_REGION = "us-east-1" + +# Authoritative list, read back from the ValidationException the API itself +# raises on an unknown Service. Do not extend this by guessing service names — +# re-probe with an invalid value and copy the "Supported value(s)" list. +RI_SERVICES = ( + "Amazon Elastic Compute Cloud - Compute", + "Amazon Relational Database Service", + "Amazon Redshift", + "Amazon ElastiCache", + "Amazon Elasticsearch Service", + "Amazon OpenSearch Service", + "Amazon MemoryDB Service", + "Amazon DynamoDB Service", +) + +# Short labels for report headings, keyed by the API's Service string. +RI_SERVICE_LABELS = { + "Amazon Elastic Compute Cloud - Compute": "EC2", + "Amazon Relational Database Service": "RDS", + "Amazon Redshift": "Redshift", + "Amazon ElastiCache": "ElastiCache", + "Amazon Elasticsearch Service": "Elasticsearch (legacy)", + "Amazon OpenSearch Service": "OpenSearch", + "Amazon MemoryDB Service": "MemoryDB", + "Amazon DynamoDB Service": "DynamoDB", +} + +SP_TYPES = ("COMPUTE_SP", "EC2_INSTANCE_SP", "SAGEMAKER_SP", "DATABASE_SP") + +SP_TYPE_LABELS = { + "COMPUTE_SP": "Compute Savings Plan", + "EC2_INSTANCE_SP": "EC2 Instance Savings Plan", + "SAGEMAKER_SP": "SageMaker Savings Plan", + "DATABASE_SP": "Database Savings Plan", +} + +# Cost Optimization Hub resource types that represent a commitment purchase, +# used to reconcile CE recommendations against COH's independent pipeline. +COH_COMMITMENT_RESOURCE_TYPES = ( + "ComputeSavingsPlans", + "Ec2InstanceSavingsPlans", + "SageMakerSavingsPlans", + "Ec2ReservedInstances", + "RdsReservedInstances", + "OpenSearchReservedInstances", + "RedshiftReservedInstances", + "ElastiCacheReservedInstances", + "DynamoDbReservedCapacity", + "MemoryDbReservedInstances", +) + +HOURS_PER_MONTH = 730.0 + +# Rough month length, used only to label a term as 12 or 36 months from the +# elapsed start->end span. 365/12 rounds both real terms correctly. +DAYS_PER_MONTH = 30.4375 + + +@dataclass(frozen=True) +class SpecShape: + """Where one service hides the purchasable spec inside a CE line item. + + `GetReservationPurchaseRecommendation` returns a count and a savings figure + per line item, but the thing you actually buy — the instance class, the + Availability Zone, whether RDS is Multi-AZ — is buried in a + service-specific sub-structure under `InstanceDetails`. A recommendation + without it is not purchasable: "buy 4 RDS reservations" does not say + `db.r6g.xlarge Multi-AZ Aurora PostgreSQL`, and buying the wrong + combination yields a reservation that matches nothing. + + `attribute_fields` are ordered for reading, most decision-relevant first. + Field names come from the botocore CE model, including the two services + that break the pattern: OpenSearch/Elasticsearch splits its type across + `InstanceClass` + `InstanceSize`, and DynamoDB has no instance at all. + """ + + key: str + container: str + size_fields: tuple[str, ...] + attribute_fields: tuple[tuple[str, str], ...] = () + family_field: str | None = "Family" + region_field: str = "Region" + + +RECOMMENDATION_SPECS = ( + SpecShape( + key="EC2InstanceDetails", + container="InstanceDetails", + size_fields=("InstanceType",), + attribute_fields=( + ("AvailabilityZone", "AZ"), + ("Platform", "platform"), + ("Tenancy", "tenancy"), + ), + ), + SpecShape( + key="RDSInstanceDetails", + container="InstanceDetails", + size_fields=("InstanceType",), + attribute_fields=( + ("DeploymentOption", "deployment"), + ("DatabaseEngine", "engine"), + ("DatabaseEdition", "edition"), + ("LicenseModel", "license"), + ("DeploymentModel", "deployment model"), + ), + ), + SpecShape( + key="ElastiCacheInstanceDetails", + container="InstanceDetails", + size_fields=("NodeType",), + attribute_fields=(("ProductDescription", "engine"),), + ), + SpecShape( + key="RedshiftInstanceDetails", + container="InstanceDetails", + size_fields=("NodeType",), + ), + SpecShape( + key="MemoryDBInstanceDetails", + container="InstanceDetails", + size_fields=("NodeType",), + ), + SpecShape( + key="ESInstanceDetails", + container="InstanceDetails", + size_fields=("InstanceClass", "InstanceSize"), + family_field=None, + ), + SpecShape( + key="DynamoDBCapacityDetails", + container="ReservedCapacityDetails", + size_fields=(), + attribute_fields=(("CapacityUnits", "capacity units"),), + family_field=None, + ), +) + + +RECOMMENDATION_SPEC_KEYS = tuple(s.key for s in RECOMMENDATION_SPECS) + + +def _attribute_display(label: str, value: Any) -> str: + """Render one attribute for a compact spec string. + + A bare number tells a reader nothing, so numeric values carry their label + ("1000 capacity units"); named values already read as themselves + ("Multi-AZ", "Linux/UNIX") and are left alone. A missing value is dropped + rather than stringified — `str(None)` is the literal "None", which would + read as a real specification in a report. + """ + if value is None: + return "" + text = str(value).strip() + if not text: + return "" + return f"{text} {label}" if text.replace(".", "", 1).isdigit() else text + + +def _spec_label(size: str, attributes: dict[str, str], region: str = "") -> str: + """Join a spec into one line: what to buy, then where. + + Region goes last because it qualifies everything before it, and the whole + string has to survive being read inside a markdown table cell. + """ + parts = [size, *attributes.values(), region] + return " · ".join(p for p in (str(x).strip() for x in parts) if p) + + +def describe_recommendation_spec(detail: dict[str, Any]) -> dict[str, Any]: + """Extract the purchasable specification from one CE recommendation detail. + + Returns `{}` when no known sub-structure is present, so a caller can degrade + to the family-level figure instead of raising — a new AWS service appearing + under `InstanceDetails` should cost the report one column, not the run. + """ + for spec in RECOMMENDATION_SPECS: + raw = (detail.get(spec.container) or {}).get(spec.key) + if not raw: + continue + size = ".".join( + str(raw.get(f) or "").strip() for f in spec.size_fields + ).strip(".") + attributes = { + label: _attribute_display(label, raw.get(field)) + for field, label in spec.attribute_fields + if str(raw.get(field) or "").strip() + } + region = str(raw.get(spec.region_field) or "") + return { + "spec_key": spec.key, + "instance_type": size, + "family": str(raw.get(spec.family_field) or "") if spec.family_field else "", + "region": region, + "attributes": {k: v for k, v in attributes.items() if v}, + # Size flexibility decides whether the recommended size is binding: + # a size-flexible reservation can be bought at another size in the + # same family and still apply, and AWS reports it per line item. + "size_flex_eligible": bool(raw.get("SizeFlexEligible")), + "current_generation": bool(raw.get("CurrentGeneration")), + "label": _spec_label(size, attributes, region), + } + return {} + + +@dataclass(frozen=True) +class InventorySpec: + """How to list one reservation family and where its fields live. + + Every reservation API names the same six concepts differently and none of + them agrees with Savings Plans, so the differences are data rather than + seven near-identical functions. Field names here were read from the + botocore service models, not guessed — an `id_field` typo silently yields + commitments with no identifier. + + Only EC2 returns an explicit end date; everywhere else it must be derived + from `start_field` plus `Duration` (seconds). + + `attribute_fields` names the extras that decide *what a renewal has to + match*: an RDS reservation covers one deployment option and one engine, and + an EC2 one is pinned to an Availability Zone when its scope is zonal. Renew + against the wrong value and the discount silently does not apply, so these + travel with the instance type rather than being dropped. + """ + + key: str + label: str + service: str + method: str + response_key: str + id_field: str + count_field: str + type_field: str + start_field: str + end_field: str | None = None + arn_field: str | None = None + payment_field: str = "OfferingType" + attribute_fields: tuple[tuple[str, str], ...] = () + + +# Regional APIs, unlike the us-east-1-pinned Cost Explorer calls above: a +# reservation is only visible in the region that holds it. +RESERVATION_INVENTORY = ( + InventorySpec( + key="ec2", + label="EC2", + service="ec2", + method="describe_reserved_instances", + response_key="ReservedInstances", + id_field="ReservedInstancesId", + count_field="InstanceCount", + type_field="InstanceType", + start_field="Start", + end_field="End", + attribute_fields=( + ("Scope", "scope"), + ("AvailabilityZone", "AZ"), + ("ProductDescription", "platform"), + ("OfferingClass", "class"), + ("InstanceTenancy", "tenancy"), + ), + ), + InventorySpec( + key="rds", + label="RDS", + service="rds", + method="describe_reserved_db_instances", + response_key="ReservedDBInstances", + id_field="ReservedDBInstanceId", + count_field="DBInstanceCount", + type_field="DBInstanceClass", + start_field="StartTime", + arn_field="ReservedDBInstanceArn", + # MultiAZ is a bool on the wire; ProductDescription carries the engine + # ("aurora-postgresql", "postgresql"). Both are part of what an RDS or + # Aurora reservation matches against, so neither can be dropped. + attribute_fields=(("MultiAZ", "deployment"), ("ProductDescription", "engine")), + ), + InventorySpec( + key="elasticache", + label="ElastiCache", + service="elasticache", + method="describe_reserved_cache_nodes", + response_key="ReservedCacheNodes", + id_field="ReservedCacheNodeId", + count_field="CacheNodeCount", + type_field="CacheNodeType", + start_field="StartTime", + arn_field="ReservationARN", + attribute_fields=(("ProductDescription", "engine"),), + ), + InventorySpec( + key="redshift", + label="Redshift", + service="redshift", + method="describe_reserved_nodes", + response_key="ReservedNodes", + id_field="ReservedNodeId", + count_field="NodeCount", + type_field="NodeType", + start_field="StartTime", + attribute_fields=(("ReservedNodeOfferingType", "offering"),), + ), + InventorySpec( + key="opensearch", + label="OpenSearch", + service="opensearch", + method="describe_reserved_instances", + response_key="ReservedInstances", + id_field="ReservedInstanceId", + count_field="InstanceCount", + type_field="InstanceType", + start_field="StartTime", + payment_field="PaymentOption", + ), + InventorySpec( + key="memorydb", + label="MemoryDB", + service="memorydb", + method="describe_reserved_nodes", + response_key="ReservedNodes", + id_field="ReservationId", + count_field="NodeCount", + type_field="NodeType", + start_field="StartTime", + arn_field="ARN", + ), +) + +INVENTORY_SPECS = {spec.key: spec for spec in RESERVATION_INVENTORY} +INVENTORY_KEYS = tuple(INVENTORY_SPECS) + +# Fields whose raw value does not read as an attribute on its own. `MultiAZ` is +# the important one: printing "MultiAZ: False" invites a reader to skim past the +# single most expensive detail of an RDS or Aurora reservation, whereas +# "Single-AZ" states it. +INVENTORY_ATTRIBUTE_VALUES: dict[str, dict[Any, str]] = { + "MultiAZ": {True: "Multi-AZ", False: "Single-AZ"}, +} + +# DynamoDB reserved capacity is deliberately absent: there is no +# describe-reserved-capacity API on any SDK, so its expiry cannot be read. +# Cost Explorer can still *size* a DynamoDB reservation (RI_SERVICES above), +# it just cannot tell you when an existing one lapses. +INVENTORY_BLIND_SPOTS = ("DynamoDB reserved capacity (no describe API exists)",) + +# States that mean "this commitment is still costing or saving money". A +# retired/expired row is history, not something to renew. +ACTIVE_RESERVATION_STATES = ("active", "payment-pending", "pending", "retired-pending") +ACTIVE_SP_STATES = ("active", "payment-pending") + + +@dataclass(frozen=True) +class Clients: + """Immutable bundle of the read-only clients a collection run needs. + + `make_client` is the escape hatch for the regional inventory calls: unlike + Cost Explorer there is no single client that can answer them, so the host + supplies a factory instead of a fixed client. It is optional and defaults + to None, which makes expiry collection degrade to a reported warning rather + than an exception on hosts that do not grant the extra Describe* + permissions. + """ + + ce: Any + coh: Any + account_id: str + profile: str | None + make_client: Callable[[str, str], Any] | None = None + + +def build_clients(profile: str | None = None) -> Clients: + session = ( + boto3.Session(profile_name=profile) if profile else boto3.Session() + ) + sts = session.client("sts", region_name=CE_REGION) + return Clients( + ce=session.client("ce", region_name=CE_REGION), + coh=session.client("cost-optimization-hub", region_name=COH_REGION), + account_id=sts.get_caller_identity()["Account"], + profile=profile, + make_client=lambda service, region: session.client( + service, region_name=region + ), + ) + + +def _error(exc: ClientError) -> dict[str, str]: + """Normalize a ClientError into a reportable dict. + + Cost Explorer raises DataUnavailableException with an EMPTY message when an + account has no commitments of the requested kind. Substituting a readable + explanation here keeps that case from surfacing as a blank error in the + report. + """ + err = exc.response.get("Error", {}) + code = err.get("Code", "Unknown") + message = err.get("Message") or "" + if not message: + if code == "DataUnavailableException": + message = ( + "No data for this period — usually means no active commitment " + "of this type, or the account is too new to have billing data." + ) + else: + message = "(API returned no error message)" + return {"error_code": code, "error": message} + + +# -------------------------------------------------------------------------- +# Purchase recommendations +# -------------------------------------------------------------------------- + + +def get_sp_recommendation( + clients: Clients, + sp_type: str, + term: str, + payment: str, + lookback: str, + account_scope: str, +) -> dict[str, Any]: + """Fetch one Savings Plans purchase recommendation permutation.""" + try: + resp = clients.ce.get_savings_plans_purchase_recommendation( + SavingsPlansType=sp_type, + TermInYears=term, + PaymentOption=payment, + LookbackPeriodInDays=lookback, + AccountScope=account_scope, + PageSize=100, + ) + except ClientError as exc: + return {"sp_type": sp_type, "term": term, "payment": payment, **_error(exc)} + + rec = resp.get("SavingsPlansPurchaseRecommendation", {}) + meta = resp.get("Metadata", {}) + return { + "sp_type": sp_type, + "term": term, + "payment": payment, + "lookback": lookback, + "account_scope": account_scope, + "summary": rec.get("SavingsPlansPurchaseRecommendationSummary", {}), + "details": rec.get("SavingsPlansPurchaseRecommendationDetails", []), + "generated_at": meta.get("GenerationTimestamp", ""), + "recommendation_id": meta.get("RecommendationId", ""), + } + + +def get_ri_recommendation( + clients: Clients, + service: str, + term: str, + payment: str, + lookback: str, + account_scope: str, + offering_class: str = "STANDARD", +) -> dict[str, Any]: + """Fetch one Reserved Instance purchase recommendation permutation. + + ServiceSpecification/OfferingClass is EC2-only; sending it for RDS or + Redshift is rejected, so it is applied conditionally. + """ + params: dict[str, Any] = { + "Service": service, + "TermInYears": term, + "PaymentOption": payment, + "LookbackPeriodInDays": lookback, + "AccountScope": account_scope, + "PageSize": 100, + } + if service == "Amazon Elastic Compute Cloud - Compute": + params["ServiceSpecification"] = { + "EC2Specification": {"OfferingClass": offering_class} + } + + try: + resp = clients.ce.get_reservation_purchase_recommendation(**params) + except ClientError as exc: + return {"service": service, "term": term, "payment": payment, **_error(exc)} + + recs = resp.get("Recommendations", []) + meta = resp.get("Metadata", {}) + # One Recommendations entry per (term, payment, scope); details hold the + # per-instance-family line items. + summary = recs[0].get("RecommendationSummary", {}) if recs else {} + details = recs[0].get("RecommendationDetails", []) if recs else [] + return { + "service": service, + "label": RI_SERVICE_LABELS.get(service, service), + "term": term, + "payment": payment, + "lookback": lookback, + "account_scope": account_scope, + "offering_class": offering_class if "EC2" in service else None, + "summary": summary, + "details": details, + "generated_at": meta.get("GenerationTimestamp", ""), + "recommendation_id": meta.get("RecommendationId", ""), + } + + +# -------------------------------------------------------------------------- +# Existing-commitment posture: coverage and utilization +# -------------------------------------------------------------------------- + + +def _time_period(days: int) -> dict[str, str]: + end = date.today() + return {"Start": (end - timedelta(days=days)).isoformat(), "End": end.isoformat()} + + +def get_sp_coverage(clients: Clients, days: int) -> dict[str, Any]: + """Share of SP-eligible spend already covered by a Savings Plan.""" + try: + resp = clients.ce.get_savings_plans_coverage( + TimePeriod=_time_period(days), Granularity="MONTHLY" + ) + except ClientError as exc: + return _error(exc) + return {"periods": resp.get("SavingsPlansCoverages", [])} + + +def get_sp_utilization(clients: Clients, days: int) -> dict[str, Any]: + """How much of what you already committed to is actually being used.""" + try: + resp = clients.ce.get_savings_plans_utilization( + TimePeriod=_time_period(days), Granularity="MONTHLY" + ) + except ClientError as exc: + return _error(exc) + return { + "total": resp.get("Total", {}), + "periods": resp.get("SavingsPlansUtilizationsByTime", []), + } + + +def get_ri_coverage(clients: Clients, days: int) -> dict[str, Any]: + try: + resp = clients.ce.get_reservation_coverage( + TimePeriod=_time_period(days), Granularity="MONTHLY" + ) + except ClientError as exc: + return _error(exc) + return {"total": resp.get("Total", {}), "periods": resp.get("CoveragesByTime", [])} + + +def get_ri_utilization(clients: Clients, days: int) -> dict[str, Any]: + try: + resp = clients.ce.get_reservation_utilization( + TimePeriod=_time_period(days), Granularity="MONTHLY" + ) + except ClientError as exc: + return _error(exc) + return { + "total": resp.get("Total", {}), + "periods": resp.get("UtilizationsByTime", []), + } + + +# -------------------------------------------------------------------------- +# Cost Optimization Hub — independent second opinion for reconciliation +# -------------------------------------------------------------------------- + + +def get_coh_enrollment(clients: Clients) -> dict[str, Any]: + try: + resp = clients.coh.list_enrollment_statuses(includeOrganizationInfo=True) + except ClientError as exc: + return {"enrolled": False, **_error(exc)} + items = resp.get("items", []) + if not items: + return { + "enrolled": False, + "status": "NOT_ENROLLED", + "error": "Cost Optimization Hub is not enabled for this account.", + } + status = items[0].get("status", "Inactive") + return { + "enrolled": status == "Active", + "status": status, + "account_id": items[0].get("accountId", ""), + "include_member_accounts": resp.get("includeMemberAccounts", False), + } + + +def get_coh_commitment_recommendations(clients: Clients) -> dict[str, Any]: + """List only COH recommendations that are commitment purchases. + + Filtered to the commitment resource types so rightsizing/idle findings — + which the platform's cost-optimization-hub tool already surfaces — do not + dilute this report. + """ + try: + paginator = clients.coh.get_paginator("list_recommendations") + pages = paginator.paginate( + filter={ + "actionTypes": ["PurchaseSavingsPlans", "PurchaseReservedInstances"], + "resourceTypes": list(COH_COMMITMENT_RESOURCE_TYPES), + }, + includeAllRecommendations=True, + ) + items = [item for page in pages for item in page.get("items", [])] + except ClientError as exc: + return _error(exc) + + return { + "recommendations": [ + { + "recommendation_id": i.get("recommendationId", ""), + "account_id": i.get("accountId", ""), + "region": i.get("region", ""), + "current_resource_type": i.get("currentResourceType", ""), + "recommended_resource_type": i.get("recommendedResourceType", ""), + "action_type": i.get("actionType", ""), + "estimated_monthly_savings": i.get("estimatedMonthlySavings", 0) or 0, + "estimated_savings_percentage": i.get("estimatedSavingsPercentage", 0) + or 0, + "implementation_effort": i.get("implementationEffort", ""), + } + for i in items + ], + "count": len(items), + } + + +def get_eligible_spend(clients: Clients, days: int) -> dict[str, Any]: + """Monthly unblended spend by service, for sizing the opportunity. + + Used to state what share of the bill is even commitment-addressable, so a + "$0 savings" result can be distinguished from "no eligible spend". + """ + try: + resp = clients.ce.get_cost_and_usage( + TimePeriod=_time_period(days), + Granularity="MONTHLY", + Metrics=["UnblendedCost"], + GroupBy=[{"Type": "DIMENSION", "Key": "SERVICE"}], + ) + except ClientError as exc: + return _error(exc) + + periods = [] + for result in resp.get("ResultsByTime", []): + groups = { + g["Keys"][0]: float(g["Metrics"]["UnblendedCost"]["Amount"]) + for g in result.get("Groups", []) + } + periods.append( + { + "start": result["TimePeriod"]["Start"], + "end": result["TimePeriod"]["End"], + "total": round(sum(groups.values()), 2), + "by_service": groups, + } + ) + return {"periods": periods} + + +# --------------------------------------------------------------------------- +# Commitment inventory — the only calls here that are not Cost Explorer +# --------------------------------------------------------------------------- + + +def _as_datetime(value: Any) -> datetime | None: + """Coerce whatever an SDK or a CLI JSON dump handed us into UTC datetime. + + boto3 returns real datetimes; `aws ... --output json` returns ISO strings; + a hand-built stub may return a plain date. All three have to work, and an + unparseable value must return None rather than raise, because one odd row + must not lose the rest of the inventory. + """ + if value is None or value == "": + return None + if isinstance(value, datetime): + return value if value.tzinfo else value.replace(tzinfo=timezone.utc) + if isinstance(value, date): + return datetime(value.year, value.month, value.day, tzinfo=timezone.utc) + try: + parsed = datetime.fromisoformat(str(value).replace("Z", "+00:00")) + except ValueError: + return None + return parsed if parsed.tzinfo else parsed.replace(tzinfo=timezone.utc) + + +def _iso_day(moment: datetime | None) -> str: + return moment.date().isoformat() if moment else "" + + +def _term_months(start: datetime | None, end: datetime | None) -> int | None: + """Label the term from the elapsed span, in months.""" + if not start or not end: + return None + months = round((end - start).days / DAYS_PER_MONTH) + return months or None + + +def _reservation_attributes(spec: InventorySpec, row: dict) -> dict[str, str]: + """Read the match-critical extras for one reservation family. + + A bool has to be tested against None rather than truthiness: `MultiAZ: + False` is the meaningful value "Single-AZ", and dropping it because it is + falsy would leave the reader assuming Multi-AZ. + """ + attributes: dict[str, str] = {} + for field, label in spec.attribute_fields: + value = row.get(field) + if value is None or value == "": + continue + mapped = INVENTORY_ATTRIBUTE_VALUES.get(field, {}).get(value) + attributes[label] = mapped or _attribute_display(label, value) + return {k: v for k, v in attributes.items() if v} + + +def _normalize_reservation(spec: InventorySpec, row: dict, region: str) -> dict: + """Flatten one reservation row into the shape every family shares.""" + start = _as_datetime(row.get(spec.start_field)) + end = _as_datetime(row.get(spec.end_field)) if spec.end_field else None + if end is None: + duration = row.get("Duration") + if start and duration: + end = start + timedelta(seconds=int(duration)) + instance_type = str(row.get(spec.type_field) or "") + attributes = _reservation_attributes(spec, row) + return { + "family": "reservation", + "service": spec.key, + "label": spec.label, + "commitment_id": str(row.get(spec.id_field) or ""), + "arn": str(row.get(spec.arn_field) or "") if spec.arn_field else "", + "instance_type": instance_type, + "attributes": attributes, + # What a renewal has to match, in one string. Region is already a column + # of its own in the report, so it is left out here. + "spec": _spec_label(instance_type, attributes), + "quantity": float(row.get(spec.count_field) or 0), + "unit": "units", + "region": region, + "state": str(row.get("State") or ""), + "payment_option": str(row.get(spec.payment_field) or ""), + "start": _iso_day(start), + "end": _iso_day(end), + "term_months": _term_months(start, end), + } + + +def get_reservation_inventory( + clients: Clients, service: str, region: str +) -> dict[str, Any]: + """List one reservation family in one region, normalized. + + Returns `{"service", "region", "items"}` on success, or an `_error()` dict + carrying the same two keys so the caller can name the failed query. + """ + spec = INVENTORY_SPECS.get(service) + if spec is None: + raise ValueError( + f"Unknown reservation family {service!r}. " + f"Expected one of: {', '.join(INVENTORY_KEYS)}" + ) + if clients.make_client is None: + return { + "service": service, + "region": region, + "error": "No client factory configured — reservation inventory " + "needs regional Describe* access this host did not grant.", + } + try: + client = clients.make_client(spec.service, region) + resp = getattr(client, spec.method)() + except ClientError as exc: + return {"service": service, "region": region, **_error(exc)} + + items = [ + _normalize_reservation(spec, row, region) + for row in resp.get(spec.response_key, []) + if str(row.get("State", "")).lower() in ACTIVE_RESERVATION_STATES + ] + return {"service": service, "region": region, "items": items} + + +def get_savings_plan_inventory( + clients: Clients, region: str = CE_REGION +) -> dict[str, Any]: + """List active Savings Plans — account-level, so called once, not per region. + + Savings Plans are the one family that returns an explicit `end`, and the + only one whose commitment is denominated in dollars per hour rather than a + unit count. `region` selects the API endpoint, not a filter on the plans. + """ + if clients.make_client is None: + return { + "service": "savingsplans", + "region": region, + "error": "No client factory configured — Savings Plan inventory " + "needs savingsplans:DescribeSavingsPlans, which this host did not " + "grant.", + } + items: list[dict] = [] + try: + client = clients.make_client("savingsplans", region) + token: str | None = None + while True: + kwargs: dict[str, Any] = {"states": list(ACTIVE_SP_STATES)} + if token: + kwargs["nextToken"] = token + resp = client.describe_savings_plans(**kwargs) + for row in resp.get("savingsPlans", []): + start = _as_datetime(row.get("start")) + end = _as_datetime(row.get("end")) + # Only an EC2 Instance Savings Plan is pinned to a family and a + # region; a Compute plan commits to dollars and nothing else, so + # an empty spec here is correct rather than missing data. + instance_family = str(row.get("ec2InstanceFamily") or "") + items.append( + { + "family": "savings-plan", + "service": "savingsplans", + "label": f"{row.get('savingsPlanType') or 'Savings'} " + "Savings Plan", + "commitment_id": str(row.get("savingsPlanId") or ""), + "arn": str(row.get("savingsPlanArn") or ""), + "instance_type": instance_family, + "attributes": {}, + "spec": instance_family, + "quantity": float(row.get("commitment") or 0), + "unit": "USD/hour", + "region": str(row.get("region") or "") or "global", + "state": str(row.get("state") or ""), + "payment_option": str(row.get("paymentOption") or ""), + "start": _iso_day(start), + "end": _iso_day(end), + "term_months": _term_months(start, end), + } + ) + token = resp.get("nextToken") + if not token: + break + except ClientError as exc: + return {"service": "savingsplans", "region": region, **_error(exc)} + return {"service": "savingsplans", "region": region, "items": items} diff --git a/src/lambda/mcp/commitments/commitments/collect.py b/src/lambda/mcp/commitments/commitments/collect.py new file mode 100644 index 0000000..5f422ba --- /dev/null +++ b/src/lambda/mcp/commitments/commitments/collect.py @@ -0,0 +1,581 @@ +"""Parallel collection and envelope shaping — shared by every entrypoint. + +This module exists so that a caller only has to supply credentials. Everything +between "here are my clients" and "here is the finished payload" lives here: +the permutation sweep, the thread pools, the per-query error tolerance, and the +two output shapes (markdown payload, JSON envelope). + +Any host can drive the analysis with four lines:: + + from commitments import api, collect, report + clients = api.build_clients(profile=None) # or build api.Clients yourself + payload = collect.collect_all(clients) + print(report.render(payload)) + +Nothing here imports boto3, argparse, or reads the filesystem. Every AWS call +goes through the `api.Clients` record handed in, so a host that builds its +clients some other way (assumed role, injected stub, cross-account session) +gets the same pipeline without touching this file. +""" + +from __future__ import annotations + +import re +from collections.abc import Callable +from datetime import date, datetime, timezone +from typing import Any + +from .analyze import ( + EXPIRY_HORIZON_DAYS, + Finding, + LineItem, + analyze_expiry, + analyze_ri_recommendation, + analyze_sp_recommendation, + assess_existing_posture, + reconcile_with_coh, + select_best_findings, +) +from .api import ( + INVENTORY_BLIND_SPOTS, + INVENTORY_KEYS, + RI_SERVICE_LABELS, + RI_SERVICES, + SP_TYPE_LABELS, + SP_TYPES, + Clients, + get_coh_commitment_recommendations, + get_coh_enrollment, + get_eligible_spend, + get_reservation_inventory, + get_ri_coverage, + get_ri_recommendation, + get_ri_utilization, + get_savings_plan_inventory, + get_sp_coverage, + get_sp_recommendation, + get_sp_utilization, +) + +# Cost Explorer throttles aggressively on the recommendation APIs; 6 keeps a +# full permutation sweep inside the rate limit while still finishing in +# reasonable wall-clock time (and inside a 300s Lambda timeout). +MAX_WORKERS = 6 + +TERMS = ("ONE_YEAR", "THREE_YEARS") +PAYMENTS = ("NO_UPFRONT", "PARTIAL_UPFRONT", "ALL_UPFRONT") +LOOKBACKS = ("SEVEN_DAYS", "THIRTY_DAYS", "SIXTY_DAYS") +ACCOUNT_SCOPES = ("PAYER", "LINKED") +FAMILIES = ("sp", "ri") + +# Defaults chosen to keep a first run cheap: Cost Explorer bills $0.01 per +# recommendation request, so evaluating both terms against the two payment +# extremes (rather than all three) halves the sweep without losing the +# no-upfront/all-upfront spread that drives the break-even discussion. +DEFAULT_TERMS = ("ONE_YEAR", "THREE_YEARS") +DEFAULT_PAYMENTS = ("NO_UPFRONT", "ALL_UPFRONT") +DEFAULT_LOOKBACK = "THIRTY_DAYS" +DEFAULT_ACCOUNT_SCOPE = "PAYER" +DEFAULT_POSTURE_DAYS = 30 +DEFAULT_SPEND_DAYS = 60 +DEFAULT_EXPIRY_HORIZON_DAYS = EXPIRY_HORIZON_DAYS + +# Region names reach the AWS SDK as an endpoint component, so they are checked +# against the shape AWS actually uses rather than passed through. Caller input +# that is not region-shaped is a caller error, not something to send onward. +REGION_PATTERN = re.compile(r"^[a-z]{2}(-[a-z]+)+-\d$") + + +# --------------------------------------------------------------------------- +# Parameter resolution +# --------------------------------------------------------------------------- + + +def resolve_ri_services(tokens: list[str]) -> list[str]: + """Resolve RI service tokens to Cost Explorer service names. + + Accepts the full API name or the short label, case-insensitively. Labels + carrying a parenthetical qualifier ("Elasticsearch (legacy)") also match on + their base word alone, since nobody types the parentheses. Duplicates are + dropped: Cost Explorer bills per recommendation request, so asking for + "EC2, ec2" must not pay twice for the same answer. + + Raises `ValueError` on an unknown token — callers translate that into + whatever their host expects (a CLI `SystemExit`, a tool error envelope). + """ + if len(tokens) == 1 and tokens[0].strip().lower() == "all": + return list(RI_SERVICES) + + by_label: dict[str, str] = {} + for service, label in RI_SERVICE_LABELS.items(): + by_label[label.lower()] = service + by_label.setdefault(label.split("(")[0].strip().lower(), service) + by_name = {s.lower(): s for s in RI_SERVICES} + + resolved: list[str] = [] + for raw in tokens: + key = raw.strip().lower() + if not key: + continue + match = by_name.get(key) or by_label.get(key) + if match is None: + raise ValueError( + f"Unknown RI service {raw.strip()!r}. Valid short labels: " + + ", ".join(sorted(RI_SERVICE_LABELS.values())) + ) + if match not in resolved: + resolved.append(match) + return resolved + + +def resolve_sp_types(tokens: list[str]) -> list[str]: + """Resolve Savings Plan type tokens, case-insensitively. `ValueError` on miss. + + Duplicates are dropped for the same reason as `resolve_ri_services`. + """ + if len(tokens) == 1 and tokens[0].strip().lower() == "all": + return list(SP_TYPES) + + upper = [t.strip().upper() for t in tokens if t.strip()] + unknown = [t for t in upper if t not in SP_TYPES] + if unknown: + raise ValueError( + f"Unknown savings plan type {unknown[0]!r}. Choose from " + + ", ".join(SP_TYPES) + ) + return list(dict.fromkeys(upper)) + + +def validate_choices(values: list[str], allowed: tuple[str, ...], label: str) -> list[str]: + """Return *values* unchanged, or raise `ValueError` naming the first bad one.""" + for value in values: + if value not in allowed: + raise ValueError( + f"Invalid {label} {value!r}. Choose from {', '.join(allowed)}." + ) + return values + + +# --------------------------------------------------------------------------- +# Parallel collection +# --------------------------------------------------------------------------- + + +def run_jobs(jobs: list[tuple[str, Callable[[], dict]]]) -> tuple[list[dict], list[dict]]: + """Run labelled callables in parallel, tolerating per-query failure. + + One throttled or unauthorized permutation must not lose the other 40, so + failures are collected as warnings and the report is reported as a lower + bound rather than aborted. + """ + # Imported lazily so that importing this module costs nothing on hosts that + # only want the pure helpers (serialize_finding, envelope). + from concurrent.futures import ThreadPoolExecutor + + results: list[dict] = [] + errors: list[dict] = [] + if not jobs: + return results, errors + + with ThreadPoolExecutor(max_workers=MAX_WORKERS) as pool: + futures = [(label, pool.submit(fn)) for label, fn in jobs] + for label, future in futures: + try: + result = future.result() + except Exception as exc: # noqa: BLE001 - one bad query must not abort + errors.append({"query": label, "error": str(exc)}) + continue + if result.get("error"): + errors.append( + { + "query": label, + "error_code": result.get("error_code", ""), + "error": result["error"], + } + ) + continue + results.append(result) + return results, errors + + +def sweep_savings_plans( + clients: Clients, + sp_types: list[str], + terms: list[str], + payments: list[str], + lookback: str, + account_scope: str, +) -> tuple[list[dict], list[dict]]: + """Query every (type × term × payment) SP permutation. Returns (recs, errors).""" + jobs = [ + ( + f"SP {SP_TYPE_LABELS.get(sp_type, sp_type)} {term} {payment}", + lambda s=sp_type, t=term, p=payment: get_sp_recommendation( + clients, s, t, p, lookback, account_scope + ), + ) + for sp_type in sp_types + for term in terms + for payment in payments + ] + return run_jobs(jobs) + + +def sweep_reservations( + clients: Clients, + services: list[str], + terms: list[str], + payments: list[str], + lookback: str, + account_scope: str, +) -> tuple[list[dict], list[dict]]: + """Query every (service × term × payment) RI permutation. Returns (recs, errors).""" + jobs = [ + ( + f"RI {RI_SERVICE_LABELS.get(service, service)} {term} {payment}", + lambda s=service, t=term, p=payment: get_ri_recommendation( + clients, s, t, p, lookback, account_scope + ), + ) + for service in services + for term in terms + for payment in payments + ] + return run_jobs(jobs) + + +def collect_posture( + clients: Clients, + posture_days: int = DEFAULT_POSTURE_DAYS, + spend_days: int = DEFAULT_SPEND_DAYS, +) -> dict: + """Fetch existing coverage/utilization, COH enrollment, and eligible spend. + + Every key is always present; a failed query lands as `{"error": ...}` in its + own slot so the caller can report partial posture instead of nothing. + """ + from concurrent.futures import ThreadPoolExecutor + + jobs: dict[str, Callable[[], dict]] = { + "sp_coverage": lambda: get_sp_coverage(clients, posture_days), + "sp_utilization": lambda: get_sp_utilization(clients, posture_days), + "ri_coverage": lambda: get_ri_coverage(clients, posture_days), + "ri_utilization": lambda: get_ri_utilization(clients, posture_days), + "coh_enrollment": lambda: get_coh_enrollment(clients), + "eligible_spend": lambda: get_eligible_spend(clients, spend_days), + } + collected: dict[str, dict] = {} + with ThreadPoolExecutor(max_workers=MAX_WORKERS) as pool: + submitted = {key: pool.submit(fn) for key, fn in jobs.items()} + for key, future in submitted.items(): + try: + collected[key] = future.result() + except Exception as exc: # noqa: BLE001 + collected[key] = {"error": str(exc)} + return collected + + +def resolve_regions(tokens: list[str] | None, default_region: str = "") -> list[str]: + """Validate and de-duplicate a caller-supplied region list. + + Reservations are regional, so a sweep needs an explicit list; Savings Plans + are account-level and are fetched once regardless. Defaults to the single + region the host itself runs in, because widening the sweep multiplies API + calls and a caller who wants org-wide coverage should say so. + """ + values = [t.strip().lower() for t in (tokens or []) if t and t.strip()] + if not values and default_region: + values = [default_region.strip().lower()] + if not values: + raise ValueError("At least one AWS region is required for expiry inventory.") + bad = [v for v in values if not REGION_PATTERN.match(v)] + if bad: + raise ValueError( + f"Not a valid AWS region name: {', '.join(bad)}. " + "Expected e.g. us-east-1, ap-northeast-1." + ) + return list(dict.fromkeys(values)) + + +def collect_expiry( + clients: Clients, + regions: list[str], + services: list[str] | None = None, + horizon_days: int = DEFAULT_EXPIRY_HORIZON_DAYS, + *, + sp_utilization_pct: float | None = None, + ri_utilization_pct: float | None = None, + as_of: date | None = None, +) -> tuple[dict[str, Any], list[dict]]: + """Inventory every commitment that carries a date, and judge its renewal. + + One job per (family, region) plus one un-multiplied Savings Plans job — the + SP API is account-level, so sweeping it per region would return the same + plans N times and quietly inflate every total. + """ + families = [s.strip().lower() for s in (services or list(INVENTORY_KEYS)) if s.strip()] + validate_choices(families, INVENTORY_KEYS, "reservation family") + + jobs: list[tuple[str, Callable[[], dict]]] = [ + ( + f"{family} reservations ({region})", + lambda f=family, r=region: get_reservation_inventory(clients, f, r), + ) + for region in regions + for family in families + ] + jobs.append(("savings plan inventory", lambda: get_savings_plan_inventory(clients))) + + results, errors = run_jobs(jobs) + items = [item for result in results for item in result.get("items", [])] + expiry = analyze_expiry( + items, + as_of or datetime.now(timezone.utc).date(), + horizon_days, + sp_utilization_pct=sp_utilization_pct, + ri_utilization_pct=ri_utilization_pct, + ) + expiry["regions"] = list(regions) + expiry["blind_spots"] = list(INVENTORY_BLIND_SPOTS) + return expiry, errors + + +def fetch_coh(clients: Clients, coh_enrollment: dict) -> dict: + """Fetch COH recommendations, or an error dict explaining why we cannot. + + Calling `ListRecommendations` while unenrolled returns an unhelpful access + error, so enrollment is checked first and the reason is passed through to + the report instead. + """ + if coh_enrollment.get("enrolled"): + return get_coh_commitment_recommendations(clients) + reason = coh_enrollment.get("error") or coh_enrollment.get("status", "not enrolled") + return {"error": f"Cost Optimization Hub not available: {reason}"} + + +# --------------------------------------------------------------------------- +# Output shaping +# --------------------------------------------------------------------------- + + +def findings_from(sp_recs: list[dict], ri_recs: list[dict]) -> list[Finding]: + """Turn raw recommendation responses into risk-adjusted findings.""" + findings = [ + f for f in (analyze_sp_recommendation(r) for r in sp_recs) if f is not None + ] + findings += [ + f for f in (analyze_ri_recommendation(r) for r in ri_recs) if f is not None + ] + return findings + + +def serialize_line_item(item: LineItem) -> dict: + """Flatten one purchasable line of a recommendation. + + A consumer that only reads the finding-level total cannot act on it: a + reservation applies to usage matching its exact specification, so `spec` and + `region` are what a purchase is actually placed against. + """ + return { + "spec": item.spec, + "region": item.region, + "commitment_unit": item.unit, + "aws_recommended_commitment": round(item.recommended, 4), + "achievable_commitment": round(item.achievable, 4), + "minimum_observed_units": round(item.floor, 4), + "average_observed_units": round(item.average, 4), + "estimated_monthly_savings": round(item.monthly_savings, 2), + "upfront_cost": round(item.upfront_cost, 2), + "monthly_on_demand_cost": round(item.monthly_on_demand, 2), + "estimated_utilization_percentage": ( + round(item.utilization_pct, 2) if item.utilization_pct is not None else None + ), + "size_flexible": item.size_flex_eligible, + "current_generation": item.current_generation, + "account_id": item.account_id, + } + + +def serialize_finding(f: Finding) -> dict: + """Flatten a Finding into a flat recommendation dict. + + Key names mirror what an AWS Cost Optimization Hub style recommendation + list looks like, so a host that already renders those needs no translation + layer. + """ + return { + "commitment_family": f.family, + "commitment_type": f.label, + "term": f.term, + "payment_option": f.payment, + "aws_recommended_commitment": round(f.api_hourly_commitment, 4), + "achievable_commitment": round(f.safe_hourly_commitment, 4), + "commitment_unit": "USD/hour" if f.family == "savings-plan" else "units", + "estimated_monthly_savings": round(f.safe_monthly_savings, 2), + "aws_best_case_monthly_savings": round(f.api_monthly_savings, 2), + "estimated_savings_percentage": round(f.savings_percentage, 2), + "upfront_cost": round(f.upfront_cost, 2), + "break_even_months": ( + round(f.break_even_months, 1) if f.break_even_months else None + ), + "waste_exposure_monthly": round(f.waste_exposure_monthly, 2), + "confidence": f.confidence, + "spend_profile": f.volatility, + "implementation_effort": "Medium", + "rationale": f.rationale, + "line_items": [serialize_line_item(i) for i in f.line_items], + } + + +def envelope(findings: list[Finding], reconciliation: dict | None = None) -> dict: + """Build the JSON response shape: recommendations plus roll-up totals.""" + payload = { + "recommendations": [serialize_finding(f) for f in findings], + "count": len(findings), + "total_estimated_monthly_savings": round( + sum(f.safe_monthly_savings for f in findings), 2 + ), + "aws_best_case_monthly_savings": round( + sum(f.api_monthly_savings for f in findings), 2 + ), + } + if reconciliation is not None: + payload["reconciliation"] = reconciliation + return payload + + +def build_meta( + clients: Clients, + lookback: str, + account_scope: str, + profile: str | None = None, +) -> dict: + """Provenance block for the rendered report.""" + return { + "account_id": clients.account_id, + "profile": profile, + "generated_at": datetime.now(timezone.utc).strftime("%Y-%m-%d %H:%M UTC"), + "lookback": lookback, + "account_scope": account_scope, + } + + +# --------------------------------------------------------------------------- +# Full pipeline +# --------------------------------------------------------------------------- + + +def collect_all( + clients: Clients, + *, + families: tuple[str, ...] | list[str] = FAMILIES, + sp_types: list[str] | None = None, + ri_services: list[str] | None = None, + terms: list[str] | None = None, + payments: list[str] | None = None, + lookback: str = DEFAULT_LOOKBACK, + account_scope: str = DEFAULT_ACCOUNT_SCOPE, + posture_days: int = DEFAULT_POSTURE_DAYS, + spend_days: int = DEFAULT_SPEND_DAYS, + regions: list[str] | None = None, + inventory_services: list[str] | None = None, + expiry_horizon_days: int = DEFAULT_EXPIRY_HORIZON_DAYS, + profile: str | None = None, +) -> dict[str, Any]: + """Run the whole pipeline and return the payload `report.render` expects. + + Sweeps recommendations, fetches existing posture, reconciles against Cost + Optimization Hub, and keeps the best finding per (family, term, payment). + Every parameter is validated here, so a host can pass user input straight + through and catch `ValueError`. + + The returned dict carries `meta`, `findings`, `posture`, `reconciliation`, + `eligible_spend`, `expiry`, `errors`, `sweep_errors` — plus `raw` for a + caller that wants the unanalyzed responses. + + `regions` is opt-in: expiry inventory needs regional Describe* permissions + beyond the Cost Explorer set, so passing nothing leaves `expiry` as None + and makes no extra calls rather than failing a caller that only granted + `ce:Get*`. + """ + families = [f.strip().lower() for f in families if f.strip()] + validate_choices(families, FAMILIES, "family") + terms = validate_choices(list(terms or DEFAULT_TERMS), TERMS, "term") + payments = validate_choices( + list(payments or DEFAULT_PAYMENTS), PAYMENTS, "payment option" + ) + validate_choices([lookback], LOOKBACKS, "lookback") + validate_choices([account_scope], ACCOUNT_SCOPES, "account scope") + if regions: + regions = resolve_regions(regions) + + sp_recs: list[dict] = [] + ri_recs: list[dict] = [] + errors: list[dict] = [] + + if "sp" in families: + recs, errs = sweep_savings_plans( + clients, + resolve_sp_types(sp_types or ["all"]), + terms, + payments, + lookback, + account_scope, + ) + sp_recs, errors = recs, errors + errs + + if "ri" in families: + recs, errs = sweep_reservations( + clients, + resolve_ri_services(ri_services or ["all"]), + terms, + payments, + lookback, + account_scope, + ) + ri_recs, errors = recs, errors + errs + + posture_data = collect_posture(clients, posture_days, spend_days) + coh = fetch_coh(clients, posture_data["coh_enrollment"]) + + best = select_best_findings(findings_from(sp_recs, ri_recs)) + posture = assess_existing_posture( + posture_data["sp_coverage"], + posture_data["sp_utilization"], + posture_data["ri_coverage"], + posture_data["ri_utilization"], + ) + reconciliation = reconcile_with_coh(best, coh) + + # Reuses the utilization already measured above rather than re-querying: + # Cost Explorer has no per-commitment utilization API, so the account-level + # figure is the only one there is, and it is what drives the renewal call. + # Kept separate from `errors` because a caller counting queries is counting + # BILLABLE ones — Cost Explorer charges $0.01 per recommendation request, + # while the inventory Describe* calls are free. Folding expiry failures into + # that count would overstate the bill. + sweep_errors = list(errors) + + expiry = None + if regions: + expiry, expiry_errors = collect_expiry( + clients, + regions, + inventory_services, + expiry_horizon_days, + sp_utilization_pct=posture.get("sp_utilization_pct"), + ri_utilization_pct=posture.get("ri_utilization_pct"), + ) + errors = errors + expiry_errors + + return { + "meta": build_meta(clients, lookback, account_scope, profile), + "findings": best, + "posture": posture, + "reconciliation": reconciliation, + "eligible_spend": posture_data["eligible_spend"], + "expiry": expiry, + "errors": errors, + "sweep_errors": sweep_errors, + "raw": {"sp_recs": sp_recs, "ri_recs": ri_recs, "coh": coh, **posture_data}, + } diff --git a/src/lambda/mcp/commitments/commitments/report.py b/src/lambda/mcp/commitments/commitments/report.py new file mode 100644 index 0000000..d1b2950 --- /dev/null +++ b/src/lambda/mcp/commitments/commitments/report.py @@ -0,0 +1,645 @@ +"""Render the commitment analysis as a markdown report. + +Report order is deliberate: posture blockers come before recommendations, so a +reader cannot skim the savings number without first seeing that existing +commitments are under-utilized. +""" + +from __future__ import annotations + +from typing import Any + +from .analyze import TERM_MONTHS, Finding, LineItem + +TERM_LABELS = {"ONE_YEAR": "1-year", "THREE_YEARS": "3-year"} +PAYMENT_LABELS = { + "NO_UPFRONT": "No upfront", + "PARTIAL_UPFRONT": "Partial upfront", + "ALL_UPFRONT": "All upfront", +} +LOOKBACK_LABELS = { + "SEVEN_DAYS": "7 days", + "THIRTY_DAYS": "30 days", + "SIXTY_DAYS": "60 days", +} +CONFIDENCE_MARKS = {"High": "High", "Medium": "Medium", "Low": "Low"} + +RECONCILE_VERDICTS = { + "reconciled": ( + "Reconciled — Cost Explorer and Cost Optimization Hub agree within 10%. " + "Figures below are defensible against the console." + ), + "minor-variance": ( + "Minor variance — the two AWS pipelines differ by 10-30%. Usually a " + "lookback-window difference; state the CE figure and note the spread." + ), + "material-variance": ( + "MATERIAL VARIANCE — the two AWS pipelines differ by more than 30%. Do " + "not quote a single number until the cause is identified (commonly a " + "different account scope, or COH data lagging a recent usage change)." + ), + "agree-zero": ( + "Both pipelines report no commitment opportunity. Consistent, and the " + "eligible-spend section below shows why." + ), + "unavailable": ( + "Not reconciled — Cost Optimization Hub could not be queried, so these " + "figures rest on Cost Explorer alone." + ), +} + + +def _money(value: float) -> str: + return f"${value:,.2f}" + + +def _pct(value: float) -> str: + return f"{value:.1f}%" + + +def _commitment_str(f: Finding, value: float) -> str: + if f.family == "savings-plan": + return f"${value:,.4f}/hr" + return f"{value:,.0f} unit(s)" + + +def _size_str(item: dict[str, Any]) -> str: + """Format a commitment's size in its own unit. + + Savings Plans commit in dollars per hour and reservations in unit counts. + Printing an hourly-dollar commitment as a bare number misreads it by + roughly 1000x, so the unit is never dropped. + """ + quantity = item.get("quantity", 0.0) + if item.get("unit") == "USD/hour": + return f"${quantity:,.4f}/hr" + return f"{quantity:,.0f} unit(s)" + + +ACTION_LABELS = { + "renew": "**renew**", + "renew-smaller": "**renew smaller**", + "let-lapse": "let lapse", + "review": "review", +} + + +def _line_quantity(value: float, unit: str) -> str: + return f"${value:,.4f}/hr" if unit == "USD/hour" else f"{value:,.0f}" + + +def _line_spec(item: LineItem) -> str: + """The spec cell, with the two flags that change what you should buy. + + Size flexibility means the recommended size is not binding — the discount + follows any size in the family. A previous-generation instance is the + opposite kind of signal: committing to one for three years locks the account + out of the cheaper current generation for the whole term. + """ + flags = [] + if item.size_flex_eligible: + flags.append("size-flexible") + if not item.current_generation: + flags.append("**previous generation**") + return item.spec + (f" ({', '.join(flags)})" if flags else "") + + +def _line_items_table(add: Any, f: Finding) -> None: + """Render what to actually buy, one row per purchasable specification. + + The family total above is the ranking figure; a reservation applies only to + usage matching its exact instance type, deployment option and Availability + Zone, so without this table the recommendation cannot be acted on. + """ + items = [i for i in f.line_items if i.recommended > 0] + if not items: + return + add("Line items — what to buy:") + add("") + add( + "| Buy | Region | AWS units | Floor | Achievable | Utilization " + "| Savings/mo |" + ) + add("|---|---|---:|---:|---:|---:|---:|") + for item in items: + util = ( + _pct(item.utilization_pct) + if item.utilization_pct is not None + else "unknown" + ) + add( + f"| {_line_spec(item)} | {item.region or '—'} " + f"| {_line_quantity(item.recommended, item.unit)} " + f"| {_line_quantity(item.floor, item.unit)} " + f"| {_line_quantity(item.achievable, item.unit)} " + f"| {util} | {_money(item.monthly_savings)} |" + ) + add("") + if f.family == "reserved-instance": + allocated = sum(i.achievable for i in items) + # Each line rounds down to a whole reservation, so the rounding losses + # accumulate. Saying which line absorbs the remainder is what keeps the + # table addable to the headline figure. + if allocated < f.safe_hourly_commitment: + shortfall = f.safe_hourly_commitment - allocated + add( + f"Rounding each line down to whole reservations leaves " + f"{shortfall:,.0f} unit(s) unallocated against the " + f"{f.safe_hourly_commitment:,.0f} achievable total — add them to " + "the line with the highest floor." + ) + add("") + add( + "*Floor* is the count that line never dropped below during the " + "lookback, so it is the part of the recommendation that carries no " + "unused-commitment risk." + ) + add("") + + +def _expiry_section(add: Any, expiry: dict[str, Any]) -> None: + """Render the expiry/renewal section. + + Deliberately placed between existing-commitment health and the new-purchase + recommendations: a commitment lapsing in three weeks is a decision with a + deadline, and it belongs ahead of an optional purchase. + """ + add("## Commitment expiry and renewal") + add("") + regions = ", ".join(expiry.get("regions", [])) or "(none)" + add( + f"Inventory taken {expiry['as_of']} over a {expiry['horizon_days']}-day " + f"horizon. Reservation regions swept: {regions}. Savings Plans are " + "account-level and are listed once regardless of region." + ) + add("") + + expiring = expiry.get("expiring") or [] + expired = expiry.get("expired") or [] + counts = expiry.get("counts", {}) + + if not expiring: + add( + f"**No commitment expires within {expiry['horizon_days']} days.** " + f"{expiry.get('total_active', 0)} active commitment(s) were " + "inventoried." + ) + add("") + else: + add( + f"**{len(expiring)} commitment(s) expire within " + f"{expiry['horizon_days']} days** — {counts.get('urgent', 0)} within " + f"30 days, {counts.get('soon', 0)} within 60, " + f"{counts.get('upcoming', 0)} within 90." + ) + add("") + monthly = expiry.get("monthly_committed_spend_expiring", 0.0) + units = expiry.get("reserved_units_expiring", 0.0) + exposure = [] + if monthly: + exposure.append( + f"{_money(monthly)}/mo of committed Savings Plan spend " + f"({_money(expiry.get('hourly_commitment_expiring', 0.0))}/hr) " + "reverts to on-demand rates if not renewed" + ) + if units: + exposure.append( + f"{units:,.0f} reserved unit(s) lose their discount. The dollar " + "value of that is not stated because it needs per-instance " + "pricing this analysis does not query" + ) + if exposure: + add("Exposure: " + "; ".join(exposure) + ".") + add("") + + add( + "| Ends | Days | Commitment | Spec | Size | Region | Utilization " + "| Action |" + ) + add("|---|---:|---|---|---:|---|---:|---|") + for item in expiring: + util = item.get("utilization_pct") + util_str = _pct(util) if util is not None else "unknown" + ident = item.get("commitment_id") or "(no id)" + add( + f"| {item.get('end', '')} | {item['days_remaining']} " + f"| {item.get('label', '')} `{ident}` " + f"| {item.get('spec') or '—'} " + f"| {_size_str(item)} | {item.get('region', '')} " + f"| {util_str} " + f"| {ACTION_LABELS.get(item['action'], item['action'])} |" + ) + add("") + add( + "> Utilization is the account-level figure from Cost Explorer, not " + "per-commitment — there is no API that reports utilization for an " + "individual Savings Plan or reservation. Treat it as the portfolio " + "signal it is, and confirm a specific commitment in the console " + "before acting." + ) + add("") + add( + "> *Spec* is what a renewal has to match. A reservation bought " + "against a different instance class, deployment option (Single-AZ " + "vs Multi-AZ) or engine does not cover the same usage, so a renewal " + "that changes any of these is a new purchase and needs fresh " + "sizing. An empty spec on a Compute Savings Plan is correct — it " + "commits to dollars, not to a family." + ) + add("") + add("### Why") + add("") + for item in expiring: + ident = item.get("commitment_id") or item.get("label", "commitment") + add(f"- `{ident}` — {item['rationale']}") + add("") + + if expired: + add("### Already ended but still listed as active") + add("") + for item in expired: + ident = item.get("commitment_id") or item.get("label", "commitment") + spec = item.get("spec") + described = f"{item.get('label', '')} {spec}".strip() if spec else item.get( + "label", "" + ) + add(f"- `{ident}` ({described}) — {item['rationale']}") + add("") + + undated = expiry.get("undated") or [] + if undated: + add( + f"{len(undated)} commitment(s) returned no usable end date and could " + "not be assessed: " + + ", ".join( + f"`{i.get('commitment_id') or i.get('label', '?')}`" for i in undated + ) + + "." + ) + add("") + + blind_spots = expiry.get("blind_spots") or [] + if blind_spots: + add("Not covered by this inventory: " + "; ".join(blind_spots) + ".") + add("") + + +def render(data: dict[str, Any]) -> str: + """Build the full markdown report from a collected+analyzed payload.""" + meta = data["meta"] + findings: list[Finding] = data["findings"] + posture = data["posture"] + recon = data["reconciliation"] + spend = data["eligible_spend"] + errors = data.get("errors", []) + + out: list[str] = [] + add = out.append + + # ---------------------------------------------------------------- header + add("# AWS Discounted Commitments Report") + add("") + add(f"**Account:** {meta['account_id']}") + if meta.get("profile"): + add(f"**Profile:** `{meta['profile']}`") + add(f"**Generated:** {meta['generated_at']}") + add( + f"**Lookback:** {LOOKBACK_LABELS.get(meta['lookback'], meta['lookback'])}" + f"  |  **Account scope:** {meta['account_scope']}" + ) + add( + "**Source APIs:** Cost Explorer (`GetSavingsPlansPurchaseRecommendation`, " + "`GetReservationPurchaseRecommendation`, coverage + utilization), " + "Cost Optimization Hub (`ListRecommendations`)" + ) + add("") + add("All data is read-only. This report does not purchase anything.") + add("") + + # -------------------------------------------------------- bottom line + total_api = sum(f.api_monthly_savings for f in findings) + total_safe = sum(f.safe_monthly_savings for f in findings) + high_conf = [f for f in findings if f.confidence == "High"] + total_high = sum(f.safe_monthly_savings for f in high_conf) + + add("## Bottom line") + add("") + # An expiry inside 30 days outranks a purchase recommendation: it has a + # deadline attached, and missing it silently raises the bill. + urgent = (data.get("expiry") or {}).get("counts", {}).get("urgent", 0) + if urgent: + add( + f"**{urgent} existing commitment(s) expire within 30 days.** That " + "deadline comes before any new purchase — see *Commitment expiry " + "and renewal*." + ) + add("") + if not findings: + add( + "**No commitment opportunity found.** Neither Cost Explorer nor Cost " + "Optimization Hub recommends a Savings Plan or Reserved Instance " + "purchase for this account at the requested term and payment options." + ) + add("") + add("") + add( + "This is a real result, not a failure — see *Eligible spend* below " + "for whether the account simply has no commitment-addressable usage." + ) + add("") + else: + add( + f"| Measure | Monthly | Annual |\n" + f"|---|---:|---:|\n" + f"| AWS best-case savings (as the console shows) | {_money(total_api)} " + f"| {_money(total_api * 12)} |\n" + f"| **Risk-adjusted achievable savings** | **{_money(total_safe)}** " + f"| **{_money(total_safe * 12)}** |\n" + f"| High-confidence subset only | {_money(total_high)} " + f"| {_money(total_high * 12)} |" + ) + add("") + if total_api > 0: + haircut = (1 - total_safe / total_api) * 100 + add( + f"The risk-adjusted figure is {haircut:.0f}% below the AWS " + "best case. That gap is unused-commitment risk on variable " + "workloads — see *Method* for how it is derived." + ) + add("") + if posture["blockers"]: + add( + "**Do not act on these numbers yet.** " + f"{len(posture['blockers'])} blocker(s) on existing commitments " + "must be resolved first — see the next section." + ) + add("") + + # ------------------------------------------------------ reconciliation + add("## Reconciliation against AWS native tools") + add("") + add(RECONCILE_VERDICTS.get(recon["status"], recon["status"])) + add("") + if recon["status"] != "unavailable": + add("| Pipeline | Recommended monthly savings |") + add("|---|---:|") + add(f"| Cost Explorer purchase recommendations | {_money(recon['ce_monthly_savings'])} |") + add(f"| Cost Optimization Hub ({recon['coh_count']} commitment recs) | {_money(recon['coh_monthly_savings'])} |") + add(f"| Delta | {_money(recon['delta'])} ({recon['delta_pct']}%) |") + add("") + if recon.get("coh_by_resource_type"): + add("Cost Optimization Hub breakdown by commitment type:") + add("") + add("| Resource type | Monthly savings |") + add("|---|---:|") + for rtype, amount in recon["coh_by_resource_type"].items(): + add(f"| {rtype} | {_money(amount)} |") + add("") + else: + add(f"Reason: {recon.get('reason', 'unknown')}") + add("") + + # ------------------------------------------------- existing commitments + add("## Existing commitment health") + add("") + if posture["blockers"]: + add("### Blockers") + add("") + for b in posture["blockers"]: + add(f"- **{b}**") + add("") + + rows = [ + ("Savings Plans coverage", posture.get("sp_coverage_pct"), "%"), + ("Savings Plans utilization", posture.get("sp_utilization_pct"), "%"), + ("Unused SP commitment", posture.get("sp_unused_commitment"), "$"), + ("Reservation coverage", posture.get("ri_coverage_pct"), "%"), + ("Reservation utilization", posture.get("ri_utilization_pct"), "%"), + ("Unused reservation hours", posture.get("ri_unused_hours"), "h"), + ("Realized RI savings", posture.get("ri_realized_savings"), "$"), + ] + # A table of all-zeros reads as "we measured 0% utilization", which implies + # a broken commitment. Absent commitments produce zeros across the board, so + # that case gets prose instead of a misleading table. + present = [(label, val, unit) for label, val, unit in rows if val is not None] + all_zero = bool(present) and all(val == 0 for _, val, _ in present) + if all_zero: + add( + "No active Savings Plans or Reserved Instances detected — every " + "coverage and utilization metric reads zero because there is nothing " + "to measure, not because an existing commitment is being wasted. " + "Any recommendation below is therefore a net-new purchase with no " + "inherited utilization risk." + ) + add("") + elif present: + add("| Metric | Value |") + add("|---|---:|") + for label, val, unit in present: + if unit == "%": + shown = _pct(val) + elif unit == "$": + shown = _money(val) + else: + shown = f"{val:,.0f}" + add(f"| {label} | {shown} |") + add("") + else: + # Neither table nor zero-prose applies: every posture query failed. The + # per-query reasons print as notes just below. + add( + "Existing commitment posture could not be measured — every coverage " + "and utilization query failed. Treat the recommendations below as " + "unvalidated against current commitments." + ) + add("") + + for note in posture.get("notes", []): + add(f"> {note}") + if posture.get("notes"): + add("") + + # ------------------------------------------------------ expiry/renewal + # Read with .get so a payload produced before expiry collection existed + # still renders — the section is simply omitted. + if data.get("expiry"): + _expiry_section(add, data["expiry"]) + + # ---------------------------------------------------- recommendations + add("## Recommended commitments") + add("") + if not findings: + add("None. See *Bottom line* above.") + add("") + else: + add( + "| # | Commitment | Term | Payment | AWS commitment | " + "Achievable commitment | Achievable savings/mo | Discount | " + "Confidence |" + ) + add("|---|---|---|---|---:|---:|---:|---:|---|") + for i, f in enumerate(findings, 1): + add( + f"| {i} | {f.label} | {TERM_LABELS.get(f.term, f.term)} " + f"| {PAYMENT_LABELS.get(f.payment, f.payment)} " + f"| {_commitment_str(f, f.api_hourly_commitment)} " + f"| {_commitment_str(f, f.safe_hourly_commitment)} " + f"| {_money(f.safe_monthly_savings)} " + f"| {_pct(f.savings_percentage)} | {f.confidence} |" + ) + add("") + + add("### Detail") + add("") + for i, f in enumerate(findings, 1): + add( + f"#### {i}. {f.label} — {TERM_LABELS.get(f.term, f.term)}, " + f"{PAYMENT_LABELS.get(f.payment, f.payment)}" + ) + add("") + add(f"- **Confidence:** {f.confidence} (spend profile: {f.volatility})") + add( + f"- **Commitment:** AWS recommends " + f"{_commitment_str(f, f.api_hourly_commitment)}; this report " + f"recommends {_commitment_str(f, f.safe_hourly_commitment)}" + ) + add( + f"- **Savings:** {_money(f.safe_monthly_savings)}/mo achievable " + f"({_money(f.api_monthly_savings)}/mo at the AWS best case)" + ) + if f.upfront_cost > 0: + add(f"- **Upfront cost:** {_money(f.upfront_cost)}") + if f.break_even_months: + term_months = TERM_MONTHS.get(f.term) + # Break-even past the end of the term means the upfront payment + # never returns — say so on the same line as the number. + if term_months and f.break_even_months > term_months: + add( + f"- **Break-even:** {f.break_even_months:.1f} months " + f"— **longer than the {term_months}-month term, so this " + "purchase cannot pay back**" + ) + else: + add(f"- **Break-even:** {f.break_even_months:.1f} months") + if f.waste_exposure_monthly > 0: + add( + f"- **Waste exposure at the AWS figure:** up to " + f"{_money(f.waste_exposure_monthly)}/mo unused if usage " + "falls to its observed floor" + ) + if f.detail_count: + add(f"- **Line items analyzed:** {f.detail_count}") + add("") + _line_items_table(add, f) + if f.rationale: + add("Why:") + add("") + for note in f.rationale: + add(f"- {note}") + add("") + + # ------------------------------------------------------ eligible spend + add("## Eligible spend") + add("") + periods = spend.get("periods") or [] + if spend.get("error"): + add(f"Could not retrieve spend: {spend['error']}") + elif periods: + add("| Period | Total unblended spend |") + add("|---|---:|") + for p in periods: + add(f"| {p['start']} → {p['end']} | {_money(p['total'])} |") + add("") + latest = periods[-1] + top = sorted( + latest["by_service"].items(), key=lambda kv: -kv[1] + )[:10] + add(f"Top services, {latest['start']} → {latest['end']}:") + add("") + add("| Service | Spend |") + add("|---|---:|") + for svc, amount in top: + add(f"| {svc} | {_money(amount)} |") + add("") + add( + "Commitments only apply to compute and database *instance* usage. " + "Serverless, data transfer, storage, support, and most managed-API " + "spend (for example Bedrock inference) cannot be committed against, " + "so a large bill with little instance usage will correctly yield no " + "recommendation." + ) + else: + add("No spend data returned for the requested period.") + add("") + + # ------------------------------------------------------------- method + add("## Method") + add("") + add( + "1. **Purchase recommendations** are pulled from Cost Explorer for every " + "requested (term, payment) permutation across all four Savings Plan " + "types and all eight RI-eligible services. The strongest permutation " + "per family is reported; the rest are collected but suppressed to keep " + "the decision legible." + ) + add( + "2. **Risk adjustment.** The AWS recommendation maximizes savings by " + "assuming the lookback window repeats. This report also reads the " + "*minimum* and *average* hourly on-demand spend AWS returns alongside " + "it, and classifies the workload:" + ) + add("") + add( + " - trough ≥ 80% of average → **stable**, take the AWS figure as-is " + "(High confidence)\n" + " - trough ≥ 50% of average → **moderate**, commit to the midpoint of " + "floor and AWS figure (Medium confidence)\n" + " - trough < 50% of average → **spiky**, clamp the commitment to the " + "measured floor (Low confidence)" + ) + add("") + add( + " Savings scale linearly with commitment size at a fixed discount " + "rate, so a trimmed commitment carries proportionally trimmed savings." + ) + add( + "3. **Reconciliation.** Cost Optimization Hub runs an independent " + "recommendation pipeline over the same billing data. Its commitment " + "recommendations are summed and compared to the Cost Explorer total; a " + "gap over 30% is flagged rather than averaged away." + ) + add( + "4. **Posture gate.** Coverage and utilization of *existing* " + "commitments are checked first. Utilization below 95% is reported as a " + "blocker, because buying on top of an under-used commitment compounds " + "waste instead of saving money." + ) + add("") + + # ------------------------------------------------------------- errors + if errors: + add("## Collection warnings") + add("") + add( + "These queries failed and are excluded from the totals above. The " + "report is therefore a lower bound on the opportunity." + ) + add("") + add("| Query | Error | Message |") + add("|---|---|---|") + for e in errors: + msg = e.get("error", "").replace("|", "\\|")[:160] + add(f"| {e['query']} | {e.get('error_code', '')} | {msg} |") + add("") + + add("---") + add("") + add( + "*Commitments are non-cancellable financial obligations. Verify the " + "figures in the AWS console before purchasing, and confirm the " + "underlying workload is not scheduled for migration, " + "re-architecture, or decommissioning within the commitment term.*" + ) + add("") + + return "\n".join(out) diff --git a/src/lambda/mcp/commitments/handler.py b/src/lambda/mcp/commitments/handler.py new file mode 100644 index 0000000..96c89aa --- /dev/null +++ b/src/lambda/mcp/commitments/handler.py @@ -0,0 +1,603 @@ +""" +AWS Discounted Commitments MCP Tool — Lambda Implementation for AgentCore Gateway + +Sizes Savings Plan and Reserved Instance purchases the workload can actually +sustain, rather than the best case the AWS recommendation APIs return. + +Tools (4): +- generate_commitment_analysis: ONE-SHOT aggregator — sweeps every requested + (term, payment) permutation across Savings Plan types and RI-eligible + services in parallel, measures existing commitment posture, reconciles + against Cost Optimization Hub, and returns a fully rendered markdown report + plus the structured envelope. Preferred fast path for reports. +- size_savings_plans: Savings Plans purchase recommendations only, risk-adjusted. +- size_reservations: Reserved Instance purchase recommendations only, risk-adjusted. +- get_commitment_posture: Coverage/utilization of EXISTING commitments, the + blockers they imply, COH enrollment, and commitment-addressable spend. + +Design note: + All of the deterministic work — the permutation sweep, volatility + classification, floor-based commitment sizing, break-even math, COH + reconciliation, and markdown rendering — runs in Python here. The agent makes + ONE tool call and gets a finished report, so the LLM spends its cycles on + judgment (which purchase to make, in what order) rather than orchestrating + dozens of sequential Cost Explorer calls. Same optimization as the + lambda-runtime tool's generate_upgrade_analysis. + + This file owns only the two things that are specific to being a gateway tool: + turning a caller-supplied JSON event into validated parameters, and building + AWS clients from the platform's cross-account helper. Everything between + those — the sweep, the thread pools, the analysis, the envelope — comes from + `commitments.collect`. Logic added here rather than there is logic that + tests/unit/test_commitments_{api,analyze,collect}.py do not cover, and + `TestHandlerDiscipline` fails the build over it. + + The `commitments/` subpackage carries no platform imports, so it stays + testable on its own, and it never opens a boto3 Session: the one + profile-aware function it defines, `api.build_clients`, is unused here. + `_get_clients` below builds the same `api.Clients` record from + `shared.cross_account` instead. + +Read-only: every AWS call is a Get*/List* operation. Nothing is purchased and +no billable analysis is started. + +Required IAM Permissions: +- ce:GetSavingsPlansPurchaseRecommendation +- ce:GetReservationPurchaseRecommendation +- ce:GetSavingsPlansCoverage +- ce:GetSavingsPlansUtilization +- ce:GetReservationCoverage +- ce:GetReservationUtilization +- ce:GetCostAndUsage +- cost-optimization-hub:ListEnrollmentStatuses +- cost-optimization-hub:ListRecommendations +- sts:GetCallerIdentity +""" + +import json +import os + +from commitments import api, collect, report +from commitments.analyze import assess_existing_posture, select_best_findings +from shared.cross_account import get_aws_client + +# Aliased, not restated. A copy of these values here would let the gateway tool +# accept a term the shared pipeline rejects (or default to a different sweep +# than the skill), and nothing would catch it. +TERMS = collect.TERMS +PAYMENTS = collect.PAYMENTS +LOOKBACKS = collect.LOOKBACKS +ACCOUNT_SCOPES = collect.ACCOUNT_SCOPES +FAMILIES = collect.FAMILIES +INVENTORY_KEYS = api.INVENTORY_KEYS + +DEFAULT_TERMS = collect.DEFAULT_TERMS +DEFAULT_PAYMENTS = collect.DEFAULT_PAYMENTS +DEFAULT_LOOKBACK = collect.DEFAULT_LOOKBACK +DEFAULT_ACCOUNT_SCOPE = collect.DEFAULT_ACCOUNT_SCOPE +DEFAULT_POSTURE_DAYS = collect.DEFAULT_POSTURE_DAYS +DEFAULT_SPEND_DAYS = collect.DEFAULT_SPEND_DAYS +DEFAULT_EXPIRY_HORIZON_DAYS = collect.DEFAULT_EXPIRY_HORIZON_DAYS + + +def _default_region() -> str: + """The region a reservation sweep defaults to. + + Reservations are regional and Lambda always sets AWS_REGION, so the tool's + own region is the one region we can assume is interesting. Widening the + sweep multiplies Describe* calls, so it stays opt-in via the `regions` + parameter. + """ + return os.environ.get("AWS_REGION") or os.environ.get( + "AWS_DEFAULT_REGION", api.CE_REGION + ) + + +def handler(event, context): + print(f"Event: {json.dumps(event)}") + extended_tool_name = context.client_context.custom["bedrockAgentCoreToolName"] + tool_name = extended_tool_name.split("___")[1] + print(f"Tool name: {tool_name}") + + handlers = { + "generate_commitment_analysis": handle_generate_commitment_analysis, + "size_savings_plans": handle_size_savings_plans, + "size_reservations": handle_size_reservations, + "get_commitment_posture": handle_get_commitment_posture, + "get_commitment_expiry": handle_get_commitment_expiry, + } + fn = handlers.get(tool_name) + if fn: + response = fn(event) + print(f"Response: {json.dumps(response, default=str)}") + return response + return { + "error": f"Unknown tool: {tool_name}", + "available_tools": list(handlers.keys()), + } + + +# --------------------------------------------------------------------------- +# Clients +# --------------------------------------------------------------------------- + +_clients = None + + +def _account_id() -> str: + """Resolve the account the analysis speaks for. + + A failure here must not abort the report — the recommendations are still + valid, they just carry an unknown account label. + """ + try: + sts = get_aws_client("sts", region_name=api.CE_REGION) + return sts.get_caller_identity()["Account"] + except Exception as exc: # noqa: BLE001 - label only, never fatal + print(f"Could not resolve account id: {exc}") + return "unknown" + + +def _get_clients() -> api.Clients: + """Build the `api.Clients` record the analysis package expects. + + The skill's `api.build_clients` is bypassed on purpose: it constructs a + profile-based `boto3.Session`, which has no meaning in Lambda. Cross-account + role assumption is the platform's concern, so it comes from + `shared.cross_account` — the COH client uses the same `COH` role alias as + the cost-optimization-hub tool, and Cost Explorer uses the default role. + Cached at module scope so a warm container makes no repeat STS calls. + """ + global _clients + if _clients is None: + _clients = api.Clients( + ce=get_aws_client("ce", region_name=api.CE_REGION), + coh=get_aws_client( + "cost-optimization-hub", + region_name=api.COH_REGION, + role_alias="COH", + ), + account_id=_account_id(), + profile=None, + # Reservation and Savings Plan inventory are regional Describe* + # calls with no single fixed client, so the factory routes each + # through the same cross-account role Cost Explorer uses. Commitments + # live in the payer/linked account being analyzed, not in the ops + # account running this Lambda. + make_client=lambda service, region: get_aws_client( + service, region_name=region + ), + ) + return _clients + + +# --------------------------------------------------------------------------- +# Parameter parsing — the gateway passes tool params straight through as JSON, +# so every value arrives as caller-controlled data and is validated here. +# --------------------------------------------------------------------------- + + +class ParamError(ValueError): + """A tool parameter the caller must fix. Surfaced as {"error": ...}.""" + + +def _as_list(value, default: tuple[str, ...]) -> list[str]: + """Accept either a JSON array or a comma-separated string.""" + if value is None or value == "": + return list(default) + if isinstance(value, str): + return [v.strip() for v in value.split(",") if v.strip()] + if isinstance(value, (list, tuple)): + return [str(v).strip() for v in value if str(v).strip()] + raise ParamError(f"Expected a list or comma-separated string, got {type(value).__name__}.") + + +def _validate_all(values: list[str], allowed: tuple[str, ...], label: str) -> list[str]: + bad = [v for v in values if v not in allowed] + if bad: + raise ParamError( + f"Invalid {label}: {', '.join(bad)}. Choose from {', '.join(allowed)}." + ) + if not values: + raise ParamError(f"No {label} requested. Choose from {', '.join(allowed)}.") + return values + + +def _validate_one(value, allowed: tuple[str, ...], label: str, default: str) -> str: + resolved = str(value).strip() if value else default + if resolved not in allowed: + raise ParamError( + f"Invalid {label}: {resolved}. Choose from {', '.join(allowed)}." + ) + return resolved + + +def _positive_int(value, label: str, default: int) -> int: + if value is None or value == "": + return default + try: + days = int(value) + except (TypeError, ValueError): + raise ParamError(f"{label} must be a whole number of days.") from None + if days < 1: + raise ParamError(f"{label} must be at least 1 day.") + return days + + +def _resolve_ri_services(spec) -> list[str]: + """Adapt the event's RI-service parameter to the shared resolver. + + The event may carry a JSON array or a comma-separated string; the resolver + takes a list and raises `ValueError`. Restating its label/case matching here + is what caused this tool to accept names the skill rejected. + """ + values = _as_list(spec, ("all",)) + try: + return collect.resolve_ri_services(values) + except ValueError as exc: + raise ParamError(str(exc)) from None + + +def _resolve_sp_types(spec) -> list[str]: + """Adapt the event's savings-plan-type parameter to the shared resolver.""" + values = _as_list(spec, ("all",)) + try: + return collect.resolve_sp_types(values) + except ValueError as exc: + raise ParamError(str(exc)) from None + + +def _resolve_regions(spec) -> list[str]: + """Adapt the event's region parameter to the shared resolver. + + Region strings become part of an SDK endpoint, so they are validated by the + shared resolver rather than passed through — the same reason the enums + above are aliased instead of restated. + """ + values = _as_list(spec, ()) + try: + return collect.resolve_regions(values, _default_region()) + except ValueError as exc: + raise ParamError(str(exc)) from None + + +def _resolve_inventory_services(spec) -> list[str]: + """Parse which reservation families to inventory.""" + values = [v.lower() for v in _as_list(spec, INVENTORY_KEYS)] + return _validate_all(values, INVENTORY_KEYS, "reservation family") + + +def _common_params(event: dict) -> dict: + """Parse the term/payment/lookback/scope parameters every sizing tool takes.""" + return { + "terms": _validate_all( + _as_list(event.get("terms"), DEFAULT_TERMS), TERMS, "term" + ), + "payments": _validate_all( + _as_list(event.get("payment_options"), DEFAULT_PAYMENTS), + PAYMENTS, + "payment option", + ), + "lookback": _validate_one( + event.get("lookback"), LOOKBACKS, "lookback", DEFAULT_LOOKBACK + ), + "account_scope": _validate_one( + event.get("account_scope"), + ACCOUNT_SCOPES, + "account scope", + DEFAULT_ACCOUNT_SCOPE, + ), + } + + +# --------------------------------------------------------------------------- +# Tool: generate_commitment_analysis +# --------------------------------------------------------------------------- + + +def handle_generate_commitment_analysis(event): + """One-shot: sweep, size, gate on posture, reconcile, and render.""" + try: + params = _common_params(event) + families = [ + f.lower() for f in _as_list(event.get("families"), FAMILIES) + ] + _validate_all(families, FAMILIES, "commitment family") + sp_types = _resolve_sp_types(event.get("savings_plan_types")) + ri_services = _resolve_ri_services(event.get("ri_services")) + posture_days = _positive_int( + event.get("posture_days"), "posture_days", DEFAULT_POSTURE_DAYS + ) + regions = _resolve_regions(event.get("regions")) + expiry_horizon_days = _positive_int( + event.get("expiry_horizon_days"), + "expiry_horizon_days", + DEFAULT_EXPIRY_HORIZON_DAYS, + ) + except ParamError as exc: + return {"error": str(exc)} + + try: + clients = _get_clients() + payload = collect.collect_all( + clients, + families=families, + sp_types=sp_types, + ri_services=ri_services, + terms=params["terms"], + payments=params["payments"], + lookback=params["lookback"], + account_scope=params["account_scope"], + posture_days=posture_days, + spend_days=DEFAULT_SPEND_DAYS, + regions=regions, + expiry_horizon_days=expiry_horizon_days, + ) + + posture = payload["posture"] + errors = payload["errors"] + raw = payload["raw"] + return { + "report_markdown": report.render(payload), + "account_id": clients.account_id, + "generated_at": payload["meta"]["generated_at"], + "lookback": params["lookback"], + "account_scope": params["account_scope"], + **collect.envelope(payload["findings"]), + "reconciliation": payload["reconciliation"], + "existing_commitment_posture": posture, + "expiry": payload["expiry"], + "blockers": posture["blockers"], + # Billable Cost Explorer requests only. The expiry Describe* calls + # are free and are excluded on purpose — this number is what the + # caller is charged $0.01 apiece for. + "queries_run": ( + len(raw["sp_recs"]) + + len(raw["ri_recs"]) + + len(payload["sweep_errors"]) + ), + "collection_warnings": errors, + "data_source": "live", + } + except ValueError as exc: + # collect_all re-validates every parameter; anything it rejects that got + # past _common_params is still the caller's to fix, not a 500. + return {"error": str(exc)} + except Exception as e: + return _aws_error(e) + + +# --------------------------------------------------------------------------- +# Tool: size_savings_plans / size_reservations +# --------------------------------------------------------------------------- + + +def handle_size_savings_plans(event): + """Savings Plans purchase recommendations, risk-adjusted, no posture gate.""" + try: + params = _common_params(event) + sp_types = _resolve_sp_types(event.get("savings_plan_types")) + except ParamError as exc: + return {"error": str(exc)} + + try: + clients = _get_clients() + recs, errors = collect.sweep_savings_plans( + clients, + sp_types, + params["terms"], + params["payments"], + params["lookback"], + params["account_scope"], + ) + best = select_best_findings(collect.findings_from(recs, [])) + return { + "account_id": clients.account_id, + "lookback": params["lookback"], + "account_scope": params["account_scope"], + "savings_plan_types": sp_types, + **collect.envelope(best), + "collection_warnings": errors, + "note": ( + "Achievable figures are risk-adjusted against the measured " + "hourly spend floor. Check get_commitment_posture before " + "acting — buying on top of an under-utilized commitment " + "compounds waste." + ), + "data_source": "live", + } + except Exception as e: + return _aws_error(e) + + +def handle_size_reservations(event): + """Reserved Instance purchase recommendations, risk-adjusted.""" + try: + params = _common_params(event) + ri_services = _resolve_ri_services(event.get("ri_services")) + except ParamError as exc: + return {"error": str(exc)} + + try: + clients = _get_clients() + recs, errors = collect.sweep_reservations( + clients, + ri_services, + params["terms"], + params["payments"], + params["lookback"], + params["account_scope"], + ) + best = select_best_findings(collect.findings_from([], recs)) + return { + "account_id": clients.account_id, + "lookback": params["lookback"], + "account_scope": params["account_scope"], + "services": [api.RI_SERVICE_LABELS.get(s, s) for s in ri_services], + **collect.envelope(best), + "collection_warnings": errors, + "note": ( + "Reservation commitments are quoted in whole instance or " + "capacity units, rounded down to the level the workload " + "sustains on its quietest hour." + ), + "data_source": "live", + } + except Exception as e: + return _aws_error(e) + + +# --------------------------------------------------------------------------- +# Tool: get_commitment_posture +# --------------------------------------------------------------------------- + + +def handle_get_commitment_posture(event): + """Health of EXISTING commitments, plus what share of spend is committable.""" + try: + posture_days = _positive_int( + event.get("posture_days"), "posture_days", DEFAULT_POSTURE_DAYS + ) + spend_days = _positive_int( + event.get("spend_days"), "spend_days", DEFAULT_SPEND_DAYS + ) + except ParamError as exc: + return {"error": str(exc)} + + try: + clients = _get_clients() + collected = collect.collect_posture(clients, posture_days, spend_days) + posture = assess_existing_posture( + collected["sp_coverage"], + collected["sp_utilization"], + collected["ri_coverage"], + collected["ri_utilization"], + ) + spend = collected["eligible_spend"] + periods = [ + {"start": p["start"], "end": p["end"], "total": p["total"]} + for p in spend.get("periods", []) + ] + latest = spend.get("periods", []) + top_services = ( + dict( + sorted(latest[-1]["by_service"].items(), key=lambda kv: -kv[1])[:10] + ) + if latest + else {} + ) + return { + "account_id": clients.account_id, + "window_days": posture_days, + "posture": posture, + "blockers": posture["blockers"], + "safe_to_buy_more": not posture["blockers"], + "cost_optimization_hub": collected["coh_enrollment"], + "spend_periods": periods, + "top_services_latest_period": top_services, + "spend_note": ( + "Commitments only apply to compute and database instance " + "usage. Serverless, storage, data transfer, and managed-API " + "spend cannot be committed against, so a large bill with " + "little instance usage correctly yields no recommendation." + ), + "notes": posture.get("notes", []), + "data_source": "live", + } + except Exception as e: + return _aws_error(e) + + +# --------------------------------------------------------------------------- +# Tool: get_commitment_expiry +# --------------------------------------------------------------------------- + + +def handle_get_commitment_expiry(event): + """When existing commitments lapse, and what to renew. + + The one tool here that is not Cost Explorer: expiry dates only exist on the + per-service Describe* APIs, and those are regional. Utilization for the + renewal call still comes from Cost Explorer, since it is the only source + for it. + """ + try: + regions = _resolve_regions(event.get("regions")) + services = _resolve_inventory_services(event.get("services")) + horizon_days = _positive_int( + event.get("horizon_days"), "horizon_days", DEFAULT_EXPIRY_HORIZON_DAYS + ) + posture_days = _positive_int( + event.get("posture_days"), "posture_days", DEFAULT_POSTURE_DAYS + ) + except ParamError as exc: + return {"error": str(exc)} + + try: + clients = _get_clients() + collected = collect.collect_posture(clients, posture_days, DEFAULT_SPEND_DAYS) + posture = assess_existing_posture( + collected["sp_coverage"], + collected["sp_utilization"], + collected["ri_coverage"], + collected["ri_utilization"], + ) + expiry, errors = collect.collect_expiry( + clients, + regions, + services, + horizon_days, + sp_utilization_pct=posture.get("sp_utilization_pct"), + ri_utilization_pct=posture.get("ri_utilization_pct"), + ) + return { + "account_id": clients.account_id, + **expiry, + "renewal_actions": expiry["actions"], + "collection_warnings": errors, + "note": ( + "Utilization is account-level (Cost Explorer publishes no " + "per-commitment figure), so a renewal verdict is a portfolio " + "signal, not a per-commitment measurement. Read-only: this " + "reports expiry, it never renews or purchases." + ), + "data_source": "live", + } + except ValueError as exc: + return {"error": str(exc)} + except Exception as e: + return _aws_error(e) + + +# --------------------------------------------------------------------------- +# Errors +# --------------------------------------------------------------------------- + + +def _aws_error(exc: Exception) -> dict: + """Normalize an unexpected failure, calling out the actionable cases.""" + message = str(exc) + if "AccessDenied" in message or "UnauthorizedOperation" in message: + return { + "error": message, + "hint": ( + "The tool role needs ce:GetSavingsPlansPurchaseRecommendation, " + "ce:GetReservationPurchaseRecommendation, the coverage and " + "utilization reads, and cost-optimization-hub:ListRecommendations. " + "Cost Explorer must also be enabled for the payer account. " + "Expiry inventory additionally needs " + "savingsplans:DescribeSavingsPlans plus the per-service " + "Describe* reads (ec2, rds, elasticache, redshift, es, " + "memorydb) in each swept region." + ), + } + if "DataUnavailable" in message: + return { + "error": message, + "hint": ( + "Cost Explorer has no usage history for the requested lookback " + "window. Try a shorter lookback, or accept that the account has " + "no commitment-addressable usage yet." + ), + } + return {"error": message} diff --git a/src/lambda/mcp/commitments/requirements.txt b/src/lambda/mcp/commitments/requirements.txt new file mode 100644 index 0000000..f974a74 --- /dev/null +++ b/src/lambda/mcp/commitments/requirements.txt @@ -0,0 +1,8 @@ +# Empty — Lambda Python 3.12 runtime provides boto3/botocore at runtime. +# +# When adding a dep, ALWAYS pin to a specific version (==X.Y.Z), not a +# floor (>=X.Y) or unconstrained name. The build packages this file via +# `pip install -r requirements.txt -t ./package/` — without pins, every +# rebuild can pull a newer minor that introduces breaking changes (we +# hit this with ag-ui-strands 0.1.4 → 0.1.9 silently changing default +# history-replay behavior). Bump deliberately, retest, then update. diff --git a/src/lambda/mcp/tools.json b/src/lambda/mcp/tools.json index dfa4fef..a087153 100644 --- a/src/lambda/mcp/tools.json +++ b/src/lambda/mcp/tools.json @@ -802,6 +802,228 @@ } ] }, + "commitments": { + "handler": "handler.handler", + "runtime": "python3.12", + "timeout": 300, + "memory": 1024, + "env_vars": { + "CROSS_ACCOUNT_ROLE_ARN": "$CROSS_ACCOUNT_ROLE_ARN", + "CROSS_ACCOUNT_ROLE_ARN_COH": "$CROSS_ACCOUNT_ROLE_ARN_COH" + }, + "iam_actions": [ + "ce:GetSavingsPlansPurchaseRecommendation", + "ce:GetReservationPurchaseRecommendation", + "ce:GetSavingsPlansCoverage", + "ce:GetSavingsPlansUtilization", + "ce:GetReservationCoverage", + "ce:GetReservationUtilization", + "ce:GetCostAndUsage", + "cost-optimization-hub:ListEnrollmentStatuses", + "cost-optimization-hub:ListRecommendations", + "savingsplans:DescribeSavingsPlans", + "ec2:DescribeReservedInstances", + "rds:DescribeReservedDBInstances", + "elasticache:DescribeReservedCacheNodes", + "redshift:DescribeReservedNodes", + "es:DescribeReservedInstances", + "memorydb:DescribeReservedNodes", + "sts:GetCallerIdentity" + ], + "tools": [ + { + "name": "generate_commitment_analysis", + "description": "ONE-SHOT: produce a complete AWS discounted-commitment (Savings Plan + Reserved Instance) purchase analysis. Sweeps every requested term/payment permutation in parallel, risk-adjusts each AWS recommendation down to the commitment level the workload sustains on its quietest hour, checks whether EXISTING commitments are healthy enough to add more, and reconciles totals against Cost Optimization Hub. Returns a fully rendered 'report_markdown' plus structured recommendations. PREFER this over calling the sizing tools separately — it is one call and includes the posture gate and reconciliation the numbers are only defensible with. Read-only: purchases nothing. Takes up to ~90s for a full sweep.", + "input_schema": { + "type": "object", + "properties": { + "lookback": { + "type": "string", + "description": "Usage history window AWS bases recommendations on: SEVEN_DAYS, THIRTY_DAYS, SIXTY_DAYS (default: THIRTY_DAYS)" + }, + "terms": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Commitment terms to evaluate: ONE_YEAR, THREE_YEARS (default: both)" + }, + "payment_options": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Payment options to evaluate: NO_UPFRONT, PARTIAL_UPFRONT, ALL_UPFRONT (default: NO_UPFRONT and ALL_UPFRONT)" + }, + "account_scope": { + "type": "string", + "description": "PAYER aggregates the whole organization; LINKED scopes to this account only (default: PAYER)" + }, + "families": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Commitment families to analyze: 'sp' (Savings Plans), 'ri' (Reserved Instances), or both (default: both)" + }, + "savings_plan_types": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Savings Plan types: COMPUTE_SP, EC2_INSTANCE_SP, SAGEMAKER_SP, DATABASE_SP, or 'all' (default: all)" + }, + "ri_services": { + "type": "array", + "items": { + "type": "string" + }, + "description": "RI-eligible services, by short label (EC2, RDS, Redshift, ElastiCache, OpenSearch, Elasticsearch, MemoryDB, DynamoDB) or exact Cost Explorer name, or 'all' (default: all). Narrow this to cut runtime." + }, + "posture_days": { + "type": "integer", + "description": "Window for existing coverage/utilization metrics (default: 30)" + }, + "regions": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Pass regions to ALSO inventory existing commitments and report what expires soon (e.g. ['ap-northeast-1','us-east-1']). Reservations are regional, so only the regions listed are checked; Savings Plans are account-level and are fetched once regardless. Omit to skip the expiry inventory entirely — it needs Describe* permissions beyond Cost Explorer." + }, + "expiry_horizon_days": { + "type": "integer", + "description": "How far ahead to look for expiring commitments, in days (default: 90). Only used when 'regions' is supplied." + } + } + } + }, + { + "name": "size_savings_plans", + "description": "Savings Plans purchase recommendations only, risk-adjusted to the commitment level the workload sustains at its measured hourly spend floor. Returns achievable vs AWS best-case monthly savings, upfront cost, break-even months, waste exposure, and confidence per plan type. Use when the user asks specifically about Savings Plans; use generate_commitment_analysis for a full report. Read-only.", + "input_schema": { + "type": "object", + "properties": { + "savings_plan_types": { + "type": "array", + "items": { + "type": "string" + }, + "description": "COMPUTE_SP, EC2_INSTANCE_SP, SAGEMAKER_SP, DATABASE_SP, or 'all' (default: all)" + }, + "lookback": { + "type": "string", + "description": "SEVEN_DAYS, THIRTY_DAYS, SIXTY_DAYS (default: THIRTY_DAYS)" + }, + "terms": { + "type": "array", + "items": { + "type": "string" + }, + "description": "ONE_YEAR, THREE_YEARS (default: both)" + }, + "payment_options": { + "type": "array", + "items": { + "type": "string" + }, + "description": "NO_UPFRONT, PARTIAL_UPFRONT, ALL_UPFRONT (default: NO_UPFRONT and ALL_UPFRONT)" + }, + "account_scope": { + "type": "string", + "description": "PAYER or LINKED (default: PAYER)" + } + } + } + }, + { + "name": "size_reservations", + "description": "Reserved Instance purchase recommendations only, risk-adjusted. Commitments are quoted in whole instance or capacity units, rounded down to the count the workload never drops below. Covers EC2, RDS, Redshift, ElastiCache, OpenSearch, MemoryDB, and DynamoDB. Use when the user asks specifically about reservations; use generate_commitment_analysis for a full report. Read-only.", + "input_schema": { + "type": "object", + "properties": { + "ri_services": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Short labels (EC2, RDS, Redshift, ElastiCache, OpenSearch, Elasticsearch, MemoryDB, DynamoDB), exact Cost Explorer service names, or 'all' (default: all)" + }, + "lookback": { + "type": "string", + "description": "SEVEN_DAYS, THIRTY_DAYS, SIXTY_DAYS (default: THIRTY_DAYS)" + }, + "terms": { + "type": "array", + "items": { + "type": "string" + }, + "description": "ONE_YEAR, THREE_YEARS (default: both)" + }, + "payment_options": { + "type": "array", + "items": { + "type": "string" + }, + "description": "NO_UPFRONT, PARTIAL_UPFRONT, ALL_UPFRONT (default: NO_UPFRONT and ALL_UPFRONT)" + }, + "account_scope": { + "type": "string", + "description": "PAYER or LINKED (default: PAYER)" + } + } + } + }, + { + "name": "get_commitment_posture", + "description": "Health of EXISTING Savings Plans and Reserved Instances: coverage, utilization, unused commitment, unused reservation hours, and realized savings. Returns explicit blockers (utilization under 95%, coverage over 90%) plus 'safe_to_buy_more', Cost Optimization Hub enrollment, and commitment-addressable spend by service. Call this BEFORE recommending any purchase — buying on top of an under-utilized commitment compounds waste instead of saving money. Read-only.", + "input_schema": { + "type": "object", + "properties": { + "posture_days": { + "type": "integer", + "description": "Window for coverage/utilization metrics (default: 30)" + }, + "spend_days": { + "type": "integer", + "description": "Window for the eligible-spend breakdown (default: 60)" + } + } + } + }, + { + "name": "get_commitment_expiry", + "description": "Which EXISTING Savings Plans and Reserved Instances expire soon, and what to do about each. Inventories active commitments across EC2, RDS, ElastiCache, Redshift, OpenSearch, MemoryDB and Savings Plans, derives each end date, and returns them bucketed as urgent (<=30 days), soon (<=60) or upcoming, each with a renew / renew-smaller / let-lapse / review verdict driven by measured utilization. Use this when the user asks what is expiring, what needs renewing, or before recommending a new purchase — a lapsing commitment silently returns that spend to on-demand rates, and expiry is the one moment resizing costs nothing. Note: AWS publishes utilization only at the account level, not per commitment, so the verdicts are directional. DynamoDB reserved capacity has no describe API and is reported as a blind spot. Read-only: renews nothing.", + "input_schema": { + "type": "object", + "properties": { + "regions": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Regions to inventory reservations in (e.g. ['ap-northeast-1','us-east-1']). Reservations are regional and are only found in the regions listed; Savings Plans are account-level and are fetched once regardless. Defaults to the Lambda's own region." + }, + "services": { + "type": "array", + "items": { + "type": "string" + }, + "description": "Reservation families to inventory: ec2, rds, elasticache, redshift, opensearch, memorydb (default: all). Savings Plans are always included." + }, + "horizon_days": { + "type": "integer", + "description": "How far ahead to look, in days (default: 90). Commitments ending beyond this are counted but not listed." + }, + "posture_days": { + "type": "integer", + "description": "Window for the utilization figures the renew/let-lapse verdicts are based on (default: 30)" + } + } + } + } + ] + }, "health-events": { "handler": "handler.handler", "runtime": "python3.12", diff --git a/tests/unit/conftest.py b/tests/unit/conftest.py index fda883d..ea8185a 100644 --- a/tests/unit/conftest.py +++ b/tests/unit/conftest.py @@ -1,10 +1,24 @@ """Shared setup for the unit-test package. -Three things are centralized here so individual test modules don't each +Four things are centralized here so individual test modules don't each re-implement them (which is how conventions drift — see the sys.modules -pollution that once lived in test_lambda_runtime_tool.py): +pollution that once lived in test_lambda_runtime_tool.py). They are listed in +the order the code below applies them: -1. Deterministic AWS env for import time. Several Lambda handlers read +1. MCP shared-package import path. Tests exercise multiple Lambda packages + that import `shared.*` during collection. Prepending src/lambda/mcp makes + the deployed shared package resolve before test modules are imported. + +2. The `commitments` analysis package. In the deployed zip the commitments tool + directory IS the import root, so its handler says `from commitments import + api`. Reproducing that name here needs an explicit binding: because step 1 + puts src/lambda/mcp on sys.path, a bare `import commitments` would resolve + to the *tool* directory as a namespace package, one level above the real + package — every `commitments.api` import would then fail. Binding the name + to the inner directory cannot be left to a test module, because whichever + module imported first would decide it. + +3. Deterministic AWS env for import time. Several Lambda handlers read AWS_REGION (and clients like MemoryClient default to us-west-2 if it is unset — see CLAUDE.md), and they are imported at test-collection time via importlib. conftest.py is imported BEFORE the test modules in its @@ -12,29 +26,41 @@ place before any handler import. Dummy credentials ensure a stray boto3 call can never reach a real account from the unit suite. -2. MCP shared-package import path. Tests exercise multiple Lambda packages - that import `shared.*` during collection. Prepending src/lambda/mcp makes - the deployed shared package resolve before test modules are imported. - -3. Automatic `unit` marker. Every test under tests/unit/ is a unit test, so +4. Automatic `unit` marker. Every test under tests/unit/ is a unit test, so rather than repeat `pytestmark = pytest.mark.unit` in 15 files (only one did), we tag them all here. This makes `pytest -m unit` actually select the whole unit suite. """ +import importlib.util import os import sys from pathlib import Path import pytest +_REPO_ROOT = Path(__file__).resolve().parents[2] + # --- 1. MCP shared-package import path -------------------------------------- -MCP_ROOT = Path(__file__).resolve().parents[2] / "src" / "lambda" / "mcp" +MCP_ROOT = _REPO_ROOT / "src" / "lambda" / "mcp" if str(MCP_ROOT) not in sys.path: sys.path.insert(0, str(MCP_ROOT)) -# --- 2. Deterministic AWS env (module scope → runs at conftest import, +# --- 2. Bind `commitments` to the analysis package, not the tool directory --- +COMMITMENTS_PKG = MCP_ROOT / "commitments" / "commitments" +if "commitments" not in sys.modules: + _spec = importlib.util.spec_from_file_location( + "commitments", + COMMITMENTS_PKG / "__init__.py", + submodule_search_locations=[str(COMMITMENTS_PKG)], + ) + _pkg = importlib.util.module_from_spec(_spec) + sys.modules["commitments"] = _pkg + _spec.loader.exec_module(_pkg) + + +# --- 3. Deterministic AWS env (module scope → runs at conftest import, # before test modules import their handlers) -------------------------- os.environ.setdefault("AWS_REGION", "us-east-1") os.environ.setdefault("AWS_DEFAULT_REGION", "us-east-1") @@ -47,7 +73,7 @@ os.environ.setdefault("AWS_SESSION_TOKEN", "testing") -# --- 3. Auto-apply the `unit` marker to everything in this package ---------- +# --- 4. Auto-apply the `unit` marker to everything in this package ---------- def pytest_collection_modifyitems(config, items): """Tag every test collected under tests/unit/ with the `unit` marker, so `pytest -m unit` selects the full suite without each file having to diff --git a/tests/unit/test_commitments_analyze.py b/tests/unit/test_commitments_analyze.py new file mode 100644 index 0000000..6dfea8b --- /dev/null +++ b/tests/unit/test_commitments_analyze.py @@ -0,0 +1,511 @@ +"""Tests for the risk-adjustment logic. + +Fixtures mirror real Cost Explorer response shapes, including its habit of +returning money and percentages as STRINGS. +""" + +from __future__ import annotations + +import pytest + +from commitments.analyze import ( + CONFIDENCE_HIGH, + CONFIDENCE_LOW, + CONFIDENCE_MEDIUM, + analyze_ri_recommendation, + analyze_sp_recommendation, + assess_existing_posture, + classify_volatility, + reconcile_with_coh, + select_best_findings, +) +from commitments.report import render + + +def sp_rec(minimum: str, average: str, hourly: str = "10.0", monthly: str = "1000.0"): + """Build an SP recommendation with a given hourly spend envelope.""" + return { + "sp_type": "COMPUTE_SP", + "term": "ONE_YEAR", + "payment": "NO_UPFRONT", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + "summary": { + "HourlyCommitmentToPurchase": hourly, + "EstimatedMonthlySavingsAmount": monthly, + "EstimatedSavingsPercentage": "20.5", + "CurrentOnDemandSpend": "8000.0", + }, + "details": [ + { + "CurrentMinimumHourlyOnDemandSpend": minimum, + "CurrentAverageHourlyOnDemandSpend": average, + "CurrentMaximumHourlyOnDemandSpend": "20.0", + "UpfrontCost": "0.0", + "EstimatedAverageUtilization": "97.5", + } + ], + "generated_at": "2026-08-07T00:00:00Z", + "recommendation_id": "rec-1", + } + + +# --------------------------------------------------------------- volatility + + +@pytest.mark.unit +@pytest.mark.parametrize( + "floor,average,expected", + [ + (9.0, 10.0, "stable"), + (8.0, 10.0, "stable"), + (7.9, 10.0, "moderate"), + (5.0, 10.0, "moderate"), + (4.9, 10.0, "spiky"), + (0.0, 10.0, "spiky"), + (5.0, 0.0, "unknown"), + ], +) +def test_classify_volatility_bands(floor, average, expected): + label, _ = classify_volatility(floor, average) + assert label == expected + + +# ------------------------------------------------------- SP risk adjustment + + +@pytest.mark.unit +def test_stable_workload_keeps_api_recommendation(): + f = analyze_sp_recommendation(sp_rec(minimum="9.0", average="10.0")) + assert f.confidence == CONFIDENCE_HIGH + assert f.safe_hourly_commitment == pytest.approx(10.0) + assert f.safe_monthly_savings == pytest.approx(1000.0) + + +@pytest.mark.unit +def test_moderate_workload_takes_midpoint(): + # floor 6.0 / avg 10.0 = 0.6 ratio -> moderate. Midpoint of 6 and 10 = 8. + f = analyze_sp_recommendation(sp_rec(minimum="6.0", average="10.0")) + assert f.confidence == CONFIDENCE_MEDIUM + assert f.safe_hourly_commitment == pytest.approx(8.0) + # Savings scale with commitment: 8/10 of $1000. + assert f.safe_monthly_savings == pytest.approx(800.0) + + +@pytest.mark.unit +def test_spiky_workload_clamps_to_floor(): + f = analyze_sp_recommendation(sp_rec(minimum="2.0", average="10.0")) + assert f.confidence == CONFIDENCE_LOW + assert f.safe_hourly_commitment == pytest.approx(2.0) + assert f.safe_monthly_savings == pytest.approx(200.0) + + +@pytest.mark.unit +def test_safe_commitment_never_exceeds_api_figure(): + """A floor above the API recommendation must not inflate the commitment.""" + f = analyze_sp_recommendation( + sp_rec(minimum="50.0", average="55.0", hourly="10.0") + ) + assert f.safe_hourly_commitment <= 10.0 + assert f.safe_monthly_savings <= 1000.0 + + +@pytest.mark.unit +def test_waste_exposure_measured_against_floor(): + f = analyze_sp_recommendation(sp_rec(minimum="2.0", average="10.0")) + # (10.0 api - 2.0 floor) * 730 hours + assert f.waste_exposure_monthly == pytest.approx(8.0 * 730.0) + + +@pytest.mark.unit +def test_upfront_cost_produces_break_even(): + rec = sp_rec(minimum="9.0", average="10.0") + rec["details"][0]["UpfrontCost"] = "6000.0" + f = analyze_sp_recommendation(rec) + assert f.break_even_months == pytest.approx(6.0) + + +@pytest.mark.unit +def test_empty_string_numerics_do_not_raise(): + """Cost Explorer returns "" for absent numbers; float("") would crash.""" + rec = sp_rec(minimum="", average="", hourly="10.0", monthly="500.0") + f = analyze_sp_recommendation(rec) + assert f is not None + assert f.confidence == CONFIDENCE_LOW + + +@pytest.mark.unit +def test_zero_recommendation_is_dropped(): + assert analyze_sp_recommendation(sp_rec("0", "0", hourly="0.0", monthly="0.0")) is None + + +@pytest.mark.unit +def test_errored_recommendation_is_dropped(): + assert analyze_sp_recommendation({"error": "boom", "error_code": "X"}) is None + + +# ------------------------------------------------------- RI risk adjustment + + +@pytest.mark.unit +def test_ri_recommendation_rounds_to_whole_units(): + rec = { + "service": "Amazon Relational Database Service", + "label": "RDS", + "term": "ONE_YEAR", + "payment": "ALL_UPFRONT", + "summary": { + "TotalEstimatedMonthlySavingsAmount": "900.0", + "TotalEstimatedMonthlySavingsPercentage": "31.0", + }, + "details": [ + { + "RecommendedNumberOfInstancesToPurchase": "10", + "MinimumNumberOfInstancesUsedPerHour": "6", + "AverageNumberOfInstancesUsedPerHour": "10", + "UpfrontCost": "12000.0", + "RecurringStandardMonthlyCost": "0.0", + "EstimatedBreakEvenInMonths": "8.5", + "AverageUtilization": "92.0", + } + ], + } + f = analyze_ri_recommendation(rec) + # ratio 0.6 -> moderate -> midpoint 8.0, already whole. + assert f.safe_hourly_commitment == 8.0 + assert f.safe_hourly_commitment == int(f.safe_hourly_commitment) + assert f.break_even_months == pytest.approx(8.5) + + +@pytest.mark.unit +def test_ri_capacity_unit_fallback(): + """DynamoDB reports capacity units, not instance counts.""" + rec = { + "service": "Amazon DynamoDB Service", + "label": "DynamoDB", + "term": "ONE_YEAR", + "payment": "NO_UPFRONT", + "summary": {"TotalEstimatedMonthlySavingsAmount": "400.0"}, + "details": [ + { + "RecommendedNumberOfCapacityUnitsToPurchase": "100", + "MinimumNumberOfCapacityUnitsUsedPerHour": "95", + "AverageNumberOfCapacityUnitsUsedPerHour": "100", + } + ], + } + f = analyze_ri_recommendation(rec) + assert f.api_hourly_commitment == pytest.approx(100.0) + assert f.confidence == CONFIDENCE_HIGH + + +@pytest.mark.unit +def test_ri_rounds_down_and_note_quotes_the_rounded_figure(): + """The prose must not contradict the number the table shows.""" + rec = { + "service": "Amazon Relational Database Service", + "label": "RDS", + "term": "ONE_YEAR", + "payment": "PARTIAL_UPFRONT", + "summary": {"TotalEstimatedMonthlySavingsAmount": "900.0"}, + "details": [ + { + # floor 9 / avg 12 = 0.75 -> moderate -> midpoint 10.5 -> 10 + "RecommendedNumberOfInstancesToPurchase": "12", + "MinimumNumberOfInstancesUsedPerHour": "9", + "AverageNumberOfInstancesUsedPerHour": "12", + } + ], + } + f = analyze_ri_recommendation(rec) + assert f.safe_hourly_commitment == 10.0 + trimmed = [n for n in f.rationale if "trimmed to" in n] + assert trimmed, "expected a trim note" + # Reservations are counted, not priced per hour — and 10.5 was rounded away. + assert "10 unit(s)" in trimmed[0] + assert "$" not in trimmed[0] + assert "10.5" not in trimmed[0] + + +@pytest.mark.unit +def test_sp_trim_note_uses_dollars_per_hour(): + f = analyze_sp_recommendation(sp_rec(minimum="6.0", average="10.0")) + trimmed = [n for n in f.rationale if "trimmed to" in n] + assert "$8.0000/hr" in trimmed[0] + + +# -------------------------------------------------- break-even vs term guard + + +@pytest.mark.unit +def test_break_even_beyond_term_is_flagged_and_downgraded(): + """A 1-year commitment that pays back in 15 years is a loss, not a saving.""" + rec = sp_rec(minimum="2.0", average="10.0", hourly="10.0", monthly="1000.0") + rec["term"] = "ONE_YEAR" + rec["details"][0]["UpfrontCost"] = "38000.0" + f = analyze_sp_recommendation(rec) + assert f.break_even_months > 12 + assert f.confidence == CONFIDENCE_LOW + assert any("Do not buy" in n for n in f.rationale) + # The warning must lead, not trail the supporting detail. + assert "Do not buy" in f.rationale[0] + + +@pytest.mark.unit +def test_break_even_inside_term_is_not_flagged(): + rec = sp_rec(minimum="9.0", average="10.0", monthly="1000.0") + rec["details"][0]["UpfrontCost"] = "6000.0" + f = analyze_sp_recommendation(rec) + assert f.break_even_months == pytest.approx(6.0) + assert f.confidence == CONFIDENCE_HIGH + assert not any("Do not buy" in n for n in f.rationale) + + +@pytest.mark.unit +def test_three_year_term_allows_longer_break_even(): + """24 months is fatal on a 1-year term and fine on a 3-year one.""" + rec = sp_rec(minimum="9.0", average="10.0", monthly="1000.0") + rec["details"][0]["UpfrontCost"] = "24000.0" + + rec["term"] = "THREE_YEARS" + assert not any( + "Do not buy" in n for n in analyze_sp_recommendation(rec).rationale + ) + + rec["term"] = "ONE_YEAR" + assert any("Do not buy" in n for n in analyze_sp_recommendation(rec).rationale) + + +@pytest.mark.unit +def test_ri_break_even_beyond_term_is_flagged(): + rec = { + "service": "Amazon Redshift", + "label": "Redshift", + "term": "ONE_YEAR", + "payment": "ALL_UPFRONT", + "summary": {"TotalEstimatedMonthlySavingsAmount": "100.0"}, + "details": [ + { + "RecommendedNumberOfInstancesToPurchase": "4", + "MinimumNumberOfInstancesUsedPerHour": "4", + "AverageNumberOfInstancesUsedPerHour": "4", + "UpfrontCost": "50000.0", + "EstimatedBreakEvenInMonths": "40.0", + } + ], + } + f = analyze_ri_recommendation(rec) + assert f.confidence == CONFIDENCE_LOW + assert any("Do not buy" in n for n in f.rationale) + + +# ------------------------------------------------------------- posture gate + + +@pytest.mark.unit +def test_low_sp_utilization_is_a_blocker(): + posture = assess_existing_posture( + sp_coverage={"periods": [{"Coverage": {"CoveragePercentage": "40.0", "OnDemandCost": "100"}}]}, + sp_utilization={"total": {"Utilization": {"UtilizationPercentage": "72.0", "UnusedCommitment": "500.0"}}}, + ri_coverage={}, + ri_utilization={}, + ) + assert any("72.0% utilized" in b for b in posture["blockers"]) + + +@pytest.mark.unit +def test_saturated_coverage_is_a_blocker(): + posture = assess_existing_posture( + sp_coverage={"periods": [{"Coverage": {"CoveragePercentage": "97.0", "OnDemandCost": "10"}}]}, + sp_utilization={}, + ri_coverage={}, + ri_utilization={}, + ) + assert any("97.0%" in b for b in posture["blockers"]) + + +@pytest.mark.unit +def test_healthy_posture_has_no_blockers(): + posture = assess_existing_posture( + sp_coverage={"periods": [{"Coverage": {"CoveragePercentage": "60.0", "OnDemandCost": "900"}}]}, + sp_utilization={"total": {"Utilization": {"UtilizationPercentage": "99.5", "UnusedCommitment": "1.0"}}}, + ri_coverage={"total": {"CoverageHours": {"CoverageHoursPercentage": "55.0", "OnDemandHours": "100"}}}, + ri_utilization={"total": {"UtilizationPercentage": "99.0", "UnusedHours": "2", "RealizedSavings": "300"}}, + ) + assert posture["blockers"] == [] + + +@pytest.mark.unit +def test_empty_datauavailable_message_is_explained(): + """A blank DataUnavailableException must not surface as an empty note.""" + posture = assess_existing_posture( + sp_coverage={}, + sp_utilization={"error": "No data for this period — no active commitment."}, + ri_coverage={}, + ri_utilization={}, + ) + assert posture["notes"] + assert all(n.strip() for n in posture["notes"]) + + +# ---------------------------------------------------------- reconciliation + + +@pytest.mark.unit +@pytest.mark.parametrize( + "ce,coh,expected", + [ + (1000.0, 1000.0, "reconciled"), + (1000.0, 950.0, "reconciled"), + (1000.0, 800.0, "minor-variance"), + (1000.0, 500.0, "material-variance"), + (0.0, 0.0, "agree-zero"), + ], +) +def test_reconciliation_bands(ce, coh, expected): + f = analyze_sp_recommendation(sp_rec("9.0", "10.0", hourly="10.0", monthly=str(ce))) + findings = [f] if f else [] + result = reconcile_with_coh( + findings, + {"recommendations": [{"estimated_monthly_savings": coh, "recommended_resource_type": "ComputeSavingsPlans", "current_resource_type": ""}] if coh else [], "count": 1 if coh else 0}, + ) + assert result["status"] == expected + + +@pytest.mark.unit +def test_reconciliation_unavailable_when_coh_errors(): + result = reconcile_with_coh([], {"error": "not enrolled"}) + assert result["status"] == "unavailable" + + +@pytest.mark.unit +def test_reconciliation_compares_unadjusted_figures(): + """COH publishes a best case, so the like-for-like axis is the API figure.""" + f = analyze_sp_recommendation(sp_rec("2.0", "10.0", monthly="1000.0")) + assert f.safe_monthly_savings < f.api_monthly_savings + result = reconcile_with_coh( + [f], + {"recommendations": [{"estimated_monthly_savings": 1000.0, "recommended_resource_type": "ComputeSavingsPlans", "current_resource_type": ""}], "count": 1}, + ) + assert result["status"] == "reconciled" + assert result["ce_monthly_savings"] == pytest.approx(1000.0) + + +# ------------------------------------------------------------- selection + + +@pytest.mark.unit +def test_selection_keeps_one_permutation_per_family(): + weak = analyze_sp_recommendation(sp_rec("9.0", "10.0", monthly="500.0")) + strong = analyze_sp_recommendation(sp_rec("9.0", "10.0", monthly="1500.0")) + strong.term = "THREE_YEARS" + best = select_best_findings([weak, strong]) + assert len(best) == 1 + assert best[0].safe_monthly_savings == pytest.approx(1500.0) + + +@pytest.mark.unit +def test_selection_breaks_ties_toward_shorter_term(): + one_year = analyze_sp_recommendation(sp_rec("9.0", "10.0", monthly="1000.0")) + three_year = analyze_sp_recommendation(sp_rec("9.0", "10.0", monthly="1000.0")) + three_year.term = "THREE_YEARS" + best = select_best_findings([three_year, one_year]) + assert best[0].term == "ONE_YEAR" + + +# ---------------------------------------------------------------- report + + +def _payload(findings, posture=None, recon=None): + return { + "meta": { + "account_id": "111122223333", + "profile": "test", + "generated_at": "2026-08-07 00:00 UTC", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + }, + "findings": findings, + "posture": posture or {"blockers": [], "notes": []}, + "reconciliation": recon or {"status": "unavailable", "reason": "test"}, + "eligible_spend": {"periods": [{"start": "2026-07-01", "end": "2026-08-01", "total": 5000.0, "by_service": {"Amazon Elastic Compute Cloud - Compute": 4000.0}}]}, + "errors": [], + } + + +@pytest.mark.integration +def test_report_renders_with_findings(): + f = analyze_sp_recommendation(sp_rec("6.0", "10.0")) + md = render(_payload([f])) + assert "# AWS Discounted Commitments Report" in md + assert "Risk-adjusted achievable savings" in md + assert "Compute Savings Plan" in md + # The haircut must be stated, not hidden. + assert "below the AWS best case" in md + + +@pytest.mark.integration +def test_report_renders_empty_case(): + md = render(_payload([])) + assert "No commitment opportunity found" in md + assert "real result, not a failure" in md + + +@pytest.mark.integration +def test_report_surfaces_blockers_before_recommendations(): + f = analyze_sp_recommendation(sp_rec("9.0", "10.0")) + posture = {"blockers": ["Existing Savings Plans are only 70.0% utilized"], "notes": []} + md = render(_payload([f], posture=posture)) + assert "Do not act on these numbers yet" in md + assert md.index("Blockers") < md.index("## Recommended commitments") + + +@pytest.mark.integration +def test_report_flags_material_variance(): + f = analyze_sp_recommendation(sp_rec("9.0", "10.0")) + recon = { + "status": "material-variance", + "ce_monthly_savings": 1000.0, + "coh_monthly_savings": 400.0, + "delta": 600.0, + "delta_pct": 60.0, + "coh_count": 1, + "coh_by_resource_type": {"ComputeSavingsPlans": 400.0}, + } + md = render(_payload([f], recon=recon)) + assert "MATERIAL VARIANCE" in md + + +@pytest.mark.integration +def test_report_zero_posture_does_not_imply_wasted_commitment(): + """All-zero metrics mean no commitments exist, not 0% utilization.""" + posture = { + "blockers": [], + "notes": [], + "sp_coverage_pct": 0.0, + "ri_coverage_pct": 0.0, + "ri_utilization_pct": 0.0, + } + md = render(_payload([], posture=posture)) + assert "nothing to measure" in md + assert "| Savings Plans coverage | 0.0% |" not in md + + +@pytest.mark.integration +def test_report_marks_break_even_past_the_term(): + rec = sp_rec(minimum="2.0", average="10.0", monthly="1000.0") + rec["details"][0]["UpfrontCost"] = "38000.0" + md = render(_payload([analyze_sp_recommendation(rec)])) + assert "cannot pay back" in md + assert "longer than the 12-month term" in md + + +@pytest.mark.integration +def test_report_has_no_blank_table_rows(): + """Markdown tables break if a heading is not preceded by a blank line.""" + f = analyze_sp_recommendation(sp_rec("6.0", "10.0")) + md = render(_payload([f])) + lines = md.split("\n") + for i, line in enumerate(lines): + if line.startswith("#") and i > 0: + assert lines[i - 1] == "", f"heading {line!r} not preceded by blank line" diff --git a/tests/unit/test_commitments_api.py b/tests/unit/test_commitments_api.py new file mode 100644 index 0000000..2ba17be --- /dev/null +++ b/tests/unit/test_commitments_api.py @@ -0,0 +1,319 @@ +"""Tests for the AWS API wrapper layer, using stub clients (no network calls). + +Focus is on the two things that actually bite: Cost Explorer's blank error +messages, and correct request shaping (notably that OfferingClass is only sent +for EC2, which is the one parameter the API rejects elsewhere). +""" + +from __future__ import annotations + +import pytest +from botocore.exceptions import ClientError + +from commitments import api + + +def client_error(code: str, message: str = "") -> ClientError: + return ClientError( + {"Error": {"Code": code, "Message": message}}, "OperationName" + ) + + +class StubCE: + """Records calls and replays canned results or raises.""" + + def __init__(self, result=None, error: ClientError | None = None): + self.result = result or {} + self.error = error + self.calls: list[dict] = [] + + def _respond(self, **kwargs): + self.calls.append(kwargs) + if self.error: + raise self.error + return self.result + + get_savings_plans_purchase_recommendation = _respond + get_reservation_purchase_recommendation = _respond + get_savings_plans_coverage = _respond + get_savings_plans_utilization = _respond + get_reservation_coverage = _respond + get_reservation_utilization = _respond + get_cost_and_usage = _respond + + +def clients(ce=None, coh=None) -> api.Clients: + return api.Clients( + ce=ce or StubCE(), coh=coh or StubCE(), account_id="111122223333", profile="t" + ) + + +# ------------------------------------------------------- error normalization + + +@pytest.mark.unit +def test_blank_data_unavailable_gets_readable_message(): + """Cost Explorer returns DataUnavailableException with an EMPTY message.""" + ce = StubCE(error=client_error("DataUnavailableException", "")) + result = api.get_sp_utilization(clients(ce=ce), 30) + assert result["error_code"] == "DataUnavailableException" + assert "no active commitment" in result["error"].lower() + assert result["error"].strip() + + +@pytest.mark.unit +def test_blank_unknown_error_still_produces_text(): + ce = StubCE(error=client_error("SomeOtherException", "")) + result = api.get_sp_coverage(clients(ce=ce), 30) + assert result["error"].strip() + + +@pytest.mark.unit +def test_real_error_message_is_preserved(): + ce = StubCE(error=client_error("ValidationException", "Invalid Service.")) + result = api.get_ri_recommendation( + clients(ce=ce), "Amazon Redshift", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + assert result["error"] == "Invalid Service." + + +# ---------------------------------------------------------- request shaping + + +@pytest.mark.unit +def test_offering_class_sent_only_for_ec2(): + """RDS/Redshift reject ServiceSpecification; EC2 requires it for STANDARD.""" + ce = StubCE(result={"Recommendations": [], "Metadata": {}}) + c = clients(ce=ce) + + api.get_ri_recommendation( + c, "Amazon Elastic Compute Cloud - Compute", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + assert "ServiceSpecification" in ce.calls[0] + + api.get_ri_recommendation( + c, "Amazon Relational Database Service", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + assert "ServiceSpecification" not in ce.calls[1] + + +@pytest.mark.unit +def test_sp_request_sends_all_required_params(): + """All four are required by the API; omitting any is a ValidationException.""" + ce = StubCE(result={"SavingsPlansPurchaseRecommendation": {}, "Metadata": {}}) + api.get_sp_recommendation( + clients(ce=ce), "COMPUTE_SP", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + sent = ce.calls[0] + for required in ( + "SavingsPlansType", + "TermInYears", + "PaymentOption", + "LookbackPeriodInDays", + ): + assert required in sent + + +@pytest.mark.unit +def test_time_period_end_is_today_and_start_is_days_back(): + from datetime import date, timedelta + + ce = StubCE(result={"SavingsPlansCoverages": []}) + api.get_sp_coverage(clients(ce=ce), 30) + tp = ce.calls[0]["TimePeriod"] + assert tp["End"] == date.today().isoformat() + assert tp["Start"] == (date.today() - timedelta(days=30)).isoformat() + + +# --------------------------------------------------------- response mapping + + +@pytest.mark.unit +def test_sp_recommendation_unwraps_nested_payload(): + ce = StubCE( + result={ + "SavingsPlansPurchaseRecommendation": { + "SavingsPlansPurchaseRecommendationSummary": {"HourlyCommitmentToPurchase": "5.0"}, + "SavingsPlansPurchaseRecommendationDetails": [{"UpfrontCost": "0"}], + }, + "Metadata": {"RecommendationId": "abc", "GenerationTimestamp": "2026-08-07"}, + } + ) + result = api.get_sp_recommendation( + clients(ce=ce), "COMPUTE_SP", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + assert result["summary"]["HourlyCommitmentToPurchase"] == "5.0" + assert len(result["details"]) == 1 + assert result["recommendation_id"] == "abc" + + +@pytest.mark.unit +def test_ri_recommendation_reads_first_recommendation_entry(): + ce = StubCE( + result={ + "Recommendations": [ + { + "RecommendationSummary": {"TotalEstimatedMonthlySavingsAmount": "100"}, + "RecommendationDetails": [{"UpfrontCost": "0"}, {"UpfrontCost": "0"}], + } + ], + "Metadata": {}, + } + ) + result = api.get_ri_recommendation( + clients(ce=ce), "Amazon ElastiCache", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + assert result["summary"]["TotalEstimatedMonthlySavingsAmount"] == "100" + assert len(result["details"]) == 2 + assert result["label"] == "ElastiCache" + + +@pytest.mark.unit +def test_ri_recommendation_handles_empty_recommendations(): + ce = StubCE(result={"Recommendations": [], "Metadata": {}}) + result = api.get_ri_recommendation( + clients(ce=ce), "Amazon Redshift", "ONE_YEAR", "NO_UPFRONT", "THIRTY_DAYS", "PAYER" + ) + assert result["summary"] == {} + assert result["details"] == [] + assert "error" not in result + + +@pytest.mark.unit +def test_eligible_spend_aggregates_service_groups(): + ce = StubCE( + result={ + "ResultsByTime": [ + { + "TimePeriod": {"Start": "2026-07-01", "End": "2026-08-01"}, + "Groups": [ + {"Keys": ["EC2"], "Metrics": {"UnblendedCost": {"Amount": "100.5"}}}, + {"Keys": ["RDS"], "Metrics": {"UnblendedCost": {"Amount": "50.25"}}}, + ], + } + ] + } + ) + result = api.get_eligible_spend(clients(ce=ce), 60) + period = result["periods"][0] + assert period["total"] == pytest.approx(150.75) + assert period["by_service"]["EC2"] == pytest.approx(100.5) + + +# ------------------------------------------------- Cost Optimization Hub + + +class StubCOH: + def __init__(self, items=None, enrollment=None, error=None): + self.items = items or [] + self.enrollment = enrollment + self.error = error + self.paginate_kwargs = None + + def list_enrollment_statuses(self, **kwargs): + if self.error: + raise self.error + return self.enrollment or {} + + def get_paginator(self, name): + stub = self + + class Paginator: + def paginate(self, **kwargs): + stub.paginate_kwargs = kwargs + if stub.error: + raise stub.error + return [{"items": stub.items}] + + return Paginator() + + +@pytest.mark.unit +def test_coh_enrollment_active(): + coh = StubCOH( + enrollment={"items": [{"status": "Active", "accountId": "1"}], "includeMemberAccounts": True} + ) + result = api.get_coh_enrollment(clients(coh=coh)) + assert result["enrolled"] is True + + +@pytest.mark.unit +def test_coh_enrollment_empty_means_not_enrolled(): + result = api.get_coh_enrollment(clients(coh=StubCOH(enrollment={"items": []}))) + assert result["enrolled"] is False + assert result["status"] == "NOT_ENROLLED" + + +@pytest.mark.unit +def test_coh_enrollment_access_denied_is_not_fatal(): + coh = StubCOH(error=client_error("AccessDeniedException", "no perms")) + result = api.get_coh_enrollment(clients(coh=coh)) + assert result["enrolled"] is False + assert "error" in result + + +@pytest.mark.unit +def test_coh_filters_to_commitment_purchases_only(): + """Rightsizing/idle findings must not dilute a commitment report.""" + coh = StubCOH(items=[]) + api.get_coh_commitment_recommendations(clients(coh=coh)) + flt = coh.paginate_kwargs["filter"] + assert set(flt["actionTypes"]) == { + "PurchaseSavingsPlans", + "PurchaseReservedInstances", + } + assert "Ec2Instance" not in flt["resourceTypes"] + assert "ComputeSavingsPlans" in flt["resourceTypes"] + + +@pytest.mark.unit +def test_coh_recommendations_normalize_null_savings(): + """COH returns None for savings on some records; arithmetic must not break.""" + coh = StubCOH( + items=[ + { + "recommendationId": "r1", + "estimatedMonthlySavings": None, + "estimatedSavingsPercentage": None, + "recommendedResourceType": "ComputeSavingsPlans", + } + ] + ) + result = api.get_coh_commitment_recommendations(clients(coh=coh)) + rec = result["recommendations"][0] + assert rec["estimated_monthly_savings"] == 0 + assert sum(r["estimated_monthly_savings"] for r in result["recommendations"]) == 0 + + +# ------------------------------------------------------------- constants + + +@pytest.mark.unit +def test_ri_service_list_matches_api_supported_values(): + """Guards against someone adding a guessed service name. + + This list was read back from the ValidationException the API raises on an + unknown Service value. Changing it requires re-probing, not guessing. + """ + assert api.RI_SERVICES == ( + "Amazon Elastic Compute Cloud - Compute", + "Amazon Relational Database Service", + "Amazon Redshift", + "Amazon ElastiCache", + "Amazon Elasticsearch Service", + "Amazon OpenSearch Service", + "Amazon MemoryDB Service", + "Amazon DynamoDB Service", + ) + assert set(api.RI_SERVICE_LABELS) == set(api.RI_SERVICES) + + +@pytest.mark.unit +def test_sp_types_cover_all_four_plan_families(): + assert set(api.SP_TYPES) == { + "COMPUTE_SP", + "EC2_INSTANCE_SP", + "SAGEMAKER_SP", + "DATABASE_SP", + } + assert set(api.SP_TYPE_LABELS) == set(api.SP_TYPES) diff --git a/tests/unit/test_commitments_collect.py b/tests/unit/test_commitments_collect.py new file mode 100644 index 0000000..ebabfd7 --- /dev/null +++ b/tests/unit/test_commitments_collect.py @@ -0,0 +1,371 @@ +"""Tests for the shared pipeline: parameter resolution, sweep, JSON envelope. + +`commitments.collect` is what the tool handlers drive: they parse the event and +build clients, and everything after that happens here. The envelope key names +are asserted explicitly because they are the contract with whatever renders the +recommendations — renaming one silently breaks the report and the frontend. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from commitments import api, collect +from commitments.analyze import analyze_sp_recommendation + +from tests.unit.test_commitments_analyze import sp_rec + + +# ------------------------------------------------------------- test doubles + + +class RecordingCE: + """Counts calls per operation so the sweep's shape can be asserted.""" + + def __init__(self): + self.sp_calls: list[dict] = [] + self.ri_calls: list[dict] = [] + + def get_savings_plans_purchase_recommendation(self, **kw): + self.sp_calls.append(kw) + return {"SavingsPlansPurchaseRecommendation": {}, "Metadata": {}} + + def get_reservation_purchase_recommendation(self, **kw): + self.ri_calls.append(kw) + return {"Recommendations": [], "Metadata": {}} + + def get_savings_plans_coverage(self, **kw): + return {"SavingsPlansCoverages": []} + + def get_savings_plans_utilization(self, **kw): + return {"Total": {}, "SavingsPlansUtilizationsByTime": []} + + def get_reservation_coverage(self, **kw): + return {"Total": {}, "CoveragesByTime": []} + + def get_reservation_utilization(self, **kw): + return {"Total": {}, "UtilizationsByTime": []} + + def get_cost_and_usage(self, **kw): + return {"ResultsByTime": []} + + +class NotEnrolledCOH: + def list_enrollment_statuses(self, **kw): + return {"items": []} + + +def recording_clients(ce): + return api.Clients( + ce=ce, coh=NotEnrolledCOH(), account_id="111122223333", profile=None + ) + + +# ------------------------------------------------------ RI service resolution + + +@pytest.mark.unit +def test_all_resolves_to_every_verified_service(): + assert collect.resolve_ri_services(["all"]) == list(api.RI_SERVICES) + + +@pytest.mark.unit +def test_short_labels_resolve_to_full_api_names(): + """Users type 'RDS', the API demands the long name.""" + assert collect.resolve_ri_services(["rds", "EC2"]) == [ + "Amazon Relational Database Service", + "Amazon Elastic Compute Cloud - Compute", + ] + + +@pytest.mark.unit +def test_full_api_names_pass_through(): + assert collect.resolve_ri_services(["Amazon Redshift"]) == ["Amazon Redshift"] + + +@pytest.mark.unit +def test_unknown_service_raises_rather_than_querying(): + """A typo must fail loudly, not silently drop a service from the sweep.""" + with pytest.raises(ValueError) as exc: + collect.resolve_ri_services(["Aurora"]) + assert "Aurora" in str(exc.value) + + +@pytest.mark.unit +def test_parenthetical_label_matches_on_base_word(): + """'Elasticsearch (legacy)' must resolve from plain 'Elasticsearch'.""" + expected = ["Amazon Elasticsearch Service"] + assert collect.resolve_ri_services(["Elasticsearch"]) == expected + assert collect.resolve_ri_services(["elasticsearch"]) == expected + assert collect.resolve_ri_services(["Elasticsearch (legacy)"]) == expected + + +@pytest.mark.unit +def test_every_label_is_reachable_by_its_base_word(): + """Guards against a label nobody can type in a comma-separated flag.""" + for service, label in api.RI_SERVICE_LABELS.items(): + base = label.split("(")[0].strip() + assert collect.resolve_ri_services([base]) == [service], f"{label} unreachable" + + +@pytest.mark.unit +def test_duplicate_tokens_are_dropped(): + """Cost Explorer bills per request; 'EC2, ec2' must not pay twice.""" + assert collect.resolve_ri_services(["EC2", "ec2"]) == [ + "Amazon Elastic Compute Cloud - Compute" + ] + assert collect.resolve_sp_types(["compute_sp", "COMPUTE_SP"]) == ["COMPUTE_SP"] + + +@pytest.mark.unit +def test_blank_entries_are_ignored(): + assert collect.resolve_ri_services(["rds", "", " "]) == [ + "Amazon Relational Database Service" + ] + + +# ------------------------------------------------------- SP type resolution + + +@pytest.mark.unit +def test_sp_types_all_and_case_insensitive(): + assert collect.resolve_sp_types(["all"]) == list(api.SP_TYPES) + assert collect.resolve_sp_types(["compute_sp"]) == ["COMPUTE_SP"] + + +@pytest.mark.unit +def test_unknown_sp_type_raises(): + with pytest.raises(ValueError) as exc: + collect.resolve_sp_types(["GRAVITON_SP"]) + assert "GRAVITON_SP" in str(exc.value) + + +@pytest.mark.unit +def test_validate_choices_names_the_offender(): + with pytest.raises(ValueError) as exc: + collect.validate_choices(["TWO_YEARS"], collect.TERMS, "term") + assert "TWO_YEARS" in str(exc.value) + + +# ------------------------------------------------------------- collect_all() + + +@pytest.mark.unit +def test_rejects_invalid_term_before_calling_aws(): + ce = RecordingCE() + with pytest.raises(ValueError): + collect.collect_all(recording_clients(ce), terms=["TWO_YEARS"]) + assert ce.sp_calls == [] + + +@pytest.mark.unit +def test_rejects_invalid_payment_before_calling_aws(): + ce = RecordingCE() + with pytest.raises(ValueError): + collect.collect_all(recording_clients(ce), payments=["MONTHLY"]) + assert ce.sp_calls == [] + + +@pytest.mark.unit +def test_rejects_invalid_family_before_calling_aws(): + ce = RecordingCE() + with pytest.raises(ValueError): + collect.collect_all(recording_clients(ce), families=["spot"]) + assert ce.sp_calls == [] + + +@pytest.mark.unit +def test_sweeps_every_permutation(): + """4 SP types and 8 RI services, each across every term x payment.""" + ce = RecordingCE() + collect.collect_all( + recording_clients(ce), + terms=["ONE_YEAR", "THREE_YEARS"], + payments=["NO_UPFRONT", "ALL_UPFRONT"], + ) + + assert len(ce.sp_calls) == len(api.SP_TYPES) * 2 * 2 + assert len(ce.ri_calls) == len(api.RI_SERVICES) * 2 * 2 + assert {c["SavingsPlansType"] for c in ce.sp_calls} == set(api.SP_TYPES) + assert {c["Service"] for c in ce.ri_calls} == set(api.RI_SERVICES) + + +@pytest.mark.unit +def test_honors_family_filter(): + ce = RecordingCE() + collect.collect_all( + recording_clients(ce), families=["sp"], terms=["ONE_YEAR"], + payments=["NO_UPFRONT"], + ) + assert ce.sp_calls + assert ce.ri_calls == [] + + +@pytest.mark.unit +def test_honors_sp_type_filter(): + ce = RecordingCE() + collect.collect_all( + recording_clients(ce), families=["sp"], sp_types=["COMPUTE_SP"], + terms=["ONE_YEAR"], payments=["NO_UPFRONT"], + ) + assert {c["SavingsPlansType"] for c in ce.sp_calls} == {"COMPUTE_SP"} + + +@pytest.mark.unit +def test_marks_coh_unavailable_when_not_enrolled(): + """Reconciliation must degrade to a stated caveat, not a crash.""" + payload = collect.collect_all( + recording_clients(RecordingCE()), families=["sp"], terms=["ONE_YEAR"], + payments=["NO_UPFRONT"], + ) + coh = payload["raw"]["coh"] + assert "error" in coh + assert "not available" in coh["error"] + + +@pytest.mark.unit +def test_records_per_query_errors_without_aborting(): + """One failing permutation must not lose the other 15.""" + + class PartlyBrokenCE(RecordingCE): + def get_savings_plans_purchase_recommendation(self, **kw): + if kw["SavingsPlansType"] == "SAGEMAKER_SP": + raise RuntimeError("throttled") + return super().get_savings_plans_purchase_recommendation(**kw) + + payload = collect.collect_all( + recording_clients(PartlyBrokenCE()), families=["sp"], terms=["ONE_YEAR"], + payments=["NO_UPFRONT"], + ) + assert len(payload["errors"]) == 1 + # Labelled for humans, not with the raw enum — same wording both hosts emit. + assert "SageMaker" in payload["errors"][0]["query"] + # The other three SP types still produced results. + assert len(payload["raw"]["sp_recs"]) == len(api.SP_TYPES) - 1 + + +@pytest.mark.unit +def test_payload_carries_every_render_key(): + """`report.render` indexes these directly; a missing key is a KeyError.""" + payload = collect.collect_all( + recording_clients(RecordingCE()), families=["sp"], terms=["ONE_YEAR"], + payments=["NO_UPFRONT"], profile="prod", + ) + for key in ( + "meta", "findings", "posture", "reconciliation", "eligible_spend", "errors" + ): + assert key in payload, f"render key {key} missing" + assert payload["meta"]["account_id"] == "111122223333" + assert payload["meta"]["profile"] == "prod" + + +@pytest.mark.unit +def test_posture_is_collected_even_when_no_family_is_swept(): + """Coverage/utilization health is useful on its own.""" + payload = collect.collect_all( + recording_clients(RecordingCE()), families=[], terms=["ONE_YEAR"], + payments=["NO_UPFRONT"], + ) + assert payload["findings"] == [] + assert "posture" in payload + + +# ------------------------------------------------------------ JSON envelope + + +@pytest.mark.unit +def test_envelope_matches_consumer_key_names(): + """These keys are the contract with every host consuming the analysis.""" + f = analyze_sp_recommendation(sp_rec("6.0", "10.0")) + env = collect.envelope([f], {"status": "reconciled"}) + + assert set(env) == { + "recommendations", + "count", + "total_estimated_monthly_savings", + "aws_best_case_monthly_savings", + "reconciliation", + } + assert env["count"] == 1 + rec = env["recommendations"][0] + for key in ( + "estimated_monthly_savings", + "estimated_savings_percentage", + "implementation_effort", + "commitment_unit", + ): + assert key in rec, f"consumer key {key} missing" + + +@pytest.mark.unit +def test_envelope_omits_reconciliation_when_not_supplied(): + """A sizing-only caller has nothing to reconcile; the key must not appear empty.""" + f = analyze_sp_recommendation(sp_rec("6.0", "10.0")) + assert "reconciliation" not in collect.envelope([f]) + + +@pytest.mark.unit +def test_envelope_reports_adjusted_and_best_case_separately(): + """Collapsing these two into one number is the error this skill exists to fix.""" + f = analyze_sp_recommendation(sp_rec("2.0", "10.0", monthly="1000.0")) + env = collect.envelope([f], {}) + assert env["total_estimated_monthly_savings"] == pytest.approx(200.0) + assert env["aws_best_case_monthly_savings"] == pytest.approx(1000.0) + assert ( + env["total_estimated_monthly_savings"] < env["aws_best_case_monthly_savings"] + ) + + +@pytest.mark.unit +def test_envelope_is_serializable(): + f = analyze_sp_recommendation(sp_rec("6.0", "10.0")) + env = collect.envelope([f], {"status": "reconciled", "delta": 0.0}) + reloaded = json.loads(json.dumps(env, default=str)) + assert reloaded["recommendations"][0]["confidence"] == "Medium" + + +@pytest.mark.unit +def test_envelope_empty_findings_totals_zero(): + env = collect.envelope([], {"status": "agree-zero"}) + assert env["count"] == 0 + assert env["total_estimated_monthly_savings"] == 0 + assert env["recommendations"] == [] + + +@pytest.mark.unit +def test_envelope_break_even_is_null_not_zero(): + """A no-upfront plan has no break-even; 0 would read as 'pays back instantly'.""" + f = analyze_sp_recommendation(sp_rec("9.0", "10.0")) + env = collect.envelope([f], {}) + assert env["recommendations"][0]["break_even_months"] is None + + +@pytest.mark.unit +def test_savings_plan_commitment_unit_is_hourly_dollars(): + """RI 'commitment' is a unit count; SP is $/hr. Mislabeling misreads by 1000x.""" + f = analyze_sp_recommendation(sp_rec("6.0", "10.0")) + assert collect.serialize_finding(f)["commitment_unit"] == "USD/hour" + + +# ------------------------------------------------- clients stay the caller's + + +@pytest.mark.unit +def test_module_builds_no_clients_and_touches_no_files(): + """The pipeline must stay drivable by a caller that supplies its own clients. + + `collect` runs in Lambda, where a boto3 profile has no meaning and the only + writable path is /tmp. Keeping session construction, argument parsing and + file IO out of this module is what leaves the handler as the single place + credentials are resolved. + """ + source = ( + Path(__file__).resolve().parents[2] + / "src" / "lambda" / "mcp" / "commitments" / "commitments" / "collect.py" + ).read_text() + assert "import boto3" not in source + assert "import argparse" not in source + assert "open(" not in source diff --git a/tests/unit/test_commitments_expiry.py b/tests/unit/test_commitments_expiry.py new file mode 100644 index 0000000..0298ebf --- /dev/null +++ b/tests/unit/test_commitments_expiry.py @@ -0,0 +1,791 @@ +"""Tests for commitment expiry inventory and renewal recommendations. + +Three things here are worth more than the rest, and are what these tests are +mostly about: + +1. **Per-service field names.** Every reservation API names the same six + concepts differently and only EC2 returns an end date; the other five have + to derive it from `StartTime` + `Duration`. A typo in `InventorySpec` is + silent — you get commitments with no id and no expiry rather than an error. +2. **The unit distinction.** Savings Plans commit in USD/hour, reservations in + unit counts. Reading one as the other misreads it by roughly 1000x, so + nothing converts RI units to money. +3. **Savings Plans must not be multiplied by region.** They are account-level; + sweeping them per region would silently N-times every total. +""" + +from __future__ import annotations + +from datetime import date, datetime, timedelta, timezone + +import pytest +from botocore.exceptions import ClientError + +from commitments import analyze, api, collect, report + +ONE_YEAR_SECONDS = 31536000 +THREE_YEAR_SECONDS = 94608000 + +AS_OF = date(2026, 9, 4) + + +def client_error(code: str, message: str = "boom") -> ClientError: + return ClientError({"Error": {"Code": code, "Message": message}}, "Describe") + + +class StubDescribe: + """A regional client that replays one canned response, or raises.""" + + def __init__(self, response=None, error: ClientError | None = None): + self.response = response or {} + self.error = error + self.calls = 0 + + def _respond(self, **kwargs): + self.calls += 1 + if self.error: + raise self.error + return self.response + + describe_reserved_instances = _respond + describe_reserved_db_instances = _respond + describe_reserved_cache_nodes = _respond + describe_reserved_nodes = _respond + + +class StubSavingsPlans: + """describe_savings_plans, optionally paginated.""" + + def __init__(self, pages=None, error: ClientError | None = None): + self.pages = pages or [{"savingsPlans": []}] + self.error = error + self.calls: list[dict] = [] + + def describe_savings_plans(self, **kwargs): + self.calls.append(kwargs) + if self.error: + raise self.error + return self.pages[len(self.calls) - 1] + + +def clients(factory=None) -> api.Clients: + return api.Clients( + ce=StubDescribe(), + coh=StubDescribe(), + account_id="111122223333", + profile=None, + make_client=factory, + ) + + +# --------------------------------------------------------- date normalization + + +@pytest.mark.unit +class TestDateCoercion: + """boto3 hands back datetimes; a CLI JSON dump hands back strings.""" + + @pytest.mark.parametrize( + "value", + [ + datetime(2026, 10, 1, 12, 0, tzinfo=timezone.utc), + datetime(2026, 10, 1, 12, 0), + date(2026, 10, 1), + "2026-10-01T12:00:00Z", + "2026-10-01T12:00:00+00:00", + "2026-10-01", + ], + ) + def test_every_shape_an_sdk_or_cli_returns_normalizes(self, value): + assert api._iso_day(api._as_datetime(value)) == "2026-10-01" + + @pytest.mark.parametrize("value", [None, "", "not-a-date", 12345]) + def test_unparseable_returns_none_rather_than_raising(self, value): + """One malformed row must not lose the rest of the inventory.""" + assert api._as_datetime(value) is None + + def test_naive_datetime_is_read_as_utc(self): + naive = api._as_datetime(datetime(2026, 10, 1)) + assert naive.tzinfo is timezone.utc + + def test_term_months_labels_both_real_terms(self): + start = datetime(2026, 1, 1, tzinfo=timezone.utc) + assert api._term_months(start, start + timedelta(seconds=ONE_YEAR_SECONDS)) == 12 + assert ( + api._term_months(start, start + timedelta(seconds=THREE_YEAR_SECONDS)) == 36 + ) + + def test_term_months_is_none_when_either_end_is_unknown(self): + assert api._term_months(None, datetime.now(timezone.utc)) is None + assert api._term_months(datetime.now(timezone.utc), None) is None + + +# ------------------------------------------------------- reservation inventory + + +@pytest.mark.unit +class TestReservationInventory: + def test_ec2_uses_the_end_field_it_actually_returns(self): + """EC2 is the only reservation API that returns an explicit End.""" + stub = StubDescribe( + { + "ReservedInstances": [ + { + "ReservedInstancesId": "ri-abc", + "InstanceCount": 4, + "InstanceType": "m5.large", + "Start": datetime(2025, 10, 1, tzinfo=timezone.utc), + "End": datetime(2026, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + "OfferingType": "No Upfront", + } + ] + } + ) + result = api.get_reservation_inventory( + clients(lambda svc, region: stub), "ec2", "ap-northeast-1" + ) + [item] = result["items"] + assert item["commitment_id"] == "ri-abc" + assert item["end"] == "2026-10-01" + assert item["quantity"] == 4 + assert item["unit"] == "units" + assert item["region"] == "ap-northeast-1" + assert item["term_months"] == 12 + + @pytest.mark.parametrize( + ("service", "response"), + [ + ( + "rds", + { + "ReservedDBInstances": [ + { + "ReservedDBInstanceId": "rds-1", + "DBInstanceCount": 2, + "DBInstanceClass": "db.r5.large", + "StartTime": datetime(2025, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + "ReservedDBInstanceArn": "arn:aws:rds:::ri/rds-1", + } + ] + }, + ), + ( + "elasticache", + { + "ReservedCacheNodes": [ + { + "ReservedCacheNodeId": "ec-1", + "CacheNodeCount": 2, + "CacheNodeType": "cache.r6g.large", + "StartTime": datetime(2025, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + "ReservationARN": "arn:aws:elasticache:::ri/ec-1", + } + ] + }, + ), + ( + "redshift", + { + "ReservedNodes": [ + { + "ReservedNodeId": "rs-1", + "NodeCount": 2, + "NodeType": "ra3.xlplus", + "StartTime": datetime(2025, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + } + ] + }, + ), + ( + "opensearch", + { + "ReservedInstances": [ + { + "ReservedInstanceId": "os-1", + "InstanceCount": 2, + "InstanceType": "r6g.large.search", + "StartTime": datetime(2025, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + "PaymentOption": "NO_UPFRONT", + } + ] + }, + ), + ( + "memorydb", + { + "ReservedNodes": [ + { + "ReservationId": "mdb-1", + "NodeCount": 2, + "NodeType": "db.r6g.large", + "StartTime": datetime(2025, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + "ARN": "arn:aws:memorydb:::ri/mdb-1", + } + ] + }, + ), + ], + ) + def test_end_date_is_derived_from_start_plus_duration(self, service, response): + """Only EC2 returns End; the other five must be computed or they are blank.""" + stub = StubDescribe(response) + result = api.get_reservation_inventory( + clients(lambda svc, region: stub), service, "us-east-1" + ) + [item] = result["items"] + assert item["end"] == "2026-10-01", f"{service} end date not derived" + assert item["start"] == "2025-10-01" + assert item["commitment_id"], f"{service} id_field is wrong" + assert item["quantity"] == 2, f"{service} count_field is wrong" + assert item["instance_type"], f"{service} type_field is wrong" + assert item["term_months"] == 12 + + def test_every_spec_maps_to_a_real_boto3_method_name(self): + """A method typo only shows up at call time, which is too late.""" + for spec in api.RESERVATION_INVENTORY: + assert spec.method.startswith("describe_") + assert spec.response_key + assert spec.key in api.INVENTORY_KEYS + + def test_retired_reservations_are_excluded(self): + """A retired reservation is history, not something to renew.""" + stub = StubDescribe( + { + "ReservedInstances": [ + { + "ReservedInstancesId": "gone", + "InstanceCount": 1, + "InstanceType": "m5.large", + "Start": datetime(2023, 1, 1, tzinfo=timezone.utc), + "End": datetime(2024, 1, 1, tzinfo=timezone.utc), + "State": "retired", + } + ] + } + ) + result = api.get_reservation_inventory( + clients(lambda svc, region: stub), "ec2", "us-east-1" + ) + assert result["items"] == [] + + def test_client_error_is_reported_with_the_failed_query_identified(self): + stub = StubDescribe(error=client_error("UnauthorizedOperation")) + result = api.get_reservation_inventory( + clients(lambda svc, region: stub), "ec2", "eu-west-1" + ) + assert result["error_code"] == "UnauthorizedOperation" + assert result["service"] == "ec2" + assert result["region"] == "eu-west-1" + + def test_missing_client_factory_degrades_to_an_explained_error(self): + """A host that granted only ce:Get* must get a warning, not a crash.""" + result = api.get_reservation_inventory(clients(None), "ec2", "us-east-1") + assert "error" in result + assert "Describe" in result["error"] + + def test_unknown_family_is_a_programming_error_not_a_silent_skip(self): + with pytest.raises(ValueError, match="Unknown reservation family"): + api.get_reservation_inventory(clients(), "dynamodb", "us-east-1") + + def test_dynamodb_is_not_claimed_as_covered(self): + """There is no describe-reserved-capacity API; pretending otherwise + would report "nothing expiring" for a reservation that does.""" + assert "dynamodb" not in api.INVENTORY_KEYS + assert any("DynamoDB" in s for s in api.INVENTORY_BLIND_SPOTS) + + +# ------------------------------------------------------ savings plan inventory + + +@pytest.mark.unit +class TestSavingsPlanInventory: + def test_commitment_is_dollars_per_hour_not_a_unit_count(self): + sp = StubSavingsPlans( + [ + { + "savingsPlans": [ + { + "savingsPlanId": "sp-1", + "savingsPlanArn": "arn:aws:savingsplans::sp/sp-1", + "savingsPlanType": "Compute", + "commitment": "5.50", + "start": "2025-10-01T00:00:00Z", + "end": "2026-10-01T00:00:00Z", + "state": "active", + "paymentOption": "No Upfront", + "region": "", + } + ] + } + ] + ) + result = api.get_savings_plan_inventory(clients(lambda svc, region: sp)) + [item] = result["items"] + assert item["unit"] == "USD/hour" + assert item["quantity"] == 5.50 + assert item["family"] == "savings-plan" + assert item["end"] == "2026-10-01" + assert item["term_months"] == 12 + # An empty region on a Compute SP means account-wide, not "unknown". + assert item["region"] == "global" + + def test_only_active_states_are_requested(self): + sp = StubSavingsPlans() + api.get_savings_plan_inventory(clients(lambda svc, region: sp)) + assert sp.calls[0]["states"] == list(api.ACTIVE_SP_STATES) + + def test_pagination_is_followed(self): + """Stopping at page one silently under-reports the expiring total.""" + sp = StubSavingsPlans( + [ + { + "savingsPlans": [ + {"savingsPlanId": "sp-1", "commitment": "1.0", + "end": "2026-10-01T00:00:00Z", "state": "active"} + ], + "nextToken": "more", + }, + { + "savingsPlans": [ + {"savingsPlanId": "sp-2", "commitment": "2.0", + "end": "2026-11-01T00:00:00Z", "state": "active"} + ] + }, + ] + ) + result = api.get_savings_plan_inventory(clients(lambda svc, region: sp)) + assert [i["commitment_id"] for i in result["items"]] == ["sp-1", "sp-2"] + assert sp.calls[1]["nextToken"] == "more" + + def test_client_error_is_reported_not_raised(self): + sp = StubSavingsPlans(error=client_error("AccessDeniedException")) + result = api.get_savings_plan_inventory(clients(lambda svc, region: sp)) + assert result["error_code"] == "AccessDeniedException" + + def test_missing_client_factory_names_the_permission(self): + result = api.get_savings_plan_inventory(clients(None)) + assert "DescribeSavingsPlans" in result["error"] + + +# -------------------------------------------------------------- expiry verdict + + +def sp_item(end: str, quantity: float = 1.0, commitment_id: str = "sp-1") -> dict: + return { + "family": "savings-plan", + "service": "savingsplans", + "label": "Compute Savings Plan", + "commitment_id": commitment_id, + "quantity": quantity, + "unit": "USD/hour", + "region": "global", + "end": end, + } + + +def ri_item(end: str, quantity: float = 2.0, commitment_id: str = "ri-1") -> dict: + return { + "family": "reservation", + "service": "ec2", + "label": "EC2", + "commitment_id": commitment_id, + "quantity": quantity, + "unit": "units", + "region": "us-east-1", + "end": end, + } + + +@pytest.mark.unit +class TestExpiryBands: + @pytest.mark.parametrize( + ("days_out", "urgency"), + [(1, "urgent"), (30, "urgent"), (31, "soon"), (60, "soon"), + (61, "upcoming"), (90, "upcoming")], + ) + def test_days_remaining_maps_to_the_documented_band(self, days_out, urgency): + end = (AS_OF + timedelta(days=days_out)).isoformat() + result = analyze.analyze_expiry([sp_item(end)], AS_OF, 90) + [entry] = result["expiring"] + assert entry["urgency"] == urgency + assert entry["days_remaining"] == days_out + + def test_beyond_the_horizon_is_not_reported(self): + end = (AS_OF + timedelta(days=120)).isoformat() + result = analyze.analyze_expiry([sp_item(end)], AS_OF, 90) + assert result["expiring"] == [] + # Still counted as inventoried, so the report can say how many exist. + assert result["total_active"] == 1 + + def test_already_ended_is_separated_from_expiring(self): + """A lapsed commitment is a different, more urgent conversation than a + renewal — the spend it covered is already back at on-demand rates.""" + end = (AS_OF - timedelta(days=5)).isoformat() + result = analyze.analyze_expiry([sp_item(end)], AS_OF, 90) + assert result["expiring"] == [] + [entry] = result["expired"] + assert entry["urgency"] == "expired" + assert "5 days ago" in entry["rationale"] + + def test_undated_commitment_is_declared_not_dropped(self): + result = analyze.analyze_expiry([sp_item("")], AS_OF, 90) + assert result["expiring"] == [] + assert len(result["undated"]) == 1 + + def test_output_is_sorted_soonest_first(self): + items = [ + sp_item((AS_OF + timedelta(days=80)).isoformat(), commitment_id="late"), + sp_item((AS_OF + timedelta(days=10)).isoformat(), commitment_id="soon"), + ] + result = analyze.analyze_expiry(items, AS_OF, 90) + assert [e["commitment_id"] for e in result["expiring"]] == ["soon", "late"] + + +@pytest.mark.unit +class TestRenewalVerdict: + @pytest.mark.parametrize( + ("utilization", "action"), + [ + (99.5, analyze.RENEW), + (95.0, analyze.RENEW), + (94.9, analyze.RENEW_SMALLER), + (50.0, analyze.RENEW_SMALLER), + (49.9, analyze.LET_LAPSE), + (0.0, analyze.LET_LAPSE), + ], + ) + def test_utilization_drives_the_verdict(self, utilization, action): + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry( + [sp_item(end)], AS_OF, 90, sp_utilization_pct=utilization + ) + assert result["expiring"][0]["action"] == action + + def test_unmeasured_utilization_asks_for_review_rather_than_guessing(self): + """The utilization figure is what makes the call defensible; without it + the honest answer is "review", not a default to renew.""" + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry([sp_item(end)], AS_OF, 90) + entry = result["expiring"][0] + assert entry["action"] == analyze.REVIEW + assert "could not be measured" in entry["rationale"] + + def test_each_family_uses_its_own_utilization_figure(self): + """SP utilization must not decide a reservation's renewal.""" + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry( + [sp_item(end), ri_item(end)], + AS_OF, + 90, + sp_utilization_pct=99.0, + ri_utilization_pct=20.0, + ) + by_family = {e["family"]: e for e in result["expiring"]} + assert by_family["savings-plan"]["action"] == analyze.RENEW + assert by_family["reservation"]["action"] == analyze.LET_LAPSE + + def test_rationale_quotes_the_figure_it_relied_on(self): + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry( + [sp_item(end)], AS_OF, 90, sp_utilization_pct=72.5 + ) + assert "72.5%" in result["expiring"][0]["rationale"] + + def test_renew_smaller_names_expiry_as_a_free_resize_point(self): + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry( + [sp_item(end)], AS_OF, 90, sp_utilization_pct=70.0 + ) + assert "zero-cost resize" in result["expiring"][0]["rationale"] + + +@pytest.mark.unit +class TestExpiryRollup: + def test_savings_plan_hourly_commitment_converts_to_monthly_money(self): + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry([sp_item(end, quantity=5.0)], AS_OF, 90) + assert result["hourly_commitment_expiring"] == 5.0 + assert result["monthly_committed_spend_expiring"] == pytest.approx( + 5.0 * analyze.HOURS_PER_MONTH + ) + + def test_reservation_units_are_never_converted_to_money(self): + """Turning unit counts into dollars needs pricing this module does not + query — inventing a rate would be a fabricated figure.""" + end = (AS_OF + timedelta(days=20)).isoformat() + result = analyze.analyze_expiry([ri_item(end, quantity=7.0)], AS_OF, 90) + assert result["reserved_units_expiring"] == 7.0 + assert result["hourly_commitment_expiring"] == 0.0 + assert result["monthly_committed_spend_expiring"] == 0.0 + + def test_counts_and_actions_tally_the_reported_rows(self): + items = [ + sp_item((AS_OF + timedelta(days=10)).isoformat(), commitment_id="a"), + sp_item((AS_OF + timedelta(days=45)).isoformat(), commitment_id="b"), + sp_item((AS_OF + timedelta(days=200)).isoformat(), commitment_id="far"), + ] + result = analyze.analyze_expiry(items, AS_OF, 90, sp_utilization_pct=99.0) + assert result["counts"] == {"urgent": 1, "soon": 1, "upcoming": 0} + assert result["actions"][analyze.RENEW] == 2 + assert result["total_active"] == 3 + + def test_as_of_is_injected_so_the_function_stays_pure(self): + end = (AS_OF + timedelta(days=5)).isoformat() + assert analyze.analyze_expiry([sp_item(end)], AS_OF, 90)["as_of"] == ( + "2026-09-04" + ) + + +# ------------------------------------------------------------- collection wiring + + +@pytest.mark.unit +class TestResolveRegions: + def test_empty_falls_back_to_the_hosts_own_region(self): + assert collect.resolve_regions([], "ap-northeast-1") == ["ap-northeast-1"] + + def test_duplicates_collapse_so_totals_are_not_doubled(self): + assert collect.resolve_regions(["us-east-1", "us-east-1"]) == ["us-east-1"] + + @pytest.mark.parametrize( + "bad", ["us-east", "US-EAST-1 extra", "../../etc", "useast1", "x"] + ) + def test_non_region_input_is_rejected_before_it_reaches_an_endpoint(self, bad): + """Region names become part of an SDK endpoint, so they are validated + rather than passed through.""" + with pytest.raises(ValueError, match="valid AWS region"): + collect.resolve_regions([bad]) + + def test_case_is_normalized(self): + assert collect.resolve_regions(["US-East-1"]) == ["us-east-1"] + + def test_no_regions_and_no_default_is_a_caller_error(self): + with pytest.raises(ValueError, match="At least one AWS region"): + collect.resolve_regions([]) + + +@pytest.mark.unit +class TestCollectExpiry: + def test_savings_plans_are_fetched_once_regardless_of_region_count(self): + """SPs are account-level. Sweeping them per region would return the same + plans N times and inflate every total by N.""" + sp_calls = {"n": 0} + + def factory(service, region): + if service == "savingsplans": + sp_calls["n"] += 1 + return StubSavingsPlans( + [ + { + "savingsPlans": [ + { + "savingsPlanId": "sp-1", + "commitment": "3.0", + "end": (AS_OF + timedelta(days=10)).isoformat(), + "state": "active", + "savingsPlanType": "Compute", + } + ] + } + ] + ) + return StubDescribe() + + expiry, errors = collect.collect_expiry( + clients(factory), ["us-east-1", "eu-west-1", "ap-northeast-1"], + ["ec2"], 90, as_of=AS_OF, + ) + assert sp_calls["n"] == 1 + assert len(expiry["expiring"]) == 1 + assert errors == [] + + def test_reservations_are_swept_per_region(self): + seen: list[tuple[str, str]] = [] + + def factory(service, region): + seen.append((service, region)) + return StubDescribe() if service != "savingsplans" else StubSavingsPlans() + + collect.collect_expiry( + clients(factory), ["us-east-1", "eu-west-1"], ["ec2", "rds"], 90, + as_of=AS_OF, + ) + assert ("ec2", "us-east-1") in seen + assert ("ec2", "eu-west-1") in seen + assert ("rds", "eu-west-1") in seen + + def test_one_failed_family_does_not_lose_the_others(self): + def factory(service, region): + if service == "rds": + return StubDescribe(error=client_error("AccessDenied")) + if service == "savingsplans": + return StubSavingsPlans() + return StubDescribe( + { + "ReservedInstances": [ + { + "ReservedInstancesId": "ri-ok", + "InstanceCount": 1, + "InstanceType": "m5.large", + "Start": datetime(2025, 10, 1, tzinfo=timezone.utc), + "End": datetime(2026, 10, 1, tzinfo=timezone.utc), + "State": "active", + } + ] + } + ) + + expiry, errors = collect.collect_expiry( + clients(factory), ["us-east-1"], ["ec2", "rds"], 90, as_of=AS_OF + ) + assert [e["commitment_id"] for e in expiry["expiring"]] == ["ri-ok"] + assert len(errors) == 1 + assert "rds" in errors[0]["query"] + + def test_unknown_family_is_rejected_with_the_valid_list(self): + with pytest.raises(ValueError, match="reservation family"): + collect.collect_expiry(clients(), ["us-east-1"], ["dynamodb"]) + + def test_regions_and_blind_spots_are_carried_into_the_result(self): + expiry, _ = collect.collect_expiry( + clients(lambda s, r: StubDescribe()), ["us-east-1"], ["ec2"], 90, + as_of=AS_OF, + ) + assert expiry["regions"] == ["us-east-1"] + assert expiry["blind_spots"] == list(api.INVENTORY_BLIND_SPOTS) + + +@pytest.mark.unit +class TestCollectAllIntegration: + def test_expiry_is_skipped_when_no_regions_are_requested(self): + """Expiry needs Describe* permissions beyond ce:Get*, so a caller that + does not ask must not have the calls made on its behalf.""" + called = {"n": 0} + + def factory(service, region): + called["n"] += 1 + return StubDescribe() + + payload = collect.collect_all(clients(factory), families=["sp"]) + assert payload["expiry"] is None + assert called["n"] == 0 + + def test_expiry_errors_are_kept_out_of_the_billable_query_count(self): + """queries_run exists to track Cost Explorer's $0.01-per-request + billing; free Describe* failures must not inflate it.""" + payload = collect.collect_all( + clients(None), families=["sp"], regions=["us-east-1"] + ) + assert payload["expiry"] is not None + assert len(payload["errors"]) > len(payload["sweep_errors"]) + + def test_invalid_region_is_rejected_by_collect_all_too(self): + with pytest.raises(ValueError, match="valid AWS region"): + collect.collect_all(clients(), families=["sp"], regions=["nope"]) + + +# -------------------------------------------------------------------- reporting + + +def payload_with_expiry(expiry) -> dict: + return { + "meta": { + "account_id": "111122223333", + "profile": None, + "generated_at": "2026-09-04 09:00 UTC", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + }, + "findings": [], + "posture": {"blockers": [], "notes": []}, + "reconciliation": {"status": "unavailable", "reason": "not enrolled"}, + "eligible_spend": {"periods": []}, + "expiry": expiry, + "errors": [], + } + + +@pytest.mark.unit +class TestExpiryReporting: + def _expiry(self, **overrides): + base = analyze.analyze_expiry( + [ + sp_item((AS_OF + timedelta(days=12)).isoformat(), quantity=5.5), + ri_item((AS_OF + timedelta(days=70)).isoformat(), quantity=4.0), + ], + AS_OF, + 90, + sp_utilization_pct=99.0, + ri_utilization_pct=40.0, + ) + base["regions"] = ["ap-northeast-1"] + base["blind_spots"] = list(api.INVENTORY_BLIND_SPOTS) + base.update(overrides) + return base + + def test_section_renders_with_dates_actions_and_units(self): + md = report.render(payload_with_expiry(self._expiry())) + assert "## Commitment expiry and renewal" in md + assert "**renew**" in md + assert "let lapse" in md + # SP size must carry its unit, RI size must not gain a dollar sign. + assert "$5.5000/hr" in md + assert "4 unit(s)" in md + + def test_urgent_expiry_is_surfaced_in_the_bottom_line(self): + """A deadline outranks an optional purchase.""" + md = report.render(payload_with_expiry(self._expiry())) + bottom = md.split("## Reconciliation")[0] + assert "expire within 30 days" in bottom + + def test_reserved_units_are_not_quoted_as_money(self): + md = report.render(payload_with_expiry(self._expiry())) + assert "needs per-instance pricing" in md + + def test_account_level_utilization_caveat_is_stated(self): + md = report.render(payload_with_expiry(self._expiry())) + assert "not\nper-commitment" in md or "not per-commitment" in md + + def test_dynamodb_blind_spot_is_disclosed(self): + md = report.render(payload_with_expiry(self._expiry())) + assert "Not covered by this inventory" in md + assert "DynamoDB" in md + + def test_nothing_expiring_says_so_rather_than_printing_an_empty_table(self): + expiry = self._expiry(expiring=[], counts={"urgent": 0, "soon": 0, "upcoming": 0}) + md = report.render(payload_with_expiry(expiry)) + assert "No commitment expires within 90 days" in md + assert "| Ends | Days |" not in md + + def test_expired_but_active_gets_its_own_subsection(self): + expiry = analyze.analyze_expiry( + [sp_item((AS_OF - timedelta(days=3)).isoformat())], AS_OF, 90, + sp_utilization_pct=99.0, + ) + expiry["regions"] = ["us-east-1"] + md = report.render(payload_with_expiry(expiry)) + assert "Already ended but still listed as active" in md + + def test_payload_without_expiry_still_renders(self): + """Report templates and cached payloads predate this section.""" + payload = payload_with_expiry(None) + md = report.render(payload) + assert "## Commitment expiry and renewal" not in md + assert "## Bottom line" in md + + def test_payload_missing_the_key_entirely_still_renders(self): + payload = payload_with_expiry(None) + del payload["expiry"] + assert "## Bottom line" in report.render(payload) diff --git a/tests/unit/test_commitments_spec.py b/tests/unit/test_commitments_spec.py new file mode 100644 index 0000000..b26ac95 --- /dev/null +++ b/tests/unit/test_commitments_spec.py @@ -0,0 +1,809 @@ +"""Tests for the purchasable specification carried on commitments. + +A commitment recommendation without a specification is not actionable. "Buy 4 +RDS reservations" does not say `db.r6g.large · Multi-AZ · Aurora PostgreSQL`, +and a reservation only discounts usage matching its exact specification — so the +things these tests guard are: + +1. **Per-service field names.** Cost Explorer buries the spec in a + service-specific sub-structure and names every field differently. A typo is + silent: you get a blank spec, not an error. Two services break the pattern + outright — OpenSearch splits its type across `InstanceClass` + `InstanceSize`, + DynamoDB has no instance at all and lives under a different container. +2. **`MultiAZ` is a bool.** `False` means "Single-AZ", which is information. Drop + it as falsy and a reader is left assuming Multi-AZ. +3. **The aggregate is a budget, not an order.** One finding can span several + distinct specs, so the per-line breakdown has to survive into both the + markdown report and the JSON envelope. +""" + +from __future__ import annotations + +from datetime import date, datetime, timezone + +import pytest + +from commitments import api, collect +from commitments.analyze import ( + SPEC_UNAVAILABLE, + analyze_expiry, + analyze_ri_recommendation, + analyze_sp_recommendation, +) +from commitments.report import render + +ONE_YEAR_SECONDS = 31536000 + + +# ------------------------------------------------- recommendation spec shapes + + +@pytest.mark.unit +class TestDescribeRecommendationSpec: + def test_ec2_reports_type_az_and_platform(self): + spec = api.describe_recommendation_spec( + { + "InstanceDetails": { + "EC2InstanceDetails": { + "Family": "m5", + "InstanceType": "m5.xlarge", + "Region": "ap-northeast-1", + "AvailabilityZone": "ap-northeast-1a", + "Platform": "Linux/UNIX", + "Tenancy": "default", + "CurrentGeneration": True, + "SizeFlexEligible": False, + } + } + } + ) + assert spec["instance_type"] == "m5.xlarge" + assert spec["family"] == "m5" + assert spec["region"] == "ap-northeast-1" + assert spec["attributes"]["AZ"] == "ap-northeast-1a" + assert spec["label"].startswith("m5.xlarge · ap-northeast-1a") + assert spec["label"].endswith("ap-northeast-1") + + def test_rds_reports_the_deployment_option_and_engine(self): + """The two dimensions that decide whether a reservation applies at all.""" + spec = api.describe_recommendation_spec( + { + "InstanceDetails": { + "RDSInstanceDetails": { + "Family": "db.r6g", + "InstanceType": "db.r6g.large", + "Region": "us-east-1", + "DatabaseEngine": "Aurora PostgreSQL", + "DatabaseEdition": "", + "DeploymentOption": "Multi-AZ", + "LicenseModel": "No license required", + "CurrentGeneration": True, + "SizeFlexEligible": True, + } + } + } + ) + assert spec["instance_type"] == "db.r6g.large" + assert spec["attributes"]["deployment"] == "Multi-AZ" + assert spec["attributes"]["engine"] == "Aurora PostgreSQL" + assert "edition" not in spec["attributes"], "blank fields must not pad the spec" + assert spec["size_flex_eligible"] is True + # License model is part of what a reservation matches on (BYOL and + # license-included are priced and sold separately), so it stays. + assert spec["label"] == ( + "db.r6g.large · Multi-AZ · Aurora PostgreSQL · " + "No license required · us-east-1" + ) + + def test_single_az_rds_is_stated_rather_than_implied(self): + spec = api.describe_recommendation_spec( + { + "InstanceDetails": { + "RDSInstanceDetails": { + "InstanceType": "db.t4g.medium", + "DeploymentOption": "Single-AZ", + "DatabaseEngine": "PostgreSQL", + } + } + } + ) + assert "Single-AZ" in spec["label"] + + def test_elasticache_reads_node_type_not_instance_type(self): + spec = api.describe_recommendation_spec( + { + "InstanceDetails": { + "ElastiCacheInstanceDetails": { + "Family": "cache.r6g", + "NodeType": "cache.r6g.large", + "Region": "us-west-2", + "ProductDescription": "redis", + } + } + } + ) + assert spec["instance_type"] == "cache.r6g.large" + assert spec["attributes"]["engine"] == "redis" + + @pytest.mark.parametrize( + ("key", "payload"), + [ + ( + "RedshiftInstanceDetails", + {"Family": "ra3", "NodeType": "ra3.xlplus", "Region": "us-east-1"}, + ), + ( + "MemoryDBInstanceDetails", + {"Family": "db.r6g", "NodeType": "db.r6g.large", "Region": "us-east-1"}, + ), + ], + ) + def test_node_based_services_still_yield_a_type(self, key, payload): + spec = api.describe_recommendation_spec({"InstanceDetails": {key: payload}}) + assert spec["instance_type"] == payload["NodeType"] + assert spec["region"] == "us-east-1" + + def test_opensearch_joins_the_two_fields_it_splits_the_type_across(self): + """`ESInstanceDetails` has no InstanceType — only class and size.""" + spec = api.describe_recommendation_spec( + { + "InstanceDetails": { + "ESInstanceDetails": { + "InstanceClass": "r6g", + "InstanceSize": "large.search", + "Region": "eu-west-1", + "CurrentGeneration": True, + } + } + } + ) + assert spec["instance_type"] == "r6g.large.search" + assert spec["family"] == "", "ESInstanceDetails has no Family field" + + def test_opensearch_missing_size_does_not_leave_a_trailing_dot(self): + spec = api.describe_recommendation_spec( + {"InstanceDetails": {"ESInstanceDetails": {"InstanceClass": "r6g"}}} + ) + assert spec["instance_type"] == "r6g" + + def test_dynamodb_lives_under_a_different_container_and_has_no_instance(self): + spec = api.describe_recommendation_spec( + { + "ReservedCapacityDetails": { + "DynamoDBCapacityDetails": { + "CapacityUnits": "1000", + "Region": "us-east-1", + } + } + } + ) + assert spec["instance_type"] == "" + assert spec["attributes"]["capacity units"] == "1000 capacity units" + assert spec["label"] == "1000 capacity units · us-east-1" + + def test_unknown_shape_degrades_to_empty_rather_than_raising(self): + """A new service must cost the caller the spec, not the whole finding.""" + assert api.describe_recommendation_spec({}) == {} + assert api.describe_recommendation_spec({"InstanceDetails": {}}) == {} + assert ( + api.describe_recommendation_spec( + {"InstanceDetails": {"QuantumInstanceDetails": {"Foo": "bar"}}} + ) + == {} + ) + + def test_previous_generation_is_recorded(self): + """A 3-year commitment on an old generation locks out the cheaper one.""" + spec = api.describe_recommendation_spec( + { + "InstanceDetails": { + "EC2InstanceDetails": { + "InstanceType": "m4.large", + "CurrentGeneration": False, + } + } + } + ) + assert spec["current_generation"] is False + + def test_every_spec_key_is_distinct_so_none_shadows_another(self): + assert len(set(api.RECOMMENDATION_SPEC_KEYS)) == len(api.RECOMMENDATION_SPECS) + + +@pytest.mark.unit +class TestAttributeDisplay: + def test_a_bare_number_carries_its_label(self): + assert api._attribute_display("capacity units", "1000") == "1000 capacity units" + + def test_a_named_value_reads_as_itself(self): + assert api._attribute_display("engine", "Aurora MySQL") == "Aurora MySQL" + + def test_blank_is_dropped(self): + assert api._attribute_display("AZ", " ") == "" + assert api._attribute_display("AZ", None) == "" + + +# --------------------------------------------------- inventory spec attributes + + +class StubDescribe: + def __init__(self, response): + self.response = response + + def _respond(self, **kwargs): + return self.response + + describe_reserved_instances = _respond + describe_reserved_db_instances = _respond + describe_reserved_cache_nodes = _respond + describe_reserved_nodes = _respond + + +def clients_returning(response) -> api.Clients: + stub = StubDescribe(response) + return api.Clients( + ce=stub, + coh=stub, + account_id="111122223333", + profile=None, + make_client=lambda service, region: stub, + ) + + +def rds_reservation(**overrides) -> dict: + row = { + "ReservedDBInstanceId": "rds-1", + "DBInstanceCount": 2, + "DBInstanceClass": "db.r6g.large", + "StartTime": datetime(2025, 10, 1, tzinfo=timezone.utc), + "Duration": ONE_YEAR_SECONDS, + "State": "active", + "MultiAZ": True, + "ProductDescription": "postgresql", + } + row.update(overrides) + return {"ReservedDBInstances": [row]} + + +@pytest.mark.unit +class TestReservationAttributes: + def test_rds_inventory_reports_multi_az_and_engine(self): + result = api.get_reservation_inventory( + clients_returning(rds_reservation()), "rds", "ap-northeast-1" + ) + [item] = result["items"] + assert item["attributes"]["deployment"] == "Multi-AZ" + assert item["attributes"]["engine"] == "postgresql" + assert item["spec"] == "db.r6g.large · Multi-AZ · postgresql" + + def test_multi_az_false_is_single_az_not_missing(self): + """A bool has to be tested against None; False is meaningful here.""" + result = api.get_reservation_inventory( + clients_returning(rds_reservation(MultiAZ=False)), "rds", "us-east-1" + ) + [item] = result["items"] + assert item["attributes"]["deployment"] == "Single-AZ" + assert "Single-AZ" in item["spec"] + + def test_absent_multi_az_is_omitted_rather_than_guessed(self): + payload = rds_reservation() + del payload["ReservedDBInstances"][0]["MultiAZ"] + result = api.get_reservation_inventory( + clients_returning(payload), "rds", "us-east-1" + ) + [item] = result["items"] + assert "deployment" not in item["attributes"] + + def test_ec2_zonal_scope_reports_the_availability_zone(self): + """A zonal reservation only covers one AZ, so a renewal must match it.""" + result = api.get_reservation_inventory( + clients_returning( + { + "ReservedInstances": [ + { + "ReservedInstancesId": "ri-1", + "InstanceCount": 3, + "InstanceType": "m5.large", + "Start": datetime(2025, 10, 1, tzinfo=timezone.utc), + "End": datetime(2026, 10, 1, tzinfo=timezone.utc), + "State": "active", + "Scope": "Availability Zone", + "AvailabilityZone": "ap-northeast-1c", + "ProductDescription": "Linux/UNIX", + "OfferingClass": "convertible", + "InstanceTenancy": "default", + } + ] + } + ), + "ec2", + "ap-northeast-1", + ) + [item] = result["items"] + assert item["attributes"]["AZ"] == "ap-northeast-1c" + assert item["attributes"]["scope"] == "Availability Zone" + assert item["attributes"]["class"] == "convertible" + assert item["spec"].startswith("m5.large · Availability Zone · ap-northeast-1c") + + def test_savings_plan_spec_is_the_family_or_empty_by_design(self): + """A Compute plan commits to dollars, so a blank spec is correct.""" + + class StubSP: + def describe_savings_plans(self, **kwargs): + return { + "savingsPlans": [ + { + "savingsPlanId": "sp-1", + "savingsPlanType": "EC2Instance", + "commitment": "5.0", + "start": "2025-10-01T00:00:00Z", + "end": "2026-10-01T00:00:00Z", + "state": "active", + "ec2InstanceFamily": "m5", + "region": "ap-northeast-1", + "paymentOption": "No Upfront", + }, + { + "savingsPlanId": "sp-2", + "savingsPlanType": "Compute", + "commitment": "8.0", + "start": "2025-10-01T00:00:00Z", + "end": "2026-10-01T00:00:00Z", + "state": "active", + }, + ] + } + + stub = StubSP() + result = api.get_savings_plan_inventory( + api.Clients( + ce=stub, + coh=stub, + account_id="111122223333", + profile=None, + make_client=lambda service, region=None: stub, + ) + ) + ec2_plan, compute_plan = result["items"] + assert ec2_plan["spec"] == "m5" + assert compute_plan["spec"] == "" + assert compute_plan["attributes"] == {} + + +# ------------------------------------------------------------ RI line items + + +def ri_rec(details: list[dict], monthly: str = "900.0") -> dict: + return { + "service": "Amazon Relational Database Service", + "label": "RDS", + "term": "ONE_YEAR", + "payment": "ALL_UPFRONT", + "summary": { + "TotalEstimatedMonthlySavingsAmount": monthly, + "TotalEstimatedMonthlySavingsPercentage": "31.0", + }, + "details": details, + } + + +def rds_detail( + instance_type: str, + deployment: str, + recommended: str, + savings: str, + **overrides, +) -> dict: + detail = { + "RecommendedNumberOfInstancesToPurchase": recommended, + "MinimumNumberOfInstancesUsedPerHour": recommended, + "AverageNumberOfInstancesUsedPerHour": recommended, + "EstimatedMonthlySavingsAmount": savings, + "EstimatedMonthlyOnDemandCost": "2000.0", + "UpfrontCost": "6000.0", + "AverageUtilization": "94.0", + "AccountId": "111122223333", + "InstanceDetails": { + "RDSInstanceDetails": { + "Family": instance_type.rsplit(".", 1)[0], + "InstanceType": instance_type, + "Region": "ap-northeast-1", + "DeploymentOption": deployment, + "DatabaseEngine": "Aurora PostgreSQL", + "CurrentGeneration": True, + "SizeFlexEligible": True, + } + }, + } + detail.update(overrides) + return detail + + +@pytest.mark.unit +class TestRiLineItems: + def test_each_line_carries_its_own_spec_and_savings(self): + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0"), + rds_detail("db.t4g.medium", "Single-AZ", "2", "300.0"), + ] + ) + ) + assert [i.spec for i in f.line_items] == [ + "db.r6g.large · Multi-AZ · Aurora PostgreSQL · ap-northeast-1", + "db.t4g.medium · Single-AZ · Aurora PostgreSQL · ap-northeast-1", + ] + assert [i.monthly_savings for i in f.line_items] == [600.0, 300.0] + assert all(i.region == "ap-northeast-1" for i in f.line_items) + assert all(i.unit == "units" for i in f.line_items) + + def test_lines_are_ranked_by_savings_not_api_order(self): + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.t4g.medium", "Single-AZ", "2", "100.0"), + rds_detail("db.r6g.large", "Multi-AZ", "4", "800.0"), + ] + ) + ) + assert f.line_items[0].spec.startswith("db.r6g.large") + + def test_a_multi_spec_total_is_disclosed_as_a_budget_not_an_order(self): + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0"), + rds_detail("db.t4g.medium", "Single-AZ", "2", "300.0"), + ] + ) + ) + joined = " ".join(f.rationale) + assert "2 distinct instance specifications" in joined + assert "budget, not an order" in joined + + def test_a_single_spec_recommendation_adds_no_such_caveat(self): + f = analyze_ri_recommendation( + ri_rec([rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0")]) + ) + assert not any("distinct instance specifications" in n for n in f.rationale) + + def test_achievable_per_line_is_whole_reservations(self): + """Reservations are sold whole, so a line never asks for 2.4 of one.""" + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0"), + rds_detail("db.t4g.medium", "Single-AZ", "3", "300.0"), + ] + ) + ) + for item in f.line_items: + assert item.achievable == int(item.achievable) + assert item.achievable <= item.recommended + + def test_missing_sub_structure_is_labelled_rather_than_left_blank(self): + detail = rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0") + del detail["InstanceDetails"] + f = analyze_ri_recommendation(ri_rec([detail])) + assert f.line_items[0].spec == SPEC_UNAVAILABLE + + def test_capacity_unit_services_fall_back_to_their_own_field_names(self): + f = analyze_ri_recommendation( + { + "service": "Amazon DynamoDB Service", + "label": "DynamoDB", + "term": "ONE_YEAR", + "payment": "NO_UPFRONT", + "summary": {"TotalEstimatedMonthlySavingsAmount": "400.0"}, + "details": [ + { + "RecommendedNumberOfCapacityUnitsToPurchase": "100", + "MinimumNumberOfCapacityUnitsUsedPerHour": "95", + "AverageNumberOfCapacityUnitsUsedPerHour": "100", + "EstimatedMonthlySavingsAmount": "400.0", + "ReservedCapacityDetails": { + "DynamoDBCapacityDetails": { + "CapacityUnits": "100", + "Region": "us-east-1", + } + }, + } + ], + } + ) + [item] = f.line_items + assert item.recommended == pytest.approx(100.0) + assert item.floor == pytest.approx(95.0) + assert item.spec == "100 capacity units · us-east-1" + + def test_absent_utilization_is_none_not_zero(self): + """Zero utilization would read as a warning that AWS never issued.""" + detail = rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0") + del detail["AverageUtilization"] + f = analyze_ri_recommendation(ri_rec([detail])) + assert f.line_items[0].utilization_pct is None + + +@pytest.mark.unit +class TestSpLineItems: + def sp_rec(self, details: list[dict]) -> dict: + return { + "sp_type": "EC2_INSTANCE_SP", + "term": "ONE_YEAR", + "payment": "NO_UPFRONT", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + "summary": { + "HourlyCommitmentToPurchase": "10.0", + "EstimatedMonthlySavingsAmount": "1000.0", + "EstimatedSavingsPercentage": "20.5", + "CurrentOnDemandSpend": "8000.0", + }, + "details": details, + "recommendation_id": "rec-1", + } + + def test_ec2_instance_plan_reports_family_and_region(self): + f = analyze_sp_recommendation( + self.sp_rec( + [ + { + "HourlyCommitmentToPurchase": "6.0", + "CurrentMinimumHourlyOnDemandSpend": "6.0", + "CurrentAverageHourlyOnDemandSpend": "7.0", + "CurrentMaximumHourlyOnDemandSpend": "9.0", + "EstimatedMonthlySavingsAmount": "700.0", + "EstimatedAverageUtilization": "98.0", + "SavingsPlansDetails": { + "Region": "ap-northeast-1", + "InstanceFamily": "m5", + "OfferingId": "off-1", + }, + }, + { + "HourlyCommitmentToPurchase": "4.0", + "CurrentMinimumHourlyOnDemandSpend": "4.0", + "CurrentAverageHourlyOnDemandSpend": "5.0", + "EstimatedMonthlySavingsAmount": "300.0", + "SavingsPlansDetails": { + "Region": "us-east-1", + "InstanceFamily": "c6g", + }, + }, + ] + ) + ) + assert [(i.spec, i.region) for i in f.line_items] == [ + ("m5", "ap-northeast-1"), + ("c6g", "us-east-1"), + ] + assert all(i.unit == "USD/hour" for i in f.line_items) + + def test_a_compute_plan_says_it_is_flexible_rather_than_blank(self): + f = analyze_sp_recommendation( + self.sp_rec( + [ + { + "HourlyCommitmentToPurchase": "10.0", + "CurrentMinimumHourlyOnDemandSpend": "9.0", + "CurrentAverageHourlyOnDemandSpend": "10.0", + "EstimatedMonthlySavingsAmount": "1000.0", + } + ] + ) + ) + assert f.line_items[0].spec == "any instance family" + + def test_dollar_commitments_are_not_rounded_to_whole_units(self): + f = analyze_sp_recommendation( + self.sp_rec( + [ + { + "HourlyCommitmentToPurchase": "6.5", + "CurrentMinimumHourlyOnDemandSpend": "4.0", + "CurrentAverageHourlyOnDemandSpend": "8.0", + "EstimatedMonthlySavingsAmount": "700.0", + } + ] + ) + ) + [item] = f.line_items + assert item.achievable != int(item.achievable) or item.achievable == 6.5 + + +# ---------------------------------------------------------------- report surface + + +def payload(findings, expiry=None) -> dict: + return { + "meta": { + "account_id": "111122223333", + "profile": "test", + "generated_at": "2026-09-04 00:00 UTC", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + }, + "findings": findings, + "posture": {"blockers": [], "notes": []}, + "reconciliation": {"status": "unavailable", "reason": "test"}, + "eligible_spend": {"periods": []}, + "expiry": expiry, + "errors": [], + } + + +@pytest.mark.integration +class TestReportShowsTheSpec: + def test_line_items_table_names_each_instance_type_and_deployment(self): + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0"), + rds_detail("db.t4g.medium", "Single-AZ", "2", "300.0"), + ] + ) + ) + md = render(payload([f])) + assert "Line items — what to buy" in md + assert "db.r6g.large · Multi-AZ" in md + assert "db.t4g.medium · Single-AZ" in md + + def test_size_flexibility_and_generation_are_flagged_per_line(self): + detail = rds_detail("db.m4.large", "Single-AZ", "2", "300.0") + detail["InstanceDetails"]["RDSInstanceDetails"]["CurrentGeneration"] = False + detail["InstanceDetails"]["RDSInstanceDetails"]["SizeFlexEligible"] = True + md = render(payload([analyze_ri_recommendation(ri_rec([detail]))])) + assert "size-flexible" in md + assert "previous generation" in md + + def test_a_line_with_nothing_to_buy_is_not_listed(self): + detail = rds_detail("db.r6g.large", "Multi-AZ", "0", "0.0") + md = render(payload([analyze_ri_recommendation(ri_rec([detail]))])) + assert "Line items — what to buy" not in md + + def test_rounding_shortfall_is_disclosed_rather_than_padded(self): + """Per-line whole-unit rounding can undershoot the family total.""" + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.r6g.large", "Multi-AZ", "5", "600.0", **{ + "MinimumNumberOfInstancesUsedPerHour": "3", + "AverageNumberOfInstancesUsedPerHour": "5", + }), + rds_detail("db.t4g.medium", "Single-AZ", "3", "300.0", **{ + "MinimumNumberOfInstancesUsedPerHour": "2", + "AverageNumberOfInstancesUsedPerHour": "3", + }), + ] + ) + ) + allocated = sum(i.achievable for i in f.line_items) + md = render(payload([f])) + if allocated < f.safe_hourly_commitment: + assert "unallocated against the" in md + assert "carries no" in md and "unused-commitment risk" in md + + def test_expiry_table_shows_what_a_renewal_has_to_match(self): + expiry, _ = None, None + inventory = [ + { + "family": "reserved-instance", + "label": "RDS Reserved Instance", + "commitment_id": "rds-1", + "instance_type": "db.r6g.large", + "spec": "db.r6g.large · Multi-AZ · postgresql", + "attributes": {"deployment": "Multi-AZ", "engine": "postgresql"}, + "quantity": 2, + "unit": "units", + "region": "ap-northeast-1", + "start": "2025-10-01", + "end": "2026-10-01", + "state": "active", + "term_months": 12, + "payment_option": "All Upfront", + } + ] + expiry = analyze_expiry(inventory, date(2026, 9, 4), 90, ri_utilization_pct=95.0) + md = render(payload([], expiry=expiry)) + assert "| Spec |" in md + assert "db.r6g.large · Multi-AZ · postgresql" in md + assert "Single-AZ vs Multi-AZ" in md + + def test_a_specless_expiring_commitment_renders_a_dash(self): + expiry = analyze_expiry( + [ + { + "family": "savings-plan", + "label": "Compute Savings Plan", + "commitment_id": "sp-1", + "instance_type": "", + "spec": "", + "attributes": {}, + "quantity": 5.0, + "unit": "USD/hour", + "region": "", + "start": "2025-10-01", + "end": "2026-10-01", + "state": "active", + "term_months": 12, + "payment_option": "No Upfront", + } + ], + date(2026, 9, 4), + 90, + sp_utilization_pct=99.0, + ) + md = render(payload([], expiry=expiry)) + assert "| — |" in md + assert "commits to dollars, not to a family" in md + + +# --------------------------------------------------------------- JSON envelope + + +@pytest.mark.unit +class TestEnvelopeCarriesLineItems: + def test_line_items_reach_the_json_a_caller_reads(self): + f = analyze_ri_recommendation( + ri_rec( + [ + rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0"), + rds_detail("db.t4g.medium", "Single-AZ", "2", "300.0"), + ] + ) + ) + [rec] = collect.envelope([f])["recommendations"] + assert len(rec["line_items"]) == 2 + first = rec["line_items"][0] + assert first["spec"] == "db.r6g.large · Multi-AZ · Aurora PostgreSQL · ap-northeast-1" + assert first["region"] == "ap-northeast-1" + assert first["commitment_unit"] == "units" + assert first["aws_recommended_commitment"] == 4.0 + assert first["minimum_observed_units"] == 4.0 + assert first["estimated_monthly_savings"] == 600.0 + assert first["size_flexible"] is True + assert first["current_generation"] is True + assert first["account_id"] == "111122223333" + + def test_absent_utilization_serializes_as_null_not_zero(self): + detail = rds_detail("db.r6g.large", "Multi-AZ", "4", "600.0") + del detail["AverageUtilization"] + [rec] = collect.envelope([analyze_ri_recommendation(ri_rec([detail]))])[ + "recommendations" + ] + assert rec["line_items"][0]["estimated_utilization_percentage"] is None + + def test_savings_plan_lines_are_serialized_in_dollars_per_hour(self): + f = analyze_sp_recommendation( + { + "sp_type": "COMPUTE_SP", + "term": "ONE_YEAR", + "payment": "NO_UPFRONT", + "lookback": "THIRTY_DAYS", + "account_scope": "PAYER", + "summary": { + "HourlyCommitmentToPurchase": "10.0", + "EstimatedMonthlySavingsAmount": "1000.0", + "EstimatedSavingsPercentage": "20.0", + "CurrentOnDemandSpend": "8000.0", + }, + "details": [ + { + "HourlyCommitmentToPurchase": "10.0", + "CurrentMinimumHourlyOnDemandSpend": "9.0", + "CurrentAverageHourlyOnDemandSpend": "10.0", + "EstimatedMonthlySavingsAmount": "1000.0", + } + ], + } + ) + [rec] = collect.envelope([f])["recommendations"] + assert rec["line_items"][0]["commitment_unit"] == "USD/hour" + + def test_a_finding_with_no_line_items_serializes_an_empty_list(self): + """Absent is not the same as unknown; the key is always present.""" + f = analyze_ri_recommendation(ri_rec([], monthly="0.0")) + assert f is None or collect.envelope([f])["recommendations"][0][ + "line_items" + ] == [] diff --git a/tests/unit/test_commitments_tool.py b/tests/unit/test_commitments_tool.py new file mode 100644 index 0000000..02ccd44 --- /dev/null +++ b/tests/unit/test_commitments_tool.py @@ -0,0 +1,771 @@ +"""Unit tests for the commitments MCP tool — the event-adapter layer. + +The analysis itself (`commitments/api.py`, `analyze.py`, `collect.py`, +`report.py`) is covered by test_commitments_api.py, test_commitments_analyze.py +and test_commitments_collect.py. What this module covers is everything the +handler adds on top: + * Dispatcher — unknown tool returns error + tool list; known tool routes + * _get_clients — builds api.Clients from shared.cross_account (never + api.build_clients, which would need a boto3 profile that has no meaning + in Lambda), with the COH role alias, and caches across invocations + * Parameter validation — every ParamError path surfaces as {"error": ...} + * _resolve_ri_services / _resolve_sp_types — label + case tolerance + * collect.run_jobs — a throttled permutation is a warning, not a lost sweep + * collect.serialize_finding — commitment_unit differs by family + * handle_generate_commitment_analysis — end-to-end through the REAL + api/analyze/report modules against a stub Cost Explorer client, asserting + report_markdown is rendered and posture blockers are surfaced + * handle_get_commitment_posture / size_* — envelope shape + * Handler discipline — the sweep and the shared constants must not be + re-implemented here, where none of the above tests would see them + +The handler imports `commitments.*` and `shared.cross_account` at module scope, +so both must resolve before exec_module; conftest.py binds each. Note that the +handler is loaded from its file rather than imported by name, because the tool +directory also contains a `handler.py` and a sys.path entry for it would shadow +the top-level `handler` name every other Lambda test module imports (it +silently broke all 182 network-resilience tests when tried). +""" + +from __future__ import annotations + +import importlib.util +import re +import sys +from pathlib import Path +from types import SimpleNamespace +from unittest.mock import MagicMock + +import pytest + +_REPO_ROOT = Path(__file__).resolve().parents[2] +_TOOL_DIR = _REPO_ROOT / "src" / "lambda" / "mcp" / "commitments" + +_HANDLER_PATH = _TOOL_DIR / "handler.py" +_spec = importlib.util.spec_from_file_location("commitments_handler", _HANDLER_PATH) +handler = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(handler) +sys.modules["commitments_handler"] = handler + + +# --------------------------------------------------------------------------- +# Fixtures +# --------------------------------------------------------------------------- + + +@pytest.fixture(autouse=True) +def _reset_client_cache(): + """Clear the module-scope client cache between tests. + + _get_clients memoizes for warm-container reuse; leaving a previous test's + stub in place would make later tests pass for the wrong reason. + """ + handler._clients = None + yield + handler._clients = None + + +def _make_context(tool_name: str) -> SimpleNamespace: + """Build the AgentCore Gateway context shape the dispatcher reads.""" + return SimpleNamespace( + client_context=SimpleNamespace( + custom={"bedrockAgentCoreToolName": f"commitments___{tool_name}"} + ) + ) + + +def _stub_clients(monkeypatch, ce=None, coh=None, account_id="123456789012"): + """Install a stub api.Clients so no test can construct a real boto3 client.""" + clients = handler.api.Clients( + ce=ce if ce is not None else MagicMock(), + coh=coh if coh is not None else MagicMock(), + account_id=account_id, + profile=None, + ) + monkeypatch.setattr(handler, "_get_clients", lambda: clients) + return clients + + +def _sp_response(hourly="10.0", monthly_savings="1000.0", minimum="6.0", + average="10.0", upfront="0.0"): + """A Cost Explorer SP purchase recommendation, with CE's string numerics. + + The default floor/average ($6 trough vs $10 average = 0.6) lands in the + "moderate" volatility band on purpose, so the risk adjustment is exercised: + a flat 0.8+ workload would take the AWS figure unchanged and the tests + would not distinguish the adjusted path from a passthrough. + """ + return { + "SavingsPlansPurchaseRecommendation": { + "SavingsPlansPurchaseRecommendationSummary": { + "HourlyCommitmentToPurchase": hourly, + "EstimatedMonthlySavingsAmount": monthly_savings, + "EstimatedSavingsPercentage": "22.5", + "CurrentOnDemandSpend": "9000.0", + }, + "SavingsPlansPurchaseRecommendationDetails": [ + { + "CurrentMinimumHourlyOnDemandSpend": minimum, + "CurrentAverageHourlyOnDemandSpend": average, + "UpfrontCost": upfront, + "EstimatedAverageUtilization": "95.0", + } + ], + }, + "Metadata": {"GenerationTimestamp": "2026-09-01T00:00:00Z", + "RecommendationId": "sp-rec-1"}, + } + + +def _ce_stub(**overrides): + """Cost Explorer stub covering every call the collection path makes.""" + ce = MagicMock() + ce.get_savings_plans_purchase_recommendation.return_value = _sp_response() + ce.get_reservation_purchase_recommendation.return_value = { + "Recommendations": [], "Metadata": {}, + } + ce.get_savings_plans_coverage.return_value = { + "SavingsPlansCoverages": [ + {"Coverage": {"CoveragePercentage": "40.0", "OnDemandCost": "5000.0"}} + ] + } + # 62% utilization is below the warn threshold → must produce a blocker. + ce.get_savings_plans_utilization.return_value = { + "Total": {"Utilization": {"UtilizationPercentage": "62.0", + "UnusedCommitment": "1234.56"}} + } + ce.get_reservation_coverage.return_value = { + "Total": {"CoverageHours": {"CoverageHoursPercentage": "30.0", + "OnDemandHours": "700.0"}}, + "CoveragesByTime": [], + } + ce.get_reservation_utilization.return_value = { + "Total": {"UtilizationPercentage": "99.0", "UnusedHours": "1.0"}, + "UtilizationsByTime": [], + } + ce.get_cost_and_usage.return_value = { + "ResultsByTime": [ + { + "TimePeriod": {"Start": "2026-08-01", "End": "2026-09-01"}, + "Groups": [ + {"Keys": ["Amazon Elastic Compute Cloud - Compute"], + "Metrics": {"UnblendedCost": {"Amount": "9000.0"}}}, + {"Keys": ["Amazon Simple Storage Service"], + "Metrics": {"UnblendedCost": {"Amount": "1500.0"}}}, + ], + } + ] + } + for name, value in overrides.items(): + getattr(ce, name).return_value = value + return ce + + +def _coh_stub(enrolled=True, recommendations=()): + coh = MagicMock() + coh.list_enrollment_statuses.return_value = { + "items": [{"status": "Active" if enrolled else "Inactive", + "accountId": "123456789012"}], + "includeMemberAccounts": True, + } + paginator = MagicMock() + paginator.paginate.return_value = [{"items": list(recommendations)}] + coh.get_paginator.return_value = paginator + return coh + + +# --------------------------------------------------------------------------- +# Handler discipline — keep logic where the analysis tests can see it +# --------------------------------------------------------------------------- + + +class TestHandlerDiscipline: + """The handler must stay an event adapter over `commitments.collect`. + + Its job is to turn a JSON event into validated parameters and to build AWS + clients. Analysis logic that migrates up into it leaves the coverage in + test_commitments_{api,analyze,collect}.py behind, which is how the + `commitment_unit` drift below got in. + """ + + def test_handler_never_calls_the_profile_based_client_builder(self): + """api.build_clients opens a boto3 Session with a named profile, which + does not exist in Lambda. The handler must build Clients itself.""" + assert "build_clients(" not in _HANDLER_PATH.read_text(encoding="utf-8") + + def test_handler_drives_the_shared_pipeline_instead_of_its_own(self): + """The sweep must not be re-implemented alongside the shared pipeline. + + It was, once: two thread-pool fan-outs with the same shape drifted on + `commitment_unit` before anyone noticed, because nothing compared them. + Anything the handler orchestrates itself is orchestration that + test_commitments_collect.py cannot reach. + """ + source = _HANDLER_PATH.read_text(encoding="utf-8") + assert "ThreadPoolExecutor" not in source + for private in ("_run_jobs", "_sweep_savings_plans", "_sweep_reservations", + "_collect_posture", "_findings_from", "_serialize_finding", + "_envelope(", "_meta("): + assert private not in source, ( + f"{private} belongs in commitments/collect.py, where the skill " + "tests and the drift guard can see it" + ) + + def test_handler_does_not_restate_the_shared_defaults(self): + """A local copy of TERMS/DEFAULT_* could accept what the pipeline rejects.""" + assert handler.TERMS is handler.collect.TERMS + assert handler.PAYMENTS is handler.collect.PAYMENTS + assert handler.LOOKBACKS is handler.collect.LOOKBACKS + assert handler.ACCOUNT_SCOPES is handler.collect.ACCOUNT_SCOPES + assert handler.FAMILIES is handler.collect.FAMILIES + assert handler.DEFAULT_TERMS is handler.collect.DEFAULT_TERMS + assert handler.DEFAULT_PAYMENTS is handler.collect.DEFAULT_PAYMENTS + assert handler.DEFAULT_LOOKBACK is handler.collect.DEFAULT_LOOKBACK + assert handler.DEFAULT_POSTURE_DAYS is handler.collect.DEFAULT_POSTURE_DAYS + assert handler.DEFAULT_SPEND_DAYS is handler.collect.DEFAULT_SPEND_DAYS + + +# --------------------------------------------------------------------------- +# Dispatcher +# --------------------------------------------------------------------------- + + +class TestDispatcher: + def test_unknown_tool_returns_error_with_tool_list(self): + result = handler.handler({}, _make_context("nonexistent_tool")) + assert result["error"].startswith("Unknown tool: nonexistent_tool") + assert result["available_tools"] == [ + "generate_commitment_analysis", + "size_savings_plans", + "size_reservations", + "get_commitment_posture", + "get_commitment_expiry", + ] + + def test_known_tool_routes_correctly(self, monkeypatch): + called = {} + + def fake(event): + called["event"] = event + return {"ok": True} + + monkeypatch.setattr(handler, "handle_get_commitment_posture", fake) + result = handler.handler({"posture_days": 7}, _make_context("get_commitment_posture")) + assert called["event"] == {"posture_days": 7} + assert result == {"ok": True} + + def test_routing_strips_target_prefix(self, monkeypatch): + """bedrockAgentCoreToolName is `target___tool` — dispatcher splits on ___.""" + monkeypatch.setattr( + handler, "handle_size_savings_plans", lambda event: {"routed": True} + ) + ctx = SimpleNamespace( + client_context=SimpleNamespace( + custom={"bedrockAgentCoreToolName": "cop-rt-commitments___size_savings_plans"} + ) + ) + assert handler.handler({}, ctx) == {"routed": True} + + +# --------------------------------------------------------------------------- +# Client construction +# --------------------------------------------------------------------------- + + +class TestGetClients: + def test_builds_clients_from_cross_account_with_coh_role_alias(self, monkeypatch): + calls = [] + + def fake_get_aws_client(service, region_name=None, role_alias=None, **kw): + calls.append((service, region_name, role_alias)) + client = MagicMock() + if service == "sts": + client.get_caller_identity.return_value = {"Account": "999888777666"} + return client + + monkeypatch.setattr(handler, "get_aws_client", fake_get_aws_client) + + clients = handler._get_clients() + assert clients.account_id == "999888777666" + assert clients.profile is None + # COH must go through its own role alias, matching cost-optimization-hub. + assert ("cost-optimization-hub", handler.api.COH_REGION, "COH") in calls + assert ("ce", handler.api.CE_REGION, None) in calls + + def test_clients_are_cached_across_invocations(self, monkeypatch): + monkeypatch.setattr( + handler, "get_aws_client", lambda *a, **k: MagicMock() + ) + monkeypatch.setattr(handler, "_account_id", lambda: "111122223333") + first = handler._get_clients() + second = handler._get_clients() + assert first is second + + def test_account_id_failure_is_not_fatal(self, monkeypatch): + def boom(service, **kw): + raise RuntimeError("STS unavailable") + + monkeypatch.setattr(handler, "get_aws_client", boom) + assert handler._account_id() == "unknown" + + +# --------------------------------------------------------------------------- +# Parameter validation — every value arrives as caller-controlled JSON +# --------------------------------------------------------------------------- + + +class TestParameterParsing: + def test_as_list_accepts_comma_separated_string(self): + assert handler._as_list("ONE_YEAR, THREE_YEARS", ()) == ["ONE_YEAR", "THREE_YEARS"] + + def test_as_list_accepts_json_array(self): + assert handler._as_list(["ONE_YEAR"], ()) == ["ONE_YEAR"] + + def test_as_list_empty_falls_back_to_default(self): + assert handler._as_list(None, ("A", "B")) == ["A", "B"] + assert handler._as_list("", ("A",)) == ["A"] + + def test_as_list_rejects_wrong_type(self): + with pytest.raises(handler.ParamError): + handler._as_list({"term": "ONE_YEAR"}, ()) + + def test_validate_all_rejects_unknown_value(self): + with pytest.raises(handler.ParamError) as exc: + handler._validate_all(["FIVE_YEARS"], handler.TERMS, "term") + assert "FIVE_YEARS" in str(exc.value) + assert "ONE_YEAR" in str(exc.value) + + def test_validate_one_defaults_when_absent(self): + assert handler._validate_one( + None, handler.LOOKBACKS, "lookback", handler.DEFAULT_LOOKBACK + ) == "THIRTY_DAYS" + + def test_validate_one_rejects_unknown_value(self): + with pytest.raises(handler.ParamError): + handler._validate_one("NINETY_DAYS", handler.LOOKBACKS, "lookback", "THIRTY_DAYS") + + @pytest.mark.parametrize("bad", ["abc", 0, -5]) + def test_positive_int_rejects_non_positive_and_non_numeric(self, bad): + with pytest.raises(handler.ParamError): + handler._positive_int(bad, "posture_days", 30) + + def test_positive_int_default_and_coercion(self): + assert handler._positive_int(None, "posture_days", 30) == 30 + assert handler._positive_int("14", "posture_days", 30) == 14 + + def test_resolve_ri_services_all(self): + assert handler._resolve_ri_services(None) == list(handler.api.RI_SERVICES) + assert handler._resolve_ri_services("all") == list(handler.api.RI_SERVICES) + + def test_resolve_ri_services_accepts_short_labels_case_insensitively(self): + assert handler._resolve_ri_services("ec2, RDS") == [ + "Amazon Elastic Compute Cloud - Compute", + "Amazon Relational Database Service", + ] + + def test_resolve_ri_services_accepts_full_api_name(self): + assert handler._resolve_ri_services(["Amazon Redshift"]) == ["Amazon Redshift"] + + def test_resolve_ri_services_strips_parenthetical_label(self): + """"Elasticsearch (legacy)" must also match on the bare word.""" + assert handler._resolve_ri_services("Elasticsearch") == [ + "Amazon Elasticsearch Service" + ] + + def test_resolve_ri_services_deduplicates(self): + assert handler._resolve_ri_services("EC2, ec2") == [ + "Amazon Elastic Compute Cloud - Compute" + ] + + def test_resolve_ri_services_rejects_unknown(self): + with pytest.raises(handler.ParamError) as exc: + handler._resolve_ri_services("Fargate") + assert "Fargate" in str(exc.value) + + def test_resolve_sp_types_uppercases(self): + assert handler._resolve_sp_types("compute_sp") == ["COMPUTE_SP"] + + def test_resolve_sp_types_rejects_unknown(self): + with pytest.raises(handler.ParamError): + handler._resolve_sp_types("LAMBDA_SP") + + def test_common_params_defaults_omit_partial_upfront(self): + params = handler._common_params({}) + assert params["terms"] == ["ONE_YEAR", "THREE_YEARS"] + assert params["payments"] == ["NO_UPFRONT", "ALL_UPFRONT"] + assert params["lookback"] == "THIRTY_DAYS" + assert params["account_scope"] == "PAYER" + + +class TestParamErrorsSurfaceAsToolErrors: + """A ParamError must come back as {"error": ...}, never as a 500.""" + + @pytest.mark.parametrize("tool,event", [ + ("handle_generate_commitment_analysis", {"terms": ["FIVE_YEARS"]}), + ("handle_generate_commitment_analysis", {"families": ["gpu"]}), + ("handle_generate_commitment_analysis", {"posture_days": -1}), + ("handle_size_savings_plans", {"savings_plan_types": "LAMBDA_SP"}), + ("handle_size_savings_plans", {"account_scope": "MEMBER"}), + ("handle_size_reservations", {"ri_services": "Fargate"}), + ("handle_get_commitment_posture", {"spend_days": "many"}), + ]) + def test_invalid_parameter_returns_error(self, tool, event): + result = getattr(handler, tool)(event) + assert "error" in result + assert "report_markdown" not in result + + +# --------------------------------------------------------------------------- +# Parallel collection — one bad permutation must not lose the sweep +# --------------------------------------------------------------------------- + + +class TestRunJobs: + """Exercised through the vendored module the handler actually calls. + + The skill's own suite covers this logic too; these stay because they are the + only check that the copy shipped in the zip behaves, and because a sweep + that silently loses permutations produces a plausible-looking report. + """ + + def test_empty_job_list(self): + assert handler.collect.run_jobs([]) == ([], []) + + def test_raised_exception_becomes_a_warning(self): + def boom(): + raise RuntimeError("ThrottlingException") + + results, errors = handler.collect.run_jobs( + [("ok", lambda: {"v": 1}), ("bad", boom)] + ) + assert results == [{"v": 1}] + assert errors == [{"query": "bad", "error": "ThrottlingException"}] + + def test_api_level_error_dict_becomes_a_warning(self): + """api.get_* returns {"error": ...} for ClientError instead of raising.""" + results, errors = handler.collect.run_jobs([ + ("ok", lambda: {"v": 1}), + ("denied", lambda: {"error": "AccessDeniedException", "error_code": "AccessDenied"}), + ]) + assert results == [{"v": 1}] + assert errors[0]["query"] == "denied" + assert errors[0]["error_code"] == "AccessDenied" + + def test_sweep_builds_one_job_per_permutation(self, monkeypatch): + seen = [] + + def fake_sp(clients, sp_type, term, payment, lookback, scope): + seen.append((sp_type, term, payment)) + return {"sp_type": sp_type, "term": term, "payment": payment} + + monkeypatch.setattr(handler.collect, "get_sp_recommendation", fake_sp) + results, errors = handler.collect.sweep_savings_plans( + MagicMock(), ["COMPUTE_SP", "EC2_INSTANCE_SP"], + ["ONE_YEAR", "THREE_YEARS"], ["NO_UPFRONT"], "THIRTY_DAYS", "PAYER", + ) + assert len(results) == 4 # 2 types x 2 terms x 1 payment + assert errors == [] + assert len(set(seen)) == 4 # late-binding closure bug would collapse these + + +# --------------------------------------------------------------------------- +# Serialization +# --------------------------------------------------------------------------- + + +def _finding(**kw): + base = dict( + family="savings-plan", label="Compute Savings Plan", term="ONE_YEAR", + payment="NO_UPFRONT", api_hourly_commitment=10.0, + safe_hourly_commitment=8.0, api_monthly_savings=1000.0, + safe_monthly_savings=800.0, savings_percentage=22.5, upfront_cost=0.0, + confidence="High", volatility="stable", rationale=["because"], + break_even_months=None, waste_exposure_monthly=1460.0, + ) + base.update(kw) + return handler.collect.Finding(**base) + + +class TestSerializeFinding: + def test_savings_plan_unit_is_dollars_per_hour(self): + out = handler.collect.serialize_finding(_finding()) + assert out["commitment_unit"] == "USD/hour" + assert out["achievable_commitment"] == 8.0 + assert out["aws_recommended_commitment"] == 10.0 + assert out["estimated_monthly_savings"] == 800.0 + assert out["aws_best_case_monthly_savings"] == 1000.0 + assert out["rationale"] == ["because"] + + def test_reservation_unit_is_instance_units(self): + out = handler.collect.serialize_finding( + _finding(family="reservation", label="EC2") + ) + assert out["commitment_unit"] == "units" + + def test_break_even_none_stays_none(self): + f = handler.collect.serialize_finding(_finding()) + assert f["break_even_months"] is None + rounded = handler.collect.serialize_finding(_finding(break_even_months=13.27)) + assert rounded["break_even_months"] == 13.3 + + def test_envelope_totals_both_axes(self): + env = handler.collect.envelope( + [_finding(), _finding(label="EC2 Instance Savings Plan")] + ) + assert env["count"] == 2 + assert env["total_estimated_monthly_savings"] == 1600.0 + assert env["aws_best_case_monthly_savings"] == 2000.0 + + +# --------------------------------------------------------------------------- +# One-shot aggregator — real api/analyze/report against a stub CE client +# --------------------------------------------------------------------------- + + +class TestGenerateCommitmentAnalysis: + def test_returns_rendered_report_and_structured_envelope(self, monkeypatch): + _stub_clients(monkeypatch, ce=_ce_stub(), coh=_coh_stub()) + + result = handler.handle_generate_commitment_analysis( + {"families": ["sp"], "savings_plan_types": "COMPUTE_SP", + "terms": ["ONE_YEAR"], "payment_options": ["NO_UPFRONT"]} + ) + + assert "error" not in result + md = result["report_markdown"] + assert md and isinstance(md, str) + assert "Compute Savings Plan" in md + assert result["data_source"] == "live" + assert result["account_id"] == "123456789012" + assert result["lookback"] == "THIRTY_DAYS" + assert result["account_scope"] == "PAYER" + assert result["count"] == 1 + + rec = result["recommendations"][0] + # Risk adjustment: the $8/hr floor caps the AWS-recommended $10/hr. + assert rec["achievable_commitment"] < rec["aws_recommended_commitment"] + assert rec["estimated_monthly_savings"] < rec["aws_best_case_monthly_savings"] + + def test_underutilized_existing_commitment_becomes_a_blocker(self, monkeypatch): + """62% SP utilization must be surfaced before any purchase advice.""" + _stub_clients(monkeypatch, ce=_ce_stub(), coh=_coh_stub()) + result = handler.handle_generate_commitment_analysis({"families": ["sp"]}) + assert result["blockers"], "under-utilized SPs must block" + assert any("utilized" in b for b in result["blockers"]) + assert result["existing_commitment_posture"]["sp_utilization_pct"] == 62.0 + + def test_coh_is_skipped_when_not_enrolled(self, monkeypatch): + """Querying COH unenrolled only yields an access error, so gate on it.""" + coh = _coh_stub(enrolled=False) + _stub_clients(monkeypatch, ce=_ce_stub(), coh=coh) + result = handler.handle_generate_commitment_analysis({"families": ["sp"]}) + assert result["reconciliation"]["status"] == "unavailable" + coh.get_paginator.assert_not_called() + + def test_reconciles_against_coh_when_enrolled(self, monkeypatch): + coh = _coh_stub(recommendations=[{ + "recommendationId": "r-1", "accountId": "123456789012", + "region": "us-east-1", "currentResourceType": "", + "recommendedResourceType": "ComputeSavingsPlans", + "actionType": "PurchaseSavingsPlans", + "estimatedMonthlySavings": 1000.0, + "estimatedSavingsPercentage": 22.0, + "implementationEffort": "VeryLow", + }]) + _stub_clients(monkeypatch, ce=_ce_stub(), coh=coh) + result = handler.handle_generate_commitment_analysis( + {"families": ["sp"], "savings_plan_types": "COMPUTE_SP", + "terms": ["ONE_YEAR"], "payment_options": ["NO_UPFRONT"]} + ) + recon = result["reconciliation"] + # CE best case ($1000) matches COH exactly. + assert recon["status"] == "reconciled" + assert recon["coh_monthly_savings"] == 1000.0 + assert recon["ce_monthly_savings"] == 1000.0 + + def test_no_opportunity_is_a_real_result_not_an_error(self, monkeypatch): + ce = _ce_stub(get_savings_plans_purchase_recommendation=_sp_response( + hourly="0.0", monthly_savings="0.0", minimum="0.0", average="0.0", + )) + _stub_clients(monkeypatch, ce=ce, coh=_coh_stub()) + result = handler.handle_generate_commitment_analysis({"families": ["sp"]}) + assert "error" not in result + assert result["count"] == 0 + assert result["recommendations"] == [] + assert result["report_markdown"] + + def test_failed_permutation_is_reported_as_a_warning(self, monkeypatch): + ce = _ce_stub() + ce.get_savings_plans_purchase_recommendation.side_effect = RuntimeError( + "ThrottlingException" + ) + _stub_clients(monkeypatch, ce=ce, coh=_coh_stub()) + result = handler.handle_generate_commitment_analysis( + {"families": ["sp"], "savings_plan_types": "COMPUTE_SP", + "terms": ["ONE_YEAR"], "payment_options": ["NO_UPFRONT"]} + ) + assert "error" not in result + assert result["collection_warnings"] + assert result["queries_run"] == 1 + + def test_families_filter_skips_the_other_sweep(self, monkeypatch): + ce = _ce_stub() + _stub_clients(monkeypatch, ce=ce, coh=_coh_stub()) + handler.handle_generate_commitment_analysis({"families": ["sp"]}) + ce.get_reservation_purchase_recommendation.assert_not_called() + + def test_unexpected_failure_returns_access_denied_hint(self, monkeypatch): + def boom(): + raise RuntimeError("AccessDeniedException: ce:GetSavingsPlansPurchaseRecommendation") + + monkeypatch.setattr(handler, "_get_clients", boom) + result = handler.handle_generate_commitment_analysis({}) + assert "AccessDenied" in result["error"] + assert "ce:GetSavingsPlansPurchaseRecommendation" in result["hint"] + + def test_data_unavailable_gets_its_own_hint(self, monkeypatch): + def boom(): + raise RuntimeError("DataUnavailableException") + + monkeypatch.setattr(handler, "_get_clients", boom) + result = handler.handle_generate_commitment_analysis({}) + assert "shorter lookback" in result["hint"] + + +# --------------------------------------------------------------------------- +# Narrow sizing tools +# --------------------------------------------------------------------------- + + +class TestSizingTools: + def test_size_savings_plans_envelope(self, monkeypatch): + _stub_clients(monkeypatch, ce=_ce_stub(), coh=_coh_stub()) + result = handler.handle_size_savings_plans( + {"savings_plan_types": "COMPUTE_SP", "terms": ["ONE_YEAR"], + "payment_options": ["NO_UPFRONT"]} + ) + assert result["savings_plan_types"] == ["COMPUTE_SP"] + assert result["count"] == 1 + assert result["data_source"] == "live" + # The narrow tool has no posture gate, so it must say so. + assert "get_commitment_posture" in result["note"] + + def test_size_reservations_reports_short_service_labels(self, monkeypatch): + _stub_clients(monkeypatch, ce=_ce_stub(), coh=_coh_stub()) + result = handler.handle_size_reservations( + {"ri_services": "EC2, RDS", "terms": ["ONE_YEAR"], + "payment_options": ["NO_UPFRONT"]} + ) + assert result["services"] == ["EC2", "RDS"] + assert result["count"] == 0 # stub returns no RI recommendations + + def test_size_reservations_does_not_query_savings_plans(self, monkeypatch): + ce = _ce_stub() + _stub_clients(monkeypatch, ce=ce, coh=_coh_stub()) + handler.handle_size_reservations({"ri_services": "EC2"}) + ce.get_savings_plans_purchase_recommendation.assert_not_called() + + +class TestGetCommitmentPosture: + def test_returns_posture_blockers_and_top_services(self, monkeypatch): + _stub_clients(monkeypatch, ce=_ce_stub(), coh=_coh_stub()) + result = handler.handle_get_commitment_posture({"posture_days": 30}) + + assert result["window_days"] == 30 + assert result["safe_to_buy_more"] is False # 62% utilization blocks + assert result["blockers"] + assert result["cost_optimization_hub"]["enrolled"] is True + assert result["spend_periods"][0]["total"] == 10500.0 + top = result["top_services_latest_period"] + assert list(top)[0] == "Amazon Elastic Compute Cloud - Compute" + # Commitments do not apply to S3 — the note must say why a big bill can + # still yield no recommendation. + assert "cannot be committed against" in result["spend_note"] + + def test_healthy_posture_allows_buying(self, monkeypatch): + ce = _ce_stub(get_savings_plans_utilization={ + "Total": {"Utilization": {"UtilizationPercentage": "99.5", + "UnusedCommitment": "1.00"}} + }) + _stub_clients(monkeypatch, ce=ce, coh=_coh_stub()) + result = handler.handle_get_commitment_posture({}) + assert result["blockers"] == [] + assert result["safe_to_buy_more"] is True + + def test_no_spend_data_yields_empty_top_services(self, monkeypatch): + ce = _ce_stub(get_cost_and_usage={"ResultsByTime": []}) + _stub_clients(monkeypatch, ce=ce, coh=_coh_stub()) + result = handler.handle_get_commitment_posture({}) + assert result["spend_periods"] == [] + assert result["top_services_latest_period"] == {} + + +# --------------------------------------------------------------------------- +# Deployment wiring — tools.json / hierarchy.json / report template +# --------------------------------------------------------------------------- + + +class TestDeploymentWiring: + """The tool is only reachable if all four registration points agree.""" + + @staticmethod + def _json(path: Path): + import json + + return json.loads(path.read_text(encoding="utf-8")) + + def test_tools_json_declares_every_dispatched_tool(self): + cfg = self._json(_REPO_ROOT / "src" / "lambda" / "mcp" / "tools.json") + assert "commitments" in cfg + entry = cfg["commitments"] + declared = {t["name"] for t in entry["tools"]} + # Read the dispatcher out of the source rather than restating it: a tool + # the handler answers but tools.json omits is undeployable, and one + # tools.json advertises but the handler drops is a broken promise to the + # agent. Hardcoding the list here only catches the first the day someone + # remembers to edit it. + dispatched = set( + re.findall( + r'^\s+"(\w+)": handle_\w+,', + _HANDLER_PATH.read_text(encoding="utf-8"), + re.MULTILINE, + ) + ) + assert dispatched, "dispatcher table not found in handler.py" + assert declared == dispatched + assert entry["handler"] == "handler.handler" + + def test_tools_json_grants_the_documented_iam_actions(self): + cfg = self._json(_REPO_ROOT / "src" / "lambda" / "mcp" / "tools.json") + actions = set(cfg["commitments"]["iam_actions"]) + # The purchase-recommendation APIs are the whole point of the tool and + # exist nowhere else in the deployment. + assert "ce:GetSavingsPlansPurchaseRecommendation" in actions + assert "ce:GetReservationPurchaseRecommendation" in actions + # Every action named in the handler docstring must actually be granted. + documented = { + line.strip("- ").strip() + for line in _HANDLER_PATH.read_text(encoding="utf-8").splitlines() + if line.startswith("- ") and (":Get" in line or ":List" in line) + } + assert documented <= actions + + def test_agent_is_wired_to_the_tool(self): + hierarchy = self._json(_REPO_ROOT / "src" / "agents" / "hierarchy.json") + agent = hierarchy["cost-operations-agent"] + assert "commitments" in agent["tools"] + assert "generate_commitment_analysis" in agent["prompt"] + + def test_report_template_is_bundled_for_both_consumers(self): + """The agent loads templates from src/agents/shared; the frontend gets + its list from the core-api Lambda's own bundled copy. Both must have it.""" + agent_copy = (_REPO_ROOT / "src" / "agents" / "shared" / + "report_templates" / "discounted_commitments.json") + api_copy = (_REPO_ROOT / "src" / "lambda" / "frontend" / "core-api" / + "report_templates" / "discounted_commitments.json") + assert agent_copy.read_bytes() == api_copy.read_bytes() + template = self._json(agent_copy) + assert len(template["sections"]) == 1 + prompt = template["sections"][0]["prompt"] + assert "generate_commitment_analysis" in prompt + assert "report_markdown" in prompt From 46bc99d032145f5e902f77626df721ab460d06fe Mon Sep 17 00:00:00 2001 From: raytoo Date: Wed, 16 Sep 2026 11:37:16 +0800 Subject: [PATCH 2/3] test(commitments): contract-test AWS field names, cover gateway target paging MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes two of the three TODOs left on the discounted-commitments PR by replacing "verify by hand against a live account" with checks that run offline in CI. Field-name contract tests (tests/unit/test_commitments_contracts.py) SpecShape and InventorySpec are lookup tables of AWS response field names, and every read goes through .get(field). That returns None for a typo exactly as it does for a field AWS genuinely omitted, so InstanceTpye does not raise — it produces a recommendation with no instance type and the report renders around the hole. Fixtures cannot catch this: a fixture written from the same table as the code agrees with the typo and stays green. botocore ships the service models the SDK dispatches on, so the real names are already on disk. These 57 tests assert every name in RECOMMENDATION_SPECS and RESERVATION_INVENTORY is a member of the matching output shape — containers, size fields, attribute fields, Family, Region, the six differently-named identifier/count/type fields, and the Duration fallback for the five families returning no explicit end date. Three further assertions pin types a rename would quietly break: MultiAZ is still boolean (which is why it is tested against None, not truthiness — False means Single-AZ), commitment is still a string (AWS returns "1.00000000", so the float() conversion is load-bearing), and ACTIVE_SP_STATES are valid SavingsPlanState enum values (they go to the API as a server-side filter, so an invalid one is a ValidationException in a Lambda rather than a red test here). Two facts are now asserted rather than assumed. DynamoDB capacity models no SizeFlexEligible/CurrentGeneration — there is no instance, so there is no size to flex; reading both unconditionally yields False, which is correct, and that is pinned so nobody "fixes" the absence by inventing a field name. And test_every_recommendation_container_is_covered compares the modelled *InstanceDetails/*CapacityDetails containers against the ones we know, because describe_recommendation_spec degrades to {} for an unknown shape — a service AWS adds later would otherwise cost the report its spec column silently. Verified by mutation: typoing ESInstanceDetails size_fields, memorydb id_field and the RDS MultiAZ attribute each turns exactly the expected test red. Gateway target paging (tests/scripts/test_sync_gateway_targets.sh) Covers the sync.sh fix in the parent commit. boto3 does not follow nextToken and the AWS CLI does, which is what made the bug invisible — the CLI showed 11 targets while the same boto3 call returned 10. A dropped target is indistinguishable from one that was never created, so the sync reported "target not found" for a target that was present; it shipped once and left the lambda-runtime target with zero tool schemas. 25 assertions against a boto3 stub serving scripted pages: a target on the last page is found and updated, paging arguments are correct (no token first, token forwarded after, maxResults=100 throughout), a single page makes no second call, a genuinely absent target still fails loudly and clears the hash file so the next deploy retries, schema-less tools.json entries are skipped, and an unchanged tools.json makes no API calls at all. Verified by reverting sync.sh to the pre-fix single call: 13 of the 25 go red, including the exact "target not found" symptom. Docs: new test file and count (274 -> 331) in the commitments testing section, plus a contract-test subsection on what these do and do not prove; the list_gateway_targets paging trap added to the development.md gotchas beside its tools/list sibling. Not closed: the third TODO still needs an account with OpenSearch or DynamoDB steady-state usage. The dev account holds no reservations and no recommendations, so populated-field verification for those two shapes remains outstanding. The contract tests narrow it to "AWS omitted it" rather than "we spelled it wrong". --- docs/development.md | 1 + docs/skills/discounted-commitments.md | 58 +++- tests/scripts/test_sync_gateway_targets.sh | 319 ++++++++++++++++++ tests/unit/test_commitments_contracts.py | 357 +++++++++++++++++++++ 4 files changed, 733 insertions(+), 2 deletions(-) create mode 100755 tests/scripts/test_sync_gateway_targets.sh create mode 100644 tests/unit/test_commitments_contracts.py diff --git a/docs/development.md b/docs/development.md index 8a5c3f6..607d194 100644 --- a/docs/development.md +++ b/docs/development.md @@ -75,6 +75,7 @@ These are load-bearing and not obvious from reading the code: - **MCP import name collision**: use `streamablehttp_client` (no underscores) from `mcp.client.streamable_http` for SigV4 `auth=` support. The similarly-named `streamable_http_client` silently drops `auth=`, tools fail to load, and the model hallucinates fake `` XML with fabricated data. - **AgentCore Gateway paginates `tools/list` at 30 per page.** `MCPClient.list_tools_sync()` returns a `PaginatedList` — `len()` is the current page only, and the cursor is exposed as `.pagination_token`. Any target whose tools land on page 2+ is invisible unless you drain the cursor. `agent_base.py::load_gateway_tools` drains pages in a loop; do NOT revert that to a single call. Symptom when skipped: filter reports `Filtered 0/30 gateway tools` even though the gateway has the target READY with correct inline schemas, model has zero tools, platform no-fabrication preamble fires "no tools available" or model fabricates plausible numbers. +- **`list_gateway_targets` paginates too, and boto3 does not follow `nextToken`.** Same class of bug as the entry above, one API layer down, and harder to spot because the AWS CLI *does* auto-paginate: `aws bedrock-agentcore-control list-gateway-targets` showed all 11 targets while the identical boto3 call returned 10 plus a `nextToken` nobody read. `sync_gateway_tools` in `scripts/lib/sync.sh` drains the token in a loop; do NOT revert that to a single call. Symptom when skipped: `: target not found in gateway, skipping` → `[WARN] Gateway tool schema sync failed (non-fatal)` for a target that is present and READY — because a dropped target is indistinguishable from one that was never created. This shipped once: it left the `lambda-runtime` target with zero tool schemas, making all 8 of its tools unreachable. Guarded by `tests/scripts/test_sync_gateway_targets.sh`, which stubs boto3 to serve scripted pages and asserts a target on the last page is still found (and that a genuinely absent one still fails loudly). - **Hallucination guardrail is platform-level, not per-prompt.** `agent_base.py` prepends a non-negotiable `_NO_FABRICATION_PREAMBLE` to every agent's system prompt AND refuses to invoke a leaf with zero tools (returns a clear error instead of letting the model improvise). Agent authors do not opt in; the factories apply this unconditionally. If you add a new agent factory, call `_apply_platform_preamble(_inject_tool_inventory(prompt, tools))` — do not bypass. - **Bedrock Guardrail (standalone ApplyGuardrail API).** NOT attached to the model (which caused false positives on system prompts). Instead, `shared/guardrail.py::check_user_input()` calls `bedrock-runtime:ApplyGuardrail` on ONLY the raw user message at the supervisor entry point (in `agui_server.py`, BEFORE the agent is built/run). Prompt attack detection + sensitive info filters + topic policy. System prompts never reach the classifier (they're assembled later in `agent_base.py` and aren't user-modifiable). Env vars: `BEDROCK_GUARDRAIL_ID`, `BEDROCK_GUARDRAIL_VERSION`, `GUARDRAIL_MODE` (supervisor-only). `GUARDRAIL_MODE` is `block` (default — refuse flagged input) or `detect` (log-only, non-blocking; set via root tfvar `guardrail_mode`). Layered with: (a) `_NO_FABRICATION_PREAMBLE` behavioral constraints, (b) IAM least-privilege per-tool (hard backstop), (c) `shared/redact.py` strips genuine secrets (access keys, external IDs, role-session names) from persisted data (memory + reports); account IDs and ARNs are identifiers kept by default, scrubbed only when `REDACT_IDENTIFIERS=true`. **The CMK on DynamoDB requires every agent role to have `kms:Decrypt` on the platform key — without it, registry reads fail closed and the agent reports "no child agents deployed".** - **MCPClient lifecycle in leaf agents**: manual `__enter__()` → `list_tools_sync()` → pass tools (not the client) to `Agent(tools=tools)` → `__exit__` in `finally`. Passing `MCPClient` directly to `Agent` causes `"client failed to initialize"` in Runtime containers. diff --git a/docs/skills/discounted-commitments.md b/docs/skills/discounted-commitments.md index 5f423ea..1976624 100644 --- a/docs/skills/discounted-commitments.md +++ b/docs/skills/discounted-commitments.md @@ -86,7 +86,7 @@ interchangeable. | Who executes | The host agent, via Bash | The Lambda, called through the gateway | | Arithmetic by | The agent, following `reference/method.md` | `commitments/analyze.py` | | Portable | Yes — copy the directory anywhere | No — platform-coupled | -| Tests | None (no code to test) | 274 tests, `tests/unit/test_commitments_*.py` | +| Tests | None (no code to test) | 331 tests, `tests/unit/test_commitments_*.py` | The skill is the portable expression: it must run in any coding agent with a shell, with nothing installed, so it drives the AWS CLI and states @@ -926,7 +926,7 @@ registration. ```bash .venv/bin/python -m pytest tests/unit/test_commitments_*.py -q -# 274 passed +# 331 passed ``` | File | Covers | @@ -937,6 +937,7 @@ registration. | `test_commitments_expiry.py` | Per-family field mapping and derived end dates, date coercion across SDK/CLI shapes, expiry buckets and renewal verdicts at their boundaries, unit separation, region validation, expiry rendering, and that the Savings Plans job is **not** multiplied per region | | `test_commitments_spec.py` | The per-service spec shapes (all seven, including the OpenSearch two-field type and the DynamoDB container), `MultiAZ` as a boolean, per-line breakdown and ranking, whole-unit rounding and its disclosure, the report's line-items and expiry `Spec` columns, and `line_items` in the envelope | | `test_commitments_tool.py` | Handler parameter validation, error normalization, `TestHandlerDiscipline`, and `tools.json` wiring | +| `test_commitments_contracts.py` | Every AWS field name in `RECOMMENDATION_SPECS` and `RESERVATION_INVENTORY` checked against botocore's shipped service models — see below | The band tests pin the risk-adjustment thresholds *at* their boundaries (0.80, 0.50, 95%, 90%, and the 30/60/90-day expiry cutoffs), which is what @@ -957,6 +958,59 @@ exception, and writing them surfaced a real defect — `str(None)` rendering as literal `"None"`, which would have printed a fabricated specification into a customer-facing report. +### Contract tests: field names, checked against botocore + +`test_commitments_spec.py` proves the *logic* using fixtures, but a fixture +written from the same table as the code cannot catch a wrong field name — it +agrees with the typo and stays green. Every read goes through `.get(field)`, +which returns `None` for a misspelling exactly as it does for a field AWS +genuinely omitted, so `InstanceTpye` does not raise: it produces a +recommendation with no instance type, and the report renders around the hole. + +`test_commitments_contracts.py` closes that gap without needing an account. +botocore ships the same service models the SDK dispatches on, so the real field +names are already on disk: + +```python +model = session.get_service_model("ce").operation_model( + "GetReservationPurchaseRecommendation" +) +``` + +Every name in `RECOMMENDATION_SPECS` and `RESERVATION_INVENTORY` is asserted to +be a member of the matching output shape — containers, size fields, attribute +fields, `Family`, `Region`, the six differently-named identifier/count/type +fields, and the `Duration` fallback for the five families that return no explicit +end date. Three further assertions pin things a rename would quietly break: + +- **`MultiAZ` is still `boolean`.** This is why the code tests it against `None` + rather than truthiness — `False` means Single-AZ, a real and expensive + specification. +- **`commitment` is still a `string`.** AWS returns `"1.00000000"`, so the + `float()` conversion is load-bearing; summing without it concatenates. +- **`ACTIVE_SP_STATES` are valid `SavingsPlanState` enum values.** These go to + the API as a server-side filter, so an invalid one is a ValidationException in + a Lambda against a live account rather than a red test here. + +Two coverage facts worth knowing, both asserted rather than assumed: + +- **DynamoDB has no `SizeFlexEligible`/`CurrentGeneration`.** There is no + instance, so there is no size to flex. `describe_recommendation_spec` reading + both unconditionally yields `False`, which is the correct answer, not a data + gap — pinned so nobody "fixes" the absence by inventing a field name. +- **`test_every_recommendation_container_is_covered`** compares the modelled + `*InstanceDetails`/`*CapacityDetails` containers against the ones + `RECOMMENDATION_SPECS` knows. `describe_recommendation_spec` degrades to `{}` + for an unknown shape, so a service AWS adds later would cost the report its + spec column silently; this turns that into a failing test instead. + +What these tests do **not** do is prove a field is populated for a given +account. That needs an account holding the commitment or recommendation in +question — specifically one with OpenSearch or DynamoDB steady-state usage for +those two shapes, which the dev account has none of. What they guarantee is that +when such an account is used, a blank column means "AWS omitted it", never "we +spelled it wrong". + `test_tools_json_declares_every_dispatched_tool` reads the dispatcher table out of `handler.py` rather than restating it, so a tool added to one and not the other fails the build in both directions. The skill's markdown copy of the diff --git a/tests/scripts/test_sync_gateway_targets.sh b/tests/scripts/test_sync_gateway_targets.sh new file mode 100755 index 0000000..7aba77e --- /dev/null +++ b/tests/scripts/test_sync_gateway_targets.sh @@ -0,0 +1,319 @@ +#!/usr/bin/env bash +# Verify sync_gateway_tools pages through list_gateway_targets. +# +# The bug this guards against: the AWS API returns a bounded page of gateway +# targets and boto3 does NOT follow nextToken on its own (the AWS CLI does, +# which is what made this invisible — `aws ... list-gateway-targets` showed all +# 11 targets while the same call through boto3 returned 10). With a single +# call, any target past the first page is missing from `existing`, and a missing +# target is indistinguishable from one that was never created: the sync prints +# "target not found in gateway, skipping" and fails. That is exactly how the +# lambda-runtime target ended up deployed with zero tool schemas, making its 8 +# tools unreachable through the gateway. +# +# boto3 is replaced by a stub that serves a scripted list of pages and records +# every call, so the paging behaviour is asserted without touching AWS. +# +# Run from project root: +# bash tests/scripts/test_sync_gateway_targets.sh + +set -u + +TESTS_RUN=0 +TESTS_FAILED=0 + +assert_eq() { + local actual="$1" expected="$2" name="$3" + TESTS_RUN=$((TESTS_RUN + 1)) + if [ "$actual" = "$expected" ]; then + printf " PASS %s\n" "$name" + else + TESTS_FAILED=$((TESTS_FAILED + 1)) + printf " FAIL %s\n expected: %q\n actual: %q\n" \ + "$name" "$expected" "$actual" + fi +} + +assert_contains() { + local haystack="$1" needle="$2" name="$3" + TESTS_RUN=$((TESTS_RUN + 1)) + if printf '%s' "$haystack" | grep -qF -- "$needle"; then + printf " PASS %s\n" "$name" + else + TESTS_FAILED=$((TESTS_FAILED + 1)) + printf " FAIL %s\n expected to contain: %q\n actual: %q\n" \ + "$name" "$needle" "$haystack" + fi +} + +assert_not_contains() { + local haystack="$1" needle="$2" name="$3" + TESTS_RUN=$((TESTS_RUN + 1)) + if printf '%s' "$haystack" | grep -qF -- "$needle"; then + TESTS_FAILED=$((TESTS_FAILED + 1)) + printf " FAIL %s\n expected NOT to contain: %q\n actual: %q\n" \ + "$name" "$needle" "$haystack" + else + printf " PASS %s\n" "$name" + fi +} + +PROJECT_ROOT="$(cd "$(dirname "$0")/../.." && pwd)" +cd "$PROJECT_ROOT" + +SANDBOX=$(mktemp -d) +trap 'rm -rf "$SANDBOX"' EXIT + +REPO="$SANDBOX/repo" +mkdir -p "$REPO/scripts/lib" "$REPO/.venv/bin" "$REPO/src/lambda/mcp" \ + "$REPO/.lambda-hashes" "$REPO/fakelib" +cp "$PROJECT_ROOT/scripts/lib"/{common,sync}.sh "$REPO/scripts/lib/" +ln -sf "$PROJECT_ROOT/.venv/bin/python" "$REPO/.venv/bin/python" + +# --- boto3 stub ------------------------------------------------------------- +# Pages come from $FAKE_TARGET_PAGES (JSON list of lists of target names); every +# call is appended to $FAKE_CALL_LOG as one JSON object per line. +cat > "$REPO/fakelib/boto3.py" <<'PYEOF' +"""boto3 stand-in for sync_gateway_tools tests. + +Only the three AgentCore control-plane calls sync_gateway_tools makes are +implemented. list_gateway_targets serves the pages named by +$FAKE_TARGET_PAGES and issues a nextToken for every page but the last, so a +caller that ignores the token sees only the first page — the exact failure the +real bug had. +""" + +import json +import os + + +def _log(call, payload): + with open(os.environ["FAKE_CALL_LOG"], "a") as handle: + handle.write(json.dumps({"call": call, **payload}) + "\n") + + +class _Client: + def __init__(self, service, region): + self.service = service + self.region = region + _log("client", {"service": service, "region": region}) + + def list_gateway_targets(self, **kwargs): + _log("list", { + "nextToken": kwargs.get("nextToken"), + "maxResults": kwargs.get("maxResults"), + "gatewayIdentifier": kwargs.get("gatewayIdentifier"), + }) + with open(os.environ["FAKE_TARGET_PAGES"]) as handle: + pages = json.load(handle) + token = kwargs.get("nextToken") + index = 0 if token is None else int(token.split(":")[1]) + response = { + "items": [ + {"name": name, "targetId": "tid-" + name} for name in pages[index] + ] + } + if index + 1 < len(pages): + response["nextToken"] = "page:%d" % (index + 1) + return response + + def get_gateway_target(self, **kwargs): + _log("get", {"targetId": kwargs.get("targetId")}) + return { + "targetConfiguration": { + "mcp": {"lambda": {"lambdaArn": "arn:aws:lambda:x:1:function:f"}} + } + } + + def update_gateway_target(self, **kwargs): + schemas = kwargs["targetConfiguration"]["mcp"]["lambda"]["toolSchema"] + _log("update", { + "name": kwargs.get("name"), + "targetId": kwargs.get("targetId"), + "tool_count": len(schemas["inlinePayload"]), + "tool_names": [t["name"] for t in schemas["inlinePayload"]], + }) + return {} + + +def client(service, region_name=None, **_kwargs): + return _Client(service, region_name) +PYEOF + +# --- tools.json ------------------------------------------------------------- +# Two tools on the target under test; a second entry with no `tools` array to +# confirm schema-less entries are skipped rather than reported as missing. +cat > "$REPO/src/lambda/mcp/tools.json" <<'EOF' +{ + "commitments": { + "tools": [ + { + "name": "get_commitment_recommendations", + "description": "Size SP/RI purchases", + "input_schema": { + "type": "object", + "properties": {"lookback_days": {"type": "integer"}} + } + }, + { + "name": "get_commitment_expiry", + "description": "Upcoming expiries", + "input_schema": {"type": "object", "properties": {}} + } + ] + }, + "no-schemas": {} +} +EOF + +cd "$REPO" +export AWS_REGION=ap-northeast-1 +export HASH_DIR=".lambda-hashes" +export PYTHONPATH="$REPO/fakelib" +export FAKE_CALL_LOG="$SANDBOX/calls.jsonl" +export FAKE_TARGET_PAGES="$SANDBOX/pages.json" + +# shellcheck disable=SC1091 +source scripts/lib/common.sh +# shellcheck disable=SC1091 +source scripts/lib/sync.sh + +# Stub the Terraform lookup — no state, no terraform binary needed. +tf_output() { echo "GW-TEST-123"; } + +# Count list calls / read the nth logged call of a kind. +# +# `grep -c` prints 0 AND exits 1 when there is no match, so the usual +# `|| echo 0` fallback emits a second zero and every count comparison against +# "0" fails on a two-line value. Capture first, then default. +count_calls() { + local count + count=$(grep -c "\"call\": \"$1\"" "$FAKE_CALL_LOG" 2>/dev/null) || count=0 + echo "$count" +} +nth_call() { + grep "\"call\": \"$1\"" "$FAKE_CALL_LOG" 2>/dev/null | sed -n "${2}p" +} + +reset_run() { + : > "$FAKE_CALL_LOG" + rm -f "$HASH_DIR/gateway-tools.sha" +} + +# --------------------------------------------------------------------------- +# Test 1: target on the LAST page is still found and updated +# +# This is the regression. Pre-fix, only page 1 was read, so `commitments` +# (page 3) was absent from `existing` and the sync failed. +# --------------------------------------------------------------------------- +echo "Test 1: target on final page is found" +cat > "$FAKE_TARGET_PAGES" <<'EOF' +[["cost-explorer", "health"], ["network-resilience", "lambda-runtime"], ["commitments"]] +EOF +reset_run +OUTPUT=$(sync_gateway_tools 2>&1) +STATUS=$? + +assert_eq "$STATUS" "0" "sync_gateway_tools succeeds" +assert_contains "$OUTPUT" "commitments: updated with 2 tool schemas" "target updated with both schemas" +assert_not_contains "$OUTPUT" "target not found" "no spurious not-found" +assert_not_contains "$OUTPUT" "sync failed" "no failure warning" +assert_eq "$(count_calls list)" "3" "all 3 pages were requested" +assert_eq "$(count_calls update)" "1" "exactly one target updated" + +# --------------------------------------------------------------------------- +# Test 2: paging arguments are correct +# --------------------------------------------------------------------------- +echo +echo "Test 2: paging arguments" +FIRST=$(nth_call list 1) +SECOND=$(nth_call list 2) +THIRD=$(nth_call list 3) + +assert_contains "$FIRST" '"nextToken": null' "first call sends no token" +assert_contains "$SECOND" '"nextToken": "page:1"' "second call forwards page-1 token" +assert_contains "$THIRD" '"nextToken": "page:2"' "third call forwards page-2 token" +assert_contains "$FIRST" '"maxResults": 100' "maxResults asks for a full page" +assert_contains "$SECOND" '"maxResults": 100' "maxResults persists across pages" +assert_contains "$FIRST" '"gatewayIdentifier": "GW-TEST-123"' "gateway id forwarded" + +# --------------------------------------------------------------------------- +# Test 3: a single unpaginated page still works (no token → no second call) +# --------------------------------------------------------------------------- +echo +echo "Test 3: single page, no token" +cat > "$FAKE_TARGET_PAGES" <<'EOF' +[["commitments", "cost-explorer"]] +EOF +reset_run +OUTPUT=$(sync_gateway_tools 2>&1) +STATUS=$? + +assert_eq "$STATUS" "0" "single-page sync succeeds" +assert_eq "$(count_calls list)" "1" "no extra call when no token returned" +assert_contains "$OUTPUT" "commitments: updated with 2 tool schemas" "target updated" + +# --------------------------------------------------------------------------- +# Test 4: a genuinely absent target still fails loudly +# +# Paging must not paper over the real not-found case: an undeployed target has +# to keep failing the sync, otherwise a target with zero schemas ships silently. +# --------------------------------------------------------------------------- +echo +echo "Test 4: genuinely missing target fails" +cat > "$FAKE_TARGET_PAGES" <<'EOF' +[["cost-explorer"], ["health"]] +EOF +reset_run +OUTPUT=$(sync_gateway_tools 2>&1) +STATUS=$? + +assert_eq "$STATUS" "0" "sync_gateway_tools returns 0 (failure is non-fatal by design)" +assert_contains "$OUTPUT" "commitments: target not found in gateway, skipping" "missing target reported" +assert_contains "$OUTPUT" "Gateway tool schema sync failed" "non-fatal warning emitted" +assert_eq "$(count_calls list)" "2" "both pages searched before giving up" +assert_eq "$(count_calls update)" "0" "nothing updated" +TESTS_RUN=$((TESTS_RUN + 1)) +if [ ! -f "$HASH_DIR/gateway-tools.sha" ]; then + printf " PASS %s\n" "hash file removed so the next deploy retries" +else + TESTS_FAILED=$((TESTS_FAILED + 1)) + printf " FAIL %s\n" "hash file removed so the next deploy retries" +fi + +# --------------------------------------------------------------------------- +# Test 5: entries without a `tools` array are skipped, not reported missing +# --------------------------------------------------------------------------- +echo +echo "Test 5: schema-less entries skipped" +assert_not_contains "$OUTPUT" "no-schemas" "tools.json entry with no schemas ignored" + +# --------------------------------------------------------------------------- +# Test 6: success writes the hash file so an unchanged tools.json skips +# --------------------------------------------------------------------------- +echo +echo "Test 6: hash-based skip after success" +cat > "$FAKE_TARGET_PAGES" <<'EOF' +[["cost-explorer"], ["commitments"]] +EOF +reset_run +sync_gateway_tools >/dev/null 2>&1 + +TESTS_RUN=$((TESTS_RUN + 1)) +if [ -f "$HASH_DIR/gateway-tools.sha" ]; then + printf " PASS %s\n" "hash file written on success" +else + TESTS_FAILED=$((TESTS_FAILED + 1)) + printf " FAIL %s\n" "hash file written on success" +fi + +: > "$FAKE_CALL_LOG" +OUTPUT=$(sync_gateway_tools 2>&1) +assert_contains "$OUTPUT" "unchanged, skipping sync" "unchanged tools.json skips the sync" +assert_eq "$(count_calls list)" "0" "no API calls on the skipped run" + +echo +echo "----------------------------------------" +echo "Ran $TESTS_RUN tests, $TESTS_FAILED failed." +echo "----------------------------------------" +exit "$TESTS_FAILED" diff --git a/tests/unit/test_commitments_contracts.py b/tests/unit/test_commitments_contracts.py new file mode 100644 index 0000000..0a050fc --- /dev/null +++ b/tests/unit/test_commitments_contracts.py @@ -0,0 +1,357 @@ +"""Contract tests: every AWS field name the commitments tool reads must exist. + +Why these are separate from the rest of the suite +------------------------------------------------- +The other commitments tests feed hand-written fixtures through the collection +and analysis code. That proves the logic, but it cannot catch the one failure +mode this tool is most exposed to: a field name that does not exist. + +`SpecShape` and `InventorySpec` are lookup tables of AWS response field names. +Every read goes through `.get(field)`, which returns `None` for a typo exactly +as it does for a field AWS genuinely omitted. So `InstanceTpye` does not raise — +it produces a recommendation with no instance type, or a commitment with no +identifier, and the report renders around the hole. A fixture written from the +same table as the code under test agrees with the typo and stays green. + +botocore ships the service models the SDK itself dispatches on, so the real +field names are already on disk. These tests read them and assert every name in +our tables is a member of the corresponding output shape. That makes a typo, or +an AWS rename, a red test offline rather than a silent blank column in a live +report. + +Scope and limits +---------------- +This validates *names and shape membership*. It does not validate that a field +is populated for any given account — that needs a live account holding the +commitment or recommendation in question, and specifically an account with +OpenSearch or DynamoDB steady-state usage for those two shapes. What it does +guarantee is that when such an account is finally used, a blank column means +"AWS omitted it", never "we spelled it wrong". +""" + +import botocore.session +import pytest +from commitments.api import ( + ACTIVE_SP_STATES, + RECOMMENDATION_SPECS, + RESERVATION_INVENTORY, +) + + +@pytest.fixture(scope="module") +def botocore_session(): + """One botocore session for the module — loading service models is slow.""" + return botocore.session.get_session() + + +def structure_members(shape): + """Members of a shape, unwrapping list nesting. + + AWS models these as `list[structure]` at almost every level + (`Recommendations`, `RecommendationDetails`, `ReservedInstances`), and only + the member structure carries field names. Returns `{}` for a scalar so a + caller gets an empty membership test rather than an AttributeError. + """ + while shape.type_name == "list": + shape = shape.member + return shape.members if shape.type_name == "structure" else {} + + +def operation_for(service_model, method): + """Map a boto3 snake_case method to its model operation name. + + Comparing `method.replace("_", "")` case-insensitively against the model's + operation names avoids hand-maintaining a title-case map — the naive + `str.title()` transform gets `DescribeReservedDBInstances` wrong, because + boto3's `describe_reserved_db_instances` capitalizes DB but title() does not. + """ + target = method.replace("_", "").lower() + for name in service_model.operation_names: + if name.lower() == target: + return service_model.operation_model(name) + return None + + +# --------------------------------------------------------------------------- +# Cost Explorer: the purchasable spec inside a recommendation +# --------------------------------------------------------------------------- +@pytest.fixture(scope="module") +def recommendation_detail_members(botocore_session): + """Members of one `RecommendationDetails[]` entry from the CE model.""" + model = botocore_session.get_service_model("ce").operation_model( + "GetReservationPurchaseRecommendation" + ) + recommendations = structure_members(model.output_shape)["Recommendations"] + details = structure_members(recommendations)["RecommendationDetails"] + return structure_members(details) + + +@pytest.mark.parametrize("spec", RECOMMENDATION_SPECS, ids=lambda s: s.key) +def test_recommendation_spec_container_exists(spec, recommendation_detail_members): + """The sub-object a spec lives under is a real field on the detail. + + Two containers are in play and the difference is easy to get wrong: + everything instance-shaped hangs off `InstanceDetails`, but DynamoDB + reserved capacity hangs off `ReservedCapacityDetails`. + """ + assert spec.container in recommendation_detail_members, ( + f"{spec.key} is declared under RecommendationDetails.{spec.container}, " + f"which the CE model does not define. Available: " + f"{sorted(recommendation_detail_members)}" + ) + + +@pytest.mark.parametrize("spec", RECOMMENDATION_SPECS, ids=lambda s: s.key) +def test_recommendation_spec_key_exists(spec, recommendation_detail_members): + """The service-specific sub-object itself exists inside its container.""" + container = structure_members(recommendation_detail_members[spec.container]) + assert spec.key in container, ( + f"{spec.container}.{spec.key} is not in the CE model. " + f"Available: {sorted(container)}" + ) + + +@pytest.mark.parametrize("spec", RECOMMENDATION_SPECS, ids=lambda s: s.key) +def test_recommendation_spec_fields_exist(spec, recommendation_detail_members): + """Every field a spec reads is a member of its sub-object. + + This is the assertion that would have caught `InstanceClass`/`InstanceSize` + being wrong for OpenSearch, or `Family` being assumed present on a shape + that has no family at all. + """ + container = structure_members(recommendation_detail_members[spec.container]) + members = structure_members(container[spec.key]) + + expected = [*spec.size_fields, spec.region_field] + expected += [field for field, _label in spec.attribute_fields] + if spec.family_field: + expected.append(spec.family_field) + + missing = [field for field in expected if field not in members] + assert not missing, ( + f"{spec.key} reads {missing}, which the CE model does not define. " + f"Available: {sorted(members)}" + ) + + +@pytest.mark.parametrize("spec", RECOMMENDATION_SPECS, ids=lambda s: s.key) +def test_size_flex_fields_present_only_where_meaningful( + spec, recommendation_detail_members +): + """`SizeFlexEligible`/`CurrentGeneration` exist for instances, not DynamoDB. + + `describe_recommendation_spec` reads both unconditionally via + `bool(raw.get(...))`. For DynamoDB reserved capacity AWS models neither + field — there is no instance, so there is no size to flex and no generation + to be current. `False` is therefore the correct answer, not a data gap, and + this test pins that so nobody "fixes" the absence by inventing a field name. + """ + container = structure_members(recommendation_detail_members[spec.container]) + members = structure_members(container[spec.key]) + flex_fields = ("SizeFlexEligible", "CurrentGeneration") + + if spec.key == "DynamoDBCapacityDetails": + assert all(field not in members for field in flex_fields), ( + "DynamoDB capacity now models size flexibility — " + "describe_recommendation_spec can report it for real instead of False" + ) + else: + missing = [field for field in flex_fields if field not in members] + assert not missing, f"{spec.key} no longer models {missing}" + + +def test_every_recommendation_container_is_covered(recommendation_detail_members): + """No CE detail sub-object goes unread without a deliberate decision. + + AWS adds services to this API over time. `describe_recommendation_spec` + degrades to `{}` for an unknown shape, so a new service costs the report its + spec column silently. Listing the known containers here turns that into a + failing test the next time AWS adds one. + """ + known = {spec.container for spec in RECOMMENDATION_SPECS} + modelled = { + name + for name in recommendation_detail_members + if name.endswith(("InstanceDetails", "CapacityDetails")) + } + assert modelled == known, ( + f"CE models detail containers {sorted(modelled - known)} that " + f"RECOMMENDATION_SPECS does not cover" + ) + + +# --------------------------------------------------------------------------- +# Reservation inventory: the describe APIs behind expiry tracking +# --------------------------------------------------------------------------- +@pytest.fixture(scope="module") +def inventory_members(botocore_session): + """`{spec.key: members of one response row}` for every inventory API.""" + resolved = {} + for spec in RESERVATION_INVENTORY: + service = botocore_session.get_service_model(spec.service) + operation = operation_for(service, spec.method) + assert operation is not None, ( + f"{spec.service} has no operation for {spec.method}" + ) + top = structure_members(operation.output_shape) + assert spec.response_key in top, ( + f"{spec.service}.{spec.method} has no {spec.response_key}; " + f"available: {sorted(top)}" + ) + resolved[spec.key] = structure_members(top[spec.response_key]) + return resolved + + +@pytest.mark.parametrize("spec", RESERVATION_INVENTORY, ids=lambda s: s.key) +def test_inventory_identity_fields_exist(spec, inventory_members): + """id/count/type/payment fields exist — a typo here yields blank rows. + + Six APIs name the same four concepts six ways + (`ReservedInstancesId` vs `ReservedDBInstanceId` vs `ReservationId`), which + is precisely why the table exists and precisely why it needs checking. + """ + members = inventory_members[spec.key] + expected = (spec.id_field, spec.count_field, spec.type_field, spec.payment_field) + missing = [field for field in expected if field not in members] + assert not missing, ( + f"{spec.key} reads {missing}, absent from {spec.service}.{spec.method}. " + f"Available: {sorted(members)}" + ) + + +@pytest.mark.parametrize("spec", RESERVATION_INVENTORY, ids=lambda s: s.key) +def test_inventory_term_fields_exist(spec, inventory_members): + """Start exists, and expiry is readable either directly or via Duration. + + Only EC2 returns an explicit `End`. Everywhere else the end date is derived + from start + `Duration` seconds, so a family with neither `end_field` nor + `Duration` would silently report no expiry — the one thing this feature is + for. + """ + members = inventory_members[spec.key] + assert spec.start_field in members, ( + f"{spec.key} start field {spec.start_field} absent from " + f"{spec.service}.{spec.method}" + ) + if spec.end_field: + assert spec.end_field in members, ( + f"{spec.key} end field {spec.end_field} absent from " + f"{spec.service}.{spec.method}" + ) + else: + assert "Duration" in members, ( + f"{spec.key} has no end_field and {spec.service}.{spec.method} " + f"models no Duration, so its expiry cannot be derived" + ) + + +@pytest.mark.parametrize("spec", RESERVATION_INVENTORY, ids=lambda s: s.key) +def test_inventory_state_field_exists(spec, inventory_members): + """`State` exists — it is what separates a live commitment from history.""" + assert "State" in inventory_members[spec.key], ( + f"{spec.service}.{spec.method} models no State, so " + f"ACTIVE_RESERVATION_STATES cannot filter retired rows" + ) + + +@pytest.mark.parametrize("spec", RESERVATION_INVENTORY, ids=lambda s: s.key) +def test_inventory_attribute_and_arn_fields_exist(spec, inventory_members): + """Renewal-matching attributes exist on the wire. + + These are the dimensions a renewal has to match — RDS `MultiAZ` and + `ProductDescription`, EC2 `Scope`/`AvailabilityZone`. A typo drops one + silently, and a renewal bought against an unstated deployment option or + engine does not apply the discount. + """ + members = inventory_members[spec.key] + expected = [field for field, _label in spec.attribute_fields] + if spec.arn_field: + expected.append(spec.arn_field) + missing = [field for field in expected if field not in members] + assert not missing, ( + f"{spec.key} reads {missing}, absent from {spec.service}.{spec.method}. " + f"Available: {sorted(members)}" + ) + + +def test_rds_multi_az_is_still_boolean(inventory_members, botocore_session): + """`MultiAZ` is a bool, which is why it is tested against None, not truth. + + `False` means Single-AZ — a real, expensive specification — so the code + checks `is not None` and renders through INVENTORY_ATTRIBUTE_VALUES rather + than treating a falsy value as absent. If AWS ever changed this to a string + that logic would need revisiting, so the type is pinned here. + """ + service = botocore_session.get_service_model("rds") + operation = operation_for(service, "describe_reserved_db_instances") + members = structure_members( + structure_members(operation.output_shape)["ReservedDBInstances"] + ) + assert members["MultiAZ"].type_name == "boolean" + + +# --------------------------------------------------------------------------- +# Savings Plans: the one family denominated in dollars +# --------------------------------------------------------------------------- +@pytest.fixture(scope="module") +def savings_plan_model(botocore_session): + return botocore_session.get_service_model("savingsplans") + + +def test_savings_plan_fields_exist(savings_plan_model): + """Every `describe_savings_plans` field the collector reads exists. + + `ec2InstanceFamily` matters most: it is present for an EC2 Instance Savings + Plan and absent for a Compute plan, and the collector renders the absence as + an empty spec on purpose. That distinction only holds if the field name is + right — a typo would make *every* plan look like a Compute plan. + """ + members = structure_members( + structure_members( + savings_plan_model.operation_model("DescribeSavingsPlans").output_shape + )["savingsPlans"] + ) + expected = ( + "savingsPlanId", + "savingsPlanArn", + "savingsPlanType", + "ec2InstanceFamily", + "commitment", + "region", + "state", + "paymentOption", + "start", + "end", + ) + missing = [field for field in expected if field not in members] + assert not missing, ( + f"describe_savings_plans no longer models {missing}. " + f"Available: {sorted(members)}" + ) + + +def test_active_sp_states_are_valid_enum_values(savings_plan_model): + """`ACTIVE_SP_STATES` are real API enum values. + + These go to the API as a server-side `states` filter, so an invalid value is + a ValidationException at collection time — in a Lambda, against a live + account, rather than here. + """ + valid = savings_plan_model.shape_for("SavingsPlanState").enum + invalid = [state for state in ACTIVE_SP_STATES if state not in valid] + assert not invalid, f"{invalid} are not SavingsPlanState values; valid: {valid}" + + +def test_savings_plan_commitment_is_a_string_on_the_wire(savings_plan_model): + """`commitment` arrives as a string, so the float() conversion is required. + + AWS returns "1.00000000", not 1.0. Summing these without converting would + concatenate them, which is the kind of bug that produces a plausible-looking + but wrong dollar figure in a report. + """ + members = structure_members( + structure_members( + savings_plan_model.operation_model("DescribeSavingsPlans").output_shape + )["savingsPlans"] + ) + assert members["commitment"].type_name == "string" From 01008a48faee8d992792440f9defc9c91a2d792c Mon Sep 17 00:00:00 2001 From: raytoo Date: Wed, 16 Sep 2026 11:43:07 +0800 Subject: [PATCH 3/3] docs(commitments): OpenSearch/DynamoDB spec coverage is scope, not a gap Neither service is in scope for this deployment, so there is no live usage to size a reservation against and no live assertion to make. Reframes the contract-test limitation from an outstanding verification step to a stated boundary: those two shapes rest on the contract tests and unit fixtures, which already guarantee a blank spec column means AWS omitted the field rather than that the field name is wrong. --- docs/skills/discounted-commitments.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/docs/skills/discounted-commitments.md b/docs/skills/discounted-commitments.md index 1976624..450813f 100644 --- a/docs/skills/discounted-commitments.md +++ b/docs/skills/discounted-commitments.md @@ -1004,12 +1004,13 @@ Two coverage facts worth knowing, both asserted rather than assumed: for an unknown shape, so a service AWS adds later would cost the report its spec column silently; this turns that into a failing test instead. -What these tests do **not** do is prove a field is populated for a given -account. That needs an account holding the commitment or recommendation in -question — specifically one with OpenSearch or DynamoDB steady-state usage for -those two shapes, which the dev account has none of. What they guarantee is that -when such an account is used, a blank column means "AWS omitted it", never "we -spelled it wrong". +What these tests do **not** do is prove a field is *populated* for a given +account — that needs an account holding the commitment or recommendation in +question. OpenSearch and DynamoDB are the two shapes with no live coverage, +because neither service is in scope for this deployment and there is no usage to +size a reservation against; both rest on the contract tests and unit fixtures. +That is the guarantee worth having here: a blank spec column can only mean AWS +omitted the field, never that the field name is wrong. `test_tools_json_declares_every_dispatched_tool` reads the dispatcher table out of `handler.py` rather than restating it, so a tool added to one and not the