Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -247,8 +247,6 @@ scripts/

docs/
architecture.md # system internals
skills/
developer-guide/SKILL.md # Adding agents / tools / collectors (interactive)
agents/ # Per-leaf-agent references (deploy modes, data model, gotchas)
README.md # When to add a file + section template
cost-operations.md # Cost Explorer / CUR / COH reference
Expand All @@ -257,8 +255,14 @@ skills/
network-resiliency.md # Direct Connect topology + resilience rules
tag-governance.md # Tag governance feature reference
lambda-upgrade.md # Lambda runtime upgrade discovery + migration
skills/ # Per-skill references (method, API surface, portability)
discounted-commitments.md # SP/RI risk-adjusted sizing + commitments MCP tool
observability-tuning.md # X-Ray + Transaction Search knobs

skills/ # Portable agent skills (invocable as /skill-name)
developer-guide/SKILL.md # Adding agents / tools / collectors (interactive)
discounted-commitments/ # SP/RI sizing — 3 markdown files, no code

tests/unit/ # pytest + moto
```

Expand Down Expand Up @@ -339,6 +343,13 @@ See `.env.example` for the canonical identity-only `.env` template and
setup, tag-policy bring-up commands, read-only-by-design rationale.
- [docs/agents/lambda-upgrade.md](docs/agents/lambda-upgrade.md)
— Lambda runtime discovery, code analysis, and migration guidance.
- [docs/skills/](docs/skills/) — per-skill reference files for skills
with a non-trivial method or a platform counterpart.
- [docs/skills/discounted-commitments.md](docs/skills/discounted-commitments.md)
— Savings Plan / Reserved Instance risk-adjusted sizing: the
volatility bands and break-even guards, the Cost Explorer API
surface and its per-request cost, the `commitments` MCP tool, and
how the portable skill and the deployed Lambda stay in agreement.
- [docs/observability-tuning.md](docs/observability-tuning.md) —
X-Ray sampling and Transaction Search indexing knobs.

Expand All @@ -356,6 +367,7 @@ Skills provide the system's analytical capabilities as portable workflows that w
| `/health-events-digest` | "any critical health events?" | Health event digest with risk scoring |
| `/tag-governance-assessment` | "how's my tag compliance?" | Tag compliance scoring + remediation links |
| `/lambda-runtime-upgrade` | "find deprecated lambda functions" | Multi-region deprecated-runtime discovery + migration report |
| `/discounted-commitments` | "what savings plans should we buy?" | Risk-adjusted SP/RI sizing — achievable vs AWS best case ([docs](docs/skills/discounted-commitments.md)) |

### How to use skills

Expand Down
21 changes: 20 additions & 1 deletion docs/agents/cost-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ FinOps domain under `finops-agent` (peer of `pricing-agent`).

## 1. What the feature does

Answers spending questions using three complementary surfaces, from
Answers spending questions using four complementary surfaces, from
narrowest to broadest:

- **Cost Explorer API** — monthly / daily cost and usage, optional
Expand All @@ -19,6 +19,12 @@ narrowest to broadest:
- **Cost Optimization Hub** — savings recommendations (right-sizing,
idle resources, Savings Plans, Reserved Instances) aggregated
across the org.
- **Commitments** — risk-adjusted Savings Plan / Reserved Instance
purchase sizing. The only surface that reaches
`GetSavingsPlansPurchaseRecommendation` and
`GetReservationPurchaseRecommendation`, so it is the only one that can
size a commitment rather than just report existing coverage. Full
reference: [`docs/skills/discounted-commitments.md`](../skills/discounted-commitments.md).

Representative prompts:

Expand Down Expand Up @@ -182,6 +188,17 @@ Lambda hardcodes `region_name="us-east-1"`. Same pattern as
}
```

### `generate_commitment_analysis` (commitments)

Returns a `report_markdown` field holding the complete pre-formatted
report, alongside the structured envelope (`recommendations`, `count`,
`total_estimated_monthly_savings`, `aws_best_case_monthly_savings`,
`reconciliation`, `existing_commitment_posture`, `blockers`). The agent
prompt requires emitting `report_markdown` verbatim rather than rebuilding
the table from the structured fields. Response shape and the other three
commitment tools:
[`docs/skills/discounted-commitments.md`](../skills/discounted-commitments.md).

### `start_query_execution` (CUR/Athena)

Synchronous — the Lambda waits for the Athena query to complete (up
Expand Down Expand Up @@ -209,3 +226,5 @@ savings, anomalies, and forecast sections. See
| Cost Explorer shows different totals than the bill | Metric mismatch (UnblendedCost vs AmortizedCost vs NetAmortizedCost) + Credits/Refunds exclusion | Specify the metric explicitly; match what the finance team uses for reconciliation |
| Forecast returns an error for start_date in the past | `get_cost_forecast` requires future start_date | Use `get_cost_and_usage` for historical; forecast is future-only |
| Model called `group_by=["SERVICE"]` when the user only asked "how much" | Model embellishing beyond the ask | The worker prompt already gates this; if it recurs, tighten the "ONE call, no group_by unless asked" rule |
| Model answered a "should we buy an SP?" question from `cost-optimization-hub` alone | COH publishes commitment recommendations but cannot size one | The worker prompt routes all SP/RI purchase questions to the `commitments` tools; see [`docs/skills/discounted-commitments.md`](../skills/discounted-commitments.md) |
| Unexpected Cost Explorer charges after a commitment question | CE bills $0.01 per recommendation request; a full default sweep is 48 | Narrow `savings_plan_types` / `ri_services` on `generate_commitment_analysis` |
3 changes: 2 additions & 1 deletion docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This file documents project-specific conventions, architecture, and gotchas for

## Project

CloudOps Multi-Agent System — a hierarchical multi-agent system for AWS cloud operations built on Amazon Bedrock AgentCore with Strands Agents SDK and an AG-UI streaming Next.js frontend. See `README.md` for architecture diagrams and `skills/developer-guide/SKILL.md` for step-by-step how-tos. `docs/agents/` has one reference file per leaf agent (deploy modes, data model, gotchas) — `health-events.md` and `tag-governance.md` today; add a new file here for any new leaf with non-trivial deploy or operational surface.
CloudOps Multi-Agent System — a hierarchical multi-agent system for AWS cloud operations built on Amazon Bedrock AgentCore with Strands Agents SDK and an AG-UI streaming Next.js frontend. See `README.md` for architecture diagrams and `skills/developer-guide/SKILL.md` for step-by-step how-tos. `docs/agents/` has one reference file per leaf agent (deploy modes, data model, gotchas); add a new file there for any new leaf with a non-trivial deploy or operational surface. `docs/skills/` does the same for skills whose method needs documenting beyond `SKILL.md` — typically because a platform Lambda implements the same logic and the two copies have to stay in agreement.

The richest source of project-specific conventions and gotchas is this file plus the `docs/` directory — treat them as authoritative. `docs/architecture.md` covers agent topology decisions.

Expand Down Expand Up @@ -75,6 +75,7 @@ These are load-bearing and not obvious from reading the code:

- **MCP import name collision**: use `streamablehttp_client` (no underscores) from `mcp.client.streamable_http` for SigV4 `auth=` support. The similarly-named `streamable_http_client` silently drops `auth=`, tools fail to load, and the model hallucinates fake `<function_calls>` XML with fabricated data.
- **AgentCore Gateway paginates `tools/list` at 30 per page.** `MCPClient.list_tools_sync()` returns a `PaginatedList` — `len()` is the current page only, and the cursor is exposed as `.pagination_token`. Any target whose tools land on page 2+ is invisible unless you drain the cursor. `agent_base.py::load_gateway_tools` drains pages in a loop; do NOT revert that to a single call. Symptom when skipped: filter reports `Filtered 0/30 gateway tools` even though the gateway has the target READY with correct inline schemas, model has zero tools, platform no-fabrication preamble fires "no tools available" or model fabricates plausible numbers.
- **`list_gateway_targets` paginates too, and boto3 does not follow `nextToken`.** Same class of bug as the entry above, one API layer down, and harder to spot because the AWS CLI *does* auto-paginate: `aws bedrock-agentcore-control list-gateway-targets` showed all 11 targets while the identical boto3 call returned 10 plus a `nextToken` nobody read. `sync_gateway_tools` in `scripts/lib/sync.sh` drains the token in a loop; do NOT revert that to a single call. Symptom when skipped: `<target>: target not found in gateway, skipping` → `[WARN] Gateway tool schema sync failed (non-fatal)` for a target that is present and READY — because a dropped target is indistinguishable from one that was never created. This shipped once: it left the `lambda-runtime` target with zero tool schemas, making all 8 of its tools unreachable. Guarded by `tests/scripts/test_sync_gateway_targets.sh`, which stubs boto3 to serve scripted pages and asserts a target on the last page is still found (and that a genuinely absent one still fails loudly).
- **Hallucination guardrail is platform-level, not per-prompt.** `agent_base.py` prepends a non-negotiable `_NO_FABRICATION_PREAMBLE` to every agent's system prompt AND refuses to invoke a leaf with zero tools (returns a clear error instead of letting the model improvise). Agent authors do not opt in; the factories apply this unconditionally. If you add a new agent factory, call `_apply_platform_preamble(_inject_tool_inventory(prompt, tools))` — do not bypass.
- **Bedrock Guardrail (standalone ApplyGuardrail API).** NOT attached to the model (which caused false positives on system prompts). Instead, `shared/guardrail.py::check_user_input()` calls `bedrock-runtime:ApplyGuardrail` on ONLY the raw user message at the supervisor entry point (in `agui_server.py`, BEFORE the agent is built/run). Prompt attack detection + sensitive info filters + topic policy. System prompts never reach the classifier (they're assembled later in `agent_base.py` and aren't user-modifiable). Env vars: `BEDROCK_GUARDRAIL_ID`, `BEDROCK_GUARDRAIL_VERSION`, `GUARDRAIL_MODE` (supervisor-only). `GUARDRAIL_MODE` is `block` (default — refuse flagged input) or `detect` (log-only, non-blocking; set via root tfvar `guardrail_mode`). Layered with: (a) `_NO_FABRICATION_PREAMBLE` behavioral constraints, (b) IAM least-privilege per-tool (hard backstop), (c) `shared/redact.py` strips genuine secrets (access keys, external IDs, role-session names) from persisted data (memory + reports); account IDs and ARNs are identifiers kept by default, scrubbed only when `REDACT_IDENTIFIERS=true`. **The CMK on DynamoDB requires every agent role to have `kms:Decrypt` on the platform key — without it, registry reads fail closed and the agent reports "no child agents deployed".**
- **MCPClient lifecycle in leaf agents**: manual `__enter__()` → `list_tools_sync()` → pass tools (not the client) to `Agent(tools=tools)` → `__exit__` in `finally`. Passing `MCPClient` directly to `Agent` causes `"client failed to initialize"` in Runtime containers.
Expand Down
Loading