diff --git a/README.md b/README.md index b3c34ee..191991f 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ **Run every AI model your teams need - one API, your AWS account!** -This sample solution provides a self-service platform that lets teams use large language models the way they ship code: commit a few lines of YAML, git push, and the platform handles the rest - GPUs, serving, scaling, routing, and monitoring. Use frontier **models from Amazon Bedrock** out of the box with no GPUs to manage or **deploy any open-source models**, provisioned and served automatically. +This sample solution provides a self-service platform that lets teams use large language models the way they ship code: commit a few lines of YAML, git push, and the platform handles the rest - GPUs, serving, scaling, routing, and monitoring. Enroll frontier **models from Amazon Bedrock** with a single command - no GPUs to manage - or **deploy any open-source model**, provisioned and served automatically. And it all runs in your AWS account and desired AWS region - including the AWS European Sovereign Cloud - so your data and models stay where you control them. @@ -41,11 +41,19 @@ The custom resources **are** the self-service interface: | **`VLLMEndpoint`** | Serve a model on vLLM - the simple default: one model, one pod, one instance (any Hugging Face model ID) | | **`LLMDEndpoint`** | Serve a model on the llm-d scale tier - KV-cache/load/prefix-aware routing across replicas (the `inference-gateway` substrate ships on every cluster; no toggle) | | **`LLMDDisaggEndpoint`** | Serve on the llm-d scale + performance tier - independently autoscaled prefill/decode pools (same llm-d substrate; no toggle) | +| **`BedrockModel`** | Enroll a managed Amazon Bedrock model - no GPUs, no serving pod (`platformctl new-model --source bedrock`); litellm-sync registers it on the same `/v1` API | | **`AITeam`** | Onboard a team: namespace, RBAC, budget, rate limits, scoped API key | -Bedrock models need no resource - they're a few lines of LiteLLM config (`litellm.yaml`), live the -moment the cluster is up. KRO definitions live in `platform/config/kro/`; extend -them there and every model/team inherits the change. +Bedrock models need no GPUs and no serving pod - enroll one with +`./platformctl new-model --source bedrock ` and the platform commits a +`BedrockModel` CR; litellm-sync registers it on the `/v1` API in seconds. +Nothing ships by default, so the committed config never pins a model id that +would be wrong in another region or partition - the CLI resolves the +region/partition-correct invocation id (a cross-region inference profile where +one exists, else the base model id) and its price at enrollment. KRO definitions +live in `platform/config/kro/` and the `BedrockModel` CRD in +`platform/config/bedrockmodel-crd.yaml`; extend them there and every model/team +inherits the change. Every model answers through the same LiteLLM `/v1` API, so governance, budgets, and tracing apply uniformly - including the optional **llm-d** scale tier @@ -64,7 +72,7 @@ tracing apply uniformly - including the optional **llm-d** scale tier - **Terraform**, **kubectl**, **make**, **jq**, **git**, and **python3** with **boto3** **AWS account setup**: -- (Optional) If using **Amazon Bedrock models**. Enable desired model(s) in the AWS console and specify model id(s) in `litellm.yaml`. +- (Optional) If using **Amazon Bedrock models**: enable the desired model(s) in the AWS console (one-time model-access toggle). You enroll them after the cluster is up with `./platformctl new-model --source bedrock` (nothing is pinned in git) - see the Quick start. - (Optional) For any **self deployed model**, sufficient **service quota** for the GPU instance types you plan to self-host on (not needed for the Bedrock-only path) - (Optional) If using EKS Managed Capabilities (`eks_capabilities = true`): An **IAM Identity Center** instance for managed ArgoCD - its ARN and the SSO user who should get @@ -106,22 +114,7 @@ cd terraform/00.global/vars && cp example.tfvars dev.tfvars # In case of using EKS Managed Capabilities `eks_capabilities = true`: Your Identity Center ARN + **its region** (`argocd_idc_region`, may differ from `region`) + your **SSO user id** (`argocd_rbac_mappings`), ``` -2. Optional for use with Bedrock, adjust `litellm.yaml`, and update model_list: - -```yaml - - model_name: opus-4-8 - litellm_params: - model: bedrock/global.anthropic.claude-opus-4-8 - aws_region_name: os.environ/AWS_REGION -``` - -For AWS European Sovereign Cloud (ESC) -```yaml - - model_name: nova-lite - litellm_params: - model: bedrock/amazon.nova-lite-v1:0 - aws_region_name: os.environ/AWS_REGION -``` +2. No models ship by default. You enroll them after the cluster is up - Amazon Bedrock in one command (step 5) or a self-hosted model (step 6). Nothing model-specific is pinned in git, so the committed config is correct in any region and partition (including the ESC partition). 3. Provision everything (VPC → EKS + capabilities → Karpenter → secrets). ```bash @@ -132,10 +125,25 @@ For AWS European Sovereign Cloud (ESC) 4. Test - no GPUs yet (up already pointed kubectl at the new cluster) ```bash ./platformctl tunnel # forward the UIs (WebUI / LiteLLM / Langfuse / Grafana / ArgoCD) -./platformctl status --check # verify Bedrock + models answer AND Langfuse tracing works +./platformctl status --check # verify the platform is healthy + Langfuse tracing works (no models answer yet - you enroll them next) +``` + +5. Enroll an Amazon Bedrock model - no GPUs, live in seconds. +```bash +# See what's invokable in your region/partition (alias, invocation id, pricing): +./platformctl new-model --source bedrock --list-available-models +# Enroll one: resolves the region/partition-correct invocation id (a cross-region +# inference profile where one exists, else the base model id) + per-token price, +# commits a BedrockModel CR, and litellm-sync registers it on the /v1 API. +./platformctl new-model --source bedrock nova-lite --deploy # add -y to skip the confirm prompt ``` +Drop `--deploy` to preview the `BedrockModel` CR without pushing. Requires Bedrock +model access enabled in-account (a one-time console toggle). For a partition the +AWS Price List API doesn't cover (e.g. ESC), pass `--input-cost`/`--output-cost` +to record accurate per-token pricing. Remove it later with +`./platformctl new-model --undeploy nova-lite`. -5. Deploy a self-hosted model with one command. +6. Deploy a self-hosted model with one command. ```bash # `new-model` right-sizes it and ships it end to end: # - reads the model's config from Hugging Face and computes its VRAM + @@ -176,7 +184,7 @@ Example with fine tuning > hourly cost) may differ from the recommendation. The model still fits and serves > correctly; only the specific instance may vary. To override the selection, you may specify `--instance-type`. -6. Removing a model +7. Removing a model Folder `workloads/models/inference` ```bash @@ -206,7 +214,7 @@ Known Issues - Public ECR Repo equivalent not available. Karpenter, ACK, aws-application-networking (LB/Gateway), eks-distro will be loaded from `public.ecr.aws`. - Requires NAT Gateway - EKS Managed Capability not available - add-ons will be installed via Helm automatically instead `eks_capabilities = false` -- Bedrock: Limited models available, make sure to adjust `litellm.yaml` +- Bedrock (ESC): limited catalog - list what's invokable with `./platformctl new-model --source bedrock --list-available-models`. The AWS Price List API doesn't cover the ESC partition, so pass `--input-cost`/`--output-cost` when enrolling for accurate per-token cost tracking. ## Beyond the basics @@ -342,6 +350,7 @@ aws iam list-roles --query "Roles[?contains(RoleName, '')].RoleNam ``` argocd/bootstrap/ ApplicationSets (platform services + self-service workloads) platform/ + config/ BedrockModel CRD (bedrockmodel-crd.yaml) - Bedrock enrollment config/kro/ VLLMEndpoint · LLMDEndpoint · LLMDDisaggEndpoint · AITeam (the API) services/ litellm, litellm-sync, open-webui, langfuse, gpu-operator, cluster-dashboard (+ Platform Health Agent), inference-gateway diff --git a/argocd/bootstrap/templates/projects.yaml b/argocd/bootstrap/templates/projects.yaml index 98bf971..c787c77 100644 --- a/argocd/bootstrap/templates/projects.yaml +++ b/argocd/bootstrap/templates/projects.yaml @@ -63,11 +63,24 @@ spec: namespace: inference - server: https://kubernetes.default.svc namespace: team-* + # EKS-managed ArgoCD (managed capabilities) registers the in-cluster + # destination as the NAMED cluster `local-cluster` whose server URL is NOT + # https://kubernetes.default.svc, so the server-based entries above don't + # match and every workload app fails with InvalidSpecError. Also allow the + # destination BY NAME — same namespaces, same enforced boundary — so both + # self-managed and EKS-managed ArgoCD work. + - name: local-cluster + namespace: ai-platform + - name: local-cluster + namespace: inference + - name: local-cluster + namespace: team-* # Deny ALL cluster-scoped resources: an empty whitelist permits nothing. clusterResourceWhitelist: [] - # Permit ONLY the KRO endpoint/team kinds. Every other namespaced kind - # (Deployment, Secret, Role/RoleBinding, ServiceAccount, raw Pod, …) is - # refused, so a workloads commit cannot escalate privilege or read secrets. + # Permit ONLY the KRO endpoint/team kinds plus the (infra-less) BedrockModel + # marker. Every other namespaced kind (Deployment, Secret, Role/RoleBinding, + # ServiceAccount, raw Pod, …) is refused, so a workloads commit cannot escalate + # privilege or read secrets. namespaceResourceWhitelist: - group: kro.run kind: VLLMEndpoint @@ -77,3 +90,8 @@ spec: kind: LLMDDisaggEndpoint - group: kro.run kind: AITeam + # Self-service Amazon Bedrock enrollment (platformctl new-model --source + # bedrock). Without this, ArgoCD refuses to sync a BedrockModel CR + # ("resource bedrock.ai-platform:BedrockModel is not permitted in project"). + - group: bedrock.ai-platform + kind: BedrockModel diff --git a/ops/lib/recommend_instance/bedrock.py b/ops/lib/recommend_instance/bedrock.py new file mode 100644 index 0000000..64b2fce --- /dev/null +++ b/ops/lib/recommend_instance/bedrock.py @@ -0,0 +1,436 @@ +"""Self-service Amazon Bedrock model enrollment for `platformctl new-model +--source bedrock`. + +Unlike the GPU serving path there is nothing to size or schedule: a Bedrock model +runs in AWS. This module + + 1. lists what's actually invokable in the deploy region from the LIVE Bedrock + control-plane APIs (ListFoundationModels + ListInferenceProfiles) — so it is + correct in any partition (the AWS European Sovereign Cloud included) with no + hardcoded catalog to drift; + 2. resolves the region/partition-correct invocation id — a cross-region system + inference profile (e.g. us.amazon.nova-lite-v1:0) where one exists, else the + bare foundation-model id; + 3. looks up per-token pricing from the AWS Price List API (best-effort — the + Price List API is commercial-only, so ESC/other partitions fall back to the + --input-cost/--output-cost overrides or LiteLLM's built-in map); + 4. emits a BedrockModel CR under workloads/models/ that litellm-sync registers + in LiteLLM — reusing the exact same git-push -> ArgoCD -> registration path + as the serving tiers. + +Nothing is offered by default; users enroll exactly the models they want. +""" + +from __future__ import annotations + +import argparse +import json +import re +import sys + +from .paths import MODELS_ROOT, is_valid_model_name +from .pricing import detect_region +from .ux import _palette, _should_use_colour + +# Bedrock models are namespaced to `inference` (like the platform-default vLLM +# placement); litellm-sync watches cluster-wide so the namespace is only about +# where the workloads ApplicationSet syncs the file. +BEDROCK_DIR = f"{MODELS_ROOT}/inference" + +# A Bedrock invocation id: foundation-model id or inference-profile id, e.g. +# amazon.nova-lite-v1:0 or us.anthropic.claude-3-5-sonnet-20240620-v1:0. Charset +# is letters/digits plus . : - (NO quotes/spaces/newlines) so a resolved value is +# safe to interpolate into the YAML manifest committed + applied by ArgoCD. +_INVOKE_ID_RE = re.compile(r"\A[A-Za-z0-9][A-Za-z0-9._:\-]*\Z") + + +# --------------------------------------------------------------------------- # +# Partition / endpoint helpers (mirror terraform/30.eks/30.cluster/bedrock.tf) # +# --------------------------------------------------------------------------- # + +def partition_for(region: str) -> str: + if region.startswith("eusc-"): + return "aws-eusc" # AWS European Sovereign Cloud + if region.startswith("us-gov-"): + return "aws-us-gov" + if region.startswith("cn-"): + return "aws-cn" + return "aws" + + +def dns_suffix_for(partition: str) -> str: + return { + "aws": "amazonaws.com", + "aws-eusc": "amazonaws.eu", + "aws-cn": "amazonaws.com.cn", + "aws-us-gov": "amazonaws.com", + }.get(partition, "amazonaws.com") + + +def runtime_endpoint(region: str, partition: str) -> str: + return f"https://bedrock-runtime.{region}.{dns_suffix_for(partition)}" + + +def _region_geo(region: str) -> str: + """The cross-region inference-profile geography prefix for a region.""" + if region.startswith(("eu-", "eusc-")): + return "eu" + if region.startswith("ap-"): + return "apac" + return "us" # us-*, ca-*, sa-*, us-gov-* + + +# --------------------------------------------------------------------------- # +# Live Bedrock catalog # +# --------------------------------------------------------------------------- # + +def _client(service: str, region: str): + """Return a boto3 client, or None if boto3/credentials are unavailable.""" + try: + import boto3 # type: ignore + except ImportError: + return None + try: + return boto3.client(service, region_name=region) + except Exception: + return None + + +def list_text_models(region: str) -> list[dict]: + """Text-output foundation models invokable in `region` (on-demand or via an + inference profile), newest-listed first. Raises RuntimeError on API failure.""" + cli = _client("bedrock", region) + if cli is None: + raise RuntimeError("boto3 or AWS credentials unavailable — cannot list Bedrock models.") + try: + resp = cli.list_foundation_models(byOutputModality="TEXT") + except Exception as e: # noqa: BLE001 + raise RuntimeError(f"bedrock:ListFoundationModels failed in {region}: {e}") from e + out: list[dict] = [] + for m in resp.get("modelSummaries", []) or []: + # Only models we can actually invoke: on-demand or via an inference + # profile. Skip PROVISIONED-only and non-ACTIVE lifecycle models. + types = m.get("inferenceTypesSupported") or [] + if types and not ({"ON_DEMAND", "INFERENCE_PROFILE"} & set(types)): + continue + if (m.get("modelLifecycle") or {}).get("status") not in (None, "ACTIVE"): + continue + out.append(m) + return out + + +def system_profiles_by_model(region: str) -> dict[str, list[str]]: + """Map foundation-model id -> [system inference-profile ids] usable from + `region`. Empty when none exist (e.g. single-region / sovereign partitions).""" + cli = _client("bedrock", region) + if cli is None: + return {} + mapping: dict[str, list[str]] = {} + try: + token = None + for _ in range(20): # bounded pagination + kwargs = {"typeEquals": "SYSTEM_DEFINED", "maxResults": 100} + if token: + kwargs["nextToken"] = token + resp = cli.list_inference_profiles(**kwargs) + for p in resp.get("inferenceProfileSummaries", []) or []: + pid = p.get("inferenceProfileId") + if not pid or (p.get("status") not in (None, "ACTIVE")): + continue + for mdl in p.get("models", []) or []: + arn = mdl.get("modelArn", "") + fm_id = arn.split("foundation-model/", 1)[-1] if "foundation-model/" in arn else "" + if fm_id: + mapping.setdefault(fm_id, []).append(pid) + token = resp.get("nextToken") + if not token: + break + except Exception: # noqa: BLE001 — profiles are an optimization; degrade to bare ids + return mapping + return mapping + + +def _best_profile(region: str, profile_ids: list[str]) -> str | None: + """Prefer the profile whose geography matches the region, then a global. + profile, then any.""" + if not profile_ids: + return None + geo = _region_geo(region) + for pid in profile_ids: + if pid.startswith(f"{geo}."): + return pid + for pid in profile_ids: + if pid.startswith("global."): + return pid + return profile_ids[0] + + +def default_alias(fm_id: str) -> str: + """Kebab alias for a foundation-model id, e.g. amazon.nova-lite-v1:0 -> + nova-lite, anthropic.claude-3-5-sonnet-20240620-v1:0 -> claude-3-5-sonnet.""" + tail = fm_id.split(".", 1)[-1] if "." in fm_id else fm_id # drop provider + tail = re.sub(r"[-_.]v\d+.*$", "", tail) # drop -v1:0 / -20240620-v1:0 + tail = tail.replace(":", "-").replace(".", "-").replace("_", "-").lower() + tail = re.sub(r"-+", "-", tail).strip("-") + return tail + + +def resolve(region: str, query: str, fms: list[dict], + profiles: dict[str, list[str]]) -> tuple[dict, str, bool]: + """Resolve a user query to (foundation_model, invocation_id, used_profile). + + Accepts a foundation-model id, an inference-profile id (with geo prefix), a + bare model name, or a substring. Raises ValueError with guidance on no/ambiguous + match.""" + q = query.strip().lower() + # If the query is itself a geo-prefixed profile id, reduce to its base fm id. + base_q = re.sub(r"^(us|eu|apac|global)\.", "", q) + + def _matches(fm: dict) -> bool: + mid = (fm.get("modelId") or "").lower() + mname = (fm.get("modelName") or "").lower() + return q in (mid, mname) or base_q == mid or q in mid or base_q in mid \ + or q in mname or default_alias(fm.get("modelId") or "") == q + + exact = [fm for fm in fms if (fm.get("modelId") or "").lower() in (q, base_q)] + cands = exact or [fm for fm in fms if _matches(fm)] + if not cands: + raise ValueError( + f"no Bedrock model matches '{query}' in {region}. " + f"Run: ./platformctl new-model --source bedrock --list-available-models") + if len(cands) > 1: + ids = ", ".join(sorted((c.get("modelId") or "") for c in cands)[:8]) + raise ValueError( + f"'{query}' is ambiguous in {region} — matches: {ids}. " + f"Re-run with a more specific model id.") + fm = cands[0] + fm_id = fm.get("modelId") or "" + pid = _best_profile(region, profiles.get(fm_id, [])) + return (fm, pid or fm_id, pid is not None) + + +# --------------------------------------------------------------------------- # +# Pricing (AWS Price List API — best-effort) # +# --------------------------------------------------------------------------- # + +def token_prices(region: str, fm: dict) -> tuple[float | None, float | None]: + """Best-effort (input, output) USD-per-token from the AWS Price List API. + + Returns (None, None) on any uncertainty — the Price List API is commercial + partition only, its Bedrock schema varies by model family, and we never want + a pricing miss to block enrollment. The --input-cost/--output-cost flags + override this.""" + cli = _client("pricing", "us-east-1") # Price List API: us-east-1 / ap-south-1 / eu-central-1 + if cli is None: + return (None, None) + fm_id = fm.get("modelId") or "" + model_token = default_alias(fm_id).replace("-", "") # e.g. novalite + if not model_token: + return (None, None) + in_price = out_price = None + try: + token = None + for _ in range(6): # bounded pagination + kwargs = dict( + ServiceCode="AmazonBedrock", + Filters=[{"Type": "TERM_MATCH", "Field": "regionCode", "Value": region}], + MaxResults=100, + ) + if token: + kwargs["NextToken"] = token + resp = cli.get_products(**kwargs) + for raw in resp.get("PriceList", []): + data = raw if isinstance(raw, dict) else json.loads(raw) + attrs = (data.get("product") or {}).get("attributes") or {} + blob = json.dumps(attrs).lower() + if model_token not in blob.replace("-", "").replace(" ", "").replace(".", ""): + continue + for term in (data.get("terms") or {}).get("OnDemand", {}).values(): + for dim in (term.get("priceDimensions") or {}).values(): + desc = (dim.get("description") or "").lower() + unit = (dim.get("unit") or "").lower() + if "token" not in (desc + unit): + continue + usd = (dim.get("pricePerUnit") or {}).get("USD") + if usd is None: + continue + per_unit = float(usd) + if per_unit <= 0: + continue + # Bedrock token dims are typically per 1,000 tokens. + divisor = 1_000_000.0 if ("1m" in unit or "million" in desc) else 1_000.0 + per_token = per_unit / divisor + if "input" in desc and in_price is None: + in_price = per_token + elif "output" in desc and out_price is None: + out_price = per_token + token = resp.get("NextToken") + if not token or (in_price is not None and out_price is not None): + break + except Exception: # noqa: BLE001 — best-effort only + return (None, None) + return (in_price, out_price) + + +# --------------------------------------------------------------------------- # +# CR emission # +# --------------------------------------------------------------------------- # + +def _fmt_cost(v: float) -> str: + # Per-token costs are tiny; render without scientific notation and trim zeros. + return f"{v:.12f}".rstrip("0").rstrip(".") or "0" + + +def build_yaml(alias: str, invocation_id: str, partition: str, region: str, + in_cost: float | None, out_cost: float | None) -> tuple[str, str, str, str]: + """Build the BedrockModel manifest. Returns (name, yaml_path, yaml_body, commit_msg).""" + if not is_valid_model_name(alias): + sys.exit(f"error: '{alias}' is not a valid Kubernetes name (RFC 1123 label). " + f"Pass --model-name with a valid alias.") + if not _INVOKE_ID_RE.match(invocation_id): + sys.exit(f"error: refusing to emit an unsafe Bedrock invocation id: {invocation_id!r}.") + + lines = [ + "# BedrockModel — generated by 'platformctl new-model --source bedrock'.", + f"# Enrolls Amazon Bedrock model '{invocation_id}' as LiteLLM alias '{alias}'.", + "# litellm-sync registers it (bedrock/ upstream, IRSA auth). Remove with", + f"# ./platformctl new-model --undeploy {alias}", + "apiVersion: bedrock.ai-platform/v1alpha1", + "kind: BedrockModel", + "metadata:", + f" name: {alias}", + " namespace: inference", + "spec:", + f' model: "bedrock/{invocation_id}"', + f" modelName: {alias}", + ] + # Sovereign / non-commercial partitions need the explicit runtime endpoint; + # the standard aws partition uses LiteLLM's default (omit the field). + if partition != "aws": + lines.append(f' bedrockEndpoint: "{runtime_endpoint(region, partition)}"') + if in_cost is not None: + lines.append(f' inputCostPerToken: "{_fmt_cost(in_cost)}"') + if out_cost is not None: + lines.append(f' outputCostPerToken: "{_fmt_cost(out_cost)}"') + + yaml_body = "\n".join(lines) + "\n" + yaml_path = f"{BEDROCK_DIR}/{alias}.yaml" + commit_msg = f"feat: enroll Bedrock model {alias} ({invocation_id})" + return alias, yaml_path, yaml_body, commit_msg + + +# --------------------------------------------------------------------------- # +# Commands # +# --------------------------------------------------------------------------- # + +def _list_available(region: str, args: argparse.Namespace) -> int: + C = _palette(_should_use_colour(args)) + partition = partition_for(region) + try: + fms = list_text_models(region) + except RuntimeError as e: + sys.stderr.write(f"error: {e}\n") + return 2 + profiles = system_profiles_by_model(region) + + print(f"\n{C.BOLD}Amazon Bedrock text models available in " + f"{region}{C.RESET} {C.DIM}(partition {partition}){C.RESET}") + print(f"{C.DIM}Enroll one with: ./platformctl new-model --source bedrock " + f" --deploy{C.RESET}\n") + header = f" {'ALIAS':<24} {'INVOCATION ID (use this)':<48} {'$/1K IN':>9} {'$/1K OUT':>9}" + print(f"{C.DIM}{header}{C.RESET}") + print(f"{C.DIM} {'-'*24} {'-'*48} {'-'*9} {'-'*9}{C.RESET}") + + for fm in fms: + fm_id = fm.get("modelId") or "" + pid = _best_profile(region, profiles.get(fm_id, [])) + invocation = pid or fm_id + alias = default_alias(fm_id) + in_c, out_c = token_prices(region, fm) + in_s = f"${in_c*1000:.5f}" if in_c is not None else "—" + out_s = f"${out_c*1000:.5f}" if out_c is not None else "—" + print(f" {alias:<24} {invocation:<48} {in_s:>9} {out_s:>9}") + + ep_note = (f" Runtime endpoint {runtime_endpoint(region, partition)} is baked " + f"into each CR." if partition != "aws" else "") + print(f"\n{C.DIM}'INVOCATION ID' is a cross-region inference profile where one " + f"exists in this region, else the base model id.{ep_note}{C.RESET}") + print(f"{C.DIM}Pricing is best-effort from the AWS Price List API ('—' = not " + f"resolved / not available in this partition; override with " + f"--input-cost/--output-cost).{C.RESET}") + if fms: + ex = default_alias(fms[0].get("modelId") or "") + print(f"\n{C.BOLD}Example:{C.RESET} {C.CYAN}./platformctl new-model " + f"--source bedrock {ex} --deploy{C.RESET}") + return 0 + + +def _enroll(region: str, args: argparse.Namespace) -> int: + C = _palette(_should_use_colour(args)) + partition = partition_for(region) + if not args.model: + sys.stderr.write("error: a Bedrock model id/name is required " + "(or use --list-available-models).\n") + return 2 + try: + fms = list_text_models(region) + except RuntimeError as e: + sys.stderr.write(f"error: {e}\n") + return 2 + profiles = system_profiles_by_model(region) + try: + fm, invocation_id, used_profile = resolve(region, args.model, fms, profiles) + except ValueError as e: + sys.stderr.write(f"error: {e}\n") + return 2 + + alias = getattr(args, "model_name", None) or default_alias(fm.get("modelId") or "") + + # Pricing: explicit overrides win; else best-effort from the Price List API. + in_cost = float(args.input_cost) if getattr(args, "input_cost", None) is not None else None + out_cost = float(args.output_cost) if getattr(args, "output_cost", None) is not None else None + if in_cost is None and out_cost is None: + in_cost, out_cost = token_prices(region, fm) + + prof_note = "cross-region inference profile" if used_profile else "base foundation-model id" + print(f"\n{C.BOLD}Bedrock model:{C.RESET} {fm.get('modelId')} " + f"{C.DIM}({fm.get('providerName','')}){C.RESET}") + print(f" {C.BOLD}Invocation id:{C.RESET} {invocation_id} {C.DIM}({prof_note}){C.RESET}") + print(f" {C.BOLD}LiteLLM alias:{C.RESET} {alias}") + if partition != "aws": + print(f" {C.BOLD}Runtime endpoint:{C.RESET} {runtime_endpoint(region, partition)}") + price_line = ( + f"in ${in_cost*1000:.5f}/1K, out ${out_cost*1000:.5f}/1K" + if (in_cost is not None and out_cost is not None) + else "not resolved (LiteLLM will use its built-in map; set --input-cost/--output-cost for accuracy)" + ) + print(f" {C.BOLD}Price:{C.RESET} {price_line}") + print(f" {C.DIM}Requires Bedrock model access enabled for this model in the " + f"console; ./platformctl status --check validates the live call.{C.RESET}") + + name, yaml_path, yaml_body, commit_msg = build_yaml( + alias, invocation_id, partition, region, in_cost, out_cost) + + if not args.deploy: + print(f"\n{C.BOLD}You'll enroll this{C.RESET} {C.DIM}→ {yaml_path}{C.RESET}") + for ln in yaml_body.rstrip().splitlines(): + print(f" {C.DIM}│{C.RESET} {ln}") + print(f"\n{C.BOLD}Enroll it:{C.RESET} {C.CYAN}./platformctl new-model " + f"--source bedrock {args.model} --deploy{C.RESET}") + print(f" {C.DIM}writes the CR, commits, pushes, and triggers ArgoCD — " + f"litellm-sync then registers it on the /v1 API.{C.RESET}") + return 0 + + # Bedrock has no HF weights — make sure the shared deploy path never tries to + # provision an hf-token Secret from an ambient $HF_TOKEN. + args.hf_token = None + from .gitops import deploy_model + return deploy_model(name, yaml_path, yaml_body, commit_msg, args) + + +def run_bedrock(args: argparse.Namespace) -> int: + """Entry point for `new-model --source bedrock` (dispatched from cli.main).""" + region = detect_region(getattr(args, "region", None)) + if getattr(args, "list_available_models", False): + return _list_available(region, args) + return _enroll(region, args) diff --git a/ops/lib/recommend_instance/cli.py b/ops/lib/recommend_instance/cli.py index 10a7d00..c939976 100644 --- a/ops/lib/recommend_instance/cli.py +++ b/ops/lib/recommend_instance/cli.py @@ -147,6 +147,26 @@ def main(argv: list[str] | None = None) -> int: "is not required with --undeploy.") p.add_argument("--yes", "-y", action="store_true", help="Skip the confirmation prompt for --deploy/--undeploy.") + + # --- Amazon Bedrock enrollment (--source bedrock) ------------------------ # + # A Bedrock model runs in AWS — no GPU sizing. These flags drive a separate + # code path (recommend_instance/bedrock.py) that lists/resolves models from + # the live Bedrock APIs and emits a BedrockModel CR. + p.add_argument("--source", choices=["huggingface", "bedrock"], default="huggingface", + help="Model source. 'huggingface' (default): self-host on GPUs " + "(the sizing path). 'bedrock': enroll a managed Amazon " + "Bedrock model (no GPUs) via a BedrockModel CR.") + p.add_argument("--list-available-models", action="store_true", + help="With --source bedrock: list the Bedrock models invokable " + "in this region (alias, invocation id, pricing) and exit.") + p.add_argument("--model-name", default=None, metavar="ALIAS", + help="With --source bedrock: override the LiteLLM alias " + "(default: derived from the model id, e.g. 'nova-lite').") + p.add_argument("--input-cost", type=float, default=None, metavar="USD_PER_TOKEN", + help="With --source bedrock: override input $/token (e.g. for " + "partitions the AWS Price List API doesn't cover, like ESC).") + p.add_argument("--output-cost", type=float, default=None, metavar="USD_PER_TOKEN", + help="With --source bedrock: override output $/token. See --input-cost.") args = p.parse_args(argv) # --undeploy is a pure file/git operation: short-circuit before any HF fetch @@ -155,6 +175,13 @@ def main(argv: list[str] | None = None) -> int: from .gitops import undeploy_model return undeploy_model(args.undeploy, args) + # --source bedrock: enroll or list Amazon Bedrock models. Handled entirely by + # the bedrock module; short-circuit BEFORE any HF fetch / VRAM sizing / + # instance recommendation, none of which apply to a managed Bedrock model. + if args.source == "bedrock": + from .bedrock import run_bedrock + return run_bedrock(args) + # Every other path needs a model to size/recommend. if not args.model: p.error("the 'model' argument is required (unless using --undeploy)") diff --git a/ops/lib/recommend_instance/gitops.py b/ops/lib/recommend_instance/gitops.py index b986b45..9a2ed2c 100644 --- a/ops/lib/recommend_instance/gitops.py +++ b/ops/lib/recommend_instance/gitops.py @@ -17,6 +17,7 @@ from __future__ import annotations +import base64 import json import os import re @@ -189,33 +190,36 @@ def _ensure_hf_token_secret(namespace: str, token: str, root: str, C: type) -> N serving pod reads via `secretKeyRef` (env HF_TOKEN). Without it the pod pulls HuggingFace unauthenticated (rate-limited/slow, and gated models fail). - Deliberately imperative and idempotent (`create --dry-run=client | apply`): - the secret is a credential and must NOT be committed to git, so it lives - outside the GitOps flow. Best-effort — a failure here does not undo the push - (git remains the source of truth); we print a manual fallback. - - Note: the token is passed on the kubectl argv (visible briefly in `ps`), - which is the standard kubectl secret-creation tradeoff.""" + The secret is a credential and must NOT be committed to git, so it lives + outside the GitOps flow. We build the Secret manifest IN-PROCESS (the token is + base64-encoded in memory) and pipe it to `kubectl apply -f -`, so the token + never appears on any argv — it would otherwise be readable in + /proc//cmdline by a co-tenant on a shared operator/CI host and could leak + into shell history or CI logs. Idempotent (apply creates or updates). + Best-effort — a failure here does not undo the push (git remains the source of + truth); we print a manual fallback.""" + manual = (f' printf %s "$HF_TOKEN" | kubectl create secret generic hf-token ' + f'-n {namespace} --from-file=token=/dev/stdin') if shutil.which("kubectl") is None: print(f"{C.YELLOW}kubectl not found{C.RESET} — could not create the hf-token " - f"secret; the pod will pull HuggingFace unauthenticated. Create it with:\n" - f" kubectl create secret generic hf-token -n {namespace} " - f"--from-literal=token=YOUR_TOKEN") - return - gen = _run(["kubectl", "create", "secret", "generic", "hf-token", - "-n", namespace, "--from-literal=token=" + token, - "--dry-run=client", "-o", "yaml"], cwd=root) - if gen.returncode != 0: - print(f"{C.YELLOW}Could not render the hf-token secret{C.RESET} " - f"({gen.stderr.strip() or 'kubectl failed'}).") + f"secret; the pod will pull HuggingFace unauthenticated. Create it with:\n{manual}") return + # Manifest built here (not via `kubectl create secret --from-literal`, which + # puts the token on argv). base64(data) is the Secret wire format; apply reads + # it from stdin only. + manifest = json.dumps({ + "apiVersion": "v1", + "kind": "Secret", + "metadata": {"name": "hf-token", "namespace": namespace}, + "type": "Opaque", + "data": {"token": base64.b64encode(token.encode()).decode()}, + }) apply = subprocess.run(["kubectl", "apply", "-n", namespace, "-f", "-"], - input=gen.stdout, capture_output=True, text=True, cwd=root) + input=manifest, capture_output=True, text=True, cwd=root) if apply.returncode != 0: print(f"{C.YELLOW}Could not apply the hf-token secret{C.RESET} " f"({apply.stderr.strip() or 'kubectl failed'}).") - print(f"{C.DIM}Create it manually: kubectl create secret generic hf-token " - f"-n {namespace} --from-literal=token=YOUR_TOKEN{C.RESET}") + print(f"{C.DIM}Create it manually:\n{manual}{C.RESET}") return print(f"{C.GREEN}✓ hf-token secret ensured{C.RESET} in namespace '{namespace}' — " f"the pod authenticates to HuggingFace. (An already-running pod must be " diff --git a/ops/lib/recommend_instance/paths.py b/ops/lib/recommend_instance/paths.py index c11b98b..8a1d7c9 100644 --- a/ops/lib/recommend_instance/paths.py +++ b/ops/lib/recommend_instance/paths.py @@ -26,6 +26,9 @@ "VLLMEndpoint": f"{MODELS_ROOT}/inference", "LLMDEndpoint": SCALE_MODELS_DIR, "LLMDDisaggEndpoint": SCALE_MODELS_DIR, + # Bedrock models (no GPU/infra) live alongside the vLLM models; the recursive + # find_model_files() search below makes `--undeploy ` work for them too. + "BedrockModel": f"{MODELS_ROOT}/inference", } # A model name doubles as a Kubernetes object name, so it must be an RFC 1123 diff --git a/ops/lib/recommend_instance/render.py b/ops/lib/recommend_instance/render.py index aa58adb..02af20b 100644 --- a/ops/lib/recommend_instance/render.py +++ b/ops/lib/recommend_instance/render.py @@ -1002,19 +1002,26 @@ def _valid_quantity(value: str, flag: str) -> str: def _yaml_scalar(v: str) -> str: - """Quote an arbitrary extraArgs token safely for YAML. Handles values that - contain double quotes (e.g. JSON like {\"method\":\"mtp\"}) by using single - quotes; rejects newlines to prevent manifest injection.""" + """Quote an arbitrary extraArgs token safely for YAML. + + YAML is a strict superset of JSON, so a ``json.dumps`` string is always a + valid YAML double-quoted scalar that decodes back to exactly ``v``. We + delegate to the stdlib JSON encoder so every escape hazard (backslashes, + both quote styles, \\t / \\0 / control chars, \\uXXXX, non-ASCII) is handled + by a tested library rather than hand-rolled quoting. + + This matters because the token is user-supplied via ``--extra-arg`` and + interpolated into a manifest that ``--deploy`` commits and ArgoCD applies: + the previous hand-rolled logic left backslashes unescaped in its + single-quote and no-quote branches, so a token ending in ``\\`` escaped the + closing quote (unterminated scalar -> broken manifest) and a token like + ``a\\tc`` was silently reinterpreted (``\\t`` -> TAB). ``json.dumps`` + (ensure_ascii=True) emits only escapes YAML also understands, so the round + trip is exact. Newlines are still rejected up front as an input-sanity guard + (a newline in a `vllm serve` arg is almost certainly a mistake).""" if "\n" in v or "\r" in v: sys.exit(f"error: --extra-arg value must not contain newlines: {v!r}") - if '"' in v and "'" not in v: - return f"'{v}'" - if "'" in v and '"' not in v: - return f'"{v}"' - if '"' not in v and "'" not in v: - return f'"{v}"' - # Both quote styles present — escape within a double-quoted scalar. - return '"' + v.replace("\\", "\\\\").replace('"', '\\"') + '"' + return json.dumps(v) def _expand_extra_args(raw: list) -> list[str]: diff --git a/platform/config/bedrockmodel-crd.yaml b/platform/config/bedrockmodel-crd.yaml new file mode 100644 index 0000000..e2e9e6f --- /dev/null +++ b/platform/config/bedrockmodel-crd.yaml @@ -0,0 +1,96 @@ +# BedrockModel — self-service enrollment of an Amazon Bedrock model. +# +# A Bedrock model has nothing to deploy or scale (no GPU, no pod, no serving +# Deployment), so — unlike the vLLM/llm-d serving tiers — this is NOT a KRO +# ResourceGraphDefinition (there is no resource graph to expand). It is a plain, +# infra-less marker CR: `litellm-sync` watches it and registers/deregisters the +# model in LiteLLM (POST /model/new) exactly like it does for the serving-tier +# CRs, but with Bedrock `litellm_params` (model: bedrock/) instead of an +# in-cluster openai/ api_base. Auth at invocation time is the `litellm` +# ServiceAccount's IRSA role — no keys in the CR. +# +# Because a CR backs the model, it is a first-class citizen of litellm-sync's +# register/finalizer/reconcile lifecycle: the reconcile sweep only deregisters +# DB models whose name has NO live CR, so a BedrockModel is protected the same +# way a VLLMEndpoint is — no special-casing needed. +# +# Authored by `./platformctl new-model --source bedrock ` (which resolves +# the region/partition-correct invocation id, the sovereign runtime endpoint for +# non-commercial partitions, and the per-token price from the AWS Price List +# API, then commits the CR under workloads/models/). Removal is `git rm` the +# file (or `./platformctl new-model --undeploy `). +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + name: bedrockmodels.bedrock.ai-platform + labels: + app.kubernetes.io/part-of: ai-platform +spec: + group: bedrock.ai-platform + scope: Namespaced + names: + plural: bedrockmodels + singular: bedrockmodel + kind: BedrockModel + shortNames: + - bedrockmodel + - brm + versions: + - name: v1alpha1 + served: true + storage: true + schema: + openAPIV3Schema: + type: object + properties: + spec: + type: object + required: + - model + properties: + model: + type: string + description: >- + Full LiteLLM Bedrock model string, including the provider + prefix and the region/partition-correct invocation id — a + cross-region system inference profile where one exists + (e.g. "bedrock/us.amazon.nova-lite-v1:0"), otherwise the + bare foundation-model id (e.g. "bedrock/amazon.nova-lite-v1:0" + in single-region / sovereign partitions). + modelName: + type: string + description: >- + LiteLLM alias callers use on the /v1 API. Defaults to + metadata.name when unset. + bedrockEndpoint: + type: string + description: >- + Optional Bedrock runtime endpoint URL. Set only for + non-commercial partitions (e.g. the AWS European Sovereign + Cloud's amazonaws.eu host); omit in the standard aws + partition so LiteLLM uses its default endpoint. + inputCostPerToken: + type: string + description: >- + Optional USD cost per input token, for accurate per-region + spend tracking (LiteLLM's built-in map is US-centric). + Populated from the AWS Price List API at enrollment when + available. + outputCostPerToken: + type: string + description: Optional USD cost per output token. See inputCostPerToken. + status: + type: object + x-kubernetes-preserve-unknown-fields: true + subresources: + status: {} + additionalPrinterColumns: + - name: Alias + type: string + jsonPath: .spec.modelName + - name: Model + type: string + jsonPath: .spec.model + - name: Age + type: date + jsonPath: .metadata.creationTimestamp diff --git a/platform/services/langfuse/helm-values.yaml b/platform/services/langfuse/helm-values.yaml index 14905d8..f789669 100644 --- a/platform/services/langfuse/helm-values.yaml +++ b/platform/services/langfuse/helm-values.yaml @@ -150,6 +150,14 @@ clickhouse: ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) TTL event_date + INTERVAL 3 DAY DELETE SETTINGS ttl_only_drop_parts=1 ENGINE = MergeTree PARTITION BY toYYYYMM(event_date) ORDER BY (event_date, event_time) TTL event_date + INTERVAL 3 DAY DELETE SETTINGS ttl_only_drop_parts=1 + # PVCs intentionally omit `storageClass` so they bind the cluster DEFAULT + # StorageClass, which Terraform marks per cluster mode: `gp3` (ebs.csi.aws.com) + # on classic/self-managed clusters incl. ESC (capabilities.blockstorage), and + # `auto-ebs-sc` (ebs.csi.eks.amazonaws.com) on EKS Auto Mode — see + # terraform/30.eks/30.cluster/main.tf. Do NOT pin `storageClass: gp3` here: + # that class does not exist in Auto Mode, so the PVC would stay Pending. + # (A non-Auto cluster with capabilities.blockstorage=false has no default SC at + # all — stateful components like Langfuse aren't supported in that config.) persistence: enabled: true size: 50Gi @@ -176,6 +184,9 @@ redis: existingSecret: langfuse-secrets existingSecretPasswordKey: redis-password primary: + # Binds the cluster default StorageClass (see the clickhouse.persistence note + # above) — no `storageClass` pinned, so it works on classic/self-managed (gp3) + # and Auto Mode (auto-ebs-sc) alike. persistence: enabled: true size: 8Gi diff --git a/platform/services/litellm-sync/README.md b/platform/services/litellm-sync/README.md index c12527a..c258cd8 100644 --- a/platform/services/litellm-sync/README.md +++ b/platform/services/litellm-sync/README.md @@ -1,14 +1,15 @@ # litellm-sync -The **single owner of LiteLLM's model registry**. Watches the three serving-tier -custom resources cluster-wide and registers each model with LiteLLM on create, -deregisters it on delete — so LiteLLM's `/v1/models` always matches what's deployed. +The **single owner of LiteLLM's model registry**. Watches the serving-tier custom +resources and `BedrockModel`s cluster-wide and registers each model with LiteLLM on +create, deregisters it on delete — so LiteLLM's `/v1/models` always matches what's deployed. ## Why this exists Models are declared as KRO custom resources — `VLLMEndpoint` (simple vLLM), `LLMDEndpoint` (llm-d scale tier), and `LLMDDisaggEndpoint` (llm-d prefill/decode -disaggregation). Something has to tell LiteLLM about them (`POST /model/new`) and +disaggregation) — plus `BedrockModel` (a managed Amazon Bedrock model; a plain +CRD, no KRO/infra). Something has to tell LiteLLM about them (`POST /model/new`) and remove them when they're deleted. Doing that with a per-CR registration Job (the previous design) has two problems: @@ -49,12 +50,13 @@ Registration is idempotent and self-healing: a model already live in `/v1/models is left alone; otherwise a stale DB entry is removed and the model re-added so it lands back in the running router. -### Static models are protected +### What the reconcile sweep can and can't delete Deregistration only ever touches **DB-registered** models (`model_info.db_model == -true`). Static config-file models declared in `litellm.yaml` — notably the Bedrock -`claude-opus-4-8` baseline — report `db_model: false` and are skipped. They can -never be deleted by this controller. +true`), and only when **no live CR** (a serving-tier CR or a `BedrockModel`) still +claims the name — so every CR-backed model is protected, including Bedrock models +enrolled via `platformctl new-model --source bedrock`. Any truly static entry in +`litellm.yaml` (none by default) reports `db_model: false` and is skipped outright. ## Operational notes diff --git a/platform/services/litellm-sync/deployment.yaml b/platform/services/litellm-sync/deployment.yaml index 5ec32c1..ca1365b 100644 --- a/platform/services/litellm-sync/deployment.yaml +++ b/platform/services/litellm-sync/deployment.yaml @@ -1,6 +1,6 @@ # litellm-sync controller. # -# Long-running pod that watches the three serving-tier CRs cluster-wide +# Long-running pod that watches the serving-tier CRs and BedrockModels cluster-wide # (VLLMEndpoint / LLMDEndpoint / LLMDDisaggEndpoint) and is the sole owner of # LiteLLM's model registry — registering a model on create and deregistering it # on delete via a finalizer (see scripts/litellm_sync.py). This keeps the LiteLLM diff --git a/platform/services/litellm-sync/rbac.yaml b/platform/services/litellm-sync/rbac.yaml index c165441..cbb44ec 100644 --- a/platform/services/litellm-sync/rbac.yaml +++ b/platform/services/litellm-sync/rbac.yaml @@ -23,6 +23,11 @@ rules: # watch/list to observe; get for re-reads; patch/update to add and remove # the finalizer on metadata. verbs: ["get", "list", "watch", "patch", "update"] + - apiGroups: ["bedrock.ai-platform"] + resources: ["bedrockmodels"] + # Bedrock models enrolled via `platformctl new-model --source bedrock` — same + # observe + finalizer-patch access as the serving tiers. + verbs: ["get", "list", "watch", "patch", "update"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding diff --git a/platform/services/litellm-sync/scripts/litellm_sync.py b/platform/services/litellm-sync/scripts/litellm_sync.py index b2af9f9..2eebd5b 100644 --- a/platform/services/litellm-sync/scripts/litellm_sync.py +++ b/platform/services/litellm-sync/scripts/litellm_sync.py @@ -1,15 +1,20 @@ #!/usr/bin/env python3 """litellm-sync — the single owner of LiteLLM's model registry. -Watches the three serving-tier custom resources cluster-wide and keeps LiteLLM's -model list in sync, so the master key never has to enter a workload namespace and -models can be deployed into any namespace (e.g. per-team `team-*` namespaces): +Watches the serving-tier custom resources AND BedrockModels cluster-wide and +keeps LiteLLM's model list in sync, so the master key never has to enter a +workload namespace and models can be deployed into any namespace (e.g. per-team +`team-*` namespaces): - Tiers (kro.run/v1alpha1), each in ANY namespace: + Serving tiers (kro.run/v1alpha1), each in ANY namespace: - vllmendpoints -> api_base http://-vllm..svc.cluster.local:8000/v1 - llmdendpoints -> api_base http://-epp..svc.cluster.local:80/v1 - llmddisaggendpoints -> api_base http://-epp..svc.cluster.local:80/v1 + Bedrock models (bedrock.ai-platform/v1alpha1), in ANY namespace: + - bedrockmodels -> native LiteLLM bedrock/ upstream (no api_base; + auth = the litellm pod's IRSA role) + Lifecycle (finalizer-driven, self-healing): 1. WATCH each kind cluster-wide. @@ -30,8 +35,10 @@ /model/info can only ever shrink the row count — so a model always converges to exactly one entry and can never accumulate duplicates. Deregistration deletes ALL rows for a name and only ever touches DB-registered models -(model_info.db_model == True) — static config models (e.g. the Bedrock -claude-opus-4-8 in litellm.yaml) have db_model == False and are never deleted. +(model_info.db_model == True); a model is swept only when NO live CR still claims +its name, so every CR-backed model — including Bedrock models enrolled via +`platformctl new-model --source bedrock` — is protected. Any static config-file +model in litellm.yaml (db_model == False) is excluded outright and never deleted. Single replica, no database. If killed mid-loop, the next start re-lists current state (watch is list-then-watch) and the reconcile loop repairs any drift. All @@ -72,19 +79,35 @@ HTTP_TIMEOUT_SEC = int(os.environ.get("HTTP_TIMEOUT_SEC", "15")) WATCH_TIMEOUT_SEC = int(os.environ.get("WATCH_TIMEOUT_SEC", "300")) -CR_GROUP = "kro.run" CR_VERSION = "v1alpha1" -# Serving tiers this controller owns, and how to build each one's LiteLLM -# api_base from the CR name + namespace. All are kro.run/v1alpha1, cluster-wide. +# Serving tiers (KRO, group kro.run): registered as an in-cluster, +# OpenAI-compatible upstream. Value = how to build each one's LiteLLM api_base +# from the CR name + namespace. All cluster-wide. # vLLM (simple): the model-server Service, port 8000 # llm-d / disagg: the llm-d Endpoint-Picker (EPP) Service, port 80 -KINDS = { +SERVING_GROUP = "kro.run" +SERVING_KINDS = { "vllmendpoints": "http://{name}-vllm.{ns}.svc.cluster.local:8000/v1", "llmdendpoints": "http://{name}-epp.{ns}.svc.cluster.local:80/v1", "llmddisaggendpoints": "http://{name}-epp.{ns}.svc.cluster.local:80/v1", } +# Bedrock models (plain CRD, group bedrock.ai-platform): registered as a native +# LiteLLM Bedrock upstream (model: bedrock/, NO api_base — auth at call time +# is the litellm pod's IRSA role). Authored by +# `platformctl new-model --source bedrock`. Same finalizer/register/reconcile +# lifecycle as the serving tiers; only the emitted litellm_params differ. +BEDROCK_GROUP = "bedrock.ai-platform" +BEDROCK_KINDS = {"bedrockmodels"} + +# Every watched CR kind -> its API group. One watch thread per kind. +KIND_GROUP = { + **{plural: SERVING_GROUP for plural in SERVING_KINDS}, + **{plural: BEDROCK_GROUP for plural in BEDROCK_KINDS}, +} +KINDS = list(KIND_GROUP) # stable iteration order for watch threads + reconcile + stop_event = threading.Event() # Serializes ALL LiteLLM registry mutations (register/deregister). The watch @@ -150,9 +173,12 @@ def list_db_model_ids() -> dict[str, list[str]] | None: return result -def register_model(name: str, model_id: str, api_base: str) -> bool: +def register_model(name: str, litellm_params: dict) -> bool: """Ensure `name` is registered exactly once in LiteLLM. Duplicate-safe. + `litellm_params` is the provider-specific block (an in-cluster openai/ + api_base + for the serving tiers, or a native bedrock/ upstream for BedrockModels). + Held under REGISTRY_LOCK so a concurrent watch + reconcile can't both add the same model. Crucially, this is *monotonic*: it only ADDS when there is no DB row for the name, and otherwise only REMOVES extra rows (keep the first, @@ -164,6 +190,10 @@ def register_model(name: str, model_id: str, api_base: str) -> bool: *grow* duplicates under startup churn. Because this version never adds when a row is present, repeated passes can only shrink the row count — so it always converges to exactly one, even if reads lag writes. + + NOTE: because it never re-adds an existing row, it does NOT push param changes + (e.g. a refreshed Bedrock price) onto an already-registered model — undeploy + + redeploy the CR to apply changed litellm_params. """ with REGISTRY_LOCK: db_models = list_db_model_ids() @@ -178,11 +208,11 @@ def register_model(name: str, model_id: str, api_base: str) -> bool: return True resp = _litellm_request("POST", "/model/new", { "model_name": name, - "litellm_params": {"model": f"openai/{model_id}", "api_base": api_base, "api_key": "no-key"}, + "litellm_params": litellm_params, }) if resp is None: return False - log.info("registered model %s -> %s", name, api_base) + log.info("registered model %s (%s)", name, litellm_params.get("model")) return True @@ -224,12 +254,56 @@ def _has_deletion_timestamp(obj: dict) -> bool: def _api_base(plural: str, name: str, ns: str) -> str: - return KINDS[plural].format(name=name, ns=ns) + return SERVING_KINDS[plural].format(name=name, ns=ns) + + +def _spec(obj: dict) -> dict: + return obj.get("spec") or {} -def _model_id(obj: dict) -> str: - # Upstream served-model-name — the serving tiers pass --served-model-name spec.model. - return (obj.get("spec") or {}).get("model", "") +def _model_alias(plural: str, obj: dict) -> str: + """The LiteLLM model_name (alias) for a CR. Serving tiers use the CR name; a + BedrockModel may override it via spec.modelName (default: the CR name). + This is the name registered in LiteLLM and tracked by the reconcile sweep.""" + name = (obj.get("metadata") or {}).get("name", "") + if plural in BEDROCK_KINDS: + return _spec(obj).get("modelName") or name + return name + + +def _litellm_params_for(plural: str, obj: dict) -> dict | None: + """Build the LiteLLM litellm_params for a CR, or None if it isn't ready to + register (no spec.model).""" + spec = _spec(obj) + model = spec.get("model") + if not model: + return None + if plural in BEDROCK_KINDS: + # Native Bedrock upstream: no api_base/api_key (auth = the litellm pod's + # IRSA role). spec.model already carries the "bedrock/" provider + # string with the region/partition-correct invocation id. + params: dict = {"model": model} + endpoint = spec.get("bedrockEndpoint") + if endpoint: + params["aws_bedrock_runtime_endpoint"] = endpoint + name = (obj.get("metadata") or {}).get("name", "") + for cr_key, ll_key in (("inputCostPerToken", "input_cost_per_token"), + ("outputCostPerToken", "output_cost_per_token")): + val = spec.get(cr_key) + if val in (None, ""): + continue + try: + params[ll_key] = float(val) + except (TypeError, ValueError): + log.warning("bedrockmodel %s: ignoring non-numeric %s=%r", name, cr_key, val) + return params + # Serving tiers: in-cluster OpenAI-compatible upstream. + meta = obj.get("metadata") or {} + return { + "model": f"openai/{model}", + "api_base": _api_base(plural, meta.get("name", ""), meta.get("namespace", "")), + "api_key": "no-key", + } def _patch_finalizers(custom: client.CustomObjectsApi, plural: str, ns: str, name: str, @@ -237,7 +311,7 @@ def _patch_finalizers(custom: client.CustomObjectsApi, plural: str, ns: str, nam patch = {"metadata": {"finalizers": finalizers}} try: custom.patch_namespaced_custom_object( - group=CR_GROUP, version=CR_VERSION, namespace=ns, + group=KIND_GROUP[plural], version=CR_VERSION, namespace=ns, plural=plural, name=name, body=patch, ) return True @@ -249,12 +323,13 @@ def _patch_finalizers(custom: client.CustomObjectsApi, plural: str, ns: str, nam def process(custom: client.CustomObjectsApi, plural: str, obj: dict) -> None: - """Route one serving-tier object to the right handler.""" + """Route one CR (serving tier or BedrockModel) to the right handler.""" meta = obj.get("metadata") or {} name = meta.get("name") ns = meta.get("namespace", "") if not name: return + alias = _model_alias(plural, obj) if _has_deletion_timestamp(obj): current = _finalizers(obj) @@ -262,7 +337,7 @@ def process(custom: client.CustomObjectsApi, plural: str, obj: dict) -> None: return # Deregister first; only drop the finalizer once LiteLLM confirms, else # retry on the next event / reconcile. - if not deregister_model(name): + if not deregister_model(alias): log.warning("keeping finalizer on %s/%s until deregistration succeeds", ns, name) return remaining = [f for f in current if f != FINALIZER] @@ -275,9 +350,9 @@ def process(custom: client.CustomObjectsApi, plural: str, obj: dict) -> None: if FINALIZER not in current: if _patch_finalizers(custom, plural, ns, name, current + [FINALIZER]): log.info("added finalizer to %s/%s", ns, name) - model_id = _model_id(obj) - if model_id: - register_model(name, model_id, _api_base(plural, name, ns)) + params = _litellm_params_for(plural, obj) + if params: + register_model(alias, params) # --------------------------------------------------------------------------- @@ -291,7 +366,7 @@ def watch_kind(plural: str) -> None: w = watch.Watch() stream = w.stream( custom.list_cluster_custom_object, - group=CR_GROUP, version=CR_VERSION, plural=plural, + group=KIND_GROUP[plural], version=CR_VERSION, plural=plural, timeout_seconds=WATCH_TIMEOUT_SEC, ) for event in stream: @@ -326,7 +401,7 @@ def reconcile_once(custom: client.CustomObjectsApi) -> None: for plural in KINDS: try: items = custom.list_cluster_custom_object( - group=CR_GROUP, version=CR_VERSION, plural=plural, + group=KIND_GROUP[plural], version=CR_VERSION, plural=plural, ).get("items", []) except ApiException as e: if e.status != 404: @@ -334,7 +409,7 @@ def reconcile_once(custom: client.CustomObjectsApi) -> None: continue any_kind_listed = True for obj in items: - live_names.add(obj["metadata"]["name"]) + live_names.add(_model_alias(plural, obj)) process(custom, plural, obj) # Sweep orphaned DB models only if we successfully listed at least one kind @@ -374,15 +449,23 @@ class Handler(http.server.BaseHTTPRequestHandler): def do_GET(self) -> None: # noqa: N802 try: client.CustomObjectsApi().list_cluster_custom_object( - group=CR_GROUP, version=CR_VERSION, plural="vllmendpoints", limit=1, + group=SERVING_GROUP, version=CR_VERSION, plural="vllmendpoints", limit=1, ) - self.send_response(200) - self.end_headers() - self.wfile.write(b"ok") + except ApiException: + # API server reachable, but the CRD may be absent (a Bedrock-only + # / kro=false install has no vllmendpoints CRD -> 404) or the list + # may be RBAC-scoped (403). Either way the API is up and the + # watch/reconcile loops handle missing CRDs by backoff, so we're + # healthy. Only a transport failure (below) is unhealthy. + pass except Exception as e: # noqa: BLE001 self.send_response(503) self.end_headers() self.wfile.write(str(e).encode()) + return + self.send_response(200) + self.end_headers() + self.wfile.write(b"ok") def log_message(self, *_: object) -> None: pass diff --git a/platform/services/litellm/litellm.yaml b/platform/services/litellm/litellm.yaml index 04fff08..5ead1e5 100644 --- a/platform/services/litellm/litellm.yaml +++ b/platform/services/litellm/litellm.yaml @@ -179,32 +179,26 @@ metadata: data: config.yaml: | # Two model sources merge at runtime: - # 1. Static models declared here (committed to git) — e.g. Bedrock, which - # has nothing to deploy/scale, so it needs no serving CR. - # 2. Self-hosted models registered dynamically via the LiteLLM API - # (POST /model/new) by the litellm-sync controller, which watches the - # serving-tier CRs (VLLMEndpoint/LLMDEndpoint/LLMDDisaggEndpoint) and + # 1. Static models declared here (committed to git). None by default — see + # the note on model_list below. + # 2. Models registered dynamically via the LiteLLM API (POST /model/new) by + # the litellm-sync controller, which watches the serving-tier CRs + # (VLLMEndpoint/LLMDEndpoint/LLMDDisaggEndpoint) AND BedrockModels and # registers/deregisters each model centrally. # LiteLLM merges this model_list with DB-registered models (STORE_MODEL_IN_DB=True). - model_list: - # Amazon Bedrock — Amazon Nova Lite. Static endpoint: zero GPUs, no serving - # CR. Auth comes from the `litellm` ServiceAccount's IRSA role (terraform: - # enable_bedrock=true) — no static keys. Uses the direct foundation-model ID - # (no global./eu./us. cross-region inference profile — those don't apply in - # the single-region ESC partition). Requires Bedrock model access enabled - # for this model in the ESC console (one-time toggle) — - # `./platformctl status --check` validates it. - - model_name: nova-lite - litellm_params: - model: bedrock/eu.amazon.nova-lite-v1:0 #ESC: bedrock/amazon.nova-lite-v1:0 - aws_region_name: os.environ/AWS_REGION - # Bedrock runtime endpoint, injected from the Terraform litellm-env - # ConfigMap (AWS_ENDPOINT_URL_BEDROCK_RUNTIME) so this file stays - # partition-agnostic. ESC needs the sovereign amazonaws.eu host (the - # default bedrock-runtime..amazonaws.com doesn't exist in the - # aws-eusc partition); commercial regions get the .com host. Terraform - # sets it for both — see terraform/30.eks/30.cluster/bedrock.tf. - aws_bedrock_runtime_endpoint: os.environ/AWS_ENDPOINT_URL_BEDROCK_RUNTIME + # + # No models are shipped by default — including Bedrock. Both self-hosted + # models AND Amazon Bedrock models are enrolled self-service and registered + # dynamically by litellm-sync (source #2 above): + # - self-hosted: ./platformctl new-model --deploy (VLLMEndpoint/…) + # - Bedrock: ./platformctl new-model --source bedrock --deploy + # (a BedrockModel CR; litellm-sync registers bedrock/ + # with the region/partition-correct invocation id + price) + # This keeps the committed config partition-agnostic: nothing here names a + # region- or partition-specific model id that would be wrong in another + # region or in the ESC partition. See platform/config/bedrockmodel-crd.yaml + # and workloads/models/README.md. + model_list: [] litellm_settings: # Register the Langfuse callback HERE — this is the only place LiteLLM # reads it. (There is no LITELLM_CALLBACKS env var; setting one is a no-op.) diff --git a/platformctl b/platformctl index e4c50b6..45f7e9f 100755 --- a/platformctl +++ b/platformctl @@ -340,7 +340,7 @@ cmd_status() { kubectl get pods -n "$NS_PLATFORM" 2>/dev/null || true echo log "Serving endpoints (all namespaces)" - kubectl get vllmendpoints,llmdendpoints,llmddisaggendpoints -A 2>/dev/null || echo " (none deployed)" + kubectl get vllmendpoints,llmdendpoints,llmddisaggendpoints,bedrockmodels -A 2>/dev/null || echo " (none deployed)" echo _status_access @@ -399,7 +399,7 @@ _status_access() { _status_check() { log "Validating the platform end-to-end (--check)" need aws - local mk code base="http://127.0.0.1:4000" pf_pid="" + local mk code base="http://127.0.0.1:4000" pf_pid="" failures=0 mk=$(kubectl get secret litellm-secrets -n "$NS_PLATFORM" -o jsonpath='{.data.master-key}' 2>/dev/null | base64 -d) \ || { warn "cannot read litellm master key — is the platform up?"; return 1; } kubectl port-forward -n "$NS_PLATFORM" svc/litellm 4000:4000 >/dev/null 2>&1 & @@ -417,6 +417,7 @@ _status_check() { fi else warn "LiteLLM gateway not responding" + failures=$((failures + 1)) fi if [ -n "$first_model" ]; then code=$(curl -s -o /dev/null -w '%{http_code}' -X POST "$base/v1/chat/completions" \ @@ -426,22 +427,31 @@ _status_check() { ok "Model answered ($first_model)" else warn "Model call ($first_model) returned HTTP $code — check model access (Bedrock: per-account console toggle) / config" + failures=$((failures + 1)) fi else - warn "No models registered in /v1/models yet — expected before you deploy one. Add a model with ./platformctl new-model (or configure a Bedrock model in litellm.yaml). Skipping the chat test." + warn "No models registered in /v1/models yet — expected before you enroll one (not a failure). Add a self-hosted model with ./platformctl new-model, or a Bedrock model with ./platformctl new-model --source bedrock. Skipping the chat test." fi if kubectl run pfcheck-langfuse --rm -i --restart=Never --image=curlimages/curl:8.11.0 -n "$NS_PLATFORM" --quiet -- \ -fsS http://langfuse-web.ai-platform.svc.cluster.local:3000/api/public/health >/dev/null 2>&1; then ok "Langfuse tracing reachable" else warn "Langfuse health check failed (tracing may be starting)" + failures=$((failures + 1)) fi [ -n "${pf_pid:-}" ] && kill "$pf_pid" 2>/dev/null || true - # --check reports health via ok/warn lines; a warning (e.g. no models yet, or - # Langfuse still starting) is not a failure, so always exit 0. Hard failures - # use die (non-zero) explicitly earlier. - return 0 + # Exit code is meaningful for CI/automation: 0 only when every real check + # passed. The legitimately-empty case (no models registered yet) is expected + # and is NOT counted as a failure. Hard failures — gateway unreachable, a + # registered model returning non-200, or Langfuse unhealthy — return non-zero + # so an automated smoke test can detect a broken platform. + if [ "$failures" -eq 0 ]; then + ok "Platform healthy" + return 0 + fi + warn "$failures check(s) failed — the platform is not fully healthy (see above)" + return 1 } # ---- tunnel ---------------------------------------------------------------- # diff --git a/terraform/00.global/vars/example-esc.tfvars b/terraform/00.global/vars/example-esc.tfvars index 34d6150..afaa62c 100644 --- a/terraform/00.global/vars/example-esc.tfvars +++ b/terraform/00.global/vars/example-esc.tfvars @@ -14,9 +14,10 @@ shared_config = { # private_eks_cluster = false (below): a plan-time check refuses to expose the # control plane to 0.0.0.0/0. Set this to the public egress IP/CIDR(s) you run # platformctl/kubectl from (office, VPN, CI): -# e.g. as given by https://www.whatismyip.com/ - -cluster_endpoint_public_access_cidrs = ["/32"] +# e.g. as given by https://www.whatismyip.com/ — then uncomment the line below. +# Left commented, the variable defaults to [] and the plan fails closed with a +# clear "not scoped" message — instead of a CIDR parse error on the placeholder. +# cluster_endpoint_public_access_cidrs = ["/32"] # e.g. ["203.0.113.10/32"] cluster_config = { kubernetes_version = "1.36" diff --git a/terraform/00.global/vars/example.tfvars b/terraform/00.global/vars/example.tfvars index 0039a37..6652dd3 100644 --- a/terraform/00.global/vars/example.tfvars +++ b/terraform/00.global/vars/example.tfvars @@ -18,8 +18,10 @@ shared_config = { # Operator CIDR allowlist for the EKS PUBLIC API endpoint. REQUIRED whenever # private_eks_cluster = false (below): a plan-time check refuses to expose the # control plane to 0.0.0.0/0. Set this to the public egress IP/CIDR(s) you run -# platformctl/kubectl from (office, VPN, CI). -cluster_endpoint_public_access_cidrs = ["/32"] +# platformctl/kubectl from (office, VPN, CI), then uncomment the line below. +# Left commented, the variable defaults to [] and the plan fails closed with a +# clear "not scoped" message — instead of a CIDR parse error on the placeholder. +# cluster_endpoint_public_access_cidrs = ["/32"] # e.g. ["203.0.113.10/32"] cluster_config = { kubernetes_version = "1.36" diff --git a/terraform/10.networking/.terraform.lock.hcl b/terraform/10.networking/.terraform.lock.hcl new file mode 100644 index 0000000..bd0454b --- /dev/null +++ b/terraform/10.networking/.terraform.lock.hcl @@ -0,0 +1,51 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.28.0" + constraints = ">= 5.20.0, ~> 6.28.0" + hashes = [ + "h1:2bDndcCvti7hgXw4MkMo37cyAAu1gk+JvsU9/UbRJNQ=", + "h1:RwoFuX1yGMVaKJaUmXDKklEaQ/yUCEdt5k2kz+/g08c=", + "h1:bMiTeecRXGBmx/btqJX57X0KuJ3j9BmM/ph3KZ3FGj0=", + "h1:wzZdGs0FFmNqIgPyo9tKnGKJ37BGNSgwRrEXayL29+0=", + "zh:0ba0d5eb6e0c6a933eb2befe3cdbf22b58fbc0337bf138f95bf0e8bb6e6df93e", + "zh:23eacdd4e6db32cf0ff2ce189461bdbb62e46513978d33c5de4decc4670870ec", + "zh:307b06a15fc00a8e6fd243abde2cbe5112e9d40371542665b91bec1018dd6e3c", + "zh:37a02d5b45a9d050b9642c9e2e268297254192280df72f6e46641daca52e40ec", + "zh:3da866639f07d92e734557d673092719c33ede80f4276c835bf7f231a669aa33", + "zh:480060b0ba310d0f6b6a14d60b276698cb103c48fd2f7e2802ae47c963995ec6", + "zh:57796453455c20db80d9168edbf125bf6180e1aae869de1546a2be58e4e405ec", + "zh:69139cba772d4df8de87598d8d8a2b1b4b254866db046c061dccc79edb14e6b9", + "zh:7312763259b859ff911c5452ca8bdf7d0be6231c5ea0de2df8f09d51770900ac", + "zh:8d2d6f4015d3c155d7eb53e36f019a729aefb46ebfe13f3a637327d3a1402ecc", + "zh:94ce589275c77308e6253f607de96919b840c2dd36c44aa798f693c9dd81af42", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:adaceec6a1bf4f5df1e12bd72cf52b72087c72efed078aef636f8988325b1a8b", + "zh:d37be1ce187d94fd9df7b13a717c219964cd835c946243f096c6b230cdfd7e92", + "zh:fe6205b5ca2ff36e68395cb8d3ae10a3728f405cdbcd46b206a515e1ebcf17a1", + ] +} + +provider "registry.terraform.io/hashicorp/random" { + version = "3.9.1" + hashes = [ + "h1:PYbnOqRuGn4c0/Ae1f7yOS/0zvmXNHFJRZjuF4KECnM=", + "h1:PlW+UZ4EElQF3NQwf41KQwavFujab3Czc51zu9dyVM8=", + "h1:g40qr7yDmIpaur4SsK5BcOda3HSo1RJ6zHVMqN4EJ+0=", + "h1:nozfr4CZq73d4HjubKJundX/8A+Mj282oS/hyNfjUPU=", + "zh:05f4734c1f0be840b711b3eff259ebc5fca436784c728955b1678078466f48d7", + "zh:0b91bf19371d012434eba1deeb6aab77158def9b39601dcbd94450b3974a2a26", + "zh:0ee6eacd47ec00183d55d726a4b6c4ce951a199f944bf22f1aa58392ebdfa7a2", + "zh:19388a4074b76a89a43a6c8328d7ae8ee2e7de3d346af51e80d3e6d3d12925f1", + "zh:23e74d48c5e2ac2e823fd527f49fee9db37d32a1990c9e3bf126ead697b843eb", + "zh:3cabf7fbd096c520064aae3aba61aba670af83ab91291a71fa1b1332929c2b7f", + "zh:5c0a3b8af0be60be4eca12ddee385cfa8babc1ec8e98cdf9de2f2274c73eabfa", + "zh:60b4f8a8ef18f52bf8e19215229dae408bee732825964092db7c989fd2de4097", + "zh:7359015acfedcbd6366f2329c854cf8d3c8ca5cd0faa89d2d37db358d6eba6c5", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:7b38758402f0e13a1071162da28994023cd2ac676e54af350c9ffd8dfa73fa7b", + "zh:7c7fbb8895eb75bb4de1f933e98553bd99c8d048c89a925ddba490aa5a67f7dc", + "zh:8c2b8c6a7ccdec16b73e2fb9f3700ea097f58c592571e4c5de60c93d2301732c", + ] +} diff --git a/terraform/20.iam-roles-for-eks/.terraform.lock.hcl b/terraform/20.iam-roles-for-eks/.terraform.lock.hcl new file mode 100644 index 0000000..2e3d328 --- /dev/null +++ b/terraform/20.iam-roles-for-eks/.terraform.lock.hcl @@ -0,0 +1,28 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.28.0" + constraints = "~> 6.28.0" + hashes = [ + "h1:2bDndcCvti7hgXw4MkMo37cyAAu1gk+JvsU9/UbRJNQ=", + "h1:RwoFuX1yGMVaKJaUmXDKklEaQ/yUCEdt5k2kz+/g08c=", + "h1:bMiTeecRXGBmx/btqJX57X0KuJ3j9BmM/ph3KZ3FGj0=", + "h1:wzZdGs0FFmNqIgPyo9tKnGKJ37BGNSgwRrEXayL29+0=", + "zh:0ba0d5eb6e0c6a933eb2befe3cdbf22b58fbc0337bf138f95bf0e8bb6e6df93e", + "zh:23eacdd4e6db32cf0ff2ce189461bdbb62e46513978d33c5de4decc4670870ec", + "zh:307b06a15fc00a8e6fd243abde2cbe5112e9d40371542665b91bec1018dd6e3c", + "zh:37a02d5b45a9d050b9642c9e2e268297254192280df72f6e46641daca52e40ec", + "zh:3da866639f07d92e734557d673092719c33ede80f4276c835bf7f231a669aa33", + "zh:480060b0ba310d0f6b6a14d60b276698cb103c48fd2f7e2802ae47c963995ec6", + "zh:57796453455c20db80d9168edbf125bf6180e1aae869de1546a2be58e4e405ec", + "zh:69139cba772d4df8de87598d8d8a2b1b4b254866db046c061dccc79edb14e6b9", + "zh:7312763259b859ff911c5452ca8bdf7d0be6231c5ea0de2df8f09d51770900ac", + "zh:8d2d6f4015d3c155d7eb53e36f019a729aefb46ebfe13f3a637327d3a1402ecc", + "zh:94ce589275c77308e6253f607de96919b840c2dd36c44aa798f693c9dd81af42", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:adaceec6a1bf4f5df1e12bd72cf52b72087c72efed078aef636f8988325b1a8b", + "zh:d37be1ce187d94fd9df7b13a717c219964cd835c946243f096c6b230cdfd7e92", + "zh:fe6205b5ca2ff36e68395cb8d3ae10a3728f405cdbcd46b206a515e1ebcf17a1", + ] +} diff --git a/terraform/30.eks/30.cluster/.terraform.lock.hcl b/terraform/30.eks/30.cluster/.terraform.lock.hcl new file mode 100644 index 0000000..4a25e21 --- /dev/null +++ b/terraform/30.eks/30.cluster/.terraform.lock.hcl @@ -0,0 +1,218 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/alekc/kubectl" { + version = "2.4.1" + constraints = "2.4.1" + hashes = [ + "h1:5ABzWzl+SvCYjhR6z/Pz+aZqTPhxLnmYa3NxRl3IVwM=", + "h1:8xq9mBqJQf1qNdwj7hmzdhqUTuvhkRBiUVsUHWADf28=", + "h1:oFhUB/LqtnraVc/XUSCSuy/4h/g6e3AtGkS6XADieaQ=", + "h1:pDnerh94NFiSv5z/Vuq+c1mEdn117P9KQI8zXeEEAEI=", + "zh:00a27c51a6f1082f20cbee52e0b996e4bed1dbc90de682cc309618aee2857e35", + "zh:16f94d63152a8a7c637fa3c609a104416272565b7a9b620e3b895ddb9912e288", + "zh:18a5c663c2aa6e673d75650d27888f1f91894f46d504fbf4a87f045c96f7afc1", + "zh:1a7c76db49ec94c0669baa6e416007f76acf09dbd141ef20036e4e13840651d7", + "zh:2baf0f93ff27fa0ff7265ee9eed5cc773311ace3bf936eb72bfbbd579d436894", + "zh:3ea7f773485b08ed55eedd29a7c83988c91ba2cc346b2436e7d1347ab061625a", + "zh:4bb3a241078f7b62dc41e9ac156c83b628933248b76cee3ec0d0264c0a2a25b7", + "zh:57571e1d9230fbde986a265228e17662eeb5ec63c12b84742564d453ad21f955", + "zh:6e1b82fde0037368a9e156304a580364ef926f26ffe235bb5a67b4c9f2edc2c4", + "zh:9dec7ed91b39d9815d9c37b78eca0327f631c66b2bcc61cbb4212c6d2ec76a22", + "zh:a347aebd398769c35ed2d3169025a7a11c1937d94efead7a7bee212aabba1c73", + "zh:b74e24ec0e2e176ee6cc18f33ac7e3609cfd6ca377b49cbd64b0f36edbb54342", + "zh:f6844cf7ead4443638dfdb3f837481585f4a4c910bcf7b030a19820cc5109ebc", + ] +} + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.28.0" + constraints = ">= 4.0.0, >= 6.0.0, >= 6.9.0, ~> 6.28.0" + hashes = [ + "h1:2bDndcCvti7hgXw4MkMo37cyAAu1gk+JvsU9/UbRJNQ=", + "h1:RwoFuX1yGMVaKJaUmXDKklEaQ/yUCEdt5k2kz+/g08c=", + "h1:bMiTeecRXGBmx/btqJX57X0KuJ3j9BmM/ph3KZ3FGj0=", + "h1:wzZdGs0FFmNqIgPyo9tKnGKJ37BGNSgwRrEXayL29+0=", + "zh:0ba0d5eb6e0c6a933eb2befe3cdbf22b58fbc0337bf138f95bf0e8bb6e6df93e", + "zh:23eacdd4e6db32cf0ff2ce189461bdbb62e46513978d33c5de4decc4670870ec", + "zh:307b06a15fc00a8e6fd243abde2cbe5112e9d40371542665b91bec1018dd6e3c", + "zh:37a02d5b45a9d050b9642c9e2e268297254192280df72f6e46641daca52e40ec", + "zh:3da866639f07d92e734557d673092719c33ede80f4276c835bf7f231a669aa33", + "zh:480060b0ba310d0f6b6a14d60b276698cb103c48fd2f7e2802ae47c963995ec6", + "zh:57796453455c20db80d9168edbf125bf6180e1aae869de1546a2be58e4e405ec", + "zh:69139cba772d4df8de87598d8d8a2b1b4b254866db046c061dccc79edb14e6b9", + "zh:7312763259b859ff911c5452ca8bdf7d0be6231c5ea0de2df8f09d51770900ac", + "zh:8d2d6f4015d3c155d7eb53e36f019a729aefb46ebfe13f3a637327d3a1402ecc", + "zh:94ce589275c77308e6253f607de96919b840c2dd36c44aa798f693c9dd81af42", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:adaceec6a1bf4f5df1e12bd72cf52b72087c72efed078aef636f8988325b1a8b", + "zh:d37be1ce187d94fd9df7b13a717c219964cd835c946243f096c6b230cdfd7e92", + "zh:fe6205b5ca2ff36e68395cb8d3ae10a3728f405cdbcd46b206a515e1ebcf17a1", + ] +} + +provider "registry.terraform.io/hashicorp/cloudinit" { + version = "2.4.1" + constraints = ">= 2.0.0" + hashes = [ + "h1:8ittBnx/AJ461WxB0hbsOBDeh3IL2nK7TqQdcoxGKmI=", + "h1:TKuE7Sq6387hl4eBo0/Gur73M9IP7OM0soSr6IqXWr0=", + "h1:dc0Fu1Hl/FpbjWufibEvUypXEP1A97ilVScmTuP0VXM=", + "h1:dszOEMe90K5HTVVHhYIqvNa8QMQFd7bCWjFQISQmdrQ=", + "zh:0290af752ce1617c9258e11ccac1787ff134c59f444197db759af65e6e4fed2e", + "zh:489e950720f129c51a3868575c8656e165cd58652c34cb5d2efd9f89057f46b4", + "zh:5688e28d20f3a4cfea7808d882824e75ebdd6b2cb305df126d13b04f7a874ba2", + "zh:59cf1a815c8f1316c7e05ffd34fd78a9bf6e1de4823f40b4951ef2e538b9d6d6", + "zh:637a27f88d25e549dd096936a4c1eac85cf2a6d1cc44cf592690b4158cd2f969", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:95c80caef31829666a71adc9deaf70010a1fa8a9bda8a742b469e218b49c6442", + "zh:a23fc664a6f53d432684b8b9745c209e0eb8df8ad8455d98180d97fe8626afbf", + "zh:b74fe251542d65137dbf0b4b8cc597fbb89f862776456179db05865c717646e6", + "zh:d15dfa0a50147cd34c6c8192990d3c880f38ab64db825da8a322301930aa10f9", + "zh:d1757d138f00e79b8ad3f53c07d9569de7b44b6343e40407faeaaf07b43382c3", + "zh:daac40bbcd0e3fc8fc6be6e57b746995ed40996d3bfd077db60f1ec32b7a205e", + "zh:f2dfc80edb573514ce98add5eb4c178fb5b041ba6a8e94d1098d28fa2b0988a5", + ] +} + +provider "registry.terraform.io/hashicorp/helm" { + version = "3.3.0" + constraints = "~> 3.0" + hashes = [ + "h1:FmfW45oPjF6qqO+24Qzh/fnOSkxINHIEVco0xsujENM=", + "h1:Jk/eQ65jQ7L+Db1yQqtvUFc+BPs1QwW8Qoil+TdwGdU=", + "h1:Qbt2+dtOJty3glPZI9jMCXq82GSAKBk/T2+wBMIdu0c=", + "h1:Y9Z+1+RqmCYRCqlncKEKPjYkRqlNIBiK3jdEIbUn1Fc=", + "zh:24a942352667cf49bed467c51a951f17a6c13e0cdf1c505b01b41f75a1a60d0f", + "zh:2c057eafd1e4d8b50acdc676f9a3096e38fb7d30c3fab3d64af82fa04a2f767e", + "zh:44252c60034ca163f2f68eaeece2583411bca25ac83bc58350e24d4f1c2d60a7", + "zh:4628da7c781eebbce1b13d12b498856815bde0e174d7a0d0b85a5d89a1d36d17", + "zh:59b3f4fd8e236c5733032c289a9636ac8bb6c92bc11b540d979b92d005f6cf98", + "zh:5d5f11201b986485ca1d33096f3545c6074b3b4a592864620a8df7f7c399b59f", + "zh:5e656afadb2db3f418eb254b2da02b00851eba8ec63196d1f22b2a2854fe9fb9", + "zh:77515e5576f4ea3d8d9e404237882de62518950ccd7b312371f5c03580f22fb0", + "zh:a4e601eddf9781b5f8678ee9c2b6fb0791fbe6ab85a53707efdf80f714a3011c", + "zh:eaa58edc8bb43aeb0fe6c0f99975cb24e2e9f5441e938a389fe848795a30b825", + "zh:ec97b17b82222d2008256f9b10b88f564d30f717a43438fe425a802ac0ccf2f3", + "zh:f18613d2aa3fd3945d394911e210af7bf58968f64631b6b2e229852b60a2803e", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + ] +} + +provider "registry.terraform.io/hashicorp/kubernetes" { + version = "2.38.0" + constraints = "~> 2.22" + hashes = [ + "h1:5CkveFo5ynsLdzKk+Kv+r7+U9rMrNjfZPT3a0N/fhgE=", + "h1:7nJdsd1RMPBtOjDXidB37+KSDN5VcOWkbkow69qJVGc=", + "h1:XCkL/mxjWTawg6gg+jlpCQhF/+SNRoCEZxbbkDTj42s=", + "h1:soK8Lt0SZ6dB+HsypFRDzuX/npqlMU6M0fvyaR1yW0k=", + "zh:0af928d776eb269b192dc0ea0f8a3f0f5ec117224cd644bdacdc682300f84ba0", + "zh:1be998e67206f7cfc4ffe77c01a09ac91ce725de0abaec9030b22c0a832af44f", + "zh:326803fe5946023687d603f6f1bab24de7af3d426b01d20e51d4e6fbe4e7ec1b", + "zh:4a99ec8d91193af961de1abb1f824be73df07489301d62e6141a656b3ebfff12", + "zh:5136e51765d6a0b9e4dbcc3b38821e9736bd2136cf15e9aac11668f22db117d2", + "zh:63fab47349852d7802fb032e4f2b6a101ee1ce34b62557a9ad0f0f0f5b6ecfdc", + "zh:924fb0257e2d03e03e2bfe9c7b99aa73c195b1f19412ca09960001bee3c50d15", + "zh:b63a0be5e233f8f6727c56bed3b61eb9456ca7a8bb29539fba0837f1badf1396", + "zh:d39861aa21077f1bc899bc53e7233262e530ba8a3a2d737449b100daeb303e4d", + "zh:de0805e10ebe4c83ce3b728a67f6b0f9d18be32b25146aa89116634df5145ad4", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + "zh:faf23e45f0090eef8ba28a8aac7ec5d4fdf11a36c40a8d286304567d71c1e7db", + ] +} + +provider "registry.terraform.io/hashicorp/null" { + version = "3.3.2" + constraints = ">= 3.0.0, ~> 3.0" + hashes = [ + "h1:/Wbz7DHFO92KAqyDBMxGA8nwOJYECM8iLGhR7+wVWhQ=", + "h1:IQ1qrkht1sC1nibUR+AJ3ulryyhVDHfCHZhoJi0sg2Y=", + "h1:SsePRc2TCwjh/+zp9OZRUchvaC96ik8pEIs2vxfW1nA=", + "h1:vpRtoWIdmRPuUP+2DgvgeWNkhA9HB7h6w4xVnaS2J/U=", + "zh:10ec43b8b7b18d5639238c7fb9e111f6a4b038523dd66c7a426bf27b25fa4c08", + "zh:60beb9cc2ad5b871c710860cee75b42850cc6acd43db0d77cb5e00fda7288b55", + "zh:62538582d0a4a2f10ad8a8d9a6c3cd3f05af6c6d91c6641ffc78d4f0e8e69b27", + "zh:64a8f9ce7852d9efc5b464c12306c946366d59f5e2757def97969c9fd64bd1d6", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:92a374fb736a52f465283326d0a5bf4f495132eb99be209dfb4c75ec803fe8db", + "zh:98da9c42785d27a50f0604758bcb61a30f6278b9f2acd92bb3b2046e0e71916c", + "zh:b0f7896fae554729cdf4a24ac06359a050cff5817e6cd8597cba8a4ae01a7409", + "zh:bc8179ee35d67c72fb03012e7023b9f9816f033a7ec4109c001dd6d29752e812", + "zh:d23a598f713bfb6098bc003571d7de90b5a33b78f9be240488252fe5f3c2a60d", + "zh:d2855b922ea345dbd89ea287e4c6c4757e38bc0aaffeb2b79aa0b8004f9c53ff", + "zh:d3a60422bc6a2f9244d076c5222c07060c826ef91bdbaf4634cb752b86057473", + "zh:faa01928c25d2a6ecd9c7eb8b88134cb08de55a6b11ca6c703ac0092845344ba", + ] +} + +provider "registry.terraform.io/hashicorp/random" { + version = "3.9.1" + hashes = [ + "h1:PYbnOqRuGn4c0/Ae1f7yOS/0zvmXNHFJRZjuF4KECnM=", + "h1:PlW+UZ4EElQF3NQwf41KQwavFujab3Czc51zu9dyVM8=", + "h1:g40qr7yDmIpaur4SsK5BcOda3HSo1RJ6zHVMqN4EJ+0=", + "h1:nozfr4CZq73d4HjubKJundX/8A+Mj282oS/hyNfjUPU=", + "zh:05f4734c1f0be840b711b3eff259ebc5fca436784c728955b1678078466f48d7", + "zh:0b91bf19371d012434eba1deeb6aab77158def9b39601dcbd94450b3974a2a26", + "zh:0ee6eacd47ec00183d55d726a4b6c4ce951a199f944bf22f1aa58392ebdfa7a2", + "zh:19388a4074b76a89a43a6c8328d7ae8ee2e7de3d346af51e80d3e6d3d12925f1", + "zh:23e74d48c5e2ac2e823fd527f49fee9db37d32a1990c9e3bf126ead697b843eb", + "zh:3cabf7fbd096c520064aae3aba61aba670af83ab91291a71fa1b1332929c2b7f", + "zh:5c0a3b8af0be60be4eca12ddee385cfa8babc1ec8e98cdf9de2f2274c73eabfa", + "zh:60b4f8a8ef18f52bf8e19215229dae408bee732825964092db7c989fd2de4097", + "zh:7359015acfedcbd6366f2329c854cf8d3c8ca5cd0faa89d2d37db358d6eba6c5", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:7b38758402f0e13a1071162da28994023cd2ac676e54af350c9ffd8dfa73fa7b", + "zh:7c7fbb8895eb75bb4de1f933e98553bd99c8d048c89a925ddba490aa5a67f7dc", + "zh:8c2b8c6a7ccdec16b73e2fb9f3700ea097f58c592571e4c5de60c93d2301732c", + ] +} + +provider "registry.terraform.io/hashicorp/time" { + version = "0.14.2" + constraints = ">= 0.9.0, ~> 0.9" + hashes = [ + "h1:eQRXh8mZFlUJfzYXKdaYRHRMhiS2cFyCfgP1mjkrtuI=", + "h1:gnP2hptiFIHSHUFBvAFKhE/Yh5u5yVEx+P7XSB58A/E=", + "h1:mwxQFMxGnM+Xm3F76yeHSAfPr3RZq/+XGT1vjgwiRWQ=", + "h1:pmcYDG9dBx6l+2YQFP59egVwno/yf0l1QS56gl2PzLY=", + "zh:0aa1028d91041f4dceba193e3707dac57358d0063d97e20700e554758b67baca", + "zh:32bee9f2b2678e2a0789ad86e716d09ca1d5450180b3cd8033ee7a251bfd352e", + "zh:3aded9ef4dc6f4aec202a50c68a08b40013d325f9947f10168ebc8bee54109fc", + "zh:4d924637f3115ffa4ffc7f16d3f366bc472594f7447d400adb9def7ac92e3fc8", + "zh:5c35008e1363deafaa440ab43519409866dd7ec72aabeb2317bc16cd82756784", + "zh:6b30d97c9827501d7010fe49c8889d7a6cc8b45b77cdb1668c65af7f28a27d73", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:851e0f0e9c4de395e42220de51e7fb20d0e845e629f3cf37056da19e597304e7", + "zh:85622b4779b3ba7424780f7f8efcfa585227e245201cdb68303f2118b388e971", + "zh:9039153e3d45147183804188a1ce36c3811db9a4dca80f36fa382408d5f50b72", + "zh:a3d385413dc258a53fe8f4ded5d1fcd1eba715c65a015d74e70911bab2919207", + "zh:b9310a2327f7c8ad2aa3266f89c4d026b4bf18b09f7c4257c0ecd71a32f32db3", + "zh:cee7f2143da0c494115da94984bf3630e3e5855e27dae720a7bc29bb6acd9be2", + ] +} + +provider "registry.terraform.io/hashicorp/tls" { + version = "4.4.1" + constraints = ">= 4.0.0" + hashes = [ + "h1:1Zk4TqZwWmUZHfGTtM/vw8EKoWecYboMRqwhNaoTopc=", + "h1:WOqocV4RLbVA6NX+7Pw3Y4FShzAKSBtEQHHMs+0JSuQ=", + "h1:jnHf8pgCbG6kX0CVzpZ78XW0sE9V4+5LjlldrlONqOE=", + "h1:x0sq38+DvTpCH55KdhERVDgNjHL036y4i+DITul45TA=", + "zh:0855f9a980426a0af2c4415fb23cccbbb88c2d1dd40d83da6f45622c075b398b", + "zh:189227fdae28e547b1d988f5d83c4280fce02e3152d8b62aa810047e2f9f7e59", + "zh:2557db4f493aa90b3683905d9eeb103e55a3facc4ccda763d6de6025116ae121", + "zh:2cfd8d6a543ac61c6e7c0461839ab89bb8da1ba6c8090ab946b1cf47f100d7fa", + "zh:5c04cf2a683ea60a0072c59bbcf415f232c382c891b3aaeb9bbbb28213e0d878", + "zh:6681e35d50328bc11a49bb61c002121353f6a844cc6d63a0c3db57745ec38a56", + "zh:693b810e438bb1dfbc13a42629597284ac1eb08eb30f46f18738d7a9bb768d34", + "zh:77c4425f75346b23817837f3947b6ab7e765308abbf573fec5d2cafd4b82f142", + "zh:b3fd64ec1cecea98c88d2332dbeac40498ce2506e3235bdbd1afdfbaa47bb758", + "zh:c06224b5797b129a84e2514be43f2b140622928fe9c92b55d7941be3296ea289", + "zh:debc830ec123f27944c6170d76cb4ea3542845896960bb9fc530702774df328e", + "zh:f300fcc909014550fd74b2b751665dcc049f874e7a475903f65671baf2c2d86d", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + ] +} diff --git a/terraform/30.eks/30.cluster/karpenter.tf b/terraform/30.eks/30.cluster/karpenter.tf index 420f8cf..e89d107 100644 --- a/terraform/30.eks/30.cluster/karpenter.tf +++ b/terraform/30.eks/30.cluster/karpenter.tf @@ -46,6 +46,23 @@ resource "aws_iam_role" "karpenter_node" { tags = local.tags } +# POLICY PARITY (commercial + ESC) — because the node role is created here rather +# than by the karpenter module (create_node_iam_role = false, for the ESC trust +# principal), this attachment set MUST remain a SUPERSET of what the module +# attaches to its own node role, or Karpenter-launched nodes on the commercial +# (majority) path could fail to join or lose SSM/ECR access. As of +# terraform-aws-modules/eks//modules/karpenter v21.1.5 the module's node-role base +# set is: AmazonEKSWorkerNodePolicy, AmazonEKS_CNI_Policy (ipv4), +# AmazonEC2ContainerRegistryReadOnly. Below we attach exactly those three PLUS +# AmazonSSMManagedInstanceCore (node SSM access) and, when the ECR pull-through +# cache is enabled, the import policy — so this is a strict superset. +# No instance profile is needed here: the module's create_instance_profile +# defaults false (we don't enable it), so Karpenter 1.x creates the instance +# profile at runtime from the EC2NodeClass spec.role (= this role's name, wired +# below and at the karpenter_manifests `role` var); the module's controller policy +# scopes iam:PassRole and the EKS access entry to node_iam_role_arn (this role). +# RE-VERIFY the base set above whenever the karpenter module version is bumped, so +# a module change can't silently drop a node policy this hand-rolled role omits. resource "aws_iam_role_policy_attachment" "karpenter_node" { for_each = local.capabilities.autoscaling ? merge( { diff --git a/terraform/30.eks/35.addons/.terraform.lock.hcl b/terraform/30.eks/35.addons/.terraform.lock.hcl new file mode 100644 index 0000000..62ede9e --- /dev/null +++ b/terraform/30.eks/35.addons/.terraform.lock.hcl @@ -0,0 +1,171 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/alekc/kubectl" { + version = "2.4.1" + constraints = "2.4.1" + hashes = [ + "h1:5ABzWzl+SvCYjhR6z/Pz+aZqTPhxLnmYa3NxRl3IVwM=", + "h1:8xq9mBqJQf1qNdwj7hmzdhqUTuvhkRBiUVsUHWADf28=", + "h1:oFhUB/LqtnraVc/XUSCSuy/4h/g6e3AtGkS6XADieaQ=", + "h1:pDnerh94NFiSv5z/Vuq+c1mEdn117P9KQI8zXeEEAEI=", + "zh:00a27c51a6f1082f20cbee52e0b996e4bed1dbc90de682cc309618aee2857e35", + "zh:16f94d63152a8a7c637fa3c609a104416272565b7a9b620e3b895ddb9912e288", + "zh:18a5c663c2aa6e673d75650d27888f1f91894f46d504fbf4a87f045c96f7afc1", + "zh:1a7c76db49ec94c0669baa6e416007f76acf09dbd141ef20036e4e13840651d7", + "zh:2baf0f93ff27fa0ff7265ee9eed5cc773311ace3bf936eb72bfbbd579d436894", + "zh:3ea7f773485b08ed55eedd29a7c83988c91ba2cc346b2436e7d1347ab061625a", + "zh:4bb3a241078f7b62dc41e9ac156c83b628933248b76cee3ec0d0264c0a2a25b7", + "zh:57571e1d9230fbde986a265228e17662eeb5ec63c12b84742564d453ad21f955", + "zh:6e1b82fde0037368a9e156304a580364ef926f26ffe235bb5a67b4c9f2edc2c4", + "zh:9dec7ed91b39d9815d9c37b78eca0327f631c66b2bcc61cbb4212c6d2ec76a22", + "zh:a347aebd398769c35ed2d3169025a7a11c1937d94efead7a7bee212aabba1c73", + "zh:b74e24ec0e2e176ee6cc18f33ac7e3609cfd6ca377b49cbd64b0f36edbb54342", + "zh:f6844cf7ead4443638dfdb3f837481585f4a4c910bcf7b030a19820cc5109ebc", + ] +} + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.28.0" + constraints = ">= 4.36.0, >= 4.47.0, >= 5.0.0, ~> 6.28.0" + hashes = [ + "h1:2bDndcCvti7hgXw4MkMo37cyAAu1gk+JvsU9/UbRJNQ=", + "h1:RwoFuX1yGMVaKJaUmXDKklEaQ/yUCEdt5k2kz+/g08c=", + "h1:bMiTeecRXGBmx/btqJX57X0KuJ3j9BmM/ph3KZ3FGj0=", + "h1:wzZdGs0FFmNqIgPyo9tKnGKJ37BGNSgwRrEXayL29+0=", + "zh:0ba0d5eb6e0c6a933eb2befe3cdbf22b58fbc0337bf138f95bf0e8bb6e6df93e", + "zh:23eacdd4e6db32cf0ff2ce189461bdbb62e46513978d33c5de4decc4670870ec", + "zh:307b06a15fc00a8e6fd243abde2cbe5112e9d40371542665b91bec1018dd6e3c", + "zh:37a02d5b45a9d050b9642c9e2e268297254192280df72f6e46641daca52e40ec", + "zh:3da866639f07d92e734557d673092719c33ede80f4276c835bf7f231a669aa33", + "zh:480060b0ba310d0f6b6a14d60b276698cb103c48fd2f7e2802ae47c963995ec6", + "zh:57796453455c20db80d9168edbf125bf6180e1aae869de1546a2be58e4e405ec", + "zh:69139cba772d4df8de87598d8d8a2b1b4b254866db046c061dccc79edb14e6b9", + "zh:7312763259b859ff911c5452ca8bdf7d0be6231c5ea0de2df8f09d51770900ac", + "zh:8d2d6f4015d3c155d7eb53e36f019a729aefb46ebfe13f3a637327d3a1402ecc", + "zh:94ce589275c77308e6253f607de96919b840c2dd36c44aa798f693c9dd81af42", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:adaceec6a1bf4f5df1e12bd72cf52b72087c72efed078aef636f8988325b1a8b", + "zh:d37be1ce187d94fd9df7b13a717c219964cd835c946243f096c6b230cdfd7e92", + "zh:fe6205b5ca2ff36e68395cb8d3ae10a3728f405cdbcd46b206a515e1ebcf17a1", + ] +} + +provider "registry.terraform.io/hashicorp/helm" { + version = "3.3.0" + constraints = ">= 3.0.0, ~> 3.0" + hashes = [ + "h1:FmfW45oPjF6qqO+24Qzh/fnOSkxINHIEVco0xsujENM=", + "h1:Jk/eQ65jQ7L+Db1yQqtvUFc+BPs1QwW8Qoil+TdwGdU=", + "h1:Qbt2+dtOJty3glPZI9jMCXq82GSAKBk/T2+wBMIdu0c=", + "h1:Y9Z+1+RqmCYRCqlncKEKPjYkRqlNIBiK3jdEIbUn1Fc=", + "zh:24a942352667cf49bed467c51a951f17a6c13e0cdf1c505b01b41f75a1a60d0f", + "zh:2c057eafd1e4d8b50acdc676f9a3096e38fb7d30c3fab3d64af82fa04a2f767e", + "zh:44252c60034ca163f2f68eaeece2583411bca25ac83bc58350e24d4f1c2d60a7", + "zh:4628da7c781eebbce1b13d12b498856815bde0e174d7a0d0b85a5d89a1d36d17", + "zh:59b3f4fd8e236c5733032c289a9636ac8bb6c92bc11b540d979b92d005f6cf98", + "zh:5d5f11201b986485ca1d33096f3545c6074b3b4a592864620a8df7f7c399b59f", + "zh:5e656afadb2db3f418eb254b2da02b00851eba8ec63196d1f22b2a2854fe9fb9", + "zh:77515e5576f4ea3d8d9e404237882de62518950ccd7b312371f5c03580f22fb0", + "zh:a4e601eddf9781b5f8678ee9c2b6fb0791fbe6ab85a53707efdf80f714a3011c", + "zh:eaa58edc8bb43aeb0fe6c0f99975cb24e2e9f5441e938a389fe848795a30b825", + "zh:ec97b17b82222d2008256f9b10b88f564d30f717a43438fe425a802ac0ccf2f3", + "zh:f18613d2aa3fd3945d394911e210af7bf58968f64631b6b2e229852b60a2803e", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + ] +} + +provider "registry.terraform.io/hashicorp/kubernetes" { + version = "2.38.0" + constraints = ">= 2.20.0, ~> 2.22" + hashes = [ + "h1:5CkveFo5ynsLdzKk+Kv+r7+U9rMrNjfZPT3a0N/fhgE=", + "h1:7nJdsd1RMPBtOjDXidB37+KSDN5VcOWkbkow69qJVGc=", + "h1:XCkL/mxjWTawg6gg+jlpCQhF/+SNRoCEZxbbkDTj42s=", + "h1:soK8Lt0SZ6dB+HsypFRDzuX/npqlMU6M0fvyaR1yW0k=", + "zh:0af928d776eb269b192dc0ea0f8a3f0f5ec117224cd644bdacdc682300f84ba0", + "zh:1be998e67206f7cfc4ffe77c01a09ac91ce725de0abaec9030b22c0a832af44f", + "zh:326803fe5946023687d603f6f1bab24de7af3d426b01d20e51d4e6fbe4e7ec1b", + "zh:4a99ec8d91193af961de1abb1f824be73df07489301d62e6141a656b3ebfff12", + "zh:5136e51765d6a0b9e4dbcc3b38821e9736bd2136cf15e9aac11668f22db117d2", + "zh:63fab47349852d7802fb032e4f2b6a101ee1ce34b62557a9ad0f0f0f5b6ecfdc", + "zh:924fb0257e2d03e03e2bfe9c7b99aa73c195b1f19412ca09960001bee3c50d15", + "zh:b63a0be5e233f8f6727c56bed3b61eb9456ca7a8bb29539fba0837f1badf1396", + "zh:d39861aa21077f1bc899bc53e7233262e530ba8a3a2d737449b100daeb303e4d", + "zh:de0805e10ebe4c83ce3b728a67f6b0f9d18be32b25146aa89116634df5145ad4", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + "zh:faf23e45f0090eef8ba28a8aac7ec5d4fdf11a36c40a8d286304567d71c1e7db", + ] +} + +provider "registry.terraform.io/hashicorp/null" { + version = "3.3.2" + constraints = "~> 3.0" + hashes = [ + "h1:/Wbz7DHFO92KAqyDBMxGA8nwOJYECM8iLGhR7+wVWhQ=", + "h1:IQ1qrkht1sC1nibUR+AJ3ulryyhVDHfCHZhoJi0sg2Y=", + "h1:SsePRc2TCwjh/+zp9OZRUchvaC96ik8pEIs2vxfW1nA=", + "h1:vpRtoWIdmRPuUP+2DgvgeWNkhA9HB7h6w4xVnaS2J/U=", + "zh:10ec43b8b7b18d5639238c7fb9e111f6a4b038523dd66c7a426bf27b25fa4c08", + "zh:60beb9cc2ad5b871c710860cee75b42850cc6acd43db0d77cb5e00fda7288b55", + "zh:62538582d0a4a2f10ad8a8d9a6c3cd3f05af6c6d91c6641ffc78d4f0e8e69b27", + "zh:64a8f9ce7852d9efc5b464c12306c946366d59f5e2757def97969c9fd64bd1d6", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:92a374fb736a52f465283326d0a5bf4f495132eb99be209dfb4c75ec803fe8db", + "zh:98da9c42785d27a50f0604758bcb61a30f6278b9f2acd92bb3b2046e0e71916c", + "zh:b0f7896fae554729cdf4a24ac06359a050cff5817e6cd8597cba8a4ae01a7409", + "zh:bc8179ee35d67c72fb03012e7023b9f9816f033a7ec4109c001dd6d29752e812", + "zh:d23a598f713bfb6098bc003571d7de90b5a33b78f9be240488252fe5f3c2a60d", + "zh:d2855b922ea345dbd89ea287e4c6c4757e38bc0aaffeb2b79aa0b8004f9c53ff", + "zh:d3a60422bc6a2f9244d076c5222c07060c826ef91bdbaf4634cb752b86057473", + "zh:faa01928c25d2a6ecd9c7eb8b88134cb08de55a6b11ca6c703ac0092845344ba", + ] +} + +provider "registry.terraform.io/hashicorp/random" { + version = "3.9.1" + constraints = ">= 3.6.0" + hashes = [ + "h1:PYbnOqRuGn4c0/Ae1f7yOS/0zvmXNHFJRZjuF4KECnM=", + "h1:PlW+UZ4EElQF3NQwf41KQwavFujab3Czc51zu9dyVM8=", + "h1:g40qr7yDmIpaur4SsK5BcOda3HSo1RJ6zHVMqN4EJ+0=", + "h1:nozfr4CZq73d4HjubKJundX/8A+Mj282oS/hyNfjUPU=", + "zh:05f4734c1f0be840b711b3eff259ebc5fca436784c728955b1678078466f48d7", + "zh:0b91bf19371d012434eba1deeb6aab77158def9b39601dcbd94450b3974a2a26", + "zh:0ee6eacd47ec00183d55d726a4b6c4ce951a199f944bf22f1aa58392ebdfa7a2", + "zh:19388a4074b76a89a43a6c8328d7ae8ee2e7de3d346af51e80d3e6d3d12925f1", + "zh:23e74d48c5e2ac2e823fd527f49fee9db37d32a1990c9e3bf126ead697b843eb", + "zh:3cabf7fbd096c520064aae3aba61aba670af83ab91291a71fa1b1332929c2b7f", + "zh:5c0a3b8af0be60be4eca12ddee385cfa8babc1ec8e98cdf9de2f2274c73eabfa", + "zh:60b4f8a8ef18f52bf8e19215229dae408bee732825964092db7c989fd2de4097", + "zh:7359015acfedcbd6366f2329c854cf8d3c8ca5cd0faa89d2d37db358d6eba6c5", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:7b38758402f0e13a1071162da28994023cd2ac676e54af350c9ffd8dfa73fa7b", + "zh:7c7fbb8895eb75bb4de1f933e98553bd99c8d048c89a925ddba490aa5a67f7dc", + "zh:8c2b8c6a7ccdec16b73e2fb9f3700ea097f58c592571e4c5de60c93d2301732c", + ] +} + +provider "registry.terraform.io/hashicorp/time" { + version = "0.14.2" + constraints = ">= 0.9.0" + hashes = [ + "h1:eQRXh8mZFlUJfzYXKdaYRHRMhiS2cFyCfgP1mjkrtuI=", + "h1:gnP2hptiFIHSHUFBvAFKhE/Yh5u5yVEx+P7XSB58A/E=", + "h1:mwxQFMxGnM+Xm3F76yeHSAfPr3RZq/+XGT1vjgwiRWQ=", + "h1:pmcYDG9dBx6l+2YQFP59egVwno/yf0l1QS56gl2PzLY=", + "zh:0aa1028d91041f4dceba193e3707dac57358d0063d97e20700e554758b67baca", + "zh:32bee9f2b2678e2a0789ad86e716d09ca1d5450180b3cd8033ee7a251bfd352e", + "zh:3aded9ef4dc6f4aec202a50c68a08b40013d325f9947f10168ebc8bee54109fc", + "zh:4d924637f3115ffa4ffc7f16d3f366bc472594f7447d400adb9def7ac92e3fc8", + "zh:5c35008e1363deafaa440ab43519409866dd7ec72aabeb2317bc16cd82756784", + "zh:6b30d97c9827501d7010fe49c8889d7a6cc8b45b77cdb1668c65af7f28a27d73", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:851e0f0e9c4de395e42220de51e7fb20d0e845e629f3cf37056da19e597304e7", + "zh:85622b4779b3ba7424780f7f8efcfa585227e245201cdb68303f2118b388e971", + "zh:9039153e3d45147183804188a1ce36c3811db9a4dca80f36fa382408d5f50b72", + "zh:a3d385413dc258a53fe8f4ded5d1fcd1eba715c65a015d74e70911bab2919207", + "zh:b9310a2327f7c8ad2aa3266f89c4d026b4bf18b09f7c4257c0ecd71a32f32db3", + "zh:cee7f2143da0c494115da94984bf3630e3e5855e27dae720a7bc29bb6acd9be2", + ] +} diff --git a/terraform/40.observability/40.aws-native-observability/.terraform.lock.hcl b/terraform/40.observability/40.aws-native-observability/.terraform.lock.hcl new file mode 100644 index 0000000..ce2b379 --- /dev/null +++ b/terraform/40.observability/40.aws-native-observability/.terraform.lock.hcl @@ -0,0 +1,171 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/alekc/kubectl" { + version = "2.4.1" + constraints = "2.4.1" + hashes = [ + "h1:5ABzWzl+SvCYjhR6z/Pz+aZqTPhxLnmYa3NxRl3IVwM=", + "h1:8xq9mBqJQf1qNdwj7hmzdhqUTuvhkRBiUVsUHWADf28=", + "h1:oFhUB/LqtnraVc/XUSCSuy/4h/g6e3AtGkS6XADieaQ=", + "h1:pDnerh94NFiSv5z/Vuq+c1mEdn117P9KQI8zXeEEAEI=", + "zh:00a27c51a6f1082f20cbee52e0b996e4bed1dbc90de682cc309618aee2857e35", + "zh:16f94d63152a8a7c637fa3c609a104416272565b7a9b620e3b895ddb9912e288", + "zh:18a5c663c2aa6e673d75650d27888f1f91894f46d504fbf4a87f045c96f7afc1", + "zh:1a7c76db49ec94c0669baa6e416007f76acf09dbd141ef20036e4e13840651d7", + "zh:2baf0f93ff27fa0ff7265ee9eed5cc773311ace3bf936eb72bfbbd579d436894", + "zh:3ea7f773485b08ed55eedd29a7c83988c91ba2cc346b2436e7d1347ab061625a", + "zh:4bb3a241078f7b62dc41e9ac156c83b628933248b76cee3ec0d0264c0a2a25b7", + "zh:57571e1d9230fbde986a265228e17662eeb5ec63c12b84742564d453ad21f955", + "zh:6e1b82fde0037368a9e156304a580364ef926f26ffe235bb5a67b4c9f2edc2c4", + "zh:9dec7ed91b39d9815d9c37b78eca0327f631c66b2bcc61cbb4212c6d2ec76a22", + "zh:a347aebd398769c35ed2d3169025a7a11c1937d94efead7a7bee212aabba1c73", + "zh:b74e24ec0e2e176ee6cc18f33ac7e3609cfd6ca377b49cbd64b0f36edbb54342", + "zh:f6844cf7ead4443638dfdb3f837481585f4a4c910bcf7b030a19820cc5109ebc", + ] +} + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.28.0" + constraints = ">= 4.0.0, >= 4.36.0, >= 4.47.0, >= 5.0.0, ~> 6.28.0" + hashes = [ + "h1:2bDndcCvti7hgXw4MkMo37cyAAu1gk+JvsU9/UbRJNQ=", + "h1:RwoFuX1yGMVaKJaUmXDKklEaQ/yUCEdt5k2kz+/g08c=", + "h1:bMiTeecRXGBmx/btqJX57X0KuJ3j9BmM/ph3KZ3FGj0=", + "h1:wzZdGs0FFmNqIgPyo9tKnGKJ37BGNSgwRrEXayL29+0=", + "zh:0ba0d5eb6e0c6a933eb2befe3cdbf22b58fbc0337bf138f95bf0e8bb6e6df93e", + "zh:23eacdd4e6db32cf0ff2ce189461bdbb62e46513978d33c5de4decc4670870ec", + "zh:307b06a15fc00a8e6fd243abde2cbe5112e9d40371542665b91bec1018dd6e3c", + "zh:37a02d5b45a9d050b9642c9e2e268297254192280df72f6e46641daca52e40ec", + "zh:3da866639f07d92e734557d673092719c33ede80f4276c835bf7f231a669aa33", + "zh:480060b0ba310d0f6b6a14d60b276698cb103c48fd2f7e2802ae47c963995ec6", + "zh:57796453455c20db80d9168edbf125bf6180e1aae869de1546a2be58e4e405ec", + "zh:69139cba772d4df8de87598d8d8a2b1b4b254866db046c061dccc79edb14e6b9", + "zh:7312763259b859ff911c5452ca8bdf7d0be6231c5ea0de2df8f09d51770900ac", + "zh:8d2d6f4015d3c155d7eb53e36f019a729aefb46ebfe13f3a637327d3a1402ecc", + "zh:94ce589275c77308e6253f607de96919b840c2dd36c44aa798f693c9dd81af42", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:adaceec6a1bf4f5df1e12bd72cf52b72087c72efed078aef636f8988325b1a8b", + "zh:d37be1ce187d94fd9df7b13a717c219964cd835c946243f096c6b230cdfd7e92", + "zh:fe6205b5ca2ff36e68395cb8d3ae10a3728f405cdbcd46b206a515e1ebcf17a1", + ] +} + +provider "registry.terraform.io/hashicorp/helm" { + version = "3.3.0" + constraints = ">= 3.0.0, ~> 3.0" + hashes = [ + "h1:FmfW45oPjF6qqO+24Qzh/fnOSkxINHIEVco0xsujENM=", + "h1:Jk/eQ65jQ7L+Db1yQqtvUFc+BPs1QwW8Qoil+TdwGdU=", + "h1:Qbt2+dtOJty3glPZI9jMCXq82GSAKBk/T2+wBMIdu0c=", + "h1:Y9Z+1+RqmCYRCqlncKEKPjYkRqlNIBiK3jdEIbUn1Fc=", + "zh:24a942352667cf49bed467c51a951f17a6c13e0cdf1c505b01b41f75a1a60d0f", + "zh:2c057eafd1e4d8b50acdc676f9a3096e38fb7d30c3fab3d64af82fa04a2f767e", + "zh:44252c60034ca163f2f68eaeece2583411bca25ac83bc58350e24d4f1c2d60a7", + "zh:4628da7c781eebbce1b13d12b498856815bde0e174d7a0d0b85a5d89a1d36d17", + "zh:59b3f4fd8e236c5733032c289a9636ac8bb6c92bc11b540d979b92d005f6cf98", + "zh:5d5f11201b986485ca1d33096f3545c6074b3b4a592864620a8df7f7c399b59f", + "zh:5e656afadb2db3f418eb254b2da02b00851eba8ec63196d1f22b2a2854fe9fb9", + "zh:77515e5576f4ea3d8d9e404237882de62518950ccd7b312371f5c03580f22fb0", + "zh:a4e601eddf9781b5f8678ee9c2b6fb0791fbe6ab85a53707efdf80f714a3011c", + "zh:eaa58edc8bb43aeb0fe6c0f99975cb24e2e9f5441e938a389fe848795a30b825", + "zh:ec97b17b82222d2008256f9b10b88f564d30f717a43438fe425a802ac0ccf2f3", + "zh:f18613d2aa3fd3945d394911e210af7bf58968f64631b6b2e229852b60a2803e", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + ] +} + +provider "registry.terraform.io/hashicorp/kubernetes" { + version = "2.38.0" + constraints = ">= 2.20.0, ~> 2.22" + hashes = [ + "h1:5CkveFo5ynsLdzKk+Kv+r7+U9rMrNjfZPT3a0N/fhgE=", + "h1:7nJdsd1RMPBtOjDXidB37+KSDN5VcOWkbkow69qJVGc=", + "h1:XCkL/mxjWTawg6gg+jlpCQhF/+SNRoCEZxbbkDTj42s=", + "h1:soK8Lt0SZ6dB+HsypFRDzuX/npqlMU6M0fvyaR1yW0k=", + "zh:0af928d776eb269b192dc0ea0f8a3f0f5ec117224cd644bdacdc682300f84ba0", + "zh:1be998e67206f7cfc4ffe77c01a09ac91ce725de0abaec9030b22c0a832af44f", + "zh:326803fe5946023687d603f6f1bab24de7af3d426b01d20e51d4e6fbe4e7ec1b", + "zh:4a99ec8d91193af961de1abb1f824be73df07489301d62e6141a656b3ebfff12", + "zh:5136e51765d6a0b9e4dbcc3b38821e9736bd2136cf15e9aac11668f22db117d2", + "zh:63fab47349852d7802fb032e4f2b6a101ee1ce34b62557a9ad0f0f0f5b6ecfdc", + "zh:924fb0257e2d03e03e2bfe9c7b99aa73c195b1f19412ca09960001bee3c50d15", + "zh:b63a0be5e233f8f6727c56bed3b61eb9456ca7a8bb29539fba0837f1badf1396", + "zh:d39861aa21077f1bc899bc53e7233262e530ba8a3a2d737449b100daeb303e4d", + "zh:de0805e10ebe4c83ce3b728a67f6b0f9d18be32b25146aa89116634df5145ad4", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + "zh:faf23e45f0090eef8ba28a8aac7ec5d4fdf11a36c40a8d286304567d71c1e7db", + ] +} + +provider "registry.terraform.io/hashicorp/null" { + version = "3.3.2" + constraints = "~> 3.0" + hashes = [ + "h1:/Wbz7DHFO92KAqyDBMxGA8nwOJYECM8iLGhR7+wVWhQ=", + "h1:IQ1qrkht1sC1nibUR+AJ3ulryyhVDHfCHZhoJi0sg2Y=", + "h1:SsePRc2TCwjh/+zp9OZRUchvaC96ik8pEIs2vxfW1nA=", + "h1:vpRtoWIdmRPuUP+2DgvgeWNkhA9HB7h6w4xVnaS2J/U=", + "zh:10ec43b8b7b18d5639238c7fb9e111f6a4b038523dd66c7a426bf27b25fa4c08", + "zh:60beb9cc2ad5b871c710860cee75b42850cc6acd43db0d77cb5e00fda7288b55", + "zh:62538582d0a4a2f10ad8a8d9a6c3cd3f05af6c6d91c6641ffc78d4f0e8e69b27", + "zh:64a8f9ce7852d9efc5b464c12306c946366d59f5e2757def97969c9fd64bd1d6", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:92a374fb736a52f465283326d0a5bf4f495132eb99be209dfb4c75ec803fe8db", + "zh:98da9c42785d27a50f0604758bcb61a30f6278b9f2acd92bb3b2046e0e71916c", + "zh:b0f7896fae554729cdf4a24ac06359a050cff5817e6cd8597cba8a4ae01a7409", + "zh:bc8179ee35d67c72fb03012e7023b9f9816f033a7ec4109c001dd6d29752e812", + "zh:d23a598f713bfb6098bc003571d7de90b5a33b78f9be240488252fe5f3c2a60d", + "zh:d2855b922ea345dbd89ea287e4c6c4757e38bc0aaffeb2b79aa0b8004f9c53ff", + "zh:d3a60422bc6a2f9244d076c5222c07060c826ef91bdbaf4634cb752b86057473", + "zh:faa01928c25d2a6ecd9c7eb8b88134cb08de55a6b11ca6c703ac0092845344ba", + ] +} + +provider "registry.terraform.io/hashicorp/random" { + version = "3.9.1" + constraints = ">= 3.6.0" + hashes = [ + "h1:PYbnOqRuGn4c0/Ae1f7yOS/0zvmXNHFJRZjuF4KECnM=", + "h1:PlW+UZ4EElQF3NQwf41KQwavFujab3Czc51zu9dyVM8=", + "h1:g40qr7yDmIpaur4SsK5BcOda3HSo1RJ6zHVMqN4EJ+0=", + "h1:nozfr4CZq73d4HjubKJundX/8A+Mj282oS/hyNfjUPU=", + "zh:05f4734c1f0be840b711b3eff259ebc5fca436784c728955b1678078466f48d7", + "zh:0b91bf19371d012434eba1deeb6aab77158def9b39601dcbd94450b3974a2a26", + "zh:0ee6eacd47ec00183d55d726a4b6c4ce951a199f944bf22f1aa58392ebdfa7a2", + "zh:19388a4074b76a89a43a6c8328d7ae8ee2e7de3d346af51e80d3e6d3d12925f1", + "zh:23e74d48c5e2ac2e823fd527f49fee9db37d32a1990c9e3bf126ead697b843eb", + "zh:3cabf7fbd096c520064aae3aba61aba670af83ab91291a71fa1b1332929c2b7f", + "zh:5c0a3b8af0be60be4eca12ddee385cfa8babc1ec8e98cdf9de2f2274c73eabfa", + "zh:60b4f8a8ef18f52bf8e19215229dae408bee732825964092db7c989fd2de4097", + "zh:7359015acfedcbd6366f2329c854cf8d3c8ca5cd0faa89d2d37db358d6eba6c5", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:7b38758402f0e13a1071162da28994023cd2ac676e54af350c9ffd8dfa73fa7b", + "zh:7c7fbb8895eb75bb4de1f933e98553bd99c8d048c89a925ddba490aa5a67f7dc", + "zh:8c2b8c6a7ccdec16b73e2fb9f3700ea097f58c592571e4c5de60c93d2301732c", + ] +} + +provider "registry.terraform.io/hashicorp/time" { + version = "0.14.2" + constraints = ">= 0.9.0" + hashes = [ + "h1:eQRXh8mZFlUJfzYXKdaYRHRMhiS2cFyCfgP1mjkrtuI=", + "h1:gnP2hptiFIHSHUFBvAFKhE/Yh5u5yVEx+P7XSB58A/E=", + "h1:mwxQFMxGnM+Xm3F76yeHSAfPr3RZq/+XGT1vjgwiRWQ=", + "h1:pmcYDG9dBx6l+2YQFP59egVwno/yf0l1QS56gl2PzLY=", + "zh:0aa1028d91041f4dceba193e3707dac57358d0063d97e20700e554758b67baca", + "zh:32bee9f2b2678e2a0789ad86e716d09ca1d5450180b3cd8033ee7a251bfd352e", + "zh:3aded9ef4dc6f4aec202a50c68a08b40013d325f9947f10168ebc8bee54109fc", + "zh:4d924637f3115ffa4ffc7f16d3f366bc472594f7447d400adb9def7ac92e3fc8", + "zh:5c35008e1363deafaa440ab43519409866dd7ec72aabeb2317bc16cd82756784", + "zh:6b30d97c9827501d7010fe49c8889d7a6cc8b45b77cdb1668c65af7f28a27d73", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:851e0f0e9c4de395e42220de51e7fb20d0e845e629f3cf37056da19e597304e7", + "zh:85622b4779b3ba7424780f7f8efcfa585227e245201cdb68303f2118b388e971", + "zh:9039153e3d45147183804188a1ce36c3811db9a4dca80f36fa382408d5f50b72", + "zh:a3d385413dc258a53fe8f4ded5d1fcd1eba715c65a015d74e70911bab2919207", + "zh:b9310a2327f7c8ad2aa3266f89c4d026b4bf18b09f7c4257c0ecd71a32f32db3", + "zh:cee7f2143da0c494115da94984bf3630e3e5855e27dae720a7bc29bb6acd9be2", + ] +} diff --git a/terraform/40.observability/45.aws-oss-observability/.terraform.lock.hcl b/terraform/40.observability/45.aws-oss-observability/.terraform.lock.hcl new file mode 100644 index 0000000..4fee605 --- /dev/null +++ b/terraform/40.observability/45.aws-oss-observability/.terraform.lock.hcl @@ -0,0 +1,146 @@ +# This file is maintained automatically by "terraform init". +# Manual edits may be lost in future updates. + +provider "registry.terraform.io/alekc/kubectl" { + version = "2.4.1" + constraints = "2.4.1" + hashes = [ + "h1:5ABzWzl+SvCYjhR6z/Pz+aZqTPhxLnmYa3NxRl3IVwM=", + "h1:8xq9mBqJQf1qNdwj7hmzdhqUTuvhkRBiUVsUHWADf28=", + "h1:oFhUB/LqtnraVc/XUSCSuy/4h/g6e3AtGkS6XADieaQ=", + "h1:pDnerh94NFiSv5z/Vuq+c1mEdn117P9KQI8zXeEEAEI=", + "zh:00a27c51a6f1082f20cbee52e0b996e4bed1dbc90de682cc309618aee2857e35", + "zh:16f94d63152a8a7c637fa3c609a104416272565b7a9b620e3b895ddb9912e288", + "zh:18a5c663c2aa6e673d75650d27888f1f91894f46d504fbf4a87f045c96f7afc1", + "zh:1a7c76db49ec94c0669baa6e416007f76acf09dbd141ef20036e4e13840651d7", + "zh:2baf0f93ff27fa0ff7265ee9eed5cc773311ace3bf936eb72bfbbd579d436894", + "zh:3ea7f773485b08ed55eedd29a7c83988c91ba2cc346b2436e7d1347ab061625a", + "zh:4bb3a241078f7b62dc41e9ac156c83b628933248b76cee3ec0d0264c0a2a25b7", + "zh:57571e1d9230fbde986a265228e17662eeb5ec63c12b84742564d453ad21f955", + "zh:6e1b82fde0037368a9e156304a580364ef926f26ffe235bb5a67b4c9f2edc2c4", + "zh:9dec7ed91b39d9815d9c37b78eca0327f631c66b2bcc61cbb4212c6d2ec76a22", + "zh:a347aebd398769c35ed2d3169025a7a11c1937d94efead7a7bee212aabba1c73", + "zh:b74e24ec0e2e176ee6cc18f33ac7e3609cfd6ca377b49cbd64b0f36edbb54342", + "zh:f6844cf7ead4443638dfdb3f837481585f4a4c910bcf7b030a19820cc5109ebc", + ] +} + +provider "registry.terraform.io/hashicorp/aws" { + version = "6.28.0" + constraints = ">= 5.0.0, ~> 6.28.0" + hashes = [ + "h1:2bDndcCvti7hgXw4MkMo37cyAAu1gk+JvsU9/UbRJNQ=", + "h1:RwoFuX1yGMVaKJaUmXDKklEaQ/yUCEdt5k2kz+/g08c=", + "h1:bMiTeecRXGBmx/btqJX57X0KuJ3j9BmM/ph3KZ3FGj0=", + "h1:wzZdGs0FFmNqIgPyo9tKnGKJ37BGNSgwRrEXayL29+0=", + "zh:0ba0d5eb6e0c6a933eb2befe3cdbf22b58fbc0337bf138f95bf0e8bb6e6df93e", + "zh:23eacdd4e6db32cf0ff2ce189461bdbb62e46513978d33c5de4decc4670870ec", + "zh:307b06a15fc00a8e6fd243abde2cbe5112e9d40371542665b91bec1018dd6e3c", + "zh:37a02d5b45a9d050b9642c9e2e268297254192280df72f6e46641daca52e40ec", + "zh:3da866639f07d92e734557d673092719c33ede80f4276c835bf7f231a669aa33", + "zh:480060b0ba310d0f6b6a14d60b276698cb103c48fd2f7e2802ae47c963995ec6", + "zh:57796453455c20db80d9168edbf125bf6180e1aae869de1546a2be58e4e405ec", + "zh:69139cba772d4df8de87598d8d8a2b1b4b254866db046c061dccc79edb14e6b9", + "zh:7312763259b859ff911c5452ca8bdf7d0be6231c5ea0de2df8f09d51770900ac", + "zh:8d2d6f4015d3c155d7eb53e36f019a729aefb46ebfe13f3a637327d3a1402ecc", + "zh:94ce589275c77308e6253f607de96919b840c2dd36c44aa798f693c9dd81af42", + "zh:9b12af85486a96aedd8d7984b0ff811a4b42e3d88dad1a3fb4c0b580d04fa425", + "zh:adaceec6a1bf4f5df1e12bd72cf52b72087c72efed078aef636f8988325b1a8b", + "zh:d37be1ce187d94fd9df7b13a717c219964cd835c946243f096c6b230cdfd7e92", + "zh:fe6205b5ca2ff36e68395cb8d3ae10a3728f405cdbcd46b206a515e1ebcf17a1", + ] +} + +provider "registry.terraform.io/hashicorp/helm" { + version = "3.3.0" + constraints = "~> 3.0" + hashes = [ + "h1:FmfW45oPjF6qqO+24Qzh/fnOSkxINHIEVco0xsujENM=", + "h1:Jk/eQ65jQ7L+Db1yQqtvUFc+BPs1QwW8Qoil+TdwGdU=", + "h1:Qbt2+dtOJty3glPZI9jMCXq82GSAKBk/T2+wBMIdu0c=", + "h1:Y9Z+1+RqmCYRCqlncKEKPjYkRqlNIBiK3jdEIbUn1Fc=", + "zh:24a942352667cf49bed467c51a951f17a6c13e0cdf1c505b01b41f75a1a60d0f", + "zh:2c057eafd1e4d8b50acdc676f9a3096e38fb7d30c3fab3d64af82fa04a2f767e", + "zh:44252c60034ca163f2f68eaeece2583411bca25ac83bc58350e24d4f1c2d60a7", + "zh:4628da7c781eebbce1b13d12b498856815bde0e174d7a0d0b85a5d89a1d36d17", + "zh:59b3f4fd8e236c5733032c289a9636ac8bb6c92bc11b540d979b92d005f6cf98", + "zh:5d5f11201b986485ca1d33096f3545c6074b3b4a592864620a8df7f7c399b59f", + "zh:5e656afadb2db3f418eb254b2da02b00851eba8ec63196d1f22b2a2854fe9fb9", + "zh:77515e5576f4ea3d8d9e404237882de62518950ccd7b312371f5c03580f22fb0", + "zh:a4e601eddf9781b5f8678ee9c2b6fb0791fbe6ab85a53707efdf80f714a3011c", + "zh:eaa58edc8bb43aeb0fe6c0f99975cb24e2e9f5441e938a389fe848795a30b825", + "zh:ec97b17b82222d2008256f9b10b88f564d30f717a43438fe425a802ac0ccf2f3", + "zh:f18613d2aa3fd3945d394911e210af7bf58968f64631b6b2e229852b60a2803e", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + ] +} + +provider "registry.terraform.io/hashicorp/kubernetes" { + version = "2.38.0" + constraints = "~> 2.22" + hashes = [ + "h1:5CkveFo5ynsLdzKk+Kv+r7+U9rMrNjfZPT3a0N/fhgE=", + "h1:7nJdsd1RMPBtOjDXidB37+KSDN5VcOWkbkow69qJVGc=", + "h1:XCkL/mxjWTawg6gg+jlpCQhF/+SNRoCEZxbbkDTj42s=", + "h1:soK8Lt0SZ6dB+HsypFRDzuX/npqlMU6M0fvyaR1yW0k=", + "zh:0af928d776eb269b192dc0ea0f8a3f0f5ec117224cd644bdacdc682300f84ba0", + "zh:1be998e67206f7cfc4ffe77c01a09ac91ce725de0abaec9030b22c0a832af44f", + "zh:326803fe5946023687d603f6f1bab24de7af3d426b01d20e51d4e6fbe4e7ec1b", + "zh:4a99ec8d91193af961de1abb1f824be73df07489301d62e6141a656b3ebfff12", + "zh:5136e51765d6a0b9e4dbcc3b38821e9736bd2136cf15e9aac11668f22db117d2", + "zh:63fab47349852d7802fb032e4f2b6a101ee1ce34b62557a9ad0f0f0f5b6ecfdc", + "zh:924fb0257e2d03e03e2bfe9c7b99aa73c195b1f19412ca09960001bee3c50d15", + "zh:b63a0be5e233f8f6727c56bed3b61eb9456ca7a8bb29539fba0837f1badf1396", + "zh:d39861aa21077f1bc899bc53e7233262e530ba8a3a2d737449b100daeb303e4d", + "zh:de0805e10ebe4c83ce3b728a67f6b0f9d18be32b25146aa89116634df5145ad4", + "zh:f569b65999264a9416862bca5cd2a6177d94ccb0424f3a4ef424428912b9cb3c", + "zh:faf23e45f0090eef8ba28a8aac7ec5d4fdf11a36c40a8d286304567d71c1e7db", + ] +} + +provider "registry.terraform.io/hashicorp/null" { + version = "3.3.2" + constraints = "~> 3.0" + hashes = [ + "h1:/Wbz7DHFO92KAqyDBMxGA8nwOJYECM8iLGhR7+wVWhQ=", + "h1:IQ1qrkht1sC1nibUR+AJ3ulryyhVDHfCHZhoJi0sg2Y=", + "h1:SsePRc2TCwjh/+zp9OZRUchvaC96ik8pEIs2vxfW1nA=", + "h1:vpRtoWIdmRPuUP+2DgvgeWNkhA9HB7h6w4xVnaS2J/U=", + "zh:10ec43b8b7b18d5639238c7fb9e111f6a4b038523dd66c7a426bf27b25fa4c08", + "zh:60beb9cc2ad5b871c710860cee75b42850cc6acd43db0d77cb5e00fda7288b55", + "zh:62538582d0a4a2f10ad8a8d9a6c3cd3f05af6c6d91c6641ffc78d4f0e8e69b27", + "zh:64a8f9ce7852d9efc5b464c12306c946366d59f5e2757def97969c9fd64bd1d6", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:92a374fb736a52f465283326d0a5bf4f495132eb99be209dfb4c75ec803fe8db", + "zh:98da9c42785d27a50f0604758bcb61a30f6278b9f2acd92bb3b2046e0e71916c", + "zh:b0f7896fae554729cdf4a24ac06359a050cff5817e6cd8597cba8a4ae01a7409", + "zh:bc8179ee35d67c72fb03012e7023b9f9816f033a7ec4109c001dd6d29752e812", + "zh:d23a598f713bfb6098bc003571d7de90b5a33b78f9be240488252fe5f3c2a60d", + "zh:d2855b922ea345dbd89ea287e4c6c4757e38bc0aaffeb2b79aa0b8004f9c53ff", + "zh:d3a60422bc6a2f9244d076c5222c07060c826ef91bdbaf4634cb752b86057473", + "zh:faa01928c25d2a6ecd9c7eb8b88134cb08de55a6b11ca6c703ac0092845344ba", + ] +} + +provider "registry.terraform.io/hashicorp/time" { + version = "0.10.0" + constraints = "0.10.0" + hashes = [ + "h1:EeF/Lb4db1Kl1HEHzT1StTC7RRqHn/eB7aDR3C3yjVg=", + "h1:NAl8eupFAZXCAbE5uiHZTz+Yqler55B3fMG+jNPrjjM=", + "h1:QL1ivYrUSB3zvhgXcRBfQak4HDxvesT2zktM+N6StVo=", + "h1:wraHzpOZSaxgC37HLOt0k5Mstl1iNrol7DvvqkSQ1kc=", + "zh:0ab31efe760cc86c9eef9e8eb070ae9e15c52c617243bbd9041632d44ea70781", + "zh:0ee4e906e28f23c598632eeac297ab098d6d6a90629d15516814ab90ad42aec8", + "zh:3bbb3e9da728b82428c6f18533b5b7c014e8ff1b8d9b2587107c966b985e5bcc", + "zh:6771c72db4e4486f2c2603c81dfddd9e28b6554d1ded2996b4cb37f887b467de", + "zh:78d5eefdd9e494defcb3c68d282b8f96630502cac21d1ea161f53cfe9bb483b3", + "zh:833c636d86c2c8f23296a7da5d492bdfd7260e22899fc8af8cc3937eb41a7391", + "zh:c545f1497ae0978ffc979645e594b57ff06c30b4144486f4f362d686366e2e42", + "zh:def83c6a85db611b8f1d996d32869f59397c23b8b78e39a978c8a2296b0588b2", + "zh:df9579b72cc8e5fac6efee20c7d0a8b72d3d859b50828b1c473d620ab939e2c7", + "zh:e281a8ecbb33c185e2d0976dc526c93b7359e3ffdc8130df7422863f4952c00e", + "zh:ecb1af3ae67ac7933b5630606672c94ec1f54b119bf77d3091f16d55ab634461", + "zh:f8109f13e07a741e1e8a52134f84583f97a819e33600be44623a21f6424d6593", + ] +} diff --git a/tests/test_gitops.py b/tests/test_gitops.py new file mode 100644 index 0000000..8f77775 --- /dev/null +++ b/tests/test_gitops.py @@ -0,0 +1,57 @@ +"""Tests for the GitOps helpers — focused on the hf-token Secret creation, which +must never place the token on a kubectl argv (readable via /proc//cmdline).""" + +import base64 +import json + +from recommend_instance import gitops + + +class _C: + """Stub colour palette — _ensure_hf_token_secret only reads these attrs.""" + RESET = YELLOW = GREEN = DIM = RED = BOLD = CYAN = "" + + +TOKEN = "hf_sekret_TOKEN_do_not_leak_123" + + +class TestHfTokenSecret: + def test_token_never_on_argv_and_applied_as_base64(self, monkeypatch): + calls = [] + + class _Result: + returncode = 0 + stdout = "" + stderr = "" + + def fake_run(args, **kwargs): + calls.append((args, kwargs)) + return _Result() + + monkeypatch.setattr(gitops.shutil, "which", lambda _cmd: "/usr/bin/kubectl") + monkeypatch.setattr(gitops.subprocess, "run", fake_run) + + gitops._ensure_hf_token_secret("inference", TOKEN, "/tmp", _C) + + # Exactly one kubectl call: `apply -f -` (no `create secret --from-literal`). + assert len(calls) == 1, calls + argv, kwargs = calls[0] + assert argv[:2] == ["kubectl", "apply"] + assert "-f" in argv and "-" in argv + # The token appears NOWHERE on the command line. + assert not any(TOKEN in str(a) for a in argv), argv + # It reaches kubectl only via stdin, base64-encoded in a Secret manifest. + manifest = json.loads(kwargs["input"]) + assert manifest["kind"] == "Secret" + assert manifest["metadata"]["name"] == "hf-token" + assert base64.b64decode(manifest["data"]["token"]).decode() == TOKEN + # And the raw token is not present verbatim in the stdin payload. + assert TOKEN not in kwargs["input"] + + def test_no_kubectl_makes_no_calls(self, monkeypatch): + calls = [] + monkeypatch.setattr(gitops.shutil, "which", lambda _cmd: None) + monkeypatch.setattr(gitops.subprocess, "run", + lambda *a, **k: calls.append((a, k))) + gitops._ensure_hf_token_secret("inference", TOKEN, "/tmp", _C) + assert calls == [] # prints a manual fallback; never shells the token diff --git a/tests/test_render.py b/tests/test_render.py index 3d3720d..2de81f3 100644 --- a/tests/test_render.py +++ b/tests/test_render.py @@ -60,3 +60,45 @@ def test_valid_id_round_trips_as_yaml(self): doc = list(yaml.safe_load_all(body))[0] assert doc["spec"]["model"] == mid assert doc["metadata"]["name"] == "llama-3-1-8b-instruct" + + +class TestYamlScalarQuoting: + """`--extra-arg` tokens are user-supplied and interpolated into a manifest + that `--deploy` commits and ArgoCD applies. `_yaml_scalar` must emit a YAML + scalar that round-trips to the EXACT input, with no way to alter surrounding + manifest structure. These tokens broke the previous hand-rolled quoting.""" + + ADVERSARIAL = [ + "--enable-prefix-caching", # plain + "qwen3", + 'j{"k":"v"}', # embedded double quotes (JSON payload) + "it's", # single quote + "both'\"quotes", # both quote styles present + "trailing\\", # trailing backslash -> old code escaped the closing quote + "x'y\\", # single quote + trailing backslash -> old parse error + "a\\tc", # literal backslash-t -> must NOT decode to a TAB + "a\\0b", # backslash-zero + "a\\x41b", # backslash-x escape sequence + "a\\u0041b", # backslash-u escape sequence + "--kv=a\tb", # a real TAB character in the token + "café", # non-ASCII + ] + + @pytest.mark.parametrize("tok", ADVERSARIAL) + def test_round_trips_exactly(self, tok): + emitted = render._yaml_scalar(tok) + # Exactly one key, value byte-identical to the input. + assert yaml.safe_load("v: " + emitted) == {"v": tok} + + @pytest.mark.parametrize("bad", ["a\nb", "a\rb", "\n", 'x"\nowned: yes']) + def test_rejects_newlines(self, bad): + # A newline is the only way to inject an adjacent mapping key; it's + # rejected up front rather than emitted. + with pytest.raises(SystemExit): + render._yaml_scalar(bad) + + def test_full_extraargs_fragment_round_trips(self): + # Mirror how build_endpoint_yaml emits the list, then parse the fragment. + tokens = ["--reasoning-parser", "qwen3", 'cfg={"a":"b\\t"}', "trailing\\"] + fragment = "\n".join(["extraArgs:"] + [f" - {render._yaml_scalar(t)}" for t in tokens]) + assert yaml.safe_load(fragment) == {"extraArgs": tokens} diff --git a/workloads/models/README.md b/workloads/models/README.md index 7907783..7e90db2 100644 --- a/workloads/models/README.md +++ b/workloads/models/README.md @@ -33,13 +33,16 @@ By default, models live in this repo. For real multi-team self-service, point access to the **workloads repo only** — never the platform repo. The directory-per-namespace convention above is unchanged. -## Bedrock — available out of the box - -`bedrock/amazon.nova-lite-v1:0` is a static LiteLLM entry (no GPUs, no CR), so a -fresh install serves against it immediately. To serve any other model, point a -`VLLMEndpoint` at a HuggingFace model ID — including a model you've fine-tuned and -pushed to HF (private repos need a token). The platform serves models; it does not -train them. (Serving weights directly from your own S3 bucket is on the roadmap.) +## Bedrock — self-service enrollment (no GPUs) + +Amazon Bedrock models are enrolled with `./platformctl new-model --source bedrock + --deploy`, which commits a `BedrockModel` CR (no GPUs, no serving pod) and +litellm-sync registers it on the `/v1` API. Nothing ships by default — list what's +invokable in your region/partition with `--source bedrock --list-available-models`. +To serve any open-source model instead, point a `VLLMEndpoint` at a HuggingFace +model ID — including a model you've fine-tuned and pushed to HF (private repos need +a token). The platform serves models; it does not train them. (Serving weights +directly from your own S3 bucket is on the roadmap.) ## Tool / function calling