Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
456 lines (351 loc) · 14.7 KB

ARCHITECTURE.md

File metadata and controls

456 lines (351 loc) · 14.7 KB

Architecture Guide

This document provides a detailed explanation of the Kro Serverless Application Stack architecture.

Table of Contents

  1. Overview
  2. Core Concept
  3. Three-Layer Architecture
  4. Cross-Controller Integration
  5. Resource Dependencies
  6. CEL Expressions
  7. Variable Resolution
  8. Security Model
  9. Deployment Flow

Overview

The Kro Serverless Application Stack implements a Hybrid Composition Pattern that combines:

  • Pre-provisioned shared resources (CloudFront, IAM roles, Cognito, DNS, CloudFront Function) managed outside the RGD
  • Dynamically created per-application resources (S3, Lambda, API Gateway, DynamoDB) managed by Kro/ACK/Crossplane

This pattern provides:

  • Cost efficiency through resource sharing (90%+ reduction)
  • Isolation through per-application resources
  • Flexibility through Kubernetes-native orchestration
  • Security through integrated Cognito authentication
  • Path-based routing through a single CloudFront distribution with CloudFront Function

Core Concept

The fundamental idea behind Hybrid Kro Compositions is simple: one ResourceGraphDefinition orchestrates a complete serverless stack.

Three-Layer Overview

What makes this powerful:

  • Developers create one simple YAML manifest
  • Kro automatically provisions all required resources
  • Dependencies are resolved automatically
  • The entire stack is ready in 5-8 minutes

Three-Layer Architecture

The architecture separates concerns into three distinct layers:

Three-Layer Overview

Detailed Layer View

%%{init: {'theme':'dark'}}%%
graph TB
    subgraph SharedPlatform["Layer 1: Shared Platform Resources"]
        S3Assets["S3 Platform Assets Bucket<br/><br/>• functions/ (Lambda code)<br/>• ui/ (UI templates)"]
        S3SharedUI["S3 Shared UI Bucket<br/><br/>• /developer1-app-dev/<br/>• /developer2-app-dev/<br/>• /team-app-staging/"]
        CloudFront["CloudFront Distribution<br/><br/>domain.com (path-based)<br/>Environments: ALL"]
        CFFunction["CloudFront Function<br/><br/>URL Rewriting<br/>index.html append"]
        OAC["Origin Access Control<br/><br/>Secure S3 Access<br/>Signing: Always"]
        IAMRoles["IAM Roles (Shared)<br/><br/>• lambda-execution-role<br/>• s3-uploader-role"]
        Cognito["Cognito Resources<br/><br/>• User Pool<br/>• App Client<br/>• Single Callback URL"]
        DNSRecord["Route 53 DNS Record<br/><br/>domain.com → CloudFront"]
        
        S3Assets -.->|Templates| S3SharedUI
        S3SharedUI -->|Origin| CloudFront
        CloudFront -->|viewer-request| CFFunction
        CloudFront -->|Secure Access| OAC
        OAC -->|Signs Requests| S3SharedUI
        IAMRoles -.->|Permissions| S3SharedUI
        DNSRecord -.->|Points to| CloudFront
    end
    
    subgraph RGDLayer["Layer 2: Kro ResourceGraphDefinition"]
        RGD["SmartInventoryHubV5 RGD<br/><br/>References shared resources:<br/>• Shared CloudFront<br/>• Shared S3 UI Bucket<br/>• Shared IAM roles<br/>+ Creates app-specific:<br/>• DynamoDB table<br/>• Lambda function<br/>• API Gateway<br/>• S3 prefix (via Job)"]
    end
    
    subgraph AppInstances["Layer 3: Application Instances"]
        App1["domain.com<br/>/developer1-app-dev/<br/><br/>S3: /developer1-app-dev/<br/>CloudFront: Shared"]
        App2["domain.com<br/>/developer2-app-dev/<br/><br/>S3: /developer2-app-dev/<br/>CloudFront: Shared"]
        App3["domain.com<br/>/team-app-staging/<br/><br/>S3: /team-app-staging/<br/>CloudFront: Shared"]
    end
    
    SharedPlatform -->|Referenced by| RGDLayer
    RGD -->|Creates| App1
    RGD -->|Creates| App2
    RGD -->|Creates| App3
    
    style SharedPlatform fill:#1a1a2e,stroke:#4A90E2,stroke-width:3px
    style RGDLayer fill:#1a1a2e,stroke:#FFE66D,stroke-width:3px
    style AppInstances fill:#1a1a2e,stroke:#26DE81,stroke-width:2px
    style S3Assets fill:#FF9F43,color:#000
    style S3SharedUI fill:#FF9F43,color:#000
    style CloudFront fill:#95E1D3,color:#000
    style CFFunction fill:#95E1D3,color:#000
    style OAC fill:#FFE66D,color:#000
    style IAMRoles fill:#FF6B6B
    style Cognito fill:#4A90E2
    style DNSRecord fill:#95E1D3,color:#000
    style RGD fill:#FFE66D,color:#000
    style App1 fill:#26DE81,color:#000
    style App2 fill:#26DE81,color:#000
    style App3 fill:#26DE81,color:#000
Loading

Layer 1: Shared Platform Resources

Resources deployed once and reused across all applications:

Resource Purpose Benefit
CloudFront Distribution CDN with single domain alias Eliminates 20-min provisioning per app
CloudFront Function Appends index.html to directory paths Enables SPA routing for all apps
Origin Access Control Secure S3 access Centralized security policy
IAM Roles (2) Lambda execution, S3 uploader Consistent permissions
Cognito User Pool Authentication with single callback URL Single sign-on across apps via state parameter
Platform Assets Bucket Lambda code and UI templates Centralized asset management
S3 Shared UI Bucket Hosts UI assets for all instances Path-based multi-tenancy
Route 53 DNS Record Points domain.com to CloudFront Single DNS record for all apps

Layer 2: ResourceGraphDefinition

The RGD defines a complete serverless stack:

apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
  name: smart-inventory-hub-v5
spec:
  schema:
    apiVersion: v1alpha1
    kind: SmartInventoryHubV5
    spec:
      teamName: string | required
      appName: string | required
      environment: string | required
      developerName: string | required
      region: string | default=us-east-2
  resources:
    # 15 resources defined here
    - id: dynamodbTable
    - id: lambdaFunction
    - id: apiGateway
    # ... etc

Layer 3: Application Instances

Simple YAML manifests that create complete stacks:

apiVersion: kro.run/v1alpha1
kind: SmartInventoryHubV5
metadata:
  name: john-inventory-dev
  namespace: kro-serverless-stack
spec:
  teamName: team-backend
  developerName: john
  appName: inventory
  environment: dev
  region: us-east-2

Cross-Controller Integration

The power of Hybrid Kro Compositions lies in seamless integration between different controllers:

cross controller dependencies

How Data Flows Between Controllers

  1. ACK DynamoDB creates table → exposes tableName
  2. Crossplane Lambda references tableName → exposes functionArn
  3. ACK API Gateway references functionArn → exposes apiId
  4. Crossplane Permission references apiId → authorizes invocation
  5. Kubernetes Job references outputs → syncs UI assets to S3

Dependency Resolution

Kro analyzes CEL expressions to build the dependency graph:

# Example: Lambda depends on DynamoDB
- id: lambdaFunction
  template:
    spec:
      environment:
        variables:
          TABLE_NAME: "${dynamodbTable.spec.tableName}"  # ← Implicit dependency

# Example: API Integration depends on Lambda
- id: apiIntegration
  template:
    spec:
      integrationURI: "${lambdaFunction.status.atProvider.arn}"  # ← Implicit dependency

Key Rules:

  • Resources are created in array order
  • ${resource.status...} references create implicit dependencies
  • Kro waits for referenced resources to be ready
  • No explicit dependencies field needed

CEL Expressions

Common Expression Language (CEL) enables dynamic configuration:

Environment-Based Naming

# Dev environments include developer name
name: "${schema.spec.environment == 'dev' ? 
        schema.spec.developerName + '-' + schema.spec.appName + '-dev' : 
        schema.spec.teamName + '-' + schema.spec.appName + '-' + schema.spec.environment}"

Cross-Resource References

# Reference ACK resource status
integrationURI: "arn:aws:apigateway:${schema.spec.region}:lambda:path/2015-03-31/functions/${lambdaFunction.status.atProvider.arn}/invocations"

# Reference Crossplane resource status
functionName: "${lambdaFunction.status.atProvider.functionName}"

# Reference Kubernetes resource
serviceAccountName: "${s3UploaderServiceAccount.metadata.name}"

Conditional Logic

# Different configurations per environment
billingMode: "${schema.spec.environment == 'prod' ? 'PROVISIONED' : 'PAY_PER_REQUEST'}"

Variable Resolution

Variable resolution in Hybrid Kro Compositions happens at two distinct moments:

CEL expression resolution

Moment 1: Platform Configuration (envsubst)

Shared resources injected once before RGD deployment:

  • IAM role ARNs for Lambda execution
  • Cognito User Pool IDs for authentication
  • CloudFront distribution domains for CDN
  • S3 bucket names for platform assets
  • OAC ID and CloudFront Function ARN
# Before envsubst
roleArn: "${PLATFORM_LAMBDA_ROLE_ARN}"

# After envsubst
roleArn: "arn:aws:iam::123456789012:role/platform-lambda-role"

Moment 2: Application Configuration (Kro CEL)

Per-instance values resolved dynamically:

  • Resource names (DynamoDB table, Lambda function, API Gateway)
  • Environment-specific settings (dev, staging, prod)
  • Team and developer identifiers for multi-tenancy
  • S3 path prefixes for UI assets
# CEL expression in RGD
name: "${schema.spec.teamName}-${schema.spec.appName}-${schema.spec.environment}"

# Developer creates instance with:
spec:
  teamName: "team-backend"
  appName: "inventory"
  environment: "dev"

# Kro resolves to:
name: "team-backend-inventory-dev"

Security Model

IAM Architecture

%%{init: {'theme':'dark'}}%%
graph TB
    subgraph IRSA["IRSA (IAM Roles for Service Accounts)"]
        SA["Kubernetes<br/>ServiceAccount"]
        Role["IAM Role"]
        SA -->|"annotated with"| Role
    end
    
    subgraph Resources["AWS Resources"]
        Lambda["Lambda<br/>Function"]
        S3["S3<br/>Bucket"]
        DDB["DynamoDB<br/>Table"]
    end
    
    Role -->|"AssumeRole"| Lambda
    Role -->|"GetObject/PutObject"| S3
    Lambda -->|"GetItem/PutItem"| DDB
    
    style IRSA fill:#1a1a2e,stroke:#4A90E2
    style Resources fill:#1a1a2e,stroke:#FF9F43
Loading

Authentication Flow

%%{init: {'theme':'dark'}}%%
sequenceDiagram
    participant User
    participant CF as CloudFront (domain.com)
    participant Cognito
    participant Callback as Auth Callback Handler
    participant APIGW as API Gateway
    participant Lambda
    
    User->>CF: Access domain.com/app-prefix/
    CF->>Cognito: Redirect to login (state=app-prefix)
    Cognito->>User: Login form
    User->>Cognito: Credentials
    Cognito->>Callback: Redirect to domain.com/auth/callback
    Callback->>Callback: Read state parameter
    Callback->>CF: Redirect to domain.com/app-prefix/
    CF->>User: Serve UI with JWT token
    User->>APIGW: API request + JWT
    APIGW->>APIGW: Validate JWT
    APIGW->>Lambda: Invoke function
    Lambda->>APIGW: Response
    APIGW->>User: Response
Loading

S3 Bucket Policy

The shared UI bucket uses SourceAccount condition for CloudFront access:

{
  "Condition": {
    "StringEquals": {
      "aws:SourceAccount": "ACCOUNT_ID"
    }
  }
}

This is simpler than SourceArn (which requires the distribution ID) and works with any CloudFront distribution in the account.

Security Best Practices

Practice Implementation
Least Privilege 2 shared IAM roles with minimal permissions
JWT Validation API Gateway authorizer with Cognito
S3 Security Origin Access Control (OAC) for CloudFront
Encryption DynamoDB encryption at rest
HTTPS ACM certificates for all endpoints
SPA Auth generateSecret: false for Cognito client (SPA-compatible)
Multi-tenant Auth Single callback URL with state parameter routing

Deployment Flow

The complete orchestration process happens in two phases:

%%{init: {'theme':'dark'}}%%
sequenceDiagram
    participant Dev as Developer
    participant K8s as Kubernetes
    participant Kro
    participant ACK
    participant XP as Crossplane
    participant AWS
    
    Dev->>K8s: kubectl apply instance.yaml
    K8s->>Kro: Create SmartInventoryHubV5
    
    rect rgb(30, 40, 60)
        Note over Kro: Phase 1: Analysis
        Kro->>Kro: Parse CEL expressions
        Kro->>Kro: Build dependency graph
        Kro->>Kro: Validate configuration
    end
    
    rect rgb(40, 50, 70)
        Note over Kro,AWS: Phase 2: Provisioning
        Kro->>ACK: Create DynamoDB Table
        ACK->>AWS: CreateTable API
        AWS-->>ACK: Table created
        Kro->>XP: Create Lambda Function
        XP->>AWS: CreateFunction API
        AWS-->>XP: Function created
        Kro->>ACK: Create API Gateway
        ACK->>AWS: CreateApi API
        AWS-->>ACK: API created
    end
    
    rect rgb(50, 60, 80)
        Note over Kro,K8s: Phase 3: Automation
        Kro->>K8s: Create UI Sync Job
        K8s->>AWS: Sync UI assets to S3
    end
    
    Kro-->>Dev: Instance ready (5-8 min)
Loading

Three-Phase Orchestration

KRO Orchestration Flow

Resource Distribution by Controller

The RGD orchestrates 15 resources across three different controllers:

%%{init: {'theme':'dark'}}%%
pie showData
    title Resources by Controller
    "ACK (10)" : 10
    "Crossplane (2)" : 2
    "Kubernetes (3)" : 3
Loading

Why This Distribution?

Controller Resources Rationale
ACK DynamoDB, API Gateway (+ Authorizer, Integration, Routes, Stage) Direct AWS API mapping, fast provisioning, excellent status exposure
Crossplane Lambda, Lambda Permission Complex IAM integration, S3-based code deployment (50MB limit vs 4KB inline)
Kubernetes ServiceAccount, Job, ConfigMap Native orchestration, IRSA integration, post-provisioning automation

Summary

The Hybrid Kro Compositions architecture provides:

  1. Unified Orchestration: Single RGD for complete serverless stacks (15 resources)
  2. Cross-Controller Integration: ACK, Crossplane, and Kubernetes working together
  3. Automatic Dependencies: CEL expressions enable implicit dependency resolution
  4. Cost Optimization: Shared resources reduce costs by 90%+
  5. Security Integration: Cognito authentication with single callback URL and state-based routing
  6. Path-Based Routing: Single CloudFront distribution with CloudFront Function for index.html append
  7. GitOps Ready: Declarative manifests for version control