Restore optimized workers, prepare 2 GiB resize, and patch legacy Next.js

.github/workflows/optimized-verify.yml

@@ -48,3 +48,18 @@ jobs:
           docker compose --env-file infra/optimized/runtime/release.env.example -f infra/optimized/runtime/compose.yaml config >/dev/null
           docker run --rm --env INDUS_HOSTNAME=optimized.example.com --volume "$PWD/infra/optimized/runtime/Caddyfile:/etc/caddy/Caddyfile:ro" caddy:2.10.2-alpine@sha256:4c6e91c6ed0e2fa03efd5b44747b625fec79bc9cd06ac5235a779726618e530d caddy validate --config /etc/caddy/Caddyfile --adapter caddyfile
           bash -n scripts/optimized/install-release.sh infra/optimized/runtime/bootstrap.sh
+      - name: Check optional Temporal worker and Valkey access
+        shell: bash
+        run: |
+          set -euo pipefail
+          compose=(docker compose --env-file infra/optimized/runtime/release.env.example -f infra/optimized/runtime/compose.yaml)
+          default_services="$(env -u COMPOSE_PROFILES "${compose[@]}" config --services)"
+          if grep -qx research-worker <<< "$default_services"; then
+            echo "Temporal worker must stay disabled by default" >&2
+            exit 1
+          fi
+          COMPOSE_PROFILES=temporal "${compose[@]}" config --services | grep -qx research-worker
+          "${compose[@]}" config --format json | jq -e '(.services.valkey.ports // []) | length == 0' >/dev/null
+          trap '"${compose[@]}" down --volumes' EXIT
+          "${compose[@]}" up --detach --wait valkey
+          [[ "$("${compose[@]}" run --rm --no-deps --entrypoint valkey-cli valkey -h valkey ping)" == PONG ]]

infra/optimized/COSTS.md

@@ -5,7 +5,7 @@ reviewing spend. It is an inventory, not an estimate or billing claim.
 
 | Component | Usage input to collect | Notes |
 | --- | --- | --- |
-| EC2 | `t3a.medium` instance-hours and CPU burst credits | Includes the Docker host; 2 GiB has not been proven for the full process set. |
+| EC2 | `t3a.small` instance-hours and CPU burst credits | The 2 GiB target requires a separate, reviewed resize after memory verification. |
 | EBS and public IPv4 | gp3 GiB, IOPS/throughput if changed, snapshots, IPv4 hours | Valkey data occupies host EBS. |
 | RDS | `db.t4g.micro` hours, gp3 GiB, backups, I/O if applicable | Single-AZ is not a standby. |
 | S3 | artifact GiB-months, requests, retrieval/transfer | Versioning and lifecycle affect retained bytes. |

infra/optimized/README.md

@@ -9,16 +9,16 @@ repositories, VPC, EC2 host, standard RDS PostgreSQL database, S3 artifact
 bucket, Cognito pool, secrets, and deployment workflow. It never uses a
 Terraform remote-state reference to main.
 
-The runtime is deliberately single-host and single-AZ: one `t3a.medium`
+The runtime is deliberately single-host and single-AZ: one `t3a.small`
 EC2 host, local persistent Valkey, and one `db.t4g.micro`
 RDS instance. The database subnet group includes a second empty
 subnet in a second AZ because RDS requires it; it does not create a standby.
 There is no EKS, Aurora, RDS Proxy, MSK, ElastiCache, ALB, CloudFront, or NAT
 gateway.
 
-The host keeps 4 GiB of memory for the complete multi-process container set.
-Do not reduce it to `t3a.small` without measured memory headroom and an
-approved load test.
+The host target is 2 GiB. Apply the resize only after deploying an application
+release with working Valkey access, completing a report, and measuring host and
+container memory for 24 hours as described in the [runbook](./RUNBOOK.md).
 
 The host retains an Elastic IP because the production apex record points
 directly to it. There is no separate `optimized.tryindus.ca` hosted zone or

infra/optimized/RUNBOOK.md

@@ -189,11 +189,30 @@ backups and reconcile any writes made after the recovery point.
 
 ## Capacity, cost, and monitoring
 
-The configured `t3a.medium` and `db.t4g.micro` sizes require capacity monitoring.
-Measure memory, CPU credits, disk, database connections, latency, market
-ingestion, and Sidekiq backlog under representative load. The host retains
-4 GiB because the full process set has not been demonstrated within 2 GiB;
-resize only after measuring headroom.
+The target `t3a.small` host has 2 GiB of memory; the database remains
+`db.t4g.micro`. This repository change does not resize a running instance.
+First deploy an application release with working Valkey access. Confirm that
+Sidekiq stays running and a research report finishes. Then sample host
+`MemTotal` and `MemAvailable`, container memory, swap use, and OOM events
+through Systems Manager for at least 24 hours, including a report run. Keep
+the current 4 GiB host if peak `MemTotal - MemAvailable` exceeds 1.5 GiB,
+any process is OOM killed, or Sidekiq fails to drain. This preserves at least
+512 MiB of estimated headroom on a 2 GiB host. Check CPU credits, disk,
+database connections, market ingestion, and health endpoints as well.
+
+Only after those checks, dispatch `optimized-infrastructure` with `plan` from
+`main`. Verify the saved plan changes `aws_instance.host.instance_type` in
+place from `t3a.medium` to `t3a.small` and has no unrelated changes. A private
+`terraform.tfvars` value for `instance_type` overrides the repository default;
+update the private optimized Terraform variable file and its GitHub
+Environment secret if it still pins `t3a.medium`, then plan again. Dispatch
+`apply` and review its saved plan before approving the production environment.
+The instance type update stops and starts the EC2 host, causing downtime;
+the Elastic IP and EBS-backed Valkey volume remain attached to the host.
+After it starts, verify the instance type, systemd service, containers, report
+creation, and health endpoints. If memory pressure or OOM kills appear,
+restore `t3a.medium` in the optimized Terraform configuration and apply a
+reviewed rollback plan. The same stop/start downtime applies to rollback.
 
 Budget for EC2 and any burst credits, EBS, public IPv4, RDS compute/storage/
 backups/I/O, S3, Cognito and optional SES, ECR, KMS/Secrets Manager, Route 53,

infra/optimized/runtime/compose.yaml

@@ -82,6 +82,7 @@ services:
     restart: unless-stopped
 
   research-worker:
+    profiles: [temporal]
     image: ${RESEARCH_WORKER_IMAGE:?Set RESEARCH_WORKER_IMAGE to a digest-pinned image}
     command: ["./bin/research-worker"]
     env_file:

infra/optimized/runtime/indus-optimized.service

@@ -10,7 +10,7 @@ WorkingDirectory=/opt/indus-optimized
 EnvironmentFile=/opt/indus-optimized/host.env
 ExecStartPre=/opt/indus-optimized/install-release.sh /opt/indus-optimized/release.env --refresh-secrets
 ExecStart=/usr/bin/docker compose --env-file /opt/indus-optimized/release.env up --detach --remove-orphans
-ExecStop=/usr/bin/docker compose --env-file /opt/indus-optimized/release.env down
+ExecStop=/usr/bin/docker compose --profile temporal --env-file /opt/indus-optimized/release.env down --remove-orphans
 TimeoutStartSec=10min
 
 [Install]

infra/optimized/runtime/valkey.conf

@@ -3,6 +3,7 @@ appendfsync everysec
 save 60 1
 maxmemory 384mb
 maxmemory-policy noeviction
-protected-mode yes
+# Docker Compose does not publish this service's port outside its application network.
+protected-mode no
 bind 0.0.0.0
 port 6379

infra/optimized/terraform/environment/tests/optimized.tftest.hcl

@@ -57,8 +57,8 @@ run "single_host_cost_profile" {
     error_message = "The DB subnet group needs two AZ subnets without a standby."
   }
   assert {
-    condition     = aws_instance.host.instance_type == "t3a.medium" && aws_db_instance.this.instance_class == "db.t4g.micro" && aws_db_instance.this.apply_immediately == true
-    error_message = "The minimal profile must retain a 4-GiB host and apply its db.t4g.micro resize immediately."
+    condition     = aws_instance.host.instance_type == "t3a.small" && aws_db_instance.this.instance_class == "db.t4g.micro" && aws_db_instance.this.apply_immediately == true
+    error_message = "The minimal profile must use a 2-GiB host and apply its db.t4g.micro resize immediately."
   }
   assert {
     condition     = aws_db_instance.this.publicly_accessible == false && aws_db_instance.this.storage_encrypted == true

infra/optimized/terraform/environment/variables.tf

@@ -29,7 +29,7 @@ variable "alert_email_addresses" {
 }
 variable "instance_type" {
   type    = string
-  default = "t3a.medium"
+  default = "t3a.small"
 }
 variable "root_volume_size_gib" {
   type    = number

package.json

@@ -52,7 +52,7 @@
 		"date-fns": "^4.1.0",
 		"lightweight-charts": "^5.2.0",
 		"lucide-react": "^0.525.0",
-		"next": "15.5.24",
+		"next": "15.5.27",
 		"next-themes": "^0.4.6",
 		"pdf-lib": "1.17.1",
 		"react": "^19.1.0",

scripts/optimized/install-release.sh

@@ -46,7 +46,7 @@ install -d -m 0700 "$runtime_dir"
 available_kib="$(df -Pk "$release_dir" | awk 'NR == 2 { print $4 }')"
 [[ "$available_kib" =~ ^[0-9]+$ && "$available_kib" -ge 5242880 ]] || { echo "need at least 5 GiB free disk" >&2; exit 1; }
 memory_kib="$(awk '/MemTotal:/ { print $2 }' /proc/meminfo)"
-[[ "$memory_kib" =~ ^[0-9]+$ && "$memory_kib" -ge 3670016 ]] || { echo "need at least 3.5 GiB memory" >&2; exit 1; }
+[[ "$memory_kib" =~ ^[0-9]+$ && "$memory_kib" -ge 1835008 ]] || { echo "need at least 1.75 GiB memory" >&2; exit 1; }
 refresh_secrets() {
   for secret in platform-api market-data research-worker; do
     arn_file="$release_dir/${secret}.secret-arn"