Add the database monitoring agent Terraform module

[anchoo2kewl]

Adds database-monitoring/terraform: the module that runs the Elastio database monitoring agent on AWS Fargate (elastio/blue-stack#17072). It is published as aws-elastio-database-monitoring-agent 0.1.0, so the registry source is terraform.cloudsmith.io/public/elastio-database-monitoring-agent/aws.

Source: deploy/terraform/aws-fargate at tag agent-v0.1.3 in the old agent repository, adapted for this repo.

Changes from the source module

• The image default is now public.ecr.aws/elastio/elastio-database-monitoring-agent:0.1.4. That image is not published yet.
• All customer-visible names now use the elastio-dbmon prefix:
  • log group /elastio-dbmon/agent/<name>
  • secrets elastio-dbmon/<name>/{api-key,database-url,hash-secret}-*
  • roles elastio-dbmon-<name16>-{exec,task}-*
  • EFS security group elastio-dbmon-<name16>-efs-*
  • EFS Name tag elastio-dbmon-<name>-ledger
  • access point root /elastio-dbmon
  • inline policies read-elastio-dbmon-secrets and mount-elastio-dbmon-ledger
  • container elastio-dbmon-agent
  • resource descriptions
• The name used in IAM role prefixes is cut to 16 characters. With the new prefix, the old 20-character cut exceeded IAM's 38-character limit for name_prefix.
• The EFS creation_token is removed and the provider generates one. The old token was built from the full name, so it broke EFS's 64-character limit for long names.
• IAM policies are now jsonencode, not data "aws_iam_policy_document", to follow this repo's convention (Replace data "aws_iam_policy_document" with manual jsonencode #109).
• The log driver region is now read from the log group ARN. data.aws_region's name and id are both deprecated in AWS provider 6, and required_providers is now aws >= 5.0 / random >= 3.0 (the relaxed constraints from Relax TF provider version requirements #121). required_version is ~> 1.9.
• Kept unchanged:
  • Variable names. The Elastio Portal renders a module call from them.
  • The QUELL_* environment variable names. They are the agent binary's configuration contract.
  • The in-container ledger path /var/lib/quell. It is the image's own volume, owned by the non-root user, and the ephemeral (non-EFS) mode depends on it.
• There is no moved {} block. Resource addresses are the same as before; only the AWS-side names changed.
• Added examples/basic (with its lock file, as the other examples have), a names_and_image test, and a region assertion.

Checks run locally

• terraform init -backend=false && terraform validate: valid, no warnings (AWS provider 6.66.0)
• terraform test: 4 passed, 0 failed
• terraform validate in examples/basic: valid
• terraform-docs v0.19.0, then prettier --check .: clean
• tflint --recursive (repo .tflint.hcl): clean
• typos: clean
• terraform fmt: clean on the new files

Task size (commit 387bdde)

• New inputs task_cpu (default 256) and task_memory (default 512, MiB). The product renders these names. Validation accepts only pairs that Fargate runs on ARM64: 256 with 512, 1024 or 2048; 512 with 1024 to 4096; 1024 with 2048 to 8192; 2048 with 4096 to 16384. Memory above 512 MiB goes in steps of 1024.
• GOMEMLIMIT is floor(task_memory * 0.8) MiB. The agent's budget for one open transaction is 30% of that, so the budget follows the size.
• The default image is now agent 0.1.6 (elastio/database-monitoring-agent#2). That release judges a transaction too large to hold from its row counts instead of crashing, and reports the task memory that would have held it. It also finds its memory limit on Fargate, which 0.1.5 did not. Tag agent-v0.1.6 before this module is released.
• README gains a Sizing section: the presets small 256/512, medium 512/1024, large 1024/2048 and xlarge 2048/4096; about how large a transaction each holds; the monthly cost in us-east-1 for ARM Fargate ($7.21 / $14.42 / $28.84 / $57.67); and "the Elastio UI tells you when to size up".
• Tests: 14 pass. Each preset and one in-between size are planned, with their GOMEMLIMIT. Pairs Fargate does not run are refused at plan. The default image is checked as 0.1.6.
• Checks: terraform fmt, validate (module and example), test, tflint --recursive, terraform-docs v0.19.0 plus prettier, and typos, all clean.

Release

After merge, someone with write access runs the release workflow (workflow_dispatch) on master. It publishes only modules whose name-version tag doesn't exist yet, so only this module is published.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[anchoo2kewl]

Pushed 98276af for agent 0.1.5 (elastio/database-monitoring-agent#1): the module now sets the ELASTIO_DBMON_* variables, mounts the ledger at /var/lib/elastio-dbmon and passes that path explicitly, so the EFS access point holds the same ledger file. It also sets GOMEMLIMIT=409MiB (80% of the 512 MiB task) and defaults the image to 0.1.5. The image needs 0.1.5 or later, since older agents read only QUELL_*. terraform test passes 5/5, and the README was regenerated with terraform-docs 0.19 and prettier.

[maksvet]

Checked the 0.1.6 image (VOLUME and owner at /var/lib/elastio-dbmon) and ran this head in a test account: both ledger modes start and open the ledger, and the agent reads its limits from ECS metadata.