feat(cloudformation): change sets for CREATE and UPDATE (CFN-4)

[NitinKumar004]

Summary

Adds CloudFormation change sets for ChangeSetType CREATE and UPDATE. aws cloudformation deploy now works end to end.

• New ops: CreateChangeSet, DescribeChangeSet, ListChangeSets, ExecuteChangeSet, DeleteChangeSet. Also GetTemplateSummary, because deploy calls it before it updates an existing stack.
• CREATE on a free name makes a REVIEW_IN_PROGRESS stack with its "User Initiated" event. More CREATE change sets can join that stack. CreateStack and UpdateStack refuse it. Deleting its last change set leaves the stack in REVIEW_IN_PROGRESS, and DeleteStack removes it. GetTemplateSummary on such a stack summarises the template of its latest change set.
• UPDATE plans with the same planUpdate and noChanges code that UpdateStack uses. An empty plan gives a FAILED change set with the real "The submitted information didn't contain changes" reason.
• Execution runs on the feat(cloudformation): property-level update diff, UPDATE_ROLLBACK_FAILED and ContinueUpdateRollback #1352 update engine. UPDATE goes through the shared runUpdate and CREATE goes through the shared provision, so the change set path and the direct stack path use the same code.
• Changes[] come from a new provider-neutral planner, services/cloudformation/changeplan.go:
  • Actions are Add, Modify and Remove. Modify reports Replacement True, False or Conditional, plus Scope and PolicyAction (ReplaceAndDelete or Delete).
  • Each Details entry has a Target (Attribute, Name, RequiresRecreation) and a ChangeSource of DirectModification, ParameterReference, ResourceReference or ResourceAttribute, with its CausingEntity.
  • RequiresRecreation comes from the replacement tables in feat(cloudformation): property-level update diff, UPDATE_ROLLBACK_FAILED and ContinueUpdateRollback #1352.
  • A change reaches dependent resources in dependency order. A Ref counts only when the referenced resource may be replaced or is new. A GetAtt counts when the resource changes at all.
  • IncludePropertyValues returns BeforeValue, AfterValue and the contexts. A value only known at execution shows {{changeSet:KNOWN_AFTER_APPLY}}, both as AfterValue and inside AfterContext, never its stale resolved value.
• Status lifecycle: CREATE_COMPLETE or FAILED, then AVAILABLE, EXECUTE_IN_PROGRESS, and EXECUTE_COMPLETE or EXECUTE_FAILED.
  • A direct UpdateStack marks the stack's open change sets OBSOLETE.
  • Executing deletes the stack's other change sets. The status check and the stack status change happen under one lock, so two concurrent executes cannot both run.
• Idempotency: a CreateChangeSet retry with the same name and ClientToken returns the same Id. A different token, or no token, gives AlreadyExistsException. An ExecuteChangeSet retry with the same ClientRequestToken succeeds without running the change set again.
• OnStackFailure:
  • ROLLBACK is the default.
  • DO_NOTHING leaves CREATE_FAILED or UPDATE_FAILED with DisableRollback set. Both statuses can be updated again. A failed update that is not rolled back keeps the old physical resources of replacements and records them, including in the snapshot. Only the cleanup phase of the next successful update, or DeleteStack, deletes them. Cleanup never deletes a retained resource whose physical ID the stack holds again, and an update back to a retained resource's name takes it back instead of failing, unless it was deleted out of band, in which case it is created again. The stack keeps the submitted template verbatim, as AWS does, so GetTemplate and Terraform see the body they sent. The next update, and its rollback, diff against each resource's last applied properties and type, so a retry with --use-previous-template applies the resources that failed or never ran.
  • DELETE is valid for CREATE only.
  • DisableRollback on ExecuteChangeSet, and on UpdateStack, maps to DO_NOTHING.
• Errors:
  • ChangeSetNotFound (HTTP 404) and InvalidChangeSetStatus.
  • AlreadyExistsException for a duplicate name.
  • ValidationError for a missing stack, for CREATE over a live stack, for a bad name or enum value, and for a name given with no StackName.
  • DeleteChangeSet by name on a live stack succeeds when the change set is already gone. The CDK relies on this.
• Pagination: ListChangeSets returns 100 per page. DescribeChangeSet pages its changes at 1 MB.
• Change sets are included in the persist snapshot. DescribeStacks now reports ChangeSetId and DisableRollback.
• Pre-existing data-loss bug, fixed here: provisioners never adopt a resource that already exists under a custom name. SQS used to reuse an existing queue with the same name, so a failed stack's rollback could delete another stack's queue. All 8 types (S3 bucket, DynamoDB table, SQS queue, SNS topic, Lambda function, IAM role, secret, SSM parameter) now fail to create with " already exists".

Closes

• CFN-02 (plan item CFN-4)

Testing

• Unit tests: a planner matrix in services/cloudformation, plus provider tests covering the lifecycle, errors, OnStackFailure, retained replacements, client tokens, pagination, snapshot round-trip and a concurrent execute.
• A provider-level test, run for each of the 8 types, shows that another stack's resource survives the rollback.
• Server tests with the real SDK for the op shapes, typed errors, client tokens and the change set waiter.
• The compat CFN suite is at 17/17.
• Gates: go build, then go vet and go test -race on the touched packages. golangci-lint --new-from-rev reports 0 issues. coveragegen and compatgen were regenerated.
• e2e against cloudemu serve with the aws CLI:
  • deploy covered create, "No changes to deploy", and an update with a parameter change and a queue rename.
  • create-change-set covered CREATE and UPDATE with a modify and a replacement. describe-change-set returned the expected Changes, with and without --include-property-values.
  • Also covered: execute, cleanup of the other change sets, the no-change FAILED case, delete, and deleting a review stack's change set.
  • Review repros, all passing:
    • A DO_NOTHING update that replaces a table and fails leaves UPDATE_FAILED, and the old table keeps its item.
    • After that failure, deleting the retained table out of band and updating back recreates it, and the cleanup keeps it.
    • Updating back while the retained table still exists takes it back with its item. A failing update onto it rolls back to UPDATE_ROLLBACK_COMPLETE and keeps the item.
    • After a DO_NOTHING failure, deleting the blocking table and running update-stack --use-previous-template --disable-rollback completes the update. GetTemplate returns the submitted YAML verbatim.
    • Terraform: after an out-of-band update-stack --disable-rollback fails with a commented YAML body, terraform plan with that same body is clean.
    • ClientToken and ClientRequestToken retries behave as described above.
    • A stack that tries to take the queue cq rolls back with "cq already exists", and the owner's queue survives.
• e2e with Terraform aws_cloudformation_stack: apply, then a clean plan, then an update with an in-place change and a replacement, then a clean plan, then destroy.

Deferred

• ChangeSetType IMPORT and ImportExistingResources, planned as CFN-8. IMPORT currently returns a ValidationError.
• An async window for CREATE_PENDING and CREATE_IN_PROGRESS under --async-settle, together with async stacks in CFN-6. Change sets are planned synchronously, as stacks are.
• IncludeNestedStacks, planned as CFN-10. Stack policy enforcement on execute, planned as CFN-6.
• A RollbackStack op to roll an UPDATE_FAILED stack back to the resources it retained.
• RetainExceptOnCreate on UpdateStack and ExecuteChangeSet, which depends on DeletionPolicy (CFN-5).
• TokenAlreadyExistsException for a token reused across different operations.
• GetTemplate with ChangeSetName.
• Stack tags that change are not yet listed as Tags-scope changes on each resource.
• Some error texts could not be checked against a live account: the OnStackFailure DELETE on UPDATE message, the message for OnStackFailure together with DisableRollback, and the message for deleting a change set during its execution.