Articles
CloudFormation Interview Questions: Change Sets, Drift, and Rollback Recovery
Practice CloudFormation interview questions with replacement review, drift-detection limits, and rollback recovery decisions in a worked stack timeline.
A CloudFormation interview answer should keep three kinds of evidence separate. A change set previews what a proposed template or parameter update would do; drift detection compares supported live resource properties with the stack’s expected configuration; stack events show what happened during execution and rollback. None is a universal guarantee that an update will succeed. If a stateful resource may be replaced or a rollback is failing, the next step is an evidence-based stop and recovery decision, not another blind deploy.
Official AWS facts below come from current CloudFormation documentation, including its newer drift-aware change sets. The OrdersStack timeline and replacement decision are editorial exercises, not an executed AWS deployment or a report of a company’s interview loop.
What does a change set actually preview?
AWS’s change-set guide says creating a change set compares a proposed template or parameter set with the stack and lists intended resource actions before execution. Creating the change set does not itself update the stack. The reviewer can inspect Add, Modify, and Remove actions and decide whether to execute it, revise the proposal, or stop.
Look beyond the action label. A Modify can require replacement, cause interruption, or be an in-place update depending on the resource property. AWS’s change-set examples explain Replacement values and static versus dynamic evaluation. A Conditional replacement result is a prompt to inspect the specific property and dependency, not permission to assume “probably safe.” For a data-bearing bucket or database, identify the current physical resource and retention policy before executing.
AWS explicitly says change sets do not guarantee a successful update. Runtime conditions, service quotas, naming conflicts, custom-resource logic, and dependency behavior can still fail. That is why a preview belongs in an approval workflow with a rollback plan and event monitoring rather than being treated as a simulation of every service action.
A replacement-risk review on an example stack
Consider a hypothetical OrdersStack with an application service, a retained export bucket, and a database. A proposed template changes a harmless service tag, changes a database property that the resource specification marks as replacement-requiring, and adds a security-group rule. The change set lists Modify for the service, Modify with replacement risk for the database, and Add for the rule. The exact resource types and fields would need to be checked in the real AWS template; this is a decision fixture, not a claim about a particular property.
| Proposed change | Review question | Decision before execute |
|---|---|---|
| Service tag | Is there an unexpected dependent change? | Verify scope, then proceed if within policy. |
| Database property | Does the resource specification require replacement? | Stop for data migration, backup, cutover, and rollback review. |
| Security-group rule | Which principal, port, and direction change? | Confirm least privilege and expected traffic. |
| Export bucket unchanged in template | Could another change still affect it? | Inspect references, stack events, and drift where relevant. |
The first article-specific asset is the review ledger: action, replacement risk, physical resource identity, data consequence, and owner decision. “The change set is green” is not a meaningful approval criterion. If the database replacement is unacceptable, revise the template and create a new change set; do not execute and hope rollback will preserve the original data path. AWS’s update-behavior guide explains that replacement usually creates a new physical resource, changes references, then removes the old one, subject to service-specific behavior.
How is drift different from a proposed change?
Drift is a difference between actual managed-resource configuration and the values CloudFormation expects from its template and parameters. AWS’s drift-detection guide describes stack and resource statuses such as DRIFTED, IN_SYNC, and NOT_CHECKED. A direct console edit to a resource can create drift even when no new template is proposed.
A crucial limitation: CloudFormation checks supported resource types and only property values explicitly set in the template or parameters. A property left at its service default may not be included in the comparison. Certain values cannot be read back or mapped from the service. A nested stack is not automatically examined when detecting drift on its parent; run detection on the nested stack itself where appropriate. Therefore IN_SYNC means “no detected difference within the supported, checked surface,” not “every byte of the account matches our mental model.”
For the OrdersStack fixture, suppose an operator changed an explicitly declared security-group ingress rule during an incident. Drift detection may report the rule as modified if supported. Before overwriting it, ask who made the change, whether the incident is still active, and whether the template should adopt or revert it. A drift result is evidence of a difference, not automatically evidence of an error.
Traditional versus drift-aware change sets
This distinction has changed over time and matters in a current interview answer. AWS now documents drift-aware change sets. A traditional change set compares the new desired template with the previous deployment template; it does not account for out-of-band drift. A drift-aware change set performs a three-way comparison among actual state, previous deployment state, and desired state for supported resources.
That added visibility changes the decision. If an emergency security-group edit exists in actual state but not in either template, a drift-aware preview can show whether execution would revert it. AWS says drift-aware execution may update resources to the desired state even when a template line did not change. The reviewer must examine its before-value source and ignored-property indicators, not assume “unchanged template means unchanged live resource.”
Do not overclaim the new mode. AWS lists unsupported resource types, write-only property limits, AWS-managed properties it preserves, and immutable-property restrictions. For unsupported types it can fall back to comparing previous deployment state with desired state. A three-way preview is stronger evidence for supported properties, but it is still not a proof of runtime success or a replacement-free update.
Read stack events as a timeline
After execution, CloudFormation stack events show resource and stack transitions. Do not jump straight to the last stack-level failure line. Find the earliest resource failure in the relevant operation, read StatusReason, then trace dependent events and rollback attempts. AWS’s troubleshooting guidance describes filtering by operation ID and examining service or CloudTrail details when needed.
Use this illustrative timeline: 10:00 change set execution begins; 10:01 the security-group update completes; 10:02 a new database replacement resource fails to create because of a name or quota condition; 10:03 rollback starts; 10:05 the old database or a dependent resource cannot be restored because its expected state was changed outside CloudFormation. The exact AWS order and statuses depend on the real stack. The timeline distinguishes the forward-update failure from the later rollback failure. A single “deployment failed” label conceals the two repair tasks.
The second article-specific asset is an event ledger with columns for operation phase, logical ID, physical ID, status, reason, and next evidence source. Physical IDs matter because replacement can change the underlying resource even when the logical name remains familiar. Keep the original template, parameter values, change-set ID, event export, and relevant service logs available for review before proposing another update.
What does UPDATE_ROLLBACK_FAILED require?
AWS’s continue-update-rollback guide says a stack in UPDATE_ROLLBACK_FAILED cannot accept normal updates until rollback returns it to an operable state. Usually the operator must first fix the cause that blocks rollback, then continue the rollback. In some cases a transient timeout resolves without a template change. The correct action depends on the failing resource event.
The guide allows skipping resources that failed during the rollback, but warns that skipped resources are marked complete while actual state can remain inconsistent with the template. Skip only the minimum eligible resources as a last resort and plan reconciliation before the next update. It is not a generic “force green” command. AWS also has specific rules for nested-stack skip identifiers; copying a root-resource recipe into a nested stack without checking the state can fail or worsen confusion.
Distinguish this from a failed create operation and from a deliberately chosen preserve successfully provisioned resources mode. Different stack statuses allow different recovery operations. State the current status and event reason before naming a CLI command. The interview answer should show an operator who protects data and understands stack-state constraints, not someone who memorized continue-update-rollback for every failure.
A safe recovery decision for the fixture
For the hypothetical database failure, first stop new deployments to the stack. Read the operation’s events and identify the first forward failure and first rollback failure. Confirm the physical database resources, backups, and traffic path. If rollback is blocked by a recoverable permission, quota, or missing dependency, fix that prerequisite and continue rollback. Verify the final stack status and actual resource health before resuming application traffic changes.
If the original database no longer exists, do not tell CloudFormation it is healthy without understanding how the app will recover. Review restoration, import, or replacement options with the owning team. Skipping a rollback resource may get the stack out of UPDATE_ROLLBACK_FAILED but leaves a reconciliation obligation. The exact remedy is environment-specific; this article does not prescribe live commands for an unknown production stack.
After recovery, run drift detection for supported resources and inspect critical properties the detector cannot cover. Make the template, actual resource state, and intended application behavior agree. Then create a fresh change set, review replacement implications again, and execute through normal controls. A successful stack status alone is not proof that the app points to the intended database.
Fast answers to CloudFormation follow-ups
Does a change set execute itself? No. Creation is a preview; execution applies the proposed operation.
Can Modify still replace a resource? Yes. Read Replacement and property-level update behavior, particularly for stateful resources.
Does IN_SYNC mean there is no drift anywhere? No. Unsupported resources, omitted/default properties, and certain unreadable values are outside the check.
Do traditional change sets include out-of-band edits? Traditional comparison is template-to-template. Drift-aware change sets add actual-state comparison for supported surfaces.
Is continue-update-rollback always safe? No. Fix the blocking cause first where possible; skipping a resource can leave the stack and actual infrastructure inconsistent.