cswork / libraryOpen practice workspace

Articles

HashiCorp Packer Interview Questions: Image Builds, Provisioners, and Reproducibility

Practice HashiCorp Packer interview answers with builder and provisioner lifecycle, failed builds, pinned inputs, manifests, and image promotion.

“We rebuilt the same Packer template twice and got different VM images. Is Packer broken?” An interview answer should begin with the inputs. The template may be identical while the base image selected by most_recent, a package repository, an install script, or a plugin version changes between runs. Packer automates a machine-image build; reproducibility requires controlling and recording everything that build consumes.

This guide uses an original golden-image build manifest, a failed provisioner trace, and a publish gate. Evidence boundary: HashiCorp Packer documentation supports build blocks, builders, provisioners, plugins, and manifest behavior. The failure case and release policy are editorial examples. No candidate report is used to claim a specific employer asks them.

What does Packer produce?

HashiCorp Packer builds machine images from a template. A builder creates and captures an artifact for a target platform, such as an AWS AMI or another VM image. Provisioners configure a running build machine after it boots—for example, installing packages, copying files, or running shell/PowerShell commands. Post-processors act on produced artifacts after the build, such as writing a manifest. HashiCorp's HCL build block ties sources, provisioners, and post-processing together. This lifecycle is the answer's backbone.

Do not describe Packer as a runtime configuration controller. Once an image is built, launching it and operating the resulting machines are separate concerns. A baked image can reduce launch-time configuration variance, but it can also become stale if promotion and retirement are not managed. The interviewer's question about “reproducibility” is not answered by saying “images are immutable”: an immutable output can still come from floating inputs.

The exercise targets one platform to keep the reasoning testable: an AWS EBS-backed AMI for a service fleet. It starts from an approved base AMI ID, installs a pinned application package and OS updates from a controlled snapshot, runs a health test, removes temporary credentials, then produces an AMI and a manifest. The exact AMI, package, and account identifiers are placeholders; this is not a ready-to-run cloud template.

Worked manifest: find what is actually pinned

Suppose a pipeline records these fields for two runs:

Input or outputRun ARun BInterpretation
Git template commitc8e2c8e2Template is unchanged.
Builder plugin constraint>= 1.0>= 1.0Actual resolved plugin may differ.
Base image selectormost_recent=truemost_recent=trueA new base AMI may be chosen.
Package commandapt install app-serversame commandRepository state may differ.
Output AMI IDami-Aami-BDifferent IDs are normal per build.
Installed app version4.2.14.2.2Content changed; investigate inputs.

The only proven sameness is the template commit and command text. HashiCorp's Amazon builder docs show filters with most_recent; that option intentionally selects the newest matching source, so it trades stability for freshness. HashiCorp's plugin-install docs explain that packer init installs the latest plugin satisfying the version constraint. A lower-bound constraint is not an exact lock. The package command also lacks a version or repository snapshot. Thus a different output is expected, not evidence by itself of Packer nondeterminism.

Repair the manifest before changing Packer code: record the resolved base AMI ID, plugin binary/version, Packer CLI version, repository snapshot, package versions, script checksums, environment and region, and output artifact ID. Pin inputs where operationally appropriate; when inputs intentionally move for security patches, make the change explicit and test it. Reproducibility does not mean refusing updates forever. It means being able to explain and rebuild a known generation, or explain why a newer generation differs.

Two builds with the same inputs may still have different raw disk hashes because timestamps, logs, generated IDs, or filesystem metadata vary. Define the requirement precisely: bit-for-bit identity, equivalent installed software and configuration, or repeatable build procedure. For many VM fleets, a verified software inventory, configuration assertions, and behavior tests are more useful than insisting AMI IDs match. State what the downstream rollout needs.

The HCL structure an interviewer expects

A small template can show the main relationships without hiding behind a huge provider example:

packer {
  required_plugins {
    amazon = {
      source  = "github.com/hashicorp/amazon"
      version = "= 1.3.9" # Illustrative; verify a real supported version.
    }
  }
}

variable "base_ami_id" { type = string }

source "amazon-ebs" "service" {
  region        = "us-east-1"
  source_ami    = var.base_ami_id
  instance_type = "t3.small"
  ssh_username  = "ubuntu"
  ami_name      = "service-{{timestamp}}"
}

build {
  sources = ["source.amazon-ebs.service"]

  provisioner "shell" {
    script = "scripts/install-and-test.sh"
  }

  post-processor "manifest" {
    output = "manifest.json"
  }
}

This is an illustrative skeleton, not a validated account-specific build. A real AMI build must also address credentials, subnet, security group, disk encryption, instance profile, package source, and cleanup under the chosen plugin version. packer init obtains plugins and packer validate checks template configuration, but neither proves the provisioner tests the application or that the resulting image is safe. Running a build requires cloud access and incurs resources; this article does not claim to have executed one.

The build block is where provisioners and post-processors are ordered. HashiCorp says provisioner and post-processor block order matters. If the install script fails before the manifest step, do not assume a publishable artifact was produced. If a post-processor fails after image capture, decide whether the AMI exists and must be quarantined or cleaned up; “pipeline red” and “no cloud artifact” are different observations.

Diagnose a failed provisioner without losing the evidence

Imagine the shell provisioner exits nonzero while installing app-server. A strong answer captures the failing command, exit status, selected base image, package repository response, and build instance identity. Then it determines whether Packer aborted before artifact creation, whether a temporary instance or snapshot remains, and whether any secret or temporary file was copied into the guest. HashiCorp's packer build documentation discusses error handling and keeping a failed instance for debugging under explicit options; that is a controlled investigation choice, not a default production cleanup policy.

Do not fix the failure by appending || true to the install line. That can turn a bad image into an apparently successful artifact. If transient repository failure is plausible, make retries bounded and observable. If the base AMI's package manager changed, update the script and the base-image contract. If a secret file is required during provisioning, prefer short-lived retrieval and ensure no secret remains in disk layers, shell history, logs, or captured image. A “cleanup provisioner” may never run after an earlier failure; design secret handling so failure is safe.

The release gate should separate build, test, and promotion. Capture the AMI ID into an isolated candidate channel, launch a test instance from it, verify service startup, security baseline, package inventory, and network behavior, then promote the tested artifact ID. If the manifest is missing or inconsistent, hold promotion even if an AMI exists. A later rebuild with the same template is a new artifact requiring testing; do not swap production to a floating “latest” image selector without traceability.

Use the manifest as evidence, not as a magic lockfile

HashiCorp's manifest post-processor writes JSON describing artifacts from builds, including build time and artifact IDs. It helps downstream automation consume the exact output. It does not pin upstream inputs by itself. If multiple builds append to one manifest, choose the intended entry by explicit build name/time and verify the ID; do not pick the last line casually. The documentation notes that -force can truncate a manifest, which matters if a pipeline expects history in that file.

For the exercise, add an external build-attestation record beside the Packer manifest: template commit, base AMI ID, plugin and CLI versions, script checksums, package inventory, security scan result, test instance result, artifact ID, and promotion time. That record is an editorial design, not a built-in guarantee of the manifest post-processor. Store it where promotion can verify it and where rollback can identify the previous good image.

Rollback means redeploying or relaunching the previous known-good AMI, not rerunning the old template and hoping the same result appears. If package repositories or base selectors have moved, a rebuild can differ even when the template commit is old. Keep the prior artifact available for the intended rollback window, with access controls and deprecation rules. Confirm that data and schema migrations are compatible with rollback; an old image may not understand a new database state.

Reproducibility without freezing vulnerability fixes

There is a tension between pinning and patching. An exact base AMI and package snapshot make a generation explainable, but an image can age. Establish a cadence that resolves updated inputs deliberately, builds a new candidate, scans and tests it, then promotes that exact ID. Track which fleet instances still run older generations. A security patch should be visible as a controlled input change, not as surprise drift on a supposedly repeatable nightly build.

Use consistent region and architecture. An AMI ID is regional; replicating an image to another region produces a different artifact ID and can involve different account permissions or encryption settings. HashiCorp Packer can automate multi-platform builds, but a single HCL template does not guarantee identical software across different builders or architectures. Compare the properties that matter to your workload and validate each target image.

The same reasoning applies to Docker outputs but the mechanisms differ. This article is about HashiCorp Packer machine-image construction, not warehouse packing jobs and not a general Docker layer-cache tutorial. If the interviewer asks how Packer relates to Terraform, say Packer produces a tested image artifact while infrastructure deployment tools can reference its exact ID. Keep image baking and fleet rollout as separate, observable stages.

Fast interview answers to common follow-ups

What are builders, provisioners, and post-processors? Builders create the target image artifact; provisioners configure the running build machine; post-processors act after an artifact is produced. Name the actual plugin and output platform.

Why did identical HCL produce different images? Check resolved base image, plugins, packages, scripts, clock-dependent data, and region. Identical source text does not imply identical inputs.

What should happen after a shell failure? Treat the candidate as failed, preserve enough logs to diagnose it, inspect temporary cloud resources, and ensure secrets are not left in the guest or artifact. Do not publish an untested AMI.

Does packer validate prove reproducibility? No. It checks template configuration. Only a recorded input manifest and repeated, tested build procedure support a reproducibility claim.

How do you roll back? Keep and reference the previous tested artifact ID, and verify application/data compatibility. Rerunning yesterday's template is not the same as restoring yesterday's image.

Sources and further reading