Articles

SaltStack Interview Questions: States, Pillar Data, and Requisite Failures

Practice SaltStack interview questions by tracing states, Pillar values, require, watch, onchanges, and failures with concrete configuration examples.

SaltStack interview questions become more useful when you trace a state run instead of reciting command names. A configuration file can succeed without changing, a service can run without reloading, and a dependent state can fail because its prerequisite failed. Read each state’s result, changes, and comment before proposing another deployment.

This article separates official Salt behavior from original interview exercises and operational reasoning. Examples use synthetic services and data; they are not candidate reports or production-ready formulas. Check the Salt version and service provider used in your environment before relying on a provider-specific option.

The Salt-specific skill here is reading what a state actually returned.

What is the difference between execution modules and states?

Official fact: Salt distinguishes functions that execute operations from states that describe and enforce system conditions. The state tutorial demonstrates managing packages, files, and services through SLS definitions. In an interview, explain the desired condition before naming the implementation.

An execution call can ask for a service operation now. A service.running state asks Salt to ensure that the service is running. Repeated application should be interpreted through the particular state function's behavior; attaching an arbitrary command to a formula does not automatically make that command idempotent.

For an original example, imagine a worker that reads /etc/acme/worker.conf. Your desired conditions are a present package, a managed configuration, and a running service. “Run these three commands in order” misses a crucial question: should Salt attempt to start the service when installation fails?

A state declaration and a dependency serve different purposes. The declaration describes the condition. A requisite determines relationships between state executions. File order in the YAML is not a substitute for explicitly expressing the dependency that protects a later operation.

How do require, watch, and onchanges differ?

Official behavior: Salt's requisite reference says require gates execution on prerequisite success. watch also supports additional behavior through mod_watch; without a change, its ordinary state still runs. onchanges applies a state when its monitored prerequisites succeed and a relevant change occurs. A module without mod_watch treats watch like require.

Use a short definition, then trace a concrete run. The following original trace matrix assumes one successful configuration prerequisite and no other blockers. The watch example also assumes the service is already running, the normal service state makes no changes, and the provider supports the requested reload behavior.

Configuration result	require on service.running	watch on service.running	onchanges on a validation command
Success, no changes	Check that the service runs	Check that the service runs; no change-triggered reload	Skip the command
Success, changed file	Check that the service runs	Check service, then request change-triggered reload	Execute the command
Failure	Dependent execution blocked	Dependent execution blocked	Dependent execution blocked

“Watch means restart whenever Salt runs” is therefore the wrong answer. Equally, a successful unchanged file does not mean a stopped service should remain stopped under watch. The service's normal condition still matters.

For onchanges, describe the trigger separately from the command's success. The file changing makes the validation eligible to run. It does not make a failing validator successful. Inspect both returned results so a successful file update does not hide a failed validator.

With multiple prerequisites, name the exact requisite type and its success/change logic. Do not extrapolate a single-target fixture into every _any variant. The exercise above deliberately uses one configuration target so the difference is visible without hiding additional conditions.

Trace a failed configuration dependency

Here is an original illustrative SLS, not a formula to apply to your machine. It assumes an existing acme-worker service and a managed template in the Salt file server.

worker-config:
  file.managed:
    - name: /etc/acme/worker.conf
    - source: salt://acme/worker.conf.jinja
    - template: jinja

worker-running:
  service.running:
    - name: acme-worker
    - reload: true
    - watch:
      - file: worker-config

The target file: worker-config names the file state declaration. Read that target as a relationship, not as a filesystem path to create. Salt can match requisite identifiers and names, but consistent explicit IDs make an interview trace easier to follow.

Suppose the file state returns result: false with a comment that its source cannot be retrieved. The service state then reports a requisite failure. The immediate defect is not necessarily the service manager. Start with the failed source lookup, selected environment, and relevant file-server availability.

Now change the fixture: the file returns result: true and an empty changes mapping, but the service is stopped. The service state should still enforce running. If it starts the service, read the service result and changes before claiming a separate reload occurred.

Finally, let the file succeed with a changed checksum while the service is already running and its normal check changes nothing. Under the stated assumptions, the watch path can invoke the service's reload behavior. Verify the command supported by the service provider; a reload option does not prove the application has adopted every new setting.

Why can watch fail to produce the reload you expected?

The documented default has a subtle condition: when the normal watching state itself returns changes, the additional watch handler can be suppressed. Salt’s current state-engine source also contains an explicit opt-in exception, so check the installed version and state module rather than treating this as an unconditional rule. The official reference illustrates separating service enablement from running-state enforcement. That is why a configuration-change story needs more than the sentence “the file changed.”

In an interview, use an evidence chain. First establish whether the file changed. Next establish whether the watching state succeeded and whether its normal execution already changed something. Then inspect the provider and the requested operation. Each answer narrows a different cause.

Consider an original scenario in which a service is running but disabled at boot. A combined state that enables it can return changes during ordinary execution. Separating enablement into service.enabled and running into service.running can make the watched running condition easier to reason about. Confirm the complete dependency structure and rerun in a disposable environment; do not present this split as a universal repair.

The service state documentation is the place to verify the installed version's options. Some applications support reload, while others require restart or accept only a subset of configuration changes without restarting. A healthy process alone does not prove that the application has loaded the intended configuration.

Ask what observation would prove success. For the synthetic worker, that could be a status endpoint showing the expected nonsecret configuration version, plus an accepted test job. A successful Salt result is useful deployment evidence, but the application-level behavior still needs an appropriate check.

What belongs in Pillar, and why can its value appear stale?

Official fact: Salt's Pillar walkthrough describes data compiled for targeted minions and used for configuration and templates. It also distinguishes inspecting freshly compiled Pillar from refreshing the minion's in-memory Pillar. Do not treat “the source file changed” as proof that every consumer now sees the new value.

For an original configuration exercise, the intended nonsecret value is the nested worker:queue field set to slow_jobs. A rendered configuration still contains fast_jobs. Trace the stages: the intended source, targeting, compiled data for the particular minion, the consumer's view, the rendered file, and the application's active configuration.

Compare the intended queue value with the compiled value for one named minion first. If another minion receives the expected value, that points toward targeting or environment differences, not necessarily a global template bug. If compiled data is correct but rendering is wrong, inspect the lookup path and any fallback. If the rendered file is correct but the worker still uses the old queue, investigate the reload or restart boundary.

Nested data introduces another interview trap. A key literally containing a colon and a nested dictionary path are not interchangeable without considering the lookup operation. Explain the data shape, then show how the template accesses it. A fallback default can conceal a misspelled or absent required key.

Pillar is suitable for sensitive configuration, but targeted distribution is not a reason to dump the whole structure into an interview transcript or shared log. Inspect a specific nonsecret field, redact credentials, and consider where compiled data and rendered files are stored. Do not put a secret value into a diagnostic example when its presence or type answers the question.

How would you diagnose a failed state run safely?

Start by separating rendering, compilation, execution, and application behavior. A template error can prevent the intended state from reaching execution. A missing requisite can invalidate the dependency graph. A successfully compiled state can then fail when it contacts a package repository or service manager.

The state execution module documents inspection functions such as state.show_sls and application functions such as state.apply. Use the former to understand compiled declarations before changing the target. These are diagnostic tools, not permission to run a broad deployment against an unknown fleet.

For a practice environment, compare the compiled graph with the intended three conditions: package present, config managed, worker running. Then read each returned state result, changes, and comment. A result of None in test mode expresses an anticipated change rather than the success of an applied operation.

A dry run can help predict work, but it cannot establish that a real repository download, service reload, or downstream application request will succeed. State what remains untested. Also avoid assuming automatic transaction rollback: earlier successful changes may remain after a later state fails.

An interviewer may show two red results and ask which one to fix. In an original trace, worker-config cannot fetch its template, while worker-running says a requisite failed. Explain that the second error follows from the first. Then ask whether an unrelated service state also failed independently. A run can contain both propagated failure and a separate defect; grouping every red result under one cause is another unsupported shortcut. Preserve the original state IDs and comments in your explanation so a reviewer can check that causal claim.

Recovery should follow the smallest failed boundary. If the source file is absent, repair source availability. If an application rejects the generated configuration, preserve evidence of the rejection and correct the template or data. If a service operation fails, inspect its provider and logs. When a rerun produces another wall of output, the original failure can disappear among repeated side effects.

Five questions for configuration and dependency reasoning

Question	Connection to this topic
Design a configuration subscription service	Separate a changed source from a consumer that has received and applied it.
Design a device configuration system	Reason about targeting, rollout, and device-level evidence.
Validate Nested Configuration Objects	Identify absent required values and invalid nested shapes.
Design Safe Configuration and Deployment for Hundreds of Services	Define rollout boundaries and meaningful verification.
Design a Configurable Order Validator with DAG Dependencies	Explain a dependency graph and how failed prerequisites affect later work.

Choose one configuration scenario from Software Engineer fundamentals practice, then write three possible outcomes: success without changes, success with changes, and failure. Start with one prerequisite and explain which operation runs next in each case. That is a practical way to replace vocabulary recall with a reviewable state trace.