Saga Interview Questions: Compensation, Irreversible Steps, and Stuck Workflows

A purchase reserves inventory, captures payment, and then fails before fulfillment. You propose a saga and a refund. The interviewer adds one detail: the refund succeeded, but the payment provider's response disappeared. Your coordinator still says REFUND_PENDING. What happens next?

The core of saga interview questions is deciding what the system knows, what it can safely repeat, and what evidence permits a terminal state. Listing compensating actions does not explain how recovery resumes after another failure. You also need a plan for uncertainty, irreversible effects, and recovery that takes longer than the original request.

This article builds one purchase-and-refund case rather than repeating the broader microservices interview framework.

Evidence boundary: official architecture documentation supports the pattern definitions below. The purchase, state records, and failure exercises are original engineering examples, not verified candidate reports or an employer scoring rubric. Local experiment results are labeled separately.

What Does a Saga Actually Guarantee?

Official fact: AWS describes a saga as a sequence of local transactions coordinated through continuation and compensation. Each participant commits its own work; a later failure can trigger compensating transactions. That does not turn the whole sequence into one isolated database transaction. AWS saga patterns

Start your answer with the invariant and the intermediate states that the product permits. For our example, inventory must not be oversold, the same purchase must not be charged twice, and a customer must not see “refunded” until the payment system has confirmed the refund. Temporary PAYMENT_PENDING or REFUND_PENDING states are allowed.

Each invariant needs an owner. Inventory atomically reserves stock. Payments owns captures and refunds. The coordinator owns the workflow's progress and records references to those authoritative effects. The coordinator records observed payment status; Payments remains authoritative for whether money moved.

A useful interview answer is: “I can tolerate a pending order while services reconcile, but I cannot tolerate inventing a payment outcome. I will make the pending state durable and visible.” That statement exposes the trade-off before you draw orchestration boxes.

Why Is Compensation Different from Rollback?

Official fact: Azure's compensating-transaction guidance says compensation may need business-specific rules, may not restore the original state, and can itself fail. It also need not execute in the exact reverse order of the original actions. Compensating Transaction pattern

Consider a stock counter that read 10 before our order reserved one unit. Another customer then buys two units. Restoring the counter to 10 would erase valid concurrent work. The compensation should release our specific reservation, identified by its reservation ID, using the inventory service's concurrency rules.

Likewise, refunding a captured payment is a new business operation with its own record. It does not erase the capture from history. Provider fees, settlement timing, or a partial refund can make its economics different. Keep the history that explains why the customer was charged and later credited.

Design recommendation: describe compensation as a command with a precondition, an effect, a stable identity, and a completion test. “Undo payment” is too vague. “Refund capture C17 by 100 units using key S17:refund:v1, then record the provider refund ID” is reviewable.

Do not assume every failure requires compensation. A temporary notification outage may justify retrying delivery while keeping the order complete. A confirmed inventory rejection before payment may require ending the order without any financial compensation.

Where Is the Point of No Return?

Official fact: Azure distinguishes compensable transactions, a pivot transaction, and retryable work after the pivot. The pivot represents the workflow's commitment boundary. The exact placement depends on the business process. Saga distributed transactions pattern

For an illustrative physical purchase, use this sequence:

1. Reserve inventory with an expiry and a reservation ID.
2. Authorize payment, where the provider's contract permits a later capture.
3. Complete critical checks and confirm that the reservation remains valid.
4. Capture payment and arrange fulfillment according to an explicit business policy.
5. Hand the package to a carrier; complete remaining bookkeeping and notifications.

The carrier handoff is a physical effect you cannot undo by deleting a row. A return shipment is a different workflow. Payment capture might be compensable through a refund, but the domain must define that contract. “Irreversible” and “unsafe to retry” are separate properties: a one-way effect may still have a stable operation ID and safe duplicate handling.

An interviewer can move the pivot earlier by changing the business: an immediately consumed digital entitlement, a nonrefundable supplier booking, or a legal submission. Ask which effects remain possible to compensate and which obligations must instead be completed.

Inference for this design: delay expensive or irreversible work until prerequisites are established. After commitment, track unfinished obligations explicitly. Do not promise that a retry loop guarantees success during a permanent outage; give stalled completion an escalation path.

What If a Step Succeeds but the Reply Is Lost?

A timeout gives the coordinator an uncertain observation. It does not prove the participant performed no work. That distinction matters for both the original capture and the refund.

Persist an operation identity before dispatch. On a timeout, query the participant by that identity if its API supports authoritative lookup. Otherwise, repeat the same operation only under a documented idempotency contract. A fresh key creates a fresh operation and can duplicate the effect.

The key also needs a lifetime and a scope. If the provider forgets it after a limited retention period, a week-old recovery cannot assume deduplication still applies. Store the original request parameters and provider references; reject reuse of the same key with a different amount.

The transactional outbox pattern helps atomically record local workflow state and an intent to send a command. It does not make the recipient's external effect atomic with the coordinator's next database commit. You still need participant idempotency and reconciliation across that boundary.

For the lost refund response, keep REFUND_PENDING until lookup or a safe repeated command establishes the result. Tell the customer the refund is processing. Marking it failed could provoke another refund; marking it complete could hide an unperformed obligation.

Run the Refund Crash Timeline

Local experiment: we used Python 3.12.14 and two independent in-memory SQLite connections to model a coordinator and a payment provider. The provider stub enforced a unique refund key. This is a failure-boundary model, not a test of a real payment API or durable workflow product. The simulated crash skips a coordinator update; it does not exercise operating-system crash durability or network partitions.

The initial coordinator row was S17, REFUND_PENDING, version=4. The provider accepted S17:refund:v1 and committed one refund. We injected the logical crash before updating the coordinator, then resumed recovery with the same key.

The recovery produced one provider refund and a coordinator state of COMPENSATED. A second worker trying to move version 4 forward updated zero rows because recovery had already advanced it to version 5. The relevant state transition was:

UPDATE saga
SET state = 'COMPENSATED', version = 5
WHERE id = 'S17'
  AND state = 'REFUND_PENDING'
  AND version = 4;

The SQL protects the coordinator from a stale transition. It cannot retract an external command already issued by a stale worker. That requires participant-side state checks, deduplication, or fencing where the participant supports it. Name that limitation before claiming the concurrency problem is solved.

Try three variations aloud. Remove the provider's unique key and explain the duplicate refund risk. Change the command amount while keeping the key and explain why the provider must reject the mismatch. Let the key expire and explain why the operator must investigate rather than replay blindly.

What Should a Stuck Workflow Record Contain?

A status string and an exception message are not enough to resume safely. For this example, an operational record should expose the saga ID, current state and version, intended step, stable command key, participant resource IDs, attempt count, next retry time, last confirmed outcome, and last error.

Separate three clocks: time since the workflow began, time since the last confirmed progress, and time until the business deadline. A long-running reservation may be expected; a short workflow that has made no progress past its deadline may deserve immediate attention.

Use bounded retries with backoff for transient failures. Keep retry eligibility separate from the count: a confirmed invalid account is not made valid by another hundred requests. When the automated budget ends, move the workflow to an explicit recovery queue with an owner and the preserved evidence.

An operator should be able to answer: “Has the provider already refunded this capture?” before receiving a button labeled “Retry.” A retry action should identify the exact step and key, check the current version, record who acted and why, and preserve the result. A dead-letter queue is a storage location, not an operational resolution.

Do not offer a generic “force success” action that suppresses an unresolved financial obligation. If the business accepts a manual adjustment, record that adjustment and the evidence separately from the original transaction. Future reconciliation must be able to distinguish automated completion from an authorized exception.

How Do Cancellation and Late Success Interact?

Here is the harder follow-up: the customer cancels while capture is in flight. The coordinator starts compensation, then receives a late capture-success event. Ignoring the event because the order is canceled loses the obligation to refund.

Define cancellation as a requested state transition, not a guarantee that all in-flight effects stopped. Record the late capture under its operation ID. If the business policy still requires cancellation, schedule the appropriate refund exactly once. If shipment has already crossed the irreversible boundary, route to the return or exception process instead.

For each event, validate workflow identity, step identity, and the expected transition. Duplicate events should be harmless. Out-of-order events should either update recorded evidence safely or wait for reconciliation; they should not push a terminal workflow back into arbitrary forward execution.

Orchestration makes this state machine easier to inspect centrally. Choreography can implement the same rules, but the evidence is distributed across participants. Choose based on ownership and operational complexity, then show where someone can reconstruct the entire history.

Practice the Recovery Decision

Practice question	What to rehearse
Design a Multi-Service Workflow Engine	Make workflow progress and recovery decisions durable.
Design a Durable DAG Workflow for Order Refunds	Resume failed refunds without repeating completed effects.
Design an Ordering Service That Prevents the Last Item from Being Oversold	Coordinate reservations with late payment outcomes.
Design an airline booking system	Identify irreversible supplier commitments.
Design distributed transactions protocol	Compare local compensation with atomic commit.

For a focused rehearsal, draw the happy path once, then spend most of the session on one failure: refund committed, acknowledgment lost, stale worker still alive. State the authoritative source, legal next actions, customer-visible state, and operator evidence. That produces a much stronger answer than listing every orchestration platform you know.

Continue with Backend Engineer interview practice and defend the same workflow after moving the irreversible boundary. Explain how that change affects compensation, customer status, and the evidence required to resume.

Sources and Further Reading

• AWS: Saga patterns
• Azure: Compensating Transaction pattern
• Azure: Saga distributed transactions pattern