Articles
SQS FIFO Interview Questions: Message Groups, Deduplication, and Blocked Consumers
Prepare for SQS FIFO interviews with message-group ordering, deduplication limits, visibility timeouts, blocked consumers, and side-effect safety.
An SQS FIFO queue has twenty consumers, but one customer's orders are not moving. Adding twenty more consumers changes nothing. Other customers continue processing normally. Is FIFO broken, or is the application waiting behind its own ordering boundary?
A strong interview answer starts with the message group. FIFO ordering is useful precisely because later work cannot freely overtake earlier work in the same ordered stream. That means group selection, failure handling, and business idempotency matter as much as consumer count.
Evidence boundary: Linked AWS documentation supports the queue behavior below. The account timeline and failure-window analysis are original preparation exercises, not candidate reports or observed AWS measurements. This article covers generic SQS consumers; Lambda event-source mapping has additional behavior that should be evaluated separately.
Begin with Implement a Concurrent Payment Queue with Visibility Timeouts. Ask what happens if payment succeeds just before a worker crashes. That question exposes the boundary between queue delivery and business completion.
What is ordered: the queue, the group, or the business operation?
Official fact: SQS FIFO ordering operates within a MessageGroupId. Different groups can be processed concurrently, and their relative order is not guaranteed. A receive batch can contain several messages from one group. See FIFO delivery logic.
For an original order-processing design, choose a group that matches the required serialization. If updates to one account must apply in sequence, an account identifier may be appropriate. A single group for the entire business creates a much broader bottleneck. A random group per message removes the useful account-ordering relationship.
Do not select the producer's identity automatically. Two producers may emit updates for the same account, and one producer may serve thousands of independent accounts. The correct boundary follows the business invariant, not whichever field is easiest to populate.
Also distinguish queue acceptance order from an external causal order. If two upstream systems race to submit “create order” and “cancel order,” the consumer needs a contract for valid state transitions. FIFO does not reconstruct business history that the producers failed to coordinate.
Why do additional consumers fail to unblock one account?
Official fact: While received messages from a FIFO group remain in flight, subsequent receives do not freely advance that group; deletion or renewed visibility changes what can be delivered. Other groups may continue. See FIFO delivery logic.
This original timeline assumes two groups and shows the effect:
| Moment | Account A | Account B | What more consumers can change |
|---|---|---|---|
| Start | A1 is received | B1 is received | Independent groups can progress |
| A1 stalls | A2 remains behind A1 | B1 completes; B2 can advance | They can help available groups, not bypass A1 |
| A1 visibility expires | A1 becomes eligible for another attempt | B continues | Another attempt may repeat unfinished or completed work |
| A1 completes and is deleted | A2 can advance | B continues | A's ordering boundary is released |
The interview diagnosis should ask for the group ID, first unfinished message, receive attempts, visibility handling, and downstream dependency. Queue depth alone cannot distinguish one hot group from many independently runnable groups.
If account A dominates the workload, adding consumers may leave them idle. Splitting A into smaller groups is valid only if the business operations no longer require ordering across those subgroups. Explain the invariant you would relax or preserve before proposing a throughput improvement.
For example, separate unrelated document conversions may safely run in different groups. Balance adjustments that depend on the prior balance require a stronger argument. A partitioning change is an application correctness decision, not just a tuning knob.
What does deduplication protect, and for how long?
Official fact: FIFO deduplication uses a five-minute interval. A producer retry with the same deduplication ID within the documented interval avoids introducing another queued copy. Content-based deduplication hashes the message body, excluding message attributes. See FIFO exactly-once processing.
For the original payment workflow, assign a stable application operation ID when the intent is created. Reuse it when retrying the same send. Generating a new random deduplication ID for every network retry defeats the purpose of identifying the same submission.
Do not confuse that interval with your business replay horizon. A payment may be retried after a long outage, a manual recovery, or a delayed upstream replay. If the business requires recognizing the same charge days later, its idempotency record needs an appropriate lifetime beyond the queue's short deduplication window.
Content-based deduplication has its own interview trap. If two semantically different operations share an identical body and differ only in attributes, they may look identical to that mechanism. Conversely, adding a changing timestamp to an otherwise identical body can make duplicate business intent look different. Design identity deliberately instead of relying on incidental serialization.
Official fact: SQS continues tracking the deduplication ID after a message has been received and deleted during the interval. See FIFO key terms. Deletion is not a signal that a producer should reuse the same ID for a new business operation.
Why does FIFO not guarantee exactly one external side effect?
Consider the original payment failure window:
| Worker failure point | Queue may offer another attempt | Business risk without an application protocol |
|---|---|---|
| Before the payment request | Yes | Work has not happened yet |
| After provider accepts payment, before result is recorded | Yes | A retry can charge again |
| After durable result, before message deletion | Yes | A retry must recognize completed work |
| After confirmed deletion | Normal processing is complete | Later upstream replay still needs business identity |
The critical gap spans systems. SQS cannot atomically commit an arbitrary external payment and your local completion record simply because the queue is FIFO.
Engineering inference: Use a stable operation ID at the side-effect boundary when the provider supports idempotency. Persist enough state to reconcile uncertain outcomes. If the provider does not offer such a mechanism, explain the reconciliation or compensation protocol and its limits rather than promising a universal exactly-once result.
A local “processed IDs” table also needs careful ordering. Marking an operation complete before charging can lose the charge after a crash. Charging before marking completion can repeat the charge. The design must address that gap explicitly, using the capabilities of the systems involved.
For a transaction entirely inside one database, an atomic state update with a unique operation record may be available. For an external call, that local transaction does not automatically include the remote service. Name the boundary before choosing the implementation.
How should visibility timeout and deletion be handled?
Official fact: Receiving a message does not remove it. Visibility timeout temporarily hides it, and the consumer can adjust visibility while processing. If processing does not complete and deletion does not occur, the message can become available again. See visibility timeout.
Set the initial timeout using observed processing behavior and failure-recovery needs. Too short can cause another attempt while a slow worker is still active. Too long can delay recovery after a crashed worker. Extending visibility should reflect actual progress, with a bound and monitoring, rather than keeping a stuck message hidden forever.
A timeout expiring is not proof the old process stopped. In the payment exercise, the original worker might still be waiting for a provider response while a new worker receives the message. That is another reason side-effect idempotency must survive concurrent attempts.
Official fact: DeleteMessage uses the receipt handle from the latest receive, not the message ID returned when sending. AWS warns that an old receipt handle may not delete the message even if the request succeeds. See DeleteMessage.
Keep receipt handles attached to processing attempts. Keep business operation IDs attached to business intent. Treating those identifiers as interchangeable makes recovery code difficult to reason about.
What can go wrong when a receive batch is processed in parallel?
A batch can contain A1 and A2 from the same group. If your handler launches both concurrently and A2 commits first, your application has defeated the business ordering you intended to preserve. Broker delivery order does not automatically serialize arbitrary code inside your process.
For an original consumer design, partition the batch by group, maintain order inside each group, and allow independent groups to run concurrently where the workload supports it. If A1 fails, define what happens to already received A2 before acknowledging or deleting later work.
Do not copy a generic “process every item concurrently and delete successes” loop without examining dependencies. A2 might appear successful only because it ran against state that should have been changed by A1. The acceptance criterion must include valid business transitions, not just handler return values.
The exact integration matters. If Lambda manages polling and deletion, study its FIFO and partial-batch contract separately. The existing Lambda and SQS interview guide covers that adjacent topic; this article's generic consumer sketch is not a substitute for that integration's rules.
Does a dead-letter queue solve the ordering problem?
Official fact: A DLQ isolates messages that cannot be processed successfully according to the redrive policy. AWS warns that using a DLQ with FIFO can break the exact order of operations. See SQS dead-letter queues.
In our account example, moving A1 aside may allow later operations to advance. Whether that is acceptable depends on their relationship to A1. An optional notification can perhaps be quarantined independently; a required account-creation event cannot simply be skipped before applying an account update.
State a recovery policy: pause the affected business stream, repair and replay with validation, compensate dependent operations, or explicitly permit skipping certain event types. Replaying A1 later does not restore its original position in history automatically.
Before redrive, preserve the operation ID, original business sequence, failure reason, and relevant processing state. Rate-limit recovery so a repaired backlog does not overwhelm the downstream service. A DLQ should create an actionable recovery workflow, not an indefinite storage location with no owner.
Which measurements reveal the actual bottleneck?
Official fact: SQS exposes CloudWatch metrics for backlog, in-flight messages, message age, and other queue behavior; many values are approximate. See available metrics.
Pair queue-level indicators with application measurements: active group count, per-group wait time, downstream latency, repeated operation IDs, and completed business operations. A rising receive count can reflect retries rather than useful throughput.
For the original incident, compare account A's first unfinished operation with account B's progress. If B remains healthy, prioritize A's dependency or payload before scaling the entire fleet. If many independent groups stall simultaneously, investigate shared downstream capacity or consumer failure.
Propose a controlled test with two groups, a slow first message, a worker crash after a simulated side effect, and a stale receipt handle. Record expected group progress and business outcomes. These are test designs, not experiments run for this article.
Practice ordering and recovery together
The verified questions below provide adjacent queue practice. Their follow-ups are original preparation prompts.
| Interview question | FIFO follow-up to explain |
|---|---|
| Implement a Concurrent Payment Queue with Visibility Timeouts | Close the payment-success/message-delete failure gap. |
| Explain Kafka and Message-Queue Delivery Semantics | Separate send deduplication from business idempotency. |
| Design distributed message queue service | Choose the correct serialization boundary. |
| Design a Multi-Channel Message Queue | Explain independent progress and one blocked stream. |
| Implement a Thread-Safe Pub-Sub Service | Preserve ordering when worker concurrency increases. |
Continue with Implement a Concurrent Payment Queue with Visibility Timeouts. Explain why the twentieth consumer cannot bypass A1, then show how a repeated attempt avoids a second charge.