Amazon Kinesis Interview Questions: Hot Shards, Consumer Lag, and Partial Failures

Amazon Kinesis interview questions become easier when you separate three problems: records failing to enter the stream, consumers failing to retrieve them quickly enough, and application work failing to finish. Adding shards, enabling enhanced fan-out, and retrying a batch address different boundaries. None is a universal fix.

This guide focuses on Kinesis Data Streams. It is not an Amazon Data Firehose tutorial or a description of Amazon's hiring process. Official AWS documentation supplies service behavior. No candidate reports are used to predict interview rounds.

What Does a Partition Key Actually Control?

Official service behavior: Kinesis maps a partition key through a hash-based routing mechanism to a shard. A shard is a unit of stream capacity and an ordered sequence of records; its sequence numbers are not a global business-event clock. AWS also exposes explicit hash-key routing, so clarify whether the producer uses the ordinary mapping. Kinesis terminology and concepts.

For an interview, ask what must stay together. An account identifier can preserve an account-oriented routing boundary, but a very busy account can concentrate traffic. A random key can spread records more widely while making account-level sequencing and aggregation harder.

Consider an original telemetry example: most tenants send moderate traffic, while one customer uploads a large device backlog after an outage. The stream's total traffic may look comfortable while the shard receiving that customer's key is overloaded. The diagnostic question is whether the concentration comes from one key, several keys sharing a shard, or a broad increase across the stream.

Splitting a shard can redistribute different hash values. It does not divide one ordinary partition-key hash across multiple shards at the same instant. If you propose salting the key, explain how consumers recover the required business grouping and what ordering guarantee changes. A routing change is part of the data contract, not merely a capacity setting.

Diagnose Three Different Bottlenecks

Use these original cases to practice choosing the next observation before suggesting a fix.

Case	Initial evidence	Next evidence to inspect	Observation that changes the diagnosis
Write hotspot	Producer throttling concentrated around a tenant burst	Per-shard incoming traffic and producer key distribution	Evenly distributed throttling suggests broader capacity pressure
Shared read contention	A second independent reader starts; read throttling appears	Retrieval mode, polling behavior, and combined shard reads	No read throttling but long processing calls points downstream
Slow sink	Input continues, one consumer's backlog grows	Processing duration, sink latency, retries, and worker resources	Slow retrieval before processing begins points back to the read path

Official troubleshooting guidance: AWS identifies competing reads, retrieval configuration, and expensive processing logic among reasons consumers read slowly. It documents GetRecords.IteratorAgeMilliseconds and KCL's MillisBehindLatest as useful lag signals. Use metrics appropriate to the retrieval library and mode, rather than assuming every consumer emits the same measurements. Troubleshoot Kinesis consumers.

For the write hotspot, preserve unsuccessful events and reduce retry pressure while investigating. For read contention, distinguish multiple independent applications from workers cooperating within one application. For the slow sink, look for a recent query, connection-pool, or retry-policy change before buying more ingestion capacity.

A useful answer includes a falsifiable hypothesis: “If database calls dominate processing time while retrieval is healthy, increasing read throughput will not address the immediate constraint. I would measure those durations separately and test the sink bottleneck under controlled conditions.” That is stronger than a list of every CloudWatch metric you remember.

When Does Enhanced Fan-Out Help?

Official service behavior: shared-throughput consumers retrieve records with GetRecords; their reads share a shard's available read throughput. Enhanced fan-out gives each registered consumer dedicated per-shard read throughput and delivers records through SubscribeToShard. Applicable consumer limits depend on stream mode and, for some configurations, Region. AWS enhanced fan-out documentation.

In the second case above, dedicated retrieval can be useful when separate consumers compete for the same read capacity. It isolates that part of their workload. It does not increase producer write capacity or make an expensive transformation finish faster.

Before recommending it, identify the number of independent consumers, freshness requirement, current throttling, and operational cost. If one consumer already retrieves data comfortably and spends its time waiting on a downstream API, the explanation should remain focused on processing.

Also distinguish retrieval freshness from end-to-end freshness. A consumer may receive an event quickly but leave it in an internal queue before committing the result. Measure event creation, stream arrival, retrieval, and durable sink completion where available. A dashboard based only on retrieval lag can miss stale business output.

Can PutRecords Return Success With Failed Records?

Yes. Official API behavior: PutRecords returns a result entry corresponding to each request entry. A successful entry carries a shard ID and sequence number; an unsuccessful entry carries error information. A record failure does not prevent later entries from being processed, and PutRecords does not guarantee record ordering. An HTTP success response therefore does not establish that every record succeeded. AWS PutRecords API.

Here is an original response fixture. A, B, and C share the ordinary partition key account-7:

Request position	Application event	Fixture result	Producer action
0	A	Success, sequence 100	Record acknowledgement
1	B	Throughput error	Keep B pending for retry
2	C	Success, sequence 101	Record acknowledgement

The sequence values are illustrative. If B subsequently succeeds as sequence 102, one possible append order is A, C, B. The response array's positional mapping helps identify which input failed; it does not promise the intended business order A, B, C.

Retrying all three after this known partial result needlessly resubmits acknowledged events. Discarding the entire batch loses the pending event. The immediate task is to preserve the mapping and select the failed record.

A Small Python Exercise for Partial Results

The following function operates on a parsed response fixture. It does not contact AWS, serialize payloads, or implement a complete retry scheduler. Its purpose is to verify record/result correspondence and make malformed fixture data fail visibly.

def failed_records(request, response):
    results = response["Records"]
    if len(request) != len(results):
        raise ValueError("response length mismatch")
    failed = []
    for record, result in zip(request, results):
        error = result.get("ErrorCode")
        success = bool(result.get("ShardId")
                       and result.get("SequenceNumber"))
        if bool(error) == success:
            raise ValueError("ambiguous result")
        if error:
            failed.append(record)
    if len(failed) != response["FailedRecordCount"]:
        raise ValueError("failed count mismatch")
    return failed

For the table above, the returned list contains B only. Local verification: eight tests passed, covering partial success, all success, all failure, unchanged input, response-length mismatch, failure-count mismatch, a missing result state, and a contradictory result state. These tests establish the fixture function's behavior, not AWS availability, throughput, or latency.

To extend the exercise, keep original event IDs and payloads in a durable pending queue. Classify errors, apply bounded retry attempts with backoff and jitter where appropriate, and expose events that cannot be delivered for investigation. Authentication or invalid-request failures should not become an unbounded rapid retry loop.

In a real integration, determine what your SDK already retries. Then define the application's responsibility for per-record failures and durable recovery. Two retry layers can amplify traffic if neither accounts for the other's attempts. A useful log links the stable application event ID to each attempt and eventual acknowledgement without dumping sensitive payloads.

What If the Entire Request Times Out?

A timeout without a usable result is different from the known partial response above. The producer may not know which records reached the service. A retry can preserve delivery attempts while also creating duplicate stream records.

Official guidance: AWS describes duplicate records arising from producer retries and consumer reprocessing, including restarts around checkpoints. It recommends accounting for these behaviors in application processing. Handle duplicate records.

For an original billing-event design exercise, carry a stable event_id across retries. The sink must distinguish a repeated delivery of the same event from a different event affecting the same account. Using account_id alone as a deduplication key would incorrectly suppress legitimate later activity.

Then inspect the atomicity boundary. If the consumer first changes a balance and later records the event ID, a crash between those operations can allow the change twice. If it records the ID first and crashes before changing the balance, recovery may suppress work that never happened. A transactional sink operation or another carefully designed idempotent effect boundary is needed for that business action.

Be precise about the promise. Deduplicating a durable database effect does not mean every message was delivered exactly once. State how long deduplication information is retained and what happens if an old event is replayed after that window. Conflicting payloads under the same event ID also need a defined error policy.

How Would You Preserve Order?

Begin by naming the order: producer submission order, stream append order, event-time order, or the order in which sink effects become visible. These can differ even when records share a key.

Official API detail: PutRecord exposes SequenceNumberForOrdering for strictly increasing sequence ordering with the same client and partition key, using the preceding successful sequence number. That is a narrower condition than “all records with this key are globally ordered forever.” Read the API's conditions before describing a producer strategy. AWS PutRecord API.

For the A/B/C exercise, ask whether C depends on B. If it does, allowing C to advance while B remains unresolved violates the application's requirement. One option is to serialize the relevant producer work and wait for acknowledgement; another may involve explicit application sequence numbers with bounded reordering and a missing-event policy. Both impose costs.

Consumer concurrency creates a separate boundary. Receiving A before B does not ensure A's slow external side effect finishes first. If the business requires per-account sequencing, maintain that constraint through processing and sink updates. Explain the head-of-line blocking and failure handling this introduces.

Avoid promising global ordering across independently produced events without defining a coordination mechanism. Many analytics workloads can tolerate out-of-order arrival if event-time processing and late-data rules are explicit. The right answer depends on the operation being protected.

Turn a Diagnosis Into a Recovery Plan

Finish an interview case with a sequence that protects data as well as freshness. Identify the affected producer or consumer, preserve pending work, remove the immediate bottleneck, and verify that backlog falls while new input continues.

For a slow sink, calculate whether recovery throughput exceeds ongoing arrival rate. If processing merely matches new traffic, the backlog never drains. Use actual observed rates for a real system; do not infer sustainable capacity from this article's fixtures.

Include retention in the recovery discussion. If unread data may expire before catch-up, raising retention can buy time where supported, but it does not resolve the bottleneck. Also confirm that replay will not corrupt downstream results. Recovery that clears lag while duplicating business effects is incomplete.

Sources and Further Reading

• AWS: Kinesis terminology and concepts.
• AWS: Troubleshoot Kinesis consumers.
• AWS: Enhanced fan-out consumers.
• AWS: PutRecords API.
• AWS: Handle duplicate records.
• AWS: PutRecord API.