Interview guides · Citi

Citi Software Engineer Interview Guide

Citi Software Engineer interview guide: reported rounds, Java, Kafka, SQL and live-debugging questions, a 7-day plan and worked exercises.

The 60-second brief

Citi is a global financial institution, and its Software Engineers work on applications for banking operations, wholesale credit risk analytics, trading platforms and consumer digital products. The reported work centers on Java and Spring Boot microservices, Kafka messaging, cloud-native modernization with Docker and Kubernetes, and automated security and compliance controls. Full-stack teams also use Python, React and Angular.

This guide covers the three stages candidates report for the Citi Software Engineer role (Phone Screen, Technical Interviews, Final Interviews) and the question categories that come up across them: core Java and OOP, Spring Boot, microservices and Kafka, data structures and algorithms with live debugging, SQL and data modeling, system design, and behavioral scenarios. The guide includes reported questions, original drills and three worked exercises.

Focus areas

Scope every query and cache key by tenant
Evolve APIs without breaking pinned SDK clients
Make every write idempotent under client retries

Overview

Software Engineers at Citi build and maintain applications behind banking operations, wholesale credit risk analytics, trading platforms and consumer digital products. Reported teams include backend Java and Spring Boot services, full-stack work with React, Angular or Python, and platform engineering that modernizes infrastructure. Levels run from Analyst through Assistant Vice President to Vice President and Senior Vice President.

Engineering teams are described as sitting within a Developer Services and CTO organization with an everything-as-code approach, where infrastructure capabilities, API contracts, security standards and pipeline controls are defined in repositories. The day-to-day work described includes cloud-native microservices, Kafka-based event-driven messaging, containerization with Docker and Kubernetes, automated security and compliance controls, and bringing Generative AI into banking workflows.

For interview preparation, this means covering a wide range of topics. Reported questions go from Java language internals and Spring Boot behavior to Kafka ordering, distributed transactions, SQL isolation levels and live debugging of an existing codebase. Prepare answers that connect a mechanism to its consequence in a regulated system, such as why immutability matters for concurrent code or how a messaging pipeline avoids losing a transaction event.

The Citi interview process

Citi candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Phone Screen

Candidates describe the first stage as an initial call that mixes behavioral and technical questions to test technical knowledge and problem-solving. Reports describe the process beginning with an HR screen or an automated online assessment on data structures, algorithms and logical reasoning. For many software engineering roles, reports add a live technical screen run through Karat, a third-party assessment platform. The reported Karat format starts with core technical questions on language internals and framework design. It then moves to a live debugging exercise where you fix failing test cases in a multi-class repository, along with algorithmic problems. Ask your recruiter which of these steps apply to your role, and practise live code debugging either way, since reports describe it as directly evaluated in the screening assessments.

What to demonstrate

Your understanding of your primary language's internals, such as how a HashMap handles collisions and resizing or what makes a Java class immutable, explained as mechanism rather than definition
How methodically you isolate a bug in unfamiliar multi-class code: reading the failing tests, finding the fault and making a targeted fix without breaking passing tests
Whether you can solve an algorithmic problem and state its complexity correctly

How to prepare

Break two or three methods in a small multi-class project, or have someone else do it, then fix them using only the test output. Write down the order in which you opened files.
Rehearse short spoken explanations of HashMap internals, immutability for thread safety, JVM vs JRE vs JDK, and garbage collection. End each with one production consequence.
Drill hash map and string problems out loud, tracing each on empty and single-element input before you call it finished
Ask your recruiter which screening steps apply to your role, and whether the Karat redo option described in candidate reports applies to your screen

Technical Interviews

Candidates describe this stage as a series of interviews with coding challenges and system design discussions, focused on algorithms and architecture principles. Reports describe a Superday or multi-round panels with engineering leads, hiring managers and technical architects. These panels cover core language mechanics, microservices design, live coding, resume deep-dives and situational behavioral questions. Reports put more weight on system design for mid-level and senior roles (AVP, VP, SVP).

What to demonstrate

Live code that runs and handles edge cases, with a stated complexity that matches the code you actually wrote
How deeply you understand Spring and Spring Boot, microservices and Kafka: auto-configuration, transaction boundaries without two-phase commit, message ordering and consumer-group rebalancing
For mid-level and senior candidates, designs that explicitly cover scalability, fault tolerance and security controls
Whether you can explain how you implemented the tools and projects listed on your resume

How to prepare

For each technology on your resume, prepare one concrete implementation detail and one problem it caused you, because reports describe resume items as the starting point for deep dives
Write out the Kafka answer: ordering holds only within a partition, the message key picks the partition, and each partition goes to one consumer in a group. Then explain how processing stays idempotent when a rebalance redelivers messages.
Prepare a saga or transactional-outbox walk-through for a write that spans two services, naming the compensating action and the idempotency key
Practise one full design, such as the Chat Application Design bank question or the webhook fan-out worked exercise. State the failure modes, duplicate handling and audit logging explicitly.

Final Interviews

Candidates describe the final stage as meetings with senior engineers or leadership about cultural fit and alignment with the team. Some reports describe a final techno-managerial or HR round covering team alignment, compensation and offer approval. Treat it as a mix of technical and behavioral discussion. Prepare stories that show what you personally did, and have your motivation and compensation expectations settled before you walk in.

What to demonstrate

How you talk about working under regulatory constraints: balancing delivery speed with risk controls, security and system availability
Whether your stories about ambiguity, disagreement and production incidents show the actions you personally took and their outcomes
Whether your reasons for choosing Citi and this team tie to the work described, such as Kafka-based messaging, cloud-native modernization or GenAI developer tooling

How to prepare

Prepare four STAR stories: three for the reported prompts (an ambiguous requirement or tight deadline, a disagreement with a senior architect, a challenging production bug) and one for the bank question Handling Critical Project Feedback
Prepare an answer on using GenAI developer assistants safely in a regulated industry: what code or data you would not give a tool, and how you would review its output
Bring questions about the team's modernization work, cloud adoption or automated developer controls
Settle your compensation expectations beforehand, since reports place compensation discussion in the final HR round

Where candidates lose points

Rewriting code in the live debugging exercise instead of reading the failing tests first

In a multi-class repository with failing tests, the tests are the specification. Run the suite, read each failing test's name and assertion, reproduce one failure, and trace it to the smallest wrong line before you edit anything. Make the minimal fix, re-run the whole suite and narrate as you go. Rewriting a class you have not fully read tends to break tests that were passing and leaves the interviewer nothing to follow.

Defining Java internals without explaining the mechanism or its consequence

'HashMap uses hashing' does not answer the reported question. Explain bucket index from the hash, collision chaining (with long bins converted to trees since Java 8), and resizing once size exceeds capacity times load factor. Then give a consequence, such as a mutable key whose hashCode changes after insertion becoming unreachable. Do the same for immutability: final class, private final fields, defensive copies, and why the object can then be shared across threads without locks.

Claiming global ordering or exactly-once delivery from Kafka without stating the conditions

Kafka orders messages within a partition, not across a topic. Related events only stay in order if they share a key that routes them to the same partition. During a consumer-group rebalance, partitions move between consumers and uncommitted messages can be redelivered. Say how your consumer handles a duplicate (an idempotency key or a unique constraint on the effect) before claiming no transaction event is lost or applied twice.

Proposing indexes for a slow query without reading its execution plan

For the slow-query and index-internals questions, start from the plan: look for full scans, bad row estimates and sort or join steps that dominate the cost. Only then propose a B-tree index, explaining column order for the predicate and why a leading wildcard or a function on the column prevents its use. Pair it with the write cost of each extra index. Keep isolation levels exact as well: which anomaly each level prevents, not a loose ranking of 'stricter'.

Listing tools on the resume that you cannot defend in a deep dive

Reports describe interviewers picking specific tools or projects from the resume and asking how you implemented them. Before the panels, go through every technology you list and prepare what you built with it, one decision you made, and one thing that went wrong. Remove anything you only watched someone else use.

Questions & practice

Hold a tenant to a trailing sixty-second request limit

Medium · sliding-window · two-pointer · rate-limiting · tenant-skew

The gateway must hold each tenant to R requests in any trailing 60 seconds, in aggregate across three regions and every pod, within a budget of under 10 ms added p99. Peak is 30,000 requests/second across 200,000 active tenants, and traffic is heavily skewed toward a handful of them. Give an exact single-process algorithm with its amortised per-request cost and its memory per tenant, then a bounded-memory approximation and the worst-case overshoot it actually admits. Say what the distributed version does when the counter store is unreachable.

Approach

Exact, single process: a per-tenant deque of request timestamps. On arrival, pop from the front while front <= now - 60s, then admit if the remaining length is below R and push. Each timestamp is pushed once and popped once, so the cost is O(1) amortised. The O(R) version is the one that re-filters the whole deque on every request.
Quote the memory. R = 1,000 across 200,000 active tenants is up to 2 x 10^8 timestamps at 8 bytes, about 1.6 GB, and that is the worst case rather than the mean, because the long tail of small tenants holds almost nothing. Skew helps you here and hurts you in the sharding decision.
Bounded alternative, with its real bound stated: a fixed 60-second counter is O(1) memory but admits close to 2R across a 60-second span straddling a boundary. The weighted two-bucket estimate, prev * (60 - elapsed)/60 + cur, is better on smooth traffic but assumes the previous window's arrivals were uniform; an adversary packing them at the end of that window is undercounted and can still approach 2R. Say that rather than calling it exact.
Token bucket is the usual gateway answer and a different contract: O(1) state per tenant (tokens, last_refill), a sustained rate, and a deliberate burst allowance equal to the bucket size. Choose it when a burst is acceptable and the log when the limit is contractual.
Distributed: the limit is per tenant in aggregate, so a local bucket of R/N per pod is wrong in both directions under skew. A tenant landing on one pod is throttled at R/N, and a tenant spread evenly across pods exceeds R. The shared check must be a single atomic round trip, one script or one increment-and-compare, never read-then-write, and it must fit inside the 10 ms p99 budget.
Decide the unavailable case in advance and write it down. Failing open keeps the product up and lets a tenant exceed its limit for the duration; failing closed converts a counter-store outage into a full outage. Most gateways fail open on rate limits and closed on authorisation, and those are two separate decisions made separately.

Follow-up

One tenant sends 40% of all traffic. What does that do to a single counter key, and what do you shard on instead?
Quotas rather than rate limits: the check is select used; if used < limit then insert. Name the isolation level that still permits the overshoot, and the two fixes.
How do you return an accurate Retry-After from the exact algorithm without a second scan?

Find peak concurrent sandbox usage from run intervals

Medium · sweep-line · intervals · concurrency-caps · null-semantics

Given up to 5 million job_run rows for one tenant over one day, with run_id, started_at, finished_at, status and wall_clock_limit_seconds, report the maximum number of sandboxes running at once, the earliest instant that maximum is reached, and the first run_id that would breach a per-tenant cap of C. started_at is null while a run is queued; finished_at is null both for runs still executing and for runs in status lost. Treat a run as occupying [started_at, finished_at). Give the complexity and state how you handle each null.

Approach

Turn each run into two sweep events, (started_at, +1) and (end, -1), then sort the 2n events by timestamp with -1 ordered before +1 at equal timestamps. That tie-break is what makes the interval half-open, so a run finishing at 10:00:00 and one starting at 10:00:00 never overlap.
Decide each null out loud before sweeping, because each choice moves the answer. A null started_at means queued and contributes nothing. A null finished_at with status running or leased is clipped to the window end. Status lost has no observed end at all, so clip it at started_at + wall_clock_limit_seconds on the grounds that the supervisor owns the timeout, and record that you did. The table's check (finished_at is null or started_at is not null) guarantees you never see an end without a start.
Sweep once, maintaining a running counter, the maximum, and the timestamp at which the maximum was first attained (update peak_at only on a strict increase, or you will report the last such instant instead of the earliest). Capture the first run_id whose +1 takes the counter to C+1 during the same sweep rather than in a second pass.
Complexity: O(n log n) dominated by the sort, O(n) space. If rows already arrive ordered by started_at, a min-heap of end times gives O(n log k) time and O(k) space with k the peak concurrency, which is the better shape when the rows come from an index scan on (tenant_id, started_at).
If second resolution is acceptable, counting-sort the endpoints into an 86,400-slot delta array and prefix-sum it: O(n + T) time and O(T) space, which beats the comparison sort at 5 million rows. It answers only at second granularity, so state which resolution the cap is defined in.

Worked solution (20 min)

Write the null policy as three lines of prose first, one per case, and keep them beside the output.
Emit 2n endpoint tuples (timestamp, delta, run_id) and sort on the key (timestamp, delta) so -1 precedes +1.
Sweep, tracking cur, peak, peak_at updated only on a strict increase, and the first run_id whose +1 takes cur to C+1.
Build a fixture with two runs where one ends exactly when the next starts, three genuinely overlapping runs, one run with a null finished_at and status running, and one with status lost and a 300-second wall_clock_limit_seconds.
Re-run with every timestamp shifted by a constant and confirm the peak is unchanged while peak_at shifts by the same constant.

Expected result: Peak is 3, from the overlapping trio. The exact-handoff pair yields a peak of 1, not 2. peak_at is the start instant of the third overlapping run. The lost run occupies exactly [started_at, started_at + 300s) under the stated policy.

Verify it yourself

Setting peak_at on >= rather than > moves the reported instant to the last time the peak was attained. Flip it and watch the fixture change.
Sorting by timestamp alone, without the delta tie-break, raises the reported peak on the handoff fixture.
The deltas over the whole sweep sum to zero once every open-ended run has been clipped.

Follow-up

Now report peak concurrency per tenant for 10,000 tenants from one globally sorted stream. What changes about memory and about the sort?
The cap has to be enforced at dispatch rather than reported afterwards. What does the admission check look like, and where does it race?
How would you answer 'peak concurrency within any 5-minute window' without re-sorting?

Parse and verify a timestamped multi-signature webhook header

Easy · parsing · hmac · constant-time-compare · replay-protection

An inbound webhook carries a signature header of at most 1 KiB shaped t=<unix seconds>,v1=<64 hex chars>, with up to five v1 values during secret rotation and possibly unknown scheme keys. You hold the raw request body bytes and the currently active signing secrets. Write the parser and the verifier: accept when any active secret reproduces a signature and the timestamp is within a five-minute tolerance in either direction, reject otherwise. Single left-to-right pass over the header, no regular expression. State what is inside the MAC and why.

Approach

Parse in one scan: split on ,, then on the first = only, since a value may itself contain = under a future scheme. Accept t exactly once and treat a second t as a reject rather than last-wins. Push every v1 onto a short list and ignore any other key, so a v2 can be introduced later without breaking this verifier.
Say what is signed: HMAC-SHA256 over the exact byte string <t>.<raw body bytes>, yielding 32 bytes or 64 hex characters. The timestamp sits inside the MAC because otherwise an attacker replays yesterday's body with its still-valid signature and only has to edit the header timestamp.
Hash the bytes as received. Verifying against a re-serialised JSON body is the usual defect: key order, whitespace and number formatting all change the bytes while the parsed objects compare equal, so signatures fail for honest senders and the popular 'fix' is to stop checking.
Compare in constant time over fixed-length digests. Decode the hex to 32 bytes, accumulate acc |= a[i] ^ b[i] across the whole length, and test acc == 0 at the end. Evaluate every candidate without an early exit; at five candidates that is five HMACs over the body, linear in body size and negligible beside the network.
Apply the tolerance as a two-sided bound, rejecting when |now - t| > 300 seconds. A sender whose clock runs ahead of yours is an ordinary case, and an unbounded future timestamp is a free replay window.
Complexity: O(L) over the header producing k candidates, plus k HMACs at O(|body|) each. Space is O(k) beyond the body itself. Do the cheap rejections, including the tolerance check, before any cryptography runs.

Follow-up

The body is 40 MB. What changes about where you verify, and what can you do before the whole body has arrived?
A customer reports that signatures fail for exactly the requests whose body contains a non-ASCII character. What is your first hypothesis?
How do you rotate the signing secret with no failed deliveries, and how long do both secrets stay live?

Model credential revocation so history survives the delete

Easy · soft delete · partial indexes · check constraints · audit history

tenant_api_key stores key_id, tenant_id, workspace_id, name, key_prefix, secret_hash, scopes text[], status (active, revoked, expired, compromised), auth_version, created_at, expires_at, last_used_at, revoked_at, revoked_reason. Rotation inserts a new row and revocation never deletes, because an incident review asks which credential served a request last quarter. Write the constraints that enforce: a label is unique only among a tenant's live keys, revoked_at and status can never disagree, and scopes is never empty. Then write the authentication lookup predicate, and name one column in this table that must stay out of it.

Approach

Reach for a partial unique index rather than a plain UNIQUE: create unique index on tenant_api_key (tenant_id, name) where revoked_at is null. Any number of revoked rows may share a label, the live namespace stays unique per tenant, and the revoked majority is not in the index at all, so it stays small on a table that only grows.
Tie the nullable timestamp to the enum so the two cannot drift: check ((revoked_at is not null) = (status in ('revoked','compromised'))) and check ((revoked_at is null) = (revoked_reason is null)). A revocation that records no reason is the one an incident review cannot use.
Write the emptiness check as check (cardinality(scopes) > 0), not array_length(scopes, 1) > 0. array_length returns NULL for an empty array, a CHECK constraint passes when its expression is NULL, so the array_length version accepts exactly the value it was written to reject.
Make the lookup a single index probe with every liveness condition inside it: where secret_hash = $1 and revoked_at is null and (expires_at is null or expires_at > now()) and auth_version = $2, backed by a unique index on secret_hash. Nothing is filtered in application code, so there is no path that forgets a clause.
Keep last_used_at out of that predicate. It is written asynchronously and is allowed to lag by a minute, so it is a usage signal; feeding it into an authorisation decision makes the decision depend on a write that may be late, batched away or lost.
Flag the modelling smell while you are here: expired is derivable from expires_at < now(), so storing it as a status obliges a job to keep it true and guarantees the column is wrong between the expiry instant and that job's next run. Derive it in the predicate; keep the stored status for states that are decisions rather than clock readings.

Follow-up

Rotation issues a replacement while the old key stays live for a 30-day overlap. What does the uniqueness rule become, and what does the UI show to tell two same-named keys apart?
A password reset bumps the principal's auth_version. No row in this table changed. How does the next request fail, and what query counts how many keys that bump just killed?
A key turns up in a public repository. Which columns let you find it, and what do you write to the row?

Decide which facts an invoice line copies instead of joining

Medium · denormalisation · immutable records · foreign keys · rounding

invoice_line_item already denormalises tenant_id, which is reachable through invoice_id, and stores amount_minor even though quantity times unit_price_micros would recompute it. A reviewer asks you to normalise both away, and separately asks whether the tenant's legal name and billing address should be copied onto the invoice header. Decide each case. For every field you keep denormalised, name the read pattern or the invariant that justifies it, the anomaly the copy can develop, and the mechanism that prevents that anomaly here.

Approach

Split the question into two kinds of copy, because they fail differently. A copy of a currently mutable fact is a cache: it drifts and needs invalidation. A copy of a fact frozen at write time is not a cache at all, it is the record of what happened, and normalising it away destroys information the source no longer holds.
Keep tenant_id on the line. It costs 8 bytes, it leads every index on the table so no read is ever accidentally cross-tenant, and it turns a wrong join into an empty result rather than another tenant's money. Prevent the drift structurally: a unique constraint on invoice (invoice_id, tenant_id) plus a composite foreign key from the line on (invoice_id, tenant_id) makes a mismatched pair impossible, so the database enforces agreement instead of a code review.
Keep amount_minor. Rounding must happen exactly once, at a named site, with a stated mode (half-even here). If readers recompute from quantity and unit_price_micros, every reader owns a rounding decision, and half-up and half-even diverge systematically across thousands of lines rather than cancelling out. A check constraint can bound the stored value but deliberately cannot re-derive it.
Copy the legal name and billing address onto the invoice header, written once and never updated. The statement must show what was true when it was sealed, and the tenant record will change afterwards. This is a snapshot for the same reason source_rollup_watermark is stored per line: without it, nobody can reconstruct what the customer was told.
Name the read pattern that pays for all of it. Rendering, dispute response and export are per-tenant, per-period reads over thousands of lines that would otherwise join back to slowly changing dimensions that no longer hold the historical value. The write side is a once-per-period batch, so the extra columns cost nothing that matters.
Concede the case where the reviewer is right: a mutable operational attribute such as the tenant's current plan name has no business on a line. If a report wants it, join. If a statement needs the plan as of the period, that is another snapshot and it belongs on the header with the rest.

Worked solution (25 min)

Write the DDL: unique (invoice_id, tenant_id) on invoice, the composite FK from the line, and a comment on each denormalised column saying whether it is a snapshot or a cache.
Attempt to insert a line whose tenant_id differs from its invoice's and confirm the foreign key rejects it.
Rename a tenant, re-render a sealed invoice, and confirm the rendered name is the one stored on the header.
Recompute amount_minor from quantity times unit_price_micros for a thousand synthetic lines rounding half-up, sum both ways, and record the divergence from the stored half-even values.

Expected result: tenant_id kept and enforced by a composite foreign key to (invoice_id, tenant_id); amount_minor kept because rounding happens once with a stated mode; legal name and billing address snapshotted onto the invoice header and never updated; mutable operational attributes left normalised. Each decision paired with its read pattern, its anomaly, and the mechanism that prevents the anomaly.

Verify it yourself

A line cannot be attached to an invoice belonging to another tenant, and the rejection comes from the database rather than the application.
Re-rendering a sealed invoice after editing the tenant record produces identical output.
Rounding per line and rounding once at the end differ measurably over a thousand lines, which is the argument for storing the value.
No column on the line is recomputed by any reader.

Follow-up

Write the composite foreign key and the unique constraint it requires on the parent. What does it cost on every line insert, and what does it do to a bulk load?
A tenant is renamed after being invoiced. Which rows change, and what does the customer see on last quarter's PDF?
Where does currency live, and what breaks if a tenant's billing currency changes between two periods?

What is the difference between Spring and Spring Boot, and how does auto-configuration work behind the scenes?

Medium · system design and architecture

Approach

Name the failure you are designing for, then the recovery path.
Fix the scope first: who calls this, how often, and what they do when it fails.
State the consistency you need, and where you are willing to be stale.

Follow-up

What breaks first when traffic grows ten times?
What would you drop to keep the system up under load?

Describe how Apache Kafka guarantees message ordering and how consumer groups handle partition rebalancing during scaling.

Medium · system design and architecture

Approach

Fix the scope first: who calls this, how often, and what they do when it fails.
State the consistency you need, and where you are willing to be stale.
Name the failure you are designing for, then the recovery path.

Follow-up

What would you drop to keep the system up under load?
What breaks first when traffic grows ten times?

How do you detect, debug, and prevent memory leaks and garbage collection bottlenecks in a high-throughput Spring Boot application?

Medium · system design and architecture

Approach

Fix the scope first: who calls this, how often, and what they do when it fails.
Name the failure you are designing for, then the recovery path.
State the consistency you need, and where you are willing to be stale.

Follow-up

What breaks first when traffic grows ten times?
What would you drop to keep the system up under load?

How do you manage transaction boundaries and data consistency across distributed Microservices without using heavy two-phase commits?

Medium · system design and architecture

Approach

Fix the scope first: who calls this, how often, and what they do when it fails.
Choose a partition key and say what query it makes expensive.
Name the read and write paths separately; they rarely have the same bottleneck.

Follow-up

What would you drop to keep the system up under load?
How does this behave when that dependency is down for an hour?

Explain the internal working mechanism of a HashMap in Java, including how collisions are handled and how bucket resizing works.

Medium

Approach

Say what you would check first and why it is the highest-information step.
Work from the requirement backwards to the design.
State your assumptions explicitly before working the problem.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Webhook fan-out with per-endpoint isolation and backoff

Medium · fan-out · backpressure · circuit breaker · ordering

One domain event fans out to every matching subscription, producing a webhook_delivery row per (subscription_id, event_id, redelivery_seq). Peak unique event rate is 20k/second; attempts run five to ten times that once fan-out and retries are counted. One customer endpoint has returned 503 for six hours and its backlog holds days of events; every other customer must be unaffected. Design the delivery system: how a worker claims work, the backoff schedule, the per-endpoint circuit breaker, the queue partitioning, and whether you offer ordering per subscription. State the delivery guarantee in one sentence.

Approach

State the guarantee first, because it determines the rest: at-least-once with a stable event_id, and the consumer documented as responsible for idempotency. Exactly-once over HTTP is not deliverable - the 200 can be lost after the customer has already committed - so any design that promises it is either lying or is really offering at-most-once.
Partition work per subscription rather than into one global pool, with a concurrency cap per subscription. With a shared pool, the endpoint that has been dead for six hours consumes workers on retries that will fail, and every other customer's delivery latency rises: head-of-line blocking across tenants is the exact failure being designed against here.
Claim by compare-and-set with a fencing token: UPDATE webhook_delivery SET status = 'in_flight', lease_token = $new, leased_until = now() + interval '60 seconds' WHERE delivery_id = $1 AND status IN ('pending','failed_retryable') AND (leased_until IS NULL OR leased_until < now()), and make the terminal write carry AND lease_token = $new so a paused worker's late write is rejected rather than overwriting a newer attempt. Find due work through the partial index on next_attempt_at WHERE status IN ('pending','failed_retryable'), so the scan is proportional to live rows rather than to the terminal rows that outnumber them by orders of magnitude.
Use full jitter: sleep uniformly in [0, min(cap, base x 2^(attempt-1))]. Plain exponential backoff hands a recovering endpoint its entire backlog as one synchronised herd and knocks it over again; full jitter de-correlates it. Then check the schedule actually spans the retention you promise - with base 1 s and a 3,600 s cap, twenty attempts have an expected total elapsed time of only about 4.6 hours, so a twenty-four-hour promise needs roughly fifty-nine attempts or a larger cap.
Trip a circuit per endpoint on consecutive failures or a failure ratio over a rolling window: stop dispatching, push next_attempt_at out or mark new deliveries dropped_circuit_open, and half-open with exactly one probe rather than a batch. Bound the backlog explicitly with a per-subscription cap or retention, and decide in advance whether a recovered endpoint receives six hours of events at full rate or a pointer telling it to fetch what it missed.
Offer ordering only as an opt-in mode of one in-flight attempt per subscription, and price it honestly: with parallel attempts a retried event overtakes a newer one, so ordering requires serialisation, and serialisation means one slow endpoint blocks its own queue entirely. That converts a shared problem into that customer's own problem, which is the right place for it, but it is still a real cost.

Worked solution (35 min)

Compute the attempt rate: 20k unique events/second x mean fan-out x retry multiplier, and size worker pools and the per-subscription concurrency cap from it.
Write the claim statement and the terminal write, and point at the clause that rejects a resumed worker's stale write.
Tabulate the backoff for attempts 1 to 20 with base 1 s and cap 3,600 s, take the expected value of each full-jitter sleep as half its ceiling, and sum to get total expected elapsed coverage.
Decide the policy for a subscription down six hours: events buffered, bytes held, and what the customer actually receives when it returns.

Expected result: At-least-once with a stable event_id; per-subscription lanes with a concurrency cap; lease-and-fence claiming; full jitter whose schedule matches the retention promised - with base 1 s and a one-hour cap, twenty attempts covers only about 4.6 hours of expected elapsed time, so twenty-four hours needs roughly fifty-nine attempts or a larger cap; a per-endpoint breaker with a single half-open probe; ordering offered only as opt-in single-in-flight with head-of-line blocking named as its price.

Verify it yourself

Sum the expected sleeps in your schedule and compare against the retention window you promised the customer. If the two numbers disagree, one of them is wrong and it is usually the promise.
Execute the terminal write with a stale lease_token and confirm it affects zero rows. If it succeeds, the lease was doing the work alone and the token is decorative.
The dead endpoint's backlog must not be visible in any queue another subscription's workers poll. If one index scan serves all subscriptions, the isolation is nominal.

Follow-up

The endpoint recovers. Does it receive six hours of events at full rate, and what does that do to it?
Trace the exact code path by which an event belonging to one tenant could be signed and sent to another tenant's endpoint.
A customer insists they never received an event your row marks delivered. What evidence do you have, and what does payload_digest let you prove?

Given a failing test suite in a multi-class codebase, how do you systematically isolate bugs, correct logic errors, and ensure all test cases pass?

Medium · debugging and production issues

Approach

Establish what changed and when, before forming any theory.
Check the instrumentation before believing the symptom.
Separate the trigger from the cause; the deploy is rarely the bug.

Follow-up

What would you look at first, and what would it rule out?
How would you tell a cause from a coincidence here?

Given an array of integers, how do you find two numbers that sum up to a target value using an optimal O(N) time complexity?

Medium · debugging and production issues

Approach

Pick a bisection that eliminates candidates whichever way it turns out.
Check the instrumentation before believing the symptom.
Separate the trigger from the cause; the deploy is rarely the bug.

Follow-up

How would you tell a cause from a coincidence here?
What would you add now so this is faster to diagnose next time?

Suggested practice checklist

For someone who has spent the last few years shipping features and reading other people's code, and who has not solved a timed problem from a blank file in a long time. Five days rebuild the primitives and the patterns that sit on them, working from invariants rather than remembered solutions, and the last two attach that back to the rest of the loop.

Day 1 — Java internals and object-oriented design

Explain HashMap internals aloud: bucket index from the hash, collision chaining with tree bins for long chains since Java 8, and resizing when size exceeds capacity times load factor. Then implement a small chained hash map with resize.
Write an immutable class (final class, private final fields, no setters, defensive copies in and out) and explain why it can be shared across threads without locking, as in the bank question Make a Java Class Immutable
Sketch JVM memory (heap, stack, metaspace), distinguish JVM, JRE and JDK, and describe one garbage-collection symptom and how you would diagnose it
Give one encapsulation and one polymorphism example from code you have shipped

Deliverable: A working immutable class and chained hash map, plus four rehearsed explanations that each end with a production consequence.

Day 2 — Coding patterns from the reported questions

Solve Two Sum in O(N) with a hash map, then state what changes when the array is sorted (two pointers, O(1) extra space)
Solve longest substring without repeating characters with a sliding window, and count palindromic substrings by expanding around each center (O(n^2) time, O(1) space). Trace both on empty and single-character input.
Implement a Fisher-Yates shuffle and top-K most frequent items with a size-K min-heap (O(N log K)), and explain why swapping each position with any random index gives a biased shuffle
Work the exercise 'Find peak concurrent sandbox usage from run intervals' and compare your tie-break and null handling with its checks

Deliverable: Six solved problems, each with stated complexity and the edge cases you traced before running it.

Day 3 — Live debugging for the screening stage

Take a small multi-class Java project with a test suite, plant three bugs (an off-by-one, a broken equals/hashCode pair, an unhandled null), and fix them using only failing-test output
Use one fixed routine every time: run the suite, read the failing assertions, reproduce one failure, find the fault, make the minimal fix, re-run everything
Write a log parser that streams a large file line by line and extracts specific fields, then do the same extraction with grep and awk, as in the bank question Linux grep and Microservices Ops
Narrate one full debugging session out loud and note where you went quiet

Deliverable: A written debugging routine and a log of three planted bugs with the file order you used to find each.

Day 4 — Spring Boot, microservices and Kafka

Explain Spring vs Spring Boot and how auto-configuration applies conditional configuration classes, and know how to print the condition evaluation report to see what was applied
Write the Kafka answer: per-partition ordering, keys choosing partitions, one consumer per partition within a group, rebalancing, and how an idempotent consumer absorbs redelivery
Walk through a two-service write without two-phase commit using a saga or transactional outbox, naming the compensating action, as in the bank question Distributed Spring Batch Data Operations
Outline how you would diagnose a memory leak in a Spring Boot service: heap dumps, GC logs, and common causes such as unbounded caches or ThreadLocal leaks
Prepare how you would secure a REST microservice with OAuth2 or JWT

Deliverable: One-page answers to the five reported framework and messaging questions, each with a production example.

Day 5 — SQL, indexes and data modeling

Explain how a B-tree index serves a lookup, read an execution plan for a slow query, and state what you would change and what it costs on writes, as in the bank question Optimizing Slow SQL Queries
Map each isolation level to the anomalies it prevents under the SQL standard (dirty reads, non-repeatable reads, phantoms), as in Isolation Levels and Anomalies
Design a normalized schema for a transaction booking system with primary and foreign keys, then argue SQL vs NoSQL for financial audit logs
Work the exercise 'Decide which facts an invoice line copies instead of joining' and run its checks

Deliverable: A schema with keys and constraints, one annotated execution plan, and an isolation-level table you can reproduce from memory.

Day 6 — System design for the technical panels

Design the bank question Chat Application Design end to end: components, message delivery, storage and what happens when a service is down
Work the exercise 'Webhook fan-out with per-endpoint isolation and backoff' and check your retry schedule against the retention you promise
Sketch a payments flow that cannot create duplicate payments on retry, using an idempotency key backed by a unique constraint
For each design, state the security controls and audit logging explicitly, and what you would defer

Deliverable: Two designs with named failure modes, duplicate handling and security controls, each explainable at whiteboard depth.

Day 7 — Behavioral stories and a full mock screen

Write STAR stories for an ambiguous requirement or tight deadline, a disagreement with a senior architect, a challenging production bug and handling critical feedback, keeping only sentences about your own actions
Prepare a specific answer to why Citi and this team, and a view on using GenAI developer assistants safely in a regulated setting
Run a mock screen: Java internals questions, then a timed debugging exercise on an unfamiliar repo, then one hash map or string problem, all narrated
Go through your resume line by line and prepare the implementation detail behind each tool you list

Deliverable: Four rehearsed STAR stories, a motivation answer, and notes from one full mock screen.

Bounding a vague ticket before writing code

The reported behavioral prompts cover ambiguity, disagreement, production incidents and motivation. Structure each answer with STAR. Keep the situation short and spend most of the answer on the actions you personally took and the measurable result. Where the story allows, name the risk you controlled, such as a security check, a rollback path or an audit record, since the role involves regulated systems.

Why do you want to work as a Software Engineer at Citi, and how do your technical career goals fit the firm's technology direction?
Tell me about a time you faced an ambiguous project requirement or a tight technical deadline. How did you prioritize tasks and deliver?
How do you approach a situation where you disagree with a senior architect or teammate about an architectural or design decision?
Describe a challenging production bug you encountered. How did you troubleshoot it, communicate with stakeholders and prevent it from recurring?
What are your thoughts on integrating Generative AI and developer assistants into software engineering workflows safely within regulated industries?
Tell me about a time you received direct, critical feedback on a project. How did you respond and keep the work on track?

How do you approach a situation where you disagree with a senior architect or teammate regarding an architectural or design decision?

Medium · behavioural and engineering judgement

Approach

Give the blast radius: what could have broken, and what you measured.
Name the disagreement and how you resolved it with evidence.
Pick a story where you made the decision, not one where you watched it.

Follow-up

What would you do differently if you ran that again?
What did you decide not to do, and why?

Describe a challenging production bug you encountered. How did you troubleshoot the issue, communicate with stakeholders, and prevent recurrence?

Medium · behavioural and engineering judgement

Approach

State the situation in two sentences and spend the rest on the reasoning.
Name the disagreement and how you resolved it with evidence.
Close with what you would do differently, concretely.

Follow-up

How did you know your change caused the improvement?
What did you decide not to do, and why?

Ship metered billing with a named deduplication horizon

Medium · technical debt · deduplication · deadline pressure · detectors

Metered billing must be on in three weeks. usage_event is partitioned daily, so its unique index must include the partition key and deduplicates only within a day: a producer retry that crosses midnight, or a replay run a week later, gets through. A cross-partition dedup store is two weeks you do not have. Describe shipping with debt you named in advance: what you shipped, what you wrote down, the detector you added, the trigger and date for paying it off, and what you would have refused to ship under the same pressure.

Approach

Show you can separate the two kinds of debt, because that distinction is what the question actually probes. Debt that costs engineering time later is shippable on a deadline. Debt that silently corrupts a number a customer gets charged for is not shippable unless the corruption is detectable, and detectability is the whole negotiation.
Make the exposure narrow and measured rather than gestural. The hole is duplicates whose occurrences straddle a UTC day boundary, plus any replay older than partition retention. Measure it before arguing about it: how often an idempotency_key recurs at all, and the distribution of the gap between first and last occurrence. If the ninety-ninth percentile of that gap is four minutes, the residual risk is a small band around midnight and you can say so numerically.
Add the detector before the feature, not after. A nightly job counting keys that appear in more than one partition is one grouped scan over recent partitions, and it converts a silent overcount into a page. State what it costs to run and what it fires on.
Buy the cheap half of the real fix immediately: extend partition retention so the dedup horizon exceeds the producer's maximum retry window plus the longest replay you intend to support. That reframes retention as a correctness parameter rather than a storage cost, which is the sentence you need on record before someone optimises the bill.
Make repayment mechanical instead of aspirational: a dated entry with a named owner, plus a threshold that pulls the date forward — first detector hit above N events, or first customer dispute. Debt with a trigger gets paid; debt with only a date does not.
Answer the second half honestly by naming what you would refuse under identical pressure: the sealing path, because a sealed row is frozen and a wrong number there stops being a bug and becomes an adjustment line, a dispute and an audit question.

Follow-up

The detector fires on forty duplicate events for one tenant, and two of their invoices have already sealed. What happens next?
Whom did you tell that the billing numbers had a known hole, and in what words?
Finance asks you to cut storage by shortening partition retention. What do you say, and to whom?