Interview guides · Allianz Partners
Allianz Partners Software Engineer Interview Guide
Allianz Partners Software Engineer interview guide: reported C#, OOP, SQL and system design questions, how to prepare, and a seven-day plan.
The 60-second brief
Reported role content places Allianz Partners in insurance and assistance services. Software engineers there build and maintain the applications behind those services, work from requirements given by product managers and business analysts, join code reviews, and improve existing systems, including careful refactoring of legacy code. The skills listed are C#, SQL and OOP as must-haves, with Azure or AWS, CI/CD pipelines and finance or insurance experience as nice-to-haves.
This guide is for candidates preparing for a Software Engineer interview at Allianz Partners. It covers the five questions candidates report (designing for high-volume financial transactions, the principles of OOP, querying large SQL databases for financial reporting, keeping code scalable and maintainable, and troubleshooting a distributed-system bug), plus two related scenarios: optimising a slow query and handling dependency injection. It also gives a preparation outline for the reported loop, a seven-day plan, behavioural prompts and an FAQ. No exact round order is reported, so the guide prepares you by topic and tells you what to confirm with your recruiter.
A screening is reported first, with technical and managerial conversations after it. No round order or length is reported, so this guide prepares you by topic.
Focus areas
- C#
- Object-Oriented Programming (OOP)
- SQL
Overview
The reported questions cluster around four skills: designing an application for heavy transactional load, explaining OOP with a real example, writing and tuning SQL against large tables, and showing that your code stays maintainable in production. One further question is behavioural, about troubleshooting a bug in a distributed system. Candidates also describe two scenario prompts: how you would speed up a slow query in a high-traffic SQL environment, and how you handle dependency injection in your current projects. Nothing reported is a pure algorithm puzzle, so the preparation weight sits on explaining decisions you have actually made.
The stack matters. Candidates report C# and SQL as the core languages, with OOP principles expected and application design framed around modularity and testability. Revisit .NET specifics you can be asked to defend: dependency injection lifetimes (transient, scoped, singleton), garbage collection and IDisposable, async/await, and how Entity Framework or hand-written queries reach the database. For Entity Framework, know the usual pitfalls: N+1 queries from lazy loading, AsNoTracking for read-only reports, projecting with Select instead of loading whole entities, and reading the SQL it generates. For SQL, be ready to read an execution plan, choose an index, and explain why a join inflates a sum. Azure or AWS, CI/CD and finance or insurance experience are listed as nice-to-have, so have one or two sentences ready on each if you have used them.
The day-to-day work includes code reviews and refactoring legacy systems, and the domain is insurance and financial services. That points your answers in a direction: talk about idempotent payment or claim processing, decimal types for money, audit trails, and refactoring in small, tested steps rather than rewrites. The role content stresses clear communication with non-technical stakeholders, so practise explaining each answer once in plain language before you add the detail.
Where to focus your preparation
Candidates report a screening followed by technical and managerial stages, without a fixed order or round names. The single entry below groups what to prepare across those stages.
Preparation focus
A screening is reported first, then technical and managerial stages; some of the later interviews reportedly involve several stakeholders. No round names, order or formats are reported, so treat this entry as your preparation across all of them. Ask your recruiter which stages exist, who you will meet, and whether you will write code live. For the technical conversations, expect scenario questions about C# and .NET, OOP, SQL tuning and application design, and practise thinking aloud, since candidates report that your reasoning matters more than a memorised answer. For the managerial conversations, prepare project stories that explain why you made each technical choice.
What to demonstrate
- Working knowledge of C# and the .NET ecosystem, including dependency injection and memory management
- SQL skill: efficient queries, indexing and tuning slow queries on large tables
- Ability to structure an application so it is modular, testable and able to handle high transaction volume
- Clear explanation of your own decisions, including to people who are not engineers
How to prepare
- Write a short spoken answer for each of the five reported questions, then rehearse it aloud until you can give it without notes
- Build a C# example you can narrate: a service with constructor-injected interfaces, a unit test using a fake, and one OOP refactor from a conditional chain to polymorphism
- Take one slow query from your own work and walk through its execution plan, the index you added, and the before and after
- Prepare three project stories covering a debugging case, a design trade-off and a disagreement with a product or business colleague
Where candidates lose points
Reciting the four OOP principles as definitions without an example from a project you built
Prepare one C# example, such as replacing a switch on payment or policy type with an interface and one class per variant. Explain what changed when a new variant arrived, and what you would not abstract.
Answering the SQL reporting question with "add an index" and nothing else
Walk through the sequence: state the grain, check joins for fan-out, make the filter sargable, read the execution plan, then pick the index. Mention DECIMAL for money, NULL handling for unmatched rows and what happens at ten times the data.
Designing the high-volume transaction system without idempotency, a retry policy or a reconciliation step
Include an idempotency key on the write path, a durable queue with a dead-letter queue, timeouts and circuit breaking on downstream calls, and a reconciliation job. Keep the read model separate from the write path.
Telling a distributed-bug story that ends with "restarted the service and it went away", or that blames another team
Choose a case where you found a root cause with evidence such as traces or logs, name the cause class, show the fix and the regression test, and say what you would do differently.
Explaining decisions in jargon, or guessing silently when a question is unclear
Ask for clarification when a scenario is vague, give a one-sentence plain-language version of each answer before adding detail, and say plainly when you do not know something and how you would find out.
Questions & practice
Canonicalise a request body into a stable idempotency fingerprint
Medium · parsing · canonicalisation · hashing
idempotency_key.request_fingerprint is a SHA-256 over the method, path and canonicalised body, and a retry whose fingerprint differs must be rejected with 422 rather than served the stored response. Write the canonicaliser. Bodies are JSON up to 256 KB nested at most 32 levels; clients vary key order, whitespace and unicode escaping, and some send 64-bit ids as JSON numbers. Produce a deterministic byte string such that semantically identical bodies match and any semantic difference does not. State your complexity and name two normalisations you refuse to perform.
Approach
- Parse once into a tree, then re-serialise under fixed rules: object keys sorted, array order preserved, one escaping convention, no insignificant whitespace. Parsing is O(n) and sorting keys is O(k log k) per object, so O(n log n) overall with O(depth) stack, and the 32-level cap is enforced during parsing because hostile nesting is how a canonicaliser becomes a stack overflow.
- Sort keys by their UTF-8 bytes and say why the obvious implementation is wrong in some runtimes: a default string comparison that orders by UTF-16 code units places surrogate pairs, meaning code points from U+10000 up, below U+E000 to U+FFFF, which is not UTF-8 byte order, so two services written in different languages disagree on the same document.
- Do not re-encode numbers through a double. IEEE-754 binary64 represents integers exactly only up to 2^53, so normalising a 19-digit id through a float changes it, and 1 against 1.0 cannot be reconciled without deciding whether they are the same value. Preserve the literal token, and require ids as strings at the API boundary if you want them comparable.
- Reject duplicate keys rather than picking one. JSON permits them and parsers disagree, most keeping the last, so any choice you make ties the fingerprint to a parser detail that the code handling the request does not necessarily share.
Follow-up
- A client sends the same logical request with an extra field your API ignores. Same key, different fingerprint, so you return 422. Is that the right answer?
- Where does the fingerprint get computed relative to request decompression and the body-size limit?
Identify the heaviest tenants in a five-minute window under memory pressure
Medium · top-k · heavy hitters · streaming
The edge service handles about 3,000 requests per second across roughly 50,000 tenants, peaking near 9,000. Expose the 50 heaviest tenants by request count over the trailing five minutes so limits can be tightened before one tenant's backfill starves the fleet. You may not retain five minutes of raw records. Give the exact solution and its memory, then the bounded-memory approximation with its error stated as a formula, and say which you would ship and at what tenant cardinality that choice changes.
Approach
- Do the exact version first, because it is affordable at this cardinality: a ring of 300 one-second counters per tenant, advanced lazily, is 1,200 bytes of counters per tenant and roughly 60 to 90 MB for 50,000 tenants with overhead. Carry a running total and subtract the bucket you overwrite so a window read is O(1) rather than 300 adds.
- Extract the top 50 with a size-k min-heap over the tenant sums: O(d log k) for d tenants, against O(d log d) to sort them all. Maintaining the heap continuously instead of on query requires a tenant-to-heap-index map, because incrementing a count already inside the heap means sifting from a known position, and without that map you rebuild the heap on every request.
- State the approximation precisely rather than gesturing at sketches. Misra-Gries with m counters retains every item whose true count exceeds N/(m+1), and each retained count underestimates the truth by at most N/(m+1). With m = 1,000 and N = 900,000 requests in the window the error is roughly 900 requests, which is fine for spotting a tenant sending 50,000 and useless for ranking two tenants 200 apart.
- Say what breaks when the window slides: Misra-Gries and Space-Saving are insert-only and cannot be decremented as records age out. The workable construction is one summary per sub-window, say ten seconds, with 30 summaries merged at query time, and the merged error is the sum of the per-summary errors, so the bound degrades linearly in the number of sub-windows.
Follow-up
- The heaviest tenant is heavy because of one export job rather than user traffic. Should the limiter treat those as the same tenant?
- Two tenants sit tied at the boundary of the top 50. Does your answer flap, and does the flapping matter?
Track a rolling failure rate per destination for circuit decisions
Easy · sliding window · ring buffer · circuit breaker
The egress service delivers about 1,500 webhooks per second across roughly 40,000 destinations, each call bounded by a 10 second timeout. Maintain, per destination, the failure rate over the trailing 60 seconds so a caller can ask before dispatch whether the circuit should open. Attempts arrive as (destination_id, finished_at_ms, outcome). Requirement: amortised O(1) per attempt, with total memory bounded by the destination count rather than by traffic. Give the structure, its exact memory, and the rule that stops a destination with three attempts from opening a circuit.
Approach
- Name the exact-deque version and then reject it as the default. Holding timestamps and advancing a tail pointer past anything older than now minus 60 seconds is a correct two-pointer window at amortised O(1) per attempt, but its memory tracks in-window traffic, so one destination in a retry storm holds hundreds of thousands of entries while thousands of quiet destinations hold none.
- Use a ring of 60 one-second buckets per destination, each bucket a pair of counters for attempts and failures. On an attempt, advance the ring by the elapsed whole seconds, zeroing at most min(elapsed, 60) buckets, then increment the head. That is amortised O(1) with a fixed footprint per destination.
- State the footprint: 60 buckets times two 4-byte counters is 480 bytes of payload per destination, so 40,000 destinations is roughly 20 to 25 MB with per-entry overhead, bounded by the catalogue rather than by the rate. The cost is granularity, since the oldest bucket ages out in whole seconds, which is far tighter than the decision needs.
- Require a minimum sample before the circuit may open. A destination with three attempts and three failures reads as 100 percent and is not evidence; a floor of roughly 20 attempts in the window makes the ratio meaningful, and below that floor use a run of consecutive failures as the trigger instead.
Follow-up
- The fleet is 30 instances and each sees roughly a thirtieth of a destination's traffic. Where does the rate actually live, and what does a per-instance answer get wrong?
- A destination answers in 9.5 seconds and succeeds. It is not failing but it is consuming your per-destination concurrency. What signal should open the circuit here?
What are the most critical considerations when querying large SQL databases for financial reporting?
Medium · Technical and Domain Proficiency
Approach
- Correctness comes first: state the grain of each table, aggregate before joining when a join is one-to-many (otherwise sums inflate), and handle NULLs deliberately (SUM ignores them, COUNT(col) skips them while COUNT(*) does not, LEFT JOIN results need COALESCE). Use exact DECIMAL types and round at the stage the business rule specifies.
- Get period boundaries right: half-open ranges (posted_at >= @start AND posted_at < @nextStart) instead of BETWEEN on datetime columns, and an agreed time zone for period close. For currency, the rate depends on the reporting rule: income-statement items usually use the transaction-date or period-average rate, while balance-sheet revaluation uses the period-end rate. Agree the rule in advance and store it with the report.
- Write sargable predicates and index for them: WHERE YEAR(posted_at) = 2025 prevents an index seek, so rewrite it as a range. Build composite or covering indexes that match the filter and sort columns, then read the actual execution plan for scans, key lookups and large gaps between estimated and actual rows; keep statistics current.
- Keep reports from fighting the transactional workload: run them on a replica or under snapshot-based isolation (in SQL Server, READ_COMMITTED_SNAPSHOT or SNAPSHOT) so readers see a consistent point in time without blocking writers. Avoid NOLOCK: it reads uncommitted data and can skip or double-read rows, which is unacceptable in a financial total.
- For volume, partition large fact tables by date so queries touch only the needed partitions, precompute summary tables or a warehouse for heavy month-end reports, and page with keyset pagination rather than large OFFSETs. Make results reproducible by reconciling report totals against ledger control totals.
Follow-up
- A monthly revenue report double-counts some amounts. How do you find the join that causes it?
- Why is filtering on YEAR(created_at) slow, and how do you rewrite it?
- The month-end report blocks inserts on the transactions table. What do you change?
Denormalise tenant onto revisions and backfill it live
Hard · denormalisation · online migration · backfill
resource_revision (revision_id, resource_id, version, actor_user_id, change_kind, patch, request_id, created_at) has 400M rows and no tenant column; tenant_id lives only on resource. Two reads need it: a tenant-scoped audit feed ordered by created_at DESC, and an offboarding purge. Both join back to resource today. Justify adding tenant_id to resource_revision against those two reads, name the anomaly the copy introduces and the constraint that prevents it, then give the ordered migration for a live table taking 1.2k writes/second — the lock each step takes, how the backfill is batched, and where each step stops being reversible. PostgreSQL 16.
Approach
- Justify from the access path rather than from taste. Without the column, the audit feed either scans resource_revision by created_at and discards other tenants' rows, or resolves the tenant's resource_ids first and probes with them — both proportional to the tenant's whole history rather than to one page. With (tenant_id, created_at DESC, revision_id DESC) it is a seek that stops at 50 rows, and the purge becomes a ranged delete instead of a join.
- Name the cost exactly: a second copy of a fact can disagree with the first. Make the disagreement unwritable rather than documented — add UNIQUE (resource_id, tenant_id) on resource so it can serve as a foreign-key target, then FOREIGN KEY (resource_id, tenant_id) REFERENCES resource (resource_id, tenant_id) on the revision table. A revision can then only ever carry its parent's tenant.
- Step one, expand: ALTER TABLE resource_revision ADD COLUMN tenant_id BIGINT NULL, with no default, so it is a catalogue change and no rewrite. It still needs ACCESS EXCLUSIVE for an instant, and that instant queues behind the longest open transaction on the table while every later query queues behind it — set lock_timeout to 2s and retry rather than wait.
- Step two, dual-write: deploy the writer that populates tenant_id on every new revision while reads still use the join. Reversible by redeploying the previous build, because nothing reads the column yet.
Follow-up
- The backfill is half finished and a rollback is required. What state is the table in, and what does the previous build do with a half-populated column?
- How do you verify the backfill actually finished, given rows are still being inserted while it runs?
How would you design a software application to handle high-volume financial transactions?
Medium · Technical and Domain Proficiency
Approach
- Pin the requirements before drawing boxes: which transaction types (payments, refunds, claim payouts), the peak write rate, and the non-negotiables: money is never lost or applied twice, and every movement is auditable. Then split the write path (accept, validate, record) from the read path (balances, statements, reporting), because they scale differently.
- Make every write idempotent: the caller sends an idempotency key, I store it under a unique constraint in the same database transaction as the ledger write, and a retry returns the stored result instead of creating a second charge.
- Record money in an append-only double-entry ledger: each transaction writes signed entries that sum to zero, amounts in exact types (C# decimal, SQL DECIMAL/NUMERIC, or integer minor units), never float. Protect balance updates from lost updates with row locks or optimistic concurrency (a rowversion column checked on update).
- Scale by partitioning on account id so one account's entries serialise in one place. Publish events through a transactional outbox (the event row commits with the ledger row, a relay sends it to the queue); delivery is at-least-once, so consumers deduplicate. Serve reporting from replicas or a separate read model.
- Plan for failure: timeouts plus retries with exponential backoff and jitter, only on idempotent calls; a daily reconciliation job that matches the ledger against the payment provider's or bank's settlement files; alerts on unmatched items.
Follow-up
- The client times out, but the payment provider already charged the card. How does your design avoid a second charge when the client retries?
- A transfer debits an account on one shard and credits one on another. How do you keep it consistent without a distributed transaction?
- One settlement account receives a large share of all writes. How do you stop it becoming a hot spot?
Can you explain the core principles of OOP (Object-Oriented Programming) and provide an example of how you app
Medium · Technical and Domain Proficiency
Can you explain the core principles of OOP (Object-Oriented Programming) and provide an example of how you apply them in your current projects?
Approach
- Define the four principles with the C# mechanism for each: encapsulation (private fields, properties with private setters, invariants enforced in constructors and methods), abstraction (interfaces and abstract classes that expose what, not how), inheritance (single class inheritance, virtual/override, multiple interface implementation) and polymorphism (runtime through virtual dispatch and interfaces, compile time through overloading and generics).
- Give one concrete example from your own code, in the first person: for instance, "I made a policy's premium changeable only through a method that validates the effective date, so no caller can put the object in an invalid state" (encapsulation), and "I put each product's pricing rule behind an IPremiumCalculator interface, so adding a product meant adding a class, not editing a switch" (polymorphism, open/closed).
- Show judgement about inheritance: prefer composition and interfaces over deep hierarchies, mark classes sealed unless they are designed for extension, and explain Liskov substitution with a case where a subclass that overrides to throw breaks callers of the base type.
- Tie the principles to testability: constructor injection of interfaces lets unit tests substitute fakes, and the .NET DI container wires the real implementations. Know the three lifetimes (transient, scoped, singleton) and the captive-dependency bug when a singleton holds a scoped service such as a DbContext.
- Know the interface vs abstract class distinction precisely: an abstract class can hold instance state and constructors; an interface cannot hold instance fields, although since C# 8 it can carry default method implementations.
Follow-up
- When would you choose an abstract class over an interface in C#?
- Describe a time an inheritance hierarchy caused problems and how you refactored it.
- How do you handle dependency injection in your current project, and which lifetime would you give a database context?
How do you ensure your code is both scalable and maintainable in a production environment?
Medium · Technical and Domain Proficiency
Approach
- Split the answer into two halves and give a concrete example from your own work for each, with a number (latency, error rate, deploy frequency, time to onboard a teammate).
- Maintainability: small cohesive classes behind interfaces, constructor injection, clear module boundaries; unit tests for logic and integration tests against a real database; code review; analyzers, nullable reference types and formatting enforced in CI so style debates do not reach review.
- Scalability in .NET: keep service instances stateless so they scale horizontally, with state in the database or a cache; use async/await end to end for I/O and never block on .Result or .Wait(), which can exhaust the thread pool; reuse pooled connections and HttpClient through IHttpClientFactory; move long-running work onto a queue.
- Measure before optimising: load-test the hot paths, and use structured logs, metrics and traces with a correlation id to locate the real bottleneck (often the database, not the C# code).
- Change production safely: CI/CD with automated tests, feature flags, backward-compatible schema changes (expand, migrate, then contract), staged rollout with a rollback plan. For legacy code, write characterization tests that pin current behaviour before refactoring, and replace pieces incrementally (strangler fig) instead of rewriting.
Follow-up
- Describe a legacy refactor you made. How did you prove behaviour did not change?
- API latency climbs under load but CPU stays low. What do you check first?
- How do you rename or retype a column a running service reads, without downtime?
Read latency spikes on a sixty-second sawtooth
Medium · cache stampede · single-flight · ttl
The cached listing read path serves about 14k reads/second at an 85% hit rate. p99 sits at 35 ms for 57 seconds, jumps to 900 ms for 3, and repeats. During each spike the primary shows several hundred identical listing queries starting within the same millisecond, all carrying one large tenant's id. Cache entries use a 60-second TTL. Give the mechanism, the ordered checks, the fix, and the correctness hazard your fix must not introduce.
Approach
- Match the period to a configured number before theorising about load. A spike every 60 seconds against a 60-second TTL is an entry expiring, and you confirm it by correlating spike timestamps with the entry's write time rather than with the traffic curve. If the period had matched a cron or a GC interval instead, this is a different investigation.
- Establish the concurrency of the miss. Several hundred identical queries in one millisecond means the miss path has no coalescing: every request that arrives between expiry and repopulation recomputes. The herd size is that key's arrival rate times its recompute time, so at 1.2k reads/second for the hot key and a 250 ms recompute you expect about 300 concurrent misses, which matches what is observed.
- Add single-flight on the miss path so one caller per key recomputes under a short-lived lock while the rest wait for its result. Prefer stale-while-revalidate where the read tolerates it: return the expired value immediately and refresh asynchronously, which removes the latency spike rather than serialising it into a queue of waiters.
- De-synchronise the keys. Write TTLs with jitter, for example 60 seconds plus or minus 10%, so a deploy or a mass invalidation does not align every key on the same second and turn a per-key herd into a fleet-wide one.
Follow-up
- The same sawtooth appears on a key that is invalidated on write rather than expired. Is that the same bug?
- How does your answer change if the recompute takes 4 seconds instead of 250 ms?
Suggested practice checklist
The plan follows the reported topics in order: stack fundamentals first, then OOP, SQL, transaction design, and finally maintainability and stories.
Day 1 — Map the loop and the stack
- Send your recruiter questions: which stages exist, who you will meet, and whether you will write code live.
- Write a one-page inventory of your C#, .NET, SQL and Azure or AWS experience, marking each topic strong, rusty or missing.
- List five projects from your work and, for each, the technical choice you would be asked to justify.
Deliverable: A sent recruiter message and a one-page skills inventory with five project choices listed.
Day 2 — C#, .NET and dependency injection
- Write a small C# service that takes its dependencies as interfaces through the constructor. Experiment: register a dependency under each lifetime in turn, log instance ids across two requests to see when instances are shared, then reproduce the captive-dependency error by injecting a scoped service into a singleton.
- Write a unit test for it using a fake dependency, and explain which lifetime suits a database context and why.
- Prepare a spoken answer on memory management: GC generations 0/1/2 and the large object heap, using/IDisposable for connections and streams, why finalizers are only a fallback, and one leak you have seen (event handlers, static caches). Add a short answer on async/await.
Deliverable: A working C# service with a test, a written one-paragraph answer on dependency injection lifetimes, and an outline of the memory-management answer.
Day 3 — OOP with a real example
- Take a method with a long conditional on type, and refactor it into an interface with one class per variant, keeping the tests green.
- Write a spoken answer for the OOP principles question that uses this refactor as the example, covering composition versus inheritance.
Deliverable: A before and after refactor and a spoken answer on OOP.
Day 4 — SQL for financial reporting and slow queries
- Write a report query over a one-to-many join and show the total inflating, then fix it by aggregating at the right grain.
- Take a slow query, read its execution plan, change the predicate or add an index, and record the plan before and after.
- Practise Entity Framework pitfalls: provoke an N+1 query through lazy loading, use AsNoTracking for a read-only report, project with Select instead of loading whole entities, and read the SQL that EF generates.
Deliverable: A fixed report query, a before and after plan comparison, notes on the ORM pitfalls, and a written backfill outline.
Day 5 — Designing for high-volume financial transactions
- Sketch the design on paper: write path with an idempotency key, queue with dead-letter handling, read model, and reconciliation job.
- Say the design aloud, ending with what breaks first at ten times the traffic.
Deliverable: One annotated design diagram and a spoken answer, plus completed solutions to both practice exercises.
Day 6 — Maintainable code and debugging stories
- Write your answer on keeping code scalable and maintainable, including one legacy refactor you can describe with tests around it.
- Write your distributed-system debugging story in the order symptom, evidence, cause, fix, prevention.
Deliverable: Two written answers (maintainability and the debugging story) and a debugging write-up for the practice exercise.
Day 7 — Full spoken mock
- Answer all five reported questions aloud in a row, and give a plain-language version of each for a non-technical listener.
Deliverable: A recorded mock covering all five questions, with notes on gaps, and two questions for interviewers.
Behavioural preparation: debugging and decision stories
The one behavioural question reported for this loop is about troubleshooting a complex bug in a distributed system. Candidates also report that the reasoning behind technical choices matters, and the role content stresses clear communication with non-technical stakeholders. Prepare stories where you made the decision, with evidence and a measured result, and keep each to a clear sequence: situation, your reasoning, the action, what happened.
- Tell me about a technical decision you made and can explain in plain language to a non-technical stakeholder. What trade-off did you accept?
- Describe a time you worked in a legacy codebase and improved it without breaking existing behaviour.
- Tell me about a time a requirement from a product manager or business analyst was unclear. How did you resolve it?
- Describe a time you did not know the answer to a technical question at work. What did you do next?
Describe a time you had to troubleshoot a complex bug in a distributed system.
Medium · Technical and Domain Proficiency
Approach
- Choose an incident where you traced the cause across more than one service, queue or database, such as duplicate messages, mismatched data between systems, or timeouts under load. Give the symptom and the user or business impact in two sentences, then spend the rest on how you found the cause.
- Narrate your diagnostic sequence in order: what changed recently, how you followed one request across services with a correlation ID or distributed trace, how you compared a healthy path with a failing one, what hypothesis you formed, and how you reproduced it with controlled inputs such as delayed responses, retries or reordered messages. Say that you changed one variable at a time.
- Name the class of cause and the evidence behind it: a race condition, a retry without idempotency creating duplicates, a partial failure, a stale cache, out-of-order delivery, or mismatched timeouts between callers and callees. A bug of this kind is usually found by reasoning from the logs, not by guessing, so say what ruled out the other candidates.
- Close with the fix, why it was safe to ship, and what now prevents a repeat: a regression test that forces the failing interleaving, an alert or dashboard, an idempotency key, or a changed timeout. Quote only numbers you really measured, such as the error rate before and after.
- The mistake that sinks this story is a case where you restarted something and it stopped, with no root cause, or where you credit another team and take no ownership. Pick a story where you made the call, and end by saying what you would do differently.
Follow-up
- How did you rule out the other services before settling on this one?
- How did you reproduce the problem if it only happened in production?
- What monitoring or test did you add so a repeat would be caught earlier?
Reverse your own decision and price the reversal
Medium · reversibility · measurement · migrations
Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.
Approach
- State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
- Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
- Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
- Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.
Follow-up
- What in that decision was irreversible, and did you know it was irreversible when you made it?
- How did you tell the people who had already built on top of the original decision?
Estimate work you have never done and defend the range
Hard · estimation · backfills · expand-contract
You are asked to estimate a change you have never attempted: add a column to a 100-million-row table, populate it, move reads across, and drop the old shape. Give a range with the assumptions that generate it, including batch size, the signal your backfill throttles on, and wall-clock hours, and name the three unknowns that would move the number most. Then describe a real estimate you gave under comparable ignorance: how you expressed its uncertainty, what you committed to, and how wrong you turned out to be.
Approach
- Decompose into independently deployable steps before estimating anything: add the column nullable, write both shapes, backfill in batches, verify, move reads, stop writing the old shape, drop it. That is four deploys spread over days, and the calendar estimate is dominated by them rather than by the loop's runtime.
- Do the arithmetic aloud for the part that has arithmetic in it: batch size times number of batches times per-batch duration, at a write rate the primary can absorb alongside roughly 1.2k writes per second of production traffic. The loop is throttled by replication lag and lock waits, not by how fast it can issue statements.
- Price the schema step by its lock rather than its statement duration. In PostgreSQL an ALTER TABLE taking ACCESS EXCLUSIVE waits for every open transaction on that table while later queries queue behind it, so a millisecond change issued during a thirty-second analytics query stalls that table for thirty seconds. Adding a nullable column with a non-volatile default avoids a rewrite from version 11; a new index wants CREATE INDEX CONCURRENTLY, which cannot run inside a transaction block and leaves an invalid index behind if it fails.
- Express the answer as a range whose endpoints each trace to a stated assumption, then name the cheapest experiment that collapses it, which is almost always running one real batch against the real table and multiplying.
Follow-up
- How do you verify the backfill genuinely finished, given rows written by production traffic while it ran?
- Where does the backfill resume from after a worker is killed mid-batch, and what makes that resume point trustworthy?