CanDeal Group Software Engineer Interview Guide

The 60-second brief

CanDeal Group runs an electronic marketplace for Canadian fixed-income and derivatives trading. Candidates describe the Software Engineer role as work on trading platforms, data services and connectivity, with distributed systems, networking protocols and real-time data processing in the mix. The reported responsibilities cover the whole delivery cycle: gathering requirements with product managers and other engineering teams, writing well-tested code for trading applications, deploying, supporting the platform afterwards and paying down technical debt.

This guide is for candidates preparing for the CanDeal Group Software Engineer loop. It covers the reported technical questions on TCP and UDP, the OSI model, DNS, load balancing and multi-threaded concurrency, plus the production-debugging story candidates are asked for. The requirements candidates describe name a backend language such as Java, C++ or C#, TCP/IP and HTTP/S, and relational databases, with 3+ years in backend or systems work described as a good fit.

Candidates report an HR screening followed by one or more technical rounds. No round names, order or formats beyond that are reported, so confirm the stages with your recruiter.

Focus areas

• Networking Fundamentals
• Computer Networks Concepts
• Protocol Understanding (General)

Overview

Reports for this role put networking fundamentals at the centre. The six reported questions are: TCP versus UDP, mapping the OSI model to network troubleshooting, a DNS lookup from start to finish, load balancing under heavy traffic, concurrency in multi-threaded code, and a production bug fixed under a tight deadline. Candidates also report two broader scenarios: what happens when you type a URL into a browser, and how you would handle a network partition in a distributed system. Five of these are explain-and-reason questions, so the preparation is to rehearse structured spoken answers rather than to grind coding problems.

Because the product is a marketplace for fixed-income and derivatives trading, it helps to connect each answer to a realistic trading-style case. For example, you can explain UDP through a price feed where a stale packet is worth less than the next one and gaps are detected with sequence numbers, and TCP through a session that must deliver every message in order. Do this as an illustration of your own reasoning, not as a claim about how CanDeal builds its systems. The same applies to latency versus throughput versus fault tolerance, which candidates report as the trade-offs to discuss in design answers.

Candidates also report deep questions about the architecture and challenges of projects on your resume. Pick the two or three projects you know best, and for each one be ready to draw the components, name the protocols and database interactions, and describe one failure and how it was diagnosed. That inventory doubles as raw material for the production-debugging story and the behavioral questions about stress, teamwork and learning from mistakes.

If your day job is not in Java, C++ or C#, choose one of them for concurrency examples, since the requirements name those as the backend languages. Nice-to-have items candidates mention are financial market data, cloud-native technologies and low-latency techniques; read up on them lightly after the fundamentals are solid.

Where to focus your preparation

Candidates report an HR screening followed by one or more technical rounds, with no further round names reported. The single entry below stands for the technical preparation as a whole.

Preparation focus

No named rounds are reported beyond an HR screening and one or more technical rounds, so this stage covers your preparation for all of them. Candidates report that the technical conversations lean on core computer science and networking, on solving problems live, and on detailed questions about the technologies and projects on your resume. Prepare spoken answers in a fixed order: definition, mechanism, a concrete case, then trade-offs. Practise the reported questions on TCP versus UDP, the OSI model and troubleshooting, DNS, load balancing, concurrency and the production-debugging story, and ask your recruiter whether any stage involves writing code.

What to demonstrate

• Working knowledge of the network stack and protocols, from DNS and TCP through HTTP/S
• Handling of concurrency in multi-threaded code
• Reasoning about trade-offs between latency, throughput and fault tolerance
• How well you can discuss the architecture and problems of projects on your resume

How to prepare

• Write a one-page comparison of TCP and UDP with a use case for each, then say it aloud without notes
• Walk a request from URL to rendered page, naming the DNS, TCP, TLS and HTTP steps in order
• Build a layer-by-layer troubleshooting ladder (link, IP, transport, TLS, application) with one command per rung
• Pick three resume projects and prepare a component sketch and one failure story for each

Where candidates lose points

Answering TCP versus UDP as a feature list (reliable, unreliable) with no use case or cost attached

State what TCP pays for its guarantees: the handshake, retransmission delay and head-of-line blocking. Then give one case for each protocol, such as an ordered request-response session versus a price feed where late data is useless and you detect gaps yourself.

Reciting the seven OSI layer names when asked how the model maps to troubleshooting

Turn the layers into a ladder of checks: link state, IP address and route, port reachability, TLS handshake, then the application response. Name a tool or symptom for each rung, and say how a failure at one layer rules out the layers above it.

Saying "I would add locks" for the concurrency question without saying what state is shared

Name the shared mutable state first, then pick a tool for it: immutability, confinement to one thread, an atomic, a lock or a concurrent collection. Cover lock ordering for deadlock and say what each choice costs under contention.

Telling the production-debugging story as heroics with no sequence, no root cause and no follow-up

Structure it as symptom, first check, how you narrowed it, root cause, fix under the deadline, and the change that stopped it recurring. Quote the pressure briefly and spend the time on your reasoning.

Listing technologies on your resume that you cannot discuss below the surface

Since candidates report detailed resume questions, cut or flag anything you have only touched. For each remaining item prepare how it was deployed, how it failed and what you would change.

Questions & practice

How do you handle concurrency issues in a multi-threaded application?

Medium · Technical and Networking Fundamentals

Approach

1. Start by naming the shared mutable state and the invariant it has to keep. Most bugs are check-then-act or read-modify-write races: count++ is a load, an add and a store, so two threads can lose an update.
2. Choose the tool to fit the state. A mutex or synchronized block with a short critical section protects multi-field invariants. Atomics and CAS (AtomicLong, std::atomic, Interlocked) cover single variables. Immutability and thread confinement avoid sharing altogether. Concurrent collections help, but compound operations still need their atomic methods (computeIfAbsent, merge), not a get followed by a put.
3. Address visibility as well as atomicity. Without a happens-before edge, another thread may never see a write. Java volatile gives visibility and ordering but does not make count++ atomic. Double-checked locking is only correct with a volatile field.
4. Prevent deadlock by breaking one of the four Coffman conditions: acquire locks in a fixed global order (for example by account id in a transfer), use tryLock with a timeout, and never call unknown code while holding a lock. Also mention livelock and starvation, and fairness as their trade-off.
5. Prefer structures that shrink the problem: a producer-consumer pipeline on a bounded blocking queue, a single writer that owns the state, and thread pools instead of ad hoc threads. Verify with stress tests, ThreadSanitizer for C++, and thread dumps (jstack) to find a deadlock in a running process.

Follow-up

• Two threads transfer money between the same two accounts in opposite directions and hang. Why, and how do you fix it?
• When would you pick a lock-free CAS loop over a mutex, and what is the ABA problem?
• How would you reproduce a race that shows up only under production load?

Identify the heaviest tenants in a five-minute window under memory pressure

Medium · top-k · heavy hitters · streaming

The edge service handles about 3,000 requests per second across roughly 50,000 tenants, peaking near 9,000. Expose the 50 heaviest tenants by request count over the trailing five minutes so limits can be tightened before one tenant's backfill starves the fleet. You may not retain five minutes of raw records. Give the exact solution and its memory, then the bounded-memory approximation with its error stated as a formula, and say which you would ship and at what tenant cardinality that choice changes.

Approach

1. Do the exact version first, because it is affordable at this cardinality: a ring of 300 one-second counters per tenant, advanced lazily, is 1,200 bytes of counters per tenant and roughly 60 to 90 MB for 50,000 tenants with overhead. Carry a running total and subtract the bucket you overwrite so a window read is O(1) rather than 300 adds.
2. Extract the top 50 with a size-k min-heap over the tenant sums: O(d log k) for d tenants, against O(d log d) to sort them all. Maintaining the heap continuously instead of on query requires a tenant-to-heap-index map, because incrementing a count already inside the heap means sifting from a known position, and without that map you rebuild the heap on every request.
3. State the approximation precisely rather than gesturing at sketches. Misra-Gries with m counters retains every item whose true count exceeds N/(m+1), and each retained count underestimates the truth by at most N/(m+1). With m = 1,000 and N = 900,000 requests in the window the error is roughly 900 requests, which is fine for spotting a tenant sending 50,000 and useless for ranking two tenants 200 apart.
4. Say what breaks when the window slides: Misra-Gries and Space-Saving are insert-only and cannot be decremented as records age out. The workable construction is one summary per sub-window, say ten seconds, with 30 summaries merged at query time, and the merged error is the sum of the per-summary errors, so the bound degrades linearly in the number of sub-windows.

Follow-up

• The heaviest tenant is heavy because of one export job rather than user traffic. Should the limiter treat those as the same tenant?
• Two tenants sit tied at the boundary of the top 50. Does your answer flap, and does the flapping matter?

Diff a projection against the primary without per-row point reads

Hard · reconciliation · range hashing · throttling

The listing projection has drifted and some rows show a stale version. The primary holds 40,000,000 resource rows across 12,000 tenants while serving 1,200 writes and 14,000 reads per second. The obvious repair, reading each resource row and comparing its version against the projection, is correct and would eventually finish. Explain precisely why it is unacceptable here, then give a diff that finds the differing rows, state its complexity, and make it safe to run against a live primary. Replication lag is usually under 100 ms and is not bounded.

Approach

1. Quantify the naive cost rather than calling it slow: 40,000,000 point reads at even 0.5 ms each is over five hours serialised, and the only lever is concurrency, which is exactly what you cannot spend. The primary's pool is sized for the write path, and 40,000,000 random reads evict the buffer cache that sustains the 85 percent cache hit rate, so the audit degrades the system it is auditing.
2. Replace random access with one ordered pass per side. Both sides can be read in (tenant_id, resource_id) order, which is a sequential scan on each and a merge join in O(n) time and O(1) memory. For a dense diff that is the whole answer, and it reads the primary once instead of 40,000,000 times.
3. For the expected sparse case, compare range hashes instead of rows: partition the key space, compute per range an order-independent aggregate over hash(resource_id, version), compare aggregates, and descend only into ranges that differ. With d differing rows and branching factor B, at most d ranges mismatch per level, so the drill-down examines O(d log_B(n/d)) ranges and reads full rows only in mismatching leaves.
4. Aggregate with a sum modulo 2^64 or a multiset hash, never XOR. XOR is order-independent but self-cancelling, so two rows wrong in the same way, or a row duplicated on one side, leave the range aggregate matching and the range is declared clean.

Follow-up

• The diff reports 900 stale rows. How do you decide between patching those rows and rebuilding the projection from resource_revision?
• Same job, but the projection lives in a search index that cannot be scanned in key order. What changes?

Write the update path that detects a concurrent edit

Medium · optimistic concurrency · transactional outbox · audit history

resource carries version INT NOT NULL DEFAULT 1. resource_revision holds revision_id, resource_id, version, actor_user_id, change_kind, patch JSONB, request_id, created_at with UNIQUE (resource_id, version). outbox_event holds aggregate_type, aggregate_id, aggregate_version, event_type, payload, status. A PUT carries the version the client read. Write the exact statements for the single transaction that applies the edit, records the revision and enqueues 'resource.updated', and give the handler's branch on zero affected rows. Then say what PostgreSQL 16 does under READ COMMITTED when two of these updates hit one row at once.

Approach

1. One transaction, three writes, no network call inside it: UPDATE resource SET title = $3, version = version + 1, updated_at = now() WHERE resource_id = $1 AND tenant_id = $4 AND version = $2; then INSERT the resource_revision row at version $2 + 1; then INSERT the outbox_event row at the same aggregate_version. The event goes to a table rather than a broker because no transaction spans both.
2. Branch on the affected-row count before doing anything else. Zero has three causes — stale version, wrong tenant, row gone — so re-read once and map to 409 carrying the current version, or 404 for an id outside the caller's tenant, which also stops the endpoint confirming that another tenant's id exists.
3. State the engine behaviour instead of assuming it. Under READ COMMITTED the second UPDATE blocks on the row lock, and when the first commits PostgreSQL re-evaluates the WHERE clause against the newly committed row, so the version predicate now fails and the statement reports zero rows. Under REPEATABLE READ the identical collision raises SQLSTATE 40001 instead, so the handler must fold both shapes into one conflict response.
4. Keep UNIQUE (resource_id, version) even though the predicate already serialises writers. It is what makes a lost update unwritable if any other path ever reaches the revision table, and it converts a logic bug into 23505 rather than into a silently missing history row.

Follow-up

• A client sends the version it read ten minutes ago and the resource has moved three versions. What is in your 409 so it can resolve the conflict without a full re-fetch?
• Two editors, two disjoint fields, no overlap. Does your answer still refuse the second write, and should it?

Replace offset paging on the resource feed with keyset

Easy · keyset pagination · composite index · sort order

resource holds resource_id, tenant_id, owner_user_id, title, body_ref, version, status ('draft','active','archived','deleted'), created_at, updated_at, deleted_at, with an index on (tenant_id, status, updated_at DESC, resource_id DESC). The listing endpoint returns active resources for one tenant, newest update first, 50 per page, today with LIMIT 50 OFFSET n. Tenants reach page 400 and rows are created while they read. Write the keyset query, define what the cursor carries and how it is encoded, and say which part of the index each predicate uses. Assume PostgreSQL 16.

Approach

1. Name the two failures separately. OFFSET 20000 makes the server produce and discard 20,000 rows, so page cost grows with depth rather than with page size. Independently, any write that changes how many rows sort above the offset moves the window between two fetches, and the direction decides which anomaly you get: an insert lands at the head of updated_at DESC and pushes already-returned rows down past the boundary, so they are returned a second time; a delete above the offset, or a row whose updated_at is bumped above the cursor, pulls rows up and one is never returned at all. Nothing in the response reveals either.
2. Write the seek: WHERE tenant_id = $1 AND status = 'active' AND (updated_at, resource_id) < ($2, $3) ORDER BY updated_at DESC, resource_id DESC LIMIT 50. The row-value comparison is one index range rather than a disjunction, and both columns are NOT NULL, which is what makes that comparison well defined.
3. Map each predicate onto the index: tenant_id and status are equality on the leading columns, (updated_at, resource_id) is the range, and the ORDER BY matches the index order so no Sort node appears and the scan stops after 50 rows. The DESC in the definition only matters for mixed directions — a plain ascending btree on the same columns is read backwards for this query.
4. Put both sort columns in the cursor and nothing the client can tamper with into another tenant: base64 of (updated_at, resource_id), validated server-side, with tenant_id taken from the principal.

Follow-up

• The client asks for 'jump to page 400'. What do you offer instead, and what does the honest version cost?
• Sort order becomes user-selectable across four columns. How many indexes is that, and which would you refuse to add?

Explain the difference between TCP and UDP and identify specific use cases for each.

Medium · Technical and Networking Fundamentals

Approach

1. Contrast the guarantees first. TCP sets up a connection with a SYN, SYN-ACK, ACK handshake and gives you a reliable, ordered byte stream through sequence numbers, ACKs and retransmission, plus flow control (the receive window) and congestion control (slow start, congestion avoidance). UDP sends independent datagrams with an 8-byte header. It has no handshake, no ordering and no retransmission, and it keeps message boundaries.
2. Explain the framing consequence. TCP is a byte stream, so the application has to frame its own messages with a length prefix or a delimiter, and one read can return half a message or several. With UDP, each receive returns exactly one datagram.
3. Name the latency costs of TCP. A lost segment stalls delivery of every later byte (head-of-line blocking) until it is retransmitted. Nagle's algorithm can delay small writes, which is why latency-sensitive senders set TCP_NODELAY.
4. Give use cases with a reason for each. TCP: HTTP/1.1 and HTTP/2, TLS, SSH, database connections and order or request flows where every message must arrive once and in order. UDP: DNS queries, voice and video, games, and multicast data feeds (multicast exists only over UDP), where late data is worth less than missing data.
5. Mention the middle ground. QUIC (HTTP/3) builds reliability, ordering per stream and congestion control on top of UDP to avoid TCP's cross-stream head-of-line blocking. Feeds over UDP usually add sequence numbers so the receiver can detect gaps.

Follow-up

• If you carry a price feed over UDP multicast, how does a receiver detect a gap and recover from it?
• What does Nagle's algorithm do, and why does it interact badly with delayed ACKs for small request-response messages?
• When does DNS switch from UDP to TCP?

Describe the process of a DNS lookup from start to finish.

Medium · Technical and Networking Fundamentals

Approach

1. Start on the client. The browser checks its own cache, then the OS stub resolver checks the hosts file and the OS cache, and on a miss it sends a recursive query, normally over UDP port 53, to the configured recursive resolver.
2. Describe the iterative walk the resolver does on a cache miss. It asks a root server (found from its root hints), which refers it to the TLD name servers for .com. The TLD servers refer it to the domain's authoritative name servers, with glue records when those servers sit inside the zone. The authoritative server returns the A or AAAA record, or a CNAME that the resolver then has to follow.
3. Explain caching. Every answer carries a TTL, and each layer (resolver, OS, browser, some runtimes such as the JVM's networkaddress.cache.ttl) caches up to it. NXDOMAIN answers are cached too (negative caching, bounded by the zone's SOA). That is why changes propagate slowly.
4. Cover the transport details: a truncated response (TC bit) makes the client retry over TCP, EDNS0 allows larger UDP payloads, DoT and DoH encrypt the client-to-resolver leg, and DNSSEC signs records so the resolver can verify them.
5. Finish with the edge cases that matter in production: failover that is slow because clients hold an old record past the change, an apex domain that cannot be a CNAME because it must carry SOA and NS records, and resolver outages that look like application outages.

Follow-up

• Before a migration, why lower the TTL ahead of time, and why can some clients still reach the old IP afterwards?
• How does DNS-based load balancing or failover differ from putting a load balancer behind one address?
• How would you debug a name that resolves on your laptop but not inside a container?

Can you explain how load balancing works in a high-traffic environment?

Medium · Technical and Networking Fundamentals

Approach

1. Separate layer 4 from layer 7. An L4 balancer forwards TCP or UDP flows by IP and port, which is cheap and protocol-agnostic but blind to requests. An L7 balancer terminates HTTP (usually TLS too), so it can route by path, header or cookie and retry failed requests, at the cost of more CPU and a second hop for the connection.
2. Compare the algorithms with the property that decides between them. Round robin and weighted round robin suit uniform requests. Least connections handles request costs that vary. Consistent hashing or source-IP hashing gives affinity and cache locality, and adding a node only moves about 1/N of the keys. 'Power of two random choices' gets close to least-loaded without global state.
3. Describe health and lifecycle. Active health checks plus passive outlier detection take bad backends out of rotation. Connection draining lets a deploy finish in-flight requests. A slow-start ramp keeps a recovered node from being flooded.
4. Remove the balancer as a single point of failure: an active-passive pair sharing a floating IP (VRRP or keepalived), several balancers behind DNS or anycast, and global balancing across regions with health-aware DNS.
5. Name the hard cases. Sticky sessions break when a node dies, so it is better to keep services stateless and put sessions in a shared store. Long-lived persistent connections do not rebalance when nodes are added until clients reconnect. Unbounded retries can turn one slow backend into a retry storm, so cap them with timeouts and retry budgets.

Follow-up

• Clients hold persistent TCP connections for hours. After you add three servers, how do you get load onto them?
• When would least connections give worse results than round robin?
• How do you keep the load balancer itself from being the outage?

How does the OSI model map to real-world network troubleshooting?

Medium · Technical and Networking Fundamentals

Approach

1. Say that the OSI model is a checklist that tells you which tool to reach for. In practice the TCP/IP stack folds layers 5-7 into the application, so troubleshooting means checking link, network, transport and application in turn.
2. Walk the layers with concrete checks. Link: interface state, error counters and the ARP or neighbour table (ip link, ip neigh). Network: addressing and routes (ip route), ping, traceroute or mtr. Transport: whether a port is listening (ss -tlnp), reachability (nc -vz host port), firewalls and security groups. Application: dig for DNS, openssl s_client for TLS, curl -v for HTTP status and headers.
3. Bisect rather than always starting at the bottom. If curl to the IP works but curl to the hostname fails, the network is fine and DNS is the suspect. If ping works but the port does not, look at layer 4 and above.
4. Read the failure signature. 'Connection refused' means the host answered with a RST, so routing works and nothing is listening. A timeout means packets are being dropped, usually by a firewall or a broken route. A tcpdump showing a SYN with no SYN-ACK confirms the drop.
5. Cover the classic edge cases: MTU or path-MTU black holes (small requests succeed, large responses hang; test with ping using the don't-fragment flag and a large size), a duplex mismatch causing intermittent loss, and TLS failures that look like network failures (expired certificate, SNI mismatch).

Follow-up

• A service is reachable by IP but not by hostname from one host only. What do you check next?
• Small API calls succeed but large responses hang. What is your hypothesis, and how do you confirm it?
• How would you prove whether packets are leaving the client or being dropped on the way?

Describe a time you had to debug a production issue under a tight deadline.

Medium · Problem-Solving and Logic

Approach

1. Open with a real incident in concrete terms: what broke, who was affected, what the deadline was and what was at stake. A vague 'the system was slow' story loses the listener straight away.
2. Show the order you worked in. Stabilise first (roll back, flip a feature flag, fail over) if that stops the harm, then find the root cause. Ask what changed (deploy, config, traffic, a dependency, data), then narrow it with logs, metrics and traces, testing the cheapest hypothesis first.
3. Name the evidence that confirmed the cause: the log line, the metric that diverged, the request you reproduced. Explain how you checked that the fix worked, for example error rate back at baseline or the repro now passing, rather than hoping it held.
4. Cover the people side in a sentence or two: who you kept informed and how often, and who made the call to roll back or ship the fix. Use 'I' for your own actions so your part is clear.
5. End with what changed afterwards: a new alert, a test, a runbook or a postmortem action. The story falls apart if the fix was luck, if the blame lands on someone else, or if there is no measurable result.

Follow-up

• If rolling back had not been possible, what would you have done instead?
• How did you know you had fixed the root cause and not just hidden the symptom?
• What would have caught this issue before it reached production?

Suggested practice checklist

Seven days, ordered from the transport layer up through DNS, troubleshooting, load balancing and concurrency, ending with the debugging story and a full mock. Each day produces something you can check against the reported questions.

Day 1 — Confirm the format and inventory your resume

• Ask your recruiter which stages follow the HR screening and whether any involves writing code
• List every technology and project on your resume, with one sentence on its architecture and one on a failure you handled
• Mark the three projects you could discuss in most depth and choose a language (Java, C++ or C#) for concurrency examples

Deliverable: A list of questions for your recruiter and a resume inventory with the three strongest projects marked.

Day 2 — Transport layer: TCP versus UDP

• Write a comparison covering the three-way handshake, byte-stream versus datagram, retransmission, ordering, flow control and congestion control
• Capture a TCP connection with tcpdump or Wireshark and label the SYN, SYN-ACK and ACK packets and the teardown
• Prepare one use case per protocol and the cost you accept by choosing it

Deliverable: A one-page comparison, an annotated packet capture and a spoken answer to the TCP versus UDP question.

Day 3 — DNS and the type-a-URL walkthrough

• Run dig +trace on a domain and note the root, top-level and authoritative steps
• Write the full path from stub resolver and caches through the recursive resolver, record types, TTLs and the UDP-to-TCP fallback
• Extend it into a complete URL-to-page walkthrough (DNS, TCP, TLS, HTTP request, response) and say it aloud

Deliverable: An annotated dig trace and a spoken DNS answer that continues into the full browser walkthrough.

Day 4 — OSI model as a troubleshooting ladder

• For each rung (link, IP, transport, TLS, application) pick one command and one failure it reveals, such as ping, traceroute, nc, ss, openssl s_client and curl -v
• Invent three outages (DNS misconfiguration, closed port, expired certificate) and walk the ladder for each
• Practise the answer so it starts from a symptom and ends with the layer that explains it

Deliverable: A one-page troubleshooting runbook and three worked outage walkthroughs.

Day 5 — Load balancing and network partitions

• Compare layer 4 and layer 7 balancing, and the algorithms round robin, least connections and consistent hashing
• Cover health checks, connection draining, session affinity and the balancer itself as a single point of failure
• Sketch how a service behaves during a network partition, what you refuse or serve stale, and how it recovers

Deliverable: A diagram of a balanced service with its failure modes, and spoken answers to the load balancing and partition scenarios.

Day 6 — Concurrency in multi-threaded code

• In your chosen language, write a lost-update counter race, then fix it three ways: a lock, an atomic and confinement to one thread
• Reproduce a lock-ordering deadlock and fix it with a consistent order
• Work through the practice question on detecting a concurrent edit in an update statement, which carries the same idea into a relational database

Deliverable: Running code for the race and its fixes, plus a spoken answer that names shared state before naming locks.

Day 7 — Production-debugging story and full mock

• Write the debugging story as symptom, first check, narrowing, root cause, fix and prevention, and rehearse it aloud
• Run a mock that mixes all six reported questions and one resume deep dive, with someone interrupting for detail
• Do the practice exercise on turning a review disagreement into a decision for a second collaboration example

Deliverable: A rehearsed debugging story, a completed mock and a list of the answers that still need work.

Behavioural preparation

Candidates report that behavioral questions cover handling stress, working with teams and learning from mistakes, and they are asked for a production issue fixed under a tight deadline. The reported guidance is to structure answers with Situation, Task, Action, Result and to stay composed and analytical when requirements change unexpectedly. Prepare stories where you can show your own sequence of reasoning.

1. Describe a time you had to debug a production issue under a tight deadline.
2. Tell me about a time project requirements changed unexpectedly. How did you keep the work on track?
3. Describe a mistake you made in a system you owned and what you changed afterwards.
4. Tell me about working with a product manager or another engineering team to get a feature to match a business need.
5. Describe technical debt you found and fixed before it turned into an outage.

Turn a code review disagreement into a decision

Easy · code review · optimistic concurrency · communication

A colleague's change updates a row with UPDATE resource SET version = version + 1 WHERE resource_id = $1 AND version = $2 and treats an affected-row count of zero as a successful no-op. You read that as a silently lost update; they think returning 200 is friendlier to clients than returning a conflict. Describe how you have handled a review disagreement of this shape: what goes in the comment, when you leave the thread, and who decides. Then write the comment you would leave here, in under 80 words.

Approach

1. Sort the disagreement before writing anything. A silently discarded write is a correctness claim about data; the choice between 409 and 412 is taste. Only the first justifies blocking a merge, and saying which one you are doing is most of the value of the comment.
2. Make the claim reproducible in the comment itself with an interleaving rather than a principle: A reads version 7, B reads version 7, B commits version 8, A's predicate matches zero rows, A is told it succeeded and A's edit is gone.
3. Offer the alternative with its cost attached: return 409 carrying the current version and the revision that won, so the client can re-read and re-apply. Note that automatic retry is not the fix, because a retry re-reads the winner's state and reapplies an intent formed against data that no longer exists.
4. Apply an escalation rule you can state: two round trips on the thread, then a call, and the service's owner decides rather than the reviewer. A reviewer who cannot be overruled is a bottleneck with extra steps.

Follow-up

• Where would you put the test that fails if someone reintroduces the swallowed zero rowcount?
• The author says clients cannot handle a 409. How do you check whether that is true?

Reverse your own decision and price the reversal

Medium · reversibility · measurement · migrations

Describe a technical decision you made and later reversed. Pick one that cost something: a service you split and merged back, a cache you added and removed, an index you created that pushed the planner onto a worse plan, a projection you rebuilt from scratch. State what you believed when you decided, the measurement that changed your mind, how long the wrong version ran in production, and what the reversal cost in migrations, dual writes, and a deprecation window for callers you did not own.

Approach

1. State the original rationale without irony, in the version you would still defend given what was known then. If it is not defensible, the story is about carelessness rather than judgement, and a different example serves you better.
2. Give the measurement that moved with a before and after: the p99 that did not improve, the cache hit rate that sat at 40%, the plan that flipped to a sequential scan once the table passed a size you can name.
3. Cost the reversal in steps, not adjectives: expand-and-contract deploys, the dual-write window, the callers who had to be notified, the rows already written in the wrong shape that had to be backfilled or abandoned.
4. Distinguish reversal from rewrite by naming what you kept. Most good reversals preserve the schema or the interface and undo one decision inside it, which is also why they were affordable.

Follow-up

• What in that decision was irreversible, and did you know it was irreversible when you made it?
• How did you tell the people who had already built on top of the original decision?

Unblock an engineer without taking the keyboard

Easy · mentoring · leases · at-least-once

A teammate has spent two days on a job handler that occasionally writes duplicate rows. They are certain the queue is delivering twice by mistake. You suspect a lease expiring under a slow handler, so the job is running concurrently with itself. Describe how you have unblocked someone in this position: what you asked before offering a hypothesis, what you showed them rather than told them, and what you left them owning. Then say what you would do if their theory turned out to be the right one.

Approach

1. Ask before diagnosing, and ask for things answerable from data they already have: the attempt count on the job rows that produced duplicates, the handler's observed duration against its lease expiry, and whether the duplicate rows share a natural key that a unique constraint could have caught.
2. Teach the shape rather than the answer. A lease cannot distinguish a dead worker from a slow one, so a handler that outruns its lease is running twice by design, and deploys deliver the other half by killing handlers mid-run on every rollout. Both of their candidate theories produce identical duplicate rows, which is why the evidence has to come from timings rather than from argument.
3. Hand over a checklist they execute: a natural key on every write the handler performs so the second copy collides rather than appends, the record of intent written before any external effect, a lease heartbeat while running, and the metric that shows it working.
4. Keep ownership with them deliberately. Pair on the first write, then step back; if you finish it yourself you have closed one ticket and left the same person stuck on the next redelivery.

Follow-up

• How would you distinguish a genuine double-delivery from a lease expiry using only the data already stored?
• Their handler calls an external endpoint before recording that it did. What do you tell them to change first?