Carnegie Mellon University Software Engineer Interview Questions & Guide 2026

The 60-second brief

As a Software Engineer at Carnegie Mellon University, you play a vital role in building open-source data and computational infrastructure that powers transformative research and social impact projects. You will collaborate directly with interdisciplinary teams of computer scientists, statisticians, and social scientists across groups like the School of Computer Science, Heinz College of Public Policy, and the Robotics Institute. Your daily work directly supports AI and machine learning initiatives deployed in partnership with government agencies and nonprofits to address critical challenges in education, public health, criminal justice, and environmental sustainability.

Treat capacity estimation as a conversion skill rather than a table to memorise: turn a user count and an action rate into requests per second and bytes per day, then name the component that number breaks first. The figure only matters if it changes the design.

Focus areas

• Scope every query, cache and job by tenant
• Validate a signed launch without trusting the client
• Sandbox untrusted learner code with hard limits

Overview

The impact of this position extends far beyond typical enterprise software development. You will architect and scale data-intensive Python systems, design robust ETL pipelines, and manage cloud environments that enable researchers and policymakers to turn complex data into actionable public good. Whether you are building computational infrastructure for teaching or developing tools for large-scale public policy modeling, your contributions directly shape how technology serves society.

You can expect an intellectually stimulating, mission-driven environment where technical rigor meets public service. The work demands a balance of robust software engineering fundamentals and a genuine passion for collaborative, interdisciplinary problem-solving. Success in this role requires not just technical excellence, but also the ability to communicate effectively with academic researchers, domain experts, and external stakeholders who rely on your systems to drive real-world change.

The Carnegie Mellon University interview process

Carnegie Mellon University candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Phone Screening

The person on this call usually cannot evaluate your code and does not need to. They write a short paragraph, and that paragraph is what a hiring manager skims when deciding who to put on your loop. So the test is not whether your work was hard, it is whether a non-engineer can repeat it correctly. Name systems by what they did rather than by their internal codename, give each project a shape (what was breaking, what you changed, what happened after), and keep the whole walkthrough near ninety seconds. Depth that cannot survive a paraphrase reads as vagueness.

What to demonstrate

• Whether a non-engineer can restate your projects without distorting them, since their paraphrase is what travels to the hiring manager, not your sentences
• Whether each project has a shape rather than a stack list: the failure or constraint, the change you made, the result and how it was measured
• Whether you can say what was yours inside a team project without either inflating it or disappearing into the plural

How to prepare

• Rewrite each headline project as two sentences with no internal system names and no acronyms outside your company, then say them to someone outside engineering and have them repeat them back. Fix whatever came back wrong
• Attach one measured number to each project: the baseline, the change, and the window it was measured over. Where nothing was ever measured, say that plainly rather than reaching for a plausible percentage
• Time the background walkthrough against a clock. If it runs past two minutes, compress the earliest role to a single clause and spend the recovered time on the most recent one

Technical Interviews

The same problem is scored by two different mechanisms depending on the format, and preparing for one does not cover the other. With a person watching, partial progress is visible and a hint is a correction you can absorb; silence is the expensive failure, because nobody can read a half-written function. With an automated grader there is no partial credit for what you were about to do, nobody to ask, and the worked examples in the prompt are the entire specification. Read them as a contract, down to whether an empty result should be an empty list or no output at all.

What to demonstrate

• In a live session, whether your commentary tracks what your hands are doing, and whether a hint redirects you or gets defended against
• In an automated one, whether you cover the cases the examples do not show, since the hidden cases are where the score moves
• Whether you manage the clock on purpose: abandoning an approach that is not converging while there is still time to write something simpler that finishes

How to prepare

• Have someone hand you a problem and feed you one deliberately wrong hint. Practise testing it against a concrete case instead of accepting or rejecting it on authority.
• Do one timed run a week in a plain browser editor with autocomplete, linting and your own snippets switched off, which is closer to what these environments give you
• For the automated format, write the harness before the solution: a main that feeds the worked examples plus an empty and a single-element case and prints expected against actual, so a wrong submission is caught by you first

Behavioral Interviews

Many of these questions are about something that went wrong, and the grading sits mostly in the hours after you knew. Who found out first, whether that was you or an alert or a user, how long it took you to say it out loud, and whether the people who needed the news got it while they could still act on it. Engineers under-tell this part because it feels like confessing. The pattern it is looking for is the opposite: the quiet fix, an incident absorbed without telling anyone, after which nothing changed and the same failure is still available.

What to demonstrate

• How the problem was found, and whether that route was one you had built or one that happened to you, since a user reporting it first means your instrumentation did not cover that failure
• Whether time-to-detect and time-to-tell are separate numbers in your account and whether you know both, because a fast fix that nobody heard about until the retro is a different answer from a slow one that was announced immediately
• Whether the resolution left something durable behind, a check that fires or a default that changed, rather than depending on people remembering to be careful
• Whether you can say what the failure cost without either inflating it or waving it away

How to prepare

• Reconstruct one incident you were part of as a timeline with clock times: first bad request, first signal, first person who knew, first message outside the team, mitigation, permanent fix. The gaps between those entries are what gets asked about
• Look up the configuration of the signal that caught it, including its evaluation window and threshold. An alert defined on a five-minute aggregate cannot fire until the condition holds across that window, which puts a floor under time-to-detect that has nothing to do with how severe the failure was. Be able to say what that floor was and whether anyone had chosen it deliberately
• Prepare one story where you escalated early and the severity turned out to be smaller than you thought, including what it cost the people you pulled in. Without it, every answer you give about raising alarms is unfalsifiable

Team Meetings

When a round has no standard shape, it is often there because something is still open: an area no earlier conversation reached, a round where the signal came out mixed, or a decision someone is not ready to make alone. Work out which by going back over what each earlier round actually covered rather than how it felt, and arrive able to give evidence on that point without being asked twice. Weak answers replay the loop's earlier material at the same depth. Strong ones go a level deeper and stay consistent with what you already said.

What to demonstrate

• Whether your account of a project matches the one you gave earlier in the loop, since what you said before may be available to whoever runs this round
• Whether you can go a level deeper on something already covered, reaching the decision and its alternatives rather than repeating the summary
• Whether you state your own uncertainty accurately, including parts of a system you did not build and decisions you inherited, instead of claiming even ownership across all of it
• Whether you can answer a question you handled poorly earlier by naming what you missed, rather than delivering a polished second version as if the first had not happened

How to prepare

• Reconstruct the loop on one page: for each round, the questions you were asked and the answer you actually gave, not the better one you thought of afterwards. The gaps on that page are your best available guess at why this round exists.
• Take the two claims you made earlier that carry the most weight and assemble the backing for each: the measurement, the date, what broke, the decision you would make differently now.
• Write down the three facts about your work that must not drift between tellings, such as team size, timeline and your own role, and check your stories against that list rather than trusting recall under pressure

Where candidates lose points

Editing a content item in place and thereby rewriting the meaning of every historical score.

Authors expect to fix a typo, change a distractor, or adjust a point value, and nothing warns them that thousands of stored results reference the record they are editing. If attempts and analytics join to the mutable item rather than to a version, a regrade or a rebuilt report scores past answers against an item that did not exist when they were given. Versioning has to be the default write path, because a convention that authors must remember will not hold.

Treating the launch subject identifier as a global user id, or merging identities on email.

The subject claim is stable only within its issuer, so the same human arriving from a second platform is a different subject and must be a second external identity bound to the same person. Email is worse than useless as a merge key here: privacy settings frequently suppress it from the launch entirely, districts recycle addresses between graduating and incoming students, and younger learners often have none. A merge on email eventually joins two children's work into one account, which is both a grade bug and a privacy incident.

Trusting input because it came from your own front end

Anything crossing a trust boundary is hostile: parameterise queries instead of building SQL by concatenation, validate against an allow-list rather than a deny-list, and bound the size of anything you allocate from a request. Raising this unprompted in an API or design question is a cheap and unusually strong signal.

A cache with no invalidation story

Say how an entry goes stale, how long you can serve it stale, and what happens when many requests miss the same key at the same instant. One popular key expiring under load sends every concurrent request to the origin together; single-flight coalescing, jittered expiry, or serving stale while revalidating are the standard answers.

Questions & practice

Walk us through a technical challenge you solved in a previous project using code samples.

Medium · data structures and algorithms

Approach

1. Name the brute-force solution and its complexity before improving on it.
2. Choose the data structure from the access pattern, not from familiarity.
3. Walk one small example through your approach before writing the whole thing.

Follow-up

• How does this change if the input no longer fits in memory?
• What is the worst case, and how likely is it on real data?

Given a string s containing just characters (, ), {, }, [ and ], determine if the input string is valid.

Medium · data structures and algorithms

Approach

1. Restate the input: its shape, its size, and what is guaranteed about it.
2. Walk one small example through your approach before writing the whole thing.
3. Name the brute-force solution and its complexity before improving on it.

Follow-up

• How does this change if the input no longer fits in memory?
• Which test case would catch an off-by-one here?

In a graph, find a path from one specific node to another node.

Medium · data structures and algorithms

Approach

1. Name the brute-force solution and its complexity before improving on it.
2. Restate the input: its shape, its size, and what is guaranteed about it.
3. State the target complexity and say which constraint rules the naive version out.

Follow-up

• Which test case would catch an off-by-one here?
• How does this change if the input no longer fits in memory?

Diff a roster snapshot against live enrollments in one pass

Easy · hash join · diffing · soft delete

A tenant's bulk roster snapshot has been normalised into N incoming enrollment records carrying section_id, person_id, role (learner, instructor, aide, observer), source, begin_date and end_date. You also hold M stored enrollment rows for that tenant where deleted_at IS NULL. N and M are each up to 1,000,000. In one pass over the incoming records, classify each as an add, an update or a no-op, and produce the set of stored rows the snapshot never asserts, which becomes deletes_proposed on roster_sync_run. State your time and space bounds in bytes.

Approach

1. Fix the key first, because the schema already decided it: the partial unique index is (tenant_id, section_id, person_id, role) WHERE deleted_at IS NULL, so role is part of the identity. One human legitimately holds two live rows in a section (instructor and observer), and keying on (section_id, person_id) alone collapses them into one, emitting a spurious update and a spurious delete on every run.
2. Load the M stored rows into an open-addressed hash map from that 3-tuple to (enrollment_id, comparable_fields, seen_flag). Packed, the key is 24 bytes and the payload about 8, so at a 0.7 load factor that is roughly 45-50 MB for a million rows; boxed objects in a JVM or CPython map cost 5-10x that, which is the number that decides whether this fits in a worker.
3. Stream the incoming records and probe once each: miss is an add, hit with equal comparable fields is a no-op, hit with any difference is an update. Set the seen flag on hit. After the stream, the stored entries still unseen are deletes_proposed; count them and divide by M to get the fraction the gate is evaluated on.
4. Handle revival explicitly: an incoming record that misses the live map may still match a soft-deleted row. Look that up and clear deleted_at in place rather than inserting a new enrollment_id, because the partial index constrains only live rows and will not catch the duplicate — the history simply fragments into a new id at every term boundary.
5. Complexity is O(N + M) expected time and O(M) space. State the alternative and its crossover: sorting both sides and merging is O(N log N + M log M) with O(1) working memory beyond the sort, and it wins as soon as M stops fitting in RAM, since an external sort degrades gracefully where a hash map does not.
6. Guard the semantics before any of this runs. Absence means deletion only for a bulk feed; on a delta feed absence is silence and only an explicit tobedeleted status removes anything. A tenant that switches feed modes mid-term will otherwise have one delta file read as a snapshot, which proposes deleting almost everything.

Worked solution (25 min)

1. Write the key tuple down before any code and check it against the partial unique index in the schema, not against intuition about what a class roster is.
2. Build the stored map with a seen bit per entry, then run the incoming stream through a single probe per record, appending to three lists: adds, updates, no-ops.
3. Sweep the map once at the end for unseen entries to get deletes_proposed, and compute the fraction against the live count captured before the run started.
4. Feed a fixture with a person holding learner and observer rows in one section, a row whose end_date alone changed, a previously soft-deleted row that the snapshot asserts again, and one genuinely absent row.
5. Re-run the same fixture end to end and confirm every count except rows_in is zero the second time.

Expected result: One update for the changed end_date, one revival that keeps the original enrollment_id, one proposed delete, and no rows at all for the dual-role person; the second run reports zero adds, zero updates and zero proposed deletes.

Verify it yourself

• adds + updates + no_ops equals N exactly, with no record falling through unclassified.
• After the apply, a live-row count query returns M + adds - deletes_applied.
• Shuffling the incoming file changes no counts, since a hash diff must be order-independent.
• Removing role from the key makes the dual-role fixture report one update and one delete, which is the regression test for this bug.

Follow-up

• The upstream system changes its external_sourced_id scheme, so every incoming row looks new and every stored row looks absent. How do you reconcile without creating duplicate persons?
• The apply aborts halfway through. What makes the next run finish rather than double-apply, and what does snapshot_digest do for you?
• M grows to 50 million for one tenant. Which of your two algorithms do you run, and what changes about the delete detection?

Decide whether the gradebook grid is derived or materialised

Hard · denormalisation · read patterns · cache invalidation

An instructor gradebook renders 35 learners by 60 published assignments in one request against a 300 ms p95 budget. Truth lives in attempt(tenant_id, assignment_id, person_id, attempt_no, state, is_late) and score(attempt_id, tenant_id, score_given, revision), and the policy is best attempt wins. Separately, a district report crosses 90,000 learners and 40 million attempts. Decide with arithmetic whether the grid is derived per request or materialised into gradebook_cell, and if materialised give its key, its staleness detector, and every event that invalidates a cell.

Approach

1. Do the arithmetic before choosing anything. The grid is 2,100 cells. With an index on attempt (tenant_id, assignment_id, person_id, attempt_no) and one score lookup per selected attempt, that is a few thousand index tuples and a few thousand buffer accesses against a warm cache — single-digit milliseconds, an order of magnitude inside a 300 ms budget. Deriving it wins, and the thing you avoid buying is an invalidation set you would own forever.
2. Identify the read that actually breaks, because it is not this one. The district report crosses 40 million attempts: that is scan-shaped, has no per-request deadline, and belongs in a periodic rollup with its own store and its own freshness SLA. Conflating the two is how a live-path cache gets built to fix an analytics problem.
3. Name the condition that would change the answer, so the decision is falsifiable rather than a preference: a policy of average-across-attempts with per-item weighting, a grid that grows to 200 assignments across several terms, or a term-end hour in which 10^3 sections open at once. Each multiplies the per-request work rather than the data, which is the signal that a projection is now cheaper than the recomputation.
4. If you do materialise, key gradebook_cell on (tenant_id, section_id, assignment_id, person_id) and store source_attempt_id and source_score_revision on the row. Those two columns are what turn 'is this stale' from an opinion into a comparison, and they let a repair job find divergence by joining rather than by rebuilding everything.
5. Enumerate invalidation exhaustively, since the omissions are where this design fails: a newly graded attempt, a regrade that bumps score.revision, an attempt voided after an integrity review, a change to the assignment's policy or point total, an enrollment added or soft-deleted (a withdrawn learner's cell must stop rendering without being deleted, because the score still exists), and a content version repin. Each is an event the projection consumes; a cell written inline by the request path is how a projection quietly becomes the system of record.
6. Write the consistency contract down: the projection is derived, rebuilt idempotently from (attempt_id, revision), never authoritative. When cell and score disagree, score wins and the cell is rebuilt, and a scheduled comparison over a sample proves it rather than assuming it. Carry tenant_id in the table and in every cache key — a projection keyed on assignment_id alone is exactly the shape that renders one district's grades inside another's page.

Follow-up

• An instructor edits one score by hand. How long until the grid shows it, and what does the instructor see in the meantime?
• As sections per tenant grow, what breaks first: the rebuild job, the invalidation fan-out, or the read you were trying to protect?
• Would you materialise the column totals and the class mean as well, and what does that do to the invalidation set?

Explain why the grading queue index stopped serving its ORDER BY

Medium · indexing · partial index · planner statistics

attempt holds 2 billion rows and 99.7% are state='graded'. The grading queue runs SELECT attempt_id FROM attempt WHERE tenant_id = $1 AND state IN ('submitted','grading') ORDER BY submitted_at LIMIT 100 against index ix_attempt_queue (tenant_id, state, submitted_at). At the bell the query takes 30 seconds and EXPLAIN shows a Sort above a bitmap heap scan, even though the queue is only a few thousand rows. Explain why that index cannot serve the ordering, give the replacement definition, and say how the replacement behaves as rows leave the queue.

Approach

1. Start from what the index actually orders. Its tuples are sorted by (tenant_id, state, submitted_at), so with two state values the scan yields all 'grading' rows in submitted_at order, then all 'submitted' rows in submitted_at order. The query wants one submitted_at order across both, which that physical order does not provide, so the planner must sort the whole matching set and the LIMIT cannot stop the scan early. With a single state constant the same index satisfies the ordering perfectly — the plan changed because of the IN list, not because of the data.
2. Replace it with a partial index that moves the predicate out of the key: CREATE INDEX CONCURRENTLY ix_attempt_queue ON attempt (tenant_id, submitted_at) WHERE state IN ('submitted','grading'). Ordering is now satisfied directly, the LIMIT stops after 100 index tuples, and the index covers roughly 0.3% of rows so it stays resident in cache instead of competing with the table.
3. State the precondition, because it is the part people skip: the planner uses a partial index only when it can prove the query's predicate implies the index predicate. A literal IN list matching the definition proves; state = ANY($1) with the list bound as a parameter does not, so the queue's state set has to be literal in the SQL, not passed in.
4. Fix the estimate separately from the access path. With five states the planner already has each one in its most-common-values list, so single-column selectivity is fine; what it gets wrong is the dependence between tenant_id and state, because a bell means one tenant owns nearly the whole queue while the planner multiplies the two selectivities as if independent. CREATE STATISTICS ON tenant_id, state FROM attempt corrects it, and you ANALYZE explicitly after a bulk transition rather than waiting for autovacuum, whose analyze threshold of 50 + 0.1 x reltuples needs 200 million changed rows on this table.
5. Account for the lifecycle. A row leaves the queue by UPDATE state='graded', so its new version is not in the partial index at all while the old index entry remains until vacuum reclaims it. The index therefore churns in proportion to throughput rather than to depth: size it and vacuum it against how many attempts pass through per day, not against how many sit in the queue at once.
6. Keep the alternative in your pocket for when the partial index is not acceptable: two explicit scans merged, (SELECT ... WHERE state='submitted' ORDER BY submitted_at LIMIT 100) UNION ALL (SELECT ... WHERE state='grading' ORDER BY submitted_at LIMIT 100) ORDER BY submitted_at LIMIT 100, which keeps the existing index and pushes the LIMIT into both branches. Confirm whichever you choose with EXPLAIN on your major version rather than from memory, since array-key handling in b-tree scans has changed between releases.

Worked solution (25 min)

1. Reproduce on a scaled copy: 20 million attempts, 0.3% in the queue, and confirm EXPLAIN (ANALYZE, BUFFERS) shows a Sort node above the scan.
2. Re-run the identical query with state = 'submitted' alone and observe the Sort disappear, which isolates the IN list as the cause.
3. Build the partial index CONCURRENTLY, re-run, and compare buffers touched and actual rows.
4. Change the literal list to a bound parameter and confirm the partial index is no longer chosen.

Expected result: An Index Scan using the partial index with no Sort node, actual rows 100, buffers in the tens rather than the hundreds of thousands, and runtime in single-digit milliseconds at the same queue depth that previously took 30 seconds.

Verify it yourself

• EXPLAIN shows no Sort and no Bitmap Heap Scan for the queue query.
• pg_relation_size of the partial index is roughly 0.3% of the equivalent full index, and grows with attempts processed rather than with queue depth.
• Rewriting the predicate as a bound parameter makes the plan regress, proving the implication requirement rather than assuming it.
• After draining the queue, the partial index has dead entries until vacuum runs, visible in pg_stat_user_indexes and index size.

Follow-up

• The queue drains and refills eight times a day. What is the vacuum strategy for this index, and what do you monitor to know it is losing?
• A second consumer claims rows with FOR UPDATE SKIP LOCKED. Does the plan survive, and what does skipping do to the ordering guarantee?
• Support wants oldest-first fairness across tenants rather than within one. Does the same index serve that query?

What is the difference between a binary tree and a red-black tree?

Medium

Approach

1. Work from the requirement backwards to the design.
2. Say what you would check first and why it is the highest-information step.
3. State your assumptions explicitly before working the problem.

Follow-up

• What assumption would you test first?
• How would you know your answer was wrong?

How do you approach building, testing, deploying, and maintaining production-grade Python software?

Medium

Approach

1. Say what you would check first and why it is the highest-information step.
2. State your assumptions explicitly before working the problem.
3. Work from the requirement backwards to the design.

Follow-up

• What assumption would you test first?
• How would you know your answer was wrong?

Publish scores outward at-least-once without losing a regrade

Medium · at-least-once · idempotency · ordering · retries

score_publication holds publication_id, attempt_id, target, idempotency_key, score_given, score_maximum, score_timestamp, state, attempt_count, next_attempt_at and last_error_code. The caller is your relay: it reads that table and calls an external gradebook endpoint you do not control, which rate-limits, sometimes times out after committing the write, and may or may not reject a non-increasing score_timestamp. On failure the relay reschedules the row and moves on. Design the claim, publish and mark loop; say how idempotency_key and score_timestamp are derived, what happens when the process dies after the call but before the mark, and which responses are terminal.

Approach

1. Write the score row and its publication rows in one transaction. If the publication can be lost while the score commits, no retry logic recovers it; if it commits without the score, you publish a grade that does not exist.
2. Claim with a lease rather than a read, and let next_attempt_at carry both meanings: for a pending row it is when the row becomes eligible, for an in_flight row it is when the lease expires. In Postgres the claim is an UPDATE whose target rows come from a sub-select: SELECT publication_id FROM score_publication WHERE state IN ('pending','failed_retryable','in_flight') AND next_attempt_at <= now() ORDER BY next_attempt_at LIMIT :batch FOR UPDATE SKIP LOCKED, and the outer UPDATE sets state='in_flight', attempt_count = attempt_count + 1 and next_attempt_at = now() + lease, RETURNING the rows. SKIP LOCKED lets several workers share the table without convoying on the same head rows.
3. Note what the in_flight branch is for, because leaving it out is the common bug: a worker that dies after claiming leaves a row in_flight forever, since nothing but a terminal transition would ever move it again, and the lease you wrote into next_attempt_at expires with no one reading it. A separate reaper that sweeps expired in_flight rows back to failed_retryable is the same recovery with a second moving part and a second failure mode; prefer the single predicate unless you want re-claims counted apart from retries.
4. Size the lease against the slowest real call, not the median: it must exceed the receiver's client timeout plus the mark, or you re-claim rows whose worker is still mid-call and send a rate-limited endpoint two copies of work it is already doing. The repeat is harmless to correctness because the key is deterministic, but it spends quota you do not have. On the terminal transitions set next_attempt_at = NULL alongside state='succeeded' or 'dead', which keeps them out of the claim predicate and keeps a partial index on (next_attempt_at) WHERE state IN ('pending','failed_retryable','in_flight') proportional to the queue rather than to the table.
5. Accept that a crash between the call and the mark repeats the call once the lease expires. That is the entire reason idempotency_key must be a deterministic function of attempt_id and the score revision, stored on the row: a key minted per send attempt turns every recovery into a second grade at the receiver.
6. Derive score_timestamp from a per-(assignment, person) revision counter materialised once at insert, never from now() at send time. Wall-clock derivation fails twice: two regrades inside one tick tie, and a stale retry sent after a regrade carries a newer clock than the value it would overwrite, so the wrong grade wins.
7. Classify responses rather than status codes. 2xx succeeds; 429 and 5xx retry with bounded exponential backoff plus full jitter under a per-tenant concurrency cap so one slow tenant cannot own the worker pool; a timeout is an unknown outcome and is retryable precisely because the key makes a repeat harmless; 401 retries only after a token refresh and only a bounded number of times; 404 on a deleted line item and 422 on an out-of-range score are terminal.
8. Make dead a visible state carrying last_error_code and the tenant, not a log line, and let the attempt_count increment in the claim be what bounds a crash loop: a row whose worker keeps dying is re-claimed, counted, and hits the ceiling like any other failure instead of cycling forever. The operator question is how many, whose, and since when, and a safe replay is a transition back to pending that reuses the same key.

Worked solution (35 min)

1. Write the transaction that inserts the score and its publication rows together, with idempotency_key computed from attempt_id and score revision.
2. Write the lease-based claim as an UPDATE over a FOR UPDATE SKIP LOCKED sub-select whose predicate is state IN ('pending','failed_retryable','in_flight') AND next_attempt_at <= now(); run two workers and assert no row is ever claimed by both.
3. Claim a row and abandon it in_flight. Assert it is not claimed again one second before its lease expires and is claimed one second after, so the lease is what excludes it rather than its state name.
4. Kill a worker between the HTTP call and the mark; assert the row is re-claimed once the lease expires, that attempt_count has incremented, and that the receiver's stored score is unchanged after the repeat.
5. Regrade twice and assert the two publications carry distinct keys and strictly increasing score_timestamps.
6. Deliver the older publication after the newer has succeeded and assert the receiver still holds the newer value.
7. Table every response class against the resulting state, next_attempt_at and attempt_count, and name the condition that reaches dead.

Expected result: A relay in which one score revision has one effect per target regardless of retries, a claim predicate that also picks up in_flight rows whose lease has expired so a dead worker's rows recover with no operator action, score_timestamp strictly increasing per learner and line item, timeouts retrying with the same key, and only 404/422-class rejections plus an attempt ceiling reaching a dead state that names the tenant and the error.

Verify it yourself

• Two workers over ten thousand rows produce exactly one effect per key at the receiver, even with a worker killed mid-run.
• A row abandoned in_flight is claimed again after its lease expires and not before, and its attempt_count grows on each claim, so a crash loop terminates in dead rather than running forever.
• A stale retry delivered after a regrade cannot change the receiver's value, verified by asserting the final value rather than asserting the send was skipped.
• Every dead row answers which tenant, which learner, which error code and how long it has been dead.

Follow-up

• The receiver accepts a score and the instructor then regrades twice within ten seconds. Walk both publications through your ordering token and say what value the receiver ends up holding.
• Put a number on the per-tenant concurrency cap and defend it against a tenant whose endpoint takes thirty seconds per call.
• A row has been dead for a week and the underlying score has changed since. What does a safe replay publish?

Attempts marked late an hour early after the clock change

Medium · timezone · dst · materialised columns · data repair

On the Monday after the autumn clock change, instructors in one zone report attempts submitted at 23:10 local flagged late against a 23:59 due time. attempt.is_late is true and frozen. assignment stores due_at_local, policy_time_zone ('America/Chicago'), and a materialised due_at_utc. The affected assignments were published in October and are due in November. Zones without a transition are unaffected. Produce an ordered diagnostic checklist, the root cause, the query that identifies every affected assignment, and the repair, including what learners and instructors see.

Approach

1. Order the checklist to separate a read-time bug from stored bad data, because the repairs are completely different. First, confirm is_late is frozen on the attempt rather than recomputed in the report, which the schema already tells you. Second, compare each affected assignment's stored due_at_utc against the value the wall-clock policy implies. Third, check whether the divergence is exactly one hour and confined to one zone. Fourth, check the publish timestamp relative to the transition date. That ordering reaches the answer without reading the submit path at all.
2. Name the defect precisely: due_at_utc was materialised at publish by applying the UTC offset in effect at publish time, not the offset in effect at the due instant. An assignment published in October under UTC-5 and due 23:59 on a November date under UTC-6 materialises as 04:59Z instead of 05:59Z, so everything submitted between 23:00 and 23:59 local is one hour past a deadline that is one hour early.
3. Write the detection as a comparison against the correct conversion rather than as a guess about which rows are bad: SELECT assignment_id FROM assignment WHERE due_at_local IS NOT NULL AND due_at_utc IS DISTINCT FROM (due_at_local AT TIME ZONE policy_time_zone). In Postgres that expression interprets a naive timestamp in the named zone using the rules in effect at that local instant, which is exactly the computation that was skipped.
4. Repair in two passes with different risk. Pass one recomputes due_at_utc for assignments still open, which changes only future decisions. Pass two revisits frozen is_late only for attempts whose assignment's due_at_utc moved and whose submitted_at falls inside the moved window, and emits a grade-change event for each so instructors see a reason rather than a number that changed on its own. Unfreezing every attempt would recompute history that was decided correctly.
5. Close the hole permanently rather than fixing the arithmetic once. Keep both the wall-clock policy and the materialised instant, treat the materialised column as a cache with an invariant, and run a nightly assertion that due_at_utc equals the AT TIME ZONE expression for every open assignment. That same assertion catches the other source of drift: a tz database update that changes a zone's rules after you materialised.
6. State the two edge cases the policy must handle before they become tickets. A due time inside a spring-forward gap has no instant, and a due time inside a fall-back repeated hour has two; resolution differs across databases and libraries, so reject or normalise such policy times at publish rather than depending on whichever rule the stack happens to implement.

Follow-up

• A section is moved from a school in one zone to a school in another mid-term. Which assignments recompute, and which must not?
• A government changes its zone's transition dates and the tz database ships an update. What does your nightly assertion report the next morning, and what is the approval path for the recompute?
• How would you present a corrected is_late to a learner who already saw a late penalty?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

• Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
• Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
• Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

• Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
• Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
• Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

• Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
• Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
• Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

• For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
• Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
• Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
• Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

• Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
• Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
• Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
• Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

• Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
• Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
• Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

• Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
• Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
• Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

Know where your numbers came from

Bring the two or three numbers the story rests on and know how they were collected. A p99 whose timer starts inside your handler excludes the time a request spent queued, so it can sit flat while users wait longer. Give the window, the percentile and what the measurement left out, or drop the number.

What experiences prepared you to be an ideal software or learning engineer for our team?

Medium · behavioural and engineering judgement

Approach

1. State the situation in two sentences and spend the rest on the reasoning.
2. Pick a story where you made the decision, not one where you watched it.
3. Name the disagreement and how you resolved it with evidence.

Follow-up

• How did you know your change caused the improvement?
• What would you do differently if you ran that again?

Why did you want to work at Carnegie Mellon University as opposed to an industry job?

Medium · behavioural and engineering judgement

Approach

1. Give the blast radius: what could have broken, and what you measured.
2. Close with what you would do differently, concretely.
3. Pick a story where you made the decision, not one where you watched it.

Follow-up

• How did you know your change caused the improvement?
• What would you do differently if you ran that again?

Can you explain your experience with JavaScript and modern frontend frameworks?

Medium · behavioural and engineering judgement

Approach

1. Give the blast radius: what could have broken, and what you measured.
2. Name the disagreement and how you resolved it with evidence.
3. Close with what you would do differently, concretely.

Follow-up

• What would you do differently if you ran that again?
• What did you decide not to do, and why?