Interview guides · Altron Automation

Altron Automation Software Engineer Interview Questions & Guide 2026

System design practice for Altron Automation software engineer candidates: reach a data model and an API contract, not a box diagram.

The 60-second brief

At Altron Automation, the Software Engineer role is central to the design and deployment of sophisticated industrial systems. You will be tasked with bridging the gap between high-level mechanical design and the functional programming that brings custom machinery to life. Your work directly impacts the efficiency and reliability of the automation solutions that the company delivers to its clients.

For a design discussion, a catalogue of architectures is worth less than the ability to turn a vague requirement into a data model and an API contract. A box diagram with no schema under it collapses at the first follow-up question.

Focus areas

Choose indexes from the query's access path
Detect concurrent edits instead of losing writes
Trace a symptom to a mechanism under load

Overview

This position requires a blend of technical precision and practical problem-solving. Because Altron Automation specializes in building bespoke equipment, you will often find yourself collaborating closely with cross-functional teams, including mechanical engineers and project managers. The role is critical because the software you write acts as the intelligence layer for the hardware, directly influencing the performance metrics and budget adherence of major projects.

The Altron Automation interview process

Altron Automation candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screening

Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.

What to demonstrate

Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not

How to prepare

Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window

Technical Discussions

Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.

What to demonstrate

Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply

How to prepare

For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last

In-Person Visit

You cannot drill a format you do not know, so put the preparation into material that travels. Three pieces of your own work, each rehearsed until you can take a follow-up you did not anticipate, will carry a conversation or a code walkthrough equally well. Specificity is what separates that from filler. A number needs its definition before it means anything: a p99 is over some window and measured at some hop, and a server-side figure excludes the queueing and network time a client would see. The number you cannot qualify is the one to leave out.

What to demonstrate

Whether your examples carry detail only someone who did the work would hold, such as what the binding constraint actually was, which alternative you rejected and why it was worse, and what you measured on each side of the change
Whether a number survives one follow-up, meaning you can say what it was measured over and whether it moved because of your change or merely alongside it
Whether a failure is described with the specific change that followed it, rather than a lesson stated in general terms
Whether your part in a team effort is stated accurately, including what other people did

How to prepare

Write a page on each of three projects covering the constraint, the option you rejected, the measurement before and after, and what went wrong. Cut any line you cannot take a follow-up on, since you are writing the parts you will be pressed on rather than a summary.
Recover the real figures while you still have access: request volume, data size, latency with its percentile and window, team size, timeline. Note where each came from, whether a dashboard, a design document or memory, and mark the estimates so you can say which they are out loud.
Take your weakest project story to someone who works in a different area and have them ask why four times in succession. The point where you run out of answer is the part to go and re-read before the round.

Where candidates lose points

Shipping a migration and the code that depends on it as a single change

During any rolling deploy, and for as long as a rollback remains possible, old and new code execute against the same schema at the same time. A migration that drops or renames a column breaks every instance that has not restarted yet, and code that requires a column the migration has not applied breaks every instance that restarted early. The discipline is expand then contract: add the new column nullable, write both shapes, backfill in batches, move reads across once the backfill is verified, and only then stop writing the old shape and drop it - four deploys, usually spread over days. It feels disproportionate until the first rollback, at which point it is the only reason the previous version still runs.

Letting a slow dependency consume unbounded concurrency

The failure that takes a service down is usually not an error but a delay. A dependency answering in thirty seconds instead of fifty milliseconds holds each request's worker or connection six hundred times longer, and since required concurrency is arrival rate times latency, a fleet sized for sixty in-flight requests now needs thirty-six thousand to sustain the same rate - so it queues, and requests whose clients have already abandoned them still occupy resources. Retries make it precisely worse: a policy of three attempts triples the load on a dependency at the exact moment it is least able to serve, which is how one slow dependency becomes an outage of everything sharing that pool. Containment is four specific things - a timeout on every outbound call shorter than the caller's remaining budget, a bounded pool per dependency so one cannot starve the others, backoff with full jitter rather than a fixed delay so retries do not resynchronise, and a circuit that stops sending once the failure rate makes an attempt pointless.

Comparing floating-point values for equality, or holding money in them

Binary floating point cannot represent 0.1 exactly, so repeated addition drifts and an equality check fails on values that are mathematically equal. Store currency as integer minor units or a decimal type, and compare floats against a tolerance you chose for a stated reason.

Retrying a write that is not safe to repeat

A timeout tells you nothing about whether the server applied the write, so a blind retry of a create or a charge can duplicate it. Either make the operation idempotent, with a caller-supplied key the server deduplicates on or a conditional update, or do not retry it; and use exponential backoff with jitter so the retries of many clients do not synchronise into a second outage.

Questions & practice

Canonicalise a request body into a stable idempotency fingerprint

Medium · parsing · canonicalisation · hashing

idempotency_key.request_fingerprint is a SHA-256 over the method, path and canonicalised body, and a retry whose fingerprint differs must be rejected with 422 rather than served the stored response. Write the canonicaliser. Bodies are JSON up to 256 KB nested at most 32 levels; clients vary key order, whitespace and unicode escaping, and some send 64-bit ids as JSON numbers. Produce a deterministic byte string such that semantically identical bodies match and any semantic difference does not. State your complexity and name two normalisations you refuse to perform.

Approach

Parse once into a tree, then re-serialise under fixed rules: object keys sorted, array order preserved, one escaping convention, no insignificant whitespace. Parsing is O(n) and sorting keys is O(k log k) per object, so O(n log n) overall with O(depth) stack, and the 32-level cap is enforced during parsing because hostile nesting is how a canonicaliser becomes a stack overflow.
Sort keys by their UTF-8 bytes and say why the obvious implementation is wrong in some runtimes: a default string comparison that orders by UTF-16 code units places surrogate pairs, meaning code points from U+10000 up, below U+E000 to U+FFFF, which is not UTF-8 byte order, so two services written in different languages disagree on the same document.
Do not re-encode numbers through a double. IEEE-754 binary64 represents integers exactly only up to 2^53, so normalising a 19-digit id through a float changes it, and 1 against 1.0 cannot be reconciled without deciding whether they are the same value. Preserve the literal token, and require ids as strings at the API boundary if you want them comparable.
Reject duplicate keys rather than picking one. JSON permits them and parsers disagree, most keeping the last, so any choice you make ties the fingerprint to a parser detail that the code handling the request does not necessarily share.
Frame the hash preimage so concatenation cannot collide: delimit or length-prefix the method, path and body, otherwise one request's fields can be rearranged into another request with the same byte stream and the same fingerprint.
Name the refusals and their consequence: no case folding, no dropping of null-valued keys, no Unicode normalisation. Each makes two different requests fingerprint alike, and the resulting failure is the worst one this table has, since the second request is answered with the first one's stored response and its effect never happens.

Follow-up

A client sends the same logical request with an extra field your API ignores. Same key, different fingerprint, so you return 422. Is that the right answer?
Where does the fingerprint get computed relative to request decompression and the body-size limit?
The endpoint takes 1,000 requests per second with 256 KB bodies. What does hashing cost, and does it belong at the edge or in the core service?

Diff a projection against the primary without per-row point reads

Hard · reconciliation · range hashing · throttling

The listing projection has drifted and some rows show a stale version. The primary holds 40,000,000 resource rows across 12,000 tenants while serving 1,200 writes and 14,000 reads per second. The obvious repair, reading each resource row and comparing its version against the projection, is correct and would eventually finish. Explain precisely why it is unacceptable here, then give a diff that finds the differing rows, state its complexity, and make it safe to run against a live primary. Replication lag is usually under 100 ms and is not bounded.

Approach

Quantify the naive cost rather than calling it slow: 40,000,000 point reads at even 0.5 ms each is over five hours serialised, and the only lever is concurrency, which is exactly what you cannot spend. The primary's pool is sized for the write path, and 40,000,000 random reads evict the buffer cache that sustains the 85 percent cache hit rate, so the audit degrades the system it is auditing.
Replace random access with one ordered pass per side. Both sides can be read in (tenant_id, resource_id) order, which is a sequential scan on each and a merge join in O(n) time and O(1) memory. For a dense diff that is the whole answer, and it reads the primary once instead of 40,000,000 times.
For the expected sparse case, compare range hashes instead of rows: partition the key space, compute per range an order-independent aggregate over hash(resource_id, version), compare aggregates, and descend only into ranges that differ. With d differing rows and branching factor B, at most d ranges mismatch per level, so the drill-down examines O(d log_B(n/d)) ranges and reads full rows only in mismatching leaves.
Aggregate with a sum modulo 2^64 or a multiset hash, never XOR. XOR is order-independent but self-cancelling, so two rows wrong in the same way, or a row duplicated on one side, leave the range aggregate matching and the range is declared clean.
Pin the comparison to a point in time or it reports lag as drift: consider only rows whose updated_at is older than now minus a lag margin, and re-check each candidate mismatch individually before repairing. At 1,200 writes per second a diff without this reports thousands of false positives, and an unattended repairer would then overwrite live rows with stale values.
Make the run resumable and throttled: batch by range key, persist the last completed range, and watch a signal such as replica lag or primary CPU, pausing rather than pressing on. A reconciliation that cannot be stopped and resumed gets killed halfway and restarted from zero, which is how a repair becomes an incident.

Follow-up

The diff reports 900 stale rows. How do you decide between patching those rows and rebuilding the projection from resource_revision?
Same job, but the projection lives in a search index that cannot be scanned in key order. What changes?
How would you run this continuously at low cost instead of only as incident response?

Archive a resource graph without breaking live references or recursing

Medium · graph traversal · topological order · tenant isolation

Resources reference other resources within a tenant; for the largest tenant the reference table holds up to 2,000,000 nodes and 8,000,000 edges. Archiving a resource must archive everything reachable from it that nothing outside the set still references, refuse when a live external referrer exists, and terminate when references form cycles, which they legitimately do. Produce the archive order and the refusal list, targeting O(V+E). Say what stops the traversal crossing a tenant boundary, and why recursion is the wrong control structure at this size.

Approach

Load the subgraph with the tenant predicate on both endpoints of the edge, not only on the side you started from. Scoping the left table alone is the classic cross-tenant leak: one mis-entered edge then pulls another tenant's resources into the traversal and, worse, into the archive.
Traverse iteratively with an explicit stack. A 2,000,000-node graph can hold a chain deep enough to exhaust a native stack in the low tens of thousands of frames, and that failure is a process crash rather than an error you can return.
Treat cycles as data rather than corruption: compute strongly connected components with Tarjan in O(V+E) using its own explicit stack, then condense. The condensation is a DAG, so a topological order over it gives the archive order, and every member of a component archives in one transaction because no order within a cycle is valid.
Decide refusals with reverse edges. A candidate is archivable only if every in-edge originates inside the candidate set, so build the transpose or count in-degrees restricted to the visited set, and emit each blocked resource with the id of the external referrer, which is the only part of the answer an operator can act on.
Store the graph as CSR rather than a map of lists: an offsets array of V+1 8-byte entries plus E 8-byte targets is about 80 MB at this size, where boxed adjacency lists cost several times that and lose cache locality on every hop.
Run Kahn over the condensation for the order in O(V+E). If the emitted count is short of the component count the condensation step itself is wrong, since a condensation cannot contain a cycle, which makes the check free.

Worked solution (30 min)

Write the edge-loading query with the tenant predicate on both endpoints and state what it does with a cross-tenant edge.
Implement iterative Tarjan with an explicit stack and confirm on a three-node cycle that it emits one component of size three.
Build the transpose restricted to the visited set and mark every node with an in-edge from outside it as refused, carrying the referrer id.
Run Kahn over the condensation and verify the emitted order against the referrer-before-referenced rule.
Size the CSR arrays for 2,000,000 nodes and 8,000,000 edges and compare against a boxed adjacency map.

Expected result: An iterative O(V+E) traversal over a tenant-scoped CSR subgraph, SCC condensation so cycles archive atomically as one component, a transpose-based refusal list naming the external referrer for each blocked resource, and a Kahn topological order over the condensation, with recursion replaced by an explicit stack because of graph depth rather than style.

Verify it yourself

A two-node cycle with no external referrers archives as one unit, and adding a single external referrer moves both nodes to the refusal list.
No resource appears in the archive order before something that references it, except within its own component.
A counter on edge visits matches the loaded edge count exactly, confirming each edge is traversed once.

Follow-up

The graph is read in one query and the archive writes a minute later. What can change in between, and how do you make the write safe?
The candidate set is 400,000 resources. Is that one transaction, and if not, what does a half-finished archive look like to a reader?
An edge points at a resource in another tenant. Is that a refusal, an error, or an alert?

Denormalise tenant onto revisions and backfill it live

Hard · denormalisation · online migration · backfill

resource_revision (revision_id, resource_id, version, actor_user_id, change_kind, patch, request_id, created_at) has 400M rows and no tenant column; tenant_id lives only on resource. Two reads need it: a tenant-scoped audit feed ordered by created_at DESC, and an offboarding purge. Both join back to resource today. Justify adding tenant_id to resource_revision against those two reads, name the anomaly the copy introduces and the constraint that prevents it, then give the ordered migration for a live table taking 1.2k writes/second — the lock each step takes, how the backfill is batched, and where each step stops being reversible. PostgreSQL 16.

Approach

Justify from the access path rather than from taste. Without the column, the audit feed either scans resource_revision by created_at and discards other tenants' rows, or resolves the tenant's resource_ids first and probes with them — both proportional to the tenant's whole history rather than to one page. With (tenant_id, created_at DESC, revision_id DESC) it is a seek that stops at 50 rows, and the purge becomes a ranged delete instead of a join.
Name the cost exactly: a second copy of a fact can disagree with the first. Make the disagreement unwritable rather than documented — add UNIQUE (resource_id, tenant_id) on resource so it can serve as a foreign-key target, then FOREIGN KEY (resource_id, tenant_id) REFERENCES resource (resource_id, tenant_id) on the revision table. A revision can then only ever carry its parent's tenant.
Step one, expand: ALTER TABLE resource_revision ADD COLUMN tenant_id BIGINT NULL, with no default, so it is a catalogue change and no rewrite. It still needs ACCESS EXCLUSIVE for an instant, and that instant queues behind the longest open transaction on the table while every later query queues behind it — set lock_timeout to 2s and retry rather than wait.
Step two, dual-write: deploy the writer that populates tenant_id on every new revision while reads still use the join. Reversible by redeploying the previous build, because nothing reads the column yet.
Step three, backfill: batch by primary key rather than by created_at so the cursor is dense and resumable — UPDATE resource_revision rr SET tenant_id = r.tenant_id FROM resource r WHERE r.resource_id = rr.resource_id AND rr.revision_id > $1 AND rr.revision_id <= $1 + 5000 AND rr.tenant_id IS NULL — committing per batch and persisting the cursor. Throttle on replica replay lag and on dead-tuple count, since each batch writes 5,000 new row versions. Run the backfill before the index exists so those updates can stay HOT.
Step four, index then enforce then contract: CREATE INDEX CONCURRENTLY (cannot run inside a transaction block, scans the table twice, waits on open transactions, and leaves an INVALID index to drop concurrently if it fails); ADD CONSTRAINT ... CHECK (tenant_id IS NOT NULL) NOT VALID, then VALIDATE CONSTRAINT, which takes only SHARE UPDATE EXCLUSIVE, after which SET NOT NULL uses the validated check instead of re-scanning on PostgreSQL 12 and later. Only then move the audit reads onto the column and, in a later deploy, delete the join path.

Follow-up

The backfill is half finished and a rollback is required. What state is the table in, and what does the previous build do with a half-populated column?
How do you verify the backfill actually finished, given rows are still being inserted while it runs?
A resource must now be movable between tenants. What does that do to the composite foreign key and to the revisions already written?

Explain why the owner filter ignores the listing index

Medium · composite index · query plans · sort elimination

The only index on resource is (tenant_id, status, updated_at DESC, resource_id DESC). A new endpoint returns one user's resources across all statuses, newest created first: WHERE tenant_id = $1 AND owner_user_id = $2 ORDER BY created_at DESC LIMIT 20. On a tenant with 2M rows it takes 900 ms and EXPLAIN shows a sort above a large scan. Explain precisely why the existing index cannot serve it, give the index that can, and state which of these the new index still will not help: owner_user_id alone across tenants; the same query ordered by updated_at. PostgreSQL 16.

Approach

Separate the two jobs an index does. For filtering, a composite btree is seekable only on a left prefix, so with no predicate on status the scan can at best range over tenant_id and test owner_user_id per row; PostgreSQL 16 has no btree skip scan to jump the unconstrained column.
For ordering, the index is sorted by (status, updated_at) within a tenant and not by created_at, so the LIMIT cannot stop early: every matching row is read and then sorted. That is the 'Sort Method: top-N heapsort' line, and it is why the plan reads 2M rows to answer with 20.
Derive the replacement from the access path — equality, equality, then the ordering column: CREATE INDEX CONCURRENTLY ON resource (tenant_id, owner_user_id, created_at DESC). The scan seeks to the (tenant, owner) range and walks 20 entries in order, so the Sort node disappears along with the row-read.
Treat INCLUDE (title, status) as conditional, not free. An index-only scan still visits the heap for any row whose page is not marked all-visible, so on a table taking 1.2k writes/second the win depends on autovacuum keeping the visibility map current, and the wider index costs more on every insert.
Answer the two negatives explicitly. owner_user_id alone is not a left prefix of the new index, so it degrades to a full scan of the index at best. Ordered by updated_at, the query still seeks on the (tenant, owner) pair but must sort, because only created_at is ordered within that pair.
Measure both sides with EXPLAIN (ANALYZE, BUFFERS) and compare estimated against actual rows at the lowest node — a 2M-versus-200 misestimate there is usually what chose the plan, and adding an index will not fix a statistics problem.

Worked solution (25 min)

Load 2M resource rows across 5k owners in one tenant, run the query under EXPLAIN (ANALYZE, BUFFERS), and record the node reading the most rows plus the Sort Method line.
Create (tenant_id, owner_user_id, created_at DESC) concurrently and re-run, confirming the Sort node is gone and actual rows fall to about 20.
Run the two negative cases and capture the plan for each.
Re-run the original tenant listing query to confirm the new index has not displaced the index that query depends on.

Expected result: With (tenant_id, owner_user_id, created_at DESC) the plan is an Index Scan with no Sort, roughly 20 rows and a handful of buffers, sub-millisecond. owner_user_id alone still scans; ORDER BY updated_at seeks on the pair and then sorts that owner's rows.

Verify it yourself

Actual rows at the leaf node is on the order of the LIMIT, not the tenant's row count.
No Sort node appears above the index scan.
Buffers read stays constant as one owner's row count grows; if it grows with it, the ORDER BY is not being satisfied by the index.
pg_stat_user_indexes shows idx_scan climbing on the new index and not on the listing index for this endpoint.

Follow-up

90% of rows are status='active'. Would a partial index WHERE status = 'active' change your answer, and for which of the three queries?
A dashboard runs this for 40 owners in one page load. What changes about the design?
How do you roll this index out on a table taking 1.2k writes/second, and what does it cost on every insert from then on?

How do you approach the logic design for a complex piece of automated equipment?

Medium

Approach

Clarify what is being asked and what a complete answer contains.
State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Given a set of requirements for a new machine, how would you estimate the hours required for design and implementation?

Medium

Approach

Work from the requirement backwards to the design.
Say what you would check first and why it is the highest-information step.
Clarify what is being asked and what a complete answer contains.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Shard by tenant when one tenant outgrows a single shard

Hard · sharding · hot partition · routing · online migration

One primary holds resource, resource_revision, outbox_event and idempotency_key for every tenant and is at its write ceiling at 1.2k writes/second. tenant_id leads every index. Shard across eight primaries. One tenant holds 22% of all rows and by itself exceeds a single shard's write capacity. Design the routing, the split of that tenant, and the online move of a tenant between shards with writes continuing. State what breaks for queries that are tenant-scoped today, and exactly what a write must do when it arrives at the old shard after the cutover.

Approach

Route on a unit smaller than a tenant from the start. Make the routing key (tenant_id, bucket) with a fixed bucket count - 64 over eight shards - and keep a directory mapping each (tenant_id, bucket) to a shard, carrying a version and cached in every service. An ordinary tenant has all 64 buckets pointing at one shard and behaves exactly as it does today; only the hot tenant has its buckets spread. Hashing tenant_id alone spreads tenants evenly, gives you no way to move one, and has no answer at all for a tenant larger than a node. The bucket count is the part you cannot change later without rehashing rows, so pick it well above the shard count and rebalance by moving buckets, not by re-bucketing.
For the tenant that exceeds one node, its buckets must land on different shards - that is the whole point of bucketing it, and buckets confined to its own shard would rename the rows while leaving every write on the node whose ceiling it already exceeds. Size it from measured numbers rather than from its row share: 22% of rows says nothing about write rate. The current primary tops out near 1.2k writes/second on this hardware and workload, so a tenant peaking at W writes/second needs its buckets spread over at least ceil(W / headroom-per-shard) shards, where the divisor is the share of each shard's ceiling you are willing to give it while that shard still serves other tenants - not the full 1.2k. Size on its peak, not its mean.
Fix the co-location invariant at the right grain. What must commit in one transaction is a resource, its resource_revision row and its outbox_event row, so the bucket is a property of the resource: derive it once at creation and stamp it into resource_id, and every later revision and event routes with its parent for free. Per-tenant co-location was never the requirement, and mistaking it for one is what makes a tenant look unsplittable. What genuinely breaks is an invariant spanning two resources of one tenant - a per-tenant counter, uniqueness across its resources - which now needs either a home-shard table or two-phase commit, and 2PC at this write rate is not a serious option.
Keep the idempotency constraint arbitrating, because it is now enforced per shard. PRIMARY KEY (tenant_id, idempotency_key) only continues to reject a retry if the same key always lands on the same shard, so derive the create-path bucket from hash(tenant_id, idempotency_key) and mint the new resource_id inside that bucket, which also puts the key row and the resource it guards in one transaction. A bucket chosen from anything that differs between a request and its retry - a timestamp, the worker id, a client-supplied resource id - splits one key across two shards, both inserts succeed, and the write endpoint's retry safety is silently gone.
State what the split costs the hot tenant's reads. Its listing, one 21-entry index scan today, becomes a scatter-gather: every bucket-shard returns 21 rows, a coordinator merges and discards the surplus, latency becomes the slowest shard's rather than the median's, and the keyset cursor has to carry a position per bucket instead of one (updated_at, resource_id) pair. Counts over that tenant fan out the same way. The relay becomes one leader per shard; consumers are unaffected because their ordering guarantee was always per aggregate and a resource's events never leave its bucket.
Move one bucket at a time, reversibly, and fence the straggler at the shard rather than at the caller. Copy from a snapshot while the bucket stays read-write, tail changes until the remaining delta is a few seconds of writes, fence writes for that (tenant_id, bucket) alone with a retryable status, apply the final delta, bump the routing version. Scoping the fence to a bucket is what makes a seconds-long freeze affordable. Then have each shard store the routing epoch it believes it holds for each (tenant_id, bucket) and reject any write carrying an older one: without that token, a service on a stale map commits successfully to a database nothing will ever read again, and the loss stays invisible for days. Outside the data path, anything that aggregated across tenants in one query - admin reporting, the A-Z index, global counters - becomes a fan-out across eight shards with a merge, and per-tenant uniqueness survives only on tables that stay whole on the tenant's home shard.

Worked solution (40 min)

Write the routing lookup keyed by (tenant_id, bucket), its version field, where it is cached and invalidated, and the request-path cost.
From the tenant's measured peak write rate and the per-shard headroom you will grant it, compute how many shards its buckets must span, assign them, and show no single shard carries its whole write rate.
Trace one create end to end: which value picks the bucket, where resource_id gets it stamped, and why the revision, outbox and idempotency rows land on the same shard.
Write the move steps for one bucket, then the epoch check the shard performs on every write, and trace a stale-map write through it.

Expected result: A versioned (tenant_id, bucket) to shard directory cached in every service, with ordinary tenants pointing all their buckets at one shard. The hot tenant's buckets are deliberately assigned to different shards, enough of them that its peak write rate divided across them fits the headroom granted on each, because no single node can carry it. The bucket is derived once and stamped into resource_id, so a resource, its revisions and its outbox rows commit together on one shard while the tenant spans several; only cross-resource invariants within the tenant lose their single-shard transaction. The create-path bucket comes from hash(tenant_id, idempotency_key) so the per-shard unique constraint still arbitrates retries. The tenant's listing becomes a scatter-gather with a per-bucket cursor. Migration moves one bucket at a time - snapshot copy, tail, seconds-long fence for that bucket, final delta, routing-version bump - and a routing epoch per (tenant_id, bucket) is checked by the shard so a stale caller's write is rejected rather than committed where nothing will read it.

Verify it yourself

The hot tenant's buckets resolve to more than one shard, and its peak write rate split across them is inside the headroom each shard has after its other tenants.
A resource's rows, its revisions, its outbox rows and its idempotency key row always resolve to a single shard, so no one write spans two.
The same (tenant_id, idempotency_key) routes to the same bucket on every retry, so the per-shard unique constraint still rejects the duplicate.
A write carrying an old routing epoch is rejected by the shard rather than merely misrouted by the caller.

Follow-up

The fence lasts 90 seconds instead of 4 because the final delta keeps growing. What is happening, and what do you do while the tenant is fenced?
Two tenants must merge into one account. What does that cost under this scheme, and which step is not reversible?
A shard is lost entirely. Which tenants are affected, and what is the source of truth for rebuilding them?

Edge instances grow 400 MB per hour until the nightly restart

Hard · memory leak · heap analysis · metric cardinality · gc

Edge API instances start at 700 MB resident and grow about 400 MB/hour; a nightly rolling restart has hidden it for weeks. Growth continues unchanged when request rate halves overnight, p99 degrades in the last hours before an instance is recycled, and heap used immediately after a forced full GC rises monotonically. The service holds no product state. Name the discriminating measurement that separates the plausible causes, give the most likely cause, and give the fix and how you would verify it.

Approach

Separate resident memory from live heap first, because they fail differently. Resident size can grow from fragmentation, native buffers or thread stacks while the heap is flat; heap used after a full GC rising monotonically is the measurement that says objects are reachable and not being released. You already have it, so this is retention, not fragmentation, and that closes off half the candidate list.
Use the rate's independence from traffic as the discriminator. Growth that continues at half the request rate rules out per-request objects that are merely slow to collect and points at a structure that grows with distinct values observed rather than with call volume. Write the candidates that have that property: a metrics registry keyed on a high-cardinality label, an unevicted cache, an interner, a per-key lock map.
Take two heap snapshots an hour apart and diff by retained size, reading the dominator tree, not by allocation count or instance count. Expect one root holding a map with millions of entries, then follow the reference chain to the code that inserts and never removes. Allocation profilers point at churn, which is the wrong signal here.
The candidate that fits this service is an observability label carrying an identifier, such as a request path recorded before templating so that /v1/resources/48213 becomes its own metric series. That grows with distinct ids seen, is independent of rate, and explains the late p99 degradation, since GC cost rises with the size of the live set.
Fix by bounding cardinality at the source: template the path to /v1/resources/{id} before it becomes a label, move tenant id from a label to a log field or an exemplar, and cap the registry with a bounded map that evicts. Add a cardinality ceiling that fails loudly in a lower environment rather than growing quietly in production.
Verify with a soak rather than a restart. Hold one instance out of the nightly recycle for 48 hours with the fix and compare post-GC heap and series count against an unfixed control taking the same traffic.

Follow-up

Post-GC heap is now flat but resident size still creeps. What are you looking at, and does it matter?
How would you have detected this before an OOM, given the nightly restart masked the trend?
That label is what makes one dashboard useful. How do you keep the dashboard and lose the leak?

Suggested practice checklist

For a candidate senior enough that the loop turns on design and judgement rather than on whether the coding round gets finished. Five days build one system properly and then stress it; coding gets a single maintenance day, on the assumption that the risk at this level is an unexamined tradeoff rather than a missed algorithm.

Day 1 — Numbers before diagrams

Build your own reference card of the figures you will re-derive all week: bytes for a realistic record, requests per second implied by a given daily active count, and the storage that a year at a given write rate produces. Derive each one rather than copying it, because the derivation is what survives a follow-up.
Turn one product statement into capacity requirements. From ten million daily users at four writes and forty reads each, state the peak-to-average factor you are assuming and why, then produce peak write QPS, peak read QPS and a year of storage.
Write the two numbers whose order of magnitude changes the design, the read-to-write ratio and the working-set size against memory per node, and state the threshold at which each one flips your answer.

Deliverable: A one-page numbers card and one worked capacity estimate with every assumption written down.

Day 2 — One system, from requirements to schema

Spend the first ten minutes producing only functional requirements, non-functional targets with numbers attached, a p99 latency, a durability expectation, a consistency requirement, and an explicit out-of-scope list.
Define the interface before the boxes: the three or four endpoints, their parameters, what each returns, and which of them are idempotent.
Write the data model, then write the single access pattern that justifies it, and state what the schema would have to become if the dominant access pattern were the other one.

Deliverable: One design carried to endpoint-and-schema depth, with non-functional targets expressed as numbers and a written out-of-scope list.

Day 3 — The consistency you are actually buying

Write out what a client sees under asynchronous replication when its write commits on the leader and its next read is served by a lagging follower, then write the two fixes, pinning that session's reads to the leader for a bounded window or carrying a version token the replica must reach, and the cost of each.
Work the quorum arithmetic on paper for N of three with W and R of two, and separate what R + W > N does guarantee, that any read set intersects any write set, from what it does not: on its own it is not linearizability, and a sloppy quorum that accepts writes on nodes outside the preference list breaks even the intersection.
Take two storage choices with different defaults, a single-leader relational store committing synchronously and a quorum-replicated store that converges eventually, and write the specific product behaviour that would be wrong under each, rather than a general statement about which is stronger.

Deliverable: A page separating what quorum overlap guarantees from what it does not, with one concrete product misbehaviour attached to each gap.

Day 4 — Failure is the design

For one write path, work through the case where the client times out after the server has already committed, then design the idempotency key: who generates it, how long it is retained, and what the duplicate request returns.
Express the retry policy as parameters rather than as a word: maximum attempts, base delay, backoff factor, jitter, and which error classes are retried at all. Then state why retrying a non-idempotent write without a key is a correctness bug and not merely waste.
Compute the fan-out effect on tail latency. If a request waits on ten backends and each independently exceeds its p99 one percent of the time, the chance at least one is slow is 1 - 0.99^10, about ten percent. Then write why independence is the optimistic assumption and what correlates them in practice.
Name the backpressure mechanism for one queue or one dependency in the design, a bounded queue with shedding or a concurrency limit, and write what the caller is told when it engages.

Deliverable: One write path with an idempotency design, a parameterised retry policy, and a written tail-latency calculation with its assumption named.

Day 5 — Scaling the hot path

Choose cache-aside or write-through for one read path and write the staleness window each produces, then name the invalidation event and what the system does when that event is lost.
Design against the stampede: either coalesce requests so only one recomputes a missing key, or refresh early with jittered expiry, and write why identical TTLs on keys populated in the same moment produce a synchronised expiry and a thundering herd.
Shard one table by a key you choose, then answer the two questions that break the choice: which queries now require a scatter-gather, and what happens to the distribution when one tenant is a hundred times larger than the median.
Write the cost of adding a node under plain modulo placement, where nearly every key moves, against consistent hashing, where roughly one key in n+1 moves, and state what virtual nodes are for.

Deliverable: A caching and sharding decision for one path, each with its failure mode and its rebalancing cost written beside it.

Day 6 — Keep the coding hand in, at the bar that applies to you

Solve one medium problem in thirty minutes, then spend twenty more making it production-shaped: named invariants, validation at the boundary, and errors that distinguish a caller mistake from an internal fault.
Write the tests you would require of a colleague's version of that function: one for empty input, one for the boundary, and one for the case the implementation is most likely to get wrong.
Read a piece of your own code from six months ago and write the change you would ask for, phrased as you would actually phrase it in review.

Deliverable: One problem hardened to review standard, with its test list and one written review comment.

Day 7 — Defend it while being interrupted

Run a forty-five-minute design mock with an interviewer briefed to change a requirement halfway, a tenfold traffic increase or a new strict consistency requirement, and to push on one number you estimated.
Rehearse the two sentences a senior loop is listening for: naming the tradeoff you are choosing against and why, and saying what you would measure to learn that the choice was wrong.
Prepare the design you regret: a real decision, the constraint that produced it, what it cost, and what you changed afterwards.

Deliverable: Mock notes recording how the design changed under the new requirement, plus a written account of one regretted decision.

Disagreement needs an outcome you did not choose

Conflict answers where you were right and everyone came round are the weakest ones. Stronger: the evidence you went and collected, what would have changed your mind, and what you did in the weeks after the call went against you. Implementing a design you argued against, properly, is a specific and checkable behaviour.

Tell us about your experience.

Medium · behavioural and engineering judgement

Approach

Pick a story where you made the decision, not one where you watched it.
State the situation in two sentences and spend the rest on the reasoning.
Name the disagreement and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know your change caused the improvement?

Elaborate on a prior experience as an engineer that will help you exceed in this job role.

Medium · behavioural and engineering judgement

Approach

Pick a story where you made the decision, not one where you watched it.
State the situation in two sentences and spend the rest on the reasoning.
Name the disagreement and how you resolved it with evidence.

Follow-up

How did you know your change caused the improvement?
What did you decide not to do, and why?

Can you describe a time you had to troubleshoot a technical issue on the shop floor?

Medium · behavioural and engineering judgement

Approach

Give the blast radius: what could have broken, and what you measured.
Pick a story where you made the decision, not one where you watched it.
Name the disagreement and how you resolved it with evidence.

Follow-up

How did you know your change caused the improvement?
What did you decide not to do, and why?

Why are you interested in joining the team at Altron Automation?

Medium · behavioural and engineering judgement

Approach

Close with what you would do differently, concretely.
State the situation in two sentences and spend the rest on the reasoning.
Pick a story where you made the decision, not one where you watched it.

Follow-up

What would you do differently if you ran that again?
What did you decide not to do, and why?