Interview guides · 1st St. NW Technical Staffing

1st St. NW Technical Staffing Software Engineer Interview Questions & Guide 2026

1st St. NW Technical Staffing software engineer prep: algorithm drills where every complexity claim has to name the assumption under it.

The 60-second brief

At 1st St. NW Technical Staffing, the Software Engineer role serves as a critical bridge between complex mechanical systems and the intelligent controls required to operate them. You will be instrumental in developing, optimizing, and maintaining the software frameworks that power custom machinery, industrial automation, and high-stakes infrastructure projects. Your work directly impacts the reliability and performance of systems that range from HVAC and refrigeration units to advanced custom mechanization and attractions.

Treat capacity estimation as a conversion skill rather than a table to memorise: turn a user count and an action rate into requests per second and bytes per day, then name the component that number breaks first. The figure only matters if it changes the design.

Focus areas

Diagnose failures inside environments you cannot instrument
Version an API you cannot force clients to upgrade
Make integration retries idempotent without target-side idempotency keys

Overview

This position is inherently technical and strategic, requiring you to balance low-level system logic with high-level architectural goals. You will collaborate with cross-functional teams—including mechanical, electrical, and manufacturing engineers—to ensure that the software you design integrates seamlessly with physical hardware. If you thrive in environments where code meets the real world and where your technical decisions have tangible, large-scale results, this role offers a challenging and rewarding career trajectory.

The 1st St. NW Technical Staffing interview process

1st St. NW Technical Staffing candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Half of this call is the part candidates treat as small talk: start date, notice period, work authorisation and its timing, location and time zone, on-call, and the number. Those are what kill offers late, after several engineers have each spent a day. Surfacing a hard constraint now costs you nothing and occasionally buys you something, since a loop compressed to fit a competing deadline can usually only be arranged if it is asked for early. The common failure is deflecting the compensation question twice, then discovering at offer stage that the band never reached your number.

What to demonstrate

Whether your hard constraints are compatible with the role before a loop gets booked: earliest start, notice period, what authorisation you hold and when it needs action, days on site, willingness to carry a pager
Whether you give a compensation range with something behind it, such as current total compensation or a competing timeline, rather than leaving the band untested
Whether your stated timeline is real, since a competing deadline raised now is something scheduling can sometimes work around and the same deadline raised at offer stage usually is not

How to prepare

Write each constraint down in one line before the call and state them as facts rather than negotiating them live under a question you were not expecting
Set your range from two or three current data points for that level and location, and name the structure you are quoting in, so the number is comparable to the one they are holding
If another process is running, say where it stands and by when, and ask directly whether this loop can be scheduled inside that window

Technical Deep-Dive

Input bounds are the part of the prompt most often skimmed, and they usually contain the answer. They tell you which complexity class is admissible, which narrows the search before you have thought about the problem itself. As a rough planning figure, a compiled language does on the order of 10^8 simple operations per second and an interpreted one roughly an order of magnitude less. So n up to about twenty admits enumerating subsets, a few thousand admits a quadratic pass, and a million admits neither: you need near-linear, or linear with a log factor. If the bounds are missing, ask for them.

What to demonstrate

Whether the approach is justified by the stated input size rather than by whichever pattern you recognised first
Whether you ask about the properties that change the algorithm: whether the input arrives sorted, whether duplicates occur, whether values are bounded integers, whether it all fits in memory
Whether you can name the bottleneck in your own solution and what would remove it, even when you deliberately leave it in place
Whether a claimed speedup is real, since memoising a recursion only helps when subproblems genuinely overlap and the state can be keyed cheaply

How to prepare

For each algorithm you rely on, write down the largest n it handles in roughly a second, then check two of those figures by timing them in the language you will actually type in
For two weeks, write one line naming your target complexity and the bound that justifies it before you write any code, then compare that line with what you ended up submitting
Practise the conversion backwards: given a required O(n log n), list the mechanisms that get you there (sorting, a heap, an ordered map, divide and conquer) and choose by what the problem needs to query, not by what you used last

Team Interaction

An unlabelled round is first an information problem, and the cheapest information is free. Whoever schedules it can usually tell you how long it runs, who will be in the room and what they work on, whether you will be writing code and in what environment, and whether anything is being sent beforehand. Ask in writing so the answer is on record, then prepare for the two or three formats those answers still leave open instead of betting on one. What separates a strong candidate is not guessing right; it is having an opening that works whichever one it turns out to be.

What to demonstrate

Whether you can start work from an ambiguous brief, since tolerating a vague scope without stalling is the same thing the job asks for
Whether the questions you asked beforehand were ones that change your preparation, such as duration, medium and who is joining, rather than ones whose answers you could not have acted on
Whether you adapt when the round turns out to be something other than what you were told, instead of spending the first ten minutes visibly recalibrating

How to prepare

Send one short scheduling message asking four things: how long, who is joining and what they work on, whether you will be writing code and where, and whether to prepare anything in advance. Treat a vague reply as real information, since it means the round is loosely structured and you will be shaping it yourself.
Write one opening that works in any of the formats still open: restate in your own words what you have been asked to do, then ask which of two directions is more useful to them. Say it aloud until it stops sounding recited.
Set up for the two most likely formats before the call starts, with a blank editor in the language you would choose and a shared document you can type into, so a format surprise costs you nothing in the first minutes

Where candidates lose points

Treating an ambiguous failure as a definite one

A timeout, a 502 from an intermediary, or a connection reset after the request bytes were sent all leave the target's state unknown. Classifying those as failures and retrying duplicates the effect; classifying them as successes and advancing the watermark loses data silently. Both wrong answers are common because the ambiguous case is rare in a sandbox and routine in production. The run needs a distinct ambiguous state, a dedupe key that makes the retry safe, and a reconciliation read against the target when the key alone cannot settle it.

Tenant context leaking across a connection pool

Setting the tenant on a connection and relying on it for the rest of the request looks correct in every test that runs one request at a time. Under transaction pooling the connection goes back to the pool with that session state still attached, and the next checkout, serving a different client, inherits it; row-level security then enforces the previous tenant's policy flawlessly and returns the wrong rows. It reproduces only under concurrency, which is the load profile your integration tests do not have. Use a transaction-scoped setting, and assert that the setting matches the request's tenant immediately before the first query rather than trusting that it was set correctly upstream.

Assuming the bug is in the framework

Suspect your own code first: read the stack trace top to bottom, check which versions are actually installed rather than which ones you believe are, and reproduce in isolation before blaming a library that thousands of people run daily. When the fault really is upstream, you need that minimal reproduction to say so credibly anyway.

Starting work without saying what you are about to spend time on

State the plan before executing it: the approach, roughly how long it will take, and what you intend to leave hand-waved. That gives the interviewer a chance to redirect you in ten seconds rather than watching you spend fifteen minutes on the wrong sub-problem.

Questions & practice

Find peak window utilisation and the first rejected request

Medium · sliding window · two pointers · rate limiting

One connector binding's requests against a client endpoint arrive as timestamps t[0..n-1] in milliseconds, sorted non-decreasing, n up to 1,000,000. The endpoint's budget is R requests per rolling window of W milliseconds, and a request at time t is admitted only if strictly fewer than R already-admitted requests fall in the half-open window (t-W, t]. Return (a) the maximum number of requests falling in any W-millisecond window, counting every request regardless of admission, and (b) the index of the first request the limiter would reject. Both parts in O(n) time.

Approach

Part (a) is a two-pointer scan: for each right index, advance left while t[right] - t[left] >= W, which leaves the window exactly (t[right]-W, t[right]], and track max(right-left+1). Left is monotone non-decreasing, so total work is O(n) amortised with O(1) extra space. The maximum over all windows of width W is attained at a window whose right edge is a request timestamp, so scanning right edges is sufficient.
Part (b) is not the same scan. A rejected request consumes no budget, so the window must count admitted requests only. Keep a FIFO deque of admitted timestamps: pop from the front while front <= t - W, admit if the deque holds fewer than R, and return the first index where it does not. O(n) total pushes and pops, O(R) memory.
Pin the boundary convention before writing either loop. With a half-open (t-W, t] window a request exactly W milliseconds after an earlier one does not see it; switching to a closed window changes the answer at every boundary and is where most disagreements with the client's own limiter come from.
Note what fixed buckets of width W buy and cost: O(1) memory, but they admit up to 2R inside a span shorter than W, R at the end of one bucket and R at the start of the next. If buckets are unavoidable, halve the bucket width or use a weighted estimate across two adjacent buckets.
If part (a) already exceeds R, the binding cannot run at this concurrency under any limiter shape. That is a scheduling problem, not a retry-policy problem, and no backoff tuning fixes it.

Follow-up

Three bindings and four workers share this one endpoint's quota. Where does the counter live, and what is the correct behaviour when that store is unreachable?
Add full-jitter backoff to the retry path and state precisely what it prevents that plain exponential backoff does not.
The client documents the limit as 'about 100 per minute'. How would you measure the real one without tripping it in their production?

Compare release versions and report the live estate spread

Easy · parsing · version skew · ordering

Each environment row carries reported_release as TEXT: either NULL, or a version of the form MAJOR.MINOR.PATCH with an optional -rc.N suffix. Up to 400 environments. Ordering rule: compare major, then minor, then patch numerically; a -rc.N build precedes the same MAJOR.MINOR.PATCH with no suffix, and rc numbers compare numerically. Return the lowest and highest reported versions, the minor-version spread between them, and the count of environments whose reported_release is NULL. NULL means the environment's version is unknown, not that it converged.

Approach

Parse each string once into a comparable tuple: (major, minor, patch, is_final, rc), where is_final is 1 for a plain release and 0 for a pre-release, and rc is 0 when absent. Tuple comparison then yields the stated order for free, including 3.10.0-rc.1 < 3.10.0. O(L) per string, O(n*L) total.
Track min and max in one linear scan rather than sorting: O(n) time, O(1) extra space, and n is only a few hundred so the parse dominates either way.
Fail loudly on a string that does not match the grammar instead of defaulting it to 0.0.0 or to the newest release. An unparseable reported_release is a reconciliation-ingest bug, and either default hides it in the direction that looks safe.
Compute the minor spread only when min and max share a major version. Across majors the difference of minor numbers is meaningless, so return the pair of versions and flag the condition rather than a number.
Report the NULL count as its own output line. Folding unknowns into 'converged' is how an install ends up running a release nobody knew was still live.

Follow-up

Two majors are live at once. What does 'spread' mean now, and what single number, if any, would you still publish?
reported_release is a claim from the environment, and last_heartbeat_at may be days old for a client-managed install. At what staleness does this figure stop being usable?
How would you enforce the supported window when the rollout plan is built, instead of reporting a violation after it already exists?

Measure credential exposure past engagement close without double counting

Medium · interval merge · sweep line · credential ttl

For one engagement you have credential_grant rows: (grant_id, principal_id, target_system_id, issued_at, not_after, revoked_at which may be NULL). The cutoff instant T is the engagement's ends_on plus access_grace_days. A grant is live over the half-open interval [issued_at, min(not_after, revoked_at)). There are up to 200,000 grants across target systems. For each target_system_id, return the total wall-clock time after T during which at least one grant was live, plus the disjoint segments that make it up. Grants that overlap must be counted once.

Approach

Clip each grant to [max(issued_at, T), min(not_after, coalesce(revoked_at, +inf))) and discard any interval whose start is not strictly before its end. O(n), and it removes every grant that already expired inside the engagement window.
Bucket the survivors by target_system_id and sort each bucket by start. O(n log n) overall, and sorting dominates the whole algorithm.
Sweep each bucket once, holding one open segment [s,e): if the next start is greater than e, emit [s,e) and open a new segment, otherwise set e = max(e, next_end). O(n) after the sort, O(1) working state, O(number of emitted segments) output.
Total exposure is the sum of emitted segment lengths, which is the measure of the union. Summing per-grant durations instead answers a different question and is unbounded above by the elapsed wall clock.
Produce a second, conservative figure that ignores revoked_at and uses not_after alone. revoked_at records that you asked for revocation; whether the client's identity provider honoured it is not something this table knows, and a self-contained token validated offline stays valid to its own expiry regardless.
If a bucket does not fit in memory, sort externally and sweep the stream, or push end timestamps into a min-heap keyed by end and pop those below the current start. Same O(n log n), memory proportional to the maximum number of concurrently live grants.

Worked solution (25 min)

Set T = 2026-06-30T00:00Z for one target system with four grants: g1 not_after 2026-06-29T12:00Z; g2 not_after 2026-07-01T00:00Z, revoked_at NULL; g3 not_after 2026-07-01T06:00Z, revoked_at 2026-06-30T18:00Z; g4 issued 2026-07-01T06:00Z, not_after 2026-07-02T06:00Z.
Clip: g1 becomes empty and drops out. g2 -> [Jun30 00:00, Jul1 00:00). g3 -> [Jun30 00:00, Jun30 18:00). g4 -> [Jul1 06:00, Jul2 06:00).
Sort by start and sweep: g2 and g3 merge into [Jun30 00:00, Jul1 00:00); g4 starts after that end, so it is emitted separately.
Union = 24h + 24h = 48h = 2,880 minutes, in two segments with a 6-hour uncovered gap between them. Summing the same three clipped intervals per grant instead gives 24 + 18 + 24 = 66h; the 18h excess is exactly the [Jun30 00:00, Jun30 18:00) stretch where g2 and g3 overlap.
Recompute ignoring revoked_at: g3 now runs to Jul1 06:00, which bridges the gap, and the union becomes the single segment [Jun30 00:00, Jul2 06:00) = 54h.

Expected result: Exposure after T = 2,880 minutes in segments [Jun30 00:00, Jul1 00:00) and [Jul1 06:00, Jul2 06:00). The conservative figure that ignores revoked_at is 3,240 minutes in one segment. Naively summing the clipped per-grant durations (24 + 18 + 24) would have reported 66 hours, more than the 54 hours that elapsed between T and the last expiry.

Verify it yourself

Union (48h) <= the clipped per-grant sum (66h) always, with equality only when no two clipped intervals overlap; here the 18h difference is the single g2/g3 overlap.
Union (48h) <= last expiry minus T (54h), since the union cannot exceed the elapsed span it lives in.
The emitted segments are strictly increasing and pairwise disjoint, and their count is 2, less than the 3 surviving grants.
g1 contributes nothing, confirming the clip ran before the sort rather than after it.

Follow-up

Three grants overlap and one of them belongs to an offboarded contractor. What must the sweep emit so exposure can be attributed per principal?
Which of your two numbers is the real bound on access if the client system validates tokens offline, and what does that imply about issuing TTLs in the first place?
Run this across 20,000 engagements as a nightly job with a fixed memory budget. What changes in the shape of the computation?

Denormalise the entitlement snapshot without freezing the clock

Medium · denormalisation · read patterns · caching

Authorisation currently joins engagement (effective-dated, with status, ends_on, access_grace_days, effective_from, effective_to), entitlement and principal_assignment on every request, at the platform's full read rate, against tables taking a few hundred writes a day. You are asked to publish a denormalised snapshot that callers cache with a few seconds of staleness and invalidate on change events. Specify the snapshot's key and shape, say which columns you denormalise, name the one thing that must not be precomputed, and say what a cache miss does.

Approach

Justify the shape from the ratio rather than from taste: a few hundred writes a day against the whole estate's read rate is the case for a published read model. The normalised tables stay the system of record; the snapshot is derived, versioned and disposable, and can be rebuilt from source at any time.
Key it on (client_id, principal_id) with a monotonic snapshot_version and a generated_at. Consumers cache by version, refresh on a change event, and reject anything older than the staleness bound, which turns the bound into an enforced property instead of an assumption about how fast events usually arrive.
Denormalise only what changes on a write: the open engagement version's engagement_id, status, isolation_tier, data_region, ends_on, access_grace_days, and the resolved scope set. Resolving the effective-dated row once at publish time — selecting the version WHERE effective_to IS NULL — is the entire saving, because that resolution is what the per-request join was paying for.
Do not precompute is_active or any boolean that folds now() into the stored value. A write event invalidates the snapshot; the passage of time does not, so an engagement that ended an hour ago keeps authorising until something unrelated triggers a republish. Ship ends_on and access_grace_days as data and compare them to the current time at read, so time-based expiry needs no invalidation at all.
Fail closed on a miss or on over-staleness: deny, and do not fall through to the source tables. A fallback read path means the first mass cache miss converts an availability incident into either an authorisation bypass or a stampede onto the one service whose availability target already exceeds everyone else's.
State the cost you accepted: a revoked entitlement is still honoured for up to the staleness bound. That is defensible for widening access and not for narrowing it, so pair it with an out-of-band deny signal for revocation and closure, and write the residual window into the runbook instead of leaving it for an auditor to discover.

Worked solution (25 min)

Write the snapshot record's field list and key, marking each field as write-invalidated or time-evaluated.
Write the publisher query that resolves the open engagement version and aggregates scopes per (client_id, principal_id).
Write the consumer's decision function, taking the snapshot plus a current timestamp, and show that it denies on missing, on stale, and on ends_on plus grace elapsed.
Write the reconciliation query that recomputes a sample of keys from source and diffs them against the published snapshot.

Expected result: A (client_id, principal_id)-keyed record carrying resolved scopes plus engagement_id, status, isolation_tier, data_region, ends_on, access_grace_days, snapshot_version and generated_at; every time-dependent decision evaluated at read; missing or over-stale entries denied rather than backfilled from source.

Verify it yourself

Freeze the publisher and advance the clock past ends_on plus grace: authorisation must start failing with no new event and no republish.
Delete a snapshot entry and confirm the caller denies rather than querying the source tables.
Reconciliation over a sample of keys reports zero differences outside the staleness bound; any difference inside it is explainable by a specific event.

Follow-up

How do you detect the snapshot quietly diverging from the source tables, given that a wrong snapshot is silently permissive?
An amendment creates a new engagement version mid-request. Which version does the in-flight request use, and can you defend that?
The snapshot carries data_region. What stops the snapshot itself from being replicated somewhere its own residency rule forbids?

Return the latest attempt per connector idempotency key

Easy · distinct-on · indexing · idempotency

connector_run stores one row per attempt: run_id, connector_id, idempotency_key, attempt_no SMALLINT, status (queued, running, succeeded, failed, ambiguous, cancelled, abandoned), failure_class, watermark_from, watermark_to, started_at, finished_at, unique on (connector_id, idempotency_key, attempt_no). For one connector_id, write the query returning the highest-numbered attempt per idempotency_key, keeping only those keys whose latest attempt is not succeeded, ordered by watermark_from. Give the index that serves it, and say what the plan does without that index.

Approach

Use DISTINCT ON (idempotency_key) with ORDER BY idempotency_key, attempt_no DESC. PostgreSQL requires the ORDER BY to lead with exactly the DISTINCT ON expressions, and that requirement is what makes the row picked deterministic rather than whichever the scan happened to reach first.
Apply the not-succeeded filter outside, in a wrapping SELECT. Putting it in the inner WHERE changes the question from 'keys whose latest attempt failed' to 'keys that have any non-succeeded attempt', which reports a key that failed twice and then succeeded, and is the single most common wrong answer here.
Know the portable alternative: row_number() OVER (PARTITION BY idempotency_key ORDER BY attempt_no DESC) in a subquery, filtered = 1 outside. A window function cannot be filtered in WHERE because WHERE is evaluated before the window, and PostgreSQL has no QUALIFY clause, so the subquery is mandatory rather than stylistic.
Index (connector_id, idempotency_key, attempt_no DESC). With connector_id fixed by equality, the index supplies the exact order DISTINCT ON needs and the Sort node disappears. It does not reduce rows read: through PostgreSQL 17 there is no loose index scan, so the executor still walks every attempt in range rather than skipping to each next distinct key.
Give the complexity honestly: without the index the plan is a scan plus an O(n log n) sort over the connector's attempts; with it, O(n) ordered reads and constant working memory, where n is attempts in range, not distinct keys. If the distinct keys are a tiny fraction of attempts, a recursive skip-scan is the next step.

Follow-up

attempt_no is not monotonic in time because a retry was queued out of order. Does your query still pick the right row, and which column should it actually order by?
How would you return the latest attempt per key across all connectors in one environment without the query degrading to a full sort?
An ambiguous latest attempt means the target may or may not have been written. What does this query feed, and should ambiguous be treated as failed here?

What is your methodology for testing and validating code before deployment into physical machinery?

Medium

Approach

Work from the requirement backwards to the design.
State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

How do you manage technical debt in systems that require long-term reliability?

Medium

Approach

Clarify what is being asked and what a complete answer contains.
Say what you would check first and why it is the highest-information step.
Work from the requirement backwards to the design.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Tell us about a project where you had to integrate software with a mechanical system that wasn't performing as expected.

Medium

Approach

Clarify what is being asked and what a complete answer contains.
Say what you would check first and why it is the highest-information step.
Work from the requirement backwards to the design.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Keep delivery running when the control services go unreachable

Hard · consistency vs availability · partition · credential ttl · bounded exposure

A network partition cuts one region's workers off from the Engagement and Entitlement Service and the Access Broker for 90 minutes. Residency forbids failing over to another region's copy of that data. Inside the region, connector runs hold cached credentials with not_after values minutes to hours out, and operator sessions are mid-task. The entitlement service fails closed by design. Decide what continues and what stops, give a stated exposure bound for each choice, and name the one thing that must never continue however long the partition lasts.

Approach

Classify each operation by whether it widens authority, because that is what decides the answer. Continuing on a credential already minted extends access that was already granted; minting a new one grants access you currently cannot verify. A blanket fail-closed that stops both is as unconsidered as a blanket carry-on, and neither tells you what your exposure was afterwards.
Let in-flight work continue on already-issued credentials. Their exposure is already bounded by not_after, which is the reason you capped it at issue, and the worst case is the longest outstanding not_after at the moment the partition began. That number is not fate: capping connector TTLs at the expected run duration rather than at the engagement window shrinks it in advance, which is the only time it can be shrunk.
Refuse everything that widens authority - new mints, scope increases, the first run of a binding that has never run, an operator establishing new access. Return those as readable refusals naming the cause rather than as transport errors, or every operator and automated agent in the region spends 90 minutes guessing whether the system is broken or is saying no.
Name the operation that must never continue: work against an engagement whose ends_on has already passed. Expiry is arithmetic over data every worker already holds and needs no network, so a partition is no excuse for it. This is the concrete payoff of never materialising an active flag - the one check that has to survive isolation is the one that is pure computation on a date.
Time-box the degradation by construction: refuse to serve from a snapshot older than the staleness bound, so the window ends on its own rather than when somebody notices. On reconnect, reconcile rather than resume - re-read entitlements, revoke grants for engagements that closed during the window, and resolve runs that ended ambiguous against their dedupe keys, since a partition is the single most likely producer of ambiguous outcomes.
State the cost plainly. This chooses availability for already-authorised work and consistency for every change of authority, and pays with a window in which a closure decided elsewhere is not yet honoured. That window is min(longest remaining TTL, staleness bound), it is a number you can put in front of a client, and it is shorter than the partition.

Worked solution (45 min)

Write the region's operation inventory: for each operation, whether it widens authority, what it reads, and whether that read can be served from cached state.
For each one, state the exposure if it proceeds on data 90 minutes stale, as a duration and a scope rather than as a risk rating.
Simulate the partition with a firewall rule and run two bindings: one whose credential has 40 minutes left, one whose engagement ended yesterday.
Restore connectivity and run the reconciliation, recording which grants are revoked, which runs are reclassified, and which watermarks are re-checked.

Expected result: The binding with a live credential finishes its range and then stops at expiry without entering a mint retry loop. The binding on the ended engagement refuses immediately and locally, citing ends_on, and never touches the network. Mints and scope changes refuse for the whole window. After reconnection, grants for engagements closed during the partition are revoked, ambiguous runs resolve against dedupe keys instead of blind retries, and the exposure actually realised matches the number predicted in step two.

Verify it yourself

Maximum exposure equals min(longest outstanding not_after, staleness bound) and is computed from credential_grant rather than estimated.
No code path can reach another region's entitlement data, asserted by a test on the region-scoped client rather than by policy.
Every refusal during the partition carries a distinct cause and is counted apart from client-side failures, so the blast radius is readable after the fact.

Follow-up

An engagement is terminated for cause during the partition. What is the real access window, and what would you change afterwards to shorten it?
Half the region's workers can reach the entitlement service and half cannot. What breaks in that case that a total partition would not have broken?
Your staleness bound is 10 seconds and the partition lasts 90 minutes, so delivery stops entirely. Is that bound right, and what evidence would change it?

Entitlement snapshot expires in lockstep across the fleet

Medium · cache stampede · ttl jitter · single-flight

Delivery requests across the estate return 403 for roughly 200 ms every five seconds. The entitlement service publishes one versioned snapshot; every caller caches it for five seconds and fails closed on a miss. Its request rate is near zero for most of each window and about 4,000 in a 300 ms burst, spread across the 800 application processes a rolling deploy restarted this morning. Its own handler service time is unchanged. Diagnose the shape, then give a fix that preserves fail-closed behaviour.

Approach

Establish the period before the cause. Plot the origin's request rate at one-second resolution: a sawtooth whose period equals the cache TTL, with the mass inside a few hundred milliseconds, says the misses are synchronised, not that traffic grew.
Measure two things next, in this order: key cardinality and in-flight concurrency per process. One key across the whole fleet means every process misses at the same instant. Four thousand requests from 800 processes means roughly five concurrent misses inside each process, so there is no single-flight and each in-flight request for the same key issues its own fetch.
Explain why the phases are locked: the rolling deploy populated every cache within the same few seconds, so 800 independent five-second TTLs expire together and nothing in the system ever scatters them again.
Fix three independent defects rather than one. Single-flight per key per process, so a process has at most one outstanding fetch for a key. A per-process randomised TTL, for example uniform over four to six seconds, so expiry decorrelates after any synchronising event. Refresh-ahead at about 80 percent of the TTL, so the hot key is replaced before it expires and a request never waits on a miss at all.
Keep fail-closed honest about what staleness costs. Serving a stale snapshot on a fetch error is safe for the engagement end-date case, because ends_on travels inside the snapshot and the caller evaluates it locally. It is not safe for a mid-engagement suspension, which only arrives as an invalidation event. So bound the stale window explicitly at tens of seconds, fail closed beyond it, and state the resulting suspension propagation delay as the accepted price.
Name the new steady state so nobody is surprised by it: compute it from the refresh interval rather than the TTL, because refresh-ahead is what now sets the period. Refreshing at 80 percent of a 5-second TTL means each process fetches every 4 seconds, assuming the refreshed entry restarts the TTL clock, so the floor is fleet size divided by the refresh interval: 800 / 4 = 200 requests per second, not the 800 / 5 = 160 that the TTL alone would suggest. Jitter nudges it up rather than down, because the rate averages 1/T and not 1/mean(T): with TTL uniform over 4 to 6 seconds the floor is about 203 requests per second. A process that stops asking for the key stops refreshing it, so 200 is the floor for a uniformly hot fleet and an upper bound for any other traffic mix. Verify by replaying 800 synchronised clients and confirming the origin sees a flat rate at that level with no burst above a stated ceiling.

Follow-up

At 8,000 processes the steady refresh floor becomes 8,000 / 4 = 2,000 requests per second, set by process count and refresh interval alone and independent of user traffic. What changes in the design?
A client is suspended for non-payment at 09:00. Walk the worst-case time until every process stops serving them, with your stale window in place.
How would you detect the next stampede automatically, given that the origin's average rate barely moves when one occurs?

Suggested practice checklist

Day one measures instead of guessing, under a fixed rubric, and the remaining hours are allocated in proportion to the gaps before any studying begins. The allocation is deliberately not renegotiated midweek, because the area that feels worst on day three is usually the one that is moving.

Day 1 — Diagnostic, scored before you study anything

Sit a 110-minute diagnostic in four blocks: forty-five minutes on two coding problems, twenty-five on one design prompt taken to interface and data model, twenty of short-answer fundamentals, and twenty delivering two behavioural answers aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct and 0 is stuck, grading the artifact rather than how the attempt felt.
Allocate days two to five in proportion to 3 minus each block's score, write the allocation down, and commit to leaving it alone.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Split the weakest area into named sub-skills and rate each separately. For coding those are restating the problem, choosing the structure, stating the invariant, turning the invariant into loop bounds, handling empty and single-element input, and accounting for complexity out loud.
Attempt three items positioned just above where the rating drops off, and for each write the first move you failed to make.
Re-attempt one of them from blank four hours later with nothing open.

Deliverable: A sub-skill map with the two blocking sub-skills circled.

Day 3 — Drill the blocking sub-skill by repeating the shape

Do eight short repetitions of the same shape rather than eight different problems, so what gets practised is the pattern and not the puzzle.
State the rule you now hold in one sentence, then test it against a case built to break it, a sliding window over an array containing negative values, or a cache-aside read path whose invalidation message is dropped.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on the strongest area

Run the same sub-skill decomposition on the second-largest gap in half the time.
Spend twenty-five timed minutes on the block you scored highest, choosing the hardest item you can still finish rather than a warm-up.
Write whether each area fails you on recall, on setup, or on execution, and set the fix accordingly: repetition for recall, a written checklist for setup, timed work for execution.

Deliverable: A second sub-skill map plus a one-line failure diagnosis for each area.

Day 5 — The gap that is not a skill

Record one technical and one behavioural answer, then count two things in the playback: seconds before your first clarifying question, and sentences you began without knowing where they would end.
Practise saying that you do not know, followed by how you would find out, without letting it soften into a guess, and practise stating a complexity or an estimate before being asked for it.
Redeliver one answer under a hard ninety-second cap, which forces structure ahead of detail.

Deliverable: Two recordings with a counted reduction in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 110-minute structure with new prompts of comparable difficulty and score it on the identical rubric.
For any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write down which single block you would still lose the offer on.

Deliverable: A second scored rubric placed beside the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a sixty-minute mock over the two blocks that moved least, with an interviewer briefed to interrupt and change direction mid-answer.
Write the recovery script for going blank: restate the question, state your assumption, name the first thing you would check.
Say every rule from the week aloud without reading it, and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive an interruption.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Know where your numbers came from

Bring the two or three numbers the story rests on and know how they were collected. A p99 whose timer starts inside your handler excludes the time a request spent queued, so it can sit flat while users wait longer. Give the window, the percentile and what the measurement left out, or drop the number.

Describe your experience with the Allen-Bradley Logix platform or CODESYS in a production environment.

Medium · behavioural and engineering judgement

Approach

Name the disagreement and how you resolved it with evidence.
Close with what you would do differently, concretely.
Give the blast radius: what could have broken, and what you measured.

Follow-up

What would you do differently if you ran that again?
How did you know your change caused the improvement?

How do you handle hardware limitations when optimizing software performance for real-time systems?

Medium · behavioural and engineering judgement

Approach

State the situation in two sentences and spend the rest on the reasoning.
Pick a story where you made the decision, not one where you watched it.
Name the disagreement and how you resolved it with evidence.

Follow-up

How did you know your change caused the improvement?
What did you decide not to do, and why?

Describe a time you encountered an unexpected hardware failure; how did you diagnose the root cause?

Medium · behavioural and engineering judgement

Approach

Pick a story where you made the decision, not one where you watched it.
Close with what you would do differently, concretely.
Give the blast radius: what could have broken, and what you measured.

Follow-up

What did you decide not to do, and why?
How did you know your change caused the improvement?