Interview guides · AMTEC (WI)
AMTEC (WI) Software Engineer Interview Guide
AMTEC (WI) Software Engineer interview guide: Oracle EBS, system integration, CNC troubleshooting and quality questions, with answers and a 7-day plan.
The 60-second brief
AMTEC (WI) works in manufacturing and engineering. Candidates describe a Software Engineer role that covers three kinds of work: CNC programming, development on Oracle E-Business Suite (Oracle EBS), and integration between systems. The responsibilities they list include building, testing and maintaining the software behind core business functions, migrating legacy systems, writing custom scripts for CNC machinery, extending Oracle EBS, and watching how shipped software performs so it can be improved from user feedback. The role description also has engineers working with Quality Engineers and Program Managers so that software fits project schedules and safety requirements. In practice the job sits where enterprise data held in an ERP meets equipment on the shop floor, and the interview questions candidates report reflect both sides.
This guide is for software engineers preparing for the AMTEC (WI) Software Engineer loop, whether you come from ERP development, from manufacturing or embedded work, or from general backend work and need to bridge into the other two. It covers what candidates report about the process, question-specific answers for the five reported technical and domain questions, the vocabulary you need for Oracle EBS and CNC programs if you have not used one of them, a behavioral set built around the reported experience questions, and a seven-day plan that works each reported area in turn.
No round-by-round sequence has been reported for AMTEC (WI). Candidates describe screening conversations followed by deeper technical discussions with people from the team, virtual or in person depending on location and role, so confirm the exact format with your recruiter.
Focus areas
- Oracle E-Business Suite (Oracle EBS)
- Software Integration
- Safety Engineering
Overview
All five questions candidates report for this role are discussion questions about your own process and history: how you integrate disparate systems, your Oracle EBS or ERP experience, how you troubleshoot code that drives machinery, how you keep code within quality and safety standards, and a performance optimisation you carried out. None of them is a reported algorithm puzzle. Preparation therefore means turning your past projects into step-by-step accounts with specifics: the systems involved, the failure modes you handled, the checks you ran and the numbers you measured.
The topics candidates report being tested on are Oracle E-Business Suite, software integration, safety engineering, CNC programming and quality engineering. Few engineers have worked in all five. Pick the one or two where you have real depth and make those answers detailed, then build an honest bridge for the rest: a web or backend engineer can carry integration answers through APIs, idempotent loads and reconciliation; an ERP developer can carry them through interface tables and concurrent programs; someone from embedded or controls work can anchor the machinery and safety answers.
The requirements candidates describe list programming-language proficiency, a solid grasp of the software development life cycle and systems integration experience as must-haves, with manufacturing background, Oracle EBS or another ERP, and engineering certifications as nice-to-haves. Candidates also report version control, testing and documentation coming up as part of the integration and development area. Have one concrete example of each from your own work, because generic claims about writing tests or following process are easy to say and hard to believe without a project behind them.
Where to focus your preparation
Candidates have not reported a fixed round order or round names for AMTEC (WI), so this guide treats the loop as one preparation focus built on the five reported questions and the five reported topic areas.
Preparation focus
Candidates have not reported a round-by-round sequence for AMTEC (WI). What they do report is a progression from screening conversations to deeper technical discussions with people from the team you would join, held virtually or in person depending on location and role. The reported questions are discussion questions about your own process and history across five areas: integrating disparate systems, Oracle EBS or other ERP work, troubleshooting software that drives machinery, meeting quality and safety standards, and optimising performance. Treat the whole loop as one preparation focus. For each of the five areas, have a step-by-step process you can explain and one project of your own that shows it, with numbers where you measured something. Ask your recruiter early how many conversations there are, who you will meet and whether any of them involves writing code, and adjust the plan if a coding stage is confirmed.
What to demonstrate
- Depth in the reported topic areas: Oracle EBS, software integration, safety engineering, CNC programming and quality engineering
- Whether you can explain your own process for integrating, troubleshooting and verifying software step by step
- Concrete accounts of past projects, including what you measured before and after
How to prepare
- Ask the recruiter how many conversations there are, who you will meet, and whether any of them includes live coding
- Write a step-by-step answer to the integration, troubleshooting and quality-and-safety questions, each anchored in a project you worked on
- Prepare two STAR stories: your Oracle EBS or ERP work, and a performance optimisation with before and after numbers
- Learn enough G-code and Oracle EBS vocabulary to hold a conversation about whichever of the two you have not used
Where candidates lose points
Answering the integration question with "I connected the systems through an API" and nothing about failures
Name the system of record for each entity, how keys map between systems, what makes a rerun safe, where rejected records go, and how you reconcile the two sides. Prepare one diagram-level example from your own work that covers all five.
Claiming Oracle EBS depth you cannot back up, or describing it only as "I worked on EBS"
Name the modules, the objects you built and the tools you used, and have one problem you solved. If your ERP experience is elsewhere, say so and map it to EBS concepts such as interface tables and concurrent programs.
Starting the machinery troubleshooting answer by editing code and rerunning it on the live machine
Lead with putting the machine in a safe state and verifying in simulation or dry run, then localise the fault layer by layer. Safety engineering is one of the reported topics, so make the safety step explicit rather than implied.
Reducing the quality and safety question to "I write unit tests"
Describe layered verification, fail-safe defaults, change control with review and traceability, and post-release monitoring, then give one defect your process caught before production.
Telling the optimisation story without a before number, an after number or your own role
Write the story down with the baseline, the diagnostic tool and what it showed, the change you made, the trade-off, and the measured result. Cut any part where your contribution is not clear.
Questions & practice
Find overlapping job attempts and peak concurrency from lease records
Medium · sweep line · intervals · leases
A day of job_run history yields about 50,000,000 attempt records: (job_run_id, job_type, attempt, started_at, finished_at which is NULL when the worker died, lease_expires_at). Leases expire on a clock, so a job that outran its lease ran twice. Produce (a) every job_run_id whose attempts overlapped in wall-clock time and (b) the peak number of simultaneously running attempts per job_type with the minute it occurred. Target O(n log n). State how you treat a NULL finished_at and what clock skew does to your answer.
Approach
- Define the interval before sorting anything: an attempt occupies [started_at, COALESCE(finished_at, lease_expires_at)). finished_at is observed and lease_expires_at is only a promise, so every attempt without a finish contributes an estimate and the whole result is a lower bound on overlap rather than an exact count.
- For peak concurrency, sweep: emit 2n endpoints, sort by (timestamp, kind) with ends ordered before starts at equal timestamps, then walk the sequence maintaining a counter per job_type and record each type's maximum with its timestamp. O(n log n) dominated by the sort, O(n) space, or O(1) extra if the sort is external and the walk streams.
- For overlap detection, do not compare attempts pairwise. A single global sort by (job_run_id, started_at) gives both the grouping and the order; within a group, keep the maximum end seen so far and report an overlap exactly when the next start is less than that running maximum, which is one linear pass after the sort.
- Half-open intervals matter and are easy to get wrong: with closed intervals an attempt ending at the same millisecond another begins reads as concurrency two, and across 50,000,000 records that artefact swamps the real signal.
Follow-up
- A handler is not idempotent and you have found 400 overlapping jobs. Which of them actually caused damage, and what would you query to find out?
- Peak concurrency for one job_type is 4 against a configured cap of 4. Is the cap working, or is the data hiding attempts that never started?
Diff a projection against the primary without per-row point reads
Hard · reconciliation · range hashing · throttling
The listing projection has drifted and some rows show a stale version. The primary holds 40,000,000 resource rows across 12,000 tenants while serving 1,200 writes and 14,000 reads per second. The obvious repair, reading each resource row and comparing its version against the projection, is correct and would eventually finish. Explain precisely why it is unacceptable here, then give a diff that finds the differing rows, state its complexity, and make it safe to run against a live primary. Replication lag is usually under 100 ms and is not bounded.
Approach
- Quantify the naive cost rather than calling it slow: 40,000,000 point reads at even 0.5 ms each is over five hours serialised, and the only lever is concurrency, which is exactly what you cannot spend. The primary's pool is sized for the write path, and 40,000,000 random reads evict the buffer cache that sustains the 85 percent cache hit rate, so the audit degrades the system it is auditing.
- Replace random access with one ordered pass per side. Both sides can be read in (tenant_id, resource_id) order, which is a sequential scan on each and a merge join in O(n) time and O(1) memory. For a dense diff that is the whole answer, and it reads the primary once instead of 40,000,000 times.
- For the expected sparse case, compare range hashes instead of rows: partition the key space, compute per range an order-independent aggregate over hash(resource_id, version), compare aggregates, and descend only into ranges that differ. With d differing rows and branching factor B, at most d ranges mismatch per level, so the drill-down examines O(d log_B(n/d)) ranges and reads full rows only in mismatching leaves.
- Aggregate with a sum modulo 2^64 or a multiset hash, never XOR. XOR is order-independent but self-cancelling, so two rows wrong in the same way, or a row duplicated on one side, leave the range aggregate matching and the range is declared clean.
Follow-up
- The diff reports 900 stale rows. How do you decide between patching those rows and rebuilding the projection from resource_revision?
- Same job, but the projection lives in a search index that cannot be scanned in key order. What changes?
Canonicalise a request body into a stable idempotency fingerprint
Medium · parsing · canonicalisation · hashing
idempotency_key.request_fingerprint is a SHA-256 over the method, path and canonicalised body, and a retry whose fingerprint differs must be rejected with 422 rather than served the stored response. Write the canonicaliser. Bodies are JSON up to 256 KB nested at most 32 levels; clients vary key order, whitespace and unicode escaping, and some send 64-bit ids as JSON numbers. Produce a deterministic byte string such that semantically identical bodies match and any semantic difference does not. State your complexity and name two normalisations you refuse to perform.
Approach
- Parse once into a tree, then re-serialise under fixed rules: object keys sorted, array order preserved, one escaping convention, no insignificant whitespace. Parsing is O(n) and sorting keys is O(k log k) per object, so O(n log n) overall with O(depth) stack, and the 32-level cap is enforced during parsing because hostile nesting is how a canonicaliser becomes a stack overflow.
- Sort keys by their UTF-8 bytes and say why the obvious implementation is wrong in some runtimes: a default string comparison that orders by UTF-16 code units places surrogate pairs, meaning code points from U+10000 up, below U+E000 to U+FFFF, which is not UTF-8 byte order, so two services written in different languages disagree on the same document.
- Do not re-encode numbers through a double. IEEE-754 binary64 represents integers exactly only up to 2^53, so normalising a 19-digit id through a float changes it, and 1 against 1.0 cannot be reconciled without deciding whether they are the same value. Preserve the literal token, and require ids as strings at the API boundary if you want them comparable.
- Reject duplicate keys rather than picking one. JSON permits them and parsers disagree, most keeping the last, so any choice you make ties the fingerprint to a parser detail that the code handling the request does not necessarily share.
Follow-up
- A client sends the same logical request with an extra field your API ignores. Same key, different fingerprint, so you return 422. Is that the right answer?
- Where does the fingerprint get computed relative to request decompression and the body-size limit?
Keep soft-deleted accounts from blocking re-registration
Easy · soft delete · unique constraints · engine differences
app_user holds user_id, tenant_id, email CITEXT, password_hash (NULL for SSO principals), email_verified_at, auth_version, status ('invited','active','suspended','deactivated'), created_at, updated_at, deleted_at. Two live accounts for one address inside a tenant must be impossible, but an address freed by a soft delete must be reusable, and the same tenant may delete and re-register it repeatedly. Write the uniqueness DDL for PostgreSQL 16, then the equivalent for MySQL 8 where partial indexes do not exist, and say what each permits once three deleted rows already hold that address.
Approach
- Start from what is actually unique: not (tenant_id, email), but (tenant_id, email) among live rows. PostgreSQL says that directly — CREATE UNIQUE INDEX app_user_live_email ON app_user (tenant_id, email) WHERE deleted_at IS NULL. A full constraint over the same two columns burns the address permanently the first time someone deletes an account.
- Keep case-insensitivity in the type or the index, never in the application: CITEXT as given, or UNIQUE (tenant_id, lower(email)) as an expression index where the extension is unavailable. A case-sensitive unique column is exactly how two accounts for one human appear.
- For MySQL 8 the predicate has to move inside the key: add a discriminator column that is a constant 0 while the row is live and is set to user_id on delete, with UNIQUE (tenant_id, email, deleted_marker). Live rows share the constant and still collide; deleted rows differ from each other and stop colliding.
- State the NULL variant and its dependency: leaving the marker NULL for deleted rows also works, because a unique index treats NULLs as distinct — true in MySQL, and true in PostgreSQL only under the default NULLS DISTINCT, which PostgreSQL 15 lets you reverse. Check the polarity against the three existing deleted rows: constant-on-live is what preserves the collision you want, and reversing it silently admits duplicate live accounts.
Follow-up
- A deleted account re-registers with the same address the next day. Do the old resource rows follow the new user_id, and how does the API keep the two principals apart?
- How do you honour an erasure request while resource_revision.actor_user_id still references this table?
Denormalise tenant onto revisions and backfill it live
Hard · denormalisation · online migration · backfill
resource_revision (revision_id, resource_id, version, actor_user_id, change_kind, patch, request_id, created_at) has 400M rows and no tenant column; tenant_id lives only on resource. Two reads need it: a tenant-scoped audit feed ordered by created_at DESC, and an offboarding purge. Both join back to resource today. Justify adding tenant_id to resource_revision against those two reads, name the anomaly the copy introduces and the constraint that prevents it, then give the ordered migration for a live table taking 1.2k writes/second — the lock each step takes, how the backfill is batched, and where each step stops being reversible. PostgreSQL 16.
Approach
- Justify from the access path rather than from taste. Without the column, the audit feed either scans resource_revision by created_at and discards other tenants' rows, or resolves the tenant's resource_ids first and probes with them — both proportional to the tenant's whole history rather than to one page. With (tenant_id, created_at DESC, revision_id DESC) it is a seek that stops at 50 rows, and the purge becomes a ranged delete instead of a join.
- Name the cost exactly: a second copy of a fact can disagree with the first. Make the disagreement unwritable rather than documented — add UNIQUE (resource_id, tenant_id) on resource so it can serve as a foreign-key target, then FOREIGN KEY (resource_id, tenant_id) REFERENCES resource (resource_id, tenant_id) on the revision table. A revision can then only ever carry its parent's tenant.
- Step one, expand: ALTER TABLE resource_revision ADD COLUMN tenant_id BIGINT NULL, with no default, so it is a catalogue change and no rewrite. It still needs ACCESS EXCLUSIVE for an instant, and that instant queues behind the longest open transaction on the table while every later query queues behind it — set lock_timeout to 2s and retry rather than wait.
- Step two, dual-write: deploy the writer that populates tenant_id on every new revision while reads still use the join. Reversible by redeploying the previous build, because nothing reads the column yet.
Follow-up
- The backfill is half finished and a rollback is required. What state is the table in, and what does the previous build do with a half-populated column?
- How do you verify the backfill actually finished, given rows are still being inserted while it runs?
How do you approach the integration of disparate software systems within an enterprise environment?
Medium · Technical & Domain Proficiency
Approach
- Start with an inventory: for each shared entity (item, order, work order, quality result) name the system of record, who writes it and which systems only read it. Many integration bugs come from two systems both believing they own the same field.
- Choose the pattern per data flow and say why: scheduled batch or staging-table loads for high-volume data that can lag, synchronous API calls when the caller needs an answer immediately, and a message queue or event feed when producer and consumer must survive each other's downtime. For Oracle EBS inbound data, the standard route is the open interface tables plus the matching import concurrent program, or the public PL/SQL APIs, never direct inserts into base tables.
- Make mappings explicit: a cross-reference table between each system's keys, unit and time-zone conversions, and a validation step that rejects bad rows into an error table with a reason instead of half-loading a record.
- Make every load idempotent and observable: a stable key so a rerun cannot create duplicates, retries with backoff, an error queue someone actually monitors, and a reconciliation report that compares counts and totals on both sides.
- Name the trade-off you made: real-time feeds cost more to run and debug, while batch is simpler but leaves the ERP or the shop floor working on stale data between runs.
Follow-up
- The receiving system is down for several hours. What happens to the records produced meanwhile, and how do you replay them without duplicates?
- How would you detect that two integrated systems have silently drifted apart?
- One side changes its record layout. How do you roll that out without breaking the other?
How do you ensure your code meets strict quality and safety standards?
Medium · Technical & Domain Proficiency
Approach
- Describe layers, each catching what the previous one misses: requirements written so they can be tested, peer review, static analysis with warnings treated as errors, unit and integration tests, then simulation or hardware tests for anything that commands equipment.
- Design for failure: validate inputs against physical limits, make the default on any error the safe state (stop, hold, reject) rather than carrying on, and treat hardware interlocks as the primary safety layer so software is never the only barrier.
- Control change: version control for everything, including machine programs and configuration; review and sign-off before release; traceability from requirement to test to release; and a tested rollback path.
- Close the loop after release with monitoring, logs and a defect process that records root cause, and bring quality engineers into review of anything that affects product or process quality.
- Make it concrete with one example where this process caught a defect before production, and what it would have cost if it had shipped.
Follow-up
- A release date is close and one test fails intermittently. What do you do?
- How do you decide how much testing a given change needs?
- Who signs off a change to a machine program, and what evidence do they see?
Explain a time you had to optimize a piece of software to improve system performance.
Medium · Technical & Domain Proficiency
Approach
- Situation and baseline in numbers: what was slow, who it hurt, and the measured value before the change (a report's run time, a job's throughput, an endpoint's p95 latency). Without a baseline the story cannot show a result.
- Show the diagnosis rather than a guess: the profiler, execution plan, SQL trace or timing logs you used and what they revealed, such as an N+1 query pattern, a full scan caused by a missing or unusable index, row-by-row PL/SQL loops that BULK COLLECT and FORALL would batch, or repeated work a cache removes.
- Action: the change you personally made, the alternative you rejected, and the trade-off you accepted (memory for speed, index maintenance cost on writes, staleness from caching).
- Result: the after number measured the same way as the baseline, how you confirmed the output was unchanged on the same input, and what you added to stop a regression.
- What sinks this answer: a team story where your own part is unclear, or a claimed speedup with no measurement behind it.
Follow-up
- How did you confirm the results were identical after the change?
- What would you do if the gain disappeared once production data volumes grew?
- What became the bottleneck after yours was fixed?
What is your process for troubleshooting code that interacts with complex machinery or physical interfaces?
Medium · Technical & Domain Proficiency
Approach
- Open with safety: the machine is put in a safe state, someone qualified on the equipment is involved, and nothing is tried first on a live part or with people inside the work envelope. Changes are checked in simulation, dry run or single-block mode before production.
- Localise by layer, choosing checks that rule out a whole layer each: the program logic, the data handed to the machine (for CNC, the generated G-code, units and offsets), the communication link and handshake, or the machine's own state (alarms, sensors, tool wear).
- Collect evidence from both sides with timestamps: your application logs, the controller's alarm history, and the exact program or message that was sent. Diff the failing program against the last known-good version before forming a theory.
- Name the usual culprits: inch versus metric mode (G20/G21), absolute versus incremental mode (G90/G91) left in the wrong state, a wrong work offset (G54 and its siblings), timing races in the handshake, and intermittent wiring or sensor noise that looks like a software bug.
- Finish with the fix and the guard: a simulation or regression check that would have caught it, and a note to the operators and quality engineers about what changed.
Follow-up
- The fault happens on one of three machines running the same program. Where do you look first?
- How do you test a change when the machine cannot be taken out of production?
- The failure is intermittent and leaves no alarm. What do you instrument?
Suggested practice checklist
Each day works one reported question area in turn (integration, ERP, machinery troubleshooting, quality and safety, performance) and ends with a full rehearsal; library exercises are attached where they train the same skill.
Day 1 — Confirm the format and map your background
- Email your recruiter asking how many conversations there are, who you will meet and whether any of them involves writing code.
- For each reported topic (Oracle EBS, software integration, safety engineering, CNC programming, quality engineering), write one line of evidence from your own work, or mark it as a gap.
- Choose the two projects you will draw on most, and list the systems, your role and one measured outcome for each.
Deliverable: A sent format question to the recruiter, a five-row evidence map with gaps marked, and two project summaries.
Day 2 — Integrating disparate systems
- Sketch one integration you built: systems of record, transport (batch, API or queue), key mapping, error handling and reconciliation.
- Implement the idempotency fingerprint exercise and note how the same idea prevents duplicate records when a load is rerun.
- Answer the integration question aloud, then answer its three follow-ups without notes.
Deliverable: An integration sketch from your own work, a working fingerprint solution, and a recorded spoken answer to the integration question.
Day 3 — Oracle EBS and ERP development
- List every ERP object you have built (reports, interfaces, conversions, extensions, personalisations, workflows) and pick one to walk end to end.
- Review the open-interface-table and import-concurrent-program flow, and how customisations are kept safe through patches; if EBS is new to you, map each idea to the ERP you know.
- Work the live tenant backfill exercise as practice for the data side of legacy migrations.
Deliverable: An ERP object list, a one-object walkthrough, a one-page EBS vocabulary sheet, and a completed backfill solution.
Day 4 — Troubleshooting software that drives machinery
- Learn the G-code basics you would need to discuss a faulty program: G00/G01 moves, G20/G21 units, G90/G91 modes and G54-style work offsets.
- Write your layer-by-layer troubleshooting sequence, starting with putting the machine in a safe state and verifying in simulation.
- Prepare one real debugging story involving hardware, devices or an external system, and tell it aloud.
Deliverable: A G-code cheat sheet, a written troubleshooting sequence, and one rehearsed debugging story.
Day 5 — Quality and safety standards
- Write out your verification layers from requirement to release, with the tool or practice you actually used at each layer.
- List three fail-safe design choices you have made or would make in code that commands equipment.
- Rehearse the outage narrative exercise, focusing on the follow-up that prevented a repeat.
Deliverable: A written verification pipeline, three fail-safe examples, and a rehearsed incident narrative.
Day 6 — Performance optimisation story
- Write the STAR story with a baseline number, the diagnostic tool and what it showed, your change, the trade-off and the after number.
- Review the techniques your story touches: reading an execution plan, indexing trade-offs, batching row-by-row loops, caching and staleness.
- Tell the story aloud twice, cutting anything where your own role is unclear.
Deliverable: A written STAR story with before and after numbers and a recording of the final version.
Day 7 — Full rehearsal
- Run all five reported questions aloud in one sitting, plus the follow-ups listed under each.
- Rehearse the exercise on telling callers you do not own that their integration breaks, as a cross-team behavioral answer.
- Write two questions for the team, for example about their current hardest technical problem and how software changes reach the shop floor.
Deliverable: A full recorded mock of the five questions, one cross-team story, and two questions for your interviewers.
Behavioural preparation
The reported experience questions for this role ask about your own history: the ERP work you have done and a time you improved performance. Use STAR, keep the situation short, and spend most of the answer on what you personally decided and measured. Because the role description spans legacy migrations, CNC scripts and Oracle EBS, also prepare a story about learning an unfamiliar tool or domain quickly.
- Explain a time you had to optimize a piece of software to improve system performance.
- Tell me about a time you had to learn a new tool or domain quickly to deliver on a project.
- Describe a change you shipped where a mistake could have affected a physical process or product quality. How did you verify it before release?
- Tell me about a time a schedule and a quality concern pulled in different directions, and how you worked it out with the people involved.
- Walk me through a legacy system migration you worked on: what you moved, how you validated it and what went wrong.
Can you describe your experience with Oracle EBS or similar ERP development frameworks?
Medium · Technical & Domain Proficiency
Approach
- Give the scope in one breath: which EBS release and modules you worked in (for example Inventory, Purchasing, Order Management, General Ledger) and which kinds of objects you built: reports, interfaces, conversions, extensions, form personalisations, workflows.
- Walk one object end to end: the business need, the tools (PL/SQL packages, concurrent programs, BI Publisher templates, Forms or OA Framework personalisation), how you tested it and how it moved through instances to production.
- Show you know the upgrade-safety rules: custom code lives in its own custom application and schema, seeded objects are not edited, and changes go through personalisation or extension so patches do not overwrite them.
- Bring one problem you solved, such as a failed concurrent request traced through its log and output files, rows stuck in an interface error table, or a slow report you tuned.
- If your ERP work is on another platform, say so plainly and map it: staging-table interfaces, configuration versus customisation and period-end data volumes carry over. Name what you would learn first on EBS rather than overstating experience.
Follow-up
- How did your customisations survive a patch or an upgrade?
- A data conversion loads tens of thousands of items and a few hundred fail. How do you find, fix and reload only those?
- When did you choose configuration over custom code, and why?
Tell callers you do not own that their integration breaks
Medium · deprecation · compatibility · stakeholders
A field in a write endpoint's response must change shape. You own the endpoint; you do not own the four internal callers or the outbound webhook consumers who read it. Describe a deprecation you were responsible for: what you shipped first, how you established who was actually reading the field, the window you gave and what set its length, what you did about the consumer who never moved, and how you decided removal was safe. Name the signal you used, not the announcement you sent.
Approach
- Establish the reader set empirically rather than from a wiki of owners: per-field usage counters keyed by principal, or access logs attributed to a consumer. State the blind spot of whichever you pick, since a consumer that reads the field only on a monthly job will not appear in a week of logs.
- Ship additive first. Populate the new field alongside the old one so no reader is forced to move, which is also what keeps a rolling deploy safe, because old and new instances answer the same requests at the same time and a rollback must still find the old shape present.
- Set the window from the slowest legitimate consumer's release cadence, not from your calendar, and decide separately what to do for a consumer with no release process at all, such as an external webhook endpoint you can only email.
- Convert silence into evidence before you rely on it: a short, low-traffic removal window that makes a still-dependent consumer fail visibly and loudly while you are watching, rather than at three in the morning after you have moved on.
Follow-up
- How would you detect a consumer that reads the field only during a monthly export?
- One caller refuses to move and has a commercial relationship behind it. What changes in your plan and what does not?
Narrate an outage you owned from page to postmortem
Hard · incident response · blast radius · postmortems
Pick an incident you personally drove, ideally one where writes were affected rather than reads. In six to eight minutes: state the symptom as it first appeared on a dashboard, the blast radius you established before you knew the cause, the mitigation you applied and when, the mechanism you eventually proved, and the follow-up that would prevent a repeat. Bring numbers: error rate, tenants affected, minutes to mitigate, minutes to resolve. If you cannot name what you measured, choose a different incident.
Approach
- Open on the signal rather than the cause: which metric at which percentile moved, on which service, at what time, so the listener follows the same evidence you had rather than a conclusion you already reached.
- Separate mitigation from diagnosis out loud. State what you did to stop the bleeding (flag off, shed traffic, drain a lease, roll back a deploy) and say plainly that you did it before the mechanism was known, because those are two jobs with different deadlines.
- Establish blast radius in countable terms: how many tenants, how many writes, and crucially whether the effect was loss or only delay. An append-only revision table or a pending outbox row means the change survived and the projection was merely behind, which is a repair rather than a data-loss incident.
- Prove the mechanism instead of asserting it. Name the trace span that grew, the plan that flipped to a sequential scan, the lease that expired, plus one alternative you ruled out and the signal that stayed flat while you ruled it out.
Follow-up
- What would you do differently in the first five minutes, given the same dashboard and no more information?
- Which follow-up action did you deliberately not take, and why was dropping it the right call?