Interview guides · F5 Networks

F5 Networks Data Scientist Interview Questions & Guide 2026

Data scientist prep for F5 Networks: practice defining a B2B software and infrastructure metric, picking its guardrails and defending the tradeoff.

The 60-second brief

The role of a Data Scientist at F5 Networks is pivotal for driving data-informed decision-making and enhancing the value of products through advanced analytics. As a Data Scientist, you will engage with complex datasets to extract actionable insights that influence product development, optimize user experiences, and inform strategic business initiatives. This position is fundamental in fostering a data-driven culture within the organization, ensuring that insights derived from data are seamlessly integrated into the product lifecycle.

SQL is seldom the hardest round and is often the one that eliminates people. The working bar is usually window functions, correct deduplication, and joins that do not silently fan out rows, rather than obscure syntax.

Focus areas

Measure churn only on renewal-eligible accounts
Strip CI, retry and synthetic traffic first
Separate contracted seats from actively used seats

Overview

At F5 Networks, you will contribute to various teams focused on improving application delivery and security. Your work will not only enhance the performance of existing products but will also help in developing new features that meet the evolving needs of users. The complexity and scale of the data you will handle, combined with the strategic influence you will have, make this role both challenging and rewarding. You can expect to work in a collaborative environment that values innovation and creativity, allowing your insights to directly impact user satisfaction and business outcomes.

The F5 Networks interview process

F5 Networks candidates report 6 rounds · ≈ 4-6 weeks. The stages below are what candidates describe, not a published process.

Recruiter Call

Data Scientist covers at least four different jobs: experimentation, product analytics, causal work on observational data, and applied modelling that ships into a system. A screening call is the cheapest place to find out which of them is being hired for, and doing that diagnosis openly reads as senior rather than fussy. Ask what the last few pieces of work on the team actually were, and roughly how a week splits between querying, modelling and stakeholder time. Then say which parts of that you have done and which you have not. Claiming the whole range is the fastest way to be caught one round later.

What to demonstrate

Whether you can distinguish the flavours of the role and locate your own experience inside one of them honestly
Whether you name what you have not done instead of stretching to cover every line of the posting
Whether your hard constraints (notice period, location, work authorisation, level) surface now rather than at offer stage

How to prepare

Map the last two years of your time into rough percentages across query writing, experiment design, modelling and stakeholder work, so a question about scope has a real answer
Mark every responsibility in the posting as done, adjacent or new, and prepare one sentence for each adjacent item naming the closest thing you have actually built
Decide which logistics are non-negotiable before the call so you can state them in one sentence rather than negotiating live

Interviews with Hiring Manager

This conversation decides whether you can be handed a problem nobody has finished defining and left alone with it for a few weeks. The manager is listening for how you behave when the brief is thin: what you clarify before starting, and what you settle on your own rather than escalating. Most candidates over-index on technical depth here and under-describe the decisions they actually owned. Say who wanted the work, what you chose not to do, and where you would have stopped and asked. A clean account of your own judgement carries this round further than a longer project list.

What to demonstrate

Whether you can name a decision that was yours alone, as opposed to one the team arrived at
How you respond to a request that arrives with no success metric attached to it
Whether the effort you estimate for a piece of work matches the work you just described doing
What you escalate, and how long you sit on a problem before you do

How to prepare

For each project you plan to raise, write one sentence saying what would not have happened if you had not been on it, and check that the sentence is about an outcome rather than an artefact
List the decisions in your last project that were genuinely yours, and for each one write down the option you rejected and why
Prepare the project that went badly: the point at which you knew, who you told, and what it cost before it was caught

Technical Team Interviews

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Take-home Assessment

A take-home is graded as an argument, not as a notebook. Somebody reads the submission without you in the room, so every choice has to survive on the page: why the question was framed this way, and what was deliberately left out. The gap between a strong and a weak submission is almost never model quality. It is whether the writeup names the specific question it answers and commits to a recommendation, including what evidence would overturn it. A high-accuracy model attached to no conclusion reads as effort that stopped before the decision.

What to demonstrate

Whether the question you answered is stated outright, and whether it is the question the prompt posed rather than an easier neighbour of it
Whether the recommendation is specific enough to act on, with the uncertainty attached to it instead of parked in a caveats section at the end
Whether analytical choices such as the metric definition, the population filter and the time window are justified in the prose, not merely visible in code

How to prepare

Take a dataset you have already worked with, write the one-paragraph conclusion first, then check whether the analysis you were planning actually supports it and cut whatever does not
Practise stating a metric in one sentence that fixes the population, the time window and the denominator, then confirm your query computes exactly that sentence and nothing adjacent to it
Hand a draft to someone outside the problem and ask them to tell you back what you recommended and why; anything they cannot recover is not on the page yet

Presentations

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Interviews with Senior Leadership

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Where candidates lose points

Treating raw request or usage volume as engagement

Most traffic in this domain is emitted by machines. Continuous-integration pipelines, scheduled batch jobs, synthetic monitors, backfills and client retries can all grow by an order of magnitude from one configuration change made by one engineer, and none of it represents a new decision to use the product. The inversion is what makes it dangerous: when the platform degrades, clients retry, so error-driven retry volume rises at the exact moment the customer is most likely to leave, and an engagement dashboard built on raw counts shows growth immediately before a churn. Filter on traffic_class and on successful status before anything else, and keep failed-request volume as its own separate series.

Reading consumption metrics before the metering lag window has closed

Usage pipelines land late and correct themselves, which is exactly what is_restated and restated_at record. A dashboard queried on day T sees a partially populated tail for the last several days, so the most recent points always slope downward and always look like a regression. Analysts then explain the artefact, and sometimes ship a change to fix it. Establish the empirical settling time by measuring how much a given usage_date's total moves between first_written_at and its final value, exclude that many trailing days from every reportable figure, and never compare a fresh period against a settled one.

Reading an observational correlation as a causal effect

Name the confounder you are most worried about and the design that would remove it: an experiment, a difference-in-differences with a checked pre-period trend, an instrument, or a regression discontinuity. When none is available, state which direction the bias likely runs and bound the claim accordingly.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Questions & practice

Write a function to calculate the Pearson correlation coefficient.

Medium · statistics and probability

Approach

Quantify uncertainty explicitly rather than reporting a point estimate alone.
Say what the estimate is of, and over what population it generalises.
Sanity-check the answer against a simple bound or a simulated case.

Follow-up

Which assumption here is most likely to be violated in practice?
What sample size would you need to detect an effect half this size?

Describe a project where you used machine learning techniques. What was your approach?

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Check what information would not exist at prediction time, and exclude it.
Set a baseline first, so any model has something honest to beat.

Follow-up

How would you choose the decision threshold, and who owns that choice?
What would you monitor after launch to know the model is still valid?

Join daily usage to the contract version live that day

Medium · as-of-join · pandas · numpy · contracts

fct_usage_daily has account_id, usage_date and net_amount_cents. fct_subscription_period has account_id, subscription_period_id, plan_tier, term_start_date, term_end_date, arr_cents and is_current, with one row per contract term version and every amendment inserting a new row. Attach to each usage row the subscription_period_id whose term brackets usage_date (term_start_date <= usage_date <= term_end_date). pd.merge_asof and an interval-condition merge are unavailable; use sorting and numpy.searchsorted. Then report monthly net revenue by plan_tier. Terms for one account do not overlap, and usage may fall outside every term.

Approach

Say out loud why the cheap version is wrong: joining on is_current stamps today's plan tier onto last year's usage, so every account that upgraded has its history reclassified and revenue-by-tier becomes a function of when the query ran.
Sort each account's terms by term_start_date and use np.searchsorted(term_start, usage_date, side='right') - 1 to get the last term that started on or before the usage date. Do it per account group, or globally after encoding (account_id, date) into one monotone key.
searchsorted only enforces the left edge. Validate the right edge afterwards — usage_date <= the candidate's term_end_date — and set the match to NA where it fails. That NA is usage in a gap between contracts and must stay visible instead of being folded into the expired term.
Assert non-overlap before trusting the lookup, and write the assertion so it is capable of passing. prev_end = terms.groupby('account_id').term_end_date.shift() is NaT on each account's first row, and NaT < Timestamp evaluates to False rather than NA, so a comparison followed by .fillna(True) has nothing left to fill and the assertion fires on every account's first term whatever the data looks like. Guard the null yourself: assert (prev_end.isna() | (prev_end < terms.term_start_date)).all(). The failure mode of getting this wrong is not a false alarm you notice once — it is an assertion someone deletes because it never passes, after which overlapping terms make searchsorted return one of them with no trace in the output.
Aggregate after the join, grouping by (usage_date month, plan_tier) with dropna=False so the unmatched bucket appears as its own row and the total still ties to the ungrouped sum of net_amount_cents.

Worked solution (35 min)

terms = terms.sort_values(['account_id','term_start_date']); prev_end = terms.groupby('account_id').term_end_date.shift(); assert (prev_end.isna() | (prev_end < terms.term_start_date)).all()
Per account group: idx = np.searchsorted(g.term_start_date.values, u.usage_date.values, side='right') - 1; rows with idx < 0 are unmatched.
Gather subscription_period_id, plan_tier and term_end_date by positional index, then null the match wherever usage_date > the gathered term_end_date.
monthly = joined.assign(month=joined.usage_date.dt.to_period('M')).groupby(['month','plan_tier'], dropna=False).net_amount_cents.sum()

Expected result: Exactly one output row per input usage row, a subscription_period_id that is NA precisely for usage outside every term, and a monthly table whose values sum to the ungrouped total of net_amount_cents once the NA-tier bucket is included.

Verify it yourself

len(joined) == len(usage); the join must not fan out.
joined.net_amount_cents.sum() == usage.net_amount_cents.sum().
For every matched row, assert term_start_date <= usage_date <= term_end_date directly rather than trusting searchsorted.
Run the non-overlap assertion on a frame where every account has exactly one term: prev_end is NaT throughout, so the assertion must pass. An assertion that fires there is testing the shift, not the data.
Spot-check one account that upgraded mid-term: its usage before the amendment must still carry the old plan_tier.

Follow-up

An amendment takes effect on the 17th of a month. How do you report that month's revenue by tier?
What changes if terms can overlap because of a co-term amendment?
How would you verify this against a SQL implementation using a BETWEEN condition?

Write a code snippet to clean and preprocess a dataset.

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

How do you optimize SQL queries for performance?

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Net revenue retention on a cohort frozen twelve months back

Hard · retention · as-of join · cohort arithmetic

fct_subscription_period carries subscription_period_id, account_id, arr_cents, plan_code, term_start_date, term_end_date, booked_at, amendment_type, superseded_by_id (the subscription_period_id of the version that replaced this one, null on the live version of a lineage) and is_current. Compute net revenue retention for month M: the summed arr_cents at M for the set of accounts holding arr_cents > 0 at M-12, divided by that same set's arr_cents at M-12. An account can hold more than one live subscription, a churned account contributes zero rather than dropping out, and nothing signed after M-12 may enter either side. Return the ratio plus the expansion, contraction and churn components in cents.

Approach

Write one reusable as-of ARR snapshot parameterised by a date: rows whose term brackets the date AND whose booked_at is at or before the date, then only the version of each lineage that is still live at that date, then sum arr_cents per account. The booked_at guard matters because an amendment signed in advance otherwise co-exists with the term it replaces and double counts the account.
Collapse the lineage on superseded_by_id, not on any attribute of the contract. Keep a row when superseded_by_id IS NULL, or when the successor it points at was booked after the date. Ranking with ROW_NUMBER() OVER (PARTITION BY account_id, plan_code ...) instead is wrong in both directions: an amendment that moves the account from one plan_code to another puts the old and new versions in different partitions, so both are rank 1, both bracket the date, and the account's arr_cents is counted twice; and two genuinely concurrent subscriptions that happen to share a plan_code land in one partition, so one of them is deleted.
Resolve the successor with a LEFT JOIN back to fct_subscription_period on subscription_period_id, and treat a missing successor as not superseded. An inner join would silently delete an account's ARR on a dangling pointer, which is a data-quality bug in the source, not a retention movement.
Never use is_current for the M-12 side. is_current describes today; using it at the historical snapshot backdates the present contract onto last year's cohort and makes retention look like 100 percent by construction.
Freeze the cohort from the M-12 snapshot where arr_cents > 0, then LEFT JOIN the M snapshot onto it and COALESCE the missing side to zero. An inner join deletes exactly the churned accounts, which is the single largest way this number gets overstated.
Return a ratio of sums, not a mean of per-account ratios. The two are different estimands: contraction is floored at zero while expansion is unbounded, so the mean of ratios is both biased relative to the aggregate and far noisier on a skewed revenue base.
Decompose per account on the delta: positive delta is expansion, negative delta with a non-zero M value is contraction, a zero M value is churn. The three components must reconcile to numerator minus denominator.
Prove no leakage: any account whose first contract began after M-12 must be absent from both sides, and the cohort row count must be identical in the numerator and denominator.

Worked solution (45 min)

Write arr_asof(d) as a CTE or lateral: from fct_subscription_period s take rows with term_start_date <= d AND term_end_date >= d AND booked_at <= d, LEFT JOIN fct_subscription_period succ ON succ.subscription_period_id = s.superseded_by_id, keep the row when s.superseded_by_id IS NULL OR succ.subscription_period_id IS NULL OR succ.booked_at > d, then sum arr_cents per account_id across every surviving version.
Sanity-check the lineage rule on one amended account before going further: at a date after the amendment, the account must contribute exactly one version per lineage even when the amendment changed plan_code, term dates or both.
Materialise base = arr_asof(M-12) filtered to arr_cents > 0, and curr = arr_asof(M).
LEFT JOIN curr onto base on account_id and COALESCE(curr.arr_cents, 0) AS arr_now.
Compute nrr = sum(arr_now)::numeric / NULLIF(sum(base.arr_cents), 0), and the three components with SUM(...) FILTER on the sign of arr_now - base.arr_cents and on arr_now = 0.
Reconcile: assert sum(arr_now) - sum(base.arr_cents) = expansion - contraction - churn, and assert the cohort account count is identical on both sides.

Expected result: A single row with nrr as a ratio, plus expansion_cents, contraction_cents and churn_cents as positive magnitudes satisfying numerator minus denominator equals expansion minus contraction minus churn, over a cohort whose membership is fixed at M-12.

Verify it yourself

At each snapshot date, the surviving row count per account equals the number of distinct live subscription lineages on that account, not the number of contract versions bracketing the date.
For an account whose amendment changed plan_code, the snapshot returns one version and its snapshot ARR equals the amended arr_cents; swapping in a ROW_NUMBER partition on (account_id, plan_code) returns two versions for that account and inflates the side it sits on, which is the fastest way to see the bug.
The three components reconcile to the numerator-denominator delta to the cent.
The distinct account count is the same in the numerator and the denominator.
No account with a first term_start_date after M-12 appears anywhere in the output.
Re-running after new accounts sign leaves the ratio unchanged, which is the defining property of a frozen cohort.

Follow-up

Net revenue retention can rise while the business shrinks. Show one mechanism and name the guardrail that catches it.
How do you handle an account that co-terms two subscriptions into one mid-window, so the subscription count changes but the money does not?
Finance computes this from invoiced amounts and gets a different number. Which is right for which question?

How do you prioritize tasks when working on multiple projects?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

What steps would you take to validate your findings?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Given a dataset, how would you identify key trends?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Describe a case where you had to make a decision based on incomplete data.

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

How would you design an A/B test for a new feature?

Medium · a/b testing and experimentation

Approach

Say whether units interfere with each other, and switch design if they do.
Decide the analysis before seeing data, including how long it runs and when you look.
Name the randomisation unit first; it decides the variance and what the test can detect.

Follow-up

What would you conclude if the result is positive but the test is underpowered?
What would you do if you could not randomise at all?

Explain the difference between supervised and unsupervised learning.

Medium

Approach

Say what you would check first and why it is the highest-information step.
State your assumptions explicitly before working the problem.
Clarify what is being asked and what a complete answer would contain.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

Define success for widening the free usage allowance

Easy · north star · guardrails · gameable metrics · unit economics

A proposal triples the free-tier included allowance on the requests_thousands SKU. Finance expects the count of weekly active organisations to rise. You have fct_usage_daily (billable_quantity, included_quantity_applied, overage_quantity, net_amount_cents, cogs_cents, is_restated), dim_account (account_status, is_internal, converted_to_paid_at) and fct_api_request (traffic_class, http_status, environment). Define the primary metric this change should be judged on, two guardrails, and the numeric decision rule you would agree before launch. State explicitly how the obvious metric can be satisfied without the product getting better for anyone.

Approach

Name the mechanism before naming any metric. A larger allowance moves quantity out of overage_quantity into included_quantity_applied, so net_amount_cents falls while cogs_cents is unchanged, and accounts that previously sat under the production volume floor now cross it without paying anything. A count of active accounts therefore moves for reasons that have nothing to do with the product.
Set the primary metric at the account grain: weekly active organisations with at least one successful request (http_status < 400) in a workspace where environment = 'production' and traffic_class in ('interactive','batch'), over accounts with is_internal = false. Report accounts that existed before launch separately from the post-launch signup cohort, so the number is not quietly an acquisition metric.
Attach the two guardrails that close the gaming routes. First, gross margin per account, (sum(net_amount_cents) - sum(cogs_cents)) / sum(net_amount_cents) computed per account per month and reported as a distribution, specifically the count and revenue share of margin-negative accounts. Second, the free and trial share of weekly active organisations.
Add the conversion link the change is supposed to buy: free-to-paid conversion within 90 days for the affected cohort, read from converted_to_paid_at. Without it, the primary metric rewards permanent free use indefinitely.
Write the decision rule with numbers before launch: ship if weekly active organisations rise by at least the pre-agreed threshold, 90-day free-to-paid conversion does not fall, and the count of margin-negative accounts stays under a stated cap. Note that the margin guardrail can only be read after the metering settling window has closed, since fct_usage_daily rows restate in place.

Worked solution (20 min)

Compute per-account monthly gross margin from fct_usage_daily as (sum(net_amount_cents) - sum(cogs_cents)) / sum(net_amount_cents), grouped by account_id and calendar month, restricted to usage_date values older than the measured metering settling window.
Simulate the allowance change on the trailing settled month: for each account, move min(overage_quantity, allowance_delta) from overage_quantity into included_quantity_applied and recompute net_amount_cents at that account's existing list and discount rates, leaving cogs_cents untouched.
Recount weekly active organisations under the simulated state with the production volume floor applied, and split the delta into accounts that were already active and accounts newly crossing the floor.
Report the before and after margin distribution, the count of accounts crossing into negative margin, and the revenue share those accounts hold.

Expected result: A two-part readout: the simulated change in weekly active organisations split by whether the account was already active, and the change in the count and revenue share of margin-negative accounts. The answer names the primary metric (weekly active organisations on qualifying production workloads), two guardrails (the per-account margin distribution and the free/trial share of actives), a pre-agreed numeric rule, and states plainly that the primary metric alone can be satisfied entirely by accounts that will never pay.

Verify it yourself

Simulated net_amount_cents can only fall or stay flat for every account; any account whose simulated revenue rises means the allowance was applied in the wrong direction.
cogs_cents must be byte-identical before and after, because an allowance changes what is charged, not what is served.
The sum of per-account margins must reconcile to the aggregate margin computed directly from fct_usage_daily totals over the same accounts and months.

Follow-up

Weekly active organisations jump 14% in the first week after launch. How much of that can you attribute to the change with no control group, and what would you need to do better?
An account settles at usage permanently just below the new allowance and never pays. Does your primary metric count it as a success, and should it?

Billable units per account jumped while nothing shipped

Medium · retries · idempotency · guardrail metrics

Billable units per paying account rose 22 percent month over month with no pricing or packaging change. From fct_api_request (account_id, endpoint, http_status, is_retry, idempotency_key, traffic_class, billable_units, request_at) and fct_usage_daily (account_id, sku_code, usage_date, billable_quantity), determine how much of the rise is delivered value and how much is duplicated work. Deliverable: the decomposed figure, the accounts it concentrates in, and a recommendation on whether to report the 22 percent at all.

Approach

Check the guardrail before the headline: compute the customer-visible server error rate per account for both months, numerator http_status >= 500 and denominator excluding synthetic_monitor and load_test. Metered volume rising alongside an error rate is the known failure mode in this domain.
Deduplicate logical work by counting billable_units once per (account_id, idempotency_key) at the first successful request rather than once per row. Rows with a null idempotency_key cannot be deduplicated, so report their share as an explicit uncertainty band instead of assuming they are all unique.
Split by traffic_class before interpreting anything, because one change to a continuous-integration configuration can multiply request volume overnight without a human deciding anything about the product.
Test concentration: compute the per-account distribution of the increase and its top-decile share. A rise carried by a few accounts scaling one batch job is a different finding from a broad shift and gets a different recommendation.
Reconcile against fct_usage_daily for the same accounts and dates, and be ready to explain the expected gap in two sentences: request rows include retries and failures carrying zero billable_units, and the usage table restates after first write.

Follow-up

A client retrying a request the server already completed produces duplicate billed work. What protocol or product change removes that, and what would you measure to confirm it worked?
If the duplicated volume was genuinely invoiced, what should finance do, and how does that change what belongs in the metric?

Suggested practice checklist

Instead of guessing where the week should go, day one measures it under a fixed rubric and allocates the remaining hours in proportion to the gaps. The method is deliberately rigid: the allocation is written down before any studying starts and is not renegotiated when a topic turns out to be unpleasant.

Day 1 — Diagnostic, scored before you study anything

Sit a 100-minute timed diagnostic in four blocks: 30 minutes of SQL across three prompts, 25 minutes of short-answer statistics, 25 minutes on one modelling or case prompt, and 20 minutes delivering one behavioural story aloud.
Score each block from 0 to 3 on a fixed rubric where 3 is correct and fluent, 2 is correct but slow or prompted, 1 is partially correct, and 0 is stuck, grading the output rather than how the attempt felt.
Allocate the hours for days two to five roughly in proportion to 3 minus the score in each block, write the allocation down, and commit to not revising it midweek.

Deliverable: A scored rubric and a fixed hour allocation for the rest of the week.

Day 2 — Largest gap: find the boundary rather than the subject

Break the weakest area into five named sub-skills (for query work: grain control, window frames, date arithmetic, set logic with NULLs, and reading a query plan) and rate each one, so the rest of the week targets a sub-skill instead of a subject.
Solve three problems chosen to sit just above where the rating drops off, and for each write the first move you failed to make.
Re-solve one of them from memory four hours later, on paper, with nothing open.

Deliverable: A five-item sub-skill map with the two blocking sub-skills circled.

Day 3 — Largest gap: drill the blocking sub-skill

Do eight short repetitions of the same shape rather than eight different problems, so what you practise is the pattern and not the puzzle.
Write the rule you now hold in one sentence, then test it against a case built to break it: a ranking function over a column with ties, or a two-sample test on observations that are obviously dependent.
Have someone else read your one-sentence rule and find the precondition you left out.

Deliverable: One rule statement with its preconditions attached and one counterexample that would have caught the incomplete version.

Day 4 — Second gap, plus maintenance on your strongest area

Run the same sub-skill map and boundary protocol on the second-largest gap, compressed into half the day.
Spend 25 timed minutes on your strongest area to stop it decaying, choosing the hardest problem you can still finish rather than an easy warm-up.
Compare how the two areas fail: whether you lose time on recall, on setup, or on arithmetic, because the fix differs for each.

Deliverable: A second sub-skill map plus a one-line diagnosis of how each area fails you.

Day 5 — The gap that is not a skill

Record yourself answering one technical and one behavioural prompt, then count two things in the playback: how many seconds before your first clarifying question, and how many sentences you started without knowing where they ended.
Rewrite your three most-used stock phrases into shorter versions, and practise saying "I do not know, here is how I would find out" without softening it into a guess.
Deliver one answer again with a hard 90-second limit to force structure before detail.

Deliverable: Two recordings with a counted improvement in time-to-first-question.

Day 6 — Retest under day-one conditions

Sit the same 100-minute diagnostic structure with new prompts of comparable difficulty and score it on the identical rubric.
Compare block by block, and for any block that did not move, change the method rather than adding hours: a block stuck at 1 usually means the practice was too varied, not too short.
Write which single block you would still lose the offer on.

Deliverable: A second scored rubric placed next to the first, with one named remaining risk.

Day 7 — Full loop under interview conditions

Run a 60-minute mock covering the two blocks that moved least, with an interviewer instructed to interrupt and change direction.
Write your recovery script for the moment you go blank: restate the question, state your assumption, name the first thing you would check.
Reduce the week to the rule statements you wrote, each with its preconditions attached, then say every one of them out loud without reading it and cut any you cannot state in a single sentence, since a rule you have to reconstruct mid-answer will not survive being interrupted.

Deliverable: A one-page card holding the recovery script and only the rules you could state from memory.

Being wrong in public

A number you shipped turned out to be wrong, and someone had already acted on it. That is one of the most useful stories a data person can carry. What is being scored is how fast you noticed, who you told first, and what you changed in the process so the same class of error could not repeat quietly.

Describe a situation where you had to convince stakeholders to adopt your recommendations.

Medium · behavioural and stakeholder questions

Approach

Pick a story where you drove the decision, not one where you observed it.
Quantify the outcome, including what you would not claim credit for.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Allocate one analyst week across three competing requests

Easy · prioritisation · stakeholders · scoping

Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.

Approach

The interviewer is probing whether you prioritise on decision timing and reversibility or on who asked most forcefully. Sort by the date each decision is actually taken and by what the default outcome is if nothing arrives.
Apply that sort concretely. The Thursday readout has a hard irreversible deadline and no value afterwards. The pricing review has three weeks of slack. The renewal list has a rolling deadline set by term_end_date, so part of it is urgent this week and the rest is not, which means it can be split rather than deferred whole.
Find the cheapest sufficient version of each request rather than the full version. The readout goes in full. The renewal list ships as a filtered query over renewal-eligible accounts ranked by two inspectable signals rather than as a model. The margin work is scoped to the accounts that dominate the pricing decision, since revenue is heavily skewed and the tail will not change the conclusion.
Make the trade visible in one written note to all three at once, with dates. Telling each person separately that they are the priority is how an allocation becomes a credibility problem.
Refuse something explicitly and say why. The model version of the renewal list is the usual candidate, because it cannot be evaluated without a holdout nobody has agreed to yet, and building it this week forecloses that.
Leave slack. A plan with none is a plan to miss the one deadline that cannot move.

Follow-up

The sales leader escalates to your manager. What did you already do that makes that a short conversation?
Which of the three deadlines would you push back on, and what exactly would you ask for?
What would you change about how these requests reach you so next week is not the same?

State the measured impact of your own work honestly

Hard · impact · causal inference · self-assessment

You are asked for the business impact of a renewal-risk worklist you shipped nine months ago. Customer success used it, and renewals in the covered segment came in 6 points above the prior year. Coverage was assigned by the team itself: they worked the top of your list and also the accounts they were already worried about. Produce the impact claim you are willing to defend, the number you refuse to claim, and the design you would have asked for at the start.

Approach

The interviewer is probing whether you can separate what you shipped from what you caused, and whether you would have built the measurement in rather than reconstructing it afterwards. Both halves are being scored.
Name the confound precisely. Coverage assignment is doubly selected: the largest accounts get an owner because they are valuable, and distressed accounts get one because they are at risk. The naive covered versus uncovered comparison mixes a strong positive selection with a strong negative one and can come out with either sign depending on which rule dominated. Matching on account size does not fix it, because the risk signal that triggered coverage is the same signal that predicts the outcome.
Split the claims by what each needs to be true. Ranking quality is defensible from precision at k on out-of-time renewals. Adoption is defensible from timestamps showing what share of listed accounts were contacted. The outcome claim is not defensible without a design, and saying so is the point of the exercise.
Look for identification before giving up on it. A capacity cut-off, a territory boundary, or a period in which the list existed but was unstaffed can assign coverage for reasons unrelated to account health, and any of those supports a bounded estimate.
State the design you would ask for now and its price: a randomly withheld slice of the list, held for two renewal quarters, with the expected cost in renewals stated openly. That cost is what it takes to be able to answer this question at all.
Give a bounded number rather than none. Six points with an explicit statement of how much of it you can attribute is more useful than either claiming the whole figure or declining to quantify anything.

Follow-up

Your manager wants the 6 points in a promotion packet. What wording do you accept, and what do you strike?
What would have had to be true for the naive covered versus uncovered comparison to be valid?
If the holdout costs the team real renewals, how do you justify asking for it, and to whom?