cswork / libraryOpen practice workspace

Interview guides · Kyndryl

Kyndryl Data Scientist Interview Questions & Guide 2026

Modeling drills for a Kyndryl data scientist interview: baselines, regression interpretation, class imbalance, model defense.

The 60-second brief

As a Data Scientist at Kyndryl, you operate at the intersection of massive-scale infrastructure management and cutting-edge artificial intelligence. Your work is pivotal in transforming how Kyndryl services its global client base, moving beyond traditional IT support into proactive, data-driven optimization. You are tasked with turning complex, heterogeneous data streams into actionable insights that enhance system reliability, automate operational workflows, and drive efficiency for large-scale enterprise environments.

Learn the economics of the product category before the loop. Marketplaces, subscription products and ad-supported products turn on different core quantities (match rate and liquidity, retention and churn, fill rate and yield) and fail in different characteristic ways.

Focus areas

  • Separate contracted seats from actively used seats
  • Strip CI, retry and synthetic traffic first
  • Power experiments for heavy-tailed account revenue

Overview

The role is inherently strategic and collaborative. Because Kyndryl often acts as the backbone for its clients' critical operations, your models and analytical frameworks must be robust, scalable, and highly interpretable. You will likely work closely with external clients, meaning your ability to translate technical findings into business value is just as important as your statistical rigor. This position offers the unique challenge of solving real-world problems in environments where downtime is not an option.

Since you may be servicing specific clients, your interviewers might be client stakeholders rather than internal Kyndryl managers. Researching the client’s industry and common pain points can provide a significant advantage.

The Kyndryl interview process

Kyndryl candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

HR Screen

Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.

What to demonstrate

  • Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
  • Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
  • The substance of the questions you ask back, which an experienced screener reads as a level signal

How to prepare

  • Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
  • Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
  • Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered

Technical Assessments

A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.

What to demonstrate

  • Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
  • Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
  • Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.

How to prepare

  • Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
  • Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
  • If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows

Discussion with Hiring Manager

Expect a live problem with pieces of it missing, closer to a conversation than an exam. A metric moved, or somebody wants to know whether a change worked, and you are asked how you would find out. The manager is watching the first ninety seconds, specifically whether you establish what decision hangs on the answer before you start proposing methods. Candidates who open with a technique get steered back. Once the decision is clear, describe what the data would look like if the story were true, and say what you would accept as evidence that it is not.

What to demonstrate

  • Whether you fix the decision the analysis serves before choosing an approach
  • How you continue when you are told the data you just asked for does not exist
  • Whether you state what would change your mind, not only what would confirm the hypothesis you started with
  • How you size an effect before you have measured it

How to prepare

  • Take a metric you know well and practise explaining in under two minutes the four things that could have moved it and how you would separate them
  • Pick a recent launch or experiment and write the single number you would ask for first, plus what you would conclude if it came back flat
  • Practise being interrupted: have someone remove a data source halfway through your answer and carry on without restarting

Where candidates lose points

Comparing accounts that received a sales or customer-success touch against those that did not

Assignment of coverage is deliberate and pulls in both directions at once: the largest accounts get a named owner because they are valuable, and the accounts showing distress get one because they are at risk. The comparison therefore mixes a strong positive selection with a strong negative one, and the naive estimate can come out with either sign depending on which assignment rule dominated during the period examined. Nothing about matching on observed size fixes this, because the risk signal that triggered coverage is usually the same signal that predicts the outcome. It needs either an actual randomised or staggered rollout of coverage, or a design built on a capacity constraint or territory boundary that assigns coverage for reasons unrelated to account health.

Reading consumption metrics before the metering lag window has closed

Usage pipelines land late and correct themselves, which is exactly what is_restated and restated_at record. A dashboard queried on day T sees a partially populated tail for the last several days, so the most recent points always slope downward and always look like a regression. Analysts then explain the artefact, and sometimes ship a change to fix it. Establish the empirical settling time by measuring how much a given usage_date's total moves between first_written_at and its final value, exclude that many trailing days from every reportable figure, and never compare a fresh period against a settled one.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Dropping rows with missing values without naming the mechanism

Say whether the values are missing at random, missing by a known process, or missing in a way that depends on the outcome, and handle them accordingly. Deleting incomplete rows silently redefines the population whenever missingness correlates with what you are measuring.

Questions & practice

Walk me through your process for troubleshooting a model that is underperforming in production.

Medium · machine learning and modelling

Approach

  1. Say how the offline result would be validated online before it is trusted.
  2. Pick an evaluation metric that matches the cost of each error type, not a default.
  3. Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

  • How would you choose the decision threshold, and who owns that choice?
  • What would you monitor after launch to know the model is still valid?

If a client asks for a model that provides 100% accuracy, how do you manage their expectations?

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Set a baseline first, so any model has something honest to beat.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • Where could label leakage enter this setup?

Audit daily usage rows for grain and arithmetic violations

Easy · data-quality · pandas · grain

You are handed fct_usage_daily as a pandas DataFrame with account_id, workspace_id, sku_code, usage_date, billable_quantity, included_quantity_applied, overage_quantity, list_amount_cents, discount_amount_cents, net_amount_cents, cogs_cents, is_restated, first_written_at and restated_at. The declared grain is one row per (account_id, workspace_id, sku_code, usage_date). Write audit(df) returning a DataFrame with one row per failing check: check name, failing row count, and one example key. Cover at minimum grain duplication, negative quantities or amounts, the identity net = list - discount, billable = included + overage, and rows where is_restated is true but restated_at is null.

Approach

  1. Check the grain before anything else with df.duplicated(subset=key, keep=False), and count rows rather than groups so a key appearing twice contributes 2 — if the grain is broken every arithmetic count below it is uninterpretable.
  2. Express each invariant as a boolean Series over the whole frame. The cent columns are integers and compare exactly, so use !=; the numeric(18,6) quantity columns need np.isclose with atol=1e-6 because included + overage is a decimal sum.
  3. Handle null as its own failure mode. Comparisons against NaN return False, so a check written as rows_that_pass = (a == b - c) silently files every null-amount row wherever the negation happens to land; build each check as violations = ~condition | column.isna().
  4. Collect the checks as a list of (name, mask) pairs and assemble the output in one pass, so adding a check is one line and every check reports in the same shape.
  5. Order the output with structural failures (grain, null keys) above arithmetic failures, and report zero-count checks too — a check that silently disappears when it passes is indistinguishable from a check that was never run.

Worked solution (20 min)

  1. key = ['account_id','workspace_id','sku_code','usage_date']; dup = df.duplicated(key, keep=False); record dup.sum() and df.loc[dup, key].iloc[0].to_dict() as the example.
  2. neg = (df[['billable_quantity','included_quantity_applied','overage_quantity','list_amount_cents','net_amount_cents','cogs_cents']] < 0).any(axis=1); net_bad = df.net_amount_cents.isna() | (df.net_amount_cents != df.list_amount_cents - df.discount_amount_cents).
  3. qty_bad = ~np.isclose(df.billable_quantity, df.included_quantity_applied + df.overage_quantity, atol=1e-6) | df.billable_quantity.isna(); restated_bad = df.is_restated & df.restated_at.isna().
  4. Assemble pd.DataFrame([{'check': n, 'failing_rows': int(m.sum()), 'example': first_key(m)} for n, m in checks]) with the grain and null checks listed first.

Expected result: One row per check with failing_rows as an integer count of rows rather than groups, and a JSON-serialisable example key for any check with a non-zero count; a single doubled key contributes 2 to the grain check.

Verify it yourself

  • df.groupby(key).size().max() == 1 if and only if the grain check reports zero.
  • When the net check reports zero, sum(net_amount_cents) == sum(list_amount_cents) - sum(discount_amount_cents) exactly.
  • Running audit on df.sample(frac=1, random_state=0) gives identical counts; the checks must not depend on row order.
  • Corrupting one net_amount_cents by a single cent raises exactly one check's count by exactly 1.

Follow-up

  • Which of these should block a dashboard refresh and which should only warn?
  • Rows with is_restated = true legitimately change value after first write. How do you make yesterday's audit result reproducible?
  • How would you extend this to catch a partition that is missing entirely rather than wrong?

Running commitment burn-down and the date consumption crosses it

Medium · window functions · running totals · committed consumption

fct_subscription_period gives committed_amount_cents, term_start_date and term_end_date for each account's current version where pricing_model = 'committed_consumption'. fct_usage_daily gives account_id, workspace_id, sku_code, usage_date and net_amount_cents, at one row per workspace and SKU per day. Inside each account's current term, return the running total of net_amount_cents by usage_date, the first usage_date on which that running total reaches committed_amount_cents, and the fraction of the term elapsed at that point. Accounts that have not reached their commitment must still appear, with a null crossing date.

Approach

  1. Collapse usage to one row per (account_id, usage_date) first. The fact is grained by workspace and SKU, so a raw running total leaves several rows per date and the first-crossing date becomes dependent on the arbitrary order of rows inside that day.
  2. Restrict to the term with usage_date BETWEEN term_start_date AND term_end_date on the account's current row, so no prior term's consumption leaks into this term's burn-down.
  3. Compute SUM(net_cents) OVER (PARTITION BY account_id ORDER BY usage_date ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW). Name ROWS explicitly: the default frame is RANGE, which includes all peer rows at the same ORDER BY value and would hand back the whole day's total on each of that day's rows.
  4. Extract the crossing with MIN(usage_date) FILTER (WHERE running_cents >= committed_amount_cents) grouped per account, which returns NULL for accounts still under commitment instead of dropping them.
  5. Elapsed fraction is (crossing_date - term_start_date)::numeric / NULLIF(term_end_date - term_start_date, 0); Postgres date subtraction yields whole days, so the guard matters for same-day terms.
  6. Exclude the trailing days still inside the metering settling window, measured from first_written_at against restated_at, and say how many days you cut and why.

Follow-up

  • Turn this into an end-of-term overage forecast. What breaks if you extrapolate a linear run rate on a consumption product?
  • An account crosses its commitment at 40 percent of the term. Is that an expansion signal or a billing-surprise risk, and what would you check to tell them apart?

Collapse retried requests into logical operations per account

Medium · window functions · deduplication · null semantics

fct_api_request carries request_id, account_id, api_key_id, idempotency_key (null when the caller supplied none), is_retry, request_at and http_status. For one ISO week, count logical operations rather than HTTP calls: requests sharing (account_id, api_key_id, idempotency_key) collapse to the earliest one, while every request with a null idempotency_key is its own operation. Return per account the raw request count, the logical operation count and the ratio between them, and state which direction a rising 5xx rate pushes that ratio.

Approach

  1. Split the stream before deduplicating. GROUP BY and PARTITION BY treat NULLs as equal to one another, while the = operator does not, so a single grouping pass folds every null-key request in an account into one 'operation' and can cut the count by orders of magnitude.
  2. Deduplicate the keyed rows with ROW_NUMBER() OVER (PARTITION BY account_id, api_key_id, idempotency_key ORDER BY request_at, request_id) and keep rn = 1. Including request_id in the ORDER BY makes the result deterministic when two rows share a millisecond, which they will.
  3. UNION ALL the null-key rows back in unchanged; they need no dedup and must not pass through the partition.
  4. Aggregate per account: count() over the raw week, count() over the union, and the ratio of the two. Report the ratio, not the difference, so accounts of very different sizes are comparable.
  5. Cross-check against is_retry, which flags a resend of the same idempotency key: raw minus logical should sit close to the count of is_retry rows, and a large gap means clients are resending without a key and the dedup is under-counting duplicates.
  6. Name the inversion explicitly: 5xx responses provoke client retries, so amplification rises exactly when reliability falls, and any engagement metric built on raw request counts will show growth during an outage.

Worked solution (25 min)

  1. Filter to the ISO week with request_at >= :week_start AND request_at < :week_start + interval '7 days', and record the raw count per account as the baseline.
  2. Build keyed_dedup: SELECT ... ROW_NUMBER() OVER (PARTITION BY account_id, api_key_id, idempotency_key ORDER BY request_at, request_id) AS rn FROM the week WHERE idempotency_key IS NOT NULL, then keep rn = 1.
  3. Build unkeyed: the same week's rows WHERE idempotency_key IS NULL, taken as-is.
  4. UNION ALL the two, then GROUP BY account_id selecting the logical count; join back to the raw counts and compute raw::numeric / logical.
  5. Validate on one account by comparing raw - logical against COUNT(*) FILTER (WHERE is_retry) for that account and explaining any gap.

Expected result: One row per account with raw_requests, logical_operations and an amplification ratio at or above 1.0, where logical_operations counts each distinct (account_id, api_key_id, idempotency_key) once and each null-key request once.

Verify it yourself

  • logical_operations is never greater than raw_requests, and the two are equal for accounts whose keys are all distinct.
  • For accounts with no null idempotency_key, raw_requests - logical_operations is close to the count of is_retry rows.
  • Deliberately grouping on the key without the null split reproduces a visibly lower count, confirming the NULL collapse is the mechanism.
  • Re-running the dedup returns the same surviving request_id set, proving the ORDER BY tiebreak made it deterministic.

Follow-up

  • Which of these two counts belongs in the billable-units metric, and which in an engagement metric?
  • An account's amplification ratio jumps from 1.05 to 3.0 in a day. Name three causes and the query that separates them.

How would you design an automated system to identify security vulnerabilities in a network log?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. Fix the population and the time window before naming any metric.
  3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

What metrics would you use to evaluate a model designed to optimize cloud resource allocation?

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. Decompose the metric into the rates that drive it, and say which one you would check first.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

How would you approach a project involving predictive maintenance for server hardware?

Medium

Approach

  1. Say what you would check first and why it is the highest-information step.
  2. State your assumptions explicitly before working the problem.
  3. Clarify what is being asked and what a complete answer would contain.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Explain the difference between supervised and unsupervised learning in the context of anomaly detection.

Medium

Approach

  1. State your assumptions explicitly before working the problem.
  2. Say what you would check first and why it is the highest-information step.
  3. Work from the decision backwards to the evidence you would need.

Follow-up

  • What assumption would you test first?
  • How would you know your answer was wrong?

Stop metering retried requests: design the metric that decides it

Medium · guardrail conflict · billing integrity · retries · expected value

The platform meters accepted requests. A proposal carries three clauses: stop metering fct_api_request rows where is_retry = true, stop metering rows with a 4xx status, and stop metering rows with http_status >= 500. Its author has attached one figure to all three together, roughly 4% of requests_thousands volume. You have fct_api_request (account_id, is_retry, idempotency_key, http_status, traffic_class, billable_units, request_at) and fct_usage_daily (billable_quantity, net_amount_cents, cogs_cents). Size each clause separately before arguing about any of them, then define the primary metric, the guardrail that genuinely conflicts with it, and how you resolve that conflict for a decision that has to be made this quarter. Revenue falls this quarter with certainty; any benefit appears at renewals up to twelve months out.

Approach

  1. Size the three clauses before accepting the headline 4%, because one of them is a no-op. billable_units is defined as zero for requests that failed with a 5xx, so the third clause removes no metered volume at all. Confirm that in the data rather than trusting the column comment: if sum(billable_units) over rows with http_status >= 500 is not zero, the metering pipeline contradicts its own definition and that is a billing defect to file before any pricing conversation happens. The two live clauses are 4xx failures, which are metered in full, and retries that did not themselves end in a 5xx.
  2. Size the two live clauses as a union, not a sum. A retry can return 4xx, so the clauses overlap and adding their volumes counts that intersection twice. Partition the trailing 90 days into four mutually exclusive buckets instead: clean (is_retry = false, http_status < 400), non-retry 4xx, retry with http_status < 500, and http_status >= 500. Report the removable share per account as a distribution; if the mass sits in a handful of accounts this is a commercial conversation with those accounts rather than a platform-wide pricing change.
  3. State the conflict rather than dissolving it. The primary metric, net metered revenue per paying account, and the integrity guardrail, the share of metered volume that is retried or failed traffic, move in opposite directions by construction. No redefinition removes that. The job is to price the trade-off, not to make it disappear.
  4. Show the perverse coupling with data, and be exact about its mechanism. Because a 5xx already carries zero billable_units, the platform is not paid directly for its own failures; it is paid for the retries and the client-side 4xx traffic those failures provoke, which is one step removed and therefore easy to miss. Cross-tab each account's trailing 28-day 5xx rate against its metered volume in the same window. If metered volume rises with error rate, that indirect coupling is the actual argument for the change.
  5. Resolve on expected value with the uncertainty stated. The revenue loss is computable and near-certain; the renewal benefit is not, so invert it and state the break-even: how many basis points of gross logo retention on the renewal-eligible base would offset the loss. That converts an argument about values into an argument about one number. Then propose the measurement that would settle it instead of claiming a readout you do not have: stage the rollout by renewal cohort so accounts whose terms end soonest are treated first, read out on gross logo retention on the renewal-eligible base, and say honestly whether the number of annual renewals in the window can support that estimate at all.

Worked solution (30 min)

  1. Test the third clause first: over the trailing 90 days compute count(*) and sum(billable_units) from fct_api_request where http_status >= 500. The sum must be zero, because billable_units is defined as zero for 5xx failures. If it is zero the clause removes nothing and drops out of the analysis; if it is not, stop and raise a metering defect, because every volume figure downstream of that column is then suspect.
  2. Compute trailing-90-day metered volume per account in the four mutually exclusive buckets: clean, non-retry 4xx, retry with http_status < 500, and http_status >= 500. Roll the total up to the requests_thousands SKU and reconcile it against fct_usage_daily billable_quantity for the same window.
  3. Convert the two removable buckets to money using each account's realised rate, net_amount_cents / billable_quantity from fct_usage_daily, because list rate overstates revenue for every discounted account.
  4. Annualise the revenue at risk and divide it by the ARR of the renewal-eligible base to express the break-even as an improvement in gross logo retention, in basis points.
  5. Cross-tab account 28-day 5xx rate deciles against metered volume per account to establish whether the error-to-revenue coupling, which can only run through retries and 4xx rather than through the failed requests themselves, is real or a story.

Expected result: Three clause sizes rather than one blended figure: zero for the 5xx clause, since billable_units is already zero there, and separate volumes for 4xx failures and for retries that did not 5xx, combined as a union so the retry-and-4xx intersection is counted once. On top of that, a single break-even figure stated as the improvement in gross logo retention on the renewal-eligible base needed to offset the metered revenue actually removed, and the per-account concentration of that removable volume. The answer says that the primary metric and the integrity guardrail conflict by construction, and resolves it with the break-even number and a staged rollout rather than by moving a definition.

Verify it yourself

  • sum(billable_units) over rows with http_status >= 500 must be exactly zero; any other result contradicts the documented definition of the column and invalidates the volume figures derived from it.
  • The four buckets must be mutually exclusive and must sum to total billable_units, which must in turn reconcile to fct_usage_daily billable_quantity within the known restatement tolerance; a large gap means the request-to-SKU mapping is wrong.
  • The two removable buckets must be reported as a union; their arithmetic sum will exceed the union by exactly the retry-and-4xx intersection, and a write-up that quotes the sum has overstated the revenue at risk.
  • Requests sent with no idempotency_key cannot carry is_retry = true, so the measured retry share is a lower bound and the write-up must say so.

Follow-up

  • Suppose the two live clauses turn out to remove 2.6% of consumption revenue. How much improvement in gross logo retention on an annual-contract base pays that back, and over what horizon does the payback land?
  • A retry sent without an idempotency_key cannot be flagged as a retry. Which direction does that bias your estimate of the removable volume, and how can you bound it?

Weekly active organisations fell nine percent over one week

Easy · seasonality · denominators · segmentation

A dashboard reports the weekly active organisation ratio on a trailing seven-day window ending each Wednesday. This week it reads nine percent below last week. You have fct_api_request (account_id, environment, traffic_class, http_status, request_at) and dim_account (account_id, billing_country, account_status, is_internal, is_current). Nothing was released. Decide whether usage actually fell, and hand back a corrected series plus a one-paragraph explanation that a non-analyst can repeat without you in the room.

Approach

  1. Count the holiday-free business days inside each window before comparing them. A trailing seven-day window spans exactly five weekdays wherever it ends, so its business-day count can only fall to four or fewer when a public holiday lands inside it and can never reach six, while usage in this domain follows a hard five-to-two weekday cycle. Two windows holding different numbers of business days are not comparable whatever the product did.
  2. Count distinct active accounts per calendar day for the last ten weeks and overlay the two windows. A calendar problem shows as a small number of weekdays sitting at weekend level, not as every day being uniformly lower.
  3. Cut the daily series by billing_country and index each country-day to that country's trailing same-weekday median, which isolates a regional public holiday from a product change.
  4. Check the denominator on its own: the metric divides by accounts whose account_status was in trial, free or active_paid for the whole week, so a batch suspension or status backfill moves the ratio with no change in the numerator at all.
  5. Report the series with each window's business-day count and the holiday dates annotated beside it, and state the residual week-over-week change that survives once the calendar effect is removed. Compare against earlier windows holding the same number of business days rather than dividing by business days, since distinct-account counts are sublinear in window length and dividing would over-correct.

Follow-up

  • Distinct account counts are sublinear in the number of days in the window. Why does losing one of five business days reduce the count by noticeably less than twenty percent?
  • How would you make this metric comparable across countries with different holiday calendars without hand-maintaining a holiday table forever?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

  • For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
  • For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
  • Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

  • Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
  • List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
  • Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

  • Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
  • Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
  • Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

  • Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
  • Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
  • Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

  • Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
  • Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
  • Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

  • Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
  • For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
  • Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

  • Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
  • Listen back and mark every moment you proposed a solution before the success metric existed.
  • Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Being wrong in public

A number you shipped turned out to be wrong, and someone had already acted on it. That is one of the most useful stories a data person can carry. What is being scored is how fast you noticed, who you told first, and what you changed in the process so the same class of error could not repeat quietly.

How do you handle situations where you have incomplete data or ambiguous project requirements?

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Pick a story where you drove the decision, not one where you observed it.

Follow-up

  • How did you know the outcome was caused by your change?
  • What would you do differently if you ran that project again?

Describe a time you had to pivot your approach due to shifting client priorities.

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. Pick a story where you drove the decision, not one where you observed it.
  3. State the situation in two sentences and spend the rest on your reasoning.

Follow-up

  • How did you know the outcome was caused by your change?
  • What would you do differently if you ran that project again?

Allocate one analyst week across three competing requests

Easy · prioritisation · stakeholders · scoping

Three requests arrive the same morning and you have one week. Finance wants per-account gross margin from fct_usage_daily for a pricing review in three weeks. Sales wants a renewal-risk list for accounts with term_end_date inside 60 days. A product manager wants an experiment readout for a decision being taken on Thursday. Produce your allocation with hours attached, what you say to whoever receives less, and one thing you refuse to do this week, with the reason each decision is defensible to the person it costs.

Approach

  1. The interviewer is probing whether you prioritise on decision timing and reversibility or on who asked most forcefully. Sort by the date each decision is actually taken and by what the default outcome is if nothing arrives.
  2. Apply that sort concretely. The Thursday readout has a hard irreversible deadline and no value afterwards. The pricing review has three weeks of slack. The renewal list has a rolling deadline set by term_end_date, so part of it is urgent this week and the rest is not, which means it can be split rather than deferred whole.
  3. Find the cheapest sufficient version of each request rather than the full version. The readout goes in full. The renewal list ships as a filtered query over renewal-eligible accounts ranked by two inspectable signals rather than as a model. The margin work is scoped to the accounts that dominate the pricing decision, since revenue is heavily skewed and the tail will not change the conclusion.
  4. Make the trade visible in one written note to all three at once, with dates. Telling each person separately that they are the priority is how an allocation becomes a credibility problem.
  5. Refuse something explicitly and say why. The model version of the renewal list is the usual candidate, because it cannot be evaluated without a holdout nobody has agreed to yet, and building it this week forecloses that.
  6. Leave slack. A plan with none is a plan to miss the one deadline that cannot move.

Follow-up

  • The sales leader escalates to your manager. What did you already do that makes that a short conversation?
  • Which of the three deadlines would you push back on, and what exactly would you ask for?
  • What would you change about how these requests reach you so next week is not the same?