Interview guides · Criteo
Criteo Data Scientist Interview Questions & Guide 2026
Case practice for Criteo data scientist candidates: scope a vague prompt into a measurable question before you analyze it.
The 60-second brief
A Data Scientist at Criteo plays a crucial role in harnessing the power of data to drive advertising performance and enhance user experiences. This position is not just about analyzing numbers; it involves crafting algorithms that optimize real-time bidding, improving customer targeting, and ultimately driving revenue for clients. At Criteo, your work will impact millions of users and numerous online businesses, making the role both challenging and rewarding.
SQL is seldom the hardest round and is often the one that eliminates people. The working bar is usually window functions, correct deduplication, and joins that do not silently fan out rows, rather than obscure syntax.
Focus areas
- Diagnose pacing: underdelivery, overdelivery, budget exhaustion
- Reason about bid shading under first-price auctions
- Defend attribution windows and model choice
Overview
In this fast-paced environment, you will collaborate closely with cross-functional teams, including engineers, product managers, and marketing specialists, to develop innovative solutions that leverage vast amounts of data. The complexity of Criteo’s products—ranging from personalized advertising to machine learning applications—ensures that your work remains dynamic and strategically influential. The role is essential not only for product development but also for shaping the company's direction in the competitive AdTech landscape.
The Criteo interview process
Criteo candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
HR Screening Call
Most candidates lose this call inside the first two minutes, during the walkthrough of their own background. The account runs chronologically, sits at the level of tools and titles, and never arrives at a decision anyone could have disagreed with. Anchor on a problem instead of a timeline: what the team could not answer, what you did about it, what happened next. Ninety seconds is enough, and stopping on time leaves room for the half of the call that belongs to you. What you ask about how work gets prioritised signals your level more reliably than the walkthrough does.
What to demonstrate
- Whether your background summary has a shape (problem, decision, consequence) or is a chronological list of tools and employers
- Whether you can account for gaps, short stints and the reason you are looking, unprompted and without hedging
- The substance of the questions you ask back, which an experienced screener reads as a level signal
How to prepare
- Time your opening walkthrough against a clock. If it runs past two minutes, compress the earliest role into a single clause and spend the recovered time on the most recent one
- Write one honest sentence for every gap or short stint visible on your resume and offer it before being asked about it
- Prepare questions about how work arrives and gets prioritised: who writes the request, how often priorities change, and what happens to an analysis after it is delivered
Technical Assessments
A handful of shapes account for most of what gets asked in this format: a ranking or deduplication inside groups, a running or rolling total, a period-over-period comparison, and a cohort tracked forward over time. Recognising the shape quickly is most of the speed here; deriving it from scratch while a clock runs is where the time goes. Know that a window function keeps every row while a GROUP BY collapses them, and know which one the question needs. If the exercise is in Python instead of SQL, the same shapes arrive as groupby with transform, shift and merge, and the same grain mistakes are available.
What to demonstrate
- Whether you reach the right construct without a detour, such as ROW_NUMBER over a partition to deduplicate instead of a self-join against a MAX subquery
- Whether you know what your window frame actually is, since adding ORDER BY inside OVER changes the default frame and silently changes a running total
- Whether the thing runs. A near-miss that throws an error scores below a plainer query that returns the right rows.
How to prepare
- Write each of the four shapes once from memory against a small schema and keep the working version somewhere you will reread it: dedupe with ROW_NUMBER, a running total, a month-over-month change with LAG, and a retention table
- Compute one running total twice on data with tied timestamps, once on the default frame and once with ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and look at where the two disagree
- If Python is on the table, rebuild the dedupe and the running total with groupby and cumsum, then assert the two implementations return identical rows
Team Interviews
Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.
What to demonstrate
- Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
- Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
- Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs
How to prepare
- For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
- Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
- For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.
Where candidates lose points
Reading a recent week's cost per action or return on ad spend before the conversion window has closed.
Spend for a period is final within hours, but conversions attributed to that period keep arriving for as long as the click window allows plus ingestion lag, and offline uploads can land weeks later. The denominator of cost per action is therefore systematically incomplete for recent dates, so recent CPA is biased high and recent ROAS biased low, and a dashboard comparing a fresh week to a matured week will show a regression that does not exist. The correct handling is to freeze a period only after click_window_days plus the observed ingestion lag, and to publish a maturation curve so readers can see how much of a given day's conversions have landed so far.
Counting conversions without deduplicating across reporting sources and identity spaces.
The same purchase routinely arrives twice, once from a browser pixel and once from a server-side API, and the two reports carry different identifiers, so a naive count inflates conversions and deflates cost per action by the duplication rate. Deduplication depends on the advertiser populating a shared dedup_key correctly, which many do not, and the failure is silent because both rows look individually valid. The mirror-image error is under-counting: when a user clicks on a mobile app and converts in a desktop browser, the identity join fails and a real conversion is attributed to nothing, which makes the channel look worse than it is. Any conversion count reported without stating the dedup rule and the identity-match rate behind it should be treated as unverified.
Solving silently instead of narrating the reasoning
Say which branch you are taking and why you chose it over the alternative, for example checking the denominator first because it changes what the comparison means. A correct answer that arrives with no visible path scores below a rigorous one that needed a hint.
Stopping an experiment the moment it crosses significance
Fix the sample size or duration before launch, or use a method built for continuous monitoring such as a sequential test, always-valid confidence intervals, or group-sequential boundaries. Repeatedly checking a fixed-horizon p-value against 0.05 pushes the real false-positive rate well above 5 percent.
Questions & practice
Discuss the bias-variance tradeoff.
Medium · statistics and probability
Approach
- Sanity-check the answer against a simple bound or a simulated case.
- Translate the result into the decision it informs, in one plain sentence.
- Say what the estimate is of, and over what population it generalises.
Follow-up
- What sample size would you need to detect an effect half this size?
- How would you explain this result to someone who does not know statistics?
Explain how you would implement a decision tree from scratch.
Medium · machine learning and modelling
Approach
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- How would you choose the decision threshold, and who owns that choice?
- What would you monitor after launch to know the model is still valid?
What is overfitting, and how can it be prevented?
Medium · machine learning and modelling
Approach
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Check what information would not exist at prediction time, and exclude it.
- Set a baseline first, so any model has something honest to beat.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
Detect duplicate conversion reports across pixel and server sources
Easy · deduplication · data-quality · conversions · pandas
You are given a pandas DataFrame conversions holding raw conversion_event rows: conversion_id, advertiser_id, conversion_ts, received_ts, conversion_type, conversion_value_usd, order_id, device_id, click_tracking_id, source, dedup_key. The is_duplicate column has been removed because producing it is the task. Roughly 30% of rows carry a null dedup_key. Return a boolean Series aligned to the input index marking duplicate reports of one real-world conversion, plus a per-advertiser table giving the duplication rate and the dedup_key coverage behind it. State your keying rule and its fallbacks.
Approach
- Declare a keying ladder before writing code and apply the tiers in priority order so each row is matched by exactly one rule: tier 1 (advertiser_id, dedup_key) where dedup_key is present, tier 2 (advertiser_id, order_id) where it is not, tier 3 a bounded behavioural rule. Keep the tier label on the output so the result is auditable rather than a bare boolean.
- Within tier 1, sort by received_ts then conversion_id and mark every row after the first in a group. Keeping the earliest-received row is the right survivor because it is the one already counted in published numbers; keeping the latest silently restates history every time a late report lands.
- For tier 3, sort within (advertiser_id, device_id, conversion_value_usd) by conversion_ts and collapse rows whose gap to the current survivor is under 60 seconds, carrying the survivor forward rather than chaining diffs. A plain diff chain collapses a run of rows 40 seconds apart into one, which is a different and wrong rule.
- Implement tiers 1 and 2 with sort plus groupby.cumcount, never with a self-join: a self-join on advertiser_id is quadratic in the largest advertiser and will not finish on a real month of data. Tier 3 cannot be written that way. Because a flagged row does not become the survivor, each comparison depends on which earlier rows were already collapsed, so cumcount (which flags every row after the first regardless of gap) and a shifted-diff (which compares against the previous row, not the survivor) both compute a different rule. Run tier 3 as one sequential O(n) pass over the sorted arrays - a Python loop over the numpy columns, or numba if the tier is large - which is affordable because tier 3 holds only the rows with neither dedup_key nor order_id.
- Report duplication rate alongside dedup_key coverage per advertiser. Coverage is what makes the rate interpretable: at 30% nulls the measured duplication rate is a lower bound, and the gap is an advertiser tagging problem, not a conversion problem.
Worked solution (20 min)
- Add a tier column: 1 where dedup_key.notna(), 2 where dedup_key.isna() and order_id.notna(), 3 otherwise. Assert the three masks partition the frame.
- Tier 1 and 2: df.sort_values(['received_ts','conversion_id']).groupby(key).cumcount() > 0 gives the duplicate flag; write it back by index so the output stays aligned.
- Tier 3: sort within (advertiser_id, device_id, conversion_value_usd) by conversion_ts, then walk the sorted arrays once, holding the current survivor's conversion_ts per group and flagging each row whose gap to that survivor is under 60 seconds; a flagged row does not advance the survivor. Keep it a single sequential pass - it is O(n) but not vectorisable, so budget it as the slow tier and keep it small by filling dedup_key and order_id upstream.
- Aggregate per advertiser_id: total_rows, flagged_duplicates, inflation_rate = flagged/total, dedup_key_coverage = share of rows with non-null dedup_key, and the share of flags contributed by each tier.
Expected result: A boolean Series with the same index and length as the input, plus a per-advertiser frame with total_rows, flagged_duplicates, inflation_rate, dedup_key_coverage and per-tier flag shares. Summing conversion_value_usd over unflagged rows is strictly lower than the raw sum, and the drop equals the value carried by the flagged rows.
Verify it yourself
- Every tier-1 and tier-2 group retains exactly one survivor: (~flag).groupby(key).size().max() == 1.
- No row is flagged by more than one tier, and the tier column covers every row exactly once.
- Tier 3 on a hand-built group of four rows spaced 40 seconds apart flags rows 2 and 3 but leaves row 4 (120 seconds after the survivor) unflagged. A shifted-diff or cumcount implementation flags all three, which is how you catch the wrong rule.
- Advertisers whose rows all come from a single source (source.nunique() == 1) should show a near-zero rate; a high rate there means the rule is collapsing genuinely distinct purchases.
Follow-up
- Tier 3 will collapse two genuine purchases of the same item a minute apart. How do you bound that false-positive rate, and for a CPA report would you rather over-collapse or under-collapse?
- An advertiser populates dedup_key with order_id on browser_pixel rows and with a session id on server_api rows. What does your ladder do, and how would you detect that from the data alone?
- If the duplication rate is 8%, what is the effect on attributed CPA, and in which direction does it move the number the advertiser sees?
Write a function to calculate the mean and median of a list of numbers.
Medium · sql and data manipulation
Approach
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- State the window function and its partition and ordering out loud before writing it.
- Say which table is the grain you start from, and join outward from it.
Follow-up
- How would you verify this result without re-running the same query?
- What breaks if events arrive late or out of order?
Time-weighted daily budget from slowly changing line item versions
Hard · scd type 2 · window functions · interval weighting · pacing
line_item is SCD type 2: one row per version with line_item_id, daily_budget_usd, pacing_mode, status, valid_from_ts, valid_to_ts (null on the current row) and is_current. ad_impression has line_item_id, served_ts, billed_price_micros_usd and is_billable. For one UTC date, compute budget delivery rate per line item: spend divided by a time-weighted daily budget, where the day's budget is each overlapping version's daily_budget_usd weighted by the fraction of the 24 hours that version was in force. Return spend, weighted budget, the weight coverage, and attainment.
Approach
- Select versions that overlap the day with valid_from_ts < day_end AND COALESCE(valid_to_ts, 'infinity'::timestamptz) > day_start. Use strict inequality on both sides so a version that ends exactly at midnight does not also claim the following day, which would make the weights sum above 1.
- Clip each version to the day before weighting: start at GREATEST(valid_from_ts, day_start) and end at LEAST(COALESCE(valid_to_ts, 'infinity'::timestamptz), day_end). Weighting on the unclipped interval gives a month-long version a weight of 30 rather than 1.
- Derive the weight as EXTRACT(EPOCH FROM (clipped_end - clipped_start)) / 86400.0 and sum daily_budget_usd * weight per line item. Keep SUM(weight) in the output as coverage: it should be 1.0 for a line item live all day and below 1 for one created or ended mid-day, and anything above 1 means the interval predicate is wrong.
- If valid_to_ts is unreliable on superseded rows, derive the interval end with LEAD(valid_from_ts) OVER (PARTITION BY line_item_id ORDER BY valid_from_ts) instead, and say which source you trust. Mixing the two produces overlapping intervals that double-count budget.
- Aggregate spend in its own CTE at impression grain, filtered to is_billable and bucketed on served_ts in UTC, then join the two pre-aggregated results on line_item_id. Joining the SCD table to impressions before aggregating multiplies every impression by the number of versions.
- Compute attainment as spend / NULLIF(weighted_budget, 0), and decide explicitly how to treat versions whose status was paused or ended for part of the day, since including a paused stretch in the denominator reads as underdelivery that never had a chance to happen.
Worked solution (40 min)
- Declare day_start and day_end as explicit TIMESTAMPTZ bounds, then select line_item rows where valid_from_ts < day_end AND COALESCE(valid_to_ts, 'infinity'::timestamptz) > day_start.
- Add w := EXTRACT(EPOCH FROM (LEAST(COALESCE(valid_to_ts, 'infinity'::timestamptz), day_end) - GREATEST(valid_from_ts, day_start))) / 86400.0 to each surviving row.
- Build budget AS (SELECT line_item_id, SUM(daily_budget_usd * w) AS weighted_budget, SUM(w) AS coverage FROM clipped GROUP BY 1).
- Build spend AS (SELECT line_item_id, SUM(billed_price_micros_usd) / 1e6 AS spend FROM ad_impression WHERE is_billable AND served_ts >= day_start AND served_ts < day_end GROUP BY 1), entirely independent of the dimension.
- FULL OUTER JOIN budget and spend on line_item_id, compute spend / NULLIF(weighted_budget, 0), and inspect the rows where coverage differs from 1.0 as a separate list.
Expected result: Per line item: spend, weighted_budget, coverage and attainment. A budget of $1,000 in force from 00:00 to 18:00 and $2,500 from 18:00 to 24:00 gives weighted_budget = 10000.75 + 25000.25 = $1,375, and spend of $1,210 gives 88.0% attainment at coverage 1.0.
Verify it yourself
- For any line item live the whole day, the clipped intervals tile the 24 hours with no overlap and no gap, so SUM(w) is exactly 1.0.
- Exactly one row per line_item_id has is_current = TRUE. If more do, the SCD writer is broken and every weight downstream is wrong.
- Spend from this query equals a plain aggregate over ad_impression with no line_item join at all. A higher figure is proof the version join fanned out.
- A line item with a single version covering the whole day returns attainment identical to the naive spend / daily_budget_usd calculation, which is the degenerate case the weighting must reduce to.
Follow-up
- A line item shows 180% attainment. Give three explanations that are not overspend.
- How does the same calculation change for lifetime_budget_usd across a multi-week flight rather than a daily budget?
- pacing_mode is 'even' but hourly spend is front-loaded and stops at 14:00. Which query distinguishes budget exhaustion from a pacing throttle, and which column settles it?
Given a dataset with user engagement metrics, how would you determine which factors influence purchase decisions?
Medium · product sense and metrics
Approach
- Restate the decision this analysis has to support, and who acts on the answer.
- Fix the population and the time window before naming any metric.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- Which segment would you cut first, and what would that rule out?
- How would you detect that the metric is being gamed rather than genuinely improving?
Design an experiment to test the effectiveness of a new ad campaign.
Medium · a/b testing and experimentation
Approach
- State the primary metric and the minimum effect worth shipping, then size the test.
- Name the guardrails that would stop a launch even on a positive primary result.
- Decide the analysis before seeing data, including how long it runs and when you look.
Follow-up
- What would you conclude if the result is positive but the test is underpowered?
- How would you handle interference between treated and control units?
Explain the concept of Click-Through Rate (CTR) and its importance in online advertising.
Medium
Approach
- Work from the decision backwards to the evidence you would need.
- State your assumptions explicitly before working the problem.
- Say what you would check first and why it is the highest-information step.
Follow-up
- What assumption would you test first?
- How would you know your answer was wrong?
Design a performance metric the owning team cannot edit
Medium · attribution · gaming · incrementality
A bidding team proposes to be evaluated on attributed cost per action: billable spend from ad_impression over SUM(credit_fraction) from attribution_credit, which carries attribution_model, model_version, click_window_days and view_window_days. The same organisation owns the attribution configuration. Deliverable: name three configuration edits that improve this metric with no change to bids, delivery or creative; then propose a primary metric that cannot be improved that way, plus the guardrails you would keep on the attributed number.
Approach
- Write the metric as a formula and audit every input for whether it describes behaviour or policy: CPA = spend / SUM(credit_fraction). Spend is contractual and hard to fake. The denominator is the output of a rules engine, and the rules are owned by the team being measured.
- Name the three levers concretely. Widening
view_window_daysfrom 1 to 7 admits view-through conversions that previously fell outside the window. Wideningclick_window_daysfrom 7 to 28 admits late converters. Switchingattribution_modelfromlast_clicktolinear,position_basedordata_drivenspreads credit onto mid-path touchpoints that previously earned zero. Each raises the denominator with identical bids, identical clearing prices and identical creative. - Show the arithmetic is not conserved across channels: credit sums to 1.0 within one conversion for one model and version, but each channel reports its own credited total, so the sum of attributed conversions across a portfolio routinely exceeds the advertiser's own order count. That is a property of the convention, not a bug, and it is why the number cannot anchor a performance contract.
- Propose the primary whose inputs the team does not own: incremental conversions per $1,000 of spend, estimated from an always-on randomised holdout at a unit that contains the budget — a geographic market or a separately budgeted cell — or from a ghost-bid holdout where control is defined by winning the auction and withholding the creative. Randomising users inside one shared budget does not estimate this, because the arms compete for the same budget and the same auctions.
- Keep attributed CPA, but freeze it: put
attribution_model,model_version,click_window_daysandview_window_daysinto the metric's key so a configuration change creates a new series instead of moving the old one. Add a maturation guardrail — the share of a date's eventual credit that has landed by read time — so nobody reads a fresh week against a matured one.
Worked solution (35 min)
- Recompute one month of attributed CPA from unchanged delivery logs under view_window_days of 1 and 7 and record the gap.
- For one conversion with four touchpoints across two channels, hand-split credit under last_click, linear and position_based and note which line item's reported CPA improves under each.
- Specify the holdout: randomisation unit, how budget is separated per arm, the estimator, and the analysis window including the conversion tail.
- Write the frozen-config metric key and the maturation guardrail definition.
- Write the two-sentence argument for why attributed numbers stay as a budget-splitting convention while reallocation decisions are sourced from the holdout.
Expected result: Primary is incremental conversions per $1,000 spend from a budget-containing randomised holdout; attributed CPA is demoted to a reporting metric whose key pins model, version and both windows; the three named levers are stated with the direction each moves CPA.
Verify it yourself
- The same delivery logs produce materially different attributed CPA under two window settings, proving the denominator is policy.
- The holdout estimate does not move when the attribution configuration changes.
- Attributed conversions summed across the advertiser's channels exceed the advertiser's own order count.
Follow-up
- The holdout costs roughly 5% of spend in forgone delivery. How do you justify that to the advertiser paying for it?
- Data-driven attribution is fitted rather than hand-set. Does that make it harder to game, or just harder to audit?
- An advertiser demands the wider view window because another platform reports it that way. What do you do?
Cost per action spiked in the most recent week only
Easy · attribution · late-arriving data · cpa
A dashboard shows attributed CPA up 38% for the last seven days against the trailing eight-week average, on the last_click model. CPA is spend from billable ad_impression rows bucketed on served_ts, over SUM(attribution_credit.credit_fraction) bucketed on touchpoint_ts for one attribution_model and model_version. Click windows are 7 days, view windows 1 day. conversion_event carries conversion_ts and received_ts. Nothing shipped. Decide whether performance regressed, and if not, produce the correction the dashboard needs.
Approach
- Separate the two sides first. Spend for a past date is final within hours; attributed credit keeps accruing. Confirm this by re-querying spend for a date a week old and checking it is byte-identical to the value stored earlier.
- Build a maturation curve: for each touchpoint date, compute observed SUM(credit_fraction) as a function of age in days, using cohorts old enough that credit has stopped moving — click_window_days plus the observed ingestion lag from received_ts minus conversion_ts.
- Read the maturation factor for the ages of the last seven touchpoint dates and divide each day's observed credit by it. A recent-week spike that collapses into the historical band under this adjustment was never a regression.
- Quantify ingestion lag separately by source, because offline_upload lands weeks after browser_pixel; a maturation curve fitted on blended sources will mis-adjust an advertiser that uses one of them heavily.
- Fix the dashboard rather than the number: suppress or grey any touchpoint date younger than click_window_days plus the p95 ingestion lag, and publish the maturation curve next to the metric so a reader can see how complete each day is.
Follow-up
- An advertiser uploads offline conversions monthly. What does that do to your freeze rule, and would you give them a different one?
- What is the sign of the bias on recent ROAS, and why is it the opposite of the bias on recent CPA?
- How would you alert on a genuine CPA regression without the alert firing every day from immaturity?
Suggested practice checklist
Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.
Day 1 — Breadth pass: query fluency
- Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
- For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
- Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.
Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.
Day 2 — Breadth pass: statistics and inference
- Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
- Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
- Rewrite the two weakest answers the following morning from memory in full sentences.
Deliverable: Ten graded answers with an honest count of exact hits.
Day 3 — Breadth pass: modelling
- Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
- Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
- Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.
Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.
Day 4 — Breadth pass: product judgement
- Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
- For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
- Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.
Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.
Day 5 — Depth, first area
- Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
- Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
- Re-solve the two you failed the same evening with notes closed.
Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.
Day 6 — Depth, second area, and the seam between them
- Repeat the depth protocol on the second-ranked area with the same six-problem structure.
- Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
- Solve your own combined problem end to end and note where the handoff between the two areas cost you time.
Deliverable: One combined problem, solved end to end, with the handoff failure written down.
Day 7 — Integration and re-measurement
- Re-run the six prompts from day one under the same clock and compare both correctness and time.
- Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
- Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.
Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.
Rigour against a deadline
Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.
Describe a time when you had to work with a difficult team member. How did you handle the situation?
Medium · behavioural and stakeholder questions
Approach
- Close with what you would do differently, concretely.
- Name the disagreement or constraint, and how you resolved it with evidence.
- Quantify the outcome, including what you would not claim credit for.
Follow-up
- How did you know the outcome was caused by your change?
- What did you decide not to do, and why?
What motivates you to work in the field of data science?
Medium · behavioural and stakeholder questions
Approach
- State the situation in two sentences and spend the rest on your reasoning.
- Pick a story where you drove the decision, not one where you observed it.
- Name the disagreement or constraint, and how you resolved it with evidence.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Disagreeing with a proposal to widen the view window
Medium · disagreement · attribution · windows · influence
A product manager proposes changing the default view window on attribution_credit from 1 day to 7 days, to bring reported performance in line with competitors. Their mock shows reported CPA falling 22%. You believe nothing real changes. You have read access to attribution_credit, conversion_event and ad_impression, and two days before the decision review. Deliverable: the analysis you run and the one-page argument you take into the review, including what you concede and the condition under which you would support the change.
Approach
- The interviewer is probing whether you can disagree from inside the other person's numbers rather than from principle, so reproduce their 22% first under the same attribution_model and conversion cohort with view_window_days set to 7, and say out loud that it is correct. The disagreement is now about interpretation, which is a debate you can win.
- Partition the newly credited volume, which is the whole argument. Group by conversion_id under both windows and split newly credited conversions into those that already carried credit on another touchpoint, which is reshuffling between channels, and those that carried no credited touchpoint at all, which is genuinely new claiming. Report the split as a percentage.
- Test the substantive claim on the newly claimed share: look for holdout evidence that view-through exposure at 1 to 7 day latency produces measurable lift. If no such evidence exists, say so plainly and label that share unvalidated rather than incremental.
- Price the cost the proposal does not mention: changing the default changes model_version, which restates history, breaks every published time series, and moves advertiser-facing numbers that some advertisers reconcile against their own order tables.
- Offer a conditional yes with a mechanism: support a window set from the observed conversion-latency distribution and corroborated by holdout lift, shipped as a parallel model_version alongside the existing one rather than as a replacement.
Follow-up
- The product manager says competitors already report this way and we look worse by comparison. Does that change your answer?
- How would you choose a defensible view window from data rather than from convention?