cswork / libraryOpen practice workspace

Interview guides · CBRE

CBRE Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for CBRE: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

As a Data Scientist at CBRE, you are stepping into a pivotal role at the world’s largest commercial real estate services and investment firm. Your work directly influences how global property markets are analyzed, how investments are evaluated, and how physical spaces are optimized for businesses worldwide. You will not just be crunching numbers; you will be translating massive, complex datasets into actionable strategies that drive multimillion-dollar real estate decisions.

Coding rounds for this role are usually data-manipulation shaped rather than data-structure shaped: group-bys, joins, time windows, ranking within a partition. Confirm the format before spending a week on graph traversal.

Focus areas

  • Turn a vague request into a measurable question
  • Define numerator, denominator and window precisely
  • Decompose a metric move by segment and mix

Overview

The impact of this position spans across multiple facets of the business, from predictive modeling for property valuations to optimizing facility management through IoT data. You will collaborate with brokers, researchers, and global clients to build scalable data products that uncover hidden market trends. Because commercial real estate is inherently spatial and deeply tied to macroeconomic factors, the problems you solve will be multifaceted, requiring a blend of statistical rigor and sharp business acumen.

Expect to work in an environment where your insights carry significant weight. Whether you are forecasting market rents, analyzing foot traffic patterns, or building recommendation engines for property investments, your technical deliverables will empower stakeholders to make confident, data-backed decisions. This role offers the unique challenge of applying cutting-edge machine learning techniques to a traditional industry that is rapidly transforming through technology.

The CBRE interview process

CBRE candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Recruiter Phone Screen

A screening call is a matching exercise run by someone who will not evaluate your statistics. They are checking that the work described on your resume is work you personally did, and that its scope matches the level the role is written for. Logistics get settled in the same half hour so nobody spends an interviewer's afternoon on a mismatch. The answer that fails is the one narrated in the plural. If every sentence is 'we built' and 'the team decided', there is nothing specific to write down about you. Name the piece that was yours, the decision you made inside it, and what changed after.

What to demonstrate

  • Whether the ownership implied by your resume survives one round of follow-up about who actually did which part
  • Whether your described scope (data size, stakeholders, what shipped) matches the seniority the role is written at
  • Whether timeline, location and compensation expectations make the rest of the loop worth scheduling

How to prepare

  • Rewrite your top three resume bullets in the first person singular, each with the decision you made and what moved afterwards, then say them out loud once so the 'we' does not return under pressure
  • Attach one number to each project: the baseline, the change, and the window it was measured over. Where impact was never measured, say that plainly rather than inventing a figure
  • Settle your compensation range before the call and give it as a range with a reason behind it, such as current total comp or a competing timeline, instead of deflecting the question twice

HireVue Video Assessment

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

  • Whether an ambiguous term becomes a specific column and filter before any computation happens
  • Whether you read the schema for keys and cardinality rather than only for column names
  • Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

  • Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
  • On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
  • Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Interview with Hiring Manager

Much of this round runs on your own history, but the manager is not collecting a project list. They are working out what it is like when something goes wrong on your watch: how late the bad news tends to arrive, and whether a number you hand over has been checked by anyone including you. That is why the strongest material is a project where you can describe the part that did not work and what it cost. A result you cannot take full responsibility for, however clean, gives them nothing to trust you with afterwards.

What to demonstrate

  • Whether you volunteer the limits of a result you are proud of, or wait to be pushed onto them
  • How errors surfaced in your past work, and whether you or somebody else found them
  • Whether the scope you claim matches the level of detail you can still produce about it
  • What you did the first time a stakeholder acted on something of yours that turned out to be wrong

How to prepare

  • Rebuild one headline figure from memory down to the join and the filter, so a question about the denominator does not stall the conversation
  • For each project you raise, write the sentence you would say to someone who had already acted on a number that later turned out wrong
  • Mark which parts of a project were yours and which belonged to other people, and state that boundary yourself before anyone asks

Final Interview

A day of back-to-back interviews samples your floor, not your ceiling. Four hours in, the habits that carry a good answer are the first to go: restating the question before solving it, asking what the data would have to look like, checking a number before quoting it. What the day decides is whether the tired version of you is still someone to leave alone with an ambiguous problem. The round that sinks a candidate is usually not the hardest one. It is the one immediately after the round that went badly.

What to demonstrate

  • Whether the late rounds get the same clarifying questions as the first one, or whether you start answering immediately to save effort
  • Whether a weak answer stays in the room it happened in, instead of following you into the next conversation as apology or distraction
  • Whether the quality of your questions holds up, since fatigue removes curiosity about the problem before it removes knowledge of the method

How to prepare

  • Rehearse the length, not just the content: book four mock interviews of different types in one afternoon with short gaps, because the one you need to observe is the fourth
  • Put the two or three questions you ask at the start of any problem on a card in front of you, so that under fatigue it is a habit you run rather than a decision you make
  • Decide in advance what the gap between rooms is for: water, one line of notes on anything you promised to follow up, and an explicit close on the round that just ended so it does not travel
  • Prepare a different closing question for each interviewer, so the end of a long day does not produce the same one four times

Where candidates lose points

Treating last-touch attribution as the causal value of a channel

The attribution label on dim_user is the output of a rule that assigns full credit to whichever touch happened to be recorded last inside a lookback window, and that rule systematically rewards channels that sit close to the conversion, especially branded search and retargeting, which largely intercept demand that already existed. Reallocating spend on those labels moves budget toward the channels that are best at being last, which is why attributed return on ad spend often improves while total signups do not. Nothing in the touchpoint data can settle this, because the counterfactual of not running the channel was never observed. The credible reads are a geo holdout or a scheduled pause, sized in advance on the total-signups metric rather than on the attributed one, and the honest framing in the meantime is that the label describes correlation with conversion and not incremental contribution.

Reading a pooled rate that moved because the mix moved, not because any behaviour changed

A pooled conversion rate is a weighted average, and a shift in the weights can move it in the opposite direction to every one of its parts. A paid campaign that brings low-converting traffic drops overall signup conversion even if desktop, mobile web and app conversion each rose that week, which is Simpson's paradox and it is the single most common cause of an inexplicable dashboard move. The discipline is to decompose before explaining: recompute the rate holding last period's segment weights fixed, and compare that counterfactual to the actual, so the mix effect and the rate effect are separated numerically rather than argued about. Segment on the dimensions that actually reweight, which in this domain are almost always device_type, referrer_channel, country and new versus returning.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Sizing estimates built on unnamed, unrevisable assumptions

Write each assumption as a named number you can change, then show the arithmetic so the interviewer can challenge one input instead of the whole answer. Finish by saying which assumption the result is most sensitive to, which matters more than the point estimate.

Questions & practice

How do you address missing values in a dataset, and how does your approach change if the data is missing not at random?

Medium · statistics and probability

Approach

  1. Quantify uncertainty explicitly rather than reporting a point estimate alone.
  2. Write down the assumption the method needs before you use the method.
  3. Translate the result into the decision it informs, in one plain sentence.

Follow-up

  • How would you explain this result to someone who does not know statistics?
  • Which assumption here is most likely to be violated in practice?

Explain the difference between L1 and L2 regularization. When would you use each?

Medium · statistics and probability

Approach

  1. Write down the assumption the method needs before you use the method.
  2. Sanity-check the answer against a simple bound or a simulated case.
  3. Say what the estimate is of, and over what population it generalises.

Follow-up

  • Which assumption here is most likely to be violated in practice?
  • What sample size would you need to detect an effect half this size?

If a business leader asks you to build a predictive model but the underlying data is heavily flawed, how do you proceed?

Medium · machine learning and modelling

Approach

  1. Say how the offline result would be validated online before it is trusted.
  2. Check what information would not exist at prediction time, and exclude it.
  3. Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

  • Where could label leakage enter this setup?
  • How would you choose the decision threshold, and who owns that choice?

Walk me through the architecture of a Random Forest model. What are its main advantages and limitations?

Medium · machine learning and modelling

Approach

  1. Set a baseline first, so any model has something honest to beat.
  2. Check what information would not exist at prediction time, and exclude it.
  3. Say how the offline result would be validated online before it is trusted.

Follow-up

  • What would you monitor after launch to know the model is still valid?
  • How would you choose the decision threshold, and who owns that choice?

How do you determine if a model is ready to be deployed into production?

Medium · machine learning and modelling

Approach

  1. Pick an evaluation metric that matches the cost of each error type, not a default.
  2. Set a baseline first, so any model has something honest to beat.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • Where could label leakage enter this setup?
  • How would you choose the decision threshold, and who owns that choice?

Cluster bootstrap for a per-session rate randomised on users

Hard · cluster bootstrap · randomisation unit · variance

An experiment randomised on user_id reports a per-session conversion rate, so sessions inside a user are correlated. Input: one row per session with user_id, variant in {control, treatment} and converted in {0,1}. Write a cluster bootstrap from scratch: resample users with replacement within each arm, keep every session of a drawn user, recompute each arm's ratio of converted sessions to sessions, and take the difference. Return the point estimate, a 95 percent percentile interval from at least 2,000 resamples, the naive session-level interval that ignores clustering, and the ratio of their widths.

Approach

  1. Name the estimand precisely: it is a ratio of sums, sum(converted) over sum(sessions) within an arm, not the mean of per-user rates. Those differ whenever session counts vary across users, and the ratio is what the reported metric is.
  2. Resample the cluster, not the row. Draw n_users user ids with replacement inside each arm and take every session belonging to each draw, including duplicate draws of the same user. Keeping the user count fixed per arm rather than the session count is what preserves the sampling design.
  3. Precompute per-user (converted_sum, session_count) once, so each resample is two vector lookups and a division rather than a repeated filter over the session frame. That turns 2,000 resamples from minutes into under a second.
  4. Take the 2.5th and 97.5th percentiles of the 2,000 differences for the interval, and report the point estimate from the full data rather than from the bootstrap mean, since the bootstrap mean carries the resampling bias.
  5. Compute the naive interval from the session-level binomial standard error and compare widths. The expected inflation is roughly sqrt(1 + (m-1)*rho), with m the mean sessions per user and rho the intraclass correlation of converted within users, so a computed ratio far from that value points at a bug in one of the two intervals.

Worked solution (35 min)

  1. per_user = df.groupby(['variant','user_id'])['converted'].agg(['sum','size']); split into two arrays per arm.
  2. point = (t_sum.sum() / t_n.sum()) - (c_sum.sum() / c_n.sum()).
  3. For b in range(B): idx = rng.integers(0, len(t_sum), len(t_sum)); ratio_t = t_sum[idx].sum() / t_n[idx].sum(); same for control; store the difference. Vectorise by drawing a (B, n) index matrix if memory allows.
  4. ci = np.percentile(diffs, [2.5, 97.5]); naive_se = sqrt(p_t*(1-p_t)/n_sessions_t + p_c*(1-p_c)/n_sessions_c); naive_ci = point +/- 1.96*naive_se.
  5. width_ratio = (ci[1]-ci[0]) / (naive_ci[1]-naive_ci[0]).

Expected result: A point estimate identical to the full-data ratio difference, a percentile interval containing that point estimate, and a width ratio greater than 1 whose value approximates sqrt(1 + (m-1)*rho) for the data's mean cluster size m and intraclass correlation rho. A ratio of approximately 1.0 means sessions were resampled instead of users.

Verify it yourself

  • Construct data where every user has exactly one session: the cluster and naive intervals then agree to within Monte Carlo error, since m = 1 makes the design effect 1.
  • Construct data where converted is constant within each user: rho is 1, and the width ratio should approach sqrt(m).
  • The bootstrap distribution's mean sits close to the point estimate; a large offset means the resampled arms have drifted in size.
  • Doubling B narrows the Monte Carlo noise in the percentile endpoints without shifting the interval's centre.

Follow-up

  • Users average 3.4 sessions and the intraclass correlation is 0.12. What width ratio do you predict before running it, and does your bootstrap land there?
  • Give the delta-method standard error for this ratio and say when you would prefer it to the bootstrap.
  • Half the users in the treatment arm have exactly one session. What does that do to the cluster bootstrap's coverage, and how would you check it?

Find reactivation gaps in account paid-period history

Hard · gaps-and-islands · window-functions · subscriptions

fct_subscription_period holds account_id, subscription_id, period_start_utc, period_end_utc, period_status and change_reason. A mid-period plan or seat change closes one row and opens another, so a single continuous paid tenure is often many rows, and an account may hold two overlapping subscriptions. Collapse rows with period_status in ('active','past_due') into continuous tenures per account, treating gaps of three days or less as continuous. Return account_id, tenure_start, tenure_end, and for every tenure after the first, the gap in days that preceded it.

Approach

  1. Filter to paid rows only: period_status IN ('active','past_due'). Trialing periods are not tenure, and including them turns every trial that never converted into a one-period tenure followed by a fake churn.
  2. Order by period_start_utc and take a running maximum of all prior ends: MAX(period_end_utc) OVER (PARTITION BY account_id ORDER BY period_start_utc, period_end_utc, subscription_id ROWS BETWEEN UNBOUNDED PRECEDING AND 1 PRECEDING). Those three columns are the only stable ordering this schema exposes, so check first that they are unique within an account; if rows tie on all three, the island numbering is order-dependent between runs and you need a real row key before the result is reproducible.
  3. LAG on its own is wrong here because with overlapping or nested periods the immediately preceding row by start date is not the one that ends latest, so the running maximum is the part that cannot be shortcut.
  4. Flag a new island when prior_max_end IS NULL OR period_start_utc > prior_max_end + interval '3 days', then number islands with a running SUM of the flag over the same ordering and an explicit ROWS frame.
  5. Group to (account_id, island) taking MIN(period_start_utc) and MAX(period_end_utc), then LAG(tenure_end) OVER (PARTITION BY account_id ORDER BY tenure_start) to compute the preceding gap in days for every tenure after the first.
  6. Sanity-check with change_reason, which is the only lineage this schema carries: list its distinct values first, then confirm that rows recording a plan or seat change sit inside a tenure rather than opening one, and that every tenure after the first opens on a row whose reason records a restart rather than an ordinary renewal. Do not reconcile against a churn timestamp on dim_account, which this schema does not define; and where such a column does exist, a cancellation timestamp records when the request was made and routinely sits weeks before the period it ends.

Follow-up

  • Why three days of grace? What do 0 and 30 days each do to the count of accounts classed as reactivated?
  • An account runs two concurrent subscriptions for different teams. One tenure or two, and what does the revenue reader expect?
  • How would you turn these tenures into a monthly gross logo churn series without double-counting an account that churned and returned in the same month?

Read an experiment from first exposure, not assignment

Hard · experiments · window-functions · srm

fct_experiment_exposure holds experiment_id, unit_type, unit_id, variant, user_id, assigned_at_utc, first_exposed_at_utc, is_in_analysis_population and planned_end_utc. fct_event holds user_id, occurred_at_utc, is_core_action, and carries events up to a known data cut, :data_cut_utc. For one experiment randomised on unit_type = 'user', return per variant: exposed units, units with at least one core action in the seven days after that unit's own first exposure, the rate, and the variant share of exposed units. Only units whose seven-day window has fully elapsed as of the data cut belong in the readout. Units appearing under more than one variant are excluded from both arms and counted separately.

Approach

  1. Run the contamination pass as an aggregate, not a window: SELECT unit_id FROM fct_experiment_exposure WHERE experiment_id = :exp GROUP BY unit_id HAVING COUNT(DISTINCT variant) > 1, then anti-join it away. PostgreSQL rejects COUNT(DISTINCT variant) OVER (PARTITION BY unit_id) outright, since DISTINCT is not implemented for window functions; if you want the test inline, MIN(variant) OVER (PARTITION BY unit_id) <> MAX(variant) OVER (PARTITION BY unit_id) is the equivalent that does run.
  2. Do not resolve contamination by keeping the earliest variant. A unit that saw both arms carries treatment from both, so assigning it to either one biases that arm.
  3. Define the population as is_in_analysis_population = TRUE AND unit_type = 'user' AND first_exposed_at_utc < planned_end_utc AND first_exposed_at_utc + interval '7 days' <= :data_cut_utc. The horizon filter is what makes the readout reproducible next week instead of drifting with every re-run; the data-cut filter is the one that actually buys seven days of follow-up, since a unit exposed an hour before the horizon otherwise contributes an hour of observation to a seven-day rate.
  4. Measure the outcome on a per-unit relative window: LEFT JOIN fct_event on user_id with is_core_action = TRUE and occurred_at_utc in [first_exposed_at_utc, first_exposed_at_utc + interval '7 days'). LEFT JOIN so units with no outcome stay in the denominator at zero rather than being deleted by an inner join.
  5. Check the sample ratio before reading the effect: variant share of exposed units against the intended split, tested as a binomial. Run it on the truncated population as well as on the full exposed set, because if one arm exposes later on average the data-cut filter removes more of that arm and can manufacture a ratio mismatch the randomisation did not have. A mismatch on the full set means the exposure data is not a valid randomisation and invalidates the readout rather than being a footnote under it.
  6. Report the per-variant rate, the absolute difference, and the fact that the variance unit is unit_id. That is straightforward here only because the grain is already one row per user; a per-session outcome under user randomisation would need a delta-method or bootstrap standard error instead.

Worked solution (35 min)

  1. Count units per variant and units with more than one variant; report the contaminated count before anything else.
  2. Build the exposed population CTE with both the horizon filter and the seven-days-before-data-cut filter, assert one row per unit_id, and record per variant how many units the second filter removed.
  3. LEFT JOIN the bounded outcome events and flag conversion with a boolean aggregate.
  4. Aggregate per variant to exposed, converted, rate and share, and run the binomial check on the share, both before and after the truncation.
  5. Re-run with a fixed calendar window from experiment_start_utc and record the difference as the size of the dilution you avoided.

Expected result: One row per variant with exposed, converted, rate and share, plus a scalar count of excluded contaminated units. Every retained unit was first exposed at least seven days before the data cut, so each one has a full seven days of observation and the rates are directly comparable.

Verify it yourself

  • Exposed counts across variants sum to the population count minus the contaminated units.
  • MAX(first_exposed_at_utc) among retained units is at least seven days before :data_cut_utc.
  • Variant shares sum to 1.
  • No unit_id contributes to two variant rows.
  • The truncation drops a similar share of each arm; if it does not, note the imbalance alongside the SRM result rather than reading the effect.
  • Swapping the relative window for a calendar one changes the rates; if it does not, exposure timing is balanced and you can say so.

Follow-up

  • Some units were assigned days before they were exposed. What does analysing the assigned set instead do to the estimated effect, and in which direction?
  • The split is 51/49 on 400,000 exposed units. Do you read the result?
  • The treatment arm exposes on average two days later than control. What does that do to a fixed calendar outcome window, and which arm does it favour?

What metrics would you use to evaluate the success of an internal dashboard built for property managers?

Medium · product sense and metrics

Approach

  1. Name one primary metric, then the guardrail that stops it being gamed.
  2. Fix the population and the time window before naming any metric.
  3. Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

  • What would you do if the primary metric and the guardrail moved in opposite directions?
  • How would you detect that the metric is being gamed rather than genuinely improving?

Reminder volume where the guardrail opposes the primary metric

Medium · engagement · guardrails · trade-offs

The growth team proposes tripling weekly reminder email volume. The north star is weekly active accounts completing a core action, counted on account_id from fct_event (is_core_action, account_id, occurred_at_utc, surface). Its stated guardrail is week-over-week repeat rate together with notification opt-out and unsubscribe rates. Reminders will move the primary up and the guardrail down, by design, and both effects are real. Define the decision rule before the test runs: what magnitudes make this a ship and what makes it a stop. Deliverable: the rule, including the exchange rate you are using between the two quantities.

Approach

  1. Name the conflict precisely rather than calling it a balance: the reminder buys one week of an account returning and spends permission to contact that account, and permission is not renewable, so the two quantities are not comparable as percentage points.
  2. Put both sides into one unit before arguing about thresholds. Value the primary gain as incremental core-action weeks over the horizon; value an opt-out as the forgone email-driven active weeks over that account's remaining expected lifetime.
  3. Read opt-out as a stock, not a flow: accumulate it over the test, because a weekly opt-out rate that looks small is a cumulative curve that only ever rises within a fixed set of contactable accounts.
  4. Run long enough for the novelty to decay and use the late number in the trade: compare week-1 lift with week-4 lift, state the decay you observed, and refuse to price the decision on a week-1 read.
  5. Write the outcome as two numbers and a default action, including what happens when the result lands between them — hold and test a smaller volume increment rather than shipping on ambiguity.

Worked solution (30 min)

  1. Measure the starting stock: current opt-out and unsubscribe rate per 1,000 contactable accounts, and the share of weekly active accounts whose session began from referrer_channel = 'email'.
  2. Estimate the two effects separately over four weeks: incremental weekly active accounts per 1,000 extra sends, and incremental cumulative opt-outs per 1,000 extra sends.
  3. Convert opt-outs into forgone active weeks: email-driven active weeks per account per year multiplied by remaining expected account lifetime, and set that against the incremental active weeks bought.
  4. Compare week-1 with week-4 lift, state the decay rate, and carry the week-4 figure into the trade.
  5. Write the rule as two thresholds plus a default action for the middle case.

Expected result: A written ship/stop rule containing a numeric exchange rate between one incremental weekly active account and one opt-out, derived from an observed lifetime and an observed decay rather than asserted.

Verify it yourself

  • The cumulative opt-out curve within a fixed set of contactable accounts must be non-decreasing. A dip means the denominator is being recomputed each week and the metric is not a stock.
  • Week-4 incremental lift should be materially below week-1. If it is not, either the horizon is too short to see decay or the arms are contaminated by cross-sends.
  • Repeat rate must be computed on the same accounts across consecutive weeks, not on the weekly active set, whose composition changes every week by construction.

Follow-up

  • Opt-out is flat but week-over-week repeat rate falls. What is the most likely mechanism, and does it change the decision?
  • You do not have a year of data to estimate remaining account lifetime. What do you substitute, and how do you keep the answer honest about that?
  • Does randomising on account rather than on user change either the readout or the size of the test?

Attribute a flow-completion drop to one client build

Medium · release-regression · instrumentation · staged-rollout

Core-flow completion rate on surface = 'ios', defined as distinct flow_instance_id reaching 'flow_completed' within 30 minutes of 'flow_started' with no 'error_shown' on the same id in between, fell from 78% to 71% over four days. A new iOS build began a staged rollout on day one. Using fct_event columns flow_id, flow_instance_id, app_version, event_name, user_id and occurred_at_utc, produce the completion rate per build per day, size the loss in absolute completions, and say whether the fix is a rollback. Treat the possibility that the instrumentation changed rather than the flow.

Approach

  1. Cut the rate by app_version and day, each build against its own denominator. A pooled series during a staged rollout is a weighted average whose weights move daily, so it declines in proportion to the rollout share even when the old build is perfectly flat; confirming that proportionality is itself the evidence the regression is build-specific rather than environmental.
  2. Test the denominator before believing the rate. Compute flow_started events per distinct user_id on each build: if the new build re-mints flow_instance_id on retry, or fails to carry the id from start to completion, the denominator inflates and the numerator falls with nothing changing for the user.
  3. Normalise to a user-level outcome that is immune to the id question, namely completions per distinct user_id per day on each build. This is the number that says whether anyone actually failed to finish.
  4. Check the error path: pull error_shown counts and their properties for flow_instance_id values on the new build. A genuine regression produces errors; an instrumentation break produces missing completions with no corresponding error volume, and the two prescribe different fixes.
  5. Size the real loss as the per-user completion gap multiplied by daily users on the new build, and give the rollback recommendation conditional on the error evidence rather than on the rate.

Follow-up

  • The rollout is at 45% and product wants to go to 100% tomorrow. What do you say, and what would you need by when?
  • If the flow_instance_id is genuinely being re-minted, what is the correct historical treatment of the four affected days?
  • How would you have caught this on day one instead of day four?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

  • For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
  • For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
  • Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

  • Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
  • List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
  • Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

  • Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
  • Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
  • Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

  • Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
  • Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
  • Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

  • Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
  • Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
  • Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

  • Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
  • For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
  • Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

  • Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
  • Listen back and mark every moment you proposed a solution before the success metric existed.
  • Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Ambiguous requests are the test

Most of the questions in this section reduce to one thing: can you be handed a vague request and come back with something useful? Prepare an example where the ask was underspecified, you chose an interpretation, and you said out loud which interpretation you chose. Describing how you narrowed the question matters more than the technique you eventually used.

Walk me through the most complex data science project on your resume from start to finish.

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. State the situation in two sentences and spend the rest on your reasoning.
  3. Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

  • How did you know the outcome was caused by your change?
  • What did you decide not to do, and why?

How do you handle situations where project requirements are vague or constantly changing?

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. Quantify the outcome, including what you would not claim credit for.
  3. Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

  • How did you know the outcome was caused by your change?
  • What did you decide not to do, and why?

Handle a request for numbers supporting a decision already made

Hard · integrity · framing · stakeholder

A senior leader has already decided to sunset a plan tier and asks you for the analysis showing it is the right call. Accounts on that tier carry 6% of MRR at constant FX and have the highest licensed-seat utilisation in the book. The leader's support matters to your next review cycle, and the decision is being presented in four days. Deliver what you produce, what you decline to produce, and the exact sentence you will say in the meeting where the number appears on a slide.

Approach

  1. Recognise what is being probed: whether you can find the legitimate request inside an illegitimate framing instead of either complying or refusing on principle. The generic answer promises to push back; the strong one produces something genuinely useful and states its limits in the room, without ambushing anybody.
  2. Separate the decision from the justification. Sunsetting the tier may be correct for reasons the data does not hold, such as support cost, roadmap surface area or sales motion. What you decline is a one-sided document. What you produce is the case read both ways, which also happens to be more useful to the leader.
  3. Build the symmetric analysis: MRR at risk at constant FX, the share of affected accounts with a plausible migration path given seats_licensed and billing_term, the recovery rate assumed for that migration and where it came from, and the downside case in which high-utilisation accounts treat the sunset as a reason to re-evaluate the vendor entirely.
  4. Surface the inconvenient fact privately and early. The highest seat utilisation in the book is a retention signal, and the leader should hold it before the room does, so they can incorporate it rather than be caught by it.
  5. Agree the meeting sentence in advance with the leader, so that nobody is surprised. Something to the effect that the tier is 6% of MRR and its accounts are the most heavily used in the book, and that the case for sunsetting rests on cost and focus rather than on revenue. That is true, it supports the decision on its real grounds, and it stops the deck claiming the numbers endorse it.
  6. Decide your own line before you need it: what you will not put your name to, and that the route if asked anyway is your own manager rather than a confrontation in the meeting.

Follow-up

  • The deck circulates with your analysis included and the downside case removed. What do you do, and by when?
  • What changes if the honest analysis says the sunset is clearly the wrong call?
  • How do you write the same memo when the leader is your skip-level and the meeting is tomorrow?