cswork / libraryOpen practice workspace

Interview guides · Universal Orlando Resort

Universal Orlando Resort Data Scientist Interview Questions & Guide 2026

Data scientist interview prep for Universal Orlando Resort: original practice on experiment design, SQL and metric reasoning, with worked solutions.

The 60-second brief

The role of a Data Scientist at Universal Orlando Resort is vital as it directly influences decision-making and strategic initiatives across the organization. You will leverage data to enhance guest experiences, optimize operational efficiency, and drive revenue through data-driven insights. By analyzing vast datasets from various sources, including visitor behavior, park operations, and marketing campaigns, you will contribute to creating unforgettable experiences for guests and ensuring the resort's continued success.

Allocate prep to your weakest link rather than your favourite topic. Of the three things that usually gate the outcome (SQL that is correct under messy joins, sound reasoning about experiments, and structured framing of an open-ended problem), candidates tend to over-invest in modelling theory and under-invest in framing.

Focus areas

  • Separate cancelled, no-show and completed booking states
  • Model demand against availability, not observed bookings
  • Decompose rate changes into mix and within-segment

Overview

This position is not only critical for day-to-day operations but also offers the opportunity to work on complex, large-scale projects that require innovative thinking and analytical prowess. As a Data Scientist, you may collaborate with teams focusing on guest engagement strategies or operational improvements, employing machine learning models to predict trends and inform business strategies. Your work will have a tangible impact on how guests perceive and interact with the resort, making this role both exciting and rewarding.

The Universal Orlando Resort interview process

Universal Orlando Resort candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Virtual Screening

An added round often puts you in front of someone outside the core hiring team: a partner engineer, a product owner, a domain expert, sometimes a more senior manager. The question they are really asking is not whether you can do the work but whether they would trust a number that came from you. That changes what a good answer looks like. Lead with what the decision cost and what it changed, keep the method available but not central, and be plain about the limits of your evidence. Overstating a result is the fastest way to lose this round.

What to demonstrate

  • Whether you can explain a technical choice to someone who will never read your code, without either flattening it into nothing or hiding inside jargon
  • Honesty about evidence strength: what the analysis establishes, what it only suggests, and what it cannot say at all
  • How you take disagreement, specifically whether you update on a good objection, hold your position with reasons, or fold on contact

How to prepare

  • Write the two-sentence version of your most technical project for a non-specialist, then check that neither sentence needs a method name to make sense.
  • For one result you are proud of, write the strongest objection someone could raise and a response that concedes the part of it that is correct.
  • Prepare one decision that turned out to be wrong: how you found out, what it cost, and what you changed afterwards. A senior cross-functional interviewer asks for this more often than a technical one does.

Presentation

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

  • Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
  • Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
  • Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

  • For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
  • Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
  • For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Onsite Interview

Where a loop ends with a senior leader, that conversation is rarely another skills test. The technical signal already exists by then, so the questions tend to open up: what you would look at first, where a metric you have heard about could mislead, what you would push back on. The decision being made is scope, which in practice means level and how much you would be trusted to own unsupervised. Treating it as a formality is the usual mistake. An open question late in the day is still being scored, and a vague answer reads as someone who has not run anything themselves.

What to demonstrate

  • Whether your view of the business has anything specific behind it, given that you are working only from what is public and are expected to say so
  • Whether the scope of work you describe owning matches the scope of the role, instead of sitting a level below it
  • Whether you can disagree with something concrete and stay useful about it, rather than agreeing with everything said in the room
  • Whether your questions are ones only this person could answer, as opposed to ones the recruiter already covered

How to prepare

  • Build one view you could defend for two minutes using only public information: what the funnel probably looks like, which metric likely drives decisions, and where that metric could mislead. Being wrong for a stated reason survives this round; having no view does not
  • Write down the largest piece of work you have owned from question to decision, who else touched it, and what you decided alone, then check that it reads at the level you are interviewing for
  • Prepare one thing you would want changed if you joined and phrase it as a question rather than a verdict, so it opens a conversation instead of closing one

Where candidates lose points

Fitting demand or price elasticity on observed bookings, when availability and restrictions censor the data

Bookings equal the minimum of demand and what was actually sellable, so a sold-out date records the capacity, not the demand behind it, and the censoring is worst precisely on the highest-demand dates. Meanwhile price is set from a forecast of that same demand, so high-demand dates carry high prices and the raw correlation between price and bookings is biased toward zero and frequently comes out positive, which reads as 'raising price increases demand'. Restrictions compound it: a minimum-length-of-stay rule or a closed-to-arrival flag suppresses bookings with no price movement at all, so the effect lands on the price coefficient if the restriction is not in the model. Join fct_rate_availability_snapshot at snapshot_date equal to the search or booking date, restrict the estimation sample to unit-dates that were genuinely open, carry the restriction flags as controls, and lean on an instrument or a deliberate price experiment before quoting an elasticity.

Running per-user experiments on shared, finite inventory

Independence between units fails when they compete for the same rooms on the same dates: a unit booked by a treatment user is removed from the control users' result sets, so the control arm is degraded by the treatment and the measured lift is inflated, sometimes to the point where a neutral change reads as a clear win. The effect is largest exactly where it matters most, in constrained markets and peak dates, and it is invisible in the usual diagnostics because both arms look balanced on pre-period covariates. Supplier-side treatments have the same problem in reverse, since a pricing or ranking change applied to some inventory changes the alternatives that every traveller sees. Randomise at the level that contains the competition, which is normally market-by-stay-week or a switchback on a market, and power the test on the number of clusters, not the number of users.

Reporting a p-value with no effect size or interval

Give the estimated difference with a confidence interval in the units the business cares about, then say whether that whole interval is worth acting on. A p-value only addresses whether you can rule out exactly zero; it says nothing about magnitude.

Never asking what decision the analysis will inform

Open with who makes the decision, what the options are, and by when. The answer determines the precision you need, the segments worth cutting, and whether an observational read suffices or an experiment is required.

Questions & practice

How do you approach model selection and validation?

Medium · machine learning and modelling

Approach

  1. Frame the prediction: the label, the moment of prediction, and the action it triggers.
  2. Set a baseline first, so any model has something honest to beat.
  3. Check what information would not exist at prediction time, and exclude it.

Follow-up

  • How would you choose the decision threshold, and who owns that choice?
  • Where could label leakage enter this setup?

What machine learning algorithms are you most familiar with, and when would you use them?

Medium · machine learning and modelling

Approach

  1. Set a baseline first, so any model has something honest to beat.
  2. Check what information would not exist at prediction time, and exclude it.
  3. Frame the prediction: the label, the moment of prediction, and the action it triggers.

Follow-up

  • How would you choose the decision threshold, and who owns that choice?
  • What would you monitor after launch to know the model is still valid?

Simulate an overbooking limit against rate-plan no-show risk

Medium · simulation · monte carlo · inventory · probability

One property, one stay date, 40 identical units. Historical traveller no-show plus same-day cancellation rates by rate plan: non-refundable 2%, partially refundable 6%, refundable 11%. Confirmed bookings arrive in the mix 50% non-refundable, 20% partially, 30% refundable. Relocating a displaced guest costs 220 USD; an unsold unit costs 180 USD of foregone rate. Simulate to choose the number of bookings N (N at least 40) that minimises expected cost, report the Monte Carlo standard error at the chosen N, and state the assumption your simulation makes about independence.

Approach

  1. Write the cost function explicitly before simulating: cost = 220 * max(arrivals - 40, 0) + 180 * max(40 - arrivals, 0). The asymmetry is the whole problem, so name the ratio 220/180 early; it is what pushes the optimum past the point where expected arrivals equal capacity.
  2. Draw a single uniform matrix of shape (n_sims, N_max) once and assign rate plans to booking slots in an interleaved order, so that any prefix of N columns preserves the 50/20/30 mix. Reusing those same draws for every candidate N is common random numbers: it makes the cost differences between adjacent N far more precise than the level of cost at any one N.
  3. Sweep N from 40 to 48, compute mean cost and the standard error as std/sqrt(n_sims), and only then declare an argmin. If adjacent N values differ by less than a couple of standard errors, say so instead of picking one.
  4. Sanity-check the simulation against arithmetic: expected shows per booking is 0.500.98 + 0.200.94 + 0.30*0.89 = 0.945, and a normal approximation to the Poisson-binomial with that mean and variance should land on the same optimum.
  5. State the independence assumption and where it fails: a weather event, a flight disruption or a group booking split across several reservations correlates no-shows, which fattens both tails and moves the optimum toward capacity. Independent Bernoullis understate the chance of a costly relocation day.

Worked solution (30 min)

  1. Build a length-48 plan vector in interleaved 50/20/30 order and a matching show-probability vector (0.98, 0.94, 0.89).
  2. Draw U = rng.random((200_000, 48)) once; shows = U < p_show broadcast across columns.
  3. For each N in 40..48: arrivals = shows[:, :N].sum(axis=1); cost = 220np.maximum(arrivals-40,0) + 180np.maximum(40-arrivals,0); record mean and std/sqrt(n_sims).
  4. Cross-check with the normal approximation: mean arrivals = 0.945N, variance = N * 0.05045 under this mix.
  5. Report the argmin, the cost curve around it, and the standard error.

Expected result: N = 42 minimises expected cost, at roughly 220 USD per night, against roughly 270 at both N = 41 and N = 43, so the curve is nearly symmetric around the optimum and rises steeply beyond it (about 400 USD at N = 44). Expected arrivals at N = 42 is 39.7. With 200,000 draws the Monte Carlo standard error is well under 1 USD, so the ranking of 41, 42 and 43 is resolved rather than noise.

Verify it yourself

  • Mean arrivals at each N equals 0.945 * N to within simulation error
  • The normal approximation to the Poisson-binomial picks the same argmin
  • Setting both cost parameters equal moves the optimum toward the N where P(arrivals > 40) is about 0.5
  • Quadrupling n_sims halves the standard error; the argmin does not move

Follow-up

  • How does the optimum move if relocation cost rises to 600 USD, and what is the general rule relating the two costs to the optimal exceedance probability?
  • The no-show rates are estimated from 800 historical nights. How would you propagate that estimation uncertainty into the choice of N?
  • Give a design that would let you measure the true relocation cost, including the parts that do not appear on an invoice.

Rank market-weeks by qualified intents that never converted

Medium · anti-join · null-semantics · supply-gaps

Using the qualifying-intent definition over fct_search (trip_intent_key, searched_at_utc, destination_market_id, check_in_date, results_returned_count, is_bot_flagged) and fct_booking (trip_intent_key, booked_at_utc), produce a supply-acquisition list: the 20 (destination_market_id, ISO week of check_in_date) pairs with the most qualifying intents that never converted within seven days of their first search. Note that fct_booking.trip_intent_key is NULL for deep-link, offline and partner-channel bookings. Return market, check-in week, non-converting intents and total intents.

Approach

  1. Reuse the qualifying-intent set: non-bot, results_returned_count > 0, collapsed to one row per trip_intent_key carrying its first search timestamp and the check_in_date from that first search.
  2. Write the anti-join as NOT EXISTS, correlated on trip_intent_key with the seven-day window inside the subquery. NOT EXISTS is NULL-safe and lets the planner choose an anti-join.
  3. If you insist on NOT IN, add WHERE trip_intent_key IS NOT NULL to the inner query. Without it, one NULL in the subquery makes the predicate UNKNOWN for every outer row and the result set is empty.
  4. Group by destination_market_id and DATE_TRUNC('week', check_in_date), counting both non-converting and total intents so the reader sees volume and share together.
  5. Order by non-converting intents descending and limit to 20. Ordering by share alone promotes market-weeks with five intents and one miss.

Worked solution (25 min)

  1. Run the inner subquery alone and count its NULLs; that count is the reason the NOT IN form fails and is worth stating out loud before fixing it.
  2. Build the qualifying-intent CTE and record its row count as the control total for the rest of the query.
  3. Write the NOT EXISTS anti-join and confirm that non-converting plus converting equals the control total exactly.
  4. Aggregate to market-week, order by volume, and limit.

Expected result: Twenty rows of (destination_market_id, check_in_week, non_converting_intents, total_intents); the NOT IN variant of the same query returns exactly zero rows, which is the diagnostic worth demonstrating.

Verify it yourself

  • non_converting_intents plus converting_intents equals the total qualifying intents for the same window.
  • No row has non_converting_intents greater than total_intents.
  • Rewriting the anti-join as LEFT JOIN with an IS NULL test gives identical counts; if it does not, the seven-day predicate has drifted into the WHERE clause and turned the LEFT JOIN into an inner join.

Follow-up

  • Which of these market-weeks are a supply shortage and which are a restriction or price problem? Which column separates them, and what join do you need?
  • You run the query and get zero rows. Walk me through how you find out why, without changing the query at random.
  • How would you exclude intents that did convert under a different trip_intent_key, such as the same traveller shifting their dates by a week?

Pick the last availability snapshot before each stay date

Easy · window-functions · dedup · as-of-join

fct_rate_availability_snapshot is keyed on (supply_unit_id, stay_date, snapshot_date) and carries units_available, units_sold_to_date, units_blocked, listed_price_usd, is_closed_to_arrival and min_length_of_stay. For every supply_unit_id and every stay_date in one target month, return exactly one row: the values from the latest snapshot_date that is on or before that stay_date. Output supply_unit_id, stay_date, snapshot_date, units_available, units_sold_to_date and listed_price_usd. A single global maximum snapshot_date is not acceptable.

Approach

  1. Restrict to the target stay_date range and to snapshot_date <= stay_date inside the same CTE; that predicate is evaluated per row, which is what makes this an as-of pick rather than a latest-snapshot pick.
  2. Rank with ROW_NUMBER() OVER (PARTITION BY supply_unit_id, stay_date ORDER BY snapshot_date DESC) and keep rank 1. RANK() would return two rows if the table ever holds a duplicate snapshot_date; ROW_NUMBER guarantees one.
  3. Filter on the rank in an outer query, or use DISTINCT ON / QUALIFY where the engine supports it. A window function cannot be referenced in the WHERE clause of the SELECT that computes it.
  4. Leave unit-dates with no qualifying snapshot out of the result and say so explicitly; if the deliverable needs them present, LEFT JOIN from a unit-by-date spine so the absence surfaces as a NULL row instead of a missing one.
  5. Keep snapshot_date in the output, because every downstream consumer needs to know how stale the as-of value is before acting on it.

Follow-up

  • Now pick the snapshot as at exactly 30 days before arrival instead. What changes in the query, and which of the two do you use for a pace report?
  • The table is partitioned on snapshot_date. How do you keep partition pruning while still ranking within each unit-date?
  • You find two rows sharing a snapshot_date for one unit-date. What does that mean about the capture pipeline, and how do you break the tie deterministically?

Propose a method to evaluate the effectiveness of a marketing campaign using data.

Medium · product sense and metrics

Approach

  1. Restate the decision this analysis has to support, and who acts on the answer.
  2. State what result would change your recommendation, so the answer is falsifiable.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

How do you prioritize competing tasks and deadlines?

Medium · product sense and metrics

Approach

  1. Decompose the metric into the rates that drive it, and say which one you would check first.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • What would you do if the primary metric and the guardrail moved in opposite directions?

If tasked with forecasting attendance for a new attraction, what data would you consider?

Medium · product sense and metrics

Approach

  1. State what result would change your recommendation, so the answer is falsifiable.
  2. Name one primary metric, then the guardrail that stops it being gamed.
  3. Fix the population and the time window before naming any metric.

Follow-up

  • How would you detect that the metric is being gamed rather than genuinely improving?
  • Which segment would you cut first, and what would that rule out?

Given a dataset of guest feedback, how would you identify trends to improve guest satisfaction?

Medium · product sense and metrics

Approach

  1. Fix the population and the time window before naming any metric.
  2. Restate the decision this analysis has to support, and who acts on the answer.
  3. State what result would change your recommendation, so the answer is falsifiable.

Follow-up

  • Which segment would you cut first, and what would that rule out?
  • How would you detect that the metric is being gamed rather than genuinely improving?

Explain a project where your data-driven insights led to significant business improvements.

Medium

Approach

  1. Say what you would check first and why it is the highest-information step.
  2. Clarify what is being asked and what a complete answer would contain.
  3. State your assumptions explicitly before working the problem.

Follow-up

  • How would you know your answer was wrong?
  • What assumption would you test first?

Measure stay quality when only some stays leave reviews

Medium · proxy metrics · selection bias · quality

Ranking and supplier suspension both consume a quality score. The only direct quality signal is fct_booking.review_score on a 1-10 scale, with review_submitted_at_utc, and reviews exist only for bookings reaching booking_status = 'stayed'. NULL means no review, never zero. Roughly a quarter of completed stays are reviewed, and dim_supply_unit.rating_count is near zero for new supply. Define the quality metric a suspension decision can use, state explicitly what the reviewed population is a biased sample of, and say what that bias does to a decision made on it.

Approach

  1. Name the estimand before the estimator: expected stay quality for a randomly chosen future stay at this supply unit. That quantity is not in any table, and review_score is a proxy for it.
  2. Enumerate the selection mechanisms rather than gesturing at bias. Only stayed bookings can review, so cancellations, no-shows and supplier-caused failures are structurally absent from the sample. Response is non-random and skews toward extreme experiences. And rating_count near zero means the estimate for new supply is dominated by noise rather than by quality.
  3. Choose an estimator that fails safe: a mean of review_score shrunk toward the (property_class, destination_market_id) prior with weight rating_count / (rating_count + k), so a unit with three reviews cannot be suspended on them, and publish rating_count and an interval beside the score.
  4. Add signals that do not pass through reviews at all, because they cover exactly the population reviews cannot see: supplier-caused cancellation rate from fct_booking.cancellation_reason = 'supplier', the no_show rate, and servicing_contacts_count per booking.
  5. State the bias direction on the decision itself: a unit whose failures end in a supplier cancellation rather than a bad stay never generates the bad review, so a suspension rule on review score alone systematically spares that failure mode.

Worked solution (30 min)

  1. Compute review response rate by property_class, supplier_type, rate_plan and lead_time_days bucket, and check whether it varies enough to break comparability across those cuts.
  2. Fit the shrinkage constant k by holding out later reviews and comparing the shrunk estimate's error against the raw mean's error at low rating_count.
  3. Build the combined signal: shrunk review mean, supplier-caused cancellation rate, no-show rate and servicing contacts per booking, each with its own window.
  4. Set a minimum rating_count gate below which no suspension can fire on review score alone, and route those units to the non-review signals.
  5. Write the bias paragraph as a deliverable, naming which population is missing and which direction the omission pushes the score.

Expected result: Quality score = review_score mean shrunk toward the (property_class, destination_market_id) prior with weight rating_count / (rating_count + k), published with rating_count and an interval, gated so no suspension fires below a minimum rating_count. Alongside it: supplier-caused cancellation rate, no-show rate and servicing contacts per booking, covering unreviewed and never-completed stays. Bias stated as: the sample is completed, reviewed stays, which excludes the worst outcomes by construction.

Verify it yourself

  • If review response rate varies materially across property_class or rate_plan, the raw mean is not comparable across those cuts and only the shrunk, segmented version is.
  • Suspension candidates selected on review score alone versus on the combined signal should overlap substantially; if the sets barely intersect, the review-only rule is answering a different question from the one asked.
  • NULL review_score rows must never enter the numerator or the denominator as zeros; verify the count of NULLs separately and confirm it matches stays minus reviews.

Follow-up

  • Review response rate rises 8 points after a prompt change. Do historical comparisons of the score still hold?
  • A unit has rating_avg 9.4 on 6 reviews and an 11 percent supplier-caused cancellation rate. What do you do?
  • How would you put a size on the non-response bias without running a survey?

Occupancy spiked in four markets on one snapshot date

Medium · pipeline-failure · denominator · data-quality · geography-cut

Occupancy, computed as stayed nights from fct_stay_night over sellable nights from fct_rate_availability_snapshot (units_sold_to_date + units_available at the latest snapshot_date on or before stay_date), jumped from 71% to 94% in four destination markets for stay dates in a two-week band, and is normal everywhere else. Stayed nights are unchanged. You have fct_rate_availability_snapshot (supply_unit_id, stay_date, snapshot_date, units_available, units_sold_to_date, units_blocked) and dim_supply_unit (listing_status, supplier_id, supplier_type, destination_market_id). Find the cause and state what the true occupancy was.

Approach

  1. The numerator is given as unchanged, so all of the work is in the denominator. Count DISTINCT supply_unit_id per snapshot_date per market and compare it against the count of units with a sellable listing_status in dim_supply_unit. A missing feed shows up as a step down in coverage, not as a change in any per-unit value.
  2. Check row counts per partition. The snapshot table is partitioned on snapshot_date, so a failed load leaves a thin or absent partition and the as-of join quietly falls back to an older snapshot whose units_sold_to_date is lower and whose availability is stale.
  3. Separate the two failure shapes, because they push the rate in opposite directions. Units missing from the snapshot entirely shrink the denominator and inflate occupancy; a stale fallback snapshot understates units_sold_to_date and can deflate it. Naming which one you have determines the correction.
  4. Look for mechanically impossible values rather than merely surprising ones. Occupancy above 1.0 on a stay_date, or a supply_unit_id present in fct_stay_night but absent from the denominator for the same date, is proof rather than suspicion.
  5. Recompute with the denominator restricted to unit-dates that carry a genuine snapshot on the intended date, and publish coverage alongside the corrected rate so the number carries its own caveat instead of needing a footnote.

Follow-up

  • Two of the four markets share a supplier feed and two do not. What does that pattern tell you about where to look next?
  • The pipeline owner wants to backfill the missing partitions. What do you need to see before you republish the corrected series?
  • What single daily check would have caught this before anyone opened the dashboard, and what is its false-positive rate?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

  • Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
  • For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
  • Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

  • Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
  • Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
  • Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

  • Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
  • Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
  • Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

  • Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
  • For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
  • Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

  • Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
  • Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
  • Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

  • Repeat the depth protocol on the second-ranked area with the same six-problem structure.
  • Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
  • Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

  • Re-run the six prompts from day one under the same clock and compare both correctness and time.
  • Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
  • Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Bring numbers, including the ones you are unsure of

An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.

What motivates you to succeed as a Data Scientist?

Medium · behavioural and stakeholder questions

Approach

  1. Close with what you would do differently, concretely.
  2. Name the disagreement or constraint, and how you resolved it with evidence.
  3. Pick a story where you drove the decision, not one where you observed it.

Follow-up

  • What would you do differently if you ran that project again?
  • How did you know the outcome was caused by your change?

Describe a challenging situation you faced in a team project and how you handled it.

Medium · behavioural and stakeholder questions

Approach

  1. Quantify the outcome, including what you would not claim credit for.
  2. Pick a story where you drove the decision, not one where you observed it.
  3. Close with what you would do differently, concretely.

Follow-up

  • How did you know the outcome was caused by your change?
  • What would you do differently if you ran that project again?

Retract an occupancy comparison after the decision shipped

Hard · error disclosure · occupancy denominators · accountability

Three weeks ago you published a market occupancy comparison that led to marketing spend being moved out of one market. Reviewing the query, you find the two markets used different denominators: sellable nights from fct_rate_availability_snapshot (units_sold_to_date + units_available) for one, and capacity_units from dim_supply_unit for the other. The second market is mostly individual_host supply with large units_blocked, so its occupancy was understated. The spend move is already live. Prepare the correction: who you tell, in what order, and what the note says.

Approach

  1. Quantify the error before telling anyone, because the first question will be 'how wrong'. Recompute both markets on sellable nights, which excludes units_blocked, and report the corrected gap and its sign, not just that the original was wrong.
  2. Separate the numerical error from the decision error. The comparison was invalid, but the spend move may still have been right; establishing whether the decision would have flipped is a different analysis and it is the one the business needs.
  3. Tell the decision-maker directly and first, before it appears in a dashboard or a peer surfaces it. Order matters because being told by a third party converts a mistake into a credibility problem.
  4. Write the note with the correction, the size, the decision implication, and the reversal cost in that order. Three weeks of moved spend has a real cost to undo, and a correction that does not price the reversal forces the reader to do the work you skipped.
  5. Name the specific control that would have caught it and put it in place in the same note: an assertion in the query that both arms use the same denominator expression, and the denominator named in the chart title. A retraction without a mechanism reads as an apology rather than a fix.

Follow-up

  • The corrected numbers still support the original decision. Do you still send the note, and does it read differently?
  • Your manager suggests quietly fixing the dashboard and not raising it. How do you respond?
  • What would you have had to do differently three weeks ago, in the query itself, rather than in your review habits?