Interview guides · Closer Consulting

Closer Consulting Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Closer Consulting: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

As a Data Scientist at Closer Consulting, you serve as a critical bridge between complex data landscapes and actionable business strategy. You are not just building models; you are solving real-world challenges for a diverse range of clients, often operating within high-impact, fast-paced project environments. Your work directly influences how organizations interpret their data, optimize their operations, and make data-driven decisions that shape their future.

Seniority shifts the scope more than the words in the title do. Earlier-career loops mostly check that you execute a well-posed analysis correctly; senior loops check that you can decide which question is worth answering and defend what you chose not to do.

Focus areas

Cluster inference at the account, not the engagement
Compute utilisation against a defended availability denominator
Segment margin by pricing model before comparing

Overview

This role requires a unique blend of technical rigor and consultant-level communication skills. Because Closer Consulting prides itself on its young, dynamic, and professional culture, you will be expected to collaborate effectively with both internal technical teams and external stakeholders. You will find this position intellectually stimulating, as it demands the flexibility to adapt to various client needs while maintaining the high standard of excellence that defines the firm.

The Closer Consulting interview process

Closer Consulting candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Human Resources Screening

Rounds outside the standard loop often open with something deliberately under-specified: a loose business problem, an open question about a product area, a dataset described in one sentence. The common failure is surveying, listing six plausible approaches and committing to none of them. The thing that separates a strong answer is scoping out loud. State what you are treating as the goal, name the metric you would move, say what you are choosing not to do and why, then take one path through to an actual answer. An interviewer can follow you down a narrow path. Nobody can grade a menu.

What to demonstrate

Whether you turn an ambiguous prompt into a stated question with a measurable outcome before doing any work
The judgement visible in what you cut, and whether you say why you cut it rather than silently dropping it
Whether you land on a concrete recommendation with its caveat attached, rather than an unranked set of options

How to prepare

Take three vague prompts, such as 'is this feature working', 'why did retention drop', and 'should we expand into a new segment'. For each, write one sentence of goal, one primary metric with its window, and two things you are explicitly not doing.
Practise giving the recommendation first and the reasoning second, in five minutes. Loosely defined rounds are usually time-boxed, and an answer that arrives last often does not arrive.
Keep a running assumption list as you talk, on paper or in the shared doc, so the interviewer can challenge one assumption instead of your whole answer.

Team Fit Interview

This conversation decides whether you can be handed a problem nobody has finished defining and left alone with it for a few weeks. The manager is listening for how you behave when the brief is thin: what you clarify before starting, and what you settle on your own rather than escalating. Most candidates over-index on technical depth here and under-describe the decisions they actually owned. Say who wanted the work, what you chose not to do, and where you would have stopped and asked. A clean account of your own judgement carries this round further than a longer project list.

What to demonstrate

Whether you can name a decision that was yours alone, as opposed to one the team arrived at
How you respond to a request that arrives with no success metric attached to it
Whether the effort you estimate for a piece of work matches the work you just described doing
What you escalate, and how long you sit on a problem before you do

How to prepare

For each project you plan to raise, write one sentence saying what would not have happened if you had not been on it, and check that the sentence is about an outcome rather than an artefact
List the decisions in your last project that were genuinely yours, and for each one write down the option you rejected and why
Prepare the project that went badly: the point at which you knew, who you told, and what it cost before it was caught

Management Fit Interview

Because the format is not fixed, prepare the reasoning rather than the ritual. Nearly every version of this round draws on the same underlying material: a design you can defend, a metric you can define exactly, an analysis whose assumptions you can state out loud. Only the wrapper changes, whether that is a take-home, a live case, a deep dive on past work, or a rough estimate on a whiteboard. Answers rehearsed to fit one shape stall the moment the shape differs. Practise naming the assumption behind a number, then saying how much the conclusion moves if that assumption is wrong.

What to demonstrate

Whether your justification for a method survives the question 'why not the simpler thing', including when the simpler thing would have worked
Precision under pressure: what exactly counts as an active user, a conversion or a success, over what window, with what exclusions
Whether you carry an argument through to a recommendation instead of stopping at a list of tradeoffs

How to prepare

For each project you plan to mention, write the metric definition in one sentence: numerator, denominator, time window, exclusions. Say it out loud once, because vagueness shows up in speech before it shows up on paper.
Rehearse the same project at three lengths: two minutes, ten minutes, and a deep dive on one technical decision. Cutting live is harder than it sounds.
For your headline result, write down what would have had to be true for it to be wrong, and how you ruled that out.

Technical Discussion

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Where candidates lose points

Treating accounts as independent observations

Revenue is concentrated: a small number of client_ids typically carries a large share of fees, and engagements within one account share a partner, a rate card and a delivery team. Ordinary standard errors computed over engagements therefore understate uncertainty badly. Cluster at client_id, and with fewer than roughly 40 clusters use a wild cluster bootstrap or a CR2 correction, because cluster-robust standard errors are downward-biased in that regime and will manufacture significance that a replication will not reproduce.

Modelling win rate on proposals with a recorded outcome, using fields written after the decision

Two failures compound here. First, stage IN ('withdrawn','no_decision') is not missing at random: those are disproportionately deals that were going to be lost, so training on won-plus-lost only inflates apparent win rate and distorts the coefficients. Second, fields like engagement_id, final scope and revised pricing are populated after the outcome is known, so including them leaks the label and produces a model with excellent backtest accuracy and no forward value. Restrict features to values knowable at submitted_at.

Stopping an experiment the moment it crosses significance

Fix the sample size or duration before launch, or use a method built for continuous monitoring such as a sequential test, always-valid confidence intervals, or group-sequential boundaries. Repeatedly checking a fixed-horizon p-value against 0.05 pushes the real false-positive rate well above 5 percent.

Interpreting a change before checking data quality and logging

Spend the first pass on row volume by day, null rates, duplicate keys, and whether the step change lands on a release or tracking-migration date. A discontinuity that coincides with a deploy is an instrumentation hypothesis before it is a behavioural one.

Questions & practice

Measure the timesheet backfill curve and pick a reporting cutoff

Easy · late-arriving data · data quality · cohort curves

time_entries has work_date (date), entered_at (timezone-aware UTC timestamp), hours and status. Given a snapshot_date, restrict to work_date in [snapshot_date - 180 days, snapshot_date - 60 days] so every cohort is fully observed. For k = 0..45, compute F(k): the share of a work_date cohort's final hours that already existed as of work_date + k days, pooled across cohorts. Return the 46-point curve and the smallest k with F(k) >= 0.99. Some rows are entered before the work date; those lags are real, not errors.

Approach

Compute lag = (entered_at converted to the reporting timezone and taken as a date) - work_date in whole days, then clip negative lags to 0 instead of dropping them; leave and planned time are routinely entered ahead of the work date and dropping them deflates the early curve.
Take cohort totals as groupby(work_date).hours.sum() over the restricted window. These are final only because the window stops 60 days short of the snapshot, which is why the restriction is in the prompt.
Build the numerator by summing hours per (work_date, lag), sorting by lag, taking a per-cohort cumsum, then reindexing each cohort onto the full 0..45 lag grid and forward-filling, so a cohort with no entries at a given lag holds its previous level rather than disappearing.
Pool as sum(numerators) / sum(denominators) at each k, not as the mean of per-cohort shares. Holiday weeks are tiny cohorts and would otherwise carry the same weight as a full week.
Read k* off the pooled curve and report F(45) with it: if F(45) is below about 0.995 the tail runs past the grid and k* is a lower bound, not the answer.

Worked solution (25 min)

Restrict rows to the [snapshot - 180d, snapshot - 60d] window and compute lag_days = (entered_at.dt.tz_convert(tz).dt.normalize().dt.date - work_date).dt.days, then lag_days = lag_days.clip(lower=0).
cohort_total = df.groupby('work_date').hours.sum(); by_lag = df.groupby(['work_date','lag_days']).hours.sum().
Reindex by_lag onto MultiIndex.from_product([cohorts, range(0,46)]), fill 0, cumsum within work_date to get hours_by_k.
F = hours_by_k.groupby(level='lag_days').sum() / cohort_total[cohorts_in_grid].sum(); assert F is non-decreasing.
k_star = int(F[F >= 0.99].index.min()) if any, else report 'not reached within 45 days' along with F(45).

Expected result: A monotone non-decreasing 46-point series F(0)..F(45) bounded by 1.0, plus an integer k* (or an explicit 'not reached by day 45') reported together with the value of F(45).

Verify it yourself

F is non-decreasing in k and F(45) <= 1.0, with equality only if no row in the window has a lag above 45 days.
The pooled curve lies between the minimum and maximum per-cohort curve at every k, since it is a weighted average of them.
One cohort recomputed by hand from raw rows matches the vectorised result.
Removing the clip and dropping negative lags lowers F(0); the difference equals the hours entered ahead of their work date.

Follow-up

The dashboard refreshes daily. Would you hold the window back past k*, or publish an as-of-entered_at series instead, and what does each choice cost the reader?
One practice area has a tail twice as long as the rest. Does that change the firm-wide cutoff, or does it change what you publish per practice area?

Simulate the chance a capped engagement crosses its cap

Medium · monte carlo · bootstrap resampling · pricing caps

A capped time-and-materials engagement has not_to_exceed_usd = 400000, has billed 250000 to date, and bills at a blended bill_rate_usd of 250. Fourteen delivery weeks remain before planned_end_date. You have that engagement's last twenty weekly totals of approved billable hours as a pandas Series. Estimate the probability that cumulative billable value crosses the cap before the planned end, and the expected unbillable hours if it does. Resample the observed weeks; do not assume normality. Report a Monte Carlo standard error and justify the number of draws.

Approach

Reduce the deterministic part first: the cap still affords (400000 - 250000) / 250 = 600 hours, so the whole question is the distribution of the sum of fourteen future weekly hour totals against a fixed threshold of 600.
Before resampling, check the twenty observed weeks for trend and lag-1 autocorrelation. Independent resampling is defensible only if the weeks are exchangeable; on a ramping engagement use a moving-block bootstrap or model the ramp, because i.i.d. draws understate the upper tail, which is exactly the tail the question is about.
Draw one (B, 14) array with np.random.default_rng().choice(observed, size=(B,14), replace=True), sum along axis 1, and compute p_hat = mean(total > 600) in one vectorised pass.
Report expected unbillable hours two ways: unconditional mean(maximum(total - 600, 0)) for expected loss, and the mean conditional on a breach for how bad a breach is when it happens. The second is the number that drives a change-order conversation.
Attach se(p_hat) = sqrt(p_hat(1 - p_hat)/B) and pick B from the precision you need: plus or minus one point at 95% confidence needs about 1.96^2 * 0.25 / 0.01^2, roughly 9600 draws at the worst case p = 0.5.
State the assumptions that would flip the answer: constant blended rate, no scope change, no holiday weeks inside the fourteen, and a cap that applies to fees rather than to fees plus expenses.

Follow-up

Hours past the cap still cost money. Restate the result as expected gross margin rather than a probability.
Two weeks of time are entered but not yet approved. How do you fold them in without double-counting?
A change order that raises the cap is judged 60% likely. How does that change the number you present, and to whom?

Implement billable utilisation against a defended availability denominator

Medium · metric implementation · denominators · proration

Implement billable utilisation for one calendar month at consultant grain, then roll it up to practice_area. Inputs: time_entries (consultant_id, work_date, hours, is_billable, charge_code, status), consultants (consultant_id, practice_area, fte_fraction, home_region, is_billable_role, hire_date, termination_date) and holidays (region, holiday_date). Numerator is approved hours with is_billable true. Denominator is scheduled workdays in the month, clipped to [hire_date, termination_date] and net of that region's holidays, times 8 times fte_fraction, minus approved leave hours. Roll up as sum(numerator) / sum(denominator).

Approach

Build the workday set per region first: business days in the month minus that region's holiday_dates. State the Monday-to-Friday assumption out loud, because a region with a different working week makes np.busday_count wrong rather than approximate.
Prorate by clipping each consultant's interval to [max(month_start, hire_date), min(month_end, termination_date or month_end)] and counting workdays inside the clipped interval, so a mid-month start produces a smaller denominator instead of a fake dip in the ratio.
Subtract approved leave hours (charge_code in leave_paid, leave_unpaid) only for leave falling on days already counted as workdays; leave logged on a public holiday would otherwise be subtracted twice and can push a denominator negative.
Filter to is_billable_role, then drop consultant-months whose denominator is zero or negative (full-month leave, a hire on the last day) rather than emitting inf or NaN, and return the dropped count as part of the result.
Roll up by summing numerators and denominators separately. Averaging per-person ratios gives a 0.2 FTE consultant the same weight as a full-time one and produces a practice number that no individual reconciles to.

Follow-up

Utilisation rose two points this month while realisation fell. Reconcile those two movements with one mechanism.
A regional lead says their team looks four points below another region. What do you check before you answer, and in what order?

Build a client cohort revenue table from invoice lines

Easy · cohorts · date truncation · revenue recognition

dim_client holds client_id and first_engagement_date. fct_engagement maps engagement_id to client_id. fct_invoice_line holds engagement_id, line_type, period_start_date, amount_usd and status; recognised fees are lines with line_type IN ('fees','milestone','credit_note') and status <> 'draft', and credit notes carry negative amount_usd. Return one row per (cohort_month, months_since_first) giving distinct clients billed and net recognised fees. cohort_month is the month of first_engagement_date; months_since_first is the whole-month gap between period_start_date and first_engagement_date. Long format, no pivot. Decide explicitly what to do with clients whose first_engagement_date is NULL.

Approach

Filter fct_invoice_line to recognised lines first (line_type, status <> 'draft'), and key recognition on period_start_date rather than issued_at or paid_at. Billing period, issue date and payment date are three different clocks, and mixing them produces a cohort table nobody can reconcile against finance.
Join invoice lines to fct_engagement to reach client_id, since invoice lines carry engagement_id only, then to dim_client for first_engagement_date.
Truncate both dates to month, then compute months_since_first as 12 * (year(bill_month) - year(cohort_month)) + (month(bill_month) - month(cohort_month)). Doing it on truncated months is what keeps a client billed on day 2 of the third month in bucket 2 instead of bucket 1.
Aggregate to (cohort_month, months_since_first) with SUM(amount_usd) and COUNT(DISTINCT client_id), keeping credit_note lines inside the sum so the figure is net rather than gross.
Handle NULL first_engagement_date as an explicit exclusion and report how many clients and how much revenue that removes, rather than letting the join drop them silently. That exclusion is why the output's total is smaller than the filtered line set's total, so carry the excluded figure as a named number rather than discovering the gap later.

Worked solution (25 min)

Write the recognised-lines CTE with the line_type and status predicates, and check its row count against the unfiltered table so you know how much you excluded.
Join through fct_engagement to dim_client and confirm the join does not multiply rows: a line has exactly one engagement, an engagement exactly one client.
Add the two date_trunc expressions and the integer month arithmetic, then eyeball ten rows against their source dates.
Group and aggregate, then sort by cohort_month and months_since_first.
Run a separate count of clients with NULL first_engagement_date, and a separate sum of their recognised fees, and note both beside the output.

Expected result: A long-format table of (cohort_month, months_since_first, active_clients, net_fees_usd), plus two scalars reported beside it: the count of clients with a NULL first_engagement_date and the recognised fees attached to them. months_since_first is a non-negative integer on every row; a negative value means an invoice period precedes the recorded first_engagement_date, which is a data-quality finding to report, not something to clamp to zero.

Verify it yourself

SUM(net_fees_usd) over the whole result equals SUM(amount_usd) over the filtered lines whose engagement resolves to a client with a non-NULL first_engagement_date. It does not equal SUM(amount_usd) over the full filtered line set, which is larger by exactly the excluded NULL-cohort revenue: add that separately reported figure back and the two agree. Any remainder after that is a line whose engagement_id has no matching row in fct_engagement, which is a referential finding to name rather than absorb.
COUNT(DISTINCT client_id) at months_since_first = 0 within a cohort_month is less than or equal to the number of clients whose first_engagement_date falls in that month.
Zero rows have a negative months_since_first; if any appear, name the clients rather than filtering them out.
Re-running with period_end_date in place of period_start_date moves only lines whose billing period spans a month boundary.

Follow-up

How would you turn this table into a 12-month net revenue retention ratio, and what exactly becomes the denominator?
One account contributes 40 percent of a cohort's month-13 fees. How do you present the cohort so a reader sees that before they act on the ratio?
If subsidiaries roll up to parent_client_id, which first_engagement_date defines the cohort, and does the rollup change the cohort a subsidiary lands in?

Find signed engagements with no approved billable hour

Medium · anti-join · null semantics · not exists

fct_engagement holds engagement_id, start_date and status. fct_time_entry holds time_entry_id, consultant_id, engagement_id, charge_code, work_date, hours, is_billable and status, and engagement_id is NULL on bench, leave, training and internal rows. Return engagements with start_date in the trailing quarter and status IN ('active','closed_delivered','closed_early') that have no approved billable time entry with work_date within 45 days of start_date, giving engagement_id, start_date and days elapsed. Then explain why NOT IN over a subquery selecting fct_time_entry.engagement_id is the wrong instrument here.

Approach

Express the exclusion as NOT EXISTS with a correlated predicate covering all four conditions at once: te.engagement_id = e.engagement_id AND te.is_billable AND te.status = 'approved' AND te.work_date < e.start_date + INTERVAL '45 days'. NOT IN can only carry a single column, so a multi-condition anti-join forces a contorted subquery even before the NULL problem.
State the NULL semantics precisely: NOT IN (SELECT engagement_id FROM fct_time_entry) evaluates to NULL rather than TRUE for every candidate as soon as one NULL is present in the list, because x <> NULL is unknown. The WHERE clause keeps only TRUE, so the query returns zero rows and reads as a clean bill of health. This table guarantees NULLs, since bench and leave rows carry no engagement.
Note the equivalent LEFT JOIN ... WHERE te.engagement_id IS NULL form. It is correct, but it materialises a join only to discard most of it; NOT EXISTS states the intent and most planners execute both as the same anti-join.
Guard the window against truncation: an engagement that started 10 days ago cannot yet have failed a 45-day test, so require start_date <= current_date - 45 or report the too-early set separately rather than counting them as failures.
Decide on the late-timesheet risk explicitly. A missing entry may mean unstaffed work or an unsubmitted timesheet, so either hold the cutoff back past the measured submission lag or label the recent tail as provisional.

Follow-up

Turn this into a time-to-first-billable-hour distribution. What do you do with engagements that never recorded one?
Should client_pursuit hours count as the engagement being staffed, and how does that choice change the list?
How would you prove the empty result from the NOT IN version is a bug rather than good news?

How do you stay updated with the latest trends in the field of data science?

Medium · product sense and metrics

Approach

Fix the population and the time window before naming any metric.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

Can you describe your most significant project in the data science space?

Medium

Approach

Clarify what is being asked and what a complete answer would contain.
Work from the decision backwards to the evidence you would need.
State your assumptions explicitly before working the problem.

Follow-up

How would you know your answer was wrong?
What assumption would you test first?

What motivated you to apply to Closer Consulting specifically?

Medium

Approach

State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.
Clarify what is being asked and what a complete answer would contain.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Why are you looking for a change in your current career path?

Medium

Approach

State your assumptions explicitly before working the problem.
Say what you would check first and why it is the highest-information step.
Clarify what is being asked and what a complete answer would contain.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Measure a staggered process rollout across five practice areas

Hard · difference-in-differences · staggered adoption · wild cluster bootstrap · parallel trends

A delivery-review process was rolled out one practice area at a time over six quarters, and all five practice areas now have it, so there is no never-treated group. You have engagement-quarter gross margin reconstructible from fct_invoice_line and fct_time_entry, plus pricing_model on fct_engagement. Leadership wants the effect on margin. Explain why a two-way fixed effects regression of margin on a treated indicator is not trustworthy here, name the estimator you would use instead and what identifies it, and specify how you would obtain a defensible p-value from five clusters.

Approach

Explain the two-way fixed effects failure concretely. Under staggered timing the coefficient is a weighted average of every available two-by-two comparison, and that set includes comparisons that use already-treated units as controls for later-treated ones. Those comparisons difference out not only the common time shock but also the earlier cohort's ongoing treatment-effect growth, so when effects grow after adoption the aggregate is pulled toward zero and can flip sign. Decomposed onto the underlying ATT(g,t), the implied weights can be negative, which is why the coefficient is not an average of treatment effects at all. This is a bias in the estimand, not a small-sample artefact, so more data does not fix it.
Switch to a heterogeneity-robust estimator: Callaway and Sant'Anna group-time ATT(g,t) using not-yet-treated units as the comparison group, aggregated into an event-study path and a single summary. Sun and Abraham's interaction-weighted estimator or Borusyak-Jaravel-Spiess imputation answer the same objection. With every area eventually treated, identification rests entirely on not-yet-treated windows, so the last-adopting area carries disproportionate weight; say that out loud rather than burying it.
Test parallel trends honestly. Plot event-time leads with confidence intervals and refuse to read a non-rejection as confirmation, because at five clusters that test is badly underpowered. Pair it with a sensitivity analysis that bounds the post-treatment violation as a multiple of the largest pre-period violation and reports the multiple at which the conclusion breaks.
Fix the inference, and be precise about what the bootstrap can deliver at five clusters. Five sits far below the roughly forty at which cluster-robust standard errors become approximately valid, and below that they are sharply downward-biased. A wild cluster bootstrap with Rademacher weights draws from 2^5 = 32 sign vectors, but the null-imposed bootstrap t-statistic is odd in the sign vector: negating every weight negates t* and leaves |t*| unchanged, because the coefficient deviation is linear in the weights while the cluster-robust variance is quadratic in them. The 32 vectors therefore collapse to 2^4 = 16 distinct values of |t*|, so the smallest attainable two-sided p-value is 1/16 = 0.0625. That is above 0.05, so at five clusters a Rademacher bootstrap cannot reject at the conventional level however large the true effect is. Use Webb six-point weights, whose 6^5 = 7,776 vectors collapse to 3,888 distinct |t*| and a floor near 0.00026, or randomisation inference permuting the five observed adoption dates across the five practice areas, which gives 5! = 120 assignments and a floor of 1/120 = 0.0083.
Deal with the mix confound before reporting anything. Fixed-fee margin falls with hours worked while uncapped time-and-materials margin rises with them, so a rollout that coincided with a pricing-mix shift contaminates the pooled estimate. Estimate within pricing_model and report the within-model effects alongside the mix decomposition.

Worked solution (45 min)

Build the engagement-quarter panel with an adoption quarter per practice area, keyed on the quarter the process went live rather than when it was announced.
Run the Goodman-Bacon decomposition of the two-way fixed effects estimate and read off how much of the total weight sits on already-treated-as-control comparisons; that share is the diagnostic there, since those weights are non-negative by construction. For negative weights on the underlying ATT(g,t), run the de Chaisemartin and D'Haultfoeuille decomposition and report the negative share.
Estimate ATT(g,t) with not-yet-treated controls, aggregate to event time, and plot leads and lags with confidence bands.
Compute the p-value three ways, cluster-robust, Rademacher wild bootstrap and Webb wild bootstrap, and report the Webb figure while noting that the Rademacher floor at five clusters is 1/16 = 0.0625 and therefore cannot clear 0.05 at all.
Re-estimate separately for fixed_fee and time_and_materials engagements and report the mix decomposition against the pooled change.

Expected result: An event-study path whose pre-period leads are statistically indistinguishable from zero, a post-period aggregate ATT reported with a Webb wild-bootstrap p-value, an explicit note that a Rademacher bootstrap at five clusters cannot produce a p below 1/16 = 0.0625 and so cannot reject at 0.05 on any data, and separate estimates by pricing model with the mix shift quantified.

Verify it yourself

Goodman-Bacon weights are non-negative and sum to one by construction, so a negative one there is an implementation bug rather than a finding; the number to read is the share of weight resting on already-treated-as-control comparisons. Negative weights on the underlying ATT(g,t) come from the de Chaisemartin and D'Haultfoeuille decomposition, and a large negative share means the two-way fixed effects coefficient should not be reported as an average effect.
Restricted to one adoption cohort and two periods against not-yet-treated units, the group-time estimator and two-way fixed effects must return the identical two-by-two difference in differences; a gap there is an implementation bug, not heterogeneity.
At five clusters, enumerate all 32 Rademacher sign vectors rather than drawing B of them. The resulting |t*| must come back as 16 identical pairs, one per sign-flip orbit; 32 distinct values means the null is not being imposed or the statistic being bootstrapped is not the cluster-robust t.
Wild bootstrap p-values should be weakly larger than cluster-robust ones at five clusters; if they come out smaller, check that the null is being imposed in the bootstrap.
Within-pricing-model effects plus the mix-shift term must reconcile to the pooled period-over-period change.

Follow-up

If the rollout order was not random, say the worst-performing practice area went first, what does that do to parallel trends and what would you do about it?
How would you choose between a synthetic control built on a single practice area and the group-time estimator?
What would the event-study path have to look like for you to believe the effect is real rather than a continuation of a pre-existing trend?

Fees per billable head fell nine percent; rank the causes

Hard · decomposition · seasonality · clustered inference

Recognised fees per billable head fell about 9% quarter over quarter and leadership wants a ranked cause by Friday. You have fct_time_entry, fct_invoice_line, fct_engagement and dim_consultant, plus a workday and regional holiday calendar. Decompose the move into available hours per head, billable utilisation and realised dollars per billable hour, separate calendar effects from behaviour, split within-segment change from mix, and deliver a ranked attribution with uncertainty. Name the one further cut you would run if the leading term is ambiguous.

Approach

Write the identity before computing anything: fees per head = (available hours per head) x (billable utilisation) x (realised dollars per billable hour), where the last term is recognised fees divided by approved billable hours. Verify the product reproduces the reported 9% exactly; if it does not, the scope of the three inputs disagrees and nothing downstream is trustworthy.
Decompose in logs: d ln(fees per head) = d ln(available per head) + d ln(utilisation) + d ln(realised rate). The log form is exactly additive for a product, so contributions sum to the total without an interaction residual and cannot attribute more than the whole move.
Strip the calendar first, because it is the term nobody can act on: available hours per head moves with scheduled workdays, regional public holidays, leave and any change in fte_fraction mix. Quantify how many of the nine points are calendar and quote the remainder as the behavioural move.
Split each surviving term by practice_area, level, region and pricing_model, and within each split separate within-segment change from mix using the weighted three-term decomposition, since a shift toward lower-rate levels reads as a rate decline that no individual rate produced.
Attach uncertainty at the unit that generates it: anything fee-side clusters at client_id because engagements in one account share a partner, a rate card and a team. With fewer than roughly forty clusters use a wild cluster bootstrap or a CR2 correction, since cluster-robust standard errors are downward-biased in that regime.
If realised rate leads and is ambiguous, the disambiguating cut is discount versus write-off versus fixed-fee overrun: compare invoiced fees to the standard-rate value of hours worked, and read written_off_hours and scope overrun by pricing_model.

Follow-up

Two of the nine points are calendar. How do you say that to an executive without sounding like you are explaining the number away?
Utilisation is up and realised rate is down. What single operational behaviour produces both, and how would you confirm it?
What would you need to see to claim this is a demand problem rather than a delivery or pricing one?

Suggested practice checklist

Four days spend equal time on query work, statistics, modelling and product judgement at deliberately shallow depth, which produces a scored map of where you actually stand. The last three days spend everything on the two areas the role weights most, and close by re-running day one to measure movement.

Day 1 — Breadth pass: query fluency

Solve six prompts spanning aggregation, joins, window functions and date arithmetic in 60 minutes total, stopping at 10 minutes each whether or not it works, and mark every prompt as solved, solved slowly, or stuck.
For each unsolved prompt write the single blocking sentence (I lost the grain, I did not know the frame clause, I could not express the date boundary) instead of reading the solution.
Translate one pandas transformation you know well into SQL and one SQL query into pandas, checking that both return the same row count and the same totals.

Deliverable: A scored six-row table, one line per prompt, saved for the day-seven re-run.

Day 2 — Breadth pass: statistics and inference

Answer ten short questions in writing with nothing open: what a p-value is conditional on, what a 95 percent interval covers across repeated samples, when a paired test is the right one, what the bootstrap estimates, why multiple comparisons inflate false positives, how controlling the family-wise error rate differs from controlling the false discovery rate, what power depends on, what a missed real effect costs a product, the three situations where the central limit theorem does not rescue you (small n, very heavy tails, dependent observations), and what a standard error is the standard deviation of.
Grade yourself against a reference and count only the answers that were exactly right, not the ones that were nearly right.
Rewrite the two weakest answers the following morning from memory in full sentences.

Deliverable: Ten graded answers with an honest count of exact hits.

Day 3 — Breadth pass: modelling

Take one tabular dataset end to end in 90 minutes: a leakage-safe split, a baseline that is not a model (majority class or historical mean), one regularized linear model, one gradient-boosted tree, and a single evaluation metric chosen before you look at any result.
Write why that metric fits the cost structure: precision at a fixed recall for alerting, calibration for anything feeding a price or a threshold, ranking metrics for retrieval, and note that area under the ROC curve is insensitive to class balance in a way that can flatter a rare-positive problem.
Name the leak you were most likely to introduce (an encoding fit on all rows before splitting, or a feature computed after the label's timestamp) and write the check that would have caught it.

Deliverable: A notebook whose first cell states the metric and the baseline, plus two lines on what beat what and by how much.

Day 4 — Breadth pass: product judgement

Answer three case prompts aloud at 15 minutes each, timing how long passes before you state a success metric.
For one case write the first segmentation you would run and the row counts you expect per segment, so that a tiny segment cannot quietly drive the conclusion.
Take a metric definition you did not write, from a public dashboard, a textbook, or documentation you already have open, and list every place two analysts implementing it would diverge: which rows the denominator admits, whether the unit is an account or a person, what the time window is anchored to, and what happens to data that arrives late. Then write the one question that would close the largest of those gaps.

Deliverable: Three recorded case answers plus an ambiguity list for a metric someone else defined, ending in the single question you would ask about it.

Day 5 — Depth, first area

Rank the four areas by how many bullet points in the role description each one covers, pick the top one, and spend the entire day inside it.
Work the six hardest problems you can find in that area and for each write the generalizable move you should have reached for first, rather than the answer.
Re-solve the two you failed the same evening with notes closed.

Deliverable: Six generalizable moves written as instructions to yourself, not as solutions.

Day 6 — Depth, second area, and the seam between them

Repeat the depth protocol on the second-ranked area with the same six-problem structure.
Construct one problem that requires both areas at once, for example a metric redefinition whose effect you must validate with a test whose readout you then have to query.
Solve your own combined problem end to end and note where the handoff between the two areas cost you time.

Deliverable: One combined problem, solved end to end, with the handoff failure written down.

Day 7 — Integration and re-measurement

Re-run the six prompts from day one under the same clock and compare both correctness and time.
Run a 60-minute mixed mock that moves between areas without warning, since switching cost is what breadth passes do not train.
Write the two areas you would still fail on, and the sentence you will use in the interview when you hit one of them.

Deliverable: A before-and-after score table plus a written plan for the two remaining gaps.

Ownership when the result is inconvenient

Sometimes the honest read is that the initiative did not work, and the person who commissioned the analysis was hoping otherwise. Interviewers want to know whether you softened it. Prepare the case where you delivered an unwelcome result, how you presented the uncertainty without hiding behind it, and what the team did next.

How do you handle working with clients who may have limited technical understanding of your findings?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Pick a story where you drove the decision, not one where you observed it.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Can you describe a time you worked in a team and faced a disagreement? How did you resolve it?

Medium · behavioural and stakeholder questions

Approach

Name the disagreement or constraint, and how you resolved it with evidence.
Close with what you would do differently, concretely.
Pick a story where you drove the decision, not one where you observed it.

Follow-up

What would you do differently if you ran that project again?
How did you know the outcome was caused by your change?

Defending your own impact claim without randomisation or clean units

Hard · self-selection · clustered inference · impact measurement

At your review you plan to claim that the realisation dashboard you built recovered 1.4 million dollars. The evidence is that engagement-month realisation rose four points over two quarters among engagements whose leads used it. Adoption was voluntary. There are about sixty client accounts and the top five carry most fees. A new rate card shipped in the same quarter. Write the claim you can defend, the estimate you would actually produce, and what you say when asked for a causal number you cannot get.

Approach

Name the probe: whether you can separate the number you want from the number the data supports, under review pressure, without either inflating it or retreating to saying nothing can be known.
State both identification problems concretely. Voluntary adoption means adopting leads are plausibly the ones who already manage realisation, so the comparison is confounded at the person level. The rate card changes bill_rate_usd, which sits in the realisation denominator, so part of the four-point move is arithmetic rather than behavioural.
Neutralise what you can. Recompute realisation with bill rates snapshotted on work_date, or hold the denominator at the old rate card, so the rate-card change cannot move the metric by construction. Then rerun the comparison.
Get the inference right for the unit count. Cluster at client_id, not engagement, because engagements in one account share a partner, a rate card and a team. With sixty accounts and five carrying most fees, the effective cluster count is far below sixty, so report a wild cluster bootstrap interval rather than plain cluster-robust standard errors, which are biased downward in that regime.
Report both weightings and explain the divergence: an account-weighted estimate describes the typical account, a value-weighted one describes the revenue, and if they disagree a small number of accounts is carrying the result. Then give the decision-relevant sentence: the defensible range, whether its lower bound still clears the build cost, and what a proper staggered rollout would have bought.

Follow-up

The pre-period trends for adopters and non-adopters are not parallel. What do you report then?
You get to design the next rollout. What do you change so the same question is answerable, without randomising individual accounts?