Interview guides · Boston Staffing

Boston Staffing Data Scientist Interview Questions & Guide 2026

Timed data scientist practice for Boston Staffing: short SQL and probability sets run end to end, then scored against a rubric.

The 60-second brief

The role of a Data Scientist at Boston Staffing is pivotal in shaping data-driven decisions that directly influence the company's products and strategy. Data Scientists are responsible for analyzing complex data sets, developing predictive models, and translating their findings into actionable insights. Their work not only enhances operational efficiencies but also drives innovations that meet the needs of the staffing industry and its clients.

In modelling rounds the live question is usually why this model class for this problem, and how you would know six weeks after launch that it is still working. Deriving gradients by hand is rarely what is being probed.

Focus areas

Compute utilisation against a defended availability denominator
Correct late-entered timesheets before trending recent weeks
Segment margin by pricing model before comparing

Overview

At Boston Staffing, Data Scientists work closely with cross-functional teams, including product management, engineering, and marketing, to leverage data effectively. This role is crucial in developing algorithms that optimize staffing solutions and improve client satisfaction. You'll engage with real-world problems such as predicting hiring trends and analyzing candidate success rates, making your contributions both impactful and rewarding.

Candidates can expect a dynamic environment where their analytical skills will be tested and honed. As a Data Scientist, you'll play a strategic role in driving the business forward, making this position not just a job but a significant career opportunity.

The Boston Staffing interview process

Boston Staffing candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening Call

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Interviews

Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.

What to demonstrate

Whether the query you write matches the plan you just described
What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly

How to prepare

Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock

Behavioral Interviews

Most of the weight in this round sits on the disagreement questions. Data work routinely produces an answer someone senior did not want, and the interviewer is trying to learn what you do in that hour. Both failure modes are common: folding as soon as a director pushes back, and treating the pushback as ignorance to be corrected with a better chart. A strong answer usually contains a specific thing the other person knew that you did not, and describes how you found out whether it changed the conclusion.

What to demonstrate

Whether you can state the other side's argument accurately before you explain why you disagreed
What you treated as evidence during the disagreement, such as a rerun under their assumption or a holdout check, rather than persuasion technique
Whether you distinguish being overruled from being wrong, and can give an example of each

How to prepare

Write out one disagreement where you turned out to be wrong, and say what in the data misled you. Candidates prepare the story where they were right, and the follow-up asks for the other one.
For your main disagreement story, be ready to say what result would have made you drop your position. If no such result exists, you were not arguing from the data.
Practise stating the opposing position out loud in one sentence the stakeholder would accept, then continue the story.

Where candidates lose points

Computing days-to-pay or proposal cycle time over completed records only

At any snapshot date, invoices that have already been paid are disproportionately the fast ones, and proposals that already have a decision are disproportionately the quick ones. Averaging over the completed set alone biases both numbers downward, and the bias grows exactly when the business is deteriorating, because the slow cases are the ones still open. Unpaid and undecided records are right-censored; use Kaplan-Meier or a restricted mean up to a fixed horizon, and never fill paid_at with a placeholder.

Trending utilisation or revenue on work_date without accounting for timesheet backfill

Time entries are created days to weeks after the work happens, and the backfill tail often runs two to six weeks. A dashboard keyed on work_date therefore shows the most recent weeks as a decline that reverses on every refresh. The fix is either to hold the reporting window back past the observed backfill tail (measure the tail with the timesheet submission lag metric rather than guessing) or to report an as-of-entered_at snapshot so the series is internally consistent, and to state which one you used.

Writing SQL without stating NULL and tie-breaking behaviour

Before calling a query finished, say what it does with NULLs, ties and empty groups. NOT IN against a subquery containing a single NULL returns no rows at all, and RANK, DENSE_RANK and ROW_NUMBER differ precisely on ties, so name which one the question requires.

Reporting a p-value with no effect size or interval

Give the estimated difference with a confidence interval in the units the business cares about, then say whether that whole interval is worth acting on. A p-value only addresses whether you can rule out exactly zero; it says nothing about magnitude.

Questions & practice

What machine learning algorithms have you implemented, and how did you choose them?

Medium · machine learning and modelling

Approach

Pick an evaluation metric that matches the cost of each error type, not a default.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Say how the offline result would be validated online before it is trusted.

Follow-up

How would you choose the decision threshold, and who owns that choice?
Where could label leakage enter this setup?

How would you implement a decision tree from scratch?

Medium · machine learning and modelling

Approach

Set a baseline first, so any model has something honest to beat.
Frame the prediction: the label, the moment of prediction, and the action it triggers.
Pick an evaluation metric that matches the cost of each error type, not a default.

Follow-up

Where could label leakage enter this setup?
How would you choose the decision threshold, and who owns that choice?

Implement billable utilisation against a defended availability denominator

Medium · metric implementation · denominators · proration

Implement billable utilisation for one calendar month at consultant grain, then roll it up to practice_area. Inputs: time_entries (consultant_id, work_date, hours, is_billable, charge_code, status), consultants (consultant_id, practice_area, fte_fraction, home_region, is_billable_role, hire_date, termination_date) and holidays (region, holiday_date). Numerator is approved hours with is_billable true. Denominator is scheduled workdays in the month, clipped to [hire_date, termination_date] and net of that region's holidays, times 8 times fte_fraction, minus approved leave hours. Roll up as sum(numerator) / sum(denominator).

Approach

Build the workday set per region first: business days in the month minus that region's holiday_dates. State the Monday-to-Friday assumption out loud, because a region with a different working week makes np.busday_count wrong rather than approximate.
Prorate by clipping each consultant's interval to [max(month_start, hire_date), min(month_end, termination_date or month_end)] and counting workdays inside the clipped interval, so a mid-month start produces a smaller denominator instead of a fake dip in the ratio.
Subtract approved leave hours (charge_code in leave_paid, leave_unpaid) only for leave falling on days already counted as workdays; leave logged on a public holiday would otherwise be subtracted twice and can push a denominator negative.
Filter to is_billable_role, then drop consultant-months whose denominator is zero or negative (full-month leave, a hire on the last day) rather than emitting inf or NaN, and return the dropped count as part of the result.
Roll up by summing numerators and denominators separately. Averaging per-person ratios gives a 0.2 FTE consultant the same weight as a full-time one and produces a practice number that no individual reconciles to.

Worked solution (35 min)

Per region, workdays = np.busday_count(month_start, month_end + 1 day, holidays=region_holidays); per consultant, recompute on the clipped [start, end] interval so proration and holidays compose.
leave = time_entries[status=='approved' & charge_code.isin(['leave_paid','leave_unpaid']) & work_date.isin(region workdays)].groupby('consultant_id').hours.sum().
denominator = workdays_clipped * 8 * fte_fraction - leave, reindexed to the billable-role consultant list and filled with 0 leave where absent.
numerator = time_entries[status=='approved' & is_billable].groupby('consultant_id').hours.sum(), reindexed and zero-filled.
Drop rows with denominator <= 0 (counting them), compute utilisation_pct = 100 * numerator / denominator, then groupby practice_area and divide summed numerator by summed denominator.

Expected result: A consultant-month frame carrying numerator, denominator and utilisation_pct, a practice-area frame whose ratio equals sum(numerator)/sum(denominator), and an explicit count of consultant-months excluded for a non-positive denominator.

Verify it yourself

A full-time consultant with no leave in a 21-workday month has denominator exactly 168.0.
Every consultant's denominator is at most workdays * 8 * fte_fraction, with equality only when leave is zero.
The practice-area ratio equals the denominator-weighted mean of the per-consultant ratios, and sits between their min and max.
Summed numerators across practice areas equal total approved billable hours for in-scope consultants.

Follow-up

Utilisation rose two points this month while realisation fell. Reconcile those two movements with one mechanism.
A regional lead says their team looks four points below another region. What do you check before you answer, and in what order?

Given a dataset, how would you approach feature selection?

Medium · sql and data manipulation

Approach

Compute rates by summing numerator and denominator separately, never by averaging rates.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

What breaks if events arrive late or out of order?
How does the query change if the join becomes one-to-many?

Write a function to find the most frequent element in a list.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Say which table is the grain you start from, and join outward from it.

Follow-up

How does the query change if the join becomes one-to-many?
How would you verify this result without re-running the same query?

Build a client cohort revenue table from invoice lines

Easy · cohorts · date truncation · revenue recognition

dim_client holds client_id and first_engagement_date. fct_engagement maps engagement_id to client_id. fct_invoice_line holds engagement_id, line_type, period_start_date, amount_usd and status; recognised fees are lines with line_type IN ('fees','milestone','credit_note') and status <> 'draft', and credit notes carry negative amount_usd. Return one row per (cohort_month, months_since_first) giving distinct clients billed and net recognised fees. cohort_month is the month of first_engagement_date; months_since_first is the whole-month gap between period_start_date and first_engagement_date. Long format, no pivot. Decide explicitly what to do with clients whose first_engagement_date is NULL.

Approach

Filter fct_invoice_line to recognised lines first (line_type, status <> 'draft'), and key recognition on period_start_date rather than issued_at or paid_at. Billing period, issue date and payment date are three different clocks, and mixing them produces a cohort table nobody can reconcile against finance.
Join invoice lines to fct_engagement to reach client_id, since invoice lines carry engagement_id only, then to dim_client for first_engagement_date.
Truncate both dates to month, then compute months_since_first as 12 * (year(bill_month) - year(cohort_month)) + (month(bill_month) - month(cohort_month)). Doing it on truncated months is what keeps a client billed on day 2 of the third month in bucket 2 instead of bucket 1.
Aggregate to (cohort_month, months_since_first) with SUM(amount_usd) and COUNT(DISTINCT client_id), keeping credit_note lines inside the sum so the figure is net rather than gross.
Handle NULL first_engagement_date as an explicit exclusion and report how many clients and how much revenue that removes, rather than letting the join drop them silently. That exclusion is why the output's total is smaller than the filtered line set's total, so carry the excluded figure as a named number rather than discovering the gap later.

Worked solution (25 min)

Write the recognised-lines CTE with the line_type and status predicates, and check its row count against the unfiltered table so you know how much you excluded.
Join through fct_engagement to dim_client and confirm the join does not multiply rows: a line has exactly one engagement, an engagement exactly one client.
Add the two date_trunc expressions and the integer month arithmetic, then eyeball ten rows against their source dates.
Group and aggregate, then sort by cohort_month and months_since_first.
Run a separate count of clients with NULL first_engagement_date, and a separate sum of their recognised fees, and note both beside the output.

Expected result: A long-format table of (cohort_month, months_since_first, active_clients, net_fees_usd), plus two scalars reported beside it: the count of clients with a NULL first_engagement_date and the recognised fees attached to them. months_since_first is a non-negative integer on every row; a negative value means an invoice period precedes the recorded first_engagement_date, which is a data-quality finding to report, not something to clamp to zero.

Verify it yourself

SUM(net_fees_usd) over the whole result equals SUM(amount_usd) over the filtered lines whose engagement resolves to a client with a non-NULL first_engagement_date. It does not equal SUM(amount_usd) over the full filtered line set, which is larger by exactly the excluded NULL-cohort revenue: add that separately reported figure back and the two agree. Any remainder after that is a line whose engagement_id has no matching row in fct_engagement, which is a referential finding to name rather than absorb.
COUNT(DISTINCT client_id) at months_since_first = 0 within a cohort_month is less than or equal to the number of clients whose first_engagement_date falls in that month.
Zero rows have a negative months_since_first; if any appear, name the clients rather than filtering them out.
Re-running with period_end_date in place of period_start_date moves only lines whose billing period spans a month boundary.

Follow-up

How would you turn this table into a 12-month net revenue retention ratio, and what exactly becomes the denominator?
One account contributes 40 percent of a cohort's month-13 fees. How do you present the cohort so a reader sees that before they act on the ratio?
If subsidiaries roll up to parent_client_id, which first_engagement_date defines the cohort, and does the rollup change the cohort a subsidiary lands in?

How do you prioritize tasks when handling multiple projects?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

How do you stay updated with industry trends and advancements in data science?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
State what result would change your recommendation, so the answer is falsifiable.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
What would you do if the primary metric and the guardrail moved in opposite directions?

Given a dataset of employee performance, how would you identify key drivers of success?

Medium · product sense and metrics

Approach

Decompose the metric into the rates that drive it, and say which one you would check first.
State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
How would you detect that the metric is being gamed rather than genuinely improving?

How would you approach a client who is dissatisfied with their staffing solutions?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
Restate the decision this analysis has to support, and who acts on the answer.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Describe a data project you led and the challenges you faced.

Medium

Approach

Clarify what is being asked and what a complete answer would contain.
Say what you would check first and why it is the highest-information step.
Work from the decision backwards to the evidence you would need.

Follow-up

What assumption would you test first?
How would you know your answer was wrong?

Proxy delivery quality when no satisfaction data is collected

Medium · proxy metrics · measurement bias · quality · composites

The firm records no survey or satisfaction data. A steering group still wants a quality metric on the monthly scorecard. From fct_time_entry (written_off_hours, hours, is_billable), fct_invoice_line (line_type, status, amount_usd), fct_engagement (status, close_reason, prior_engagement_id) and dim_client (client_id, account_tier, parent_client_id), construct a composite quality proxy. For each component, state the direction of its bias and where the bias comes from, and name at least one account segment on which the proxy is systematically wrong. Then say what you would instrument to replace it.

Approach

Write the target construct down first: whether the client received what they paid for. Then show that every available column records a decision made by someone inside the firm or by the client's finance function, not an observation of the work.
Assemble four components at engagement grain: write-off rate (written_off_hours / billable hours), disputed and credit-noted share of invoiced value, early-close rate on close_reason IN ('scope_dispute','quality'), and twelve-month repeat purchase on the account rolled up through parent_client_id.
State each bias precisely. Write-offs record a partner's willingness to concede, so a strategic-tier account with a partner protecting the relationship shows high write-offs on good work while a transactional account shows low write-offs on bad work: across account_tier the proxy runs against the construct. Disputes require a client AP function that disputes at all, which scales with client size. close_reason is entered by the person closing the engagement and is the field most likely to be recorded defensively. Repeat purchase is confounded by switching cost and by whether the client has budget at all.
Refuse to blend them into one index without weights that a named person owns, and report the four as a strip stratified by account_tier so a shift in concession policy cannot read as a shift in delivery.
Specify the replacement: a post-close instrument tied to engagement_id, collected by someone outside the delivery team, with a response-rate floor agreed before launch, plus a constrained close_reason enum enforced at write time instead of after the fact.

Worked solution (25 min)

Compute the four components at engagement grain for the trailing four quarters.
Cross-tabulate write-off rate by account_tier and by pricing_model, and confirm the tier gradient exists before making any claim about quality.
Check whether write-off rate and twelve-month repeat purchase agree in sign at the account level; if they disagree, the composite has no single direction and must not be blended.
Write the one-paragraph caveat that has to travel with the number, naming the specific segment where it inverts.

Expected result: Four separately reported components, each with a stated direction and named bias source, stratified by account_tier, plus an explicit statement that write-off rate inverts on strategic-tier accounts where concessions are relationship decisions rather than quality signals. No blended index without an owner for the weights, and a named instrumentation plan that would replace the proxy.

Verify it yourself

Write-off rate differs materially across account_tier. If it does not, either the tier field is stale or concessions are centrally policed, and both are worth reporting.
The four components do not all point the same way on every account. If they do, check whether one large engagement is driving all four.
The caveat names a specific segment and direction, not a generic statement that the number is directional.

Follow-up

Firm-wide write-off rate falls 30% in one quarter. Give one benign and one alarming explanation, and the query that separates them.
The steering group insists on a single number. What do you hand them, and what do you decline to do?

Firm margin rose while every pricing model's margin fell

Easy · mix shift · margin · decomposition

Firm-wide engagement gross margin rose from 31% to 34% quarter over quarter. Split by fct_engagement.pricing_model, margin fell inside time_and_materials, fixed_fee, retainer and outcome_based. Sources are fct_invoice_line (engagement_id, line_type, amount_usd, period_start_date, period_end_date) and fct_time_entry (engagement_id, hours, cost_rate_usd, status). Explain the aggregate move, quantify how much of the 3-point change is mix and how much is within-model, and state which number you put in front of practice leadership and why.

Approach

Write the aggregate as a weighted mean: M = sum over pricing models p of w_p * m_p, where m_p is margin within p and w_p is p's share of the fee base. The weights must be the fee base, because margin is a fees-weighted ratio; weighting by engagement count answers a different question.
Apply the exact three-term decomposition: dM = sum (w_p1 - w_p0) * m_p0 + sum w_p0 * (m_p1 - m_p0) + sum (dw_p)(dm_p). Verify the three components sum to the reported +3.0 points before interpreting any of them.
Identify which weights moved and in which direction; the aggregate can only rise this way if fee share shifted toward the structurally higher-margin models, typically fixed_fee delivered under budget or retainer.
Guard against composition inside each stratum: repeat the same decomposition within pricing_model by practice_area and account_tier, so a within-model decline is not itself an unexamined mix effect.
Confirm both sides of the ratio are on the same clock and the same scope: numerator includes line_type IN ('fees','milestone','credit_note') with credits signed negative; the cost side includes all approved delivery hours, billable and non-billable alike.
Report within-model margin as the headline delivery signal and mix as a separate, named line with its own owner, since mix is a sales decision and margin is a delivery one.

Follow-up

Next quarter the mix reverts. What happens to the aggregate, and what will you have already told leadership so this is not a surprise?
Which of the four pricing models should not be pooled with the others even for a mix-adjusted number, and why?
How would you present this if the mix shift were deliberate strategy rather than accident?

Suggested practice checklist

For someone who can already write the query and train the model but stalls when asked what to measure or whether a change is worth making. Metric definition and case structure come first; the technical work is kept as maintenance rather than the centre of the week.

Day 1 — Metric anatomy

For three products you use daily, write one primary metric, two input metrics that plausibly move it, and one guardrail that would catch a cheap way of moving the primary at the cost of the product.
For one of them, specify the metric precisely enough that two analysts would return the same number: numerator, denominator, unit of observation, time window, and how returning and deleted accounts are treated.
Pick a ratio metric and write what happens to it when the denominator shrinks for reasons unrelated to the numerator, with a concrete example of that happening.

Deliverable: A one-page metric tree for one product, with the primary metric written as an unambiguous spec.

Day 2 — Diagnosing a drop without guessing

Take the prompt "weekly active users fell 8 percent week over week" and write the segmentation plan before proposing any cause: platform, region, tenure cohort, acquisition channel, and whether the movement sits in the numerator or in a changed denominator.
List the instrumentation failures that manufacture fake drops (a client release that stopped firing an event, a bot filter change, a shifted date boundary or timezone) and write the query that rules out each one.
Rehearse stating the boring explanations first, seasonality and day-of-week composition, before reaching for a product cause.

Deliverable: A drop-diagnosis checklist short enough to recite from memory in under a minute.

Day 3 — Should we build it

Take a feature idea and write it as a bet: what you believe is true, what would have to be true for it to pay off, the metric that would confirm it, and the effect size that would justify the engineering cost.
Size the opportunity top-down and bottom-up, then reconcile the two numbers in writing instead of quoting whichever is friendlier.
Write the counter-metric that would make you kill the feature even if it wins on the primary metric.

Deliverable: A one-page product memo ending in a decision rather than a list of considerations.

Day 4 — The places aggregate numbers lie

Construct a Simpson's paradox numerically: two segments where the treatment wins within each segment yet loses overall, and identify the shift in segment weights that causes it.
Take a heavy right-tailed quantity such as revenue per user and write why the mean is the wrong summary, which percentile you would report instead, and what a moving mean with a stable median tells you.
Write your definition of a session for the product from day one, then name two real behaviours it misclassifies.

Deliverable: One page holding a worked Simpson's paradox table and a session definition with its two known failure cases.

Day 5 — Technical maintenance, aimed at metrics

Solve four timed SQL prompts that all end in a ratio metric, so the question of grain stays live in every answer.
Compute a 95 percent confidence interval for a proportion on a small sample, and state why the normal approximation is unreliable when either np or n(1 minus p) falls below roughly 10, along with which interval you would use instead.
Take one metric from your day-one tree, write the query that computes it correctly, then write the query that computes it wrong in the most plausible way and explain how you would notice.

Deliverable: Four solved prompts plus a matched correct and plausible-wrong query for one metric.

Day 6 — Turning engineering work into data science stories

Write three project stories as situation, decision, trade-off, outcome, each carrying one number and one thing you got wrong.
For the story you will lead with, prepare an answer to "what would you do differently" that names a decision you made, not a constraint you were handed.
Practise the sentence that reframes a systems project as a question project: the question the work answered, ahead of the pipeline it shipped.

Deliverable: Three written stories with the lead story delivered aloud and timed under four minutes.

Day 7 — Mock case and gap list

Run a 40-minute mock case with someone playing a product manager who pushes back on your metric choice, and record it.
Listen back and mark every moment you proposed a solution before the success metric existed.
Rewrite those moments as the question you should have asked, and rehearse the first 90 seconds of the case until scoping comes before solving.

Deliverable: A recorded case plus a rewritten opening 90 seconds.

Stories where someone disagreed with you

Have two ready. In one, the data was on your side and you had to move someone who outranked you. In the other, the pushback was correct and you changed position. The second is the harder story and it lands better, because it shows you separate being right from being attached to an answer. Name the person's actual objection.

Tell us about a time you faced a significant setback. How did you handle it?

Medium · behavioural and stakeholder questions

Approach

Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.
State the situation in two sentences and spend the rest on your reasoning.

Follow-up

What did you decide not to do, and why?
What would you do differently if you ran that project again?

Defending your own impact claim without randomisation or clean units

Hard · self-selection · clustered inference · impact measurement

At your review you plan to claim that the realisation dashboard you built recovered 1.4 million dollars. The evidence is that engagement-month realisation rose four points over two quarters among engagements whose leads used it. Adoption was voluntary. There are about sixty client accounts and the top five carry most fees. A new rate card shipped in the same quarter. Write the claim you can defend, the estimate you would actually produce, and what you say when asked for a causal number you cannot get.

Approach

Name the probe: whether you can separate the number you want from the number the data supports, under review pressure, without either inflating it or retreating to saying nothing can be known.
State both identification problems concretely. Voluntary adoption means adopting leads are plausibly the ones who already manage realisation, so the comparison is confounded at the person level. The rate card changes bill_rate_usd, which sits in the realisation denominator, so part of the four-point move is arithmetic rather than behavioural.
Neutralise what you can. Recompute realisation with bill rates snapshotted on work_date, or hold the denominator at the old rate card, so the rate-card change cannot move the metric by construction. Then rerun the comparison.
Get the inference right for the unit count. Cluster at client_id, not engagement, because engagements in one account share a partner, a rate card and a team. With sixty accounts and five carrying most fees, the effective cluster count is far below sixty, so report a wild cluster bootstrap interval rather than plain cluster-robust standard errors, which are biased downward in that regime.
Report both weightings and explain the divergence: an account-weighted estimate describes the typical account, a value-weighted one describes the revenue, and if they disagree a small number of accounts is carrying the result. Then give the decision-relevant sentence: the defensible range, whether its lower bound still clears the build cost, and what a proper staggered rollout would have bought.

Follow-up

The pre-period trends for adopters and non-adopters are not parallel. What do you report then?
You get to design the next rollout. What do you change so the same question is answerable, without randomising individual accounts?

Your utilisation dashboard caused a staffing decision on incomplete data

Medium · late-arriving data · postmortem · reporting windows

Six weeks ago you shipped a weekly billable-utilisation dashboard keyed on fct_time_entry.work_date. It showed a six-point drop across the three most recent weeks. A practice lead pulled two consultants off an engagement in response. The drop reversed on the next refresh, because time entries are created days to weeks after the work happens and entered_at trails work_date. Describe what you do now: the diagnosis, the change to the artifact, and the conversation with the person who acted on your number.

Approach

Name the probe: whether you own a reporting-design error rather than reclassifying it as someone else's timesheet compliance problem, and whether you fix the class of bug instead of the single week.
Quantify before explaining. Measure the backfill curve directly from entered_at: for work_date D, the share of final hours that existed as of D plus k, for k from 1 to 45. That gives an observed tail length instead of a guessed cutoff.
Change the artifact so the incomplete region cannot be read as a trend. Either end the trended series at snapshot minus the measured tail, or publish an as-of-entered_at series that is internally consistent, and label which one is on screen.
Tell the person who acted, first and directly, with the corrected series and the specific decision to revisit. A correction that arrives after they notice costs more than the original error.
Add a standing completeness tile: timesheet submission lag, the share of entries where entered_at date minus work_date exceeds seven days, so the dashboard shows its own reliability rather than depending on you remembering.

Follow-up

Leadership still wants to see the current week. What do you show, and how do you label it?
One practice runs a three-day lag and another twenty days. Do you set one firm-wide cutoff or one per practice, and what does that cost in comparability?