Interview guides · HKT Services

HKT Services Data Scientist Interview Questions & Guide 2026

Diagnose a metric move for HKT Services data scientist prep: split by segment, rule out a logging change, separate mix shift from behaviour.

The 60-second brief

As a Data Scientist within the Data Monetisation Team at HKT Services, you are not just building models; you are acting as a bridge between complex data ecosystems and tangible business growth. This role is central to HKT Services’ mission to transform vast customer data into actionable intelligence. You will operate at the intersection of product strategy and advanced AI, delivering solutions that enhance personalization, optimize conversion, and drive revenue across diverse digital channels.

If the team owns experimentation, expect depth past a two-sample test: minimum detectable effect and its roughly inverse-square-root dependence on sample size (holding power, significance level and variance fixed), variance reduction from pre-period covariates, interference between units, and when a sequential design is the right call.

Focus areas

Separate bookings, recognised revenue and collected cash
Reconstruct pipeline stage history from mutable rows
Segment margin by pricing model before comparing

Overview

The work is high-stakes and high-impact. Whether you are optimizing Retrieval-Augmented Generation (RAG) pipelines for GenAI, deploying Reinforcement Learning models for personalization, or architecting scalable MLOps frameworks, your contributions directly influence how the business understands and interacts with its customers. You will collaborate with a multidisciplinary team of 40+ data scientists and engineers, operating in an environment that prizes ownership, technical depth, and the ability to translate abstract analytical findings into clear strategic recommendations.

The HKT Services interview process

HKT Services candidates report 4 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.

Initial Screening

Whoever runs this call is usually not a practitioner. They take notes, and a hiring manager skims those notes later, so the real question is whether your work survives being written down by someone outside the field. Test every project sentence against that: could a non-specialist repeat it correctly without knowing what a propensity score is? Carry a plain-language version of each project and one reason you want this particular role that you could not copy onto another application. Vagueness at this stage reads as inexperience, even when the underlying work was genuinely deep.

What to demonstrate

Whether a non-specialist can restate your projects accurately, since their paraphrase is what reaches the hiring manager
Whether your reason for wanting the role points at the work itself rather than the company's reputation
Whether your language signals the level being screened for: what you decided yourself versus what you were handed

How to prepare

Write a two-sentence, jargon-free version of each major project: the question nobody could answer, and the decision your work changed. Read it to someone outside data and have them repeat it back
Point your 'why this role' answer at something concrete in the job description or the product surface you would be working on, and keep it to two sentences
Have two questions ready about measurement: which metric the team is held to, and who acts on an analysis once it lands

Technical Deep Dive

Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.

What to demonstrate

Whether an ambiguous term becomes a specific column and filter before any computation happens
Whether you read the schema for keys and cardinality rather than only for column names
Whether the result answers the question at the grain it was asked at, per user or per session or per day

How to prepare

Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over

Live Technical Assessment

This round decides whether someone can hand you a schema and a question and trust the number that comes back. Correctness under a clock is the bar, not clever syntax. The habit that separates strong from weak answers is checking the grain: after every join, know how many rows you expect and whether the count moved. Most wrong answers in this format are not wrong logic, they are a fan-out from a key that turned out not to be unique, or a filter applied before an aggregate when it belonged after. Say what you expect before you run it.

What to demonstrate

Whether your row counts survive each join, and whether you notice on your own when they do not
Deliberate handling of rows that fail to match, including whether the question needs an inner join or a left join with the non-matches kept and counted
Whether NULLs are treated on purpose, given that a NULL compares equal to nothing and that COUNT of a column skips it
Reaching a defensible answer inside the window instead of a refined one after it

How to prepare

Take a two-table schema, write a join that fans out on purpose, then fix it by collapsing the many-side to one row per key before joining. Repeat until the fix is reflex rather than recall.
Write a funnel as one query and print the distinct user count at each stage, then confirm each stage is a subset of the one above it rather than assuming it
Do a few timed runs in a plain text box with no autocomplete and no formatter, since assessment editors often have neither

Final Review

A loop is not scored one interview at a time. The people you meet compare notes afterwards, usually in a meeting you are not in, and the outcome turns on what each of them can say about you when asked. That rewards something other than survival: every room needs one specific thing worth repeating, and none of them can contradict another. The common way to lose is to tell the same project four times with different numbers in it, or to be uniformly fine in a way that leaves nobody with anything to argue for.

What to demonstrate

Whether your account of a project survives being told twice, with the same scale, the same metric definition and the same numbers each time
Whether each interviewer leaves with one concrete claim they could make on your behalf later, rather than an absence of complaints
Whether a question you already answered in an earlier room gets the same answer at the same depth, without visible impatience

How to prepare

Write a one-page fact sheet for your two or three main projects that fixes the numbers you will quote: rows of data, the metric as a single sentence, the effect you measured and how long the work took. Say them aloud from the sheet until they come out identical every time
For each kind of room you expect, decide the one sentence you want that interviewer repeating in a debrief, then check during the mock that you said it outright instead of implying it
Rehearse answering the same project question twice in one sitting, the second time as though you had not just answered it, because the thing that needs fixing is the flatness that creeps into a repeated story

Where candidates lose points

Trending utilisation or revenue on work_date without accounting for timesheet backfill

Time entries are created days to weeks after the work happens, and the backfill tail often runs two to six weeks. A dashboard keyed on work_date therefore shows the most recent weeks as a decline that reverses on every refresh. The fix is either to hold the reporting window back past the observed backfill tail (measure the tail with the timesheet submission lag metric rather than guessing) or to report an as-of-entered_at snapshot so the series is internally consistent, and to state which one you used.

Treating accounts as independent observations

Revenue is concentrated: a small number of client_ids typically carries a large share of fees, and engagements within one account share a partner, a rate card and a delivery team. Ordinary standard errors computed over engagements therefore understate uncertainty badly. Cluster at client_id, and with fewer than roughly 40 clusters use a wild cluster bootstrap or a CR2 correction, because cluster-robust standard errors are downward-biased in that regime and will manufacture significance that a replication will not reproduce.

Extrapolating a first-week lift inflated by novelty effects

Plot the treatment effect by days since first exposure instead of quoting one pooled average. A lift that decays toward zero across the test window is behaviour that will not persist, and annualising it produces a forecast that misses by an order of magnitude.

Optimising accuracy on a heavily imbalanced target

State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.

Questions & practice

Simulate the chance a capped engagement crosses its cap

Medium · monte carlo · bootstrap resampling · pricing caps

A capped time-and-materials engagement has not_to_exceed_usd = 400000, has billed 250000 to date, and bills at a blended bill_rate_usd of 250. Fourteen delivery weeks remain before planned_end_date. You have that engagement's last twenty weekly totals of approved billable hours as a pandas Series. Estimate the probability that cumulative billable value crosses the cap before the planned end, and the expected unbillable hours if it does. Resample the observed weeks; do not assume normality. Report a Monte Carlo standard error and justify the number of draws.

Approach

Reduce the deterministic part first: the cap still affords (400000 - 250000) / 250 = 600 hours, so the whole question is the distribution of the sum of fourteen future weekly hour totals against a fixed threshold of 600.
Before resampling, check the twenty observed weeks for trend and lag-1 autocorrelation. Independent resampling is defensible only if the weeks are exchangeable; on a ramping engagement use a moving-block bootstrap or model the ramp, because i.i.d. draws understate the upper tail, which is exactly the tail the question is about.
Draw one (B, 14) array with np.random.default_rng().choice(observed, size=(B,14), replace=True), sum along axis 1, and compute p_hat = mean(total > 600) in one vectorised pass.
Report expected unbillable hours two ways: unconditional mean(maximum(total - 600, 0)) for expected loss, and the mean conditional on a breach for how bad a breach is when it happens. The second is the number that drives a change-order conversation.
Attach se(p_hat) = sqrt(p_hat(1 - p_hat)/B) and pick B from the precision you need: plus or minus one point at 95% confidence needs about 1.96^2 * 0.25 / 0.01^2, roughly 9600 draws at the worst case p = 0.5.
State the assumptions that would flip the answer: constant blended rate, no scope change, no holiday weeks inside the fourteen, and a cap that applies to fees rather than to fees plus expenses.

Follow-up

Hours past the cap still cost money. Restate the result as expected gross margin rather than a probability.
Two weeks of time are entered but not yet approved. How do you fold them in without double-counting?
A change order that raises the cap is judged 60% likely. How does that change the number you present, and to whom?

Attach effective-dated cost rates without merge_asof

Medium · type-2 scd · interval join · searchsorted

time_entries (consultant_id, work_date, hours) has roughly two million rows. dim_consultant is a type-2 dimension (consultant_sk, consultant_id, standard_cost_rate_usd, effective_from, effective_to nullable, is_current) with a handful of rows per consultant. Attach the cost rate in force on each entry's work_date. You may not use pandas.merge_asof and you may not materialise the consultant_id cartesian product. Treat intervals as half-open, meaning [effective_from, effective_to), with a NULL effective_to open-ended. Return unmatched entries separately with a reason rather than dropping them.

Approach

Fix the semantics before writing code: half-open intervals are what make the join single-valued on the day a rate changes. Compare work_date at start-of-day in the reporting timezone against effective_from in the same zone, and write that comparison down as a stated assumption.
Sort dim_consultant by (consultant_id, effective_from) and assert non-overlap inside each consultant, since searchsorted only returns the right answer if the intervals are ordered and disjoint.
Build the composite sort key so it cannot overflow: dense-code the consultant ids through one shared sorted category mapping, express time as whole days since a floor date earlier than every date in both frames, and take SCALE as a power of two above the largest day offset, giving key = code * SCALE + offset. Assert code.max() * SCALE + off.max() < np.iinfo(np.int64).max before using it. Multiplying a raw consultant_id by a scale larger than a nanosecond timestamp (about 1.75e18 for present-day dates) leaves an id budget of only int64_max / SCALE, roughly five consultants, before the product passes 9.22e18; beyond that numpy wraps silently, and it does not even wrap consistently negative (with SCALE = 2e18 an id of 12 lands back on a large positive key), so the order breaks without a sign flip to catch it.
With a key that fits, np.searchsorted(dim_key, entry_key, side='right') - 1 is the index of the last interval starting at or before work_date, in O((n + k) log k) and with no cartesian product. SCALE only has to exceed the span of the offsets, never an absolute epoch value, so if a rate can change intraday, switch the offset unit to seconds (a 2^31 scale covers 68 years) and re-run the same assertion.
Validate the candidate rather than trust it: the match holds only if work_date < effective_to or effective_to is NULL. Everything else is an exception, either work logged before the first dimension row or a genuine gap in the SCD.
Return matched rows carrying consultant_sk and the rate, plus an exceptions frame with counts by reason, so a gap in the dimension surfaces as a number rather than as quietly missing cost.

Worked solution (30 min)

dim = dim.sort_values(['consultant_id','effective_from']); assert within each consultant that effective_from is strictly increasing and effective_to.fillna(max) >= the next effective_from.
cats = np.union1d(time_entries.consultant_id.unique(), dim.consultant_id.unique()), which is sorted, so code = pd.Categorical(ids, categories=cats).codes.astype('int64') preserves consultant_id order and the sort above still holds; an id absent from the dimension codes to -1 instead of matching something else.
floor = min(work_date.min(), effective_from.min()).normalize(); off = (ts.dt.normalize() - floor).dt.days.to_numpy() in both frames; SCALE = 1 << 20 (1,048,576 days, about 2871 years); assert off.min() >= 0, off.max() < SCALE and code.max() * SCALE + off.max() < np.iinfo(np.int64).max; key = code * SCALE + off.
idx = np.searchsorted(dim_key, entry_key, side='right') - 1; entries whose code is -1 are exceptions with reason 'consultant_not_in_dimension'; rows with idx < 0, or whose matched dimension code differs from the entry's code, are exceptions with reason 'no_interval_before_work_date'.
Check the upper bound: keep rows where effective_to is NaT or work_date < effective_to; the rest are exceptions with reason 'gap_in_scd'.
Brute-force 200 random entries with a straight boolean filter over that consultant's rows and diff the rates against the vectorised answer.

Expected result: Matched rows equal to the input count minus exceptions, each carrying exactly one consultant_sk and standard_cost_rate_usd, plus an exceptions frame grouped by reason. No input row is duplicated and none is silently dropped.

Verify it yourself

len(matched) + len(exceptions) == len(time_entries), which is the property an inner join would break.
The 200-row brute-force sample agrees on every rate.
An entry whose work_date equals an effective_from takes the new rate, not the old one, exactly once.
Per-consultant total cost changes by (new_rate - old_rate) * hours after the change date, and by nothing before it.
np.diff(dim_key) is strictly positive, asserted rather than assumed: this is the only cheap guard against a key that has wrapped, since an overflowed product can land back on a positive value and a non-negativity check alone would pass it. Renumbering the consultant_ids (say to 4000001..4000050) must not change a single attached rate, which is the fixture a raw-id-times-nanoseconds key fails once ids run past int64_max / SCALE.

Follow-up

Rates were backdated a quarter after the fact. How do you keep last quarter's published cost from moving when someone reruns the report?
How does the implementation change if effective_to is inclusive rather than exclusive, and what breaks on the boundary day if you get that wrong?

Implement billable utilisation against a defended availability denominator

Medium · metric implementation · denominators · proration

Implement billable utilisation for one calendar month at consultant grain, then roll it up to practice_area. Inputs: time_entries (consultant_id, work_date, hours, is_billable, charge_code, status), consultants (consultant_id, practice_area, fte_fraction, home_region, is_billable_role, hire_date, termination_date) and holidays (region, holiday_date). Numerator is approved hours with is_billable true. Denominator is scheduled workdays in the month, clipped to [hire_date, termination_date] and net of that region's holidays, times 8 times fte_fraction, minus approved leave hours. Roll up as sum(numerator) / sum(denominator).

Approach

Build the workday set per region first: business days in the month minus that region's holiday_dates. State the Monday-to-Friday assumption out loud, because a region with a different working week makes np.busday_count wrong rather than approximate.
Prorate by clipping each consultant's interval to [max(month_start, hire_date), min(month_end, termination_date or month_end)] and counting workdays inside the clipped interval, so a mid-month start produces a smaller denominator instead of a fake dip in the ratio.
Subtract approved leave hours (charge_code in leave_paid, leave_unpaid) only for leave falling on days already counted as workdays; leave logged on a public holiday would otherwise be subtracted twice and can push a denominator negative.
Filter to is_billable_role, then drop consultant-months whose denominator is zero or negative (full-month leave, a hire on the last day) rather than emitting inf or NaN, and return the dropped count as part of the result.
Roll up by summing numerators and denominators separately. Averaging per-person ratios gives a 0.2 FTE consultant the same weight as a full-time one and produces a practice number that no individual reconciles to.

Follow-up

Utilisation rose two points this month while realisation fell. Reconcile those two movements with one mechanism.
A regional lead says their team looks four points below another region. What do you check before you answer, and in what order?

Explain how you would optimize a slow-running query that processes millions of rows.

Medium · sql and data manipulation

Approach

State the window function and its partition and ordering out loud before writing it.
Compute rates by summing numerator and denominator separately, never by averaging rates.
Say which table is the grain you start from, and join outward from it.

Follow-up

How would you verify this result without re-running the same query?
How does the query change if the join becomes one-to-many?

Write a query using SQL window functions to calculate a 7-day rolling average of user engagement.

Medium · sql and data manipulation

Approach

Say which table is the grain you start from, and join outward from it.
Check whether any join is one-to-many before aggregating, or the sums inflate.
Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.

Follow-up

How would you verify this result without re-running the same query?
What breaks if events arrive late or out of order?

Longest consecutive bench spell per consultant from weekly activity

Hard · gaps and islands · window functions · run length

You have a dense series (consultant_id, week_start, billable_hours, full_week_leave BOOLEAN) covering 26 weeks for every billable-role consultant, with zero-hour weeks present as 0. Return every maximal run of consecutive weeks where billable_hours = 0, as consultant_id, spell_start_week, spell_end_week and week_count, then each consultant's longest run. Weeks flagged full_week_leave must neither break a run nor count toward its length. Mark runs still open at the last week of the series rather than reporting them as finished.

Approach

Remove full_week_leave weeks from the sequence first, then re-number what remains with ROW_NUMBER() OVER (PARTITION BY consultant_id ORDER BY week_start). Deleting before numbering is precisely what makes a leave week transparent to a run instead of a break in it; filtering afterwards would leave a gap in the sequence and split the spell.
Flag each surviving week as bench (billable_hours = 0) or active, then compute two row numbers per consultant: one over all surviving weeks, one partitioned additionally by the bench flag. Their difference is constant inside any run of adjacent same-flag weeks, which is the island key.
Group the bench weeks by (consultant_id, island key) and take MIN(week_start), MAX(week_start) and COUNT(*). Do not island on week_start minus an interval times a row number: that variant assumes the spine has no holes, and the leave removal has just deliberately put holes in it.
Pick the longest spell per consultant with ROW_NUMBER() OVER (PARTITION BY consultant_id ORDER BY week_count DESC, spell_start_week) so ties resolve deterministically, rather than a MAX that cannot carry the accompanying dates.
Handle both edges as censoring. A spell touching the last week of the series is still open and its length is a lower bound, so flag it; a spell touching the first week may have started earlier, so either flag it too or exclude it from any length distribution and say which.

Worked solution (40 min)

Filter out full_week_leave rows and count the surviving weeks per consultant.
Add the bench flag and the two row numbers, then print one consultant's full sequence to see the island key stay constant across a run.
Group bench weeks by consultant and island key and aggregate to spell boundaries and lengths.
Add the open-at-the-edge flag by comparing spell_end_week to the consultant's last surviving week.
Rank spells per consultant and select the longest with the deterministic tiebreak.

Expected result: One row per maximal bench run with its first and last week, its length in weeks and an is_open flag, plus a one-row-per-consultant longest-spell table. A consultant with no bench weeks appears in neither, which is correct and worth stating.

Verify it yourself

For each consultant, the sum of week_count across all spells equals their count of bench weeks in the leave-filtered series.
No two spells for the same consultant are adjacent in the re-numbered sequence; adjacency would mean the island key was computed wrongly.
The longest-spell table has at most one row per consultant and its week_count matches the maximum in the spells table.
Hand-check one consultant by printing their 26 weeks as a strip of zeros, hours and leave markers, and reading the runs off it.

Follow-up

A spell begins before your 26-week window. How do you detect that, and what do you report as its length?
Rebuild this at daily grain. What changes about weekends, holidays and the island key?
How would you separate true bench from unbilled delivery work on a live engagement, given only charge codes?

If a key conversion metric drops by 10% overnight, how would you structure your investigation to isolate the cause?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Fix the population and the time window before naming any metric.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

How would you detect that the metric is being gamed rather than genuinely improving?
Which segment would you cut first, and what would that rule out?

How would you design the metrics for a new subscription-based feature to ensure we are measuring long-term user value?

Medium · product sense and metrics

Approach

Restate the decision this analysis has to support, and who acts on the answer.
Name one primary metric, then the guardrail that stops it being gamed.
Decompose the metric into the rates that drive it, and say which one you would check first.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
How would you detect that the metric is being gamed rather than genuinely improving?

Describe a product you admire from a data-driven perspective; what metrics would you use to define its success?

Medium · product sense and metrics

Approach

State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.
Fix the population and the time window before naming any metric.

Follow-up

What would you do if the primary metric and the guardrail moved in opposite directions?
Which segment would you cut first, and what would that rule out?

How do you prioritize competing requests from different product teams?

Medium · product sense and metrics

Approach

Name one primary metric, then the guardrail that stops it being gamed.
State what result would change your recommendation, so the answer is falsifiable.
Restate the decision this analysis has to support, and who acts on the answer.

Follow-up

Which segment would you cut first, and what would that rule out?
What would you do if the primary metric and the guardrail moved in opposite directions?

What are the most common experimentation pitfalls you encounter when running A/B tests?

Medium · a/b testing and experimentation

Approach

Name the guardrails that would stop a launch even on a positive primary result.
Name the randomisation unit first; it decides the variance and what the test can detect.
State the primary metric and the minimum effect worth shipping, then size the test.

Follow-up

How would you handle interference between treated and control units?
What would you conclude if the result is positive but the test is underpowered?

How do you determine the required sample size to achieve statistical significance for a low-traffic feature?

Medium · a/b testing and experimentation

Approach

State the primary metric and the minimum effect worth shipping, then size the test.
Name the guardrails that would stop a launch even on a positive primary result.
Decide the analysis before seeing data, including how long it runs and when you look.

Follow-up

What would you do if you could not randomise at all?
How would you handle interference between treated and control units?

Proxy delivery quality when no satisfaction data is collected

Medium · proxy metrics · measurement bias · quality · composites

The firm records no survey or satisfaction data. A steering group still wants a quality metric on the monthly scorecard. From fct_time_entry (written_off_hours, hours, is_billable), fct_invoice_line (line_type, status, amount_usd), fct_engagement (status, close_reason, prior_engagement_id) and dim_client (client_id, account_tier, parent_client_id), construct a composite quality proxy. For each component, state the direction of its bias and where the bias comes from, and name at least one account segment on which the proxy is systematically wrong. Then say what you would instrument to replace it.

Approach

Write the target construct down first: whether the client received what they paid for. Then show that every available column records a decision made by someone inside the firm or by the client's finance function, not an observation of the work.
Assemble four components at engagement grain: write-off rate (written_off_hours / billable hours), disputed and credit-noted share of invoiced value, early-close rate on close_reason IN ('scope_dispute','quality'), and twelve-month repeat purchase on the account rolled up through parent_client_id.
State each bias precisely. Write-offs record a partner's willingness to concede, so a strategic-tier account with a partner protecting the relationship shows high write-offs on good work while a transactional account shows low write-offs on bad work: across account_tier the proxy runs against the construct. Disputes require a client AP function that disputes at all, which scales with client size. close_reason is entered by the person closing the engagement and is the field most likely to be recorded defensively. Repeat purchase is confounded by switching cost and by whether the client has budget at all.
Refuse to blend them into one index without weights that a named person owns, and report the four as a strip stratified by account_tier so a shift in concession policy cannot read as a shift in delivery.
Specify the replacement: a post-close instrument tied to engagement_id, collected by someone outside the delivery team, with a response-rate floor agreed before launch, plus a constrained close_reason enum enforced at write time instead of after the fact.

Worked solution (25 min)

Compute the four components at engagement grain for the trailing four quarters.
Cross-tabulate write-off rate by account_tier and by pricing_model, and confirm the tier gradient exists before making any claim about quality.
Check whether write-off rate and twelve-month repeat purchase agree in sign at the account level; if they disagree, the composite has no single direction and must not be blended.
Write the one-paragraph caveat that has to travel with the number, naming the specific segment where it inverts.

Expected result: Four separately reported components, each with a stated direction and named bias source, stratified by account_tier, plus an explicit statement that write-off rate inverts on strategic-tier accounts where concessions are relationship decisions rather than quality signals. No blended index without an owner for the weights, and a named instrumentation plan that would replace the proxy.

Verify it yourself

Write-off rate differs materially across account_tier. If it does not, either the tier field is stale or concessions are centrally policed, and both are worth reporting.
The four components do not all point the same way on every account. If they do, check whether one large engagement is driving all four.
The caveat names a specific segment and direction, not a generic statement that the number is directional.

Follow-up

Firm-wide write-off rate falls 30% in one quarter. Give one benign and one alarming explanation, and the query that separates them.
The steering group insists on a single number. What do you hand them, and what do you decline to do?

Firm margin rose while every pricing model's margin fell

Easy · mix shift · margin · decomposition

Firm-wide engagement gross margin rose from 31% to 34% quarter over quarter. Split by fct_engagement.pricing_model, margin fell inside time_and_materials, fixed_fee, retainer and outcome_based. Sources are fct_invoice_line (engagement_id, line_type, amount_usd, period_start_date, period_end_date) and fct_time_entry (engagement_id, hours, cost_rate_usd, status). Explain the aggregate move, quantify how much of the 3-point change is mix and how much is within-model, and state which number you put in front of practice leadership and why.

Approach

Write the aggregate as a weighted mean: M = sum over pricing models p of w_p * m_p, where m_p is margin within p and w_p is p's share of the fee base. The weights must be the fee base, because margin is a fees-weighted ratio; weighting by engagement count answers a different question.
Apply the exact three-term decomposition: dM = sum (w_p1 - w_p0) * m_p0 + sum w_p0 * (m_p1 - m_p0) + sum (dw_p)(dm_p). Verify the three components sum to the reported +3.0 points before interpreting any of them.
Identify which weights moved and in which direction; the aggregate can only rise this way if fee share shifted toward the structurally higher-margin models, typically fixed_fee delivered under budget or retainer.
Guard against composition inside each stratum: repeat the same decomposition within pricing_model by practice_area and account_tier, so a within-model decline is not itself an unexamined mix effect.
Confirm both sides of the ratio are on the same clock and the same scope: numerator includes line_type IN ('fees','milestone','credit_note') with credits signed negative; the cost side includes all approved delivery hours, billable and non-billable alike.
Report within-model margin as the headline delivery signal and mix as a separate, named line with its own owner, since mix is a sales decision and margin is a delivery one.

Follow-up

Next quarter the mix reverts. What happens to the aggregate, and what will you have already told leadership so this is not a surprise?
Which of the four pricing models should not be pooled with the others even for a mix-adjusted number, and why?
How would you present this if the mix shift were deliberate strategy rather than accident?

Suggested practice checklist

For someone who has spent the last year in notebooks, dashboards or modelling work and has not written raw SQL under time pressure. The first four days rebuild query fluency against a fixture you control and can verify by hand; the last three attach that fluency to the rest of the loop.

Day 1 — Build a fixture you can check answers against

Create a local Postgres or SQLite database with four tables (users, sessions, events, orders) holding roughly 200 rows you generated yourself, so you know the contents well enough to predict every result.
Deliberately seed the cases that break queries: a user with no sessions, a session with no events, two orders sharing a timestamp, a NULL in one join key, and one duplicated user row.
Before writing any SQL, hand-compute five answers on paper (how many users placed at least one order, median orders per ordering user, and three others) and save them as the ground truth for the week.

Deliverable: A one-command seed script plus a text file of five hand-computed answers to grade every later query against.

Day 2 — Joins, filters and NULL semantics

Answer "which users have no orders" three ways (LEFT JOIN with IS NULL, NOT EXISTS, NOT IN) and confirm that the NOT IN version returns zero rows once the subquery contains a NULL, because the comparison is never TRUE.
Reproduce the LEFT JOIN that silently collapses to an inner join by putting a right-table predicate in WHERE, then fix it by moving the predicate into the ON clause, and record both row counts.
Create a fan-out bug on purpose by joining orders to order_items and summing the order total, then correct it with a pre-aggregated subquery and explain in one line which table changed the grain.

Deliverable: One annotated .sql file holding the three join traps, each with the wrong result and the corrected result side by side.

Day 3 — Window functions and frames

Write three window queries against the fixture: a running order total per user, the rank of each order within its user by value, and the day gap to that user's previous order, then check each against the day-one ground truth.
Run ROW_NUMBER, RANK and DENSE_RANK over a column containing ties, print all three side by side, and write one sentence on when each is the correct choice.
Switch one query from the default frame (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, which is what you get when ORDER BY is present and no frame is written) to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and explain why the output differs only when the ORDER BY column has duplicates.

Deliverable: Three verified window queries plus a short note explaining the RANGE versus ROWS difference in your own words.

Day 4 — The four analytical query patterns

Write a monthly retention grid: first order month per user, then months-since-first as the column, and verify that month zero equals the cohort size exactly.
Sessionize the events table under a 30-minute inactivity rule using LAG plus a cumulative sum over a new-session flag.
Build a four-step funnel that counts distinct users rather than events at each step, and state the rule you applied to a user who reaches step three without ever logging step two.

Deliverable: One file with the retention, sessionization and funnel patterns, each carrying a one-line note on the assumption it bakes in.

Day 5 — Write SQL the way you will have to write it live

Set a 12-minute timer and solve three medium prompts in a plain editor with no execution and no autocomplete, then run them and tally syntax errors separately from logic errors.
Narrate one solution aloud while writing it, stating the grain of each intermediate result (one row per user, one row per user-day) before you type its body.
Rewrite your slowest solution as a CTE chain where every CTE name states its grain, and time yourself re-solving it from blank.

Deliverable: A recording of one narrated solution plus an error tally that separates syntax from logic.

Day 6 — One day for everything that is not SQL

Write the preconditions of the two-sample t-test from memory, then check them: independent observations, and a difference in means whose sampling distribution is approximately normal, which at large sample sizes follows from the central limit theorem rather than from normality of the raw values.
Write the difference between an odds ratio from logistic regression and a relative risk, and state the condition under which the two are close (low outcome prevalence).
Prepare a 90-second answer to "how would you know this model is any good" that names the metric, the baseline you would beat, and the cost of the errors you care about.

Deliverable: One page of notes covering test preconditions, the odds-ratio caveat and the model-quality answer.

Day 7 — Full loop rehearsal

Run a 45-minute mock with someone willing to interrupt: 20 minutes of SQL, 15 minutes defining a metric, 10 minutes on a past project.
Re-solve from blank the two queries you were slowest on this week and compare the times against day five.
Write a five-line answer to "walk me through a project" that puts a number in the first sentence and names the decision the work changed.

Deliverable: Mock feedback notes plus a timed project narrative you can deliver without reading it.

Rigour against a deadline

Nearly every data role forces a trade between the analysis you want and the one that fits the decision window. Prepare a case where you deliberately shipped something less rigorous, named the weakness to the person relying on it, and said what would change your answer. The naming is the part interviewers listen for.

Describe a situation where you had to mentor a peer or lead a project through ambiguity.

Medium · behavioural and stakeholder questions

Approach

State the situation in two sentences and spend the rest on your reasoning.
Close with what you would do differently, concretely.
Name the disagreement or constraint, and how you resolved it with evidence.

Follow-up

What did you decide not to do, and why?
How did you know the outcome was caused by your change?

Answering a request to redefine the utilisation denominator

Hard · denominator integrity · restatement · escalation

Before a quarterly review, a practice lead asks for two changes to reported billable utilisation: drop consultants allocated to engagements with status = 'paused' out of the denominator, and count approved travel hours booked to client_delivery as billable. Together the changes move reported utilisation from 71 to 79 percent. You must answer in writing today. Give your answer, the principle you apply to each change separately, and what you do if you are overruled.

Approach

Name the probe: whether you can hold a measurement line by adjudicating each proposal on its merits, rather than treating the request as bad faith or accepting it to avoid a fight.
Adjudicate the paused-engagement change against what the denominator means. Billable utilisation measures the share of available consultant hours that were sold. A consultant whose engagement is paused is still available and still costs salary, which is precisely the situation the metric exists to reveal. Removing them defines away the signal, so this one is refused on definitional grounds, not on suspicion.
Adjudicate the travel change as an empirical question, because it is one. If those hours are billed to the client, is_billable and bill_rate_usd should already reflect it; if they are charged and then written off or never invoiced, counting them raises utilisation while realisation falls. Check the invoice lines for the same engagement and period rather than arguing about principle.
Quantify each change separately, and the interaction. An eight-point move made of one point and seven points is a different conversation from two four-point moves, and the lead cannot make a good decision without that split.
If a change is accepted, restate history on the new definition and publish the old series alongside it with the date of the change. A definition applied forward only makes every period-over-period comparison meaningless. If overruled, comply with the restatement and the footnote, and escalate only the part that is a misstatement of fact rather than a defensible choice.

Follow-up

Which of the two changes would you accept, and what evidence flips your answer on the other one?
The lead wants the new definition applied only from this quarter forward. Why is that worse than applying it to history, and how do you explain that in one sentence?

Three requests, one week, and the one you defer

Medium · prioritisation · stakeholder management · scope reduction

Three requests arrive in one week. Finance needs days sales outstanding recomputed for a board meeting in four days. A partner wants a proposal win-rate model for a pursuit review in three weeks. Delivery wants a staffing forecast, with no date attached. You have one week of your own capacity and no analyst. Write the prioritisation you send back, the request you defer, and the message you send to the person whose request you defer.

Approach

Name the probe: whether you prioritise on decision dates and reversibility, or on who asked most forcefully.
Score each request on four things: what decision it changes, the date that decision is made, the cost of being late, and the cost of being wrong. A board figure has a hard date and a high cost of being wrong; a forecast with no date has neither.
Reduce scope rather than dropping work. The win-rate model is the largest piece and the easiest to get wrong, because features written after the decision, such as engagement_id and revised pricing, leak the label, and because withdrawn and no_decision proposals are not missing at random. A two-day descriptive win-rate cut by is_competitive and loss_reason answers most of what a pursuit review needs, with the model scoped separately.
Defer explicitly, with a date and a smaller substitute, rather than leaving a request to decay quietly. Silence is read as agreement and then as failure.
Put the trade-off in writing so it can be overturned by someone with more context than you have, and say what you would drop if the deferred request becomes urgent.

Follow-up

The partner escalates to the practice lead. What do you change, and what do you refuse to change?
What evidence would make you drop the finance work instead?