Interview guides · BNY
BNY Data Scientist Interview Questions & Guide 2026
Data scientist interview prep for BNY: original practice on experiment design, SQL and metric reasoning, with worked solutions.
The 60-second brief
As a Data Scientist at BNY, you sit at the unique intersection of cutting-edge technology and global financial services. BNY is a premier financial institution responsible for safeguarding and managing trillions of dollars in assets. This unprecedented scale means that the data science team is tasked with solving incredibly complex, high-impact problems. From optimizing asset management portfolios and predicting market trends to building advanced fraud detection systems and integrating generative AI into legacy pipelines, your work directly influences the stability and efficiency of the global financial ecosystem.
Product-sense cases reward reasoning from a mechanism to a testable prediction. Reciting every metric you can name reads as pattern matching; naming the single quantity that would move if your explanation were true reads as thinking.
Focus areas
- Decompose expected loss into PD, LGD, EAD
- Separate authorization, settlement and dispute outcomes cleanly
- Read vintage curves, not blended portfolio averages
Overview
The role of a Data Scientist here goes far beyond traditional statistical analysis. You will be building, scaling, and deploying production-grade machine learning models that process massive, highly structured, and unstructured datasets. Whether you are joining as a generalist or stepping into a senior role such as a Vice President Data Scientist, you will be expected to drive innovation, design robust machine learning pipelines, and translate complex algorithmic outcomes into actionable business strategies for executive stakeholders.
What makes this role particularly exciting is BNY's active commitment to modernizing its technology stack. The data science division is heavily focused on the latest trends, including Large Language Models (LLMs), deep learning, and advanced neural networks. By joining this team, you will have the opportunity to work on highly sophisticated financial products, collaborate with world-class engineers, and leverage substantial computing resources to deliver measurable business impact.
The BNY interview process
BNY candidates report 3 rounds · ≈ 3-5 weeks. The stages below are what candidates describe, not a published process.
Online Technical Assessment
Much of what gets scored here happens out loud while you type. Nobody can see your reasoning inside a half-written query, so five silent minutes read as being stuck even when they are not. State the plan in plain language first: which tables, what grain you are aggregating to, and the one filter that defines the population. Then write it. The narration doubles as insurance, because a wrong plan gets caught early and cheaply while a wrong query gets caught at the end with no time left to redo it. A timed statistics section, where one exists, is a separate test with its own clock.
What to demonstrate
- Whether the query you write matches the plan you just described
- What you do with a hint, meaning whether the correction gets absorbed or the first approach gets defended
- Whether you can debug your own wrong output by reading the result set and naming which part of the query produced the anomaly
How to prepare
- Solve three problems while screen-sharing into a recording, then watch it back and mark every stretch longer than thirty seconds where you said nothing
- Practise compressing the plan into one sentence before typing, then check afterwards whether the finished query actually matched it
- Time yourself on statistics questions that carry a business reading, such as what a confidence interval does and does not claim, rather than re-reading notes without a clock
Structured Technical Rounds
Before anything else, this round is a reading test. You are given a small schema and a question phrased in business language, and most of the difficulty sits in the gap between them. Who counts as an active user, does a refunded order still count as an order, is that date column an event time or a load time. Weak answers start typing immediately and compute something precise about the wrong population. Strong ones pin the definition in one sentence, name the column that encodes it, then write the query. On a timed assessment with nobody to tell, write the definition in a comment anyway.
What to demonstrate
- Whether an ambiguous term becomes a specific column and filter before any computation happens
- Whether you read the schema for keys and cardinality rather than only for column names
- Whether the result answers the question at the grain it was asked at, per user or per session or per day
How to prepare
- Take three metrics you already use and write down the exact filter and exact grain behind each, then practise stating one of them in a single sentence out loud
- On a schema you have never seen, spend the first minute writing what one row of each table means and which key it is unique on, then predict which joins can duplicate rows
- Rehearse a version where the definition changes halfway through, and edit the query you have instead of starting over
Onsite/Virtual Panel Interviews
Where a loop ends with a senior leader, that conversation is rarely another skills test. The technical signal already exists by then, so the questions tend to open up: what you would look at first, where a metric you have heard about could mislead, what you would push back on. The decision being made is scope, which in practice means level and how much you would be trusted to own unsupervised. Treating it as a formality is the usual mistake. An open question late in the day is still being scored, and a vague answer reads as someone who has not run anything themselves.
What to demonstrate
- Whether your view of the business has anything specific behind it, given that you are working only from what is public and are expected to say so
- Whether the scope of work you describe owning matches the scope of the role, instead of sitting a level below it
- Whether you can disagree with something concrete and stay useful about it, rather than agreeing with everything said in the room
- Whether your questions are ones only this person could answer, as opposed to ones the recruiter already covered
How to prepare
- Build one view you could defend for two minutes using only public information: what the funnel probably looks like, which metric likely drives decisions, and where that metric could mislead. Being wrong for a stated reason survives this round; having no view does not
- Write down the largest piece of work you have owned from question to decision, who else touched it, and what you decided alone, then check that it reads at the level you are interviewing for
- Prepare one thing you would want changed if you joined and phrase it as a question rather than a verdict, so it opens a conversation instead of closing one
Where candidates lose points
Reading the most recent months of fraud and dispute rates as final
Consumer dispute rights commonly run around 120 days from the transaction or expected delivery date, and several reason codes run considerably longer, so the disputes belonging to a recent transaction month have simply not been filed yet. Any chart attributed by transaction date therefore slopes down at the right edge regardless of what is happening. The fix is to report only matured cohorts, or to apply development factors estimated from completed months and to show the estimate as an estimate.
Recalibrating an underwriting cutoff on approved and funded applicants only
Rejected applicants have no repayment outcome, and they were rejected because the incumbent model scored them badly, so the missingness depends directly on the outcome being modelled. Reject inference by augmentation or parcelling fills the gap using the incumbent model's own assumptions, which means it can confirm those assumptions but cannot test them. The only genuinely new information about the reject region comes from bureau performance on rejects who borrowed elsewhere, or from a deliberately randomised approval band around the cutoff.
Optimising accuracy on a heavily imbalanced target
State the base rate first, then choose the metric from the relative cost of a false positive against a false negative: precision and recall at the operating threshold, PR-AUC, or expected cost. At a 1 percent positive rate, predicting the majority class for everyone scores 99 percent accuracy and is worthless.
Ending an analysis without a recommendation or next step
Close with what you would do and what would change your mind, stated as a condition you can check later. If the evidence is genuinely inconclusive, recommend the specific next measurement and say what it costs in time or exposure.
Questions & practice
Describe how you would evaluate the performance of an unsupervised clustering model when no ground truth labels are available.
Medium · machine learning and modelling
Approach
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Pick an evaluation metric that matches the cost of each error type, not a default.
- Check what information would not exist at prediction time, and exclude it.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
Walk me through the steps you would take to fine-tune a pre-trained Large Language Model (LLM) for a domain-specific financial task.
Medium · machine learning and modelling
Approach
- Frame the prediction: the label, the moment of prediction, and the action it triggers.
- Set a baseline first, so any model has something honest to beat.
- Say how the offline result would be validated online before it is trusted.
Follow-up
- What would you monitor after launch to know the model is still valid?
- How would you choose the decision threshold, and who owns that choice?
Bootstrap a fraud loss rate that clusters within merchant
Medium · bootstrap · clustered resampling · heavy tails
You have a per-transaction frame with auth_id, merchant_id, settled_amount_reporting and net_loss_reporting, both already in one reporting currency. Most rows carry zero loss, a few carry large ones, and losses cluster within merchant. Using only numpy's random generator and no resampling helper from any library, write a bootstrap that returns a 95 percent interval for net fraud loss in basis points of settled volume, resampling merchants with replacement and taking all rows belonging to each drawn merchant. Also produce the naive row-level interval and state which you would report.
Approach
- State the estimator before writing it: total net loss divided by total settled volume, times 10,000. It is a ratio of sums, so each replicate recomputes both sums. Averaging per-transaction loss rates instead would weight a five-unit transaction like a five-thousand-unit one.
- Pre-aggregate loss and volume to merchant level once. For a ratio of sums, drawing merchants and taking all their rows is arithmetically identical to drawing merchant-level (loss_sum, volume_sum) pairs, so a replicate becomes one integer draw plus two vectorised sums rather than a groupby inside the loop.
- Draw B replicates of M merchant indices with replacement, where M is the observed merchant count, compute the ratio per replicate, and take the 2.5th and 97.5th percentiles. Say explicitly that this is a percentile interval and that BCa would correct the skew-induced bias if the decision is close.
- Repeat with independent row draws for the naive interval and compare widths on the same replicate count.
- Report the clustered interval. Rows within a merchant share an acceptance profile, a category code and a fraud exposure, so they are not independent, and the row-level interval understates variance by roughly the design effect.
Worked solution (30 min)
- Compute the point estimate directly on the full data and keep it for comparison.
- Aggregate to merchant-level loss and volume arrays, record M, and set B to 2,000 with a seeded numpy Generator.
- In a vectorised loop, draw integer indices of shape (B, M), index both arrays, sum along axis 1, and take the ratio times 10,000.
- Repeat for the row-level version using the per-transaction arrays and N draws.
- Take the 2.5 and 97.5 percentiles of each replicate array and report both intervals alongside the point estimate.
Expected result: Two intervals in basis points sharing the same point estimate. The clustered interval is materially wider, commonly two to four times, and is right-skewed because a heavy merchant can be drawn more than once in a replicate.
Verify it yourself
- The point estimate equals the direct ratio on the full data and lies inside both intervals.
- A synthetic input where every merchant holds exactly one transaction makes the clustered and naive intervals coincide; this is the test that the resampling unit is wired correctly.
- Raising B from 2,000 to 10,000 shifts the endpoints by less than the reporting precision; if it does not, a handful of merchants is driving the tail and the estimate needs that caveat.
- Replicate ratios never fall outside the range implied by drawing only the lowest-loss or only the highest-loss merchants.
Follow-up
- Your clustered interval is three times wider. How do you explain that to someone who wanted a tighter number?
- One merchant accounts for 40 percent of losses. What does that do to the interval, and what would you do about it?
- How does this change if the question is whether two months differ rather than what this month's rate is?
Explain the difference between list comprehensions and generator expressions in Python, and when you should use each for memory efficiency.
Medium · sql and data manipulation
Approach
- Say which table is the grain you start from, and join outward from it.
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
Follow-up
- What breaks if events arrive late or out of order?
- How would you verify this result without re-running the same query?
Write a Python function using pandas and numpy to clean a dataset containing missing values and outliers, and normalize the numerical features.
Medium · sql and data manipulation
Approach
- Compute rates by summing numerator and denominator separately, never by averaging rates.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
- State the window function and its partition and ordering out loud before writing it.
Follow-up
- How would you verify this result without re-running the same query?
- What breaks if events arrive late or out of order?
How would you optimize a slow-performing SQL query that involves multiple large table joins and aggregations?
Medium · sql and data manipulation
Approach
- State the window function and its partition and ordering out loud before writing it.
- Handle the rows that do not match: a LEFT JOIN with a NULL check is usually the question.
- Check whether any join is one-to-many before aggregating, or the sums inflate.
Follow-up
- How would you verify this result without re-running the same query?
- What breaks if events arrive late or out of order?
Accident-quarter loss ratio on earned rather than written premium
Hard · loss ratio · earned vs written · development age
From fct_policy_period_monthly, compute the accident-quarter loss ratio by product_line: incurred losses, being paid_loss_minor plus case_reserve_minor plus ibnr_reserve_minor, over earned_premium_minor for the same accident quarter. State explicitly whether loss_adjustment_expense_minor is included and apply that choice consistently. Also output the same ratio computed on written_premium_minor so the two can be compared. The table holds current values with no valuation-date snapshot. Say in one line which comparison this schema cannot support and what you would need to support it.
Approach
- Derive the accident quarter from as_of_month with date_trunc, and note that the table already attributes losses to the month of the loss event while earning premium pro rata into the same month, which is what makes the two sides comparable at all.
- Aggregate earned_premium_minor, written_premium_minor and the three loss components to product_line and accident quarter in one pass, keeping loss adjustment expense as its own column so the inclusion choice is a final-select decision rather than something buried in a CTE.
- Compute both ratios side by side and a third column for their difference, because the size and sign of that difference is a direct read on whether the book grew or shrank in the quarter.
- State the limitation plainly: every row carries today's reserve estimate, so each accident quarter is observed at a different development age and a cross-quarter comparison mixes development with underwriting. A fixed development age needs a valuation-date dimension, that is one row per accident period per valuation, which this table does not have.
- Guard against the mirror-image error on the numerator by confirming ibnr_reserve_minor is non-zero on recent quarters; if it is null or zero there, the recent periods are understated twice over and the series is not usable.
Worked solution (40 min)
- CTE quarterly: group fct_policy_period_monthly by product_line and date_trunc('quarter', as_of_month), summing earned_premium_minor, written_premium_minor, paid_loss_minor, case_reserve_minor, ibnr_reserve_minor and loss_adjustment_expense_minor.
- Final SELECT: build incurred_minor as the three loss components plus the LAE column, with the LAE inclusion written as a named expression so the choice is visible on the page.
- Emit loss_ratio_earned and loss_ratio_written, both cast to numeric, plus their difference and the written-to-earned premium ratio.
- Order by product_line and accident quarter, and append the one-line note about the missing valuation dimension to the query as a comment.
Expected result: One row per product_line and accident quarter with earned and written premium, incurred losses, both ratios and their difference. In quarters where written premium exceeds earned, loss_ratio_written is strictly below loss_ratio_earned, and the gap widens with the growth rate.
Verify it yourself
- For a mature product line with flat volume, summed earned premium over a full policy term is close to summed written premium, so the two ratios converge there.
- The sign of loss_ratio_earned minus loss_ratio_written matches the sign of written premium minus earned premium in every quarter, which is an arithmetic identity and not a data finding.
- Recent accident quarters have non-zero ibnr_reserve_minor; a zero there means the numerator is missing its tail and the ratio is understated.
- Within a quarter, loss_ratio_earned divided by loss_ratio_written equals written_premium_minor divided by earned_premium_minor exactly, because both ratios share one numerator. Any deviation means the LAE term was built into one ratio and not the other, which is the real test that the inclusion choice was applied consistently.
- Recomputing with LAE excluded scales each quarter's ratio by incurred_minor / (incurred_minor + lae_minor). That factor is not constant across quarters unless LAE happens to be booked as a fixed fraction of incurred losses, so a uniform shift is evidence about how LAE is loaded, not evidence that the query is self-consistent — use the previous check for that.
Follow-up
- Written premium exceeds earned premium by 18 percent this quarter and by 3 percent two years ago. What happened to the book, and what does it do to each ratio?
- How would you build a development triangle from a valuation-dated version of this table, and what would you use the chain-ladder factors for?
- Statutory presentation conventionally takes the expense ratio on written premium while the loss ratio uses earned. How do you avoid a combined ratio that quietly mixes the two bases?
How would you design a recommendation engine for financial products while ensuring strict compliance with data privacy regulations?
Medium · product sense and metrics
Approach
- Fix the population and the time window before naming any metric.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- State what result would change your recommendation, so the answer is falsifiable.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
How do you prioritize your work when managing multiple high-priority data science projects with tight deadlines?
Medium · product sense and metrics
Approach
- State what result would change your recommendation, so the answer is falsifiable.
- Name one primary metric, then the guardrail that stops it being gamed.
- Fix the population and the time window before naming any metric.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Describe the architecture for a document search and question-answering system using Retrieval-Augmented Generation (RAG) for financial reports.
Medium · product sense and metrics
Approach
- Name one primary metric, then the guardrail that stops it being gamed.
- Decompose the metric into the rates that drive it, and say which one you would check first.
- Fix the population and the time window before naming any metric.
Follow-up
- How would you detect that the metric is being gamed rather than genuinely improving?
- What would you do if the primary metric and the guardrail moved in opposite directions?
Size a two-week test on the ecommerce authorization path
Easy · power · mde · ratio metrics · clustering
Your team wants to test a new decline-retry policy on the ecommerce channel. The metric is the dollar-weighted authorization approval rate from fct_payment_authorization: approved amount_minor over attempted amount_minor, after collapsing retry chains within a 15-minute window on (card_token_id, merchant_id, amount_minor) and excluding is_reversal rows and zero-amount verifications, all converted to one reporting currency. Baseline is 87 percent on roughly 900,000 collapsed attempts in two weeks. Randomisation is by customer_id, mean six attempts per customer. Give the minimum detectable effect at 80 percent power, two-sided alpha 0.05, and say what you would change if it is too large.
Approach
- Compute the count-weighted binomial anchor first, because it is the number everyone expects and you need it to show why it is wrong. With n = 450,000 attempts per arm and p = 0.87, MDE = (1.96 + 0.8416) * sqrt(2p(1-p)/n).
- State that the real metric is a ratio of two sums with a random denominator, not a Bernoulli mean, so the binomial standard error is not the right one. Linearise: Var(R) is approximately Var(Y_i - R * D_i) / (n * Dbar^2), where Y_i and D_i are the approved and attempted amount totals for customer i, n is the number of customers, and Dbar is the mean attempted amount per customer.
- Apply the clustering correction, since assignment is by customer and attempts within a customer are correlated. Ask for or estimate the intra-customer correlation; the design effect is 1 + (m - 1) * rho with m = 6, so rho = 0.05 gives 1.25 and inflates the MDE by sqrt(1.25) = 1.118.
- Do not assume the amount-weighted variance; estimate it by bootstrapping customers from the last eight weeks of history and resampling whole customers, which captures both the skew in amount_minor and the within-customer correlation in one step.
- If the MDE exceeds the plausible effect, list the levers in order of cost: extend duration (MDE falls as 1/sqrt(n)), restrict to the segment where the rule can bind at all rather than diluting across all traffic, apply CUPED on the customer's pre-period approved amount, or switch the primary to the count-weighted rate and demote the dollar-weighted rate to a secondary read.
Worked solution (20 min)
- Split 900,000 collapsed attempts into 450,000 per arm and compute the binomial anchor: sqrt(2 * 0.87 * 0.13 / 450000) = sqrt(5.027e-7) = 7.090e-4; multiply by 2.8016 to get 1.99e-3.
- Derive the customer count: 900,000 / 6 = 150,000 customers, 75,000 per arm, and note that this is the true sample size for inference.
- Apply the design effect at rho = 0.05: 1 + 5 * 0.05 = 1.25, so the count-weighted MDE becomes 1.99e-3 * 1.118 = 2.22e-3.
- State that the dollar-weighted MDE requires the linearised variance and cannot be derived from p alone; specify the customer-level bootstrap that would produce it and note it will be larger because amount_minor is right-skewed.
Expected result: Count-weighted MDE is about 0.20 percentage points unclustered and about 0.22 percentage points at a design effect of 1.25, against an 87 percent baseline (roughly 0.26 percent relative). The dollar-weighted MDE is not computable from the stated inputs and must be estimated by a customer-level bootstrap; the candidate should say so rather than quote the binomial figure for it.
Verify it yourself
- z(0.975) + z(0.80) = 1.9600 + 0.8416 = 2.8016; any MDE quoted with a different multiplier is using different power or alpha than stated.
- Doubling the test duration should shrink the MDE by exactly sqrt(2) = 1.414 if the traffic mix is stable; if a candidate's arithmetic does not, the formula is wrong.
- Run the bootstrap on two historical periods split A/A: the bootstrap standard error of the difference times 2.8016 should reproduce the analytic MDE.
- A/A splits of historical data should reject at roughly 5 percent; materially more means the variance model is understated.
Follow-up
- The rule only changes behaviour on declined attempts, which are 13 percent of traffic. How does restricting the analysis population to attempts that could have triggered the rule change both the MDE and the estimand?
- How would your sizing change if the split were 90/10 instead of 50/50, and why is the loss more than proportional?
- Currency: the metric sums amounts converted to one reporting currency. Would you use settlement_fx_rate or a rate table pinned at test start, and what does the choice do to variance?
Fraud losses appear to halve in recent transaction months
Medium · right censoring · development factors · maturity
A weekly chart attributes fct_card_dispute cases to the requested_at month of the linked fct_payment_authorization row. The two most recent months show the first-chargeback rate falling by half, and a risk rule shipped six weeks ago. Columns: dispute_id, auth_id, dispute_category, dispute_stage, opened_at, disputed_amount_minor, liability_shift_flag, outcome, net_loss_minor, resolved_at. Decide whether the rule worked, and produce the version of the chart you would sign your name to.
Approach
- Separate the two dates explicitly. opened_at is when a case was filed, requested_at is when the transaction happened. Attributing by transaction month is the right causal choice and is exactly what makes the newest months structurally incomplete.
- Measure the filing lag rather than assuming it: the distribution of opened_at minus requested_at over fully developed months, split by dispute_category, and the age at which around 95 percent of cases have arrived.
- Build a development triangle of transaction month by months of development on cumulative case counts, and estimate age-to-age factors from the columns that are complete.
- Develop the immature months with those factors and plot the result as an estimate with a visible band, kept visually distinct from the matured series rather than blended into it.
- State the assumption the method needs: a stable development pattern across cohorts. A change in filing behaviour, merchant mix or the dispute team's own backlog breaks it, so inspect factor stability down each column before relying on the estimate.
- Only then evaluate the rule, comparing pre-change and post-change cohorts at equal development age.
Follow-up
- What leading indicator would you accept while the cohort matures, and what is its known bias?
- How does liability_shift_flag change which disputes you should expect to see in the first place?
- If the rule also blocked good transactions, where does that cost appear, and is any of it in this chart?
Suggested practice checklist
For someone who has spent the last year in notebooks, dashboards or modelling work and has not written raw SQL under time pressure. The first four days rebuild query fluency against a fixture you control and can verify by hand; the last three attach that fluency to the rest of the loop.
Day 1 — Build a fixture you can check answers against
- Create a local Postgres or SQLite database with four tables (users, sessions, events, orders) holding roughly 200 rows you generated yourself, so you know the contents well enough to predict every result.
- Deliberately seed the cases that break queries: a user with no sessions, a session with no events, two orders sharing a timestamp, a NULL in one join key, and one duplicated user row.
- Before writing any SQL, hand-compute five answers on paper (how many users placed at least one order, median orders per ordering user, and three others) and save them as the ground truth for the week.
Deliverable: A one-command seed script plus a text file of five hand-computed answers to grade every later query against.
Day 2 — Joins, filters and NULL semantics
- Answer "which users have no orders" three ways (LEFT JOIN with IS NULL, NOT EXISTS, NOT IN) and confirm that the NOT IN version returns zero rows once the subquery contains a NULL, because the comparison is never TRUE.
- Reproduce the LEFT JOIN that silently collapses to an inner join by putting a right-table predicate in WHERE, then fix it by moving the predicate into the ON clause, and record both row counts.
- Create a fan-out bug on purpose by joining orders to order_items and summing the order total, then correct it with a pre-aggregated subquery and explain in one line which table changed the grain.
Deliverable: One annotated .sql file holding the three join traps, each with the wrong result and the corrected result side by side.
Day 3 — Window functions and frames
- Write three window queries against the fixture: a running order total per user, the rank of each order within its user by value, and the day gap to that user's previous order, then check each against the day-one ground truth.
- Run ROW_NUMBER, RANK and DENSE_RANK over a column containing ties, print all three side by side, and write one sentence on when each is the correct choice.
- Switch one query from the default frame (RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, which is what you get when ORDER BY is present and no frame is written) to ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW, and explain why the output differs only when the ORDER BY column has duplicates.
Deliverable: Three verified window queries plus a short note explaining the RANGE versus ROWS difference in your own words.
Day 4 — The four analytical query patterns
- Write a monthly retention grid: first order month per user, then months-since-first as the column, and verify that month zero equals the cohort size exactly.
- Sessionize the events table under a 30-minute inactivity rule using LAG plus a cumulative sum over a new-session flag.
- Build a four-step funnel that counts distinct users rather than events at each step, and state the rule you applied to a user who reaches step three without ever logging step two.
Deliverable: One file with the retention, sessionization and funnel patterns, each carrying a one-line note on the assumption it bakes in.
Day 5 — Write SQL the way you will have to write it live
- Set a 12-minute timer and solve three medium prompts in a plain editor with no execution and no autocomplete, then run them and tally syntax errors separately from logic errors.
- Narrate one solution aloud while writing it, stating the grain of each intermediate result (one row per user, one row per user-day) before you type its body.
- Rewrite your slowest solution as a CTE chain where every CTE name states its grain, and time yourself re-solving it from blank.
Deliverable: A recording of one narrated solution plus an error tally that separates syntax from logic.
Day 6 — One day for everything that is not SQL
- Write the preconditions of the two-sample t-test from memory, then check them: independent observations, and a difference in means whose sampling distribution is approximately normal, which at large sample sizes follows from the central limit theorem rather than from normality of the raw values.
- Write the difference between an odds ratio from logistic regression and a relative risk, and state the condition under which the two are close (low outcome prevalence).
- Prepare a 90-second answer to "how would you know this model is any good" that names the metric, the baseline you would beat, and the cost of the errors you care about.
Deliverable: One page of notes covering test preconditions, the odds-ratio caveat and the model-quality answer.
Day 7 — Full loop rehearsal
- Run a 45-minute mock with someone willing to interrupt: 20 minutes of SQL, 15 minutes defining a metric, 10 minutes on a past project.
- Re-solve from blank the two queries you were slowest on this week and compare the times against day five.
- Write a five-line answer to "walk me through a project" that puts a number in the first sentence and names the decision the work changed.
Deliverable: Mock feedback notes plus a timed project narrative you can deliver without reading it.
Bring numbers, including the ones you are unsure of
An answer without a quantity is hard to interrogate, so interviewers keep probing until they find one. Come with the baseline, the change, the window it was measured over, and how confident you were. If the effect never got measured, say so and say what you would have measured. Fabricated precision is worse than an honest gap.
How do you handle concept drift and data drift in a production machine learning pipeline?
Medium · behavioural and stakeholder questions
Approach
- Quantify the outcome, including what you would not claim credit for.
- Name the disagreement or constraint, and how you resolved it with evidence.
- Pick a story where you drove the decision, not one where you observed it.
Follow-up
- What did you decide not to do, and why?
- What would you do differently if you ran that project again?
Defend a vintage finding that contradicts the portfolio dashboard
Medium · vintage analysis · mix shift · stakeholder pushback
The lending dashboard shows blended 90-plus days-past-due falling for four consecutive quarters while originations grew 60 percent. Using fct_loan_performance_monthly, you build a vintage view keyed on origination_month by months_on_book and find the three most recent vintages are worse than their predecessors at the same age. The business lead presents that dashboard weekly and pushes back hard, suggesting you picked favourable cohorts. You get one meeting and the vintage table. Present the finding so it survives the cherry-picking objection and ends in a decision.
Approach
- Reconcile before you contradict: show that aggregating your vintage table along the calendar diagonal reproduces the published blended series, so the disagreement is about age mix rather than about data quality.
- Make the mechanism arithmetic rather than rhetorical: a loan cannot reach 90 days past due before it is 90 days old, so rapid origination growth shifts weight onto young months-on-book where the rate is structurally near zero.
- Show every vintage rather than a selected pair, all indexed at months_on_book equal to 12, with cohort sizes printed beside each curve so nobody can claim the divergence rests on a thin cohort.
- Handle restructuring explicitly, because restructured_flag resets days_past_due: count each loan on its worst pre-restructure state, or recent vintages will look better than they are.
- Close on the decision rather than the chart: state what the divergence implies for the cutoff or the channel mix, and state in advance what evidence would make you withdraw the claim.
Follow-up
- Two cohorts differ at month 12. How do you separate a seasoning effect from a genuine credit-quality effect?
- Someone argues the recent vintages are simply a broker-channel mix shift. How do you test that, and what would confirm it?
Allocate one analyst-week across three competing risk requests
Medium · prioritisation · decision deadlines · stakeholder negotiation
Three requests land in the same week and you have one analyst-week. Payments wants a merchant-level decline teardown before a contract renewal in nine days. Credit wants a swap-set analysis on a cutoff change scheduled to ship in six weeks. Insurance wants accident-quarter loss ratios at 12 months development for a reserving review with no fixed date. Each sponsor believes theirs is first, and each has escalated before. Produce the allocation, the reasoning you would say out loud to all three at once, and what you explicitly drop.
Approach
- Score each request on the decision it unblocks rather than on effort or on how loudly it arrived: what changes if it is late, and is that change reversible.
- Separate deadline from value. The nine-day renewal is a hard, irreversible date with a bounded prize; the six-week cutoff has slack but a much larger downside if it ships unmeasured; the reserving number has no date but feeds external reporting, which is its own kind of hard.
- Hunt for the cheap partial in each: a decline teardown restricted to the top merchants by declined value usually answers the contract question at a fraction of the full cut.
- Sequence by hard date first, then by largest irreversible downside, and deliver the trade-off to all three sponsors in one message rather than three, so nobody negotiates privately against a version you told someone else.
- Name what is dropped and who now owns that consequence, in writing, so the trade-off is visible rather than silently absorbed by you.
Follow-up
- The credit sponsor escalates to your manager. What do you change, and what do you refuse to change?
- How would you make this allocation reproducible so the next contested week is a rule application rather than a negotiation?