Deliver an elevator pitch and impact example

Elevator Pitch + End-to-End Experimentation Case + “Why Meta?”

Context

You are interviewing for a Data Scientist role during a technical screen. Use concise, decision-oriented communication with concrete numbers.

Task

1. Elevator Pitch (≤60 seconds)

   • Who you are.
   • The scale you’ve operated at.
   • Your superpower.

2. Experimentation Project (end-to-end, with measurable business impact)

   • Problem framing.
   • Hypothesis.
   • Unit of randomization (and why).
   • Primary metric and guardrail metrics (and why).
   • Sample size, power, and planned duration.
   • Pre-registration/analysis plan.
   • Execution challenges and how you addressed them.
   • Final results with concrete numbers (e.g., +X% conversion at Y% significance, Z p.p. change in a guardrail).
   • Causal story (why it worked), trade-offs considered, and what you’d do differently.

3. Why Meta?

   • Map your motivations to a specific product surface you’d join.
   • Explain how your skills fit that team’s needs.

Solution

1) 60-second Elevator Pitch

• I’m a data scientist with 7+ years in consumer growth and marketplace/notifications. I’ve run 200+ online experiments across products reaching 100M+ MAU, shipping features that moved DAU and revenue at scale.
• My superpower is turning ambiguity into decision-ready experiments: crisp problem framing, clean metrics, and pre-registered analyses that stakeholders trust.
• I partner closely with engineering and PMs, and I’m known for fast, reliable reads (CUPED/stratification) and telling a causal story that drives roadmap choices.

Tip: Practice a 3-sentence version: role + scale, superpower + one quantified impact, collaboration style.

2) Experimentation Case Study: Send-Time Personalization for Push Notifications

Scenario: We wanted to grow high-quality sessions by sending each user notifications at their best time-of-day.

A) Problem Framing

• Observation: Notification open rates were flat, and weekly opt-out (“mute/unsubscribe”) rates were creeping up by +0.05 p.p./week.
• Goal: Increase notification-driven session starts without harming user experience.
• Decision: Build a per-user send-time model versus fixed times; test via A/B.

B) Hypothesis

• H1: Personalizing send-time will increase notification-driven session starts per user-week by ≥2% relative.
• H2 (guardrail): Opt-out rate will not worsen by more than +0.10 percentage points.

C) Unit of Randomization

• User-level randomization (1:1). Rationale: Treatment is delivered at the user level; minimal network interference; avoids contamination.
• Stratified by: app platform (iOS/Android), region (US/ROW), and engagement tier (low/med/high) to balance covariates and improve power.

D) Metrics

• Primary: Notification-driven session starts per user per week.
  • Attribution: session within 10 minutes of a received push (last-touch).
• Key secondary: Notification open-through rate (OTR).
• Guardrails:
  • Opt-out/mute rate (weekly, p.p.).
  • Negative feedback rate on notifications (p.p.).
  • Battery impact (avg CPU/network per active user).
  • Experiment collision rate (overlapping tests), crash rate.

E) Sample Size, Power, Duration

• Design: Two-sided test, α = 0.05, power = 0.80.
• Metric type: Approximate primary as continuous (sessions per user-week) with historical σ ≈ 0.90 and mean ≈ 0.80.
• Minimum Detectable Effect (MDE): +2% relative on mean = δ = 0.016 sessions/user-week.
• Formula (two-sample t-test approximation): n_per_group ≈ 2 × (Z_{1-α/2} + Z_{1-β})^2 × σ^2 / δ^2 With Z_{1-α/2} = 1.96, Z_{1-β} = 0.84: n_per_group ≈ 2 × (2.8)^2 × (0.9)^2 / (0.016)^2 ≈ 61,000 users per group per full week.
• CUPED variance reduction (25% observed historically) effectively reduces required n to ~46k per group.
• Duration: 14 days to cover two weekly cycles and weekend effects; 10% → 50% → 100% ramp within the experiment while maintaining 1:1 assignment.

F) Pre-registration / Analysis Plan

• Assignment: User-level ITT (intention-to-treat).
• Invariants check: Balance on key covariates (platform/region/engagement) and pre-period outcomes.
• Variance reduction: CUPED using prior-week sessions (X): Y_adj = Y − θ (X − E[X]), where θ = Cov(Y, X)/Var(X).
• Estimator: Difference-in-means with cluster-robust SEs at the user level; stratification fixed effects.
• Multiple metrics: Control family-wise error by pre-specifying primary and interpreting guardrails descriptively unless breached.
• Early looks: O’Brien–Fleming alpha-spending for optional stopping (checks at day 7 and 14).
• Exclusions: Known push-denied users; catastrophic log gaps; retain all others in ITT.

G) Execution Challenges

• Capacity limits: Coordinated with infra to stagger send windows; used feature flags to rate-limit.
• Time zones/daylight savings: Derived local send windows from device time; validated with synthetic tests.
• Event attribution: Implemented 10-minute last-touch rule and de-duplicated bursts.
• Experiment collisions: Registered and filtered users in high-conflict cohorts (other notif tests); monitored collision rate.
• Novelty effects: Tracked effect decay over the 2-week window; planned post-ramp holdout.

H) Results (illustrative but internally consistent)

• Primary: +2.6% sessions/user-week (ITT), 95% CI [+1.8%, +3.4%], p < 0.001.
• Secondary: +5.1% OTR, 95% CI [+3.9%, +6.3%].
• Guardrails:
  • Opt-out rate: −0.08 p.p. (improvement), 95% CI [−0.12, −0.04].
  • Negative feedback: +0.01 p.p., n.s.
  • Battery: +0.2% CPU per active user, within SLO.
• Heterogeneity (pre-specified): Larger effects for “low engagement” users (+4.3%) and evening-preferring clusters; iOS > Android.
• Business impact: At 50M eligible weekly users, +2.6% translates to ~1.3M incremental weekly sessions, with improved opt-out—approved for 100% rollout.

I) Causal Story (Why It Worked)

• Mechanism: Aligning send-time with user availability increases salience and reduces interruption cost. Higher last-touch probability leads to more opens and near-immediate sessions.
• Evidence: Lift concentrated where model confidence was high and during predicted peak times; no increase in negative feedback—suggests higher relevance rather than over-sending.

J) Trade-offs Considered

• Volume vs. quality: We held message volume constant to isolate timing; next step is jointly optimizing volume and timing.
• Fairness: Guarded against systematically deprioritizing certain time zones or work schedules; monitored subgroup effects.
• Platform complexity: Additional scheduling complexity vs. measurable lift; validated reliability under infra constraints.

K) What I’d Do Differently

• Long-run effects: Staggered geo rollouts with dark-holdout to measure persistence and novelty decay.
• Modeling: Contextual bandits for joint timing + content; incorporate cost-aware policies (battery, channel fatigue).
• Quality outcomes: Add downstream guardrails (session depth, well-being proxies) to avoid optimizing only last-touch.
• Interference checks: Small cluster-randomized holdout by household/device family to confirm negligible spillovers.

Teaching notes: The key is crisp pre-specification, a defensible primary metric that maps to business value, realistic power math, and a clean causal narrative. Guardrails should reflect user trust and system health.

3) Why Meta? Product Surface + Fit

• Motivation: I’m excited by Meta’s scale, rapid experimentation culture, and the chance to balance growth with integrity and long-term user value.
• Product surface: Instagram Reels notifications and discovery. It’s a high-leverage surface connecting creators and viewers where timing, ranking, and user well-being all matter.
• Fit: My strengths in experimental design (powering large-scale AA/A/Bs, CUPED, stratification), causal inference, and metric design map directly to optimizing alert relevance, watch-time quality, and opt-out/negative feedback guardrails. I’m comfortable partnering with engineering to build reliable experimentation plumbing and with PMs to define MDEs that matter.
• Impact plan: Start by auditing metrics and invariants, ship a fast E2E timing/content test with pre-registered guardrails, then scale via adaptive policies and heterogeneity-aware insights for creators and cohorts.

Checklist you can adapt:

• State the business problem in one sentence; name the lever (e.g., timing).
• Hypothesis with a numeric MDE that matters.
• Unit of randomization and interference rationale.
• Primary metric and 2–4 guardrails tied to user trust/system health.
• Power math with assumptions and a duration plan.
• Pre-registration: ITT, variance reduction, multiple-testing approach.
• Execution risks and mitigations.
• Results with CI/p-values and p.p. changes on guardrails.
• Causal story, trade-offs, and a concrete “do next.”
• Close with a specific team/surface and how your skills drive impact there.