Describe internship and research projects

Behavioral/Leadership Prompt: Two Projects (Internship + Research)

Context

You are interviewing for a Machine Learning Engineer role during a technical screen. The interviewer wants concise, structured evidence of end-to-end ownership, technical depth, and measurable impact.

Task

Briefly introduce two projects—one internship and one research. For each project, cover:

1. Problem and constraints (business/user goal, scale, latency/memory limits, data availability)
2. Your role and ownership (what you personally led/built/decided)
3. Key technical decisions and why (model/data/pipeline/metrics; trade-offs)
4. Notable challenges and how you addressed them (failure modes, debugging, constraints)
5. Measurable results/impact (offline and online metrics, A/B outcomes, latency/throughput)
6. One improvement if you had more time (next step, risk you’d retire, or scalability plan)

Keep each project to ~2–3 minutes. Use concrete numbers where possible (e.g., +2.1% CTR, p99 latency 45 ms, AUCPR +0.16).

Clarifying Questions to Ask

• Clarify the role, scope, timeline, stakeholders, and what success looked like.
• Use a real example with enough context for the interviewer to evaluate your judgment.
• Separate your own actions from team actions and quantify the result when possible.

What a Strong Answer Covers

• A concise STAR or STAR+Reflection story with a specific situation and clear stakes.
• Concrete actions, trade-offs, communication choices, and ownership of mistakes or risks.
• A measurable result and a reflection on what you would repeat or change.
• Answers to likely probes about conflict, ambiguity, prioritization, and follow-through.

Follow-up Questions

• What would you do differently if the same situation happened again?
• How did you keep stakeholders aligned when priorities changed?
• What evidence shows that your actions changed the outcome?

Solution

Solution Alignment

The improved prompt asks for a structured answer that states assumptions, covers edge cases, and explains trade-offs. The answer below preserves the original solution content while making the expected interview coverage explicit.

Interview Framing

• Start by restating the goal and the assumptions you need.
• Work through the main approach in the same order as the prompt.
• Call out trade-offs, edge cases, and validation steps before finalizing the recommendation.

How to Answer (Template)

Use a tight STAR+R structure (Situation, Task, Action, Result, Reflection):

• Situation/Problem: Who is the user? What metric matters? What constraints exist (e.g., p99 latency ≤ 50 ms, 10M DAU, 1% positives)?
• Role/Ownership: What did you own end-to-end? What decisions were yours vs. the team’s?
• Actions/Key Decisions: Models, features, metrics, infra; explain trade-offs and why you chose them.
• Challenges/Fixes: Data leakage, skew, latency, cost, instability; how you diagnosed and resolved them.
• Results/Impact: Offline (AUC, AUCPR, loss), online (CTR, watch time, conversion), reliability (p99, QPS), and business outcomes. Include numbers.
• Improvement: The most leveraged next step (e.g., sequence model, bandits, better labeling, caching).

Tip: Tie offline to online. Call out constraints (latency, memory, privacy). Mention guardrails (holdouts, canaries, power).

---

Example 1 — Internship: Real‑Time Feed Ranking Upgrade

• Problem

  • Goal: Increase short‑video feed engagement (CTR and watch time) without exceeding a 50 ms p99 inference budget for ranking.
  • Scale: ~30M daily ranking requests; mixed device types.

• Role & Ownership

  • ML Engineer intern. Owned candidate feature pipeline and ranking model upgrade from baseline logistic regression to a low‑latency gradient‑boosted tree model.
  • Led offline evaluation, feature ablations, and partnered with platform team for serving optimizations.

• Key Technical Decisions and Why

  1. Model: Chose XGBoost with monotonic constraints over LR and over deeper DNNs.
     • Why: Captures nonlinearities for CTR while maintaining predictable low latency on CPU; monotonic constraints stabilized business‑critical features (e.g., prior views → non‑decreasing CTR).
  2. Features: Added sequence‑aware aggregates (e.g., last‑N engagement rates), time‑decayed counts, and content‑creator embeddings.
     • Why: Recent behavior often predicts short‑term interest; decay reduces drift; embeddings generalize to sparse IDs.
  3. Data & Splits: Time‑based splits; K‑fold target encoding with out‑of‑fold leakage prevention.
     • Why: Prevents future→past leakage and overfitting via naïve encodings.
  4. Serving: Feature store with feature parity checks; model quantization (INT8) + ONNX Runtime; micro‑batching for CPU vectorization.
     • Why: Match training/serving features; meet p99 latency and reduce cost.
  5. Metrics: Optimized for calibrated pCTR and Precision@K; business eval on CTR and avg watch time.
     • Why: Ranking quality aligns with surface CTR/watch time; calibration helps auction‑style blending.

• Challenges and Fixes

  • Training–serving skew: Mismatch in time windows caused offline–online drift.
    • Fix: Feature contracts in the feature store, unit tests on window bounds; CI to diff stats (mean/std, missing rate) between offline and online.
  • p99 latency regression (63 ms) after feature expansion.
    • Fix: Quantization + cache hot features; pruned low‑gain trees; achieved 44 ms p99.
  • Cold‑start items/users.
    • Fix: Back‑off features using content metadata and creator priors; hashing for unseen IDs; calibrated fallback scores.

• Results/Impact

  • Offline: AUC +0.018; Calibrated ECE reduced from 0.045 → 0.021.
  • Online A/B (2 weeks, 50/50, CUPED): CTR +2.1% (p<0.05), avg watch time/session +1.3%, no increase in bounce; p99 44 ms (−12% vs baseline) and −18% CPU per 1k requests.

• One Improvement

  • Sequence model (DIN/DIEN or lightweight Transformer) for user–item interactions under a 50 ms budget using distillation to a tree or a small MLP; expected +0.5–1.0% CTR with careful caching.

• Guardrails Used

  • Time‑based split; canary rollout (5%) before 50/50; kill‑switch on p99 > 55 ms; feature drift alerts.

• Small Numeric Example: A/B Power Check

  • Baseline CTR p0 = 5.0%, target uplift δ = +0.10 pp (2% relative). For two‑sided α=0.05, 1−β=0.8:
    • n per group ≈ 2 × (z_{0.975}+z_{0.8})^2 × p0(1−p0) / δ^2 ≈ 2 × (1.96+0.84)^2 × 0.05×0.95 / 0.001^2 ≈ 25.5M impressions per arm.
  • Ensures observed +0.1 pp is detectable.

---

Example 2 — Research: Semi‑Supervised Multimodal Content Moderation

• Problem

  • Goal: Detect policy‑violating content in short videos with limited labeled data and heavy class imbalance (~1% positives). Prior system had poor recall at low FPR, causing reviewer overload.
  • Constraints: Keep FPR ≤ 1% to avoid creator friction; support 100+ videos/sec/GPU.

• Role & Ownership

  • Lead graduate researcher. Designed training objective, multimodal architecture, and labeling strategy; built the pipeline, ran ablations, and led offline ↔ online validation.

• Key Technical Decisions and Why

  1. Semi‑Supervised Learning: Mean‑Teacher consistency + pseudo‑labels with confidence threshold τ=0.9.
     • Why: Leverage millions of unlabeled videos; teacher EMA stabilizes training; high‑confidence pseudo‑labels mitigate noise.
  2. Losses: Focal loss for labeled data to handle imbalance; consistency loss for unlabeled.
     • Focal loss: FL(p_t) = −α (1−p_t)^γ log(p_t), used γ=2, α tuned via validation.
     • Why: Emphasizes hard positives/negatives; improves AUCPR under skew.
  3. Multimodal Late Fusion: Text (ASR/transcript) → transformer encoder; vision → EfficientNet; audio → CNN; stacked with calibration (isotonic).
     • Why: Different modalities catch different violations; late fusion is robust to missing modalities.
  4. Metrics: Optimized AUCPR offline; selected operating points by maximizing recall at FPR=1%.
     • Why: AUCPR reflects performance under imbalance; operations need stable FPR.
  5. Robustness: Strong augmentations (SpecAugment for audio, RandAugment for vision), and label‑noise filtering via small‑loss trick.

• Challenges and Fixes

  • Label noise and spurious correlations (e.g., background text).
    • Fix: Co‑teaching small‑loss filtering; SHAP audits to remove shortcut features; added text masking augmentation.
  • Distribution shift across regions and languages.
    • Fix: Domain‑specific batch norm; language‑aware heads; calibrated thresholds per locale.
  • Threshold calibration instability.
    • Fix: Isotonic regression on a time‑decayed validation set; temperature scaling for each modality prior to fusion.

• Results/Impact

  • Offline: AUCPR 0.42 → 0.58; at 1% FPR, recall 60% → 77%.
  • Ops impact (pilot, 4 weeks): −18% manual review volume at same precision; reviewer SLA variance −22%.
  • Throughput: FP16 inference, 120 vids/sec/GPU; p95 latency 38 ms per video modality pass; graceful degradation if ASR missing.

• One Improvement

  • Active learning loop with uncertainty sampling + diversity (k‑center) to label edge cases weekly; expected AUCPR +0.03 with ~5k labels/week. Longer‑term: pretrain with contrastive multimodal learning (CLIP‑style) on 100M public videos.

• Validation/Guardrails

  • Strict temporal validation; per‑locale holdouts; fairness slice checks (language/creator size). Shadow deployment before partial automation. Alerting on FPR drift using Wilson intervals.

---

Pitfalls to Avoid

• Vague ownership (“we did”); always state what you personally built/decided.
• Only offline metrics; always tie to business/user impact and latency/cost.
• Ignoring guardrails (leakage checks, canaries, power analysis, rollback).
• No trade‑offs; explain why not another model (e.g., DNN too slow, ANN retrieval unnecessary, etc.).

Use the template to swap in your own projects; keep each to 6 bullets with crisp numbers and one thoughtful improvement.

Checks and Follow-ups

• Verify that the answer addresses every requested part of the prompt.
• Identify the highest-risk assumption and explain how you would validate it.
• Be ready to discuss an alternative approach and why you did not choose it first.