Analyze failed gym-collab credit card launch

Case: Postmortem for an Underperforming Credit Card × Gym Partnership

Context

Your team launched a credit card–gym partnership offering a benefit (e.g., statement credit, discount, or rewards multiplier) intended to drive partner sign-ups/enrollments and incremental card spend at gyms. The initiative underperformed targets on both sign-ups and spend.

Task

Run a structured postmortem. Your response should:

1. Establish a credible counterfactual to estimate incremental impact.
2. Diagnose root causes across:
   • Targeting
   • Positioning/creative and channel mix
   • Incentive design (offer value and mechanics)
   • Partner operations (in-store/on-site execution, integration)
   • Seasonality and macro confounders
3. Specify the data you would collect and the analyses you would run (e.g., funnel drop-off, churn cohorts, geographic difference-in-differences).
4. Recommend immediate mitigations versus longer-term pivots.
5. Outline how you would communicate the failure to executives and to the partner while preserving the relationship.

Assumptions (clarify or adjust if needed)

• "Sign-ups" refers to enrollments in the gym offer (or partner membership sign-ups attributed to the offer); "spend" refers to incremental gym-category card spend (MCC 7991/fitness) within 30/60/90 days post-enrollment.
• The program ran in a subset of geos/channels, enabling the use of untreated controls for comparison.

Solution

Structured, Teaching-Oriented Solution

0) Define success and set the stage

• Primary metrics
  • Incremental enrollments (vs. counterfactual) and incremental gym spend per eligible cardholder within 30/60/90 days.
  • Unit economics: Incremental margin LTV − incentive cost − media/ops cost ≥ 0.
• Secondary metrics
  • CTR/CVR through the funnel; approval/activation if new-card applications were part of the flow; time-to-first-gym-transaction; retention/churn of gym spend.
• Guardrails
  • Don’t rely on raw lifts; always anchor on a credible counterfactual with checks for bias.

1) Establish the counterfactual (incrementality)

Prefer randomized designs; if unavailable, use quasi-experimental methods.

• If you had an RCT/holdout:
  • ITT (intention-to-treat): mean outcome difference between exposed and holdout groups.
  • TOT (treatment-on-the-treated): adjust ITT by take-up rate.
• If no RCT, use quasi-experimental:
  • Difference-in-differences (DiD) across treated vs. matched control geos/branches/customers: DID = (Y_treat,post − Y_treat,pre) − (Y_ctrl,post − Y_ctrl,pre)
  • Ensure parallel pre-trends via event-study plots and placebo tests.
  • Matching/PSM: match on pre-period gym spend, tenure, demographics, credit risk, channel, seasonality exposure.
  • Synthetic control (if one/few treated units) to construct a weighted counterfactual from controls.
  • Negative controls: categories unlikely affected by the partnership (e.g., grocery spend) to detect spurious shifts.
• Attribution hygiene
  • Validate exposure logs, MCC classification, partner referral tags, and that measurement pipelines were live at launch.

Small numeric example: Suppose monthly gym spend per eligible cardholder moved from $9.0 to $11.0 in treated geos, and from $9.0 to $10.5 in controls over the same period. DID = (11.0−9.0) − (10.5−9.0) = $0.5 incremental per customer-month.

2) Data to collect

• Exposure and funnel
  • Impressions, clicks, landings, enrollments (timestamped), approvals/activations (if applicable), first transaction, repeat transactions.
  • Channel/creative variant, device, geo, audience/eligibility flags.
• Customer and history
  • Pre-period gym spend, overall spend, tenure, product type, credit bands, demographics (as permitted), opt-in status.
• Partner operations
  • Store list and hours, POS capabilities, BIN recognition settings, coupon/offer code acceptance, training completion logs, signage audits, lead capture counts, referral IDs.
• Offer and pricing
  • Incentive schedule (value, thresholds, caps), redemption/crediting latency, breakage, unit costs.
• Marketing/media
  • Spend by channel/geo/day, frequency/caps, competing campaigns.
• External factors
  • Seasonality (e.g., Jan/Sept peaks), local events, weather, macro shocks, competitor promotions.
• Quality and integrity
  • MCC mappings, de-duped customer IDs, bot filters, logging gaps.

3) Analyses to run

A. Measurement/QC

• Reconcile exposures → enrollments → first gym transaction counts across systems.
• Check MCC tagging for partner locations; spot-check with test swipes.
• Latency checks: time from purchase to statement credit; broken credits depress observed value.

B. Funnel and friction

• Compute stepwise conversion: exposure → click (CTR), click → landing, landing → enroll (CVR), enroll → first gym swipe, swipe → repeat.
• Segment by channel, creative, device, geo, and customer type (existing gym spender vs. non-spender).
• Identify the biggest drop-off and investigate root causes (UX, eligibility, approval, redemption friction).

C. Cohorts and retention

• 0/30/60/90-day spend curves for enrollment cohorts; time-to-first-transaction survival curves; churn hazard after first month.
• Compare to historical gym spenders vs. new-to-category customers.

D. Incrementality and heterogeneity

• DiD by geo/branch and by customer segment; cluster-robust errors at geo level.
• Event-study plots to test parallel trends and dynamics.
• Uplift modeling/CATE: E[Y|T=1,X] − E[Y|T=0,X] to find who actually benefits (e.g., fitness-interested, urban cores).

E. Diagnose specific failure modes

• Targeting failure
  • Exposure skewed to low-propensity audiences? Check pre-period gym spend propensity vs. exposure rates.
  • Low incremental lift in low-propensity segments indicates wasted reach.
• Positioning/creative
  • CTR and on-site CVR by creative; run logistic regression controlling for channel to isolate message effects.
  • Survey/lightweight UX tests to detect misunderstanding of the benefit or hidden eligibility.
• Incentive design
  • Is the offer salient and simple? Thresholds/caps too high? Delayed credit reduces perceived value?
  • Compute elasticity: enrollment lift per $ of expected value; compare to benchmarks.
  • Unit economics: Incremental margin × retention − incentive cost − media.
• Partner operations
  • Redemption failures at POS, staff unfamiliarity, missing signage → high enroll but low first-swipe.
  • Compare store clusters with high vs. low training completion.
• Seasonality/macros
  • Launch timing vs. gym intent cycles; add month fixed effects.
  • Placebo in non-gym categories to ensure effects are category-specific.

F. Sensitivity and robustness

• Alternative control sets, varying pre-period windows, excluding overlapping campaigns.
• Synthetic control as a cross-check.

4) Likely synthesis patterns (examples)

• Big drop from enroll → first swipe with partner stores showing low offer recognition → partner ops is primary driver.
• Good CTR but poor landing → enroll CVR → positioning/UX issue.
• Decent enrollments but low incrementality after DiD → high cannibalization (existing gym spenders switching cards, not increasing total spend).
• Flat lifts in summer/off-peak months → seasonality; moving spend to Jan/Sept could materially improve ROI.
• Weak or negative CATE in suburban low-density areas → mistargeting; concentrate on urban cores and existing gym spenders or New Year’s joiners.

5) Immediate mitigations (2–6 weeks)

• Fix the leakiest funnel step
  • If POS redemption fails: enable auto-statement credit using MCC detection; refresh BIN tables; hotfix training with job aids; mystery-shop and audit top-50 locations.
  • If landing CVR is low: simplify copy, remove hidden eligibility, show "you’ll get $X credited within Y days" with examples; reduce clicks to enroll.
• Targeting tightening
  • Suppress low-propensity audiences; prioritize customers with recent fitness app/device signals or prior gym spend; retarget cart-abandoners.
• Incentive tweaks
  • Increase immediate perceived value (e.g., first-month credit, lower threshold) while capping total cost; shorten credit latency.
• Media/geo reallocation
  • Shift budget to high-lift geos/segments; pause underperforming channels/creatives; ramp near partner flagship locations.
• Measurement and guardrails
  • Stand up geo holdouts; implement unique referral codes; daily QA on crediting pipeline.

6) Longer-term pivots (6–24 weeks)

• Redesign mechanics
  • From complex tiered thresholds to simple, auto-applied statement credit; or a limited-time high-salience New Year’s offer.
  • Performance-based economics with partner (rev share per incremental membership or per incremental spend).
• Product integration
  • Add "Fitness" as a rotating or always-on accelerated category beyond a single partner to reduce concentration risk.
  • Bundle with digital fitness apps/wearables; tie benefits to membership autopay for persistence.
• Targeting science
  • Always-on uplift modeling; lifecycle triggers around January/September and after life events.
• Experimentation rigor
  • Pre-registered RCTs or geo-lift tests with power calculations; persistent holdouts for true incrementality.
• Operational excellence with partner
  • Quarterly business reviews, standardized training, launch playbooks, signage SLAs, POS certification before mass launch.

7) Communication plans that preserve trust

A) Executives

• Structure: 1-page summary + appendix
  • What happened: Under target on enrollments (−X%) and incremental spend (−Y%).
  • Counterfactual and validity: Method (e.g., DiD with matched controls), pre-trend checks, sensitivity.
  • Root-cause attribution with effect sizes: e.g., partner ops issues explain ~60% of gap; incentive salience ~25%; seasonality ~15%.
  • Financial impact: Variance to plan, unit economics, risk to annual goals.
  • Action plan: Immediate fixes (2–6 weeks), pivots (6–24 weeks), owners, timelines, success criteria, and decision asks (budget reallocation, product changes).
  • Lessons learned: Launch timing, holdout design, QA gates.
• Tone: Candid, data-first, solution-oriented; show learning and a path to ROI-positive relaunch.

B) Partner

• Acknowledge shared goals and early results; appreciate their investment.
• Share a simplified, transparent readout focused on joint fixes (avoid blame):
  • What worked (e.g., high awareness near flagship locations).
  • Where we stumbled (e.g., POS redemption confusion in 40% of stores; credit latency messaging).
  • Joint action plan: Staff refresher training, signage refresh, auto-credit integration, co-branded New Year push; clear KPIs and check-ins.
  • Propose performance-based adjustments (reduce their perceived risk) and a quick-win pilot in top-performing geos before scaling.
• Tone: Collaborative, specific, and time-bound; emphasize that the bank assumes co-ownership of issues and is committing resources.

8) Example quantitative wrap-up (illustrative)

• Target: 50k enrollments; actual: 28k (−44%).
• Incremental gym spend (DiD): +$0.50 per eligible customer-month; plan: +$1.50.
• Biggest gap: enroll → first swipe CVR 64% vs. plan 85%, concentrated in stores lacking updated BIN tables.
• Fix forecast: If ops fixes restore CVR to 80% and incentive tweak lifts enrollments +25%, modeled ROI moves from −12% to +8% at current media spend; +15% if we shift 40% budget to Jan/Sept windows.

9) Common pitfalls and guardrails

• Confounding from overlapping campaigns; isolate exposure or exclude periods.
• MCC misclassification leading to undercounted spend; validate with transaction sampling.
• Cannibalization: Existing gym spend merely shifts to our card; measure net incremental category spend, not just partner spend.
• Parallel trends violations; include event-study plots and alternative control groups.
• Overfitting uplift models; validate out-of-sample and monitor stability.

10) What good looks like in the follow-up

• A relaunch plan with powered geo holdouts, daily QA on redemption/crediting, pre-registered success criteria, and a joint QA checklist with the partner.
• A 30/60/90-day readout cadence that transparently tracks incremental outcomes and unit economics, with kill-switch thresholds if incrementality is not achieved.

This approach cleanly separates measurement (counterfactual), diagnosis (which link in the chain broke and why), actions (near-term vs. strategic), and communication (preserving executive and partner trust), which is essential for turning a miss into a durable learning and a path back to ROI-positive growth.