Compute sample size and test duration

You will run a two-arm A/B test on a signup funnel. Given: baseline conversion p0 = 4.0%; you care about detecting a 10% relative uplift (p1 = 4.4%); two-sided α = 0.05; power = 80%; Bernoulli outcomes; independent users; simple difference-in-proportions z-test approximation is acceptable.

• Compute the minimum per-variant sample size. Show the exact formula you use and the numeric result.
• Now assume you have 2 primary metrics (conversion and qualified-lead rate). Apply a Bonferroni correction and recompute the per-variant sample size. Explain trade-offs versus using a gatekeeper metric or a hierarchical test.
• Traffic: 1.2M eligible sessions/day, but only 70% meet eligibility checks; you can allocate at most 20% of eligible traffic to the experiment initially. Estimate test duration in days, then propose a safe ramp schedule that preserves statistical validity and operational risk controls.
• Explain when you would use a z-test vs. a t-test here, and interpret a 95% CI that barely excludes zero uplift.
• Describe how you would verify proper randomization and balance across country and platform before looking at outcomes.