Articles
Ads Data Scientist Interview Case: Explain Higher CTR with Lower Advertiser Value
Practice an ads data-science case where CTR rises but advertiser value falls, using a reconciled funnel, attribution checks, and testable hypotheses.
A campaign’s click-through rate rises, yet the advertiser says the campaign is worth less. That is a good data-science interview case because the headline can be true and commercially disappointing at the same time. CTR counts a transition from impression to click; the advertiser cares about valuable incremental outcomes after that click and the cost of getting them. Your answer should reconcile the funnel, test whether the measurement is sound, and separate plausible causes from proven ones.
Official metric definitions: Google Ads defines CTR as clicks divided by impressions and describes conversion value per cost as recorded conversion value divided by ad-interaction cost. Candidate reports: this is an original, platform-neutral case exercise, not a reported question from a named employer. Inference: the investigation sequence is an editorial recommendation; it does not imply that every advertiser optimizes the same value measure.
Define “advertiser value” before doing math
Ask which advertiser objective is at issue: purchase revenue, gross profit, qualified leads, subscription lifetime value, store visits, or incremental sales. A higher CTR may be desirable for awareness but irrelevant or harmful if the contract is measured on profitable acquisitions. Even “conversion” can mean checkout, lead form, app install, or a micro-action that has little economic value. The interviewer should hear you distinguish a platform optimization target from the business outcome.
Then align the reporting scope. Are the before and after numbers from the same campaign, network, geography, device mix, placement, creative, attribution window, and period length? Were impressions counted under the same eligibility rules? Did a tracking change alter which clicks or conversions are observable? Does the conversion-value field represent transaction revenue, expected lead value, or a constant $1 placeholder? Google’s conversion-value guidance makes clear that assigned values are needed to analyze business impact; a nominal placeholder does not measure actual profit.
The funnel identity is simple: impressions × CTR = clicks. Clicks × conversion rate per eligible click ≈ recorded conversions, subject to the product’s attribution and interaction definitions. Conversions × average recorded value = recorded conversion value. Recorded value divided by ad cost gives a return-on-ad-spend-like ratio. Each transition can change independently. Increasing the first rate does not guarantee increasing the last result.
Do not immediately label more clicks “low quality.” The creative could be reaching a better audience while the landing page broke; purchase tracking might be missing; prices or stock could have changed; an attribution delay might make the most recent cohort appear weak. State multiple hypotheses, then request the data that distinguishes them.
Work a numerical funnel that reconciles
Original practice case: compare equal windows with one million eligible impressions each. In Period A, there are 20,000 clicks, 1,000 recorded purchases worth $100 each, and $50,000 in ad cost. In Period B, there are 30,000 clicks, 900 recorded purchases worth $80 each, and $60,000 in ad cost. Treat every value as a simplified gross recorded purchase value, not incremental profit.
| Measure | Period A | Period B | Calculation |
|---|---|---|---|
| Impressions | 1,000,000 | 1,000,000 | Equal reach |
| Clicks | 20,000 | 30,000 | Up 50% |
| CTR | 2.0% | 3.0% | Clicks / impressions |
| Purchases | 1,000 | 900 | Down 10% |
| Purchase rate per click | 5.0% | 3.0% | Purchases / clicks |
| Average recorded purchase value | $100 | $80 | Value / purchases |
| Recorded purchase value | $100,000 | $72,000 | Purchases × value |
| Ad cost | $50,000 | $60,000 | Spend |
| Recorded value / cost | 2.0 | 1.2 | Value / cost |
Every number can be true. Period B wins on attention but loses on recorded purchases, average value, total value, and value per dollar spent. Recorded value per click falls from $5.00 to $2.40. Cost per purchase rises from $50 to about $66.67. Those calculations identify where the funnel deteriorated. They do not prove why the purchase rate or order value changed.
The actual advertiser outcome might be worse or better than this table suggests. If the cheaper orders carry better margin or repeat-purchase behavior, gross recorded purchase value could understate lifetime value. If many purchases would have happened anyway without ads, recorded value could overstate incrementality. If Period B has more returns or fraud, gross value overstates realized value. State the distinction instead of declaring 1.2 “the true ROAS.”
Notice one hidden assumption in the table: purchases are attributed to clicks inside the same window. Real reporting can delay a conversion and credit an earlier interaction. A right-censored recent period will have fewer observed purchases even if its eventual conversion rate matches A. Compare cohorts with enough time to mature or use a documented lag adjustment; never quietly compare a fully matured old month with an incomplete new week.
Test measurement before changing the model
Start with counting integrity. Deduplicate click IDs as required by the platform, ensure impressions and clicks share the same eligible population, and verify conversion tags after a recent site or app release. Check whether consent, browser restrictions, server-side events, offline uploads, or attribution settings changed between periods. A tracking regression could create the appearance of lower advertiser value while customers still buy.
Then reconcile the value field. Google Ads’ conversion tracking help distinguishes conversions, conversion value, cost per conversion, and value per conversion. If a product feed accidentally sends a default value for some transactions, the average recorded value can fall without any basket change. Compare ad-platform conversions with a first-party order ledger using a stated attribution mapping, and quantify unmatched orders. Do not expect perfect equality when windows and identifiers differ.
Watch selection effects. A campaign may broaden into mobile placements with cheap clicks and lower immediate purchase rates. The overall CTR rises because impression mix changes; a within-placement comparison might show no creative improvement. Decompose by network, placement, device, geography, query intent, audience, new versus returning customers, and creative. Use fixed-mix or standardization calculations if you want to separate within-segment performance from composition. Do not report the same blended trend twice and call it diagnosis.
Check the landing experience: page speed, broken checkout, out-of-stock products, price changes, shipping cost, and message match between ad and landing page. A compelling but misleading creative may attract curious clicks whose intent is poor; a good creative may also expose an unrelated checkout defect. Segmenting the funnel from click to qualified visit, product view, cart, checkout, and purchase gives a location for the loss.
Finally, inspect spend mechanics. More clicks can raise cost even at a similar or lower CPC. The example has click volume up 50% and spend up 20%; average CPC falls from $2.50 to $2.00, yet cost per purchase rises. Calling the campaign “more efficient because CPC fell” would ignore the value chain. A bidding strategy optimized for clicks may rationally buy cheap attention that does not meet the advertiser’s purchase-value target.
Which hypotheses can an experiment test?
After data QA, formulate a narrow causal question. For creative: does the new high-CTR ad increase incremental qualified orders or merely click propensity? Randomize creative among comparable eligible impressions, hold targeting and landing experience stable, and measure a downstream primary outcome with enough time for conversions. Include CTR as a diagnostic metric, not the only success condition. The unit of randomization and interference rules matter if users see multiple ads.
For landing page: run a page experiment among clicked users, but recognize it estimates the page effect conditional on a click, not the causal effect of the entire campaign on the reachable population. For bid policy: a holdout or geo experiment may be needed to estimate incremental advertiser outcomes when attribution itself is biased. If no experiment is feasible, use matched cohorts and sensitivity checks, with clear limits. “Incremental” is a stronger word than “attributed.”
Think about guardrails. A short-term conversion-value gain could come from excessive discounts, low margins, returns, or poor customer experience. Define acceptable cost per qualified acquisition, contribution margin, retained value, and user harm. Google’s guidance on measuring different values per conversion supports transaction-specific values where purchases differ, but your own margin and lifetime-value mapping must still be validated.
| Hypothesis | Fast check | What would change the decision |
|---|---|---|
| Broader low-intent traffic | Channel and query mix, qualified-visit rate | B worsens within the same intent groups |
| Creative overpromises | Ad-to-landing message match, bounce and funnel steps | Better message keeps value while CTR normalizes |
| Checkout or inventory issue | Errors, speed, stock, abandonment | Fix restores CVR without changing targeting |
| Value instrumentation broke | Platform-to-order reconciliation | Ledger value remains stable |
| Conversion delay | Matured click cohorts | Eventual value closes the gap |
This matrix is valuable because each row predicts a different next action. A targeting change is not the right response to a broken checkout. A conversion-tag fix is not evidence that the new creative is better. The analyst should say what observation would falsify each explanation.
How do you make a recommendation?
In the numerical case, you can recommend not scaling Period B on CTR alone. Keep the campaign stable long enough to finish measurement checks, then decide whether to revert the creative, repair the landing flow, tighten targeting, change value signals, or run a controlled test. The exact choice is conditional on the segment and ledger evidence. You do not need to invent a threshold at which CTR becomes bad.
Ask who receives the recommendation. A bidding team can change targeting and value signals, while a merchant may need to fix stock, checkout, or order-value tracking. Assign the next check to the owner of that part of the funnel and specify a decision date after delayed conversions have matured. Otherwise the diagnosis may be accurate but impossible to act on.
A two-minute answer sounds like this: “CTR increased from 2% to 3%, but click-to-purchase rate fell from 5% to 3%, average recorded order value fell from $100 to $80, and recorded value per cost fell from 2.0 to 1.2. I would first validate attribution maturity and value tracking, then decompose by channel, audience, device, creative, and landing funnel. If the decline persists within comparable segments, I would test the most likely mechanism with a downstream value metric and cost guardrails.”
If the interviewer asks whether high CTR is “good,” say it measures appeal or relevance at the click stage, as Google’s CTR definition describes, but it is not sufficient evidence of advertiser value. Practice the reverse case with Decide when CTR falls but revenue rises: the same funnel reasoning should work when the arrows reverse. A defensible answer traces the money, not just the clicks.