Present conflicting evidence without hiding the inconvenient segment

When aggregate and segment results tell different stories, show both and explain the population behind each. Do not select the view that best supports a preferred recommendation. An aggregate change can reflect a changing mix even when every segment's rate remains stable.

The example below uses original synthetic counts. It demonstrates reconciliation and communication, not a causal explanation for why the customer mix changed.

Put the apparent conflict on the page

In Period A, 800 returning users convert at 20% and 200 new users at 5%. In Period B, there are 200 returning users and 800 new users, with the same respective rates.

The overall conversion rate falls from 17% to 8%. The segment rates do not fall. Both statements are true because the share of users in each segment changes.

This table preserves the denominators. Showing only segment percentages would hide the composition change that explains the aggregate difference arithmetically.

Reconcile the two views

from fractions import Fraction

before = {'returning':(800,160),'new':(200,10)}
after = {'returning':(200,40),'new':(800,40)}
def rate(rows):
    return Fraction(sum(v[1] for v in rows.values()),sum(v[0] for v in rows.values()))
assert rate(before)==Fraction(17,100)
assert rate(after)==Fraction(8,100)
for segment in before:
    assert Fraction(before[segment][1],before[segment][0])==Fraction(after[segment][1],after[segment][0])
base_weights = {k:Fraction(v[0],1000) for k,v in before.items()}
standardized_after = sum(base_weights[k]*Fraction(after[k][1],after[k][0]) for k in after)
assert standardized_after==Fraction(17,100)
print({'observed_rates_percent':[17,8],
       'period_B_at_period_A_mix_percent':float(standardized_after)*100,
       'within_segment_rates_changed':False})

The standardized 17% uses Period A's mix as a reference. It helps isolate the arithmetic role of composition under this calculation. It does not replace the actual Period B rate of 8% or prove what a business intervention would have caused.

Write a summary that preserves both facts

A defensible summary is: “Overall conversion fell from 17% to 8%, while the two segment rates remained unchanged. The supplied counts show a larger share of new users, whose conversion rate is lower in both periods. At the earlier segment mix, the later rates would aggregate to 17%.”

This wording explains the reconciliation without claiming that marketing caused the mix shift or that the product experience is unchanged in every relevant respect.

The product analytics lab provides related exercises in denominators and population definitions. The counts in this article are an explicit teaching illustration.

Keep inconvenient evidence visible

If your recommendation concerns acquisition quality, show the new-user segment and its size. If it concerns retention, show the returning segment as well. Do not hide a segment because it weakens the headline.

Choose segments for a defensible analytical reason and document them. Searching many cuts after seeing a disappointing result can produce a favorable story by chance. Exploratory findings should be labeled as such and checked with suitable additional evidence.

Small segments may be unstable or incomplete. Report their counts and avoid treating a noisy difference as a settled pattern. Missing or unclassified users should remain visible rather than disappearing from the analysis.

Connect the evidence to a proportionate next step

The example supports investigating why the mix changed and whether that change was expected. It does not by itself justify stopping a campaign or declaring the overall decline harmless.

Consider the business objective: a larger new-user population might have different future value, acquisition cost or maturity. Those questions need additional data and a defined observation horizon.

Exercise: change the new-user conversion rate in Period B to 4%. Recalculate the aggregate and standardized rate, then write a summary that distinguishes composition change from the within-segment decline.

NeuraPath's Data Analytics with Generative AI course connects segmentation with clear communication. Conflicting views become useful when their populations and relationships are explained instead of selectively displayed.

Continue learning

This article is part of the Metrics, visualization and decision communication sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Write an executive summary that preserves caveats.
• Continue with Design a KPI tree from a business objective.