Popularity bias in a recommendation system

Popularity is a strong and transparent recommendation baseline. Recommending the same head items repeatedly can narrow catalogue exposure, make new items harder to learn and reinforce the historical policy that produced the interaction data.

Compare exposure summaries beside relevance

Our synthetic training matrix has 60 users, 20 catalogue items and 240 warm-user interactions. The four most-interacted training items define the head set: I0, I3, I6 and I10. Seen items are removed before producing five recommendations per user.

The popularity baseline exposes nine unique items, 45% of the catalogue. Head items make up 50.67% of its 300 recommendation slots. The SVD baseline exposes 17 unique items, 85% coverage, with a 23.33% head share.

from unsupervised_cases import popularity_case

r = popularity_case()
assert r['head_items'] == ['I0', 'I10', 'I3', 'I6']
assert r['popularity']['unique_items'] == 9
assert r['svd']['unique_items'] == 17
assert r['popularity']['head_share'] > r['svd']['head_share']
print(r)

Reproduce these rankings in the unsupervised lab. Coverage and head share describe exposure concentration; they do not prove the SVD list is more relevant or beneficial.

Diagnose the feedback loop

Popular items receive more exposure, which creates more interactions, which can increase future rank. New or niche items receive less evidence even when they would satisfy a user. Historical logs therefore mix preference with prior recommendation policy, position and availability.

Measure item exposure by popularity decile, supplier or content group where appropriate. Add long-tail coverage, Gini or concentration measures and per-user novelty. Keep relevance metrics and quality guardrails beside them; maximizing catalogue coverage alone can recommend random items.

Mitigations include calibrated exploration, re-ranking constraints, freshness boosts and hybrid content signals. Each can reduce short-term clicks or introduce new failure modes. Validate offline under temporal splits and test online with user and ecosystem guardrails.

Cold-start exposure is a policy choice. If item I19 has no interactions, collaborative filtering cannot learn it without exploration or metadata. Log propensities or exposure details when causal evaluation will matter.

Exercise: create a re-ranker allowing at most two head items in the top five. Compare hit rate, NDCG, coverage and head share on validation. Define an online experiment that measures discovery without sacrificing user safety or content quality.

NeuraPath's Data Science course connects recommender accuracy to exposure distribution. Popularity is both a baseline and a process that shapes future data.

Continue learning

This article is part of the Clustering, reduction and recommendations sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Ranking metrics: compare precision at k and NDCG.
• Continue with Implicit feedback: missing interaction is not dislike.