Direct vs regular mutual fund plans: measure the cost gap from NAVs

Data you need

Two AMFI sources, both free:

• The current snapshot: NAVAll.txt, a semicolon-delimited file with Scheme Code, ISINs, Scheme Name, Plan, Option, Net Asset Value and Date. Category and fund-house names appear as header lines between rows.
• History: the NAV History section of AMFI's website (under the Net Asset Value menu) lets you download NAVs for a scheme or fund house over a period. Check the downloaded file's header before you parse it, because we have not confirmed that it matches NAVAll's layout.

Our companion article on parsing NAVAll and comparing category returns covers the header lines and the parser itself.

The pairing problem is the real work

Nothing in the file says "this direct code and that regular code are the same scheme". You have to build the pairs, and the real file makes that harder than it sounds:

• In the copy we downloaded on 27 September 2026, the Plan and Option columns were blank on roughly 5,800 of about 14,400 rows. Plan and option then have to be read from the scheme name.
• Option labels vary: "Growth", "Growth Option", "GROWTH", "IDCW Option" and longer variants.
• Many rows carried old NAV dates, some from 2018, because the file holds each scheme's latest NAV, not one day's NAVs.
• Segregated portfolios appear with names containing "Segregated" and a NAV of zero.

A workable approach: normalise the name by removing plan and option words, then group by fund house plus normalised name. Accept a pair only when the group contains exactly one direct growth and one regular growth code. Send everything else to a manual review list. The pairing table is a deliverable in its own right.

Why a fund analytics team would care

Distributors, advisory platforms and research desks all field the question "how much am I paying for a regular plan?". An estimate taken from realised NAVs, rather than from a factsheet, is an independent check on disclosed costs. It is also a strong interview story, because it shows you can turn an economic mechanism into a measurable quantity and then question your own estimate.

The estimator

If both plans earn the same gross return and differ only in daily cost, then log(direct NAV ÷ regular NAV) rises linearly with time. Its slope per year is the annualised cost gap in log terms. Fit it by least squares on aligned dates.

The code below is on synthetic NAV series. The scheme is invented, and the 0.9% gap was set by us so that we can check the estimator recovers it.

import numpy as np
import pandas as pd

rng = np.random.default_rng(3)
dates = pd.bdate_range("2023-01-02", "2026-09-25")
daily_market = rng.normal(0.0005, 0.009, len(dates))
per_year = len(dates) / ((dates[-1] - dates[0]).days / 365.25)   # business days a year
cost_direct, cost_regular = 0.006 / per_year, 0.015 / per_year   # 0.6% vs 1.5% a year, synthetic
# a small autocorrelated wobble stands in for accrual timing differences between the plans
wobble = np.zeros(len(dates))
for i in range(1, len(dates)):
    wobble[i] = 0.97 * wobble[i - 1] + rng.normal(0, 0.0004)
direct = np.round(50 * np.exp(np.cumsum(daily_market - cost_direct)), 4)
regular = np.round(48 * np.exp(np.cumsum(daily_market - cost_regular) + wobble), 4)
df = pd.DataFrame({"date": dates, "direct": direct, "regular": regular})

# Simulate real-world gaps: drop some regular-plan dates
df.loc[rng.choice(len(df), 25, replace=False), "regular"] = np.nan
aligned = df.dropna().copy()
print("aligned dates:", len(aligned), "of", len(df))

years = (aligned["date"] - aligned["date"].iloc[0]).dt.days / 365.25
log_ratio = np.log(aligned["direct"] / aligned["regular"])
slope, intercept = np.polyfit(years, log_ratio, 1)
print(f"estimated annual gap: {slope:.4%} (set to 0.9000%)")

# Newey-West style caution: residuals are autocorrelated, so bootstrap by blocks
resid = log_ratio - (slope * years + intercept)
blocks = 20
idx = np.array_split(np.arange(len(aligned)), blocks)
boot = []
for _ in range(300):
    pick = np.concatenate([idx[i] for i in rng.integers(0, blocks, blocks)])
    pick.sort()
    boot.append(np.polyfit(years.iloc[pick], log_ratio.iloc[pick], 1)[0])
print("block bootstrap 95% interval:", np.round(np.percentile(boot, [2.5, 97.5]) * 100, 3), "%")

On this synthetic series the estimate lands close to the 0.9% we set, and the interval shows how far it could plausibly wander. Real data adds more noise: dividend-style events on the wrong option, and occasional changes in expense ratios. That is why the block bootstrap matters. It resamples chunks of time so autocorrelated deviations do not make the interval look falsely tight.

Cleaning checklist

1. 1Use growth options only. An IDCW payout drops the NAV and would look like a sudden cost.
2. 2Align on dates where both plans published a NAV. Do not forward-fill one side.
3. 3Remove schemes whose latest NAV is stale. A regular plan with no NAV since 2020 cannot be paired.
4. 4Drop segregated portfolios.
5. 5Start each series at the later of the two plans' first dates, and require at least a year of overlap.
6. 6Record the pairing method for every pair: automatic or manual.

Analysis plan

• Estimate the slope and bootstrap interval for every accepted pair.
• Summarise by SEBI category: median gap and interquartile range.
• Test for a break: fit the slope separately on each year. If the gap changes noticeably, check whether the fund house changed its expense ratio in that period.
• Compare your estimates with the expense ratios the fund house discloses, for a sample of schemes. Report agreement, and explain disagreements rather than hiding them.

Deliverables

In the limitations, say that this is a historical cost estimate. It is not investment advice and does not account for the advice a distributor may provide.

Interview questions

• Why is the slope of the log NAV ratio an estimate of the cost gap?
• What assumption breaks if the two plans' portfolios differ slightly on some days?
• Why a block bootstrap instead of the ordinary least-squares standard error?
• How would you detect a wrong pairing automatically?
• Your estimate disagrees with the disclosed expense ratios by 0.3 points for one scheme. Where do you look first?

Estimation with honest uncertainty is a core thread of our Data Science programme. If you are unsure whether your Python is ready for it, take the skill check.