Data freshness checks: stop a stale dashboard release

A dashboard can be generated minutes ago from a source that stopped updating days earlier. Freshness checks should evaluate the data's reporting contract, not just the modification time of the output file.

Define the expected period, source coverage and maximum acceptable lag. Then make failure block the release path rather than merely adding a small warning to a dashboard people may not notice.

Separate three checks

First, confirm that the source watermark reaches the required report end. Second, confirm that extraction occurred within the allowed preparation window. Third, decide whether that report period is still acceptable at the intended release time.

These checks answer different questions. A complete January report is valid historical evidence but is not automatically suitable for a dashboard labeled “current” in September.

The original automation lab implements the first two checks during source loading and provides a separate explicit-clock freshness function for the third. No live dashboard release is performed by this exercise.

Test the release-time boundary

Suppose a teaching release contract allows a report ending at January 12, 2026 midnight to be used within one hour of that end. The boundary is inclusive: exactly one hour is acceptable, anything later fails.

from pipeline import check_freshness,ReportError

end = '2026-01-12T00:00:00Z'
assert check_freshness(end,'2026-01-12T00:59:59Z',3600)==3599
assert check_freshness(end,'2026-01-12T01:00:00Z',3600)==3600
for current in ('2026-01-12T01:00:01Z','2026-01-13T00:00:00Z',
                '2026-01-11T23:59:59Z'):
    try:
        check_freshness(end,current,3600)
    except ReportError as error:
        assert str(error)=='report_outside_freshness_window'
    else:
        raise AssertionError('invalid release time accepted')
print({'inclusive_boundary_verified':True,'late_and_future_periods_rejected':True})

The last case rejects a report whose period has not yet ended. The clock is passed explicitly so tests remain deterministic. In a deployed release service, obtain that clock from the controlled runtime rather than accepting an arbitrary caller-supplied timestamp.

Choose a threshold that matches the cadence

The one-hour rule above is an illustrative immediate-release policy. A weekly dashboard intended to remain valid until the next weekly report needs a different expectation, usually tied to the latest scheduled period that should be available by the current time.

Do not copy a threshold without considering weekends, holidays, source delivery schedules and business timezone. A nightly batch source should not be judged by the same expectation as a near-real-time event stream.

Store the threshold and calendar policy with the release configuration. Changing them after a failure should require an explicit reason, not become an invisible way to turn a red check green.

Avoid using maximum event time as the only signal

One recent event can make MAX(event_time) look fresh while most partitions remain stale. Conversely, a quiet source can have no recent events even though extraction is healthy and correctly reports zero activity.

Use source heartbeats or watermarks whose semantics distinguish no activity from no delivery. For partitioned sources, evaluate the required partitions individually and report coverage. A global timestamp should not conceal a missing region or date partition.

The incomplete-source tutorial explains why row counts and checksums complement these time checks without proving completeness on their own.

Preserve a usable failure state

When freshness fails, retain the previous verified artifact with its actual period label if the product's policy permits historical viewing. Do not relabel it as current. Record the failed attempt, expected period, observed watermark and next recovery action.

A blocked release is useful only if someone can identify whether the source is delayed, the scheduler failed or the period configuration is wrong. The failure runbook should distinguish those cases and define when a corrected version can be reviewed.

Exercise: create two required source partitions, one current and one a day behind. Show why the maximum timestamp across both passes a naive check, then implement an all-required-partitions gate with explicit missing-partition handling.

NeuraPath's Data Analytics with Generative AI course connects data validation with reliable reporting. A meaningful freshness check asks whether the evidence is suitable for the intended decision at the intended time.

Continue learning

This article is part of the Reliable reporting automation sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Create a report manifest with timestamps and row counts.
• Continue with Retry a failed data extract without duplicating records.