Measure extraction precision and recall by field

Whole-record accuracy marks a document wrong when one field fails and can hide which extraction component needs work. Micro-averaging can let an easy, frequent field dominate. Report true positives, false positives and false negatives for each field with explicit null handling.

Reconcile the counts

The local lab evaluates order ID, date, amount and email across four invented records.

from nlp_cases import field_metrics_case

result = field_metrics_case()
assert set(result["by_field"]) == {"order_id", "date", "amount", "email"}
for field, metrics in result["by_field"].items():
    print(field, metrics)

Order ID, amount and email each have 3 true positives; date has 4. Every field has zero false positives and false negatives, so precision and recall are 1.0. These perfect results describe four parser-aligned synthetic records. They are a unit test, not a production estimate.

Define matching before scoring

Decide whether values must match exactly after normalization. For spans, report exact boundary and type. For dates and amounts, parse to canonical types before comparison while retaining the raw source. A partially correct ID should normally be wrong if it drives a lookup.

Null cases matter. Predicting a value when none exists is a false positive. Returning null for a present field is a false negative. If a document can contain several values, use set or span matching with a documented assignment policy.

Add decision-weighted views

An incorrect amount may cost more than a missing optional email. Keep per-field metrics and supplement them with a business loss or review policy. Report document types, languages and length slices. Add confidence coverage if low-confidence extraction goes to review.

Use enough examples to estimate rare formats and failure modes. Freeze normalization and matching logic before final test; changing them after seeing errors changes the metric.

The Data Science course connects entity and field extraction to auditable denominators and acceptance gates.

Exercise

Inject one spurious email, one missed amount and one off-by-one entity boundary. Recalculate field metrics and whole-record accuracy. Explain which report gives the clearest repair path.

Continue learning

This article is part of the NLP and text analytics sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Extract structured fields from text and validate each field.
• Continue with Build a review-mining project that produces actionable themes.

Reference: scikit-learn precision, recall and F-score API.