Python type conversion: reject invalid numeric records explicitly

Numeric conversion should follow a source contract. Decide which formats, units and special values are permitted, then reject or quarantine records that do not satisfy it. Turning every conversion failure into zero changes unknown or invalid data into an invented measurement.

A parser accepting a broad range of Python numeric strings is not necessarily enforcing the business field's format. Monetary inputs may need stricter rules than float() or Decimal() alone provides.

Define a canonical amount format

The Python reporting lab accepts nonnegative INR amounts written with exactly two decimal places, such as 19.00, 0.00 and 1200.00. It rejects commas, exponent notation, signs and missing values for this particular source.

This is a deliberately narrow teaching contract, not a claim that other formats are inherently invalid. A different source may legitimately use comma decimals, thousands separators or credit amounts, but its parser must define those rules explicitly.

Run this from the lab directory:

from decimal import Decimal
from report import parse_amount

assert parse_amount("19.00") == Decimal("19.00")
assert parse_amount("0.00") == Decimal("0.00")
for text in ["N/A", "1,200.00", "NaN", "Infinity", "1e3", "12", "-1.00"]:
    try:
        parse_amount(text)
    except ValueError as error:
        assert str(error) == "invalid_amount_format"
    else:
        raise AssertionError(f"Unexpected acceptance: {text}")
print("Canonical amount checks passed")

The parser validates the textual grammar before constructing Decimal. That prevents special nonfinite numeric values from entering this ordinary amount field.

Preserve the reason for rejection

The raw fixture includes R8 with N/A and R10 with 1,200.00. Both fail the canonical amount rule, but the original source remains unchanged. rejected_records.csv records their input record number, order ID and reason.

The report does not silently strip punctuation from R10. Removing commas would happen to yield a plausible amount under one locale, but the same general habit can corrupt a source using commas differently.

If the business approves a locale-specific parser, implement and test that parser as a named transformation. Preserve the raw text beside the parsed value or in the source snapshot.

Separate type validity from business validity

The text 999999999.00 is numerically valid under the simple format rule. Whether it is a reasonable order amount requires another rule based on the source and business context.

Likewise, zero can be valid for a fully discounted order or invalid for a field that requires positive value. Parsing establishes representation; validation establishes the allowed business domain.

Avoid rejecting every unusual high value as an outlier without evidence. A legitimate large transaction and a typing error can look similar until investigated.

Keep exact decimal meaning where required

Construct Decimal from the original decimal string rather than first converting through a binary float. The intermediate float can introduce a representation different from the source text.

Python's decimal documentation describes decimal arithmetic and special values. Precision and rounding still require an explicit policy when calculations create more decimal places than the source amount.

Do not assume Decimal alone validates currency, sign, maximum amount or number of fractional digits. Those are separate input rules.

Report accepted and rejected populations together

The complete fixture has eleven input records: seven accepted unique orders, three rejects and one identical replay. January's accepted Paid value is INR 47.50, but the overall run is labelled partial because records were rejected.

A scheduler or reviewer must decide whether that partial result is usable for the intended report. A process that simply continues after errors without a rejection count provides no basis for that decision.

The lab's fifteen tests verify parsing, boundary handling and the partial-result exit status. The examples here are also executable through the programme's Python checks.

Exercise: define a separate parser for a source that permits signed credit amounts with exactly two decimals. Test valid negative credits, malformed signs and nonfinite values, then explain why the original ordinary-sale parser should retain its narrower contract.

NeuraPath's Data Analytics with Generative AI course connects Python conversion with data-quality decisions. A trustworthy numeric pipeline preserves invalid evidence instead of replacing it with plausible numbers.

Continue learning

This article is part of the Python foundations for analysts sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Write a Python function with a clear input contract.
• Continue with Read CSV files without corrupting customer identifiers.