Debug a Python KeyError in messy business data

A KeyError means a requested dictionary key was not present. In reporting code, investigate the actual schema and transformation history before replacing the access with get(). A default can hide the error while changing the business result.

Common causes include whitespace in headers, inconsistent capitalization, renamed fields and a join or transformation that changed the record structure.

Inspect structure without dumping customer records

Start with the key name in the traceback and inspect the available keys. Using repr on field names makes surrounding whitespace visible.

row = {"order_id": "R1", " amount_inr ": "19.00"}
try:
    row["amount_inr"]
except KeyError as error:
    assert error.args == ("amount_inr",)
else:
    raise AssertionError("Expected missing field was present")
assert [repr(key) for key in row] == ["'order_id'", "' amount_inr '"]
print("Actual keys:", [repr(key) for key in row])

The field's value exists, but under a different key. Printing the complete customer payload is unnecessary to diagnose this schema problem.

The Python mapping documentation describes dictionary lookup and get behavior. Choosing a fallback is a business decision layered on top of those mechanics.

Distinguish absent, null and empty

These three records are different: a record without amount_inr, one with amount_inr set to None, and one with an empty string. get("amount_inr") returns None for the first two unless you supply a distinct sentinel.

missing = object()
records = [{}, {"amount_inr": None}, {"amount_inr": ""}, {"amount_inr": "0.00"}]
states = []
for record in records:
    value = record.get("amount_inr", missing)
    if value is missing:
        states.append("absent")
    elif value is None:
        states.append("null")
    elif value == "":
        states.append("empty")
    else:
        states.append("supplied")
assert states == ["absent", "null", "empty", "supplied"]
print(states)

Zero is a supplied amount, not evidence that data is missing. An expression that replaces every false-like value with a default may collapse meaningful distinctions.

Normalize only under an explicit mapping

If your source contract permits trimming header whitespace, normalize once at ingestion and record the mapping. First check whether normalization would create duplicate names: amount_inr and a padded version could both collapse to the same field.

def normalized_headers(headers):
    result = [name.strip() for name in headers]
    if len(set(result)) != len(result):
        raise ValueError("header_collision_after_normalization")
    return result

assert normalized_headers(["order_id", " amount_inr "]) == ["order_id", "amount_inr"]
try:
    normalized_headers(["amount_inr", " amount_inr "])
except ValueError as error:
    assert str(error) == "header_collision_after_normalization"
else:
    raise AssertionError("Ambiguous headers accepted")
print("Header collision rejected")

The Python reporting lab takes a stricter approach: it requires the exact expected header sequence. Both policies can be reasonable, but an accidental mixture is difficult to audit.

Trace where the field disappeared

If the raw schema is correct, inspect the output columns after each transformation. A projection may have omitted the amount, a merge may have added suffixes, or a nested object may require access at another level.

Add schema checks at meaningful boundaries rather than scattering fallback values throughout calculations. A failed boundary check locates the issue near its cause; a final zero total merely reports the consequence.

Do not turn a missing measure into zero

Using row.get("amount_inr", "0.00") prevents KeyError but claims that an unavailable amount equals zero. That changes the dataset. Reject the record, stop the report or apply an approved missing-value rule with a visible flag.

Exercise: create a source copy with a padded header and another with two headers that normalize to the same name. Implement a documented acceptance policy and verify that ambiguous inputs fail rather than selecting a value silently.

NeuraPath's Data Analytics with Generative AI course connects Python debugging with schema discipline. Fixing the exception is useful only when the resulting number still means what the report promises.

Continue learning

This article is part of the Python foundations for analysts sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Test a business calculation with normal and boundary cases.
• Continue with Python generators for processing a large file incrementally.