Writing Pytest Unit Tests for Transformation and Validation Functions

Hello! Your pipeline now distinguishes accepted records, quarantined rows, and failed batches in its logs. The next question is whether a code change can silently alter those decisions. A unit test gives a small function a known input and checks the behavior you expect, without running the whole CSV pipeline.

This lesson closes the Python foundations module. You’ll test validate_order() and paid_sale() with pytest, covering typed output, rejected input, and the transformation’s expected result. Plan on about 40 minutes, including time to inspect your existing functions: the previous lesson shows how they are called, but not every field or calculation inside them.

Put a test around one decision

A batch run checks many things at once: file paths, CSV parsing, validation, transformation, and writing. If its output changes, finding the cause can take time. A unit test narrows the question. For example: given a CSV row whose quantity is "2", does validation return the integer 2?

Watch the introductory parts of getting started with pytest by anthonywritescode. Focus on test functions, ordinary assert statements, and the failure report pytest produces—not on adopting the video’s sample project layout.

Use your existing Linux-based project environment. Install pytest inside that environment, not into an unrelated system Python. For example, a pinned development dependency is:

python -m pip install "pytest==8.4.2"

Record pytest==8.4.2 in your project’s test/development dependency file so the test environment can be recreated; keep any existing runtime dependency pins. Running python -m pytest will use the pytest installed for the same python interpreter.

Create a tests directory and a file named tests/test_pipeline_units.py. Pytest discovers files beginning with test_ and functions beginning with test_. Run commands below from the project root, where transforms.py and validated_orders.py are available to import.

Before writing expectations, briefly inspect those two modules and one accepted input row from your earlier fixture. Confirm the CSV column names, accepted spelling of the paid flag, and the exact fields produced by paid_sale(). A test’s expected value must come from the intended contract and a hand-worked example—not by calling the function once and copying whatever it returned. Otherwise, a bug can become the “expected” behavior.

Test successful validation at the type boundary

CSV supplies strings. Your validator is the boundary that turns an acceptable raw row into the typed order used by the rest of the pipeline. That makes both value and type important: the string "2" and integer 2 can look similar in output but behave differently in calculations.

Start tests/test_pipeline_units.py with the following. The sample row is illustrative; adjust it to match a row your project’s validator is supposed to accept. In particular, check its header fields and paid-flag convention before running the tests.

import pytest

from transforms import paid_sale
from validated_orders import RowValidationError, validate_order


VALID_CSV_ROW = {
    "order_id": "O-101",
    "product": "Notebook",
    "quantity": "2",
    "amount_cents": "750",
    "paid": "true",
    "coupon_code": "",
}


def test_validate_order_converts_csv_values():
    order = validate_order(VALID_CSV_ROW.copy())

    assert order["quantity"] == 2
    assert type(order["quantity"]) is int
    assert order["amount_cents"] == 750
    assert type(order["amount_cents"]) is int
    assert order["paid"] is True


def test_validate_order_preserves_unpaid_status():
    raw = {**VALID_CSV_ROW, "paid": "false"}

    assert validate_order(raw)["paid"] is False

The explicit type checks matter here because True == 1 in Python. order["paid"] is True checks that the result is actually the intended Boolean. If your established contract uses a different money representation, change the money assertion to that contract rather than forcing an incorrect integer expectation.

Each test supplies a fresh dictionary. That keeps one test’s input independent of another test’s changes. These tests do not open a CSV file or call main(); they address the validation function directly.

Test rejection without disguising a programming error

A malformed row should produce the specific RowValidationError your pipeline knows how to quarantine. A test that merely expects some exception could pass because of an unrelated KeyError or programming mistake.

Watch how pytest: testing exceptions by anthonywritescode replaces a manual try/except with pytest.raises, and why the expected exception should be specific.

For a short written reference, the pytest documentation also shows how to inspect an exception and how to match its message.

Add these tests below the successful cases:

@pytest.mark.parametrize(
    "field,bad_value",
    [
        ("quantity", "two"),
        ("amount_cents", "not-a-number"),
        ("paid", "perhaps"),
    ],
)
def test_validate_order_rejects_invalid_values(field, bad_value):
    raw = {**VALID_CSV_ROW, field: bad_value}

    with pytest.raises(RowValidationError) as error:
        validate_order(raw)

    assert str(error.value).strip()


def test_validate_order_rejects_missing_required_field():
    raw = VALID_CSV_ROW.copy()
    del raw["quantity"]

    with pytest.raises(RowValidationError):
        validate_order(raw)

Parameterization gives each malformed value a separate test result while keeping the assertion readable. The nonempty-message check protects a useful feature of your quarantine output: a rejected record needs a reason. If your validator’s error wording is an intentional, stable part of its contract, you can make an individual test stronger with match="quantity" or another appropriate pattern.

Do not automatically add every imaginable bad value. Choose cases that correspond to distinct rules in your actual validator. In particular, whether zero quantity is invalid depends on the contract you wrote earlier; the test should document that decision, not guess it.

Pin down the transformation’s result

The validation tests protect the input boundary. A transformation test protects what happens to a valid, typed order. For paid_sale(), the strongest ordinary check is an explicit expected output for one hand-worked paid order.

Add this test, but complete expected_sale from your existing transformation contract before running the suite:

def test_paid_sale_matches_expected_output():
    order = validate_order(VALID_CSV_ROW.copy())

    expected_sale = {
        # Replace this comment with EVERY expected output field and
        # its literal value for O-101, as defined by transforms.py.
        # If the contract computes an amount, calculate it by hand.
    }

    assert paid_sale(order) == expected_sale

The empty dictionary is an unfinished template, not the expected result. For example, if—and only if—your function’s contract defines a total as quantity multiplied by a unit amount, work out the value for this order yourself and put it under the actual output key. If amount_cents already represents the whole order amount, multiplying it would be a wrong test. Comparing the complete dictionary also catches missing or unexpected output fields.

This test should call paid_sale() on the typed result of validation, because that is how the pipeline calls it. The decision to skip unpaid orders happens in the pipeline’s if order["paid"] branch; do not assume paid_sale() itself rejects unpaid orders unless its own contract says so.

For exact integer-cent amounts, use ordinary equality. pytest.approx() is useful for genuine floating-point calculations, but allowing a tolerance around integer cents would weaken a money test rather than improve it.

Run the suite and interpret what it tells you

After completing the expected dictionary and aligning the sample row with your existing contract, run:

python -m pytest -q tests/

Use -v instead of -q when you want to see each case, including the separate parameterized cases:

python -m pytest -v tests/

If collection fails with an import error, first confirm that you are in the project root, using the intended Python environment, and that the module filenames match your imports. If a test runs but fails, read the assertion report before changing anything. Check, in order, whether the sample input reflects the established contract, whether your hand-worked expectation is correct, and whether the implementation violates it. Do not change an expected value merely to make a red result green.

These are unit tests, not proof that file reading, JSON-lines writing, logging, and exit statuses work together. Their advantage is speed and precision: when a future edit changes conversion, rejection, or sale calculation, they point to the smaller behavior that changed.

Conclusion

You now have a repeatable way to protect the two functions at the center of your local pipeline. Test valid input for its values and types, malformed input for a specific validation error, and transformations against independently worked-out output. Keep examples small enough that a failed assertion explains a rule, not an entire batch run.

This completes the Python foundations module. Next, you’ll move into MySQL querying, beginning with the logical order in which a SELECT query is processed—a useful basis for understanding why a query returns the rows it does.