Reading and Writing Typed CSV and JSON Data

Hello! In the previous lesson, you isolated your project’s Python packages and proved you could recreate that environment. This time, you’ll use the same batch-sales project to handle two common pipeline inputs: CSV and JSON. Both are readable text formats, but they make different promises about types. You’ll read each format, write it back out, and check that identifiers remain identifiers, numbers remain numbers, and missing values retain their intended meaning.

The examples use Python’s built-in csv and json modules, so there is nothing new to install.

CSV: columns have names, but values arrive as text

A CSV file represents rows and columns in plain text. A header can tell you that a column is called quantity, but it does not make its values Python integers. The text 2 must be converted according to the type you expect for that column. By contrast, an order ID such as 0007 must stay a string: converting it to an integer would discard the leading zero.

Watch the opening explanation and the dictionary-based reading and writing demonstration in Corey Schafer’s Python Tutorial: CSV Module - How to Read, Parse, and Write CSV Files. Focus on why a CSV parser is safer than splitting each line at commas, and why field names are clearer than positional indexes.

In your WSL terminal, return to the project and activate its environment if necessary:

cd ~/projects/batch-sales
source .venv/bin/activate
mkdir -p data

Create data/orders.csv in VS Code with these exact contents:

order_id,product,quantity,amount_cents,paid,coupon_code
0007,"Notebook, ruled",2,1599,true,
0008,Café mug,1,250,false,SAVE5

Here, amount_cents is an integer amount, avoiding floating-point rounding for money. The comma inside "Notebook, ruled" is part of the product name, not a column separator. The blank coupon_code means “no coupon” for this particular dataset.

A csv.DictReader uses the header as dictionary keys and handles the quoted product name correctly. Nevertheless, the first row’s quantity, amount_cents, and paid values initially arrive as "2", "1599", and "true"—all strings. Reading a file safely means separating two questions: Where are the field boundaries? The CSV parser answers that. What does each field mean? Your expected column types answer that.

When opening CSV files, use encoding="utf-8" to read the product name consistently and newline="" so the csv module controls newline handling. Use both settings again when writing CSV. A DictWriter needs an explicit list of fieldnames; call writeheader() if the output should include a header row.

JSON: some types are encoded, but not every business meaning is

JSON can encode objects, arrays, strings, numbers, booleans, and null. Python’s json module maps these to dictionaries, lists, strings, numbers, True or False, and None. For example, the JSON value true becomes a Python boolean; the CSV text true does not.

Watch Corey Schafer’s Python Tutorial: Working with JSON Data using the json Module. Pay particular attention to the distinction between operations on a string (loads, dumps) and operations on an open file (load, dump).

JSON has types, but it does not know your intended types. Both "0007" and "2" are JSON strings, even though one is an identifier and the other might be a quantity supplied incorrectly by a source. Nor does JSON automatically turn a date-looking string into a Python date. A reliable pipeline defines what each field should contain instead of assuming that the file format will decide.

For our CSV file, the intended record has this shape after conversion:

{
    "order_id": "0007",
    "product": "Notebook, ruled",
    "quantity": 2,
    "amount_cents": 1599,
    "paid": True,
    "coupon_code": None,
}

Notice two deliberate choices. order_id stays a string despite looking numeric. An empty CSV coupon_code becomes None, which JSON writes as null. That second conversion is reversible only because we have defined an empty coupon field to mean “no coupon.” If an empty string and a missing value had different meanings in your source, you would need a different representation.

Build and check a CSV–JSON round trip

Create a new file named formats.py in the project root. Leave your existing run.py unchanged. This script reads the CSV using an explicit type policy, writes JSON, reads that JSON back, and writes a new CSV.

import csv
import json

fieldnames = [
    "order_id",
    "product",
    "quantity",
    "amount_cents",
    "paid",
    "coupon_code",
]

# Read CSV text and convert fields according to their expected types.
orders = []
with open("data/orders.csv", newline="", encoding="utf-8") as source:
    reader = csv.DictReader(source)
    for row in reader:
        if row["paid"] not in ("true", "false"):
            raise ValueError("paid must be true or false")

        orders.append(
            {
                "order_id": row["order_id"],
                "product": row["product"],
                "quantity": int(row["quantity"]),
                "amount_cents": int(row["amount_cents"]),
                "paid": row["paid"] == "true",
                "coupon_code": row["coupon_code"] or None,
            }
        )

# Write a JSON array of typed records.
with open("data/orders.json", "w", encoding="utf-8") as destination:
    json.dump(orders, destination, indent=2, ensure_ascii=False)

# Read the JSON file into Python objects.
with open("data/orders.json", encoding="utf-8") as source:
    restored = json.load(source)

assert len(restored) == 2
assert restored == orders
assert restored[0]["order_id"] == "0007"
assert type(restored[0]["quantity"]) is int
assert type(restored[0]["amount_cents"]) is int
assert type(restored[0]["paid"]) is bool
assert restored[0]["coupon_code"] is None
assert restored[1]["paid"] is False

# Write CSV using an explicit representation for bool and None.
with open(
    "data/orders_roundtrip.csv", "w", newline="", encoding="utf-8"
) as destination:
    writer = csv.DictWriter(destination, fieldnames=fieldnames)
    writer.writeheader()

    for order in restored:
        writer.writerow(
            {
                "order_id": order["order_id"],
                "product": order["product"],
                "quantity": order["quantity"],
                "amount_cents": order["amount_cents"],
                "paid": "true" if order["paid"] else "false",
                "coupon_code": (
                    order["coupon_code"]
                    if order["coupon_code"] is not None
                    else ""
                ),
            }
        )

# Compare CSV field values after parsing, not the files' punctuation.
with open("data/orders.csv", newline="", encoding="utf-8") as source:
    original_rows = list(csv.DictReader(source))

with open(
    "data/orders_roundtrip.csv", newline="", encoding="utf-8"
) as source:
    roundtrip_rows = list(csv.DictReader(source))

assert roundtrip_rows == original_rows
print("Two orders preserved through CSV, JSON, and CSV.")

Run it from the project root, where the relative data/... paths point to your files:

python formats.py

Open both generated files. In orders.json, check that quantity and amount_cents have no surrounding quotes, paid is true or false, the first coupon_code is null, and "0007" still has its leading zeros. In orders_roundtrip.csv, check that the product containing a comma is quoted as a single field.

The final comparison parses both CSV files before comparing them. CSV writers can make different but equally valid choices about text representation; what matters here is that the field values match. The explicit conversion when writing CSV is equally important: without it, a Python boolean would be written as True or False, rather than the lowercase text our input policy expects.

This is a small fixture, so keeping its records in a list makes the type changes easy to inspect. In the next lesson, you’ll make record transformations reusable with functions and generators; later, you’ll give malformed records more informative handling than this script’s basic conversion errors.

Conclusion

CSV parsing finds fields but leaves their values as text; you supply the column type policy. JSON preserves supported types such as numbers, booleans, and null, but it cannot decide whether a numeric-looking string is an ID or an incorrectly encoded number. Reading and writing reliably therefore means defining representations at both boundaries and checking the result after a round trip.

Your formats.py script now demonstrates that policy with leading-zero IDs, integer amounts, booleans, an optional field, UTF-8 text, and a quoted comma. Next, you’ll move the per-record work out of the script’s loop so the same transformation can be reused without loading every record at once.