Reading and Writing Repository Files with pathlib

Hello! Last lesson, you created an isolated Python environment and a pinned dependency file. Now you can start using that project to inspect repository files. This lesson covers paths, source text, JSON, and two common configuration formats. You’ll build a small file-inspection script without touching an unfamiliar repository; allow about 40 minutes, including the videos and coding.

Paths first, contents second

A Path object represents a location. It does not read a file merely because you create it. That separation is useful in repository tooling: you can identify a candidate source file, inspect its name and extension, and only then decide whether to read it.

Watch these two short parts of Net Ninja’s Working with Files in Python #5 – Pathlib to Read & Write Files:

Create a disposable repository-shaped directory inside the project from last lesson. Run these commands from that project’s top-level directory:

mkdir -p demo-repo/src demo-repo/config
printf 'int add(int a, int b) { return a + b; }\n' > demo-repo/src/math.cpp
printf '[analysis]\ninclude_suffix = .cpp\nmax_files = 20\n' > demo-repo/config/triage.ini
printf '{"target": "src/math.cpp", "enabled": true}\n' > demo-repo/config/task.json
printf '[project]\nname = "tiny-demo"\n' > demo-repo/pyproject.toml

The shell commands only set up sample inputs. The Python code will do the reading and subsequent writing.

The official pathlib documentation is a useful reference for both file contents and discovery:

Create inspect_repo.py in your project’s top-level directory. You can add the following blocks to the same file, in order, running it after each addition:

from pathlib import Path

root = Path("demo-repo").resolve(strict=True)
report_dir = Path("reports")
report_dir.mkdir(parents=True, exist_ok=True)

example = root / "src" / "math.cpp"
print(example.name, example.suffix, example.parent.name)

summaries = []
for path in sorted(root.rglob("*.cpp")):
    if not path.is_file():
        continue

    text = path.read_text(encoding="utf-8")
    relative = path.relative_to(root)
    summaries.append(
        f"{relative.as_posix()}: {len(text.splitlines())} lines"
    )

summary_path = report_dir / "source-summary.txt"
summary_path.write_text(
    "\n".join(summaries) + "\n",
    encoding="utf-8",
)
print(summary_path.read_text(encoding="utf-8"), end="")

Here, / joins path components; it does not divide numbers. rglob("*.cpp") discovers matching paths recursively, is_file() excludes a directory that happens to match, and sorted() gives the report a stable order. relative_to(root) makes the reported location useful within the repository rather than printing a machine-specific absolute path.

read_text() returns the entire decoded file as a string. write_text() creates or overwrites a file and closes it for you, so the script writes only into reports/. The explicit UTF-8 encoding avoids depending on a machine’s default text encoding. A repository may contain files in another encoding or files too large to read comfortably at once; a production inspector needs policies for those cases.

JSON is data, not just text

A JSON file is text on disk, but your program usually needs its structure: objects, arrays, strings, numbers, booleans, and null values. Python’s standard-library json module performs that conversion. pathlib still handles where the file is; json handles what its contents mean.

Watch Net Ninja’s Working with Files in Python #8 – Working with JSON Files:

For the exact type conversions, read this small portion of the Python documentation:

Add this block to inspect_repo.py:

import json

task_path = root / "config" / "task.json"
with task_path.open("r", encoding="utf-8") as file:
    task = json.load(file)

if not isinstance(task, dict) or not isinstance(task.get("target"), str):
    raise ValueError("Expected a JSON object with a string 'target'")

print("Requested target:", task["target"])

task["status"] = "queued"
reviewed_path = report_dir / "task-reviewed.json"
with reviewed_path.open("w", encoding="utf-8") as file:
    json.dump(task, file, indent=2, ensure_ascii=False)
    file.write("\n")

print("Saved status:", json.loads(
    reviewed_path.read_text(encoding="utf-8")
)["status"])

json.load(file) reads and parses an open file; json.dump(task, file, ...) serializes a Python value into an open file. The final json.loads(...) shows the related operation for an already-read string. The trailing newline makes the generated file pleasant to view in a terminal.

The type check matters because syntactically valid JSON might be a list, a number, or an object missing target. Parsing establishes that the file is valid JSON; it does not establish that it has the structure your tool requires. Nor should the string in target automatically become a file your tool opens: repository-supplied paths need validation before they control filesystem access.

If you later put a Path in a JSON report, convert it to a string first—for example, relative.as_posix(). A Path itself is not one of the standard JSON value types.

Configuration files need their own parsers

“Configuration file” describes a purpose, not a single syntax. The sample repository has an INI file (triage.ini) and a TOML file (pyproject.toml). Both have sections, but a parser for one should not be used for the other. In each case, Path locates and opens the file; a format-specific module interprets it.

Add the INI portion to your script:

import configparser

ini_path = root / "config" / "triage.ini"
settings = configparser.ConfigParser()
with ini_path.open("r", encoding="utf-8") as file:
    settings.read_file(file)

suffix = settings["analysis"]["include_suffix"]
limit = settings.getint("analysis", "max_files")
print("Analysis settings:", suffix, limit)

settings["analysis"]["max_files"] = "25"
updated_ini = report_dir / "triage-updated.ini"
with updated_ini.open("w", encoding="utf-8") as file:
    settings.write(file)

INI values are strings when accessed directly; getint() performs an explicit integer conversion. settings.write() writes a valid INI representation, but it may change layout or discard comments. That is another reason to write a new file rather than silently rewriting a repository’s hand-maintained configuration.

For TOML, Python 3.11 and newer include tomllib for reading. If you used an older Python interpreter for your environment, skip this final snippet rather than assuming tomllib is available:

import tomllib

toml_path = root / "pyproject.toml"
with toml_path.open("rb") as file:
    project = tomllib.load(file)

print("Project name:", project["project"]["name"])

The binary mode "rb" is required by tomllib.load(). The standard-library tomllib does not provide a corresponding writer. For a tool that must preserve and edit an existing TOML file, choose an appropriate writer deliberately rather than treating TOML as interchangeable with INI or doing an unrestricted text replacement.

What should fail visibly?

Run the completed script with your environment’s Python:

python inspect_repo.py

You should see a source summary, the JSON target and saved status, the INI settings, and the TOML project name. Inspect the three generated files in reports/; the four inputs in demo-repo/ should remain unchanged.

File-handling failures carry useful information. A missing path can raise FileNotFoundError; bytes that cannot be decoded with your chosen text encoding can raise UnicodeDecodeError; malformed JSON raises json.JSONDecodeError. A useful repository tool reports which file failed and why rather than silently treating every failure as “no results.” Keep outputs separate from inputs until you have a deliberate policy for changing repository files.

Takeaways

pathlib gives you a consistent way to locate, discover, read, and write files. Use json, configparser, or tomllib when the contents have a format that plain text methods cannot interpret on their own. In your small inspector, stable discovery order, explicit encodings, basic structure checks, and separate output files already make the result easier to trust.

Next, you’ll extend this foundation from reading repository evidence to running compiler, test, and analysis commands from Python while capturing their results.