Running Compiler, Test, and Static Analysis Commands from Python

Hello! Last lesson, you used pathlib to inspect repository files without changing the inputs. Now you’ll use Python to run tools against those files. The goal is a small runner that compiles C++, executes a test binary, and runs static analysis while retaining each command’s exit code, standard output, and standard error. Allow about 40 minutes, including the video, reading, and implementation.

A command has three useful results

When you run a compiler in a terminal, its diagnostics appear on screen and the shell receives an exit status. From Python, subprocess.run() can give you those results as data. That distinction matters for coding automation: a later tool can report which step failed and include the relevant diagnostics instead of merely saying “the build failed.”

Watch these parts of Corey Schafer’s Python Tutorial: Calling External Commands Using the Subprocess Module:

For the precise behavior of the returned object, consult the Python documentation:

We will use check=False, the default. A compiler error is an expected result to record, not an exception that should stop the entire runner before it can report the other checks. A non-zero code means the tool reported an unsuccessful outcome; the tool’s messages explain why. It does not mean Python failed to start the tool. Conversely, a missing executable can raise FileNotFoundError before there is any tool exit code at all.

Build a three-step repository runner

Keep working in the disposable demo-repo/ from last lesson. Its src/math.cpp should contain:

int add(int a, int b) { return a + b; }

You’ll need a C++ compiler (g++) and the static analyzer cppcheck available on your Linux system. Check with g++ --version and cppcheck --version; if either is missing, install it using your system’s package manager. cppcheck is a separate executable, not a Python package in your virtual environment.

From the project directory, create a small C++ test program at demo-repo/tests/test_math.cpp:

#include <iostream>

int add(int a, int b);

int main() {
    if (add(2, 3) != 5 || add(-2, 1) != -1) {
        std::cerr << "add: FAIL\n";
        return 1;
    }

    std::cout << "add: PASS\n";
    return 0;
}

This test deliberately uses process conventions that your Python runner can observe: it prints a success message and exits with zero when checks pass; otherwise, it writes a failure message and exits with a non-zero code. Real test frameworks provide richer reports, but the subprocess interface is the same.

Create run_checks.py alongside last lesson’s inspect_repo.py:

from pathlib import Path
import subprocess
import sys

root = (Path(__file__).resolve().parent / "demo-repo").resolve(strict=True)
(root / "build").mkdir(exist_ok=True)


def run_step(name: str, argv: list[str]) -> bool:
    print(f"\n[{name}] arguments: {argv!r}")

    try:
        result = subprocess.run(
            argv,
            cwd=root,
            capture_output=True,
            text=True,
            encoding="utf-8",
            errors="replace",
            timeout=30,
            check=False,
        )
    except FileNotFoundError as exc:
        print(f"Could not start {name}: {exc}")
        return False
    except subprocess.TimeoutExpired:
        print(f"{name} exceeded 30 seconds")
        return False

    print(f"exit code: {result.returncode}")
    print("stdout:\n" + (result.stdout.strip() or "(empty)"))
    print("stderr:\n" + (result.stderr.strip() or "(empty)"))
    return result.returncode == 0


compile_ok = run_step(
    "compile",
    [
        "g++", "-std=c++17", "-Wall", "-Wextra", "-Werror",
        "src/math.cpp", "tests/test_math.cpp",
        "-o", "build/test_math",
    ],
)

if compile_ok:
    test_ok = run_step("test", ["./build/test_math"])
else:
    print("\n[test] skipped because compilation did not succeed")
    test_ok = False

analysis_ok = run_step(
    "static analysis",
    [
        "cppcheck", "--enable=warning,style",
        "--error-exitcode=1", "--quiet",
        "src/math.cpp", "tests/test_math.cpp",
    ],
)

print(
    f"\nSummary: compile={compile_ok}, "
    f"test={test_ok}, static analysis={analysis_ok}"
)
sys.exit(0 if compile_ok and test_ok and analysis_ok else 1)

Run it with the Python interpreter from your isolated environment:

python run_checks.py

On the supplied files, you should see add: PASS under the test step and a successful summary. cppcheck may produce no text when it finds no enabled diagnostics; empty output is not itself a failure.

Notice the boundaries between the steps. The compiler reads repository-relative source paths and writes only to demo-repo/build/. The test runs the resulting binary only if that compilation succeeded, so an old binary cannot masquerade as the current build. Static analysis still runs if compilation fails, because its diagnostics may provide independent evidence.

Read the results without losing the failure

The argument list keeps "src/math.cpp" as one argument and each compiler flag as another. cwd=root makes those paths relative to demo-repo/ regardless of where you launched the Python script. It changes the child process’s working directory; it does not restrict what files a program is capable of accessing.

capture_output=True stores standard output and standard error separately. With text=True, they are strings rather than bytes; the explicit encoding and replacement policy keep an unexpected diagnostic byte from crashing this small reporter. Different compilers and analyzers choose different streams for their messages, so record both. --error-exitcode=1 asks cppcheck to return a non-zero code when it reports findings at the enabled severities; other analyzers can have different exit-code rules.

To see a test failure without breaking compilation, temporarily change return a + b; in src/math.cpp to return a - b;, then rerun the script. The compiler should still succeed, while the test reports a non-zero exit code and add: FAIL on standard error. Restore the original line afterward. A syntax error, by contrast, would make compilation fail and cause the runner to skip the test while retaining the compiler diagnostic. These are different cases for a future build-log triage tool.

There are also failures that do not produce a CompletedProcess: FileNotFoundError means the requested program could not be started, and TimeoutExpired means the deadline elapsed. The runner reports those separately from an ordinary non-zero tool exit. A timeout is useful protection for a small local check, although production handling of long-running commands and their descendants needs more care. Capturing everything into memory is similarly suitable for this small example, not for unbounded CI logs.

Finally, an argument list with the default shell=False avoids shell interpretation of filenames and arguments. It is not a sandbox: compiling and especially executing code from an untrusted repository can still be dangerous. Run this pattern on repositories and tools you trust; later lessons will establish stronger boundaries for tool-enabled workflows.

The same interface works for Python tests

For a Python repository, you can replace the C++ test command with an interpreter-driven pytest command, provided pytest is installed in the environment and that repository has Python tests:

[sys.executable, "-m", "pytest", "tests"]

Using sys.executable selects the same Python interpreter running your script. This is a command replacement for a Python repository, not a step to add to the C++ demo, whose tests/ directory currently contains no pytest tests.

Takeaways

subprocess.run() lets Python treat compiler, test, and analyzer runs as inspectable results: an exit code plus captured output from two streams. Use an argument list, set the intended working directory, distinguish a tool’s non-zero exit from a failure to start it, and do not run a stale test binary after compilation fails.

Next, you’ll call an authenticated REST API from Python and parse its JSON response—another source of evidence that your automation can combine with local repository checks.