Reading and Writing UTF-8 Text, JSON, and Workflow State Files

Reading and Writing Text Files

Most simple data, like logs, notes, or basic reports, are stored in plain text files. Python makes it easy to work with them, but it's important to do it safely and correctly.

The standard way to work with files is using the with statement combined with the open() function. This approach is highly recommended because it automatically handles closing the file for you, even if errors occur.

In the last lesson, you created Path objects. A great feature is that you can pass these objects directly to the open() function.

from pathlib import Path

# Create a path to a new file in your home directory
my_file_path = Path.home() / "my_first_file.txt"

# Open the file for writing ('w') with UTF-8 encoding
with open(my_file_path, 'w', encoding='utf-8') as f:
    f.write("Hello, automation world!\n")
    f.write("This is the second line.")

# Open the same file for reading ('r')
with open(my_file_path, 'r', encoding='utf-8') as f:
    content = f.read()
    print(content)

Notice two key arguments in open():

• mode: 'w' for write (which creates the file or overwrites it if it exists) and 'r' for read.
• encoding: We explicitly state encoding='utf-8'. UTF-8 is the modern standard for text that supports characters from virtually all languages. Specifying it makes your scripts predictable and reliable.

To get a thorough walkthrough of these fundamentals, the following video by Corey Schafer is an excellent guide.

To reinforce these concepts, the official Python documentation provides a concise summary.

Writing to Files

Now let's look at writing. As mentioned, 'w' mode will overwrite a file. If you want to add to the end of an existing file, you can use append mode, 'a'.

The video you just watched also covers writing. It's a good idea to watch this segment to see it in action.

Reading and Writing Structured Data with JSON

Plain text is great for simple, unstructured data. But for structured data—like a list of contacts, configuration settings, or data from a web service—JSON (JavaScript Object Notation) is the industry standard. It's human-readable and maps very naturally to Python's dictionaries and lists.

Your goal is to "serialize" a Python object (like a dict) into a JSON-formatted string to save it, and "deserialize" a JSON string back into a Python object to use it. Python's built-in json module is all you need.

Writing Python Objects to a JSON File

The process of converting a Python object to JSON is called serialization. We use json.dump() to write an object directly to a file.

import json
from pathlib import Path

# Our structured data in Python
report_data = {
    "report_id": "q4-2023-sales",
    "generated_by": "automation_script_v2",
    "metrics": {
        "total_sales": 150394.50,
        "new_customers": 82
    },
    "regions_active": ["North", "South", "West"]
}

# The file where we'll store it
json_path = Path.home() / "report.json"

# Write the data to the file
with open(json_path, 'w', encoding='utf-8') as f:
    json.dump(report_data, f, indent=4)

print(f"Successfully wrote report data to {json_path}")

After running this, open report.json in a text editor. You'll see your Python dictionary perfectly formatted. The indent=4 argument is optional but highly recommended; it adds indentation to make the file easy for humans to read.

The Real Python article on JSON provides an excellent written explanation of this process.

Corey Schafer's video on JSON also offers a great visual demonstration of writing to a file.

Reading a JSON File into a Python Object

The reverse process, deserialization, uses json.load() to read a file and parse it back into a Python object.

# Continuing from the previous example...
# Let's read the data back from the file we just created.

with open(json_path, 'r', encoding='utf-8') as f:
    loaded_data = json.load(f)

print("--- Data loaded from JSON file ---")
print(f"Report ID: {loaded_data['report_id']}")
print(f"Total Sales: {loaded_data['metrics']['total_sales']}")

# The JSON array becomes a Python list
print(f"Active regions: {loaded_data['regions_active']}")
print(f"Type of 'regions_active': {type(loaded_data['regions_active'])}")

As you can see, the loaded_data variable is a standard Python dictionary, and you can access its contents just like any other dictionary.

Let's refer back to the Real Python article and the Corey Schafer video to see this in practice.

Applying the Concepts: A Simple Workflow-State Record

Now let's tie everything together to solve a real-world automation problem. Many scripts need to remember what they did last time. For instance, a script that processes daily sales files shouldn't re-process files it has already handled. A simple way to manage this is by saving the script's state in a JSON file.

Here, we'll write a script that simulates a task and records when it was last run successfully.

For this, we can use two convenient methods from the pathlib library:

• Path.write_text(content, encoding=...): A shortcut that opens, writes, and closes a file.
• Path.read_text(encoding=...): A shortcut that opens, reads, and closes a file.

These methods handle the with open(...) pattern for you, making the code more concise for simple read/write operations.

import json
from pathlib import Path
from datetime import datetime

# Define the path to our state file
STATE_FILE = Path("workflow_state.json")

# --- 1. Read the last known state ---
last_run_time = None
if STATE_FILE.exists():
    print(f"Reading state from {STATE_FILE}...")
    try:
        state_data_str = STATE_FILE.read_text(encoding="utf-8")
        state_data = json.loads(state_data_str)
        
        last_run_time_str = state_data.get("last_successful_run")
        if last_run_time_str:
            last_run_time = datetime.fromisoformat(last_run_time_str)
            print(f"Last successful run was at: {last_run_time}")
    except json.JSONDecodeError:
        print("Warning: State file is corrupted. Starting fresh.")
        last_run_time = None

# --- 2. Perform the main automation task ---
print("\nPerforming main automation task...")
# (Imagine your script does some work here, like processing new files)
print("Task completed successfully!")

# --- 3. Write the new state to the file ---
print(f"\nSaving current state to {STATE_FILE}...")
current_time = datetime.now()
new_state = {
    "last_successful_run": current_time.isoformat(),
    "run_status": "success",
    "version": "1.0"
}

# Convert the dictionary to a JSON string with indentation
output_json_string = json.dumps(new_state, indent=4)

# Write the string to the file
STATE_FILE.write_text(output_json_string, encoding="utf-8")
print("State saved.")

Run this script multiple times. You'll see that on the first run, it finds no state file. On subsequent runs, it reads the timestamp from the previous run. This simple mechanism is incredibly powerful for making your automations smarter and more resilient.

Conclusion

In this lesson, you've moved from just navigating the file system to actually manipulating file content. This is a critical skill for almost any automation task.

Here are the key takeaways:

• Always use the with open(...) statement for handling files to ensure they are closed properly.
• Explicitly set encoding="utf-8" when working with text and JSON files for maximum compatibility.
• Use plain text for simple, linear data like logs, and use the f.write() method or iterate line by line to read.
• Use JSON for structured data that maps to Python dictionaries and lists, using the json.dump() and json.load() functions.
• Saving a script's status in a JSON state file is a fundamental pattern for creating robust, resumable workflows.

You now know how to create paths and how to read from and write to them. However, what happens when a file you expect to read doesn't exist, or a JSON file contains invalid data? Our next lesson will address this by teaching you how to handle expected file, network, and data errors with targeted try/except blocks, making your scripts truly robust.