Reading and Writing Files

1. Reading and Writing Simple Text Files

Let's start with the basics: plain text files (.txt). While less common for structured data, they are often used for logs, notes, or configuration files. The fundamental process involves three steps: opening the file, performing an action (read or write), and closing it.

To ensure files are always closed properly, even if errors occur, Python provides a clean and safe syntax called a context manager, which uses the with keyword. This is the standard you should always use.

with open('filename.txt', 'mode') as file_variable:
    # Perform operations on the file here
# The file is automatically closed when this block is exited

The 'mode' tells Python what you want to do:

• 'r': Read (default). Throws an error if the file doesn't exist.
• 'w': Write. Overwrites the entire file if it exists, or creates a new one if it doesn't.
• 'a': Append. Adds new content to the end of an existing file, or creates a new one if it doesn't.

A Practical Example

Let's combine what we've learned. Imagine you have a file, raw_notes.txt, with some messy, user-entered notes:

raw_notes.txt:

  first idea: need to check campaign performance
  Second Idea: update stakeholder list
third idea: ANALYZE Q3 CLICK DATA  

Our goal is to read this file, clean up each line (trim whitespace, standardize to sentence case), and write the cleaned notes to a new file, cleaned_notes.txt.

# Open the input file for reading and create a new file for writing
with open('raw_notes.txt', 'r') as infile, open('cleaned_notes.txt', 'w') as outfile:
    # Loop through each line in the input file
    for line in infile:
        # Use string methods from our last lesson!
        cleaned_line = line.strip().capitalize()
        
        # Write the cleaned line to the output file, adding a newline character
        outfile.write(cleaned_line + '\n')

print("File has been cleaned and saved as cleaned_notes.txt")

After running this, cleaned_notes.txt will contain:

First idea: need to check campaign performance
Second idea: update stakeholder list
Third idea: analyze q3 click data

This simple script demonstrates a complete workflow: reading raw data, transforming it, and loading it into a new file.

2. The Power of CSV for Structured Data

Most business data isn't just a list of notes; it's tabular, with rows and columns. The Comma-Separated Values (CSV) format is the universal standard for exchanging this kind of data.

You might be tempted to read a CSV file and just use line.split(','), but this is unreliable. What if a text field contains a comma, like "New York, NY"? Your simple split would break. Python's built-in csv module is designed to handle these complexities gracefully.

To get a solid grasp of how to use this module, we'll watch a comprehensive video tutorial.

Why DictReader and DictWriter are a BA's Best Friends

As you saw in the video, csv.reader returns a list, like ['John', 'Doe', 'john.doe@example.com']. To get the email, you need to remember it's at index 2.

csv.DictReader, however, returns a dictionary-like object: {'first_name': 'John', 'last_name': 'Doe', 'email': 'john.doe@example.com'}. Now you can access the email with row['email']. This makes your code:

• More Readable: It's immediately clear what data you're working with.
• More Robust: If someone adds a new column to the CSV file, your code won't break as long as the column names you use still exist. With csv.reader, adding a column would shift all the indices and break your code.

For these reasons, you should almost always prefer DictReader and DictWriter when working with tabular data.

clicks_as_integer = int(row['Clicks'])
print(clicks_as_integer) # Output: 1500

3. Putting It All Together: An AdTech Analysis Task

Let's apply everything to a practical mini-project relevant to your AdTech background.

You've been given a file, ad_performance.csv, containing raw data from a campaign.

ad_performance.csv:

campaign_name,impressions,clicks,spend
Summer_Sale_2024, 50000, 2500, "$1,250.50"
Winter_Promo, 120000, 3600, "$2,800.00"
Back_to_School, 85000, 4000, "$1,950.75"

Notice the data has issues: spaces around the numbers and a dollar sign and comma in the spend column.

Your task:

1. Read ad_performance.csv.
2. For each campaign, calculate the Click-Through Rate (CTR), where CTR = (Clicks / Impressions) * 100.
3. Calculate the Cost Per Click (CPC), where CPC = Spend / Clicks.
4. Write a new file, campaign_report.csv, with the original data plus the two new metrics (CTR and CPC), ensuring all numbers are formatted cleanly.

Here is the complete script to accomplish this. Read through it carefully, paying attention to the comments.

import csv

# Define the headers for our output file. This includes our new calculated fields.
fieldnames = ['campaign_name', 'impressions', 'clicks', 'spend', 'ctr', 'cpc']

# Use 'with' to open both input and output files
with open('ad_performance.csv', mode='r', encoding='utf-8') as infile, \
     open('campaign_report.csv', mode='w', newline='', encoding='utf-8') as outfile:
    
    # Use DictReader for robust reading
    reader = csv.DictReader(infile)
    
    # Use DictWriter and pass it the fieldnames for the output file
    writer = csv.DictWriter(outfile, fieldnames=fieldnames)
    
    # Write the header row to our new file
    writer.writeheader()
    
    print("Processing campaign data...")
    
    # Loop through each row in the input file (each row is a dictionary)
    for row in reader:
        # --- Transformation Step ---
        # Clean and convert data types, using skills from the previous lesson
        impressions = int(row['impressions'].strip())
        clicks = int(row['clicks'].strip())
        
        # Clean the 'spend' string and convert to a float
        spend_str = row['spend'].strip().replace('$', '').replace(',', '')
        spend = float(spend_str)
        
        # Calculate our new metrics
        # Avoid division by zero if impressions or clicks are 0
        ctr = (clicks / impressions) * 100 if impressions > 0 else 0
        cpc = spend / clicks if clicks > 0 else 0
        
        # --- Prepare the output row ---
        # The writerow method expects a dictionary. We can update the original 'row' dictionary
        # with our new, calculated values.
        row['spend'] = f"{spend:,.2f}" # Format spend back to a clean string
        row['ctr'] = f"{ctr:.2f}%"  # Format CTR as a percentage string
        row['cpc'] = f"${cpc:.2f}"   # Format CPC as a currency string
        
        # Write the complete, transformed row to the output file
        writer.writerow(row)

print("Report 'campaign_report.csv' has been successfully generated.")

After running this script, you will have a new file, campaign_report.csv, that looks like this:

campaign_report.csv:

campaign_name,impressions,clicks,spend,ctr,cpc
Summer_Sale_2024,50000,2500,"1,250.50",5.00%,$0.50
Winter_Promo,120000,3600,"2,800.00",3.00%,$0.78
Back_to_School,85000,4000,"1,950.75",4.71%,$0.49

This example is a microcosm of what you'll do as a BA: take raw data, clean it, enrich it with new calculations, and present it in a structured, readable format.

Conclusion

Congratulations! You've just learned one of the most fundamental and practical skills in programming for data analysis. You can now confidently get data out of files and save your results back, forming a complete analytical workflow.

Key Takeaways:

• Always use with open(...): It's the safest and cleanest way to handle files in Python.
• Know your modes: Use 'r' for reading, 'w' for overwriting, and 'a' for appending.
• Use the csv module for CSV files: It handles the format's complexities, so you don't have to.
• Prefer DictReader and DictWriter: They make your code for handling tabular data more readable, robust, and less prone to errors.

Preview of the Next Lesson:
So far, we've only used Python's built-in capabilities. While powerful, the real magic of Python for data analysis comes from its vast ecosystem of external packages. The csv module is great, but a package like pandas can read that same CSV file in one line and provide much more powerful tools for analysis.

In our next lesson, we will learn how to understand and install external packages using pip, Python's package manager. This will unlock a whole new level of efficiency and capability for your data analysis tasks.