Analyzing Seasonal & Event-Based Violation Rate Changes

Step 1: Defining Custom Time Periods

To compare different periods, you first need to label each data point in your DataFrame according to whether it falls within a period of interest (e.g., 'monsoon') or the 'baseline'. The most flexible way to do this is by creating a new column that categorizes each date.

Let's assume your data is in a DataFrame df with a DatetimeIndex. Your goal is to create a new column, let's call it period_label, that contains strings like 'monsoon_season', 'festival_period', or 'baseline'. A robust way to achieve this is by writing a function that takes a date and returns the appropriate label.

The Stack Overflow discussion "How to groupby in pandas for multiple periods" provides an excellent template for this task.

Let's adapt this concept for your project. You can define a function that checks if a date falls within the monsoon months (e.g., June to September) or a list of specific festival dates.

Here is a practical implementation. You would first define your periods, then apply the function to your DataFrame.

import pandas as pd




# Assume 'df' is your DataFrame with a DatetimeIndex and a 'violation_rate' column.




# 1. Define your specific periods
# For a recurring period like monsoon
MONSOON_START_MONTH = 6
MONSOON_END_MONTH = 9




# For specific, non-recurring events like festivals
festival_dates = pd.to_datetime(['2023-10-24', '2023-11-12']) # Example: Dussehra, Diwali




# 2. Create a function to label each date
def get_period_label(date):
    if date.month >= MONSOON_START_MONTH and date.month <= MONSOON_END_MONTH:
        return 'monsoon'



    # Check if the date is within a few days of a festival
    elif any((date >= fest_date and date <= fest_date + pd.Timedelta(days=2)) for fest_date in festival_dates):
        return 'festival'
    else:
        return 'baseline'




# 3. Apply the function to create the new column
df['period_label'] = df.index.map(get_period_label)




# Display the result
print(df[['violation_rate', 'period_label']].sample(5))

This code snippet gives you a new period_label column, which is the foundation for the next step of our analysis.

Step 2: Aggregating and Comparing Metrics

With your data now labeled, you can easily compare the average violation rate across these different periods using a groupby operation. This aggregates the data and gives you a summary table, making the differences immediately apparent.

# Group by the new label and calculate the mean violation rate for each period
period_comparison = df.groupby('period_label')['violation_rate'].mean()

print(period_comparison)

Running this would produce a simple but powerful output, for instance:

period_label
baseline    0.15
festival    0.28
monsoon     0.25
Name: violation_rate, dtype: float64

This table clearly shows that, in this hypothetical example, the average violation rate is higher during festival and monsoon periods compared to the baseline.

Step 3: Quantifying the Change

While the table above is insightful, your report will be more impactful if you quantify the magnitude of this change. The most common way to do this is by calculating the percentage change relative to the baseline. The formula is:

In our case, the "Old Value" is the metric for the baseline period, and the "New Value" is the metric for the monsoon or festival period.

You can perform this calculation directly on the period_comparison Series we created.

# Extract the baseline value
baseline_rate = period_comparison.loc['baseline']




# Calculate the percentage change for the monsoon period
monsoon_rate = period_comparison.loc['monsoon']
monsoon_change_pct = ((monsoon_rate - baseline_rate) / baseline_rate) * 100




# Calculate the percentage change for the festival period
festival_rate = period_comparison.loc['festival']
festival_change_pct = ((festival_rate - baseline_rate) / baseline_rate) * 100

print(f"Violation rate during monsoon is {monsoon_change_pct:.2f}% higher than baseline.")
print(f"Violation rate during festivals is {festival_change_pct:.2f}% higher than baseline.")

This gives you a precise, quantifiable statement like: "The violation rate during the monsoon season was 66.67% higher than the baseline period."

The article from AbsentData, "3 Ways to Calculate Percent Change in Python," provides more context on calculating percentage changes in pandas, though it focuses on consecutive changes.

Beyond Seasons: Identifying Anomalous Periods

Your analysis can also target periods that are not defined by a calendar, but by the data itself. These are often called anomalies: days or weeks where the violation rate unexpectedly spikes. Identifying these periods can uncover issues that aren't tied to predictable events like seasons.

A simple way to define an anomaly is to flag any day where the violation rate is significantly higher than a moving average, for example, more than two standard deviations above a 30-day rolling mean.

Once you've identified these anomalous dates, you can label them just as we did for 'monsoon' and 'festival' periods and then perform the same quantitative comparison against the baseline. This adds another layer of depth to your temporal analysis.

Conclusion

In this lesson, you have moved from visualizing trends to quantifying their impact. You now have a clear, step-by-step process to measure how much meal quality metrics change during specific, important periods.

Here are the key takeaways:

• Define and Label: The first step is to create a new column in your DataFrame that labels each date based on custom criteria (e.g., 'monsoon', 'festival', 'baseline').
• Group and Aggregate: Use df.groupby() on your new label column to calculate summary statistics (like the mean violation rate) for each period.
• Quantify the Change: Calculate the percentage change of your metric for each period relative to the baseline to produce powerful, reportable findings.
• Look for Anomalies: Beyond predefined seasons, you can programmatically identify anomalous periods where metrics spike unexpectedly and analyze them using the same quantitative framework.

You are now equipped to provide quantitative answers to key temporal questions in your project. In our next and final lesson of this module, we will explore statistical testing. You will learn to use the chi-square test to formally assess the relationship between categorical variables, such as determining if there is a statistically significant association between the type of school (e.g., government vs. private) and the occurrence of meal quality violations.