Calculating Baseline Performance Metrics

From Simulation to Measurement: Defining Key Metrics

Before we can improve a system, we must first measure it. In the context of logistics and operations, a few key performance indicators (KPIs) are universally important. For our Chittagong Port gate-in process, we will focus on four primary metrics:

• Cycle Time: This is the total time an entity (a truck) spends in the system, from the moment it arrives until the moment it completes its process and departs. It is a critical measure of overall process velocity and directly impacts customer satisfaction. In our model, it's departure_time - arrival_time.
• Wait Time: The portion of the cycle time that is non-productive. It's the time a truck spends in a queue, waiting for a required resource (like the gate) to become available. High wait times are a classic symptom of a bottleneck. In our model, it's service_start_time - arrival_time.
• Throughput: The rate at which the system successfully processes entities. It's typically measured in units per period of time (e.g., trucks per hour). Throughput defines the system's maximum capacity. In our simulation, it's total trucks processed / total simulation time.
• Resource Utilization: The percentage of time a resource is busy processing entities. While 100% utilization might sound ideal, in systems with variability, it guarantees ever-growing queues. Understanding utilization is key to capacity planning and identifying overworked resources.

The Physics of Queues: Why Metrics Explode

It's tempting to think that a system running at 80% utilization will have queues that are simply "80% full." However, the reality of systems with random arrivals and service times is highly non-linear. This relationship is often described by queuing theory, and its most important lesson for a systems architect is this: as utilization approaches 100%, average wait time and cycle time increase exponentially.

This graph illustrates the non-linear relationship between throughput (as a percentage of maximum capacity) and cycle time. Notice how cycle time skyrockets as utilization nears 100%, and this effect is amplified by higher process variability.

The core of this behavior is captured in Kingman's formula, a famous approximation in queuing theory. While the full equation is complex, its essence is that wait time is a function of three key factors:

1. Utilization ($\rho$): The ratio of arrival rate to service rate.
2. Variability ($c_a^2,c_s^2$): The unpredictability in arrival and service times.
3. Processing Time ($T_p$): The time spent in active service.

This image breaks down the components of cycle time. The wait time portion is directly influenced by Variability and Utilization, confirming the explosive relationship shown in the previous graph.

As an architect, the critical insight is that reducing variability (e.g., through scheduled appointments) or managing utilization (e.g., by adding capacity or smoothing demand) can have a far greater impact on cycle time than simply trying to speed up the processing time itself. Our simulation will allow us to measure and see these effects firsthand.

Data Collection Strategy in SimPy

SimPy provides the engine but is unopinionated about data collection. It's our responsibility to "instrument" the simulation to capture the data we need. The most common and flexible method is to record timestamps at key moments in an entity's lifecycle and store them for post-simulation analysis.

We will modify your truck_process generator to do exactly this. For each truck, we'll capture its ID and the relevant timestamps in a dictionary, which we'll then append to a global list. After the simulation finishes, we can easily convert this list of dictionaries into a pandas DataFrame for powerful and efficient analysis.

For a detailed walkthrough of this exact pattern, the following article is an excellent reference. It compares SimPy and Salabim, but its SimPy implementation perfectly demonstrates the data collection and analysis technique we will use.

Hands-On: Calculating Baseline Metrics

Let's apply this pattern to your full_simulation.py script.

1. Prepare for Data Collection

At the top of your script, import pandas and initialize an empty list to store the results.

import simpy
import random
import requests
import json
import pandas as pd  # Import pandas

# ... your other constants ...

# Global list to store results for each truck
simulation_results = []

2. Instrument the truck_process

Now, modify your truck_process function to capture timestamps and store the results.

def truck_process(env, name, gate):
    """
    The process a truck follows: arrive, get checked in, and depart.
    """
    # 1. Capture arrival time
    arrival_time = env.now
    print(f'{name} arrives at the port at {arrival_time:.2f}.')
    
    # --- Integration Point with Camunda (from previous lesson) ---
    appointment_id = f"APT-{random.randint(1000, 9999)}"
    start_gate_in_process(truck_id=name, appointment_id=appointment_id)
    
    # The truck now requests the gate resource
    with gate.request() as request:
        yield request
        
        # 2. Capture service start time
        service_start_time = env.now
        print(f'{name} enters the gate at {service_start_time:.2f}.')
        
        # Simulate the time it takes to be processed at the gate
        processing_time = random.uniform(3, 6)
        yield env.timeout(processing_time)
        
        # 3. Capture service end time
        service_end_time = env.now
    
    print(f'{name} leaves the port at {service_end_time:.2f}.')
    
    # 4. Record all metrics for this truck
    simulation_results.append({
        "truck_id": name,
        "arrival_time": arrival_time,
        "service_start_time": service_start_time,
        "service_end_time": service_end_time,
        "processing_time": processing_time
    })

3. Analyze the Results After the Simulation

After your env.run() call, add the following code block to perform the analysis using pandas.

# --- Run the simulation (existing code) ---
# env.run(until=SIMULATION_TIME) 

# --- New Analysis Section ---
print("\n--- Simulation Finished ---")
print("\n--- Performance Metrics Analysis ---")

if not simulation_results:
    print("No trucks completed the process. No metrics to calculate.")
else:
    # Convert results to a pandas DataFrame
    df = pd.DataFrame(simulation_results)
    
    # --- Calculate Metrics ---
    # Cycle Time
    df['cycle_time'] = df['service_end_time'] - df['arrival_time']
    avg_cycle_time = df['cycle_time'].mean()
    
    # Wait Time
    df['wait_time'] = df['service_start_time'] - df['arrival_time']
    avg_wait_time = df['wait_time'].mean()

    # Throughput
    total_trucks_processed = len(df)
    total_simulation_time = env.now # Use the final simulation time
    throughput_per_minute = total_trucks_processed / total_simulation_time
    throughput_per_hour = throughput_per_minute * 60

    # Utilization (Total busy time / Total available time)
    total_busy_time = df['processing_time'].sum()
    gate_capacity = gate.capacity # Assuming gate is your simpy.Resource
    total_available_time = total_simulation_time * gate_capacity
    utilization = (total_busy_time / total_available_time) * 100

    # --- Print Report ---
    print(f"Total trucks processed: {total_trucks_processed}")
    print(f"Total simulation time: {total_simulation_time:.2f} minutes")
    print("\n--- Averages ---")
    print(f"Average Cycle Time: {avg_cycle_time:.2f} minutes")
    print(f"Average Wait Time: {avg_wait_time:.2f} minutes")
    print(f"Average Processing Time: {df['processing_time'].mean():.2f} minutes")
    print("\n--- System Performance ---")
    print(f"Throughput: {throughput_per_hour:.2f} trucks per hour")
    print(f"Gate Utilization: {utilization:.2f}%")

After implementing these changes, run your full_simulation.py script (with the Camunda engine and worker also running). At the end of the run, you will see a detailed performance report printed to your console. This report is your quantitative baseline for the port's gate-in process.

Conclusion

In this lesson, you've transformed your simulation from a purely descriptive model into an analytical tool. You learned to define and measure the fundamental KPIs of any queuing system: cycle time, wait time, throughput, and utilization. By instrumenting your SimPy code and leveraging pandas for analysis, you have now established a quantitative baseline for your port logistics process. These numbers represent your "ground truth" and will be the basis for evaluating every architectural improvement you implement throughout this course.

In our next lesson, we will begin to stress-test our system. Now that you have a performance baseline under ideal conditions, we will introduce Toxiproxy, a network emulator, to programmatically inject latency and simulate the unreliable network conditions common in large-scale industrial environments. This will allow us to see how our system's performance metrics degrade under pressure.