SimPy & Camunda: Driving Processes with Simulated Events

Architectural Approaches: Integrating Simulation and Workflow

When combining Discrete-Event Simulation (DES) with Business Process Management (BPM), there are two primary architectural patterns. Understanding them is key to choosing the right tool for the job.

1. Simulation-Driven Workflow (Our Approach): In this pattern, the DES model represents the "outside world." It generates entities and events (e.g., trucks arriving, ships docking) at stochastic intervals. Each significant event triggers the start of a new workflow instance in the BPM engine. This is ideal for modeling systems that react to external, often unpredictable, demand. Our Chittagong Port scenario, with trucks arriving randomly, fits this pattern perfectly. The SimPy model dictates when a process starts, and Camunda dictates what happens next.

2. Workflow-Driven Simulation: Here, the BPMN process is the master controller. Specific tasks within the workflow are themselves small, self-contained simulations. For example, a "Mill Part" task in a manufacturing process might trigger a DES model that simulates machine setup, processing time, and potential breakdowns. The result of the simulation then determines the path forward in the BPMN diagram.

The image below illustrates an example closer to the second pattern, where a high-level BPMN task ('Cutting plant') is decomposed into a detailed DES model.

This diagram shows a high-level BPMN activity being broken down into a more detailed discrete-event simulation model, representing a "workflow-driven simulation" approach.

For our port logistics system, we will implement the simulation-driven workflow pattern. Your SimPy script will act as the event generator, using Camunda's REST API to initiate a gate-in-process for every truck that arrives in the simulation.

The Bridge: Camunda's REST API

The connection between your SimPy script and the Camunda engine is the Camunda REST API. This is the same API that the external task worker uses under the hood, but we will call a different endpoint directly. To start a process instance, we need to send an HTTP POST request to:

POST /process-definition/key/{key}/start

This request requires two key pieces of information:

• The Process Definition Key: This is the ID of your process model as defined in the Camunda Modeler (e.g., gate-in-process).

• The Request Body: A JSON payload containing any initial variables required to start the process. For us, this will be the truckId and appointmentId.

Hands-On: Triggering Workflows from SimPy

Let's modify the simulation code you developed in a previous lesson. We'll add a function to handle the API call and then integrate it into the SimPy truck process.

1. Create an API Client Function

It's good practice to encapsulate API interactions in their own functions. In a new Python script (or by modifying your existing simulation script), add the requests library and create the following function. This function will be responsible for starting a workflow instance for a given truck.

import requests
import json

CAMUNDA_ENGINE_URL = "http://localhost:8080/engine-rest"
PROCESS_DEFINITION_KEY = "gate-in-process" # Ensure this matches the ID in your BPMN file

def start_gate_in_process(truck_id: str, appointment_id: str):
    """
    Starts a new 'gate-in-process' instance in Camunda via the REST API.
    """
    start_process_url = f"{CAMUNDA_ENGINE_URL}/process-definition/key/{PROCESS_DEFINITION_KEY}/start"
    
    payload = {
        "variables": {
            "truckId": {"value": truck_id, "type": "String"},
            "appointmentId": {"value": appointment_id, "type": "String"}
        }
    }
    
    headers = {
        "Content-Type": "application/json"
    }
    
    try:
        response = requests.post(start_process_url, headers=headers, data=json.dumps(payload))
        response.raise_for_status()  # Raises an exception for bad responses (4xx or 5xx)
        
        process_instance_id = response.json().get('id')
        print(f"Successfully started process instance {process_instance_id} for truck {truck_id}")
        
    except requests.exceptions.RequestException as e:
        print(f"Error starting process instance for truck {truck_id}: {e}")

This function neatly packages the logic: it constructs the URL, builds the JSON payload with the required variables, sets the content type, and makes the POST request. The error handling ensures your simulation won't crash if the Camunda engine is unreachable.

2. Integrate into the SimPy Process

Now, let's call this function from your SimPy truck process generator. Find the function that defines the truck's behavior (e.g., truck_process or run_truck). At the point where a truck "arrives", we will insert the call to start_gate_in_process.

import simpy
import random
# ... other imports including requests, json, and the function above

def truck_process(env, name, gate):
    """
    The process a truck follows: arrive, get checked in.
    """
    print(f'{name} arrives at the port at {env.now:.2f}.')
    
    # --- Integration Point ---
    # Generate a unique appointment ID for this truck's arrival
    appointment_id = f"APT-{random.randint(1000, 9999)}"
    # Trigger the Camunda workflow for this truck
    start_gate_in_process(truck_id=name, appointment_id=appointment_id)
    # -------------------------

    # The rest of the SimPy logic can continue, e.g., waiting in a queue.
    # For now, we are just triggering the workflow. The actual "check-in"
    # logic is now handled by the Camunda process and its external worker.
    with gate.request() as request:
        yield request
        print(f'{name} enters the gate at {env.now:.2f}.')
        # In a more complex model, the workflow could signal the simulation back
        # to release the truck from this wait.
        yield env.timeout(random.uniform(3, 6)) # Represents time inside the port
    
    print(f'{name} leaves the port at {env.now:.2f}.')

# Your existing SimPy setup code:
# env = simpy.Environment()
# gate = simpy.Resource(env, capacity=1)
# for i in range(5):
#     env.process(truck_process(env, f'Truck-{i}', gate))
#     yield env.timeout(random.uniform(5, 10))
# env.run()

Notice the clean separation. The simulation's only responsibility is to announce the arrival event to the orchestrator (Camunda). It doesn't need to know the details of the check-in procedure. That logic lives in the BPMN model and the external worker you already built.

Running the End-to-End System

Now you have all the pieces to run a fully integrated simulation.

1. Start Services:

   • Ensure your Camunda engine is running (via docker-compose up).
   • Open a terminal and start your Python external task worker from the previous lesson: python gate_worker.py. It will begin polling, waiting for tasks.

2. Run the Simulation:

   • Open a second terminal.
   • Run your updated SimPy script: python full_simulation.py.

3. Observe the Chain Reaction:

   • In the simulation terminal: You will see messages like Truck-0 arrives at the port at 0.00. followed by Successfully started process instance....
   • In the worker terminal: Almost immediately, you will see the worker spring to life, printing Received task... and Completing task... for the truck whose arrival was just simulated.
   • In Camunda Cockpit: Navigate to http://localhost:8080/camunda/app/cockpit/. You will see a new process instance for Truck-0, Truck-1, and so on, appearing exactly at the moments they were created by the simulation. You can click on any of them and see that the token has passed the "Check-in Truck" task, and the variables returned from your worker (checkInStatus, etc.) are now part of the process instance data.

You have now created a digital twin of your gate-in process where a DES model simulates the physical world and a BPM engine orchestrates the corresponding business logic.

For further exploration of robust API integration, the PyCamunda library, mentioned in the "Implementing an external task worker" article, offers a more object-oriented way to build and send these API requests, which can be useful in larger, more complex applications.

Conclusion

In this lesson, you have achieved a critical integration milestone. By using Camunda's REST API, you have empowered your SimPy simulation to drive the creation of workflow instances. This transforms your simulation from a standalone analytical model into a dynamic driver for a complete, orchestrated process. You have seen how a simulation event (truck arrival) triggers a business process, which in turn delegates work to your external Python worker—all without any component needing full knowledge of the others. This decoupled, message-driven architecture is a hallmark of robust, scalable enterprise systems.

In our next lesson, we will leverage this integrated system. With the simulation running and generating process data in Camunda, we will focus on how to define, extract, and calculate baseline performance metrics, such as cycle time and resource utilization, directly from the simulation outputs and process history.