Kolmogorov Equations in Multi-State Actuarial Models

From Chapman-Kolmogorov to Differential Equations

Let's start with the familiar Chapman-Kolmogorov equation, which states that to get from state $i$ to state $j$ in time $t+h$, the process must be in some intermediate state $k$ at time $t$:
$p_{ij}(t+h)={\sum }_{k\in S}{p}_{ik}(t)p_{kj}(h)$

Now, consider a very small time interval $h>0$. From our work with generator matrices, we know the transition probabilities over this small interval can be approximated as:

• For $k\ne j$, $p_{kj}(h)\approx q_{kj}h$. This is the probability of a jump from $k$ to $j$.
• For $k=j$, $p_{jj}(h)\approx 1+q_{jj}h$. This is the probability of not leaving state $j$.

Substituting these into the Chapman-Kolmogorov equation gives:
$p_{ij}(t+h)\approx {\sum }_{k\ne j}{p}_{ik}(t)(q_{kj}h)+p_{ij}(t)(1+q_{jj}h)$
Rearranging this to isolate the change in probability, we get:
$p_{ij}(t+h)-p_{ij}(t)\approx h\left({\sum }_{k\ne j}{p}_{ik}(t)q_{kj}+p_{ij}(t)q_{jj}\right)$
Dividing by $h$ and taking the limit as $h\to 0$, we arrive at a differential equation:
$\frac{d}{dt}{p}_{ij}(t)={\sum }_{k\in S}{p}_{ik}(t)q_{kj}$

This is Kolmogorov's Forward Equation in component form. It's called the "forward" equation because its derivation looks at the last small step in the time interval, from $t$ to $t+h$, considering all possible transitions into the final state $j$.

In matrix form, if $\mathbf{P}(t)$ is the matrix of transition probabilities $[p_{ij}(t)]$ and $\mathbf{Q}$ is the generator matrix $[q_{ij}]$, the entire system of equations can be written concisely as:
$\frac{d}{dt}\mathbf{P}(t)=\mathbf{P}(t)\mathbf{Q}$

The following resource formally presents the forward equations and demonstrates their application to the simple single decrement (alive/dead) model, showing how they produce the familiar results for survival probability.

The Backward Equations: A Different Perspective

There is a second, equally important set of equations derived by considering the first step in the time interval, from time 0 to $h$. This time, we write the Chapman-Kolmogorov equation as $p_{ij}(t+h)={\sum }_k{p}_{ik}(h)p_{kj}(t)$.

Following a similar derivation, we arrive at Kolmogorov's Backward Equation:
$\frac{d}{dt}{p}_{ij}(t)={\sum }_{k\in S}{q}_{ik}{p}_{kj}(t)$
In matrix form, this is:
$\frac{d}{dt}\mathbf{P}(t)=\mathbf{QP}(t)$

Notice the crucial difference: the matrix multiplication is reversed. The backward equation considers transitions out of the initial state $i$. For time-homogeneous processes, both the forward and backward equations yield the same solution for $\mathbf{P}(t)$, but one may be easier to work with depending on the problem structure.

This next reading provides the formal definition of the backward equation.

Solving the Equations: A Worked Example

Setting up the equations is the first step; solving them is the next. For a model with constant transition intensities, we have a system of linear first-order ordinary differential equations (ODEs), which can be solved analytically.

Let's consider the classic permanent disability model, a three-state model:

• State 0: Healthy
• State 1: Permanently Disabled
• State 2: Dead (absorbing state)

Possible transitions are Healthy $\to$ Disabled ($q_{01}$), Healthy $\to$ Dead ($q_{02}$), and Disabled $\to$ Dead ($q_{12}$). The generator matrix is:

Using the forward equations, ${\mathbf{P}}^{\prime }(t)=\mathbf{P}(t)\mathbf{Q}$, we can write out the component equations. For example, for the (0,0) and (0,1) entries of the matrix equation:

1. $p_{00}^{\prime }(t)=p_{00}(t)q_{00}=-p_{00}(t)(q_{01}+q_{02})$
2. $p_{01}^{\prime }(t)=p_{00}(t)q_{01}+p_{01}(t)q_{11}=p_{00}(t)q_{01}-p_{01}(t)q_{12}$

Equation (1) is a simple ODE with initial condition $p_{00}(0)=1$. Its solution is $p_{00}(t)=e^{-(q_{01}+q_{02})t}$. This makes intuitive sense, as $p_{00}(t)$ is the probability of staying in state 0, and the total exit rate is $q_{01}+q_{02}$.

Once we have $p_{00}(t)$, we can substitute it into equation (2) and solve the resulting ODE for $p_{01}(t)$. The following resource provides a complete, step-by-step solution for this exact problem with numerical values.

To see the backward equations in action, the following video demonstrates how to set them up for a similar three-state model. While it doesn't carry out the full solution, it provides excellent intuition for how the backward equations are constructed.

Conclusion

In this lesson, we derived the fundamental differential equations that govern the dynamics of continuous-time Markov processes. These equations connect the instantaneous transition rates in the generator matrix to the finite-time transition probabilities.

Key Takeaways:

• Kolmogorov's equations are systems of linear ordinary differential equations describing the evolution of transition probabilities $p_{ij}(t)$.
• The Forward Equations, ${\mathbf{P}}^{\prime }(t)=\mathbf{P}(t)\mathbf{Q}$, are derived by considering the final transition into the destination state $j$.
• The Backward Equations, ${\mathbf{P}}^{\prime }(t)=\mathbf{QP}(t)$, are derived by considering the first transition out of the initial state $i$.
• For time-homogeneous models with constant intensities, these systems of ODEs can be solved analytically, often by first solving for the occupancy probabilities and then using those solutions to find the other probabilities.

Solving these systems by hand can become tedious for models with many states. In our next lesson, we will explore a more powerful and elegant method for solving the Kolmogorov equations in the time-homogeneous case: matrix exponentiation. This will allow us to compute the entire transition probability matrix $\mathbf{P}(t)$ in a single operation.