Absorption Probabilities and Expected Time to Absorption

In our previous lesson, we established how to calculate transition probabilities over multiple time steps using matrix multiplication, a direct application of the Chapman-Kolmogorov equations. This allows us to answer questions about the state of a system at a specific future time, $t$ . Now, we will address a different but equally critical set of questions: what happens in the long run?

This lesson focuses on absorbing states—states like 'Dead' or 'Lapsed' from which there is no escape. We will develop the methods to calculate two key metrics for any actuarial model containing such states:

Absorption Probability: The probability of eventually ending up in a specific absorbing state, given a starting state.
Expected Time to Absorption: The average number of time steps until the process is absorbed into any of the absorbing states.

Given your background in probability theory, you'll recognize that these calculations are exercises in conditioning and expected values. We will explore two primary methods for these calculations: a first-step analysis leading to a system of linear equations, and a more powerful matrix-based approach using the fundamental matrix.

Absorbing States in Markov Chains

First, let's formally define our terms. In many actuarial models, certain states are final. Once a policyholder enters the 'Dead' state or a policy 'Lapses', it remains there permanently.

2.1 Markov chains and transition probabilities - Actuarial... - Fiveable

This article from Fiveable provides a concise definition of absorbing states within an actuarial context.

Please read the section on Absorbing states. This will formalize the concept we'll be working with throughout this lesson. The states that are not absorbing are called transient states.

Method 1: First-Step Analysis

The most intuitive way to calculate absorption probabilities and expected times is to consider what can happen in the very next step. This technique, known as first-step analysis or conditioning on the first step, breaks a long-term problem down into an immediate transition followed by a new long-term problem.

Calculating Absorption Probabilities

Let's say we want to find the probability of eventually being absorbed into a specific absorbing state, $S_{A}$ , starting from a transient state, $S_{i}$ . Let's call this probability $a_{i}$ . After one step, the process will transition to some state $S_{j}$ with probability $p_{ij}$ .

If $S_{j}$ is the absorbing state $S_{A}$ , the process is absorbed, and the journey is over.
If $S_{j}$ is another absorbing state, we have failed to reach $S_{A}$ .
If $S_{j}$ is a transient state, we are now at $S_{j}$ and the "new" problem is to find the probability of being absorbed into $S_{A}$ starting from $S_{j}$ . This is simply $a_{j}$ .

By summing over all possibilities, we can construct a system of linear equations.

L26.6 Absorption Probabilities

This MIT OpenCourseWare lecture provides an excellent walkthrough of setting up and solving for absorption probabilities using first-step analysis.

First, watch the brief introduction to absorbing and transient states to see a visual example. Next, focus on the detailed explanation of how to set up the system of linear equations. The lecturer methodically conditions on the outcome of the first transition to build the equations. Finally, observe how this method is generalized and can be used to find the probability of absorption into different absorbing states.

Let's formalize this. Let $T$ be the set of transient states and $A$ be the set of absorbing states. The probability $a_{ik}$ that the chain, starting in transient state $i \in T$ , is eventually absorbed into state $k \in A$ is given by the solution to the following system of linear equations:

a_{ik} = p_{ik} + j \in T \sum p_{ij} a_{j k} for each i \in T

Here, $p_{ik}$ is the probability of going directly from transient state $i$ to absorbing state $k$ in one step. The summation term covers all paths that go through another transient state $j$ first.

Calculating Expected Time to Absorption

We can use the exact same logic to find the expected number of steps until the process hits any absorbing state. Let $μ_{i}$ be the expected number of steps to absorption, starting from transient state $i$ .

After one step (which always counts as "+1" to our time), we transition to some state $j$ with probability $p_{ij}$ .

If $j$ is an absorbing state, the process stops. The additional time from state $j$ is 0.
If $j$ is a transient state, the remaining expected time from that point is $μ_{j}$ .

This leads to another system of linear equations.

L26.7 Expected Time to Absorption

This lecture, a follow-up to the previous one, applies the same first-step analysis to calculate the expected time to absorption.

Watch the core demonstration from setting up the equations. Notice the crucial "1 +" term, which accounts for the step that has just been taken. Then, watch the segment explaining how to handle multiple absorbing states by combining them into a single 'mega state'. This is a key technique when you are interested in the time to absorption, regardless of which absorbing state is entered.

The system of equations for the expected time to absorption, $μ_{i}$ , for each transient state $i \in T$ is:

μ_{i} = 1 + j \in T \sum p_{ij} μ_{j}

Note that the sum is only over the transient states $j$ . If the process moves to an absorbing state, the process stops, and that path contributes nothing further to the expected time.

Method 2: The Fundamental Matrix

While solving systems of equations is effective for small models, it becomes inefficient for models with many transient states. A more powerful and elegant matrix-based method exists, which is standard in actuarial software.

Let the full transition matrix be $P$ . We can partition it based on transient ( $T$ ) and absorbing ( $A$ ) states:

P = (Q 0 R I)

where:

$Q$ is a sub-matrix containing transition probabilities between transient states ( $T \to T$ ).
$R$ is a sub-matrix containing transition probabilities from transient to absorbing states ( $T \to A$ ).
$0$ is a matrix of zeros (you cannot leave an absorbing state).
$I$ is the identity matrix (you stay in an absorbing state).

The fundamental matrix, denoted $N$ , is defined as:

N = (I - Q)^{- 1}

The entry $N_{ij}$ of this matrix has a wonderful interpretation: it is the expected number of times the process visits transient state $j$ , given that it started in transient state $i$ .

With the fundamental matrix, calculating the expected time to absorption becomes trivial.

2.1 Markov chains and transition probabilities - Actuarial... - Fiveable

This part of the Fiveable article introduces the fundamental matrix and its direct application.

Please read the subsections Fundamental matrix and Time to absorption. This gives you the core formulas for this method.

As the reading states, the expected time to absorption starting from transient state $i$ , which we called $μ_{i}$ , is simply the sum of the $i$ -th row of the fundamental matrix $N$ .

μ_{i} = j \in T \sum N_{ij}

This makes intuitive sense: the total expected time before absorption is the sum of the expected times spent in each of the transient states.

Let's see this powerful technique applied to a practical, multi-state problem.

[PDF] A Markov Chain Model for Covid-19 Survival Analysis

This research paper uses a 5-state Markov chain with an absorbing 'Dead' state to model Covid-19 progression. It's a perfect case study for the fundamental matrix method.

Start by examining the section that <span data-type="resource_reading_textrange" data-resource-subitem-id="dd2483c7" data-range-start="To achieve this, we obtain from P the sub -matrix" data-range-end="of all transient states, by deleting the row and column">defines the matrix Q from the full transition matrix. 2. Follow the calculation of the inverse matrix (I-Q)^{-1}, which they call the "Potential matrix" (our Fundamental Matrix, N). See how they interpret its entries as the average number of visits. 3. Critically, observe how they calculate the average time to death by summing the rows of this matrix, as presented in Table #2. 4. Finally, the summary in Section 4.0 recaps this exact process.

The fundamental matrix can also be used to find the absorption probabilities. The probability of being absorbed into state $k$ , starting from state $i$ , is given by the $(i, k)$ -th entry of the matrix product $N \times R$ . While this is a useful result, for exam purposes, solving the system of linear equations for absorption probabilities is often just as fast for simple problems.

Conclusion

In this lesson, we have equipped ourselves with the tools to analyze the long-term behavior of Markov chains that include absorbing states. These methods are fundamental to pricing life insurance and annuities, modeling policy lapses, and analyzing any risk process that has a terminal outcome.

Key Takeaways:

Absorption Probability: The probability of eventually entering a specific absorbing state. Can be found by setting up and solving a system of linear equations derived from a first-step analysis.
Expected Time to Absorption: The average number of steps until entering any absorbing state. This can also be found by solving a system of linear equations.
Fundamental Matrix ( $N$ ): A more powerful tool, calculated as $N = (I - Q)^{- 1}$ , where $Q$ is the matrix of transient-to-transient transitions.
The entry $N_{ij}$ gives the expected number of visits to transient state $j$ starting from transient state $i$ .
The expected time to absorption from state $i$ is the sum of the $i$ -th row of the fundamental matrix $N$ .

We have now concluded our exploration of the core tools for discrete-time Markov chains. In the next lesson, we will shift our focus to continuous time by introducing the Poisson process. This process is the cornerstone for modeling the timing of random events, such as the arrival of insurance claims, and serves as the simplest form of a continuous-time Markov process.

Recommended resources