Expected Present Value of State-Contingent Cash Flows

Introduction

In our last lesson, we mastered a powerful technique for finding the transition probability matrix $P (t)$ for any time-homogeneous Markov jump process by calculating the matrix exponential $e^{t Q}$ . This gave us the ability to determine $p_{ij} (t)$ , the probability of being in state $j$ at time $t$ given a start in state $i$ .

Now, we will put those probabilities to practical use. This lesson introduces one of the most fundamental concepts in actuarial science: the Actuarial Present Value (APV), also known as the Expected Present Value (EPV). We will learn how to calculate the value today of future, uncertain cash flows that depend on the state of a policyholder. This is the cornerstone of pricing insurance contracts and establishing the necessary funds, or reserves, to pay future benefits. We will explore two main types of cash flows: those paid while in a particular state (like an annuity) and those paid upon a transition between states (like a death benefit).

The Principle of Actuarial Present Value

At its heart, calculating an APV is an application of concepts you are already familiar with from probability theory, primarily the law of total expectation. To find the expected value of a future uncertain payment, we consider every possible time it could occur. For each time, we calculate the value of that payment by multiplying three key components:

The Probability: The probability that the event triggering the cash flow occurs at that specific time.
The Amount: The value of the cash flow itself.
The Discount Factor: A factor that discounts the cash flow from its future payment date back to the present time, accounting for the time value of money.

The total APV is then the sum of these values over all possible future times. This is often called the "triple-product summation" or "3πΣ" approach.

Let's formalize this by looking at the two primary types of state-contingent cash flows.

Cash Flows While in a State (Annuities)

The first type of cash flow is a payment made for being in a particular state at a specific point in time. A classic example is a disability income benefit, where an individual receives a regular payment as long as they remain in the "Disabled" state.

Let's define some notation. We will assume a discrete time model for simplicity, with time steps $n, n + 1, n + 2, \dots$ .

$_{k} p_{s}^{i}$ (or $_{k} Q (s, i)_{n}$ in the notation of some texts) is the probability that a life currently in state $s$ at time $n$ will be in state $i$ at time $n + k$ .
$_{n + k} C (i)$ is the cash flow amount paid at time $n + k$ if the person is in state $i$ .
$_{k} v_{n}$ is the discount factor for a payment made $k$ periods in the future, as viewed from time $n$ . If interest is constant at rate $i$ , then $_{k} v_{n} = v^{k} = (1 + i)^{- k}$ .

The APV, as seen from time $n$ for a person in state $s$ , of all future payments for being in state $i$ , is the sum of the expected present values of each potential payment:

A P V_{s @ n} (C (i)) = k = 0 \sum \infty [_{k} p_{s}^{i}] \cdot [_{n + k} C (i)] \cdot [_{k} v_{n}]

Here, the three components of the triple-product are clear: probability, amount, and discount factor.

The following resource provides a formal definition and a helpful example.

[PDF] Multi-state transition models for actuarial applications - SOA

This document from the Society of Actuaries clearly lays out the formulas for APV calculations. We will first focus on cash flows for being in a state.

Please read the section "2.3 Cash ﬂows while in states". Focus on Notation 2.11, which defines the cash flow, and Theorem 2.12, which presents the APV formula we just discussed. Then, carefully work through Example 2.13 to see how the formula is applied.

Let's break down the calculation in Example 2.13. The subject is in State 1 at time $n$ , and a cash flow of 1 occurs for being in State 1 at times $n, n + 1,$ and $n + 2$ . The interest rate is 25% ( $v = 0.8$ ). The transition matrix is $Q = (0.4 0.8 0.6 0.2)$ .

The APV is the sum of the present values of the three potential payments:

Payment at time n: The person is certainly in State 1. Probability = 1. Amount = 1. Discount factor = $v^{0} = 1$ . PV = $1 \cdot 1 \cdot 1 = 1$ .
Payment at time n+1: The probability of being in State 1 is $_{1} p_{1}^{1} = 0.4$ . Amount = 1. Discount factor = $v^{1} = 0.8$ . PV = $0.4 \cdot 1 \cdot 0.8 = 0.32$ .
Payment at time n+2: The probability of being in State 1 is $_{2} p_{1}^{1}$ . This is the (1,1) entry of $Q^{2}$ .
$Q^{2} = (0.4 0.8 0.6 0.2) (0.4 0.8 0.6 0.2) = (0.16 + 0.48 0.32 + 0.16 0.24 + 0.12 0.48 + 0.04) = (0.64 0.48 0.36 0.52)$
So, $_{2} p_{1}^{1} = 0.64$ . Amount = 1. Discount factor = $v^{2} = 0.64$ . PV = $0.64 \cdot 1 \cdot 0.64 = 0.4096$ .

The total APV is the sum: $1 + 0.32 + 0.4096 = 1.7296$ .

Cash Flows Upon Transition (Insurances)

The second type of cash flow is a payment made immediately following a transition from one state to another. The most common example is a death benefit, paid upon transition from the "Alive" state to the "Dead" state.

The logic is the same, but the probability component becomes slightly more complex. The event is now "a transition from state $i$ to state $j$ during the period from $n + k$ to $n + k + 1$ ". By the law of conditional probability, this is:

P (in state i at n + k and transition i \to j) = P (in state i at n + k) \cdot P (transition i \to j ∣ in state i at n + k)

In our notation, this is $_{k} p_{s}^{i} \cdot p_{i}^{j}$ .

The APV formula for a cash flow $_{n + k + 1} C (i, j)$ paid at time $n + k + 1$ for a transition $i \to j$ during the preceding period is:

A P V_{s @ n} (C (i, j)) = k = 0 \sum \infty [_{k} p_{s}^{i} \cdot p_{i, n + k}^{j}] \cdot [_{n + k + 1} C (i, j)] \cdot [_{k + 1} v_{n}]

Note the time subscript on the one-step transition probability $p_{i, n + k}^{j}$ to allow for time-inhomogeneous models, and that the discount factor is for $k + 1$ periods, as the payment is made at the end of the period of transition.

Let's turn back to our reference document for the formal definition and an example.

[PDF] Multi-state transition models for actuarial applications - SOA

Now we will look at the second type of cash flow, which is contingent on a transition between states.

Please read the section "2.2 Cash ﬂows upon transitions". Focus on Notation 2.5 and Theorem 2.7. Then, carefully study Example 2.8 to see this formula in action.

Let's dissect Example 2.8. The setup is the same as before (same transition matrix, starting in State 1, $v = 0.8$ ). The benefit is a cash flow of 1 for a transition from State 2 to State 1 within the next 3 periods.

A transition $2 \to 1$ can happen at time 1 (from $t = 0$ to $t = 1$ ), time 2, or time 3.

Transition during period 1 (t=0 to t=1): The subject starts in State 1. To transition from State 2, they must be in State 2. So, the probability of this transition is 0.
Transition during period 2 (t=1 to t=2): The subject must first be in State 2 at time 1, and then transition to State 1.
- Prob. of being in State 2 at t=1, starting from State 1 at t=0: $_{1} p_{1}^{2} = 0.6$ .
- Prob. of transition $2 \to 1$ : $p_{2}^{1} = 0.8$ .
- Total probability: $0.6 \cdot 0.8 = 0.48$ .
- Amount = 1. Discount factor = $v^{2} = 0.64$ . PV = $0.48 \cdot 1 \cdot 0.64 = 0.3072$ .
Transition during period 3 (t=2 to t=3): The subject must be in State 2 at time 2, and then transition to State 1.
- Prob. of being in State 2 at t=2, starting from State 1 at t=0: $_{2} p_{1}^{2} = 0.36$ (from the (1,2) entry of $Q^{2}$ ).
- Prob. of transition $2 \to 1$ : $p_{2}^{1} = 0.8$ .
- Total probability: $0.36 \cdot 0.8 = 0.288$ .
- Amount = 1. Discount factor = $v^{3} = 0.512$ . PV = $0.288 \cdot 1 \cdot 0.512 = 0.147456$ .

The total APV is the sum: $0 + 0.3072 + 0.147456 = 0.454656$ , which matches the resource's result.

Continuous-Time Formulation

The discrete-time summations we've used are excellent for practical computation. Given your background in probability theory, it's also valuable to see the underlying continuous-time integral formulation, which is conceptually analogous.

In continuous time, the summation becomes an integral, and the one-step transition probability is replaced by the transition intensity (or force of transition), $μ$ .

Annuity (in-state): The APV of an annuity of 1 per year paid continuously while a life aged $x$ is in state $j$ , given they are currently in state $i$ , is: $a ˉ_{x}^{ij} = \int_{0}^{\infty} e^{- δ t} \cdot_{t} p_{x}^{ij} d t$ where $δ$ is the force of interest.
Insurance (on-transition): The APV of a benefit of 1 paid immediately on transition from state $k$ to state $j$ , for a life currently in state $i$ , is found by integrating over all possible transition times: $A ˉ_{x}^{i \to (k \to j)} = \int_{0}^{\infty} e^{- δ t} \cdot_{t} p_{x}^{ik} \cdot μ_{x + t}^{k j} d t$

The following resource provides these formulas, which you may find to be a more mathematically fundamental representation of the same ideas.

[PDF] life contingency models ii - vladislav kargin - Binghamton University

This text by Vladislav Kargin provides the continuous-time analogues to the discrete APV formulas.

Please read the section "2.4. Calculation of Premiums". Focus your attention on the integral formulas for the annuity value and the insurance benefit value. Note how the summations from the previous resource are replaced by integrals here.

Conclusion

In this lesson, we have laid the groundwork for pricing nearly any insurance product based on a multi-state model. By combining probabilities of future events with the time value of money, we can determine a fair value for state-contingent cash flows.

Key Takeaways:

The Actuarial Present Value (APV) is the expected value of future, uncertain, discounted cash flows.
It is calculated using the "triple-product" principle: summing the product of probability, amount, and discount factor over all possible payment events.
Cash flows can be paid while in a state (like an annuity), with an APV of $\sum (_{k} p_{s}^{i}) \cdot (C_{n + k}) \cdot (v^{k})$ .
Cash flows can be paid upon a transition (like an insurance benefit), with an APV of $\sum (_{k} p_{s}^{i} \cdot p_{i}^{j}) \cdot (C_{n + k + 1}) \cdot (v^{k + 1})$ .
The transition probabilities ( $p$ ) are the key link to our previous work on solving Markov models.

In our next lesson, we will directly apply these APV calculations. We will learn how to set a premium for an insurance policy by equating the APV of premiums with the APV of benefits (the Equivalence Principle) and how to calculate the policy reserves an insurer must hold to meet its future obligations.

Recommended resources