Bias and Smoothness Tests

Introduction

In our previous lesson, we established the first line of defense in evaluating a mortality graduation: testing for adherence to data using the Chi-squared and Standardised Deviations tests. We saw that these tests provide a powerful overall measure of fit but have a key weakness: they can miss systematic patterns in the deviations. A graduation might produce small errors on average, but if those errors are consistently positive in one age range and negative in another, it indicates a fundamental flaw that our previous tests might overlook.

This lesson addresses that gap directly. We will now explore a suite of diagnostic tests designed specifically to detect non-randomness and bias in the standardised deviations ( $z_{x}$ ). You will learn to apply the Signs Test, the Grouping of Signs Test, and the Serial Correlations Test. As a statistician, you'll find the principles behind these tests very familiar; our focus will be on their specific application in the actuarial context of assessing a graduation. By the end of this lesson, you will have a complete toolkit for rigorously evaluating both the adherence and smoothness of a graduated mortality table.

Detecting Overall Bias: The Signs Test

The most straightforward type of pattern to check for is an overall bias. Are our graduated rates systematically too high or too low across the board? The Signs Test provides a simple and effective way to answer this.

The logic is intuitive: if the graduation is unbiased, a standardised deviation $z_{x}$ is just as likely to be positive as it is negative.

Null Hypothesis ( $H_{0}$ ): The graduation has no overall bias. In statistical terms, the probability of a positive deviation is 0.5, i.e., $P (z_{x} > 0) = 0.5$ .
Test Statistic: Let $m$ be the number of age groups. The test statistic, $P$ , is simply the count of positive $z_{x}$ values.
Distribution under $H_{0}$ : Under the null hypothesis, $P$ follows a Binomial distribution, $P \sim Binomial (m, 0.5)$ . We perform a two-tailed test, checking if the observed count of positive (or negative) signs is too extreme to have occurred by chance.

For a large number of age groups $m$ , we can use a Normal approximation, where $P \sim N (\frac{m}{2}, \frac{m}{4})$ .

Let's examine a practical application of this test from an actuarial exam paper.

[PDF] Subject CS2 Paper A - Institute of Actuaries of India

This document from the Institute of Actuaries of India provides a worked solution that includes the Signs Test. The context involves rates of "getting married," but the statistical procedure is identical to what we would use for mortality rates.

Please read the section titled "Signs Test". Focus on how the simple principles we've discussed are put into practice: The statement of the purpose of the test. The identification of the null distribution (Binomial with m=21 and p=0.5). The comparison of the observed counts to what would be expected. The conclusion drawn from the test.

Detecting Clustering: The Grouping of Signs Test

The Signs Test is useful, but it wouldn't detect a pattern like +++++++++----------, where there are roughly equal numbers of positive and negative deviations. The deviations are balanced overall, but their arrangement is clearly not random. This "clumping" or "clustering" of signs suggests that the graduated curve is "waving" through the crude data, failing to capture its true shape.

The Grouping of Signs Test (also known as the Runs Test) is designed to detect this specific issue.

Null Hypothesis ( $H_{0}$ ): The signs of the deviations are randomly ordered.
Test Statistic: The test statistic, $G$ , is the number of groups of consecutive identical signs. For the pattern + + - - - + - -, the groups are (++), (---), (+), (--), so $G = 4$ .
Decision Rule: Too few groups suggest clustering. Therefore, this is a lower-tailed test. We reject $H_{0}$ if $G$ is below a certain critical value.
Distribution under $H_{0}$ : The exact distribution is based on combinatorics and can be complex. For practical purposes, a Normal approximation is often used.

The formula sheet below provides the relevant expressions for this test.

[PDF] Exam CS2 Formula & Review Sheet - ACTEX Learning

This formula sheet from ACTEX Learning contains the necessary formulas for the tests in this lesson.

Locate the section for the "Grouping of Signs Test". You will see the definition of the test statistic G and its approximate Normal distribution. Here, n_1 is the number of positive signs and n_2 is the number of negative signs. Notice that both the mean and variance of this approximation depend only on the counts of positive and negative signs.

For example, if we have 21 age groups with 8 positive and 13 negative deviations (as in the IAI paper), the expected number of groups would be:
$μ_{G} = \frac{2 n _{1} n _{2}}{n _{1} + n _{2}} + 1 = \frac{2 \times 8 \times 13}{21} + 1 \approx 10.9$
If we observed a very small number of groups, say $G = 3$ , this would be strong evidence of clustering and a poor graduation, even if the Signs Test was passed.

Detecting General Patterns: The Serial Correlations Test

The most statistically rigorous test for non-randomness is the Serial Correlations Test. This test generalizes the idea of looking for patterns by directly measuring the correlation between a deviation at one age and a deviation at a subsequent age. Your background in time series analysis will be very helpful here, as this is conceptually identical to calculating the sample autocorrelation function (ACF) of the sequence of standardised deviations.

We are primarily concerned with positive serial correlation, where a positive $z_{x}$ tends to be followed by a positive $z_{x + 1}$ , and a negative by a negative. This is the hallmark of a "smooth" graduated curve failing to follow the more jagged, but real, patterns in the crude data.

Serial correlation testing - introduction

Before looking at the specific actuarial formula, let's watch this video from Ben Lambert that introduces the general concept of testing for serial correlation in model residuals. The z_x values are our residuals.

First, watch the brief introduction to understand why we test for serial correlation. The reasoning—that our estimator is no longer optimal—applies directly to our graduation. Next, watch the segment from the main explanation. The key idea presented is regressing residuals on their own lagged values and testing the significance of the coefficient. This provides the core intuition for what we are about to do, even though our specific test formula is slightly different.

Now, let's look at the specific test used in actuarial science.

Null Hypothesis ( $H_{0}$ ): The standardised deviations are serially uncorrelated.
Test Statistic: The serial correlation coefficient of lag $j$ , denoted $r_{j}$ . The most common test uses lag-1 correlation ( $j = 1$ ). The formula is:
$r_{j} = \frac{\sum _{i = 1}^{m - j} ( z _{i} - z ˉ ) ( z _{i + j} - z ˉ )}{\sum _{i = 1}^{m} ( z _{i} - z ˉ ) ^{2}}$
In practice, since the standardised deviations $z_{i}$ are expected to have a mean of 0, we often assume $z ˉ = 0$ to simplify the calculation.
Distribution under $H_{0}$ : For a reasonably large number of age groups $m$ , the distribution of the test statistic $r_{j} m$ can be approximated by a standard normal distribution, $N (0, 1)$ .
Decision Rule: Since we are primarily concerned with clustering (++ or --), we are looking for significantly positive serial correlation. Therefore, we typically perform an upper-tailed test on $r_{1}$ .

Let's again consult our formula sheet for the formal definition.

[PDF] Exam CS2 Formula & Review Sheet - ACTEX Learning

This is the same formula sheet we viewed earlier.

Now, locate the section for the "Serial Correlations Test". You will find two formulas for the test statistic r_j. The first is the more precise definition, while the second is a common approximation. Also, note the final line, which gives the test to perform against the standard normal distribution.

Let's solidify this with a quick calculation. Suppose for $m = 50$ ages, we calculate the lag-1 serial correlation coefficient to be $r_{1} = 0.3$ .

Calculate the test statistic: $r_{1} m = 0.3 \times 50 \approx 0.3 \times 7.07 = 2.12$ .
Compare to critical value: The 95% critical value for a one-tailed $N (0, 1)$ test is 1.645.
Conclusion: Since $2.12 > 1.645$ , we reject the null hypothesis of no serial correlation. There is significant evidence of clustering in the deviations, suggesting a poor graduation.

Conclusion

In this lesson, we have completed our toolkit for assessing the quality of a mortality graduation. We moved beyond simply checking the magnitude of errors to diagnosing the patterns within them.

Key Takeaways:

A Comprehensive Assessment: A good graduation must not only adhere to the data (checked by the $χ^{2}$ test from the last lesson) but must also produce deviations that are free from systematic patterns.
The Signs Test: This is your first check for bias. It tests whether the graduation is systematically over- or under-stating the rates by checking for a balance between positive and negative deviations using a Binomial test.
The Grouping of Signs Test: This test looks for clustering (e.g., +++---). Too few "groups" or "runs" of signs indicates that the graduated curve may be inappropriately smooth or "waving" through the data.
The Serial Correlations Test: This is the most general test for non-randomness. It measures the correlation between adjacent deviations, akin to an ACF plot in time series analysis. Significant positive correlation is a strong red flag indicating a poor fit.

Together, the tests from this lesson and the previous one form a suite of diagnostics that must be applied to any proposed graduation before it can be deemed acceptable.

Preview of the Next Lesson

With this lesson, we conclude our study of graduation, which focuses on creating a single, static mortality table that best represents current experience. However, for long-term forecasting, pension funding, and annuity pricing, a static table is insufficient. We need to understand how mortality will change over time. In our next module, we will transition from static to dynamic models, beginning with the seminal Lee-Carter mortality projection model. You will see how your knowledge of time series analysis becomes directly applicable to forecasting future mortality rates.

Recommended resources