Sign Test with Binomial Distribution

From Preferences to a Test Statistic

Let's recall the procedure from our last lesson. When comparing two styles, say a 'Push-up' bra versus a 'Wireless' bra, we record the model's preference for each pairing:

• A plus sign (+) if she prefers the 'Push-up'.
• A minus sign (-) if she prefers the 'Wireless'.
• A zero (0) if there's no preference (a tie).

We discard any ties and let $n$ be the total number of non-zero preferences. We then count the number of plus signs, $n_{+}$, and minus signs, $n_{-}$. The test statistic is simply one of these counts.

The core logic of the sign test rests on the null hypothesis, $H_0:p=0.5$. If there's truly no preference, then each choice is like a fair coin flip. The number of times the model chooses the 'Push-up' bra (our $n_{+}$) in $n$ trials should therefore follow a binomial distribution.

Your background in statistics makes you very familiar with the properties of the binomial distribution, $B(n,p)$. For the sign test, the parameters are always the number of trials ($n$) and the probability of success under the null hypothesis ($p=0.5$).

Calculating the p-value with the Binomial Distribution

The p-value is the probability of observing a result at least as extreme as your data, assuming the null hypothesis is true. To calculate it, we use the binomial probability formula for a distribution with parameters $n$ and $p=0.5$.

The probability of observing exactly $k$ successes in $n$ trials is:

The p-value is the sum of probabilities for all outcomes that are "as or more extreme" than what you observed. The exact calculation depends on your alternative hypothesis.

Let's work through an example. Suppose you test a 'Push-up' bra (Style A) against a 'Wireless' bra (Style B) in $n=6$ different outfits. The model prefers the 'Push-up' in 5 of the 6 trials ($k=5$). You want to test if the 'Push-up' is generally preferred.

• Hypotheses: $H_0:p=0.5$ vs. $H_1:p>0.5$ (one-sided test)
• Observed outcome: 5 preferences for Style A out of 6 trials.
• "As or more extreme" outcomes: This means getting 5 preferences or even more (i.e., 6 preferences).
• p-value: $P(X\ge 5)=P(X=5)+P(X=6)$

In the past, these probabilities were looked up in tables. While we now use software, a table can still be illustrative.

To find our p-value using the table above:

1. Find the block for $n=6$.
2. Go to the column for $p=0.50$.
3. Read the probabilities for $r=5$ and $r=6$.
   • $P(X=5)$ is listed as 0.094.
   • $P(X=6)$ is listed as 0.016.
4. Sum them to get the p-value: $$\text{p-value}=0.094+0.016=0.110$$

Decision: Our p-value is 0.110. Using a standard significance level ($\alpha =0.05$), we find that $0.110>0.05$. Therefore, we fail to reject the null hypothesis. Even though the 'Push-up' was preferred in 5 of 6 trials, with such a small sample size, this result is not statistically significant. We don't have enough evidence to conclude that the 'Push-up' is genuinely superior.

Handling Different Alternative Hypotheses

The calculation changes slightly for different $H_1$. Let's say you observed $k=5$ preferences for Style A in $n=6$ trials.

• For $H_1:p>0.5$ (Is Style A preferred?):

  • p-value = $P(X\ge 5)=P(5)+P(6)$
  • You are looking at the upper tail of the distribution.

• For $H_1:p<0.5$ (Is Style A less preferred?):

  • This hypothesis doesn't fit our data (5/6 is not "less preferred"). But if we had observed $k=1$, the p-value would be:
  • p-value = $P(X\le 1)=P(0)+P(1)$
  • You would be looking at the lower tail.

• For $H_1:p\ne 0.5$ (Is there any preference?):

  • This is a two-sided test. We consider extreme outcomes in both tails. With an outcome of 5 (and therefore 1), the result is $5-1=4$ units away from an even 3-3 split. An equally extreme result in the other direction would be 1 (and therefore 5).
  • p-value = $P(X\ge 5)+P(X\le 1)$.
  • Since the binomial distribution with $p=0.5$ is symmetric, this simplifies to $2\times P(X\ge 5)$.
  • p-value = $2\times (0.094+0.016)=2\times 0.110=0.220$.

Conducting the Test with Modern Tools

Manually calculating binomial probabilities or using tables is cumbersome. In practice, you will use statistical software or online calculators. These tools execute the exact same logic instantly.

Notice how tools like this simplify the process. You provide the number of successes, the number of trials, and the alternative hypothesis, and the software returns the p-value and a conclusion. The online calculator from MeasuringU, which we saw referenced in the context of UX design, works on the same principle, though it is typically used to compare against different chance-proportions.

Numerical Exercise

You are in the final stages of optimizing a look for the photoshoot. You are comparing a new 'Minimalist Bralette' (Style M) against a classic 'Lace Balconette' (Style L). You want to determine if there is any significant preference between them, in either direction.

You conduct 14 pairwise tryouts with different outfits. The 'Minimalist Bralette' is preferred 10 times, and the 'Lace Balconette' is preferred 4 times. There were no ties.

Your Task:

1. State the null ($H_0$) and alternative ($H_1$) hypotheses for this scenario.
2. Identify the number of trials ($n$) and the number of "successes" ($k$, let's define this as preferring the Bralette).
3. Calculate the two-sided p-value. You will need to use the binomial formula, as $n=14$ is not in our table. Recall that for a two-sided test, the p-value is $2\times P(X\ge k)$ for $k$ in the upper tail. The probabilities for individual outcomes are:
   • $P(X=10)\approx 0.0611$
   • $P(X=11)\approx 0.0222$
   • $P(X=12)\approx 0.0056$
   • $P(X=13)\approx 0.0009$
   • $P(X=14)\approx 0.0001$
4. State your conclusion at a significance level of $\alpha =0.05$.

Solution:

1. Hypotheses:

   • $H_0:p=0.5$ (There is no preference between the Bralette and the Balconette).
   • $H_1:p\ne 0.5$ (There is a significant preference for one of the styles).

2. Trials and Successes:

   • Number of trials, $n=14$.
   • Number of successes (preferring the Bralette), $k=10$.

3. p-value Calculation:
   This is a two-sided test. Since we observed 10 successes (which is in the upper half), we calculate the probability of an outcome this extreme or more so in the upper tail, and then double it.

   • $P(X\ge 10)=P(X=10)+P(X=11)+P(X=12)+P(X=13)+P(X=14)$
   • $P(X\ge 10)\approx 0.0611+0.0222+0.0056+0.0009+0.0001=0.0899$
   • The two-sided p-value is $2\times P(X\ge 10)\approx 2\times 0.0899=0.1798$.

4. Conclusion:
   The p-value is approximately 0.18. Since $0.18>0.05$, we fail to reject the null hypothesis. Despite the Bralette being preferred in 10 out of 14 trials, this result is not strong enough to rule out the possibility that it occurred by random chance. We do not have sufficient statistical evidence to conclude there is a significant preference for either style.

Conclusion

You have now mastered the complete sign test procedure. You can move from qualitative preference data to a rigorous, quantitative conclusion about statistical significance.

Key Takeaways:

• The sign test uses the binomial distribution with $p=0.5$ to model preference counts under the null hypothesis of "no preference."
• The p-value represents the probability of observing a result as or more extreme than your data, assuming $H_0$ is true.
• The calculation of the p-value depends on the alternative hypothesis (one-sided vs. two-sided).
• A p-value below your chosen significance level (e.g., 0.05) allows you to reject the null hypothesis and conclude that a genuine preference exists.
• Even seemingly strong preferences (like 5 out of 6) may not be statistically significant with small sample sizes, highlighting the importance of formal testing.

In our next lesson, we will build on this foundation to tackle a more complex scenario. What happens when you have more than two styles to compare? You'll learn how to apply a series of sign tests to a record of pairwise preferences among three or more lingerie styles to determine if an overall winner emerges.