Understanding Sampling Schemes and Their Analytical Implications

1. The Statistical Foundation: Probability Distributions for Counts

Before we discuss study designs, let's briefly cover the probability distribution that models the counts in a contingency table's cells: the multinomial distribution.

You are already familiar with the binomial distribution, which models the number of "successes" in a fixed number of trials for an outcome with only two possibilities (e.g., heads/tails). The multinomial distribution is simply a generalization of this to situations with more than two possible outcomes.

For example, if we survey people about their blood type (O, A, B, AB), the counts we get in each category can be modeled by a multinomial distribution.

To understand the assumptions and mechanics of this distribution, please watch the following video.

As you saw, the key conditions are:

1. A fixed number of trials, $n$.
2. The trials are independent.
3. Each trial results in one of $k$ mutually exclusive and exhaustive outcomes.
4. The probability of each outcome remains constant across trials.

Now, let's see how these principles apply to real-world study designs.

2. From Theory to Practice: Sampling Schemes and Study Designs

The way we sample subjects and collect data determines which totals in a contingency table are fixed by design and which are left to chance. This distinction is what separates the main sampling schemes.

Consider the following contingency table. In the last lesson, you learned to interpret its percentages. Now, we ask a new question: how were these 600 students sampled? The answer determines what we can infer.

Design 1: Cross-Sectional Study (Multinomial Sampling)

• How it works: You take a single random sample of a fixed size, $n$, from a population and then classify each individual on two (or more) categorical variables simultaneously.
• Example: We randomly select $n=600$ students from a university and record both their gender and their major.
• Fixed vs. Random: In this design, only the grand total ($n=600$) is fixed by the researcher. All cell counts and all marginal totals (the number of men, women, humanities majors, etc.) are random variables.
• Statistical Model: The counts in all the cells of the table follow a single multinomial distribution.
• Implications for Analysis: Because you have a representative snapshot of the entire population, you can estimate any probability of interest: joint, marginal, or conditional. All common measures of association—difference in proportions, relative risk, and odds ratio—are valid and can be calculated.

Design 2: Prospective and Experimental Studies (Product-Multinomial Sampling)

This category includes cohort studies and most randomized clinical trials.

• How it works: You first identify groups based on an exposure or treatment variable and select a fixed number of subjects for each group. Then, you follow them forward in time ("prospectively") to observe an outcome.
• Example (Cohort Study): We decide to sample a fixed number of students: 200 from Humanities, 200 from Natural Sciences, and 200 from Social Sciences. We then observe the gender distribution within each of these fixed-size groups.
• Fixed vs. Random: In this design, the row totals (or column totals, depending on how you arrange the table) are fixed by the researcher. The cell counts and the other set of marginal totals are random.
• Statistical Model: You don't have one single multinomial distribution. Instead, each row is its own independent multinomial distribution. The overall model is the product of these independent distributions, hence the name product-multinomial. In a 2x2 table, this simplifies to a product of two binomial distributions.
• Implications for Analysis: You can calculate the probability of the outcome conditional on the exposure group. For our example, we could calculate P(Gender | Major). However, you cannot calculate the overall marginal probability of being a certain gender, because you artificially fixed the number of students from each major. Despite this, you can still validly calculate the difference in proportions, relative risk, and odds ratio.

Design 3: Retrospective Study (Product-Multinomial Sampling)

This is the classic case-control study design.

• How it works: You identify groups based on the outcome variable (e.g., disease status) and select a fixed number of "cases" (with the outcome) and "controls" (without the outcome). Then, you look backward in time ("retrospectively") to check their history for an exposure.
• Example: We want to study the link between smoking and a rare cancer. It would be inefficient to follow 10,000 people for 20 years. Instead, we find 100 people with the cancer (cases) and 100 similar people without it (controls). We then ask them if they were smokers.
• Fixed vs. Random: This is also product-multinomial sampling. The column totals (cases and controls) are fixed by the researcher.
• Implications for Analysis: This is where the implications become most critical.
  • You can estimate the probability of exposure conditional on the outcome, e.g., P(Smoker | Cancer).
  • You CANNOT estimate the risk of getting the disease, P(Cancer | Smoker). Why? Because you artificially fixed the proportion of people with cancer in your sample (at 50% in this example), which is not its true rate in the population.
  • Therefore, you cannot calculate the Difference in Proportions or the Relative Risk. This is a classic failure case—applying these measures here would produce meaningless results.
  • The Odds Ratio: Amazingly, the odds ratio has a unique mathematical property of invariance. The odds ratio of disease given exposure is identical to the odds ratio of exposure given disease. This makes the odds ratio the only valid measure of association in a case-control study.

This is a critical piece of functional knowledge. The choice of study design directly restricts your analytical options.

3. Summary of Sampling Schemes and Their Implications

To tie everything together, here is a summary of the three main designs and their implications for analyzing a 2x2 table.

To reinforce these distinctions, the following resource provides a concise, slide-based summary of the three designs.

A final note: while the estimation of association measures is highly dependent on the sampling scheme, the most common test of independence (the Pearson Chi-Square test, which we'll see soon) is valid under all three sampling schemes. This is because the null hypothesis of independence can be tested by comparing observed counts to expected counts, a procedure that holds true whether the margins are fixed or not.

Conclusion

In this lesson, we explored how the method of data collection fundamentally shapes the analysis of categorical data. You've learned to distinguish between the primary sampling schemes and, most importantly, to understand their consequences for analysis.

Key Takeaways:

• Counts in a contingency table are often modeled by the multinomial or product-multinomial distributions.
• Cross-sectional studies fix only the grand total, follow a multinomial model, and allow for the calculation of all common measures of association (difference in proportions, relative risk, and odds ratio).
• Prospective (cohort) studies fix the exposure group totals, follow a product-multinomial model, and also allow for all three measures.
• Retrospective (case-control) studies fix the outcome group totals and also follow a product-multinomial model. This design has a critical limitation: because the disease prevalence is artificially fixed, you cannot calculate the difference in proportions or relative risk.
• The odds ratio is a uniquely robust measure that can be validly estimated from all three study designs, making it essential for case-control studies.

Preview of the Next Lesson:

Now that we understand how sampling influences our choice of metrics, we can begin our deep dive into the measures of association themselves. In the next lesson, we will formally define, compute, and interpret the most straightforward measure for a 2x2 table: the difference in proportions.