The Problem of Priors: Subjectivism, Objectivism, and Indifference

The Great Divide: Subjectivity vs. Objectivity

The debate over priors mirrors a more fundamental disagreement in epistemology about the nature of knowledge itself.

This divide is central to the problem of the priors.

• Subjective Bayesians argue that prior probabilities represent an agent's personal, subjective degrees of belief. The only constraint is that they must be coherent (i.e., follow the probability axioms). Beyond that, any prior is rationally permissible.
• Objective Bayesians argue that there are further rational constraints. In a state of ignorance, our priors should be uniquely determined by principles of neutrality or "freedom from bias." Our beliefs should reflect the objective state of our evidence, not just our personal whims.

Let's begin by reading a brief, formal introduction to this problem from the Stanford Encyclopedia of Philosophy.

The Subjectivist Position: Rationality as Coherence

The subjectivist, or "personalist," view is perhaps the most minimalist. It claims that rationality is about the structure of your beliefs and how they change, not their initial content. As long as your belief system is internally consistent and you update via conditionalization, you are rational.

The following video explains this stance and its implications.

As the video explains, a significant objection to subjectivism is that it seems to permit too much, allowing wildly different or even outlandish priors to be considered "rational." If a scientist starts with a very high prior in a fringe theory and another starts with a very low one, subjectivism deems them both rational.

The primary defense against this is the convergence-of-opinion theorem, also known as the "washing out of the priors." The idea is that as two agents with different priors are exposed to the same stream of evidence, their posterior beliefs will inevitably converge. In the long run, the evidence will overwhelm the initial subjective differences.

We will not delve into the mathematics of this theorem today, but it's important to grasp its philosophical role. It allows the subjectivist to reclaim a form of objectivity, not by constraining priors, but by relying on the power of evidence to forge consensus over time.

The Objectivist Position: The Principle of Indifference

Objective Bayesians find the subjectivist position unsatisfactory. They argue that a theory of rationality should have more to say about what to believe in a state of ignorance. Their primary tool for this is the Principle of Indifference.

The Principle of Indifference (PI): In the absence of any evidence or reason to favor one possibility over another, we should assign equal probability to each.

For a six-sided die, if we have no reason to believe it's biased, the PI dictates we assign a probability of $1/6$ to each face. A subjectivist would say assigning $1/2$ to '6' and $1/10$ to all other faces is perfectly rational, as long as it's coherent. An objectivist would call this irrational, as it doesn't reflect our state of ignorance.

The video below introduces this idea and its immediate, powerful challenge.

Bertrand's Paradox: The Trouble with Indifference

The Principle of Indifference works beautifully for discrete, symmetrical problems like coin flips and dice rolls. However, it breaks down dramatically when applied to continuous domains, leading to what is known as Bertrand's Paradox.

The paradox demonstrates that the way we describe or parameterize a problem of ignorance changes the outcome of applying the PI, leading to contradictory results. The following video provides a brilliant visualization of the classic version of this paradox.

Bertrand's Paradox poses a formidable challenge to Objective Bayesianism. If there's no unique, non-arbitrary way to apply the Principle of Indifference in many real-world scenarios (which often involve continuous variables), then the objective project of finding a single "correct" prior to represent ignorance seems doomed.

Objective Bayesians have developed sophisticated responses, such as appealing to invariance principles (as mentioned briefly at the end of the video with Edwin Jaynes's work) to find the "natural" parameterization. However, the debate remains fierce.

Deeper Justifications and a Return to Epistemology

Given the challenges, some philosophers have tried to provide deeper justifications for the Principle of Indifference, moving beyond simple intuition. This brings us back to your core interest: the connection between probability and epistemology.

A compelling recent analysis distinguishes between two kinds of justification for norms like the PI:

1. Pragmatic Justification: We should follow the rule because it leads to better outcomes (e.g., minimizing worst-case expected loss). This is the same spirit as the Dutch Book arguments.
2. Epistemic Justification: We should follow the rule because it is the most truth-conducive or accurate representation of our doxastic state.

The paper below explores this distinction and argues that while a pragmatic argument for the PI might work, an epistemic one runs into a conflict with a plausible epistemological thesis called Evidentialism.

The clash arises because Evidentialism holds that a belief is only epistemically rational if it is compatible with one's evidence. The Principle of Indifference seems to go beyond the evidence by mandating a specific belief distribution (e.g., uniform) where the evidence is silent. This suggests that the justification for using such "uninformative" priors might be better understood as a pragmatic strategy for decision-making under uncertainty, rather than a uniquely accurate representation of what the evidence tells us.

Conclusion

Today, we've journeyed to the heart of a foundational debate in Bayesianism. The seemingly simple question of "Where do priors come from?" forces us to confront deep philosophical questions about rationality, subjectivity, and objectivity.

Key Takeaways:

• The Problem of the Priors is the debate over what rational constraints, if any, apply to our initial belief states.
• Subjective Bayesians champion minimal constraints, requiring only probabilistic coherence. They argue that personal judgments are permissible starting points, relying on the long-run power of evidence to forge consensus.
• Objective Bayesians argue for stronger constraints to codify epistemic neutrality. Their main tool, the Principle of Indifference, mandates equal probabilities for possibilities among which our evidence does not discriminate.
• The Principle of Indifference faces a severe challenge from Bertrand's Paradox, which shows that applying the principle to continuous spaces can yield contradictory results depending on how the problem is framed.
• This has led to more sophisticated justifications for priors, distinguishing between pragmatic (strategic) and epistemic (accuracy-based) rationales, further intertwining probability theory with epistemology.

In our next lesson, we will take a closer look at the subjectivist's most powerful rejoinder: the convergence-of-opinion theorem. We will evaluate just how strong the "washing out" of priors is and what it implies for the ideal of scientific objectivity.