Modeling Photoshoot Energy Transitions

1. The Markov Property and the Transition Matrix

A Markov Chain is a model for a system that transitions between a set of states over time. Let's imagine a boudoir photoshoot. The model's creative energy is a key resource that fluctuates. We can define a set of discrete energy states:

• State 0: High Energy (creative, engaged, spontaneous)
• State 1: Moderate Energy (focused, professional, executing planned poses)
• State 2: Low Energy (fatigued, needing a break, losing focus)

If we observe the model's state at regular intervals (say, every 15 minutes), we get a sequence of states. A Markov model simplifies the dynamics of this sequence with a powerful assumption: the Markov Property.

This property asserts that the probability of transitioning to the next state depends only on the current state, and not on the sequence of states that preceded it. In our scenario, this means the model's energy level in the next 15 minutes depends only on her current energy, not on whether she was full of energy for the past hour or just recovered from a dip.

The following short video introduces this core concept and how the transition probabilities are organized into a transition matrix.

The transition matrix, typically denoted by $P$, is a square matrix where the entry $P_{ij}$ is the probability of moving from state $i$ to state $j$ in one time step.

Here, $P_{12}$ would be the probability that a model in a "Moderate Energy" state transitions to a "Low Energy" state in the next 15 minutes.

2. Validating the Markov Assumption

The Markov property is a simplifying assumption. But is it a valid one? Perhaps the model's energy level has "momentum" or a "boom-bust" cycle, where a long period of high energy makes a crash more likely. This would violate the memoryless property. As a statistician, you know that assumptions must be tested.

We can frame this as a formal hypothesis test:

• Null Hypothesis ($H_0$): The process is a first-order Markov chain. The state at time $t+1$ depends only on the state at time $t$.
• Alternative Hypothesis ($H_1$): The process is a second-order Markov chain. The state at time $t+1$ depends on the states at both time $t$ and $t-1$.

If we fail to reject $H_0$, we can proceed with the simpler first-order model. The standard method for this is a likelihood-ratio test, which in this context is often implemented as a chi-squared test. We compare how well a first-order model explains the data versus a second-order model.

A highly relevant discussion on this topic can be found on the statistics forum Stack Exchange.

The core idea is to test for conditional independence: Is the future state independent of the past state, given the present state?
$P(X_{t+1}=m|X_t=j,X_{t-1}=i)\stackrel{?}{=}P(X_{t+1}=m|X_t=j)$
To conduct this test formally, we can use the methodology developed by Anderson and Goodman (1957), which uses a chi-squared statistic. A detailed guide to this test is provided in the following resource.

3. Estimating the Transition Matrix

If our hypothesis test supports the first-order Markov assumption, our next step is to estimate the transition matrix $P$. The most common method for this is Maximum Likelihood Estimation (MLE). Given your background, you'll recognize the principle: we find the parameter values (the transition probabilities) that make our observed data most probable.

For a Markov chain, the maximum likelihood estimator for the transition probability $P_{ij}$ has a very intuitive form. It is simply the observed frequency of transitions from state $i$ to state $j$.

Let $n_{ij}$ be the number of times we observed a transition from state $i$ to state $j$ in our data. Let $n_i={\sum }_k{n}_{ik}$ be the total number of times the system was in state $i$. Then the MLE is:

Example: Photoshoot Energy Data

Suppose we observed a model's energy state every 15 minutes over a portion of a shoot and recorded the following sequence of 16 states:
[0, 0, 1, 1, 0, 1, 2, 1, 2, 2, 1, 0, 1, 1, 2, 2]

First, we create a transition count matrix:

Note: There are 15 transitions in a sequence of 16 states. The final '2' in state 2 did not transition anywhere in the observed sequence.

Now, we normalize each row by its total to get the estimated transition probability matrix $\hat{P}$:

This matrix now serves as our model of the energy dynamics. For example, it predicts that from a "High Energy" state, there is a 75% chance of moving to "Moderate Energy" and a 25% chance of staying in "High Energy" in the next 15 minutes.

For a formal treatment of the MLE procedure, you can review the following resource.

4. An Extension: State-Dependent vs. State-Independent Models

The resource LINK also introduces a more subtle question we can ask about our model. In our estimated matrix $\hat{P}$, the probability of deteriorating (e.g., transitioning to a worse energy state) is different depending on the current state. For example, $P_{01}=0.75$ while $P_{12}\approx 0.43$. This is a state-dependent model.

We could hypothesize a simpler model where the probability of dropping one energy level is the same, regardless of the current state (a state-independent model). This is another testable hypothesis. The generalized likelihood-ratio test provides the framework for comparing these two nested models.

This type of model comparison is a fundamental part of the statistical modeling process, allowing us to select a model that is just complex enough to capture the dynamics in the data, but no more.

Conclusion

In this lesson, we established the foundational steps for building a discrete-time Markov Chain model from observational data. We saw that building such a model is not just a matter of calculation, but also requires rigorous statistical validation.

Key Takeaways:

• The Markov Property assumes that a process is "memoryless," with future state probabilities depending only on the present state.
• This assumption is not a given; it must be validated using data. A common method is a chi-squared or likelihood-ratio test comparing a first-order model (memoryless) against a second-order model (with memory).
• If the Markov assumption holds, the transition probabilities can be estimated using Maximum Likelihood Estimation (MLE), which simplifies to calculating the observed frequencies of each transition.
• The resulting transition matrix provides a compact, quantitative model of the system's one-step dynamics.

In our next lesson, we will address a crucial question: What do we do if the Markov assumption is violated? We will compare the suitability of Markov Chains versus Renewal Theory for modeling cyclical phenomena, exploring how Renewal Theory provides a more general framework for processes that do possess memory.