Introduction
In our last lesson, we constructed the blueprint for a sophisticated, multi-dimensional satisfaction instrument. By defining the specific facets of satisfaction within the vee triad and designing a measurement approach that mitigates ceiling effects, we established what we need to measure. Now, we turn to the crucial questions of when and how often. An experimental design, particularly a within-subject crossover design like ours, relies on capturing outcome changes with sufficient detail to attribute them to the intervention.
This lesson directly addresses the learning outcome: Specify a measurement schedule (e.g., daily diaries) that balances data granularity against participant reactivity. We will explore Ecological Momentary Assessment (EMA) as a powerful tool for collecting high-density data and then confront the challenges it poses, namely the risk that the act of measurement itself can alter participants' experiences and behaviors. Our goal is to design a concrete measurement protocol for our triadic experiment that maximizes insight while minimizing participant burden and data corruption.
1. The Power of Granularity: Ecological Momentary Assessment (EMA)
To capture the fluid, dynamic nature of satisfaction within the vee triad, we need to move beyond single, retrospective snapshots. The state-of-the-art method for this is Ecological Momentary Assessment (EMA), which involves collecting data from people in their natural environments, in real-time.
Ecological Momentary Assessment to Measure Emotion, Cognition, and Behavior in Daily Life | DIGIT-MI
This presentation from the Eisenberg Family Depression Center provides an excellent introduction to EMA. It clarifies the core principles and explains why this method is so well-suited for studying dynamic psychological phenomena.
Please watch the first two segments of the video: The first part defines EMA by its four key features: real-world data collection, focus on the current state, sampling of moments, and repeated assessments. Note the important distinction made between EMA and a simple end-of-day diary. The second part explains the primary benefits of EMA, highlighting the kinds of research questions it can answer. Pay close attention to the distinction between "within-subject" questions (e.g., when is a person's satisfaction highest?) and "between-subject" questions, as EMA is uniquely powerful for the former.
The principles of EMA are perfectly aligned with our experimental goals. By collecting multiple data points each day from all three participants, we can:
- Minimize Recall Bias: Instead of asking someone at the end of the week how they felt, we ask them in the moment. This is critical for capturing the fleeting emotional responses that define relationship quality.
- Capture Intra-Individual Variability: We can model how each person's satisfaction ebbs and flows throughout the day and across the weeks of our crossover experiment. This allows us to see not just if a living arrangement works better, but how it changes daily life.
- Analyze Temporal Sequences: With high-density data, we can start to untangle cause and effect in the short term. For example, does a reported moment of high autonomy for the pivot (W) precede a rise in her satisfaction scores a few hours later? This temporal data is foundational for the causal modeling we will perform later.
- Examine Context: We can link satisfaction scores to specific contexts. Is satisfaction higher when all three are together, or when dyads are alone? Does this differ across the living arrangement conditions?
Essentially, EMA provides the raw material—a rich, intensive longitudinal dataset—to observe the complex dynamics of the triad as they unfold.
2. The Perils of Measurement: Reactivity and Participant Burden
While frequent measurement is powerful, it is not without cost. The very act of prompting someone to report on their feelings can change those feelings, a phenomenon known as measurement reactivity. This is a significant threat to the validity of our findings.
[PDF] A Mixed-Method Investigation Into Measurement Reactivity to the ...
This paper by Eisele et al. provides a deep, mixed-method analysis of reactivity in EMA studies. It's an excellent resource for understanding the nuances of how participants respond to intensive measurement protocols.
Please read the following sections: The Introduction, which provides an overview of the issue. The section on "Measurement Reactivity in ESM Research," which formally defines the concept and its potential to undermine ecological validity. The review of prior research on changes in the construct itself, such as increased self-awareness. The discussion on "Detected Changes in Response Behavior," which contrasts "habituation" (a benign adaptation) with "fatigue" (a detrimental loss of motivation).
As the paper details, reactivity isn't a single problem but a cluster of related issues:
- Changes in the Underlying Construct: Repeatedly asking participants about their emotional clarity or equity perceptions might make them more self-aware. While this might seem positive, it's a confounding effect—we can't be sure if changes are due to our intervention (the living arrangement) or the measurement itself.
- Habituation vs. Fatigue: Participants will naturally get faster at answering surveys (habituation). This is generally harmless. However, they might also become bored or annoyed, leading them to give less thoughtful, more uniform answers (fatigue). Fatigue reduces data quality and can mask real effects.
- Behavioral Changes: Participants might alter their behavior. For instance, knowing a survey could arrive during a meal, they might avoid contentious topics to avoid having to report negative feelings.
A related concept is participant burden. This refers to the subjective experience of the study being difficult, intrusive, or annoying. High burden can lead to non-adherence (missing surveys) and fatigue. An important insight is that burden doesn't just come from the study design; it also comes from the participant's momentary state.
This distinction, highlighted in the paper "Ecological Momentary Assessment" (LINK), is critical. If a participant reports high burden, is it because our survey schedule is too demanding (a design problem) or because they had a high-stress day at work (a momentary factor)? Differentiating these is key to interpreting our data correctly.
3. Designing Our Schedule: Balancing the "Three Ds"
Our task is to find the sweet spot between data granularity and participant reactivity. A useful framework for this is the "Three Ds" of EMA study design: Density, Depth, and Duration.
Ecological Momentary Assessment to Measure Emotion, Cognition, and Behavior in Daily Life | DIGIT-MI
The presenter in this video offers a clear, practical guide to making these crucial design decisions.
Please watch the section on designing an EMA study, focusing on the "Three Ds". Pay attention to the discussion of different sampling schedules (fixed, random, semi-random, event-contingent) and how the optimal balance depends on the research question, the population, and the analysis plan.
Let's apply this framework to our vee triad experiment:
-
Density (Frequency): How often should we send surveys? Given our interest in moment-to-moment dynamics and variability within a day, we need multiple assessments. Too few (e.g., 1-2 per day) and we can't model dynamics; too many (e.g., 10+ per day) and we risk overwhelming our participants. Research suggests compliance drops more significantly with study length than with daily frequency, so a high-density, shorter-duration approach is often effective.
-
Depth (Length): How long should each survey be? The multi-dimensional instrument we designed in the last lesson is comprehensive but too long to be administered multiple times a day. We need to be strategic.
-
Duration (Length of study): How long does the measurement period last? In our crossover design, the duration of each measurement period is defined by the experimental block (e.g., two weeks in one living arrangement, followed by a washout period, then two weeks in the next).
Proposed Hybrid Measurement Schedule
To balance these factors, we will implement a hybrid measurement schedule that combines different sampling methods. This approach is standard in modern, sophisticated EMA research.
-
Signal-Contingent Surveys (Semi-Random):
- What: A brief survey containing a core subset of the quantitative items from our satisfaction instrument (e.g., one key item per dimension).
- Schedule: 4 prompts per day. The day will be divided into four-hour blocks (e.g., 9am-1pm, 1pm-5pm, 5pm-9pm, 9pm-1am), with one prompt delivered at a random time within each block.
- Rationale: This semi-random or stratified-random approach ensures coverage across the entire waking day without being predictable. It gives us the high-density time-series data needed to analyze variability and temporal dynamics. The brevity (depth) of these surveys minimizes the burden of the high frequency (density).
-
Event-Contingent Surveys:
- What: A short survey initiated by the participant.
- Schedule: Participants will be instructed to complete this survey as soon as possible after a "significant relational event." This would be defined during training as a moment of notable connection, joy, conflict, or negotiation with one or both of their partners.
- Rationale: Time-based sampling can miss the most critical moments. Event-contingent sampling ensures we capture data immediately surrounding the very events that likely drive satisfaction dynamics.
-
End-of-Day Diary Survey:
- What: A slightly longer survey taken once before bed. This is where we will implement the Multistage Testing (MST) design from our previous lesson. It will start with the global satisfaction rating, which routes the participant to the tailored, open-ended questions designed to elicit rich qualitative data.
- Schedule: One prompt delivered at a fixed time each night (e.g., 9 PM), with a long window to complete it (e.g., 3 hours).
- Rationale: This combines the benefits of a classic diary study with our advanced measurement technique. It allows for deeper reflection without interrupting the day. It also serves as the vehicle for collecting our most nuanced data, which is essential for understanding the why behind the quantitative ratings. The concept of a diary study is simple but effective for capturing user habits and behaviors over time.
This hybrid schedule is robust. The signal-contingent surveys provide the quantitative backbone for our time-series analysis. The event-contingent surveys ensure we don't miss key causal triggers. And the end-of-day diary delivers the rich qualitative insight needed to understand the complex mechanisms at play.
4. Practical Implementation and Burden Mitigation
A well-designed schedule is only effective if participants can and will follow it. Ensuring high compliance requires careful planning and support.
Ecological Momentary Assessment - PMC - NIH
This paper provides a concrete example of an EMA study design and discusses the results in terms of burden and adherence.
Please review the following sections: The "EMA Design" section, which gives a real-world example of a hybrid schedule combining signal-, event-, and end-of-day contingent surveys. The "Overview" of the Discussion, which summarizes the key finding: momentary stressors, not the design itself, were the primary drivers of burden. The section on "Momentary and Study-Related Stress and Burden," which reinforces the value of asking participants directly about burden to distinguish state vs. trait patterns.
Drawing from the research, here are the essential steps for successful implementation:
- Thorough Participant Training: Before the experiment begins, we must conduct a detailed onboarding session. All three participants should install the app, walk through each survey type, and practice initiating an event-contingent survey. As noted in the "Ecological Momentary Assessment" video (LINK), this "hand-holding" at the start significantly improves compliance.
- Individualize the Schedule: While the sampling blocks are fixed, we can and should work with participants to customize the overall window. The paper by Eisele et al. (LINK) notes that a common source of burden is prompts arriving during sleep. We should establish each person's typical waking hours and ensure prompts are only sent then.
- Establish Clear Communication: Participants need a simple way to report technical problems. We will implement daily automated checks of the data stream. If a participant misses several consecutive surveys, a research assistant will reach out non-judgmentally to troubleshoot, differentiating tech issues from non-compliance.
- Compensation Structure: As a practical matter, compensation drives compliance. We will use a model that rewards both participation and completion: a base payment for completing the study, plus a per-survey bonus. A final bonus for achieving a high completion rate (e.g., >80%) can be a powerful motivator.
Conclusion
In this lesson, we have designed a comprehensive measurement schedule for our N-of-1 experiment. We've moved from the abstract need for data to a concrete, multi-faceted protocol that balances the scientific requirement for granular data with the human reality of participant reactivity and burden.
Key Takeaways:
- EMA is the Right Tool: Ecological Momentary Assessment provides the high-density, in-the-moment data necessary to model the dynamic changes in satisfaction within our crossover design.
- Reactivity is a Real Threat: We must be aware that the act of measurement can change participant behavior and experience through mechanisms like fatigue, habituation, and increased self-awareness.
- Balance the "Three Ds": A successful schedule finds a sustainable equilibrium between Density (frequency), Depth (length), and Duration (time).
- A Hybrid Schedule is Optimal: Our proposed schedule combines semi-random signal-contingent surveys for quantitative time-series data, event-contingent surveys to capture critical moments, and an end-of-day diary for rich qualitative reflection.
- Implementation Matters: Success depends on practical steps like thorough training, schedule customization, and clear communication channels to support participants and mitigate burden.
We now have a plan for what to measure and when to measure it. The next challenge arises from the nature of our system. Because the three individuals in the vee are interconnected, their satisfaction outcomes will be statistically correlated. In our next lesson, we will develop a strategy to handle these statistically correlated outcomes, a critical step before we can move to modeling and analysis.