Using a complexity science approach to evaluate the effectiveness of just-in-time adaptive interventions: A meta-analysis

Objective: Just-in-time adaptive interventions (JITAIs), which allow individuals to receive the right amount of tailored support at the right time and place, hold enormous potential for promoting behavior change. However, research on JITAIs’ implementation and evaluation is still in its early stages, and more empirical evidence is needed. This meta-analysis took a complexity science approach to evaluate the effectiveness of JITAIs that promote healthy behaviors and assess whether key design principles can increase JITAIs’ impacts. Methods: We searched five databases for English-language papers. Study eligibility required that interventions objectively measured health outcomes, had a control condition or pre-post-test design, and were conducted in the real-world setting. We included randomized and non-randomized trials. Data extraction encompassed interventions’ features, methodologies, theoretical foundations, and delivery modes. RoB 2 and ROBINS-I were used to assess risk of bias. Results: The final analysis included 21 effect sizes with 592 participants. All included studies used pre- and post-test design. A three-level random meta-analytic model revealed a medium effect of JITAIs on objective behavior change (g = 0.77 (95% confidence interval (CI); 0.32 to 1.22), p < 0.001). The summary effect was robust to bias. Moderator analysis indicated that design principles, such as theoretical foundations, targeted behaviors, and passive or active assessments, did not moderate JITAIs’ effects. Passive assessments were more likely than a combination of passive and active assessments to relate to higher intervention retention rates. Conclusions: This review demonstrated some evidence for the efficacy of JITAIs. However, high-quality randomized trials and data on non-adherence are needed.