Why Research From Lower- and Middle-Income Countries Matters to Evidence-Based Intervention: A State-of-the-Science Review of ACT Research as an Example

Title

Why Research From Lower- and Middle-Income Countries Matters to Evidence-Based Intervention: A State of the Science Review of ACT Research as an Example

Complete Reference

Sahdra, B. K., King, G., Payne, J. S., Ruiz, F. J., Kolahdouzan, S. A., Ciarrochi, J., & Hayes, S. C. (2024). Why research from lower- and middle-income countries matters to evidence-based intervention: A state of the science review of ACT research as an example. Behavior Therapy, 55(6), 1348–1363. https://doi.org/10.1016/j.beth.2024.06.003

Type of Work

A state-of-the-science review examining the importance of research conducted in lower- and middle-income countries (LMICs) for evidence-based intervention, using Acceptance and Commitment Therapy (ACT) as an illustrative example. Published in Behavior Therapy.

---

Central Thesis and Objectives

The article argues that a critical worldwide scientific imbalance exists: despite the global nature of psychological problems, an overwhelming majority of research originates from a small segment of the world's population living in high-income countries (HICs). This imbalance risks distorting our understanding of psychological phenomena by underrepresenting the cultural and contextual diversity of human experience.

The authors identify that LMIC research is less frequently cited both because it is "apparently viewed as a special case" and because it is less well known due to language differences and biased indexing algorithms. This invisibility is particularly significant given that many clinically relevant topics may be more prevalent or present differently in LMICs than in HICs.

The primary objective is to examine ACT as a case example to demonstrate:

1. The importance of topics identified in LMIC research in randomized controlled trials (RCTs)
2. How neglect of LMIC research marginalizes the voices of LMIC researchers and practitioners
3. The structural and indexing barriers that contribute to LMIC research invisibility
4. How the field of ACT/evidence-based intervention can benefit from LMIC research

---

Theoretical Framework

Global Context of Psychological Research

The authors establish that in 2010, approximately 96% of the world's psychological research came from 12% of the world's population living in so-called WEIRD countries (Western, Educated, Industrialized, Rich, Democratic). A decade later, Talmayer et al. (2021) found similar results. This skewed distribution contrasts with the global prevalence of psychological problems, which affect populations across all economic and cultural contexts.

The authors document that the potential for knowledge distortion produced by this imbalance is enormous. Among the 100 most eminent psychologists identified by Diener et al. (2014), fewer than 10% spent a large part of their careers outside the United States. Of the 100 most cited articles in science, even one had not been coauthored by an LMIC author.

Indexing Infrastructure and Visibility

A fundamental aspect of the analysis concerns how database indexing systems (Elsevier/Web of Science, Clarivate/Scopus, PubMed) are heavily biased against LMICs. LMIC journals are:

• Less indexed, often for decades
• Less cited when indexed
• More difficult to access even with open-access publication
• Affected by prohibitive article processing charges (APCs) that are prohibitive for LMIC researchers

This infrastructure is not merely a "scientific quality" problem but a structural barrier that actively marginalizes knowledge produced in LMICs.

ACT as an Example of Process-Based Intervention

The authors select ACT as a case study because:

1. It is recognized as an evidence-based approach by bodies such as the World Health Organization and has been at the forefront of "third wave" behavioral therapy development for decades.

2. It is built upon basic psychological processes—specifically a "psychological flexibility" model that can be applied transdiagnostically. As Hayes (in press) notes, the first 1,000 ACT RCTs focused on medical illness (cancer, diabetes, chronic illness, physical impairment 26% of the time), health-related behaviors (weight loss, exercise, smoking, and other substance use—10%), performance (including academic, workplace, or athletic performance—7%), clinical outcomes (9%), chronic pain (6%), and social concerns (4%), with a variety of other less frequent topics.

3. It offers intercultural flexibility—the process-based approach of ACT can be adapted to clients' specific experiences and preferences with specific cultural backgrounds, in contrast to approaches that prescribe a list of values to be embraced.

Invisible Research and Its Implications

The authors argue that when LMIC research is ignored, entire topics of ACT research become invisible. This creates multiple problems:

• Sample bias: If certain methods are particularly popular in LMICs compared with HICs, or vice versa, the comparison of the impact or prevalence of specific approaches will be biased by the relative invisibility of LMIC research.

• Misunderstanding of processes of change: Without examining the full range of research available in LMICs and HICs, it is possible that researchers could miss important information about how processes of change actually work across diverse sociocultural contexts.

• Myopic clinical application: If the knowledge base is limited to HICs, clinical understanding of the problems needing intervention will be incomplete.

---

Argument Development

Methodology: Structural Topic Modeling (STM)

The authors employed Structural Topic Modeling (STM), an unsupervised machine learning method, to analyze latent topics in a database spanning more than a decade of ACT RCT research conducted in LMICs.

Collection Procedure:

• Extracted citations of ACT RCTs reported in Hayes & King (in press)
• Searched for abstracts online through Behavioral Science (contextualscience.org)
• Initial database comprised 1,015 ACT RCTs from 1986-2023
• Primary focus on 464 RCTs appearing in journals with home offices in LMICs
• From these, identified 457 English-language abstracts with sufficient content for analysis

Text Processing:

• Tokenization, removal of numbers and punctuation
• Stemming and other language normalization techniques
• Reduction to relevant vocabulary
• 457 documents (abstracts), 283 terms, 25,646 tokens processed

STM Model:

• Varied the number of topics from 5 to 40 in increments of 5
• Compared diagnostic properties: held-out likelihood (larger better), residual dispersion, lower bound
• "Optimal" model of 30 topics with relatively high held-out likelihood, reasonable coherence, and lowest residuals
• Final STM model with 30 topics and four covariates: year, indexing, DSM focus, country

Results: The 30 Topics

The analysis identified 30 distinct topics in ACT research in LMICs. The topics labeled by co-authors (Table 1) include:

1. Observation of Intervention Groups
2. Couples Functioning and Satisfaction
3. Mothers of Special Needs Children
4. Adolescents Mental Health
5. Cognitive-Emotion Regulation
6. Chronic Pain
7. Type 2 Diabetes
8. Bodily Concerns
9. School Students
10. Psychological Well-Being
11. ACT Mindfulness Training of Non-Clinical Populations
12. ACT Combination Treatments
13. Statistics Related Terms Usage
14. Dynamics of Marital Satisfaction Among Women
15. Physical and Mental Disability
16. Anxiety and Depression
17. Quality of Life
18. Cancer
19. ACT Compared to Other Therapies
20. Resilience and Adjustment
21. ACT During Rehabilitation
22. Stress Reduction
23. Family Functioning
24. ACT Counseling
25. Distress of Physical Diseases
26. Self-Efficacy or Self-Esteem
27. Relationship Studies
28. Pre-Post Experimental Design
29. ACT Psychotherapy Treatment
30. Psychological Flexibility

Important note: The authors indicate that topics marked with * (Observation of Intervention Groups, Distress of Physical Diseases, Psychological Flexibility) were considered meaningful but potentially non-substantive in examining the substantive topics of ACT RCT research landscape in LMICs.

Key Findings on Topics

Temporal Trends: Year of publication had a statistically significant effect on expected topic proportions. Several topics showed patterns of change from 2011-2023:

• Declining topics: Adolescents Mental Health, ACT Mindfulness Training of Non-Clinical Populations, ACT Combination Treatments, Physical and Mental Disability
• Increasing topics: Relationship Studies, Self-Efficacy or Self-Esteem, ACT Psychotherapy Treatment

Indexing Effects: Indexing had a statistically significant association with topic prevalence for six substantive topics (Figure 3):

• Resilience and Adjustment
• Pre-Post Experimental Design
• ACT Psychotherapy Treatment
• Relationships

Topics from indexed sources, relative to nonindexed ones, showed higher expected topic proportions. This indicates that even within LMIC research, certain topics are invisible when research is not indexed.

Country-Level Variation: The analysis showed that different countries contributed substantially distinct emphases:

• Adolescents Mental Health was important in India, Bulgaria, Ethiopia, Indonesia
• Chronic Pain prevalent in India, China, Brazil, Iran, Algeria
• School Students varied reliably across countries for India, China, Thailand, Iran, Egypt
• Psychological Well-Being with reliable differences across Iran, Afghanistan, Algeria, and Macedonia

DSM Focus: When examining whether articles focused on DSM diagnostic categories (24.51%, n = 112 vs. 75.49%, n = 345 without DSM focus):

• Topics with higher expected proportions among DSM-focused abstracts: Bodily Concerns, ACT Combination Treatments, Anxiety and Depression, Self-Efficacy or Self-Esteem, Adolescents Mental Health
• Topics with lower proportions: Couples Functioning and Satisfaction, Chronic Pain, Psychological Well-Being, Resilience and Adjustment, Stress, Adjustment, Distress of Physical Diseases

The Question of Research Quality and Sample Size

An important finding was that "LMIC research was often nonindexed, but study size could not explain a lack of indexing." The authors found that many objectively identified topics became invisible when LMIC research was ignored.

They examined this through a case study: a large Chinese nonindexed study on orthodontic compliance in adolescents (Gang et al., 2016). A total of 400 adolescent orthodontic patients were randomly assigned to conventional orthodontic treatment or ACT plus treatment. There were statistically significant differences between the two groups in maintaining good oral hygiene, intactness of attachment to the dental appliance, and compliance with proper wearing of adjusted appliances. The authors concluded that ACT can improve and shorten orthodontic treatment processes.

The finding was important because the number of ACT studies in dental care is limited, and because the use of objective measures was of particular interest. Yet because it was not indexed, this research was invisible in the ACT literature.

The authors underscore a statistical principle: even if sample size per study is smaller in LMICs compared with HICs, the accumulation of LMIC studies can result in substantial implications. This is analogous to the well-known argument that the volume of small-scale research is important for understanding processes of change.

Citation Impact: LMICs vs. HICs

The authors examined differential citation impact between HICs and LMICs through randomized block analyses:

Methodology: They randomly selected 3 studies from each of the four blocks of the "Hayes & King database" for each year 2015-2022, stratified by country income level (HIC vs. LMIC), indexing status (indexed vs. nonindexed), and year of publication.

Key Findings:

1. Indexing predicts citations: After controlling for years since publication and indexing, indexed LMIC publications received 410% more citations per article on average than nonindexed articles; for HICs, indexed articles received 482% more citations per article on average than nonindexed articles.

2. Sample size predicts citation in HICs, not in LMICs: In HIC articles, after controlling for years since publication and indexing status, higher sample size per study arm was associated with higher citations. In LMIC articles, after controlling for years since publication and indexing status, sample size per study arm did not predict citations. These findings indicate that although appearing in an indexed journal is important for boosting the visibility of studies from LMICs, sample size (per study arm)—as an indicator of scientific quality—is not predictive of greater visibility. This is consistent with our earlier result where sample size was not predictive of indexing in LMICs, as it was in HICs.

3. Theoretical implication: This suggests that the visibility of LMIC studies is not driven by indicators of "scientific quality" as conventionally defined (e.g., sample size), but rather by accessibility through indexing.

---

Clinical and Research Implications

Invisibility of LMIC Research and Loss of Knowledge

The findings suggest that the marginalization of the LMIC voice has real clinical implications:

1. Bias in Knowledge Base: If LMIC research is ignored, the knowledge base for evidence-based intervention will be systematically biased in favor of HIC findings, which may not generalize.

2. Lost Topics: Certain clinically relevant topics may be less well known when LMIC research is ignored. For example, research on how to culturally adapt ACT for religious and alternative wisdom traditions that differ across HICs and LMICs may be augmented or diminished, resulting in incomplete or distorted knowledge.

3. Real Practical Barriers: LMIC researchers encounter notable obstacles when attempting to publish in international peer-reviewed journals:

   • Limited funding
   • Language barriers
   • Gender disparities
   • Prohibitive APCs that bias the literature toward well-funded HIC researchers

Process-Based Approaches and Cultural Flexibility

A central argument is that ACT, as a process-based approach, offers greater intercultural flexibility than other behavioral therapy approaches. Unlike some CBT approaches that prescribe a list of values to be embraced, ACT asks clients to fit their experiences and preferences into values component of the model.

The authors suggest that this feature is particularly important for diverse populations, including those in religious contexts where cultural adaptation is critical.

Religion-Based Evidence Research

An illustrative example is provided by ACT research in religious contexts. The authors note that there are dozens of examples of studies (mostly nonindexed, mostly from Iran) that have modified ACT using stories and metaphors from the Sacred Quran (ACBS, 2022). Most Iranian research is nonindexed; if this were ignored, the global scientific community would not know what has been learned there.

The authors note that more psychological and process-based approaches to evidence-based intervention may be especially useful for delineating and fostering processes of change relevant to client goals and values, with religion being helpful for delineating these goals and fostering specific spiritual lifestyles that might include these goals and values. An illustration of what religion-based behavioral research might do so is presented by Asnaam et al.'s (2022) response to the ACBS Task Force 2021 report (Hayes et al., 2021).

---

Contribution to the Field of RNT-Focused ACT

Reinterpretation of LMIC Research

The findings presented suggest that LMIC research in ACT reflects topics in many ways similar to HIC research, but without considering LMIC studies, these trends are difficult to quantify objectively. The results also show that LMIC ACT RCTs are less cited worldwide, especially if nonindexed.

The principal conclusion is that the RCT base of ACT in LMICs is substantial and varied, covering nearly 500 studies across a wide range of topics. Despite its size, a major finding of the present study is that a lack of indexing is marginalizing the voice of LMIC researchers, and particularly LMIC researchers who are non-English speakers.

Mechanisms of Marginalization

The mechanisms of marginalization include:

1. Active seeking of voices (e.g., rejecting high-quality LMIC studies, or passively neglecting LMIC studies in literature reviews)
2. Passive neglect (ignoring LMIC studies in literature reviews)
3. Country-specific practices (e.g., LMIC countries encouraging a student to publish in a native-language journal to get an advanced degree)
4. Infrastructural barriers (e.g., prohibitive Article Processing Charges)

Importance for Process-Based Therapy

The findings suggest that for behavioral and cognitive therapists interested in evidence-based intervention more generally, paying more attention to LMIC research can benefit how research in LMICs might inform the development of ACT or other evidence-based interventions more generally.

If this pattern is replicated in CBT and evidence-based therapy more generally, it would further show that there is a produced inequality resulting from this global scientific imbalance that risks distorting our understanding of psychological phenomena worldwide.

Perspective of Process-Based Medicine

The authors note that process-based medicine (PBM) emphasizes process-based behavioral and cognitive approaches (also called process-based cognitive behavioral therapy [PBCBT] or process-based behavior therapy [PBBT]). This approach builds on the view that the core target focus of intervention should be on the bioprocessosocial processes that can be most effectively and efficiently modified (Hofmann & Hayes, 2019).

It is suggested that high temporal density in measures is connected to basic principles with higher precision and scope. For example, a principle such as reinforcement may be as "culturally narrow" as an agreed-upon list of virtues or character strengths, but an agreed-upon list of virtues or character strengths may not be an imposition of ideological synthesis that is necessarily narrower than a list, but rather an exploration of what "culture-bound syndromes" have been added to the DSM itself to recognize the fact that there are disorders that are more common in one culture and never in others.

---

Conclusion

The present research provides a demonstration of how the invisibility of LMIC research in the global indexed scientific literature creates substantial inequalities in scientific knowledge about evidence-based intervention. Using ACT as a case study, the authors document that:

1. There is a substantial and varied RCT evidence base in LMICs
2. This research is systematically less cited globally
3. The invisibility is largely driven by indexing and access factors, not "quality" in the conventional sense
4. Neglect of LMIC research means that clinically important topics may be overlooked by the global scientific community

The authors conclude that to develop truly evidence-based intervention that is culturally inclusive, diverse, and equitable, the scientific community must take deliberate steps to increase the visibility, knowledge, and indexing of LMIC research. This includes:

• Affirmation of LMIC researchers
• Efforts to change indexing practices
• Collaboration between LMIC and HIC research centers
• Recognition that small but cumulative LMIC research provides important information for clinical practice

---

---