Measurement invariance and discriminant validity of the Cognitive Fusion Questionnaire across five Spanish‑speaking countries

Title

Measurement Invariance and Discriminant Validity of the Cognitive Fusion Questionnaire across Five Spanish-Speaking Countries

Complete Reference

Rosales-Sarabia, R. M., Moore, C., Montoya, C. E., Peña-Tomás, B., Ortiz, S., Santamaría, C., Odriozola-González, P., & Ruiz, F. J. (2025). Measurement invariance and discriminant validity of the Cognitive Fusion Questionnaire across five Spanish-speaking countries. Journal of Contextual Behavioral Science, 38, 100954. https://doi.org/10.1016/j.jcbs.2025.100954

Study Type

Cross-cultural psychometric validation study with measurement invariance analysis and discriminant validity assessment.

BACKGROUND AND OBJECTIVES

Cognitive fusion is a midlevel process in Acceptance and Commitment Therapy (ACT) referring to the human tendency to become entangled with the content of one's thoughts and emotions. More technically, it refers to acting under the control of the ongoing transformation of functions without contacting alternative sources of stimulus control. This process is closely linked to psychological inflexibility and has been consistently associated with emotional symptoms such as depression, anxiety, and stress.

The Cognitive Fusion Questionnaire (CFQ; Gillanders et al., 2014) is the most widely used instrument for measuring cognitive fusion. Although the CFQ has been validated in multiple languages, including several Spanish versions, a significant gap exists in the literature: the factorial equivalence of the CFQ across countries and cultures has not been investigated. Additionally, the discriminant validity of the CFQ relative to negative emotional symptoms has not been sufficiently analyzed, which is important to ensure that the instrument measures a construct distinct from negative emotions.

The objectives of this study were threefold: (1) to evaluate the internal consistency and internal structure validity of the one-factor model of the CFQ in five Spanish-speaking countries (Colombia, Peru, Mexico, Chile, and Spain); (2) to examine the factorial invariance of the CFQ across these countries to determine whether the instrument measures the same construct regardless of population context; and (3) to analyze the discriminant validity of the CFQ in relation to symptoms of depression, anxiety, and stress.

METHOD

Participants

The study included participants from five Spanish-speaking countries with a total sample size of N = 3,389:

Colombian sample: 735 participants (69.66% female, 30.34% male), ages 18–79 years (M = 31.98, SD = 10.58). Education: 8.30% primary/secondary, 11.43% vocational training, 45.03% undergraduate/college, 35.24% postgraduate.
Mexican sample: 909 participants (67.5% female, 31.7% male, 0.8% did not disclose), ages 18–76 years (M = 30.8, SD = 12.2). Education: 25.9% primary/mid-level, 60.0% undergraduate/college, 14.2% postgraduate.
Peruvian sample: 826 participants (59.8% female, 39.5% male, 0.7% did not respond), ages 18–74 years (M = 29.4, SD = 10.5). Education: 10.8% primary/mid-level, 16.2% vocational training, 52.5% undergraduate/college, 20.5% postgraduate.
Chilean sample: 545 participants (64.4% female, 35.6% male), ages 18–73 years (M = 37.9, SD = 11.2). Education: 12.7% primary/mid-level, 27.7% vocational training, 32.9% undergraduate/college, 26.8% postgraduate.
Spanish sample: 374 participants (76.5% female, 23.3% male, 0.2% other), ages 18–72 years (M = 36.3, SD = 12.2). Education: 20.6% primary/mid-level, 17.7% vocational training, 35.6% undergraduate/college, 26.5% postgraduate.

Data were collected from July to December 2020 through online surveys distributed via social networks using snowball sampling.

Instrument(s) Evaluated

Cognitive Fusion Questionnaire (CFQ). The CFQ is a self-report instrument with 7 items rated on a 7-point Likert-type scale (1 = never true, 7 = always true). The Spanish version developed by Ruiz, Suárez-Falcón, et al. (2017) was utilized. The seven items are:

"Mis pensamientos me causan angustia o dolor emocional" [My thoughts cause me distress or emotional pain]
"Me quedo tan enganchado a mis pensamientos que no soy capaz de hacer las cosas que más quiero hacer" [I get so caught up in my thoughts that I am unable to do the things that I most want to do]
"Analizo las situaciones demasiado, hasta el punto en que no me resulta útil" [I over-analyze situations to the point where it's unhelpful to me]
"Lucho contra mis pensamientos" [I struggle with my thoughts]
"Me enfado conmigo mismo por tener determinados pensamientos" [I get upset with myself for having certain thoughts]
"Tiendo a enredarme mucho en mis pensamientos" [I tend to get very entangled in my thoughts]
"Me resulta muy difícil dejar pasar los pensamientos molestos incluso cuando sé que hacerlo me ayudaría" [It's such a struggle to let go of upsetting thoughts even when I know that letting go would be helpful]

This version has demonstrated excellent internal consistency (Cronbach's alphas between 0.89–0.93), the expected one-factor structure, measurement invariance across gender and clinical versus nonclinical samples, convergent construct validity, criterion validity, and sensitivity to treatment effects in Colombian populations. It has shown strong psychometric performance in other Spanish-speaking countries (Spain, Peru, Ecuador, Mexico).

Other Measurement Instruments

Depression Anxiety and Stress Scale – 21 (DASS-21). The DASS-21 (Lovibond & Lovibond, 1995; Spanish version by Daza et al., 2002) is a widely used self-report instrument designed to assess emotional symptoms related to depression, anxiety, and stress. It consists of 21 items rated on a 4-point Likert-type scale (0 = did not apply to me at all; 3 = applied to me very much, or most of the time). The Spanish version has demonstrated excellent internal consistency in samples from Colombia and Spain, exhibiting a hierarchical factor structure that includes three first-order factors and a second-order factor reflecting general emotional symptomatology.

Data Analysis

JASP 0.18.3.0 was used for all analyses. Corrected item-total correlations were examined to detect careless or random responding (values < 0.30 suggest poor responding). Internal consistency coefficients were calculated: Cronbach's alpha and McDonald's omega.

Confirmatory factor analyses (CFA) were conducted for each sample using the robust diagonally weighted least squares (robust DWLS) estimator. This estimator was selected due to the ordinal scale of the CFQ. Multiple goodness-of-fit indices were evaluated: Comparative Fit Index (CFI), Non-Normed Fit Index (NNFI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR). Fit criteria: CFI/NNFI ≥ 0.90 (acceptable), ≥ 0.95 (good); RMSEA < 0.08 (acceptable), < 0.05 (good); SRMR < 0.05 (very good), ≤ 0.08 (good).

Measurement invariance of the one-factor model of the CFQ across countries was analyzed following guidelines from Jöreskog (2005) and Millsap and Yun-Tein (2004), examining four progressively more restrictive models: configural invariance (item-factor loadings and number of factors consistent), metric invariance (factor loadings invariant), scalar invariance (item intercepts invariant), and strict invariance (item error variances invariant). Criteria from Cheung and Rensvold (2002) and Chen (2007) were applied: ΔRMSEA < 0.01 and ΔNNFI/ΔCFI > −0.01.

Sensitivity analysis was conducted to evaluate whether mandatory responses on items affected reliability, using model-free indicators: multivariate Mahalanobis distance (p < .01) and response uniformity/longstring (99th percentile). Flagged cases were excluded to re-estimate models.

For discriminant validity, the heterotrait-monotrait ratio of correlations (HTMT) from Henseler et al. (2015) was used via CFA on a four-factor correlated model including cognitive fusion, depression, anxiety, and stress. An HTMT < 0.850 was considered a conservative benchmark for discriminant validity.

RESULTS

Psychometric Quality of Items

All CFQ items exhibited high discrimination indexes across all samples (range 0.761–0.872). Item 6 ("Tiendo a enredarme mucho en mis pensamientos" [I tend to get very entangled in my thoughts]) recorded the highest discrimination index for Colombian, Mexican, Peruvian, and Spanish samples. For the Chilean sample, Item 4 ("Lucho contra mis pensamientos" [I struggle with my thoughts]) showed the highest index. Items with the lowest discrimination indexes varied by country: Item 1 (Mexico, Chile), Item 3 (Colombia), Item 4 (Peru), Item 5 (Spain).

Internal consistency: Estimates were nearly identical for Cronbach's alpha and McDonald's omega. They were excellent across all samples: Cronbach's alpha ranged from 0.937 (Mexico) to 0.952 (Peru); McDonald's omega ranged from 0.938 (Mexico) to 0.952 (Peru).

Validity Evidence Based on Internal Structure

Dimensionality: The one-factor model obtained very good fit according to CFI, NNFI, and SRMR values:

Country	RMSEA [90% CI]	CFI	NNFI	SRMR	S-B χ² (df)
Colombia	0.062 [0.045, 0.080]	0.999	0.999	0.019	54.645 (14)
Mexico	0.067 [0.052, 0.083]	0.999	0.998	0.024	71.018 (14)
Peru	0.083 [0.067, 0.099]	0.999	0.998	0.026	93.127 (14)
Chile	0.134 [0.115, 0.153]	0.997	0.995	0.042	149.779 (14)
Spain	0.097 [0.073, 0.122]	0.998	0.998	0.030	62.932 (14)

Although RMSEA values for the Chilean and Spanish samples were relatively elevated, the one-factor model was considered adequate because RMSEA tends to be artificially high for models with small degrees of freedom, even when the model is correctly specified.

Measurement invariance: The one-factor model of the CFQ demonstrated parameter invariance at all levels across countries:

Model	RMSEA	ΔRMSEA	CFI	ΔCFI	NNFI	ΔNNFI
Configural invariance	0.087	—	0.999	—	0.998	—
Metric invariance	0.086	0.001	0.998	−0.001	0.998	0.000
Scalar invariance	0.062	0.024	0.998	0.000	0.999	0.001
Strict invariance	0.062	0.000	0.998	0.000	0.999	0.000

CFI and NNFI values showed minimal changes across models, whereas RMSEA values improved (decreased) when fitting more constrained models, likely related to increased degrees of freedom in more constrained models.

Sensitivity Analysis by Excluding Potential Careless Responders

Insufficient-effort responding (IER) screening flagged 190 respondents (5.6% of participants). Re-estimating the one-factor model excluding these cases yielded similar fit indices (Colombia: CFI = 1.000, NNFI = 0.999; RMSEA = 0.056; Peru: CFI = 0.999, NNFI = 0.999; RMSEA = 0.075; Mexico: CFI = 0.999, NNFI = 0.999; RMSEA = 0.059; Chile: CFI = 0.996, NNFI = 0.995; RMSEA = 0.153; Spain: CFI = 0.999, NNFI = 0.999; RMSEA = 0.068). Measurement invariance across countries yielded results very similar to the full sample, resulting in the same invariance decisions with highly similar fit.

Discriminant Validity of the CFQ

The CFA of the four-factor model (cognitive fusion, depression, anxiety, stress) demonstrated good fit: RMSEA = 0.041 [95% CI 0.040, 0.043], CFI = 0.998, NNFI = 0.998, SRMR = 0.029.

HTMT values provided evidence for the discriminant validity of the CFQ regarding depression (HTMT = 0.811), anxiety (HTMT = 0.755), and stress (HTMT = 0.817), all below the conservative threshold of 0.850.

DISCUSSION AND CONCLUSIONS

This study addressed two largely unexamined aspects of the CFQ: its factorial equivalence across countries and its discriminant validity relative to negative emotions. Establishing factorial equivalence is an essential first step before conducting cross-cultural research on cognitive fusion.

Results demonstrated that the CFQ maintains excellent psychometric properties across all samples. The one-factor model obtained very good fit according to CFI, NNFI, and SRMR criteria. Measurement invariance analysis revealed that the CFQ demonstrates parameter invariance at all levels across the five Spanish-speaking countries. This finding, to the authors' knowledge, represents the first evidence of measurement invariance for the CFQ across different countries. It indicates that the scale's internal structure is robust and the construct is measured similarly across countries regardless of cultural context.

Regarding discriminant validity, HTMT values between cognitive fusion and emotional symptoms were strong but remained below the conservative threshold established by Henseler et al. (2015). The CFQ demonstrated satisfactory discriminant validity relative to negative emotions, complementing prior research that confirmed its adequacy regarding frequency of negative thoughts.

The authors acknowledge several limitations. First, the study included only five Spanish-speaking countries, although Spanish is an official language in 21 countries. Second, the Ruiz, Suárez-Falcón, et al. (2017) translation was used, so results may not extend to other Spanish versions of the CFQ. Third, discriminant validity relative to neuroticism was not examined. Fourth, the CFQ was administered only online. Fifth, samples consisted primarily of females and lacked sociodemographic data such as sexual orientation and income. Sixth, response-time data were lacking. Seventh, although measurement invariance and discriminant validity were assessed at a nomothetic level via cross-sectional data, dimensionality and relationship with negative emotion measures may differ at the individual level.

In conclusion, this study provides strong empirical evidence for the factorial equivalence of the CFQ across five Spanish-speaking countries, paving the way for future cross-cultural research on cognitive fusion. Researchers may investigate whether some countries exhibit higher levels of cognitive fusion or whether cognitive fusion associates differently with emotional symptoms and quality of life in various contexts. The study also confirmed that the CFQ measures a construct distinct from negative emotions, which is significant for both researchers and practitioners, ensuring measurement accuracy and validating the importance of cognitive fusion in psychopathology.

Significance and contribution

This study provides robust empirical evidence for the cross-cultural validity and measurement invariance of the Cognitive Fusion Questionnaire across five Spanish-speaking countries, advancing the field's understanding of cognitive fusion as a distinct psychological construct. By demonstrating that the CFQ maintains equivalent psychometric properties and internal structure across Colombia, Peru, Mexico, Chile, and Spain, the study establishes a foundation for legitimate cross-cultural research on cognitive fusion. The confirmation of discriminant validity—showing that cognitive fusion is conceptually separable from negative emotional symptoms—enriches psychological measurement by clarifying that cognitive fusion represents a distinctive process within psychopathology, independent of emotional symptomatology.