Validity evidence of the Revised Child Anxiety and Depression Scale-30 in Colombian children

Title

Validity Evidence of the Revised Child Anxiety and Depression Scale-30 in Colombian Children

Complete Reference

Barajas, H. N., & Ruiz, F. J. (2024). Validity evidence of the Revised Child Anxiety and Depression Scale-30 in Colombian children. International Journal of Psychology & Psychological Therapy, 24(2), 255-267.

Study Type

Psychometric validation study. Analysis of psychometric properties through confirmatory factor analysis (CFA), measurement invariance analysis, discriminant validity, and convergence analysis.

BACKGROUND AND OBJECTIVES

Emotional disorders present high prevalence rates in the child population worldwide. The prevalence of anxiety disorders in children is estimated between 9% and 32%, while depression affects approximately 2-6% of the child population. These disorders significantly interfere with adequate development at social, emotional, cognitive, and academic levels. Additionally, there is a high rate of comorbidity between anxiety and depression disorders in the child population, presenting greater severity and resistance to psychological and pharmacological interventions.

The Revised Child Anxiety and Depression Scale (RCADS) is a widely used instrument to assess symptoms of anxiety and depression in children and adolescents. Although the RCADS has been extensively analyzed in Spain and some Latin American countries, there is a scarcity of validation studies in Latin American contexts. The 30-item version (RCADS-30) is a shorter adaptation that maintains the original six-factor structure, encompassing panic disorder, social phobia, separation anxiety disorder (SAD), generalized anxiety disorder (GAD), obsessive-compulsive disorder (OCD), and depression.

The present study examined the psychometric properties of the RCADS-30 in a sample of Colombian children, analyzing internal consistency, factor structure, measurement invariance across gender and age groups, and discriminant and convergent validity of the scale.

METHOD

Participants

The sample consisted of a total of 585 students between 8 and 12 years of age (47.7% female, M = 10.37, SD = 1.25). All participants attended public schools located in the Department of Boyacá, Colombia. 63.34% came from elementary schools and 36.75% from middle schools.

Regarding recruitment procedures, contact was made with two public schools in the Department of Boyacá. Institutional Ethics Committee approval was obtained. Subsequently, information was presented regarding the study's objectives, procedure to be followed, potential benefits and risks, confidentiality, and voluntary participation. All children with parental or legal representative authorization were invited to participate. Of the 850 parents invited, 76.47% consented to their children's participation. Of the 585 authorized children, 98% completed informed consent. 8% of participants were absent due to moving to a new city. Subsequently, results were sent to interested parents as compensation for participation.

Instrument(s) Evaluated

Revised Child Anxiety and Depression Scale-30 (RCADS-30). This is a screening instrument that assesses symptoms of anxiety and depression in children and adolescents. It comprises 30 items with 4-point Likert-type response options (never, sometimes, often, always). The scale has a six-factor structure, each comprising five items: depressive disorder, panic disorder, social phobia, separation anxiety disorder (SAD), generalized anxiety disorder (GAD), and obsessive-compulsive disorder (OCD). The RCADS-30 was initially validated in Spanish by Sandín and colleagues (2010) and has demonstrated acceptable psychometric properties, including a six-factor structure and discriminant validity.

Other Measurement Instruments

Depression Anxiety Stress Scale-Youth (DASS-Y). This is a self-report instrument comprising 21 items with Likert-type response options. The DASS-Y was developed by Szabó and Lovibond (2002) as an abbreviated version for child populations. It assesses symptoms of depression (e.g., "I could not stop feeling sad"), anxiety (e.g., "I was afraid"), and stress (e.g., "I was worried about many things"). The Spanish-translated version has demonstrated the expected three-factor structure and acceptable internal consistency (between .77 and .81) in Colombian child populations (cited in Ruiz et al., submitted).

Data Analysis

All statistical analyses were conducted using JASP 0.18.3.0. First, the functioning of RCADS-30 items was investigated by calculating corrected item-total correlations. Items with a discrimination index below .20 were deemed fully and removed.

To examine internal construct validity, a confirmatory factor analysis (CFA) was performed on the six-factor structure of the RCADS-30 using robust maximum likelihood (RML). The Satorra-Bentler chi-square test and the following goodness-of-fit indices were compared: (a) the root mean square error of approximation (RMSEA), (b) the comparative fit index (CFI), (c) the non-normed fit index (NNFI), and (d) the standardized root mean square residual (SRMR). According to Hu and Bentler (1999), RMSEA values of 0.08 signify acceptable fit, while values below 0.05 indicate very good fit. Regarding CFI and NNFI, values above .90 suggest acceptable models, while values above .95 indicate very good fit.

Additional CFAs were conducted to examine the measurement invariance of the six-factor structure across gender (boys and girls) and age groups (8-10 years and 11-12 years). Following the recommendations of Jöreskog (2005), Kline (2005), and Millsap and Yun-Tein (2004), metric, scalar, and strict invariances were assessed by examining the equality of factor loadings, item intercepts, and item error variances across specified groups. To compare models, CFI, NNFI, and RMSEA indices were evaluated across nested models. The more restricted model was selected (e.g., the second versus the first, and the third versus the second) based on criteria outlined by Cheung and Rensvold (2002) and Chen (2007): (a) RMSEA was less than 0.01, and (b) NNFI and CFI were 0.01 or greater.

Discriminant validity of the RCADS-30 subscales was assessed by calculating the heterotrait-monotrait ratio of correlations (HTMT). The HTMT provides an estimate of correlations between constructs using structural equation modeling. According to Henseler and colleagues (2015), an HTMT value below .90 or .85 indicates evidence of discriminant validity across constructs. This method has demonstrated superior performance to the traditional Fornell-Larcker criterion (1981).

Pearson correlation coefficients were calculated between RCADS-30 subscales and the DASS-Y to evaluate convergent validity. A two-way analysis of variance (ANOVA) was conducted to examine disparities in RCADS-30 scores across gender and age group. Partial eta squared (ƞ²) was calculated to indicate the effect size of factors, with the following benchmarks: .01 for small effect, .06 for medium effect, and .14 or higher for large effect.

RESULTS

Item Quality

Table 1 showed the RCADS-30 items, their corrected item-total correlations, and descriptive data for each item. Discrimination indices were good for the Depression subscale (range .418 to .536), Panic Disorder (range .480 to .609), Social Phobia (range .424 to .620), and GAD and OCD (range .360 to .803). Two items were removed: Item 5 (r = .282) from OCD and Item 12 (r = .280) from OCD, which showed acceptable but lower values.

Internal Consistency and Reliability

Table 2 indicated that Cronbach's alpha and McDonald's omega were acceptable for all subscales except OCD. For Depression, Panic Disorder, Social Phobia, and GAD, values ranged between .702 and .779 (alpha) and .757 and .812 (omega). For SAD and OCD, alpha was .702 and omega .779 for Panic; omega was .701 for SAD; .780 for GAD; and .651 for OCD. The OCD subscale obtained an alpha of .659 and an omega of .653, which can be considered questionable values (George & Mallery, 2003; Ventura-León & Caycho-Rodríguez, 2017).

Dimensionality and Factor Structure

Regarding the dimensionality of the RCADS-30, the six-factor model obtained acceptable fit to the data according to χ²(df= 2.190) (χ² = 854.172, p < .001), RMSEA = 0.049 (90% CI [0.045, 0.053]), SRMR = 0.056, and CFI = 0.902. The NNFI value (0.890) indicated poorer fit. Considering that goodness-of-fit indices should be considered globally (Brown, 2015), it can be concluded that the six-factor model of the RCADS showed acceptable fit to the data.

Measurement Invariance

Table 3 presented the results of measurement invariance analyses. All levels of measurement invariance were supported across gender because changes in RMSEA, CFI, and NNFI were lower than 0.01. Factorial equivalence analyses across age groups also supported metric and scalar invariance. However, criteria for claiming strict invariance were not completely met because the change in CFI was higher than 0.01.

Discriminant Validity

Regarding discriminant construct validity, Table 4 presents the HTMT values of the RCADS-30 subscales. The RCADS-30 demonstrated satisfactory discriminant validity when applying the more liberal criterion suggested by Henseler and colleagues (2015) of .90. However, when applying the stricter criterion of .85, some discriminant validity issues emerged. Specifically, HTMT values between Depression and Panic Disorder (HTMT = .861) and between Panic Disorder and OCD (HTMT = .876) were slightly above this cutoff.

Convergent Validity

Table 5 presents the correlations obtained by the RCADS-30 with the DASS-Y to analyze convergent construct validity. The Depression subscale of the RCADS-30 showed a strong correlation with DASS-Depression (r = .644), which was expected. This subscale also strongly correlated with DASS-Anxiety (r = .586) and DASS-Stress (r = .535). Within the RCADS-30, Depression showed strong correlations with Panic Disorder (r = .642), Social Phobia (r = .581), and OCD (r = .549).

As expected, Panic Disorder showed a strong correlation with DASS-Anxiety (r = .723) and lower correlations with DASS-Depression and DASS-Stress (r = .577 respectively). Within the RCADS-30, Panic also strongly correlated with Social Phobia (r = .568) and OCD (r = .612). Social Phobia showed strong correlations with GAD (r = .516) and OCD (r = .539).

SAD and GAD showed similar correlations with the DASS-Y (SAD from .220 for Stress to .319 for Anxiety; GAD from .293 for Depression to .382 for Stress). Finally, OCD showed a strong correlation with DASS-Anxiety (r = .548) and a lower correlation with the suggestibility dimension. The RCADS-30 subscales showed stronger correlations among themselves than with SAD and GAD.

Differences by Gender and Age Group

Table 6 presents descriptive data of mean scores on the RCADS-30 according to gender and age group. The gender factor was statistically significant in all RCADS-30 subscales, with girls showing higher scores than boys. Differences were stronger for Depression, Panic Disorder, and Social Phobia, with small to medium effect sizes. The age group factor was statistically significant for Depression, Social Phobia, and GAD. Older children showed higher scores than younger children in Depression and Social Phobia (small effect sizes), but lower scores in GAD (medium effect). Finally, the interaction between gender and age group was statistically significant for Depression, Panic Disorder, and Social Phobia, with small effect sizes in all cases.

DISCUSSION AND CONCLUSIONS

The RCADS is one of the most used instruments to assess emotional disorders in children. In recent years, the short Spanish version of the RCADS, the RCADS-30, has been extensively analyzed in Spain and is beginning to be explored in Latin American countries. Given the lack of validated and comprehensive assessment tools for emotional disorders in Colombia, this study conducted a comprehensive analysis of the psychometric properties of the RCADS-30 in a sample of Colombian children.

Most RCADS-30 items demonstrated good discrimination indices (i.e., values above .30). Two items showed acceptable but lower indices (Item 5, r = .282; Item 12, r = .280), corresponding to the OCD subscale. However, internal consistency of all components was acceptable for both Cronbach's alpha and McDonald's omega measures, with the sole exception of the OCD subscale, which showed a value of .65 for both indicators. These results are coherent with previous research (Selva Batista & Saenz Martínez, 2013; Cervín et al., 2022; Piqueras et al., 2017).

Factor analyses indicated that the six-factor model of the RCADS-30 obtained acceptable fit to the data without requiring respecification. This factor model showed invariance at least at the scalar level across gender and age groups. Although criteria for claiming strict invariance were not completely met because the change in CFI was higher than 0.01, this does not prevent comparing mean scores on the RCADS-30 across different ages (Greiff & Scherer, 2018).

Another important contribution of this study was the discriminant validity analysis of the RCADS-30. Discriminant validity evidence was found according to the more liberal criterion suggested by Henseler and colleagues (2015) for HTMT values. However, it is important to note that some potential discriminant validity issues emerged when applying the stricter criterion of .85. Specifically, HTMT values between Depression and Panic Disorder (HTMT = .861) and between Panic Disorder and OCD (HTMT = .876) were slightly above this cutoff. To our knowledge, this is the first study that assessed the discriminant validity of the RCADS-30.

Regarding convergent validity, the RCADS-30 subscales showed statistically significant correlations with all DASS-Y subscales. Specifically, the Depression subscale of the RCADS-30 showed the strongest correlation with DASS-Depression (r = .644), which was expected. This subscale also strongly correlated with DASS-Anxiety (r = .586) and DASS-Stress (r = .535). Within the RCADS-30, Depression showed stronger correlations with other subscales than with SAD and GAD. Panic Disorder showed the strongest correlation with DASS-Anxiety (r = .723) and lower correlations with DASS-Depression and DASS-Stress (r = .577 respectively). Within the RCADS-30, Panic also strongly correlated with Social Phobia (r = .568) and OCD (r = .612). Social Phobia showed especially strong correlations with DASS-Y and DASS-Y subscales. This might be due to the content of the SAD subscale. Specifically, mean scores on the GAD subscale were among the highest, which might be due to statements such as "I worry that something bad will happen to someone in my family" (Item 11). Although the subscale content does not include references to the extent to which the respondent is worried (e.g., too much) or the interference caused by worrying (e.g., focusing on the present), the item could be answered in a way that someone necessarily does or is expected to do.

Descriptive data analyzed showed that girls obtained higher scores than boys on all RCADS-30 subscales, especially for Depression, Panic Disorder, and Social Phobia, with small to medium effect sizes. In terms of age group, older children showed higher scores than younger children for Depression and Social Phobia (small effect sizes) but lower scores for GAD (medium effect). The increase in mean scores for Depression and Social Phobia was relatively small, but the decrease in GAD scores was more notable. This is expected given the greater autonomy typically observed in preadolescents.

Some limitations of this study merit mention. First, a clinical sample was not recruited. Therefore, these findings should be replicated in children suffering from emotional disorders. Also, administering the RCADS-30 to clinical and nonclinical samples might allow establishment of cutoffs specific to Colombia. Second, data were collected from two schools in one Colombian region. Therefore, these findings need to be replicated in other departments of the country, which would facilitate data generalization. Third, RCADS-30 functioning was explored only in children between 8 and 12 years. Future studies should analyze the psychometric properties of the RCADS-30 in Colombian adolescents. Finally, treatment sensitivity of the RCADS-30 was not analyzed. Future studies should examine whether it shows a similar level of sensitivity to change as that found by Wolpert, Cheng, and Deighton (2015).

In summary, this study examined the psychometric properties of the RCADS-30 in Colombian children for the first time. The RCADS-30 demonstrated acceptable psychometric properties, similar to those obtained in previous validation studies. The six-factor model fit the data adequately and demonstrated measurement invariance across gender and age groups. The RCADS-30 showed evidence of convergent and discriminant construct validity. Therefore, the RCADS-30 demonstrates promising psychometric properties in Colombian children.

Significance and contribution

The validation of the RCADS-30 in Colombian populations contributes significantly to the development of standardized measurement instruments for assessing emotional disorders in Latin American contexts. The findings confirm that the RCADS-30 possesses solid psychometric properties, including appropriate factor structure, acceptable internal consistency, measurement invariance across gender and age groups, and convergent and discriminant validity. The availability of this validated Spanish-language instrument facilitates both empirical research and clinical assessment of anxiety and depression symptoms in Spanish-speaking child populations, enabling equitable comparisons between demographic groups and contributing to the advancement of knowledge regarding emotional disorders in children.