Comparing Confirmatory Factor Analysis Models Using DWLS in R's lavaan Package

CFA
1

Help needed with CFA model comparison in R

I’m working on a project using Confirmatory Factor Analysis (CFA) in R. I’ve got two models I want to compare:

  1. A 4-factor model
  2. A 2-factor model

I’m using the DWLS estimator in the lavaan package because I’m dealing with ordinal data. Here’s a snippet of what I’ve done:

model_4factor <- cfa(four_factor_spec, data = my_dataset, 
                     ordered = c('Q1', 'Q2', 'Q3', 'Q4', 'Q5', 'Q6'))

summary(model_4factor, fit = TRUE)

When I try to compare these models using anova(), I get an error about unconstrained parameter sets not matching.

I noticed that when using Maximum Likelihood estimation, I get AIC and BIC values, but these are missing with DWLS.

Does anyone know how to properly compare CFA models estimated with DWLS in lavaan? Or is there another way to determine which model fits best? Thanks for any help!

2

hey, i’ve faced similar issues. for DWLS models, consider the WRMR index for ordinal data. also, try the scaled chi-square difference test via lavTestLRT() in lavaan. hope this helps with ur analysis!

3

Hey there WhisperingWind! :wind_face:

I totally get your frustration with comparing CFA models using DWLS. It can be a real head-scratcher, right? :thinking:

Have you thought about using the DIFFTEST option in lavaan? It’s pretty nifty for comparing nested models with DWLS. Here’s a quick example:

model_comparison <- lavTestLRT(model_4factor, model_2factor, method = 'satorra.bentler.2001')

This gives you a scaled chi-square difference test, which is way more appropriate for ordinal data than the regular anova().

Also, don’t forget to look at those fit indices like CFI, TLI, and RMSEA. They can tell you a lot about which model is working better.

Oh, and have you considered using the semTools package? It’s got some cool functions for comparing non-nested models with DWLS.

What kind of theory are you working with that led you to these two models? I’m super curious about your research! :blush:

4

I’ve encountered similar issues when using DWLS for CFA models with ordinal data. In this case, traditional indices like AIC and BIC do not work as expected. Instead, focus on alternative fit indices such as CFI, TLI, RMSEA, and SRMR as reported in the lavaan output. You might also try a scaled chi-square difference test using the lavTestLRT() function:

lavTestLRT(model_4factor, model_2factor)

This should provide you with a test statistic and p-value for comparing the models. In the end, consider the theoretical underpinnings of your models along with these tests to select the most appropriate one.