Comparing CFA Models with DWLS Estimation in R: Seeking Guidance

CFA
1

Need help comparing CFA models using DWLS in R

I’m working on a project where I need to compare two CFA models estimated with DWLS (also known as WLSMV) for ordinal data. I’ve set up two models in lavaan:

  • A 4-factor model (let’s call it model_four)
  • A 2-factor model (let’s call it model_two)

Here’s a snippet of my code:

model_four <- cfa(four_factor_spec, data = my_data, ordered = c(
  "Item1", "Item2", "Item3", "Item4", "Item5", "Item6", 
  "Item7", "Item8", "Item9", "Item10", "Item11", "Item12"
))

summary(model_four, fit = TRUE)

When I try to compare these models using anova(model_four, model_two), I get an error:

Error: unconstrained parameter set is not the same in model_two and model_four.

Interestingly, I can compare the models when using Maximum Likelihood estimation. I noticed that ML outputs include AIC and BIC values, which are missing in DWLS outputs. These seem important for the comparison.

Any ideas on how to properly compare these DWLS-estimated models? Thanks!

2

hey, i’ve run into similar issues. DWLS doesn’t give AIC/BIC, which complicates comparisons. have you tried using the DIFFTEST option in lavaan? it’s specifically for nested models with WLSMV. might be worth a shot. also, check if your models are truly nested - that could explain the error you’re getting.

3

Hey there! I’ve been dabbling with CFA models too, and I totally get your frustration. Have you considered using the lavTestLRT() function? It’s pretty nifty for comparing DWLS models.

But here’s a thought - why not look beyond just the model comparison? What’s the theoretical basis for your 4-factor vs 2-factor models? Sometimes, the stats can only tell us so much, you know?

Oh, and speaking of fit indices, have you checked out SRMR? It’s not as common, but I find it super helpful with ordinal data. Might give you some extra insight!

Just curious - what kind of data are you working with? I’m always fascinated by the real-world applications of these models. Care to share a bit about your project? It might help us brainstorm some more creative solutions!

4

I’ve encountered this challenge before when working with ordinal data and DWLS estimation. The ANOVA approach isn’t suitable here due to the different estimation method. Instead, you might want to consider using the scaled chi-square difference test, which is more appropriate for DWLS/WLSMV estimation.

In lavaan, you can implement this with the lavTestLRT() function. Here’s how you might approach it:

scaled_diff_test <- lavTestLRT(model_four, model_two)
print(scaled_diff_test)

This should give you a more reliable comparison between your nested models. Additionally, don’t forget to examine other fit indices like CFI, TLI, and RMSEA to get a comprehensive view of model fit. These can be particularly useful when dealing with non-nested models where the chi-square difference test isn’t applicable.

Lastly, if you’re interested in model comparison beyond nested models, you might want to look into alternative fit indices like WAIC or DIC, which can be calculated for DWLS models using Bayesian estimation techniques.