Title: IMPACTS OF FACTOR ANALYTIC CHOICES ON SCALE INTERPRETATION COMMONLY USED TO ASSESS CHILDREN WITH AND WITHOUT AUTISM SPECTRUM DISORDER  

Abstract: Introduction: Autism Spectrum Disorder (ASD) measures are often designed as diagnostic tools but also used as outcome variables in research. This dual usage poses validation challenges since these tools are typically developed primarily for diagnostic accuracy, not as outcome measures. This issue is particularly pronounced for ordinal scales, where their use  as outcome variables requires different types of evidentiary support compared to their clinical and applied predictive purposes.

Objective: This study examines how various factor analytic decisions influence the perceived quality of ASD scales, using the Childhood Symptom Behavior Questionnaire (CSBQ) as a model, focusing on its development as an outcome variable.

Methods: Using responses from approximately 7,000 children in the CDC's Pathway's dataset, this study compares 4 factor analytic methods—Maximum Likelihood (ML), Categorical ML (cML), Diagonally Weighted Least Squares (DWLS), and Unweighted Least Squares (ULS). These methods are evaluated using fit indices in their regular, scale-shifted, and robust forms to determine their impact on model fit.

Results: Analysis revealed substantial variations in model fit across methods, with engagement in questionable “adjustments” (i.e., correlating errors) required to enhance fit. All approaches required minimal changes to achieve “adequate fit” for unadjusted models. Scale-shifted approaches required more “adjustments” and did not achieve adequate fit for ULS. Only ML achieved adequate fit with “adjustments” when using robust approaches.

Conclusion: Methodological decisions during factor analysis significantly impact the interpretations of clinical assessments used as outcomes, which are secondary to many tool’s primary purposes. Biostatisticians should become better acquainted with the methodological choices available when developing scales and be better equipped to identify questionable research practices used to enhance fit for marketed scales. As a deeply interdisciplinary field, public health researchers must keep abreast of the methodological cultures of allied health fields to ensure adopted measures align with immediate research purposes (e.g., as outcomes), and note limitations in relationship to evolving measurement practices.

Chair: Dr. Brian Barger