Measuring Treatment Outcomes: Apples, Oranges, Rutabagas, Nachos
Uncertainty is a major theme of The Politics of Autism. Here is how I start chapter 3:
If we can land a man on the moon, why can’t we cure autism? Frustrated parents may ask that question, remembering that when John F. Kennedy committed the United States to go to the moon, NASA scientists and engineers figured out how to get there. Ever since Neil Armstrong stepped off the lunar module in 1969, politicians have held up the Apollo project as a model for solving all kinds of problems. But autism is not rocket science. Contrary to the usual meaning of that expression, I hardly suggest that autism science is simple; rather, it is more puzzling than rocket science.
When the moon program was getting under way, there was consensus about the fundamental terms and facts. Although the engineering details were challenging, the basic math and physics behind the mission dated back to Isaac Newton. Autism is different. As we have already seen, it is a contested concept with many uncertainties. Just picture an Apollo program in which experts saw different kinds of moons in different parts of the sky and were not quite sure about the laws of motion.
Umberto Provenzani and colleagues have an article at Autism titled "What Are We Targeting When We Treat Autism Spectrum Disorder? A Systematic Review Of 406 Clinical Trials." The abstract:
The number of trials aimed at evaluating treatments for autism spectrum disorder has been increasing progressively. However, it is not clear which outcome measures should be used to assess their efficacy, especially for treatments which target core symptoms. The present review aimed to provide a comprehensive overview regarding the outcome measures used in clinical trials for people with autism spectrum disorder. We systematically searched the Web of KnowledgeSM database between 1980 and 2016 to identify published controlled trials investigating the efficacy of interventions in autism spectrum disorder. We included 406 trials in the final database, from which a total of 327 outcome measures were identified. Only seven scales were used in more than 5% of the studies, among which only three measured core symptoms (Autism Diagnostic Observation Schedule, Childhood Autism Rating Scale, and Social Responsiveness Scale). Of note, 69% of the tools were used in the literature only once. Our systematic review has shown that the evaluation of efficacy in intervention trials for autism spectrum disorder relies on heterogeneous and often non-specific tools for this condition. The fragmentation of tools may significantly hamper the comparisons between studies and thus the discovery of effective treatments for autism spectrum disorder. Greater consensus regarding the choice of these measures should be reached.
From the article:
Our findings also suggest that, given the evolving and still unclear phenomenology of ASD, there may be no single tool capable of detecting and assessing changes in symptoms and behaviours. Therefore, in the absence of an overarching consensus regarding reliable standardized measures, the use of multiple instruments (together with the CGI) [Clinical Global Impression] could be recommended. Conversely, the efficacy of treatments on some important domains which are frequently impaired in autistic people (e.g. quality of life) demand further investigation.
