Home video clips may help assess autism spectrum disorder in children one day, possibly reducing the wait time for diagnosis in the future, according to researchers who tested "feature tagging" on more than 200 videos of autistic and normally-developing young children.
In the study, Dennis Wall, PhD, of Stanford University School of Medicine in California, and colleagues tested eight machine-learning models for diagnosing autism from short videos. The videos were viewed by individuals previously untrained in autism diagnosis -- some were high school students -- who were given simple instructions on behaviors to look for.
Several of these models "performed well," the researchers wrote in , with one showing an area under the receiver-operating characteristic curve of 0.94 for correct diagnosis in children age 2-6.
"Across the United States, the average waiting list to get access to standard-of-care can last up to a year," Wall explained in a statement. "Using home videos for diagnosis has the potential to streamline the process and make it far more efficient."
Home videos also can capture a child's behavior in his or her natural environment, Wall noted: "The clinical environment can be stark and artificial, and can elicit atypical behaviors from kids."
Standard approaches to diagnosing autism evaluate between 20 and 100 behaviors and take several hours to complete. Currently, when children are ages ≥4 years, but diagnoses as early as age 2 years can be valid, according to the CDC.
Each of the models tested in the current study employed algorithms incorporating five to 12 features of children's behavior, producing an overall numerical score for each child.
Families who had been recruited through social media and internet groups submitted brief home videos for review. A total of 116 videos of children with autism (average age 4 years, 10 months) and 46 videos of typically developing children (average age 2 years, 11 months) met the study's criteria: they were 1 to 5 minutes long, showed the child's face and hands, showed direct social engagement or opportunities for engagement, and showed opportunities for use of objects such as toys, crayons, or utensils.
Three non-expert raters who were blind to each child's diagnosis received brief instructions about how to evaluate each video. The raters answered 30 yes/no questions about behavioral characteristics culled from standard autism screening tools, then fed those answers into the models. Watching and scoring the videos took the raters a median time of 4 minutes each.
One model, a logistic regression model that used five behavioral characteristics, showed an accuracy of 88.9%, sensitivity of 94.5%, and specificity of 77.4%. A prospectively collected independent validation set of 66 videos (33 of children with autism and 33 of children without) showed that model again performed best with sensitivity of 87.8% and specificity of 72.7%.
The research is still early; prospective testing in general pediatric settings with children who have not been diagnosed is needed, Wall and colleagues said. But results so far support the possibility that video analysis might be able to reduce waiting periods for diagnoses and reach underserved populations in the future, Wall noted.
"We showed that we can identify a small set of behavioral features that have high alignment with the clinical outcome, that non-experts can rapidly and independently score these features in a virtual environment online in minutes, and that the model we used to combine those features is effective in producing a score that matches the clinical outcome," he said.
Disclosures
The study is supported by the NIH, Hartwell Foundation, Bill and Melinda Gates Foundation, Coulter Foundation, Lucile Packard Foundation, and program grants from Stanford University's Human Centered Artificial Intelligence Program, Precision Health and Integrated Diagnostics Center (PHIND), Beckman Center, Bio-X Center, Predictives and Diagnostics Accelerator, and the Child Health Research Institute.
Wall and co-authors disclosed support from Bobby Dekesyer and Peter Sullivan. Wall disclosed a relevant relationship with Cognoa.
Primary Source
PLOS Medicine
Tariq Q, et al "Mobile detection of autism through machine learning on home video: A development and prospective validation study" PLOS Medicine 2018; DOI:10.1371/journal.pmed.1002705.