Can a Google Glass App Help Kids With Autism Read Emotions?

Google co-founder Sergey Brin wearing Google Class Explorer in 2012 in San Francisco. 

Photo by Kimihiro Hoshino/AFP/Getty Images

In his first year at Stanford, Catalin Voss helped build an app for Google Glass that could recognize emotions. If you put on Google’s computerized eyewear and looked at others in front of you, the software could tell you—based on their facial expressions—if they were happy, sad, surprised, or disgusted.

This face-and-eye-tracking technology was so effective that he and his three co-inventors sold it to a Japanese company called GAIA System Solutions, which is now working to put it into cars. The system could improve safety by identifying, say, when you’re falling asleep or when you’re looking the wrong way.

But at Stanford, Voss has bigger ambitions. He’s now working with Dennis Wall, a professor in the university’s school of medicine, to hone this technology into something useful for children diagnosed with autism. This was one of Voss’s original aims when he built Sension, the startup he sold to GAIA, but that didn’t fit the broader prospects of the business. So he spun off the idea as a project inside the medical school.

For Voss, Wall, and their colleague Nick Haber, a Stanford post-doc, the idea is that their Glass software will help autistic children recognize and understand facial expressions and, through them, emotions. It operates like a game or, as Voss calls it, an “interactive learning experience.” Through the Google Glass eyewear, children are asked to, say, find someone who is happy. When they look at someone who is smiling, the app recognizes this and awards “points.”

The system also records what the child does for later review. “You can plot, as they wear the glasses, how they’re improving, where they’re improving,” Wall says. “You can look at video to understand why.”

The group has tested the software in a lab with about 40 children. Now, they are starting a clinical trial with 100 kids at home, with parents driving the software (interested parents can sign up here). “First, we had make sure that a six-year-old would actually wear this,” Wall says. “We’ve been able to show that. But we’ve also been able to see change in these kids, at least in the lab.”

Steve Silberman, author of the book NeuroTribes: The Legacy of Autism and the Future of Neurodiversity and a frequent Wired contributor, points out, however, that the software may not be as effective as it appears. “This is not the first time people have tried to encourage [kids on the autism spectrum] to recognize emotions with what amounts of schematics of facial expressions,” he says. “The problem is that autism may not map the schematics—the simplified version of emotions—to more complex human expressions.”

The project isn’t the only effort to treat autism with Google Glass. Ned Sahin, a neuroscientist, runs a startup called Brain Power, which is also building Glass software in an effort to help autistic children learn at least some of the skills they need to interact with others. The company has been testing its software with children across the country and is planning its own clinical trial at Stanford, according to Sahin.

These technologies are still in the early stages, but they could have an enormous impact. According to the Centers for Disease Control and Prevention, autism affects about 1 in 68 children. “This is an enormous public health crisis,” says Jill Escher, president of the San Francisco Bay Area Autism Society. “Technology can certainly help us categorize autism and other mental disorders and understand them a little better.”

At Stanford, Wall is also working to improve the ability to diagnose autism in children at a younger age by using machine learning algorithms to analyze data collected on the behavior of autistic children, looking for the signals of the disorder. In the United States, Wall says, the average age of a child at diagnosis is 4½ years; he hopes to drive that to 2½ and perhaps ever younger.

But the bigger task lies in finding ways to help. “Early intervention is helpful,” Escher says. “But there is still no robust therapy for autism.” The Google Glass software built by people like Voss and Sahin could be a step in that direction.

Also in Wired:

Dell. EMC. HP. Cisco. These Tech Giants Are the Walking Dead

Her Code Got Humans on the Moon—And Invented Software Itself

Edward Snowden May Be the Most Powerful Person on Twitter

How to Kill the New iOS 9 Feature That’s Eating Your Data

What You Need to Know to Be Culturally Literate in 2016