This AI System Can Diagnose Depression From Instagram Photos
Researchers created a machine-learning algorithm that was able to identify clinical depression with 70 percent accuracy, while human general practitioners achieved a 42 percent success rate.
Heads up Instagram users — the pictures you post may be more revealing than you think.
This week, researchers at Harvard University and the University of Vermont released a study that suggests artificial intelligence systems can identify a depressed individual simply by looking at their Instagram photos.
According to the researchers, their algorithm has a 70 percent success rate when determining which Instagram users have been diagnosed as clinically depressed within the last three years. By comparison, general practitioners have about a 42 percent success rate when diagnosing depression through in-person evaluations. While those figures aren't particularly rigorous from a statistical point of view — more on that in a bit — they do suggest that AI could be useful in screening for clinical depression.
“Increasingly, physicians look to a variety of digital medical records to help inform patient assessments,” said co-author Andrew Reece, a postdoctoral researcher in Harvard University's department of psychology. “Patients might someday opt-in to share their social media feeds with doctors for health screening purposes. Imagine a yearly checkup where the doctor holds a tablet showing various health metric feeds, customized for that specific patient.”
Reece and co-author Chris Danforth, professor of mathematics and statistics at the University of Vermont, developed the AI system, which analyzes criteria such as color, shading, and whether or not a smartphone user touched up their image with a filter. Using previous psychological research regarding depression and color choices, the AI system initially sorts for warm and bright colors versus more muted tones.
The AI system also looks at comments and “likes” received and uses a face recognition algorithm to automatically identify people in each photo.
“We also looked at which, if any, filters were used to adjust the appearance of each image,” Reece said. “For example, depressed people preferred a black-and-white filter, called Inkwell. Healthy people preferred to make their images appear warmer and brighter, using a filter called Valencia.”
The 166 test subjects were selected so that approximately half had been clinically diagnosed with depression within the last three years. When the AI system sorted through the photos associated with each individual, it was able to accurately classify 70 percent of the subjects diagnosed with depression.
The team's research was initially published last year in the open-access online journalarXiv, which often serves as a pre-print outlet for scientists to get word out about their research. The study, funded by the National Science Foundation, was officially published this week in the peer-reviewed journal EPJ Data Science.
Reece said it's important to note that the comparison to general practitioner accuracy — 42 percent — is meant only as an informal comparison. The rate of accuracy is based on initial screenings in which the doctor doesn't have access to any scales or questionnaires typically used in mental health screening. The number also uses a different statistical model than the one employed in the Instagram study.
“It's not really fair to make a direct comparison between our model's success rate and that of GPs,” Reece said. “Doctors have a much tougher job to do, for one thing. Another more substantive issue is that we didn't make our assessments with the same sample of individuals. To really be able to compare accuracy, we'd need to have GPs assess the same Instagram users who participated in our study, and then see how our results measured up.”
That said, Reece contends it's useful to compare the two numbers when considering alternative methods for those initial, first-pass assessments on depression. If a computer program can flag depressive symptoms with 70 percent accuracy, it could help medical professionals do their jobs down the line.
“An alert might pop up saying something like, ‘Ask extra questions about depression, Instagram activity over the last 4 weeks indicates a moderate probability of depression,’” Reece said. “That's a long way off from the prototype we've developed here. But it's nice to think that people's own social media data might be used to help them, rather than simply to target them with ads and marketing.”
WATCH: How Instagram Fuels the Urban Explorer Movement