Select your localized edition:

Close ×

More Ways to Connect

Discover one of our 28 local entrepreneurial communities »

Be the first to know as we launch in new countries and markets around the globe.

Interested in bringing MIT Technology Review to your local market?

MIT Technology ReviewMIT Technology Review - logo

 

Now Available: Innovators Under 35 2013 See The 2013 List »

35 Innovators Under 35

Rana el Kaliouby, 34

Teaching devices to tell a frown from a smile

Affectiva

Photographs by Webb Chappell

Computers are good with information—but oblivious to our feelings. That’s a real shortcoming, believes MIT Media Lab scientist Rana el Kaliouby, because it leaves them unable to usefully respond to many of our needs until we take the trouble to tap out instructions. To close that gap, el Kaliouby has come up with technologies that help computers recognize facial expressions and other physical indicators of how someone is feeling. Someday this could help make our machines more adept at assisting us.

El Kaliouby is not the first researcher to try to map facial expressions. But where others have focused on trying to get computers to recognize a half-dozen exaggerated expressions recorded in the lab, she is identifying the more varied and subtle faces that people commonly make. “It’s a problem that requires pushing the state of the art of computer vision and machine learning,” she says.

To break the problem down, she zeroed in on 24 “landmarks” on the face. Then she trained a computer to identify how those parts of the face change shape in response to different emotions, creating expressions such as a furrowed brow. To ensure that the technology would work with people in different cultures, el Kaliouby, who lives in Cairo and spends one week a month at MIT, enlisted the help of thousands of people on six continents. They have allowed their computers’ embedded cameras to record their expressions while they watch a video, resulting in what she says is the largest database of facial images in the world.

One early experimental application of the technology was a set of camera-equipped glasses intended for people with Asperger’s syndrome, who tend to have difficulty recognizing others’ emotional states. The device could recognize whether someone facing the wearer appeared bored; if so, it could use small lights in the glasses to signal that to the wearer. (El Kaliouby herself was known to sport a head-cam in and out of the lab, tucked into the head scarf she wears.)

El Kaliouby has cofounded a company called Affectiva in Waltham, Massachusetts, to commercialize the facial recognition technology and a wristband that she helped develop to measure skin conductance, which is associated with emotional arousal and can be used to detect anxiety in real time. For now, Affectiva uses facial recognition mainly to give advertisers a better sense of how their ads are affecting viewers. The company convenes enormous virtual focus groups made up of online viewers who allow their expressions to be tracked, and then analyzes the resulting data. But in the longer term, el Kaliouby also wants to bring her technology to classrooms to help teachers identify which material students respond to best.

The technology could eventually become a critical component of many electronic devices, making it possible for them to recognize when we’re puzzled, frustrated, happy, or sad—and enabling them to respond with the right information, music, or human assistance. And there’s a lot to be said for getting our phones, PCs, and GPS systems to recognize when we just want to be left alone.

Karen Weintraub

2012 TR35 Winners

Rana el Kaliouby

Teaching devices to tell a frown from a smile

Saikat Guha (video)

Letting advertisers send targeted pitches to your mobile phone without ever seeing your personal information

Chris Harrison (video)

Liberating us from the touch screen by turning skin and objects into input devices

John Hering

Securing our smartphones from spyware and rogue apps, with a little help from the crowds

Drew Houston

Hiding all the complexities of remote file storage behind a small blue box

Ren Ng (video)

By tracking the direction of light, a camera takes pictures that can be refocused on different objects in a scene

Hossein Rahnama (video)

Mobile apps that tell you what you need to know before you have to ask

Leila Takayama (video)

Applying the tools of social science to make robots easier to live and work with

Eben Upton

His ultracheap computer is perfect for tinkering

Andreas Velten (video)

Spotting tiny problems with help from an ultrafast camera

See This Years' Winners

More Innovators Under 35

Advertisement