Forget emojis – here's how to show emotion digitally | Information Age

Coming to a phone near you?

By Staff Writers on Jul 21 2015 02:45 PM

Affectiva's algorithm is put to the test. (Credit: TED)

It might be the year that emojis officially went mainstream but 2015 is hardly shaping up as a pinnacle for representing how we feel when communicating digitally.

For Rana el Kaliouby, founder of MIT technology spinout Affectiva, the rise of the emoji is perhaps an antithesis for what’s possible in representing our emotional state in the online world.

“We're living more and more of our lives … in a world that's devoid of emotion,” el Kaliouby said in a recent TED talk.

“As more and more of our lives become digital, we are fighting a losing battle trying to curb our usage of devices in order to reclaim our emotions.

“What I'm trying to do instead is to bring emotions into our technology and make our technologies more responsive. I want to bring emotions back into our digital experiences.”

Affectiva has created an algorithm that uses a tablet or phone’s front-facing camera to recognise its user’s facial expressions – and therefore how they feel at the time.

“Our human face happens to be one of the most powerful channels that we all use to communicate social and emotional states, everything from enjoyment, surprise, empathy and curiosity,” el Kaliouby said.

The algorithm recognises 45 facial muscle movements that can be combined hundreds of ways to express various emotions.

Its ability to recognise emotions was initially backed by “deep learning” using “tens of thousands of examples of people” displaying each emotion type. This enabled the algorithm to recognise subtleties of expression - such as the difference between a smirk and a smile.

“The smile and the smirk look somewhat similar, but they mean very different things,” el Kaliouby said.

“It's important for a computer to be able to tell the difference between the two expressions.”

The accuracy of the algorithm has significantly improved over time as it is exposed to more test data.

“So far, we have amassed 12 billion of these emotion data points - it's the largest emotion database in the world,” el Kaliouby said.

“We've collected it from 2.9 million face videos, people who have agreed to share their emotions with us, and from 75 countries around the world. It's growing every day.

“It blows my mind away that we can now quantify something as personal as our emotions, and we can do it at this scale.”

El Kaliouby sees a number of potential applications for the technology – not least being able to communicate emotions more naturally when conversing through a computer.

"I think five years down the line, all our devices are going to have an emotion chip, and we won't remember what it was like when we couldn't just frown at our device and our device would say, 'Hmm, you didn't like that, did you?'," she said.

She also raises the prospect of “emotion-enabled wearable glasses can help individuals who are visually impaired read the faces of others, [or] help individuals on the autism spectrum interpret emotion, something that they really struggle with".

“[Or] in education, imagine if your learning apps sense that you're confused and slow down, or that you're bored, so it's sped up, just like a great teacher would in a classroom.”

The company offers software development kits (SDKs) and application programming interfaces (APIs) for developers to play with, and also a live demonstration tracking emotive responses to advertising.

Tags: