Behavioral Research Blog

Comparing machine emotion recognition with a human benchmark

Posted by Annelies Verkerk on Sep 6, 2013

Our emotions come across clear in our facial expressions. Due to this, facial expressions can be used in a wide variety of studies (to identify the reaction of a customer to a product, for instance). It has become a large part of fields such as consumer research and human-computer interaction. FaceReader is an example of an automated tool for facial expression analysis. It provides researchers with an objective assessment of a subject’s emotion based on key points found in the face.

Why this study?
Machine emotion recognition is still fairly new. In this study, Jansen et al. chose to benchmark machine emotion recognition with human performance. They chose to do this based on four reasons. To start, humans are able to recognize and report the emotions they see in an easy-to-understand way. Also, other uses of artificial intelligence have been benchmarked by human performance. Thirdly, humans are considered superior when it comes to performing mental tasks. Finally, success in artificial intelligence applications is measured by if they are able to beat human performance.

Read More

Topics: emotion recognition, FaceReader, facial expression analysis, Human-Computer Interaction

Facial expression analysis: the future of research in advertising, education, and more

Posted by Annelies Verkerk on Aug 23, 2013

A reaction to a product in a store or an assignment in the classroom can tell a lot about the effectiveness of the advertising or the learning method. These reactions can be studied using facial expression analysis, a research method which looks at the facial expression of a person to determine their emotional reaction to some sort of cue. Of course, facial expression analysis is not only limited to these settings – anywhere there is human interaction (with another human or with an object), there is the potential to gain valuable insight on consumer behaviors, emotional reactions, and much more.

FaceReader is software for automated analysis of facial expressions, which gives an objective assessment of a person’s emotion. It can increase the accuracy and reliability of research using facial expression analysis, and make data collection more effective and efficient. On our Behavioral Research Blog, we have published numerous blogs where FaceReader was used for facial expression analysis. Here are a few of them:

  • Terzis et al. discovered that empathetic and emotional feedback helped make e-learning easier. They used an avatar which emulated the student’s emotional response to an assessment if the FaceReader analysis and the human judges’ analysis matched. The empathetic avatar would match the emotion and then encourage the student. Students with the avatar found the material as easier to learn and more enjoyable and useful than those without the avatar. Read more...
  • Relational Agents (RA) are human-like avatars on the computer that are used to interact with humans. These RAs promote social bonds forming over the long term. The avatar created can be used in many ways, such as in video games, health care, or even teaching. Learn more!
  • Advertisements are designed to make us react to them. Lewinski et al. showed that specific patterns of facial expressions can partially explain the effectiveness of the advertisement in amusing persuasive video stimuli. These researchers used FaceReader to analyze facial reactions to three videos with different levels of how amusing they were. Interested? Read more...
  • An objective assessment can often be more accurate than even the participant’s personal assessment. Danner et al. tested people’s reactions to different orange juices and used FaceReader to analyze their emotional state. To read about the results, click here.
  • Drape et al. explored technology integration in the classroom. To do this, they used numerous techniques including observations, qualitative interviews, and facial expression analysis. Facial expression analysis was done with FaceReader to analyze non-verbal behavior of the students when they watched a video on their computer. Click here to learn more.
Read More

Topics: emotion recognition, on-site research, video observation, FaceReader, facial expression analysis, Neuromarketing, Human-Computer Interaction

Making a hotel reservation - Where should you go this summer?

Posted by Annelies Verkerk on Jul 29, 2013

Using facial expression analysis to study decision making processes

Where should you go this summer, who should you go with, and for how long? Which hotel or hostel to book? People around the world are currently making hotel reservations for their summer vacation. Are you one of them?

Read More

Topics: emotion recognition, FaceReader, facial expression analysis, consumer behavior, consumer behavior research, Human-Computer Interaction

Relational Agents: the use and testing of avatars for HCI

Posted by Annelies Verkerk on Jul 16, 2013

Relational Agents (RAs) are computer constructs designed to interact with humans in a way that promotes the long-term formation of social and emotional bonds. Integrating social psychological principles of relationship building such as empathy, shared self-disclosure and emotional feedback into the RA interface helps to establish these bonds. Relational agents are generally created with an anthropomorphic (i.e. human-like) form that may be more or less realistic, varying from cartoon-like two-dimensional drawings to animated three-dimensional models developed with advanced graphics software. This type of model, or avatar, is known to most as a character representation found in third-person video games in which the game player character is visible. However, RAs are usually limited to frontal views from the shoulders or neck up due to constraints imposed by limits on computer processing power and the simulation capabilities of artificial intelligence algorithms. The video shown here was created as a final project for a Motion Capture class at New York University, and is intended to illustrate the potential uses of RAs as well as methods for testing the acceptance of such agents by their target audience.

Read More

Topics: emotion recognition, FaceReader, facial expression analysis, avatar, Human-Computer Interaction

Subscribe to Email Updates

Posts by Topic

see all