Behavioral Research Blog

Empathetic and emotional feedback in e-learning: analyzing facial expressions

Posted by Annelies Verkerk on Nov 16, 2012

Follow a course onlineComputerized learning tools have already become a standard educational tool in many institutions. The break-through point was when many leading universities joined up on common platforms and offered so-called massive open online course (MOOC) such as https://www.coursera.org/universities. The courses - open for everyone - have thousands of participants. In e-learning courses, participants receive the semi-individualized feedback only thanks to the sophisticated computer algorithms. The algorithms employ the patterns of characteristics of correct and incorrect answers. Additionally, keyboard- and mouse- movements are measured to provide feedback. This entire instantaneous assessment relies on the logical and rational input while unintentionally omitting the affective and emotional factors. However, computer scientists and psychologists have recently developed tools to automatically assess and analyze patterns of emotions in the participants of the computerized courses. A computer – equipped with a standard webcam - may analyze in-real time an emotional state of the participant who is taking an online course. Research proves that such automated affective assessment improves learning outcomes due to the enriched feedback it can provide.

The scientists from University of Macedonia, Greece [1] found that empathetic and emotional feedback facilitates computerized learning and assessment (e-learning).  A presence of digital avatar that responds to a student’s emotional state increases the perceived: usefulness, ease of use and playfulness of the studied material. The researchers used FaceReader [2] - software that automatically recognize facial expressions of emotions - to prove that instantaneously assessing participants’ emotions helps in the learning process. In the experiment, the digital female avatar showed corresponding empathetic feedback, only if FaceReader and the independent human judges agreed with each other. If they both agreed that participant looked “sad” then the female digital avatar showed sad face –empathizing with the student– and then she smiled and said encouragingly “cheer up, continue trying and you will succeed.” Students that learnt the material with empathetic and emotional avatar perceived the material as more easy to learn, enjoyable and useful than the control group.

FaceReader can recognize six basic emotions – happiness, sadness, surprise, fear, anger, disgust – and a neutral face. In the learning context, smiling may indicate feeling of an accomplishment whereas the negative emotions - sad or scared - indicate feelings of frustration. Therefore, in the experiment, the digital female avatar praised the participant when the software detected happiness. However if FaceReader detected sadness or fear the avatar tried to cheer up the participant. Additionally expressing understanding and empathy is crucial in the communication. Therefore, before speaking the female avatar sympathized with the student by mirroring the same facial expressions of emotion. If participant emotional response was happiness avatar smiled and then showed neutral face. However when the student was sad or fearful, the avatar mimicked the same emotion and then smiled at participant and tried to cheer him up.

The researchers use the software that automatically assesses human facial behavior and interprets emotions in many different contexts. Helping students to learn is one of them, but facial expressions analysis helps in research of - politics (analyzing presidential debate); - consumer behavior (measuring food preferences); or psychotherapy (eliciting adequate emotions).

By Peter Lewinski
Marie Curie Research Fellow
VicarVision, Amsterdam NL

  1. Terzis, V.; Moridis, C. N.; Economides, A.A. (2012). The effect of emotional feedback on behavioral intention to use computer based assessment. Computers & Education, 59 (2), 710–721.
  2. FaceReader: http://www.noldus.com/human-behavior-research/products/facereader

Topics: FaceReader, facial expression analysis, Educational research, methods and techniques, human behavior research

Subscribe to Email Updates

Posts by Topic

see all