Frustration in the face of the driver: A simulator study on facial muscle activity during frustrated driving

Frustration in the face of the driver: A simulator study on facial muscle activity during frustrated driving. Klas Ihme, Christina Dömeland, Maria Freese, Meike Jipp. Interaction Studies, Volume 19, Issue 3, Dec 2018, p. 487 - 498. https://doi.org/10.1075/is.17005.ihm

Abstract: Frustration in traffic is one of the causes of aggressive driving. Knowledge whether a driver is frustrated may be utilized by future advanced driver assistance systems to counteract this source of crashes. One possibility to achieve this is to automatically recognize facial expressions of drivers. However, only little is known about the facial expressions of frustrated drivers. Here, we report the results of a driving simulator study investigating the facial muscle activity that comes along with frustration. Twenty-eight participants were video-taped during frustrated and non-frustrated driving situations. Their facial muscle activity was manually coded according to the Facial Action Coding System. Participants showed significantly more facial muscle activity in the mouth region. Thus, recording facial muscle behavior potentially provides traffic researchers and assistance system developers with the possibility to recognize frustration while driving.

Keyword(s): driving simulator , Facial Action Coding System , facial expressions and frustration

Check also Recognizing Frustration of Drivers From Face Video Recordings and Brain Activation Measurements With Functional Near-Infrared Spectroscopy, Klas Ihme et al. Front. Hum. Neurosci., August 17 2018. https://doi.org/10.3389/fnhum.2018.00327

Abstract: Experiencing frustration while driving can harm cognitive processing, result in aggressive behavior and hence negatively influence driving performance and traffic safety. Being able to automatically detect frustration would allow adaptive driver assistance and automation systems to adequately react to a driver’s frustration and mitigate potential negative consequences. To identify reliable and valid indicators of driver’s frustration, we conducted two driving simulator experiments. In the first experiment, we aimed to reveal facial expressions that indicate frustration in continuous video recordings of the driver’s face taken while driving highly realistic simulator scenarios in which frustrated or non-frustrated emotional states were experienced. An automated analysis of facial expressions combined with multivariate logistic regression classification revealed that frustrated time intervals can be discriminated from non-frustrated ones with accuracy of 62.0% (mean over 30 participants). A further analysis of the facial expressions revealed that frustrated drivers tend to activate muscles in the mouth region (chin raiser, lip pucker, lip pressor). In the second experiment, we measured cortical activation with almost whole-head functional near-infrared spectroscopy (fNIRS) while participants experienced frustrating and non-frustrating driving simulator scenarios. Multivariate logistic regression applied to the fNIRS measurements allowed us to discriminate between frustrated and non-frustrated driving intervals with higher accuracy of 78.1% (mean over 12 participants). Frustrated driving intervals were indicated by increased activation in the inferior frontal, putative premotor and occipito-temporal cortices. Our results show that facial and cortical markers of frustration can be informative for time resolved driver state identification in complex realistic driving situations. The markers derived here can potentially be used as an input for future adaptive driver assistance and automation systems that detect driver frustration and adaptively react to mitigate it.