Laser Microbeam and Medical Program    »     People  |   Education & Dissemination  |   Research & Resources  |   Publications

Non-contact emotion monitoring based on physics and physiology based-image analysis

Norimichi Tsumura, PhD

Non-contact video based heart rate estimation has many great potential in areas such as medical, health, and affective computing. Recently, it is proposed to use digital camera for remote heart rate (HR) and heart rate variability (HRV) measurement. From HRV spectrogram, we can visualize sympathetic nerve system. However, the accuracy of these methods are not enough to calculate heart rate variability spectrogram (HRVS).
In this talk, we propose the method of HR and HRV spectrogram estimation by analyzing extracted hemoglobin concentration from facial color images based on the biomedical knowledge and knowledge on optics. Our method does not require any special camera. In addition, stable presumption is possible even under the ambient light sources. As a result, we could get HR around 99% accuracy and could get HRV spectrogram with the same accuracy of electrocardiogram. Therefore we could identify if participants were relaxed or stressed using conventional DSLR camera. Using 27 kind of features calculated from the physiological signals from the camera, we also classified five different emotional states (amusement, anger, disgust, sadness and surprise). As a result, we could classify emotions with 94% accuracy by selecting features.

[ Back ]

sponsored by

The Laser Microbeam and Medical Program (LAMMP)
a NIH biotechnology resource facility at the Beckman Laser Institute

LAMMP Seminars

View selected seminars online

Education & Dissemination

Contact Us

  • Hanna Kim
    Resource Coordinator

Supported by