Real-time multimodal affect recognition in laughter episodes

Date of Publication

2013

Document Type

Master's Thesis

Degree Name

Master of Science in Computer Science

College

College of Computer Studies

Department/Unit

Computer Science

Thesis Adviser

Merlin Suarez

Abstract/Summary

Emotion recognition is a widely studied subject, due to its importance in human interaction and decision making. The recognition of emotion in laughter is particularly important as laughter can identify non-basic affective states such as distress, anxiety, and boredom. Existing systems are unable to classify the emotion of laughter in real-time, however. This research proposes a real-time multimodal affect recognition system for laughter episodes, using facial expressions and voiced laughter as modalities.

The system takes a video stream as input. The video stream can be either a web camera with a microphone attached for audio, or a video file. As laughter take place over a period of time, rather than frame-by-frame, the system will segment the stream into different windows of 1.62 seconds in length. Within the window, image and audio data are extracted, and the AUs in the apex of the window are detected. At the end of each window, the pitch and MFCC values of the audio data collected within the window are computed, and decision-level fusion is applied to the audio and face features. The resulting features are then be passed to the emotion recognition model, which then produces the final valence and arousal values of the window.

The emotion recognition model was able to achieve a correlation coefficient of 0.68 for valence and 0.61 for arousal using the Semaine corpus, and 0.75 for valence and 0.83 for arousal using the Pinoy Laughter 2 corpus. The overhead for the whole emotion recognition process is 610.98 ms, however the overhead will be hard to completely eliminate due to the high number of processes required to perform emotion recognition.

Abstract Format

html

Language

English

Format

Print

Accession Number

TG05360

Shelf Location

Archives, The Learning Commons, 12F Henry Sy Sr. Hall

Physical Description

vi, 64 leaves ; 28 cm.

Keywords

Emotion recognition; Laughter

This document is currently not available here.

Share

COinS