Portal de Periódicos da CAPES

Automatic highlights extraction for drama video using music emotion and human face features

2013; Elsevier BV; Volume: 119; Linguagem: Inglês

10.1016/j.neucom.2012.03.034

1872-8286

Keng-Sheng Lin, Ann Lee, Yi‐Hsuan Yang, Cheng-Te Lee, Homer H. Chen,

Human Motion and Animation

This paper describes a novel system that uses music emotion and human face as features for automatic highlights extraction for drama video. These high-level audiovisual features are used because music evokes emotion response from the viewer and characters express emotion on their faces. In addition, a novel scheme is developed to improve the accuracy of music emotion recognition in drama video. Specifically, emotion recognition is performed not on the input audio signal but on the noise-free music available from the album of the incidental music, with the presence of incidental music detected by an audio fingerprint technique. Besides the conventional subjective evaluation, we propose a new metric for quantitative performance evaluation of highlights extraction. Experiments conducted over four different types of drama videos demonstrate that the proposed system significantly outperforms baseline ones in terms of both subjective and objective measures.