Papers: [ACII 2017] [IEEE TAC 2019] [Psychiatry Research 2018] [IJCV 2019]
Xin Lu, Reginald B. Adams, Jr., Jia Li, Michelle G. Newman, James Z. Wang
The Pennsylvania State University, USA
Prior computational studies have examined hundreds of visual characteristics related to color, texture, and composition in an attempt to predict human emotional responses. Beyond those myriad features examined in computer science, roundness, angularity, and visual complexity have also been found to evoke emotions in human perceivers, as demonstrated in psychological studies of facial expressions, dance poses, and even simple synthetic visual patterns. Capturing these characteristics algorithmically to incorporate in computational studies, however, has proven difficult. Here we expand the scope of previous computer vision work by examining these three visual characteristics in computer analysis of complex scenes, and compare the results to the hundreds of visual qualities previously examined. A large collection of ecologically valid stimuli (i.e., photos that humans regularly encounter on the web), named the EmoSet and containing more than 40,000 images crawled from web albums, was generated using crowdsourcing and subjected to human subject emotion ratings. We developed computational methods to the separate indices of roundness, angularity, and complexity, thereby establishing three new computational constructs. Critically, these three new physically interpretable visual constructs achieve comparable classification accuracy to the hundreds of shape, texture, composition, and facial feature characteristics previously examined. In addition, our experimental results show that color features related most strongly with the positivity of perceived emotions, the texture features related more to calmness or excitement, and roundness, angularity, and simplicity related similarly with both of these emotions dimensions.
Full Paper
Dataset: To be made available.Citation: Xin Lu, Reginald B. Adams, Jr., Jia Li, Michelle G. Newman and James Z. Wang, ``An Investigation into Three Visual Characteristics of Complex Scenes that Evoke Human Emotion,'' Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 440-447, San Antonio, Texas, October 2017.
© 2017 All rights reserved.