Affect Display Recognition Through Tactile and Visual Stimuli in a Social Robot

New technologies are nowadays an important part of human communication and interaction. While text, facial, and voice recognition have become increasingly fluid in recent years, thanks to the development of machine learning algorithms, recognising and expressing sensations or moods via multimodal recognition is a field that the literature could further explore. This situation introduces a new challenge to social robots. In this work, the authors study how a combination of visual and tactile stimuli influences people’s perceptions of affect display and seeks to apply these findings to a social robot. In the experiments, the subjects had to determine the perceived valence and arousal of simultaneously being exposed to the two stimuli mentioned above. The analysis revealed that the combination of touch and facial expression significantly influences the valence and arousal perceived by users. Based on these findings, this work includes an application for the robot to determine the user’s affect display in real-time.