«1 Abstract: Emotional reactions to basic, artificial, yet carefully controllable point-light displays (PLDs) were investigated with ratings of ...»
Abstract: Emotional reactions to basic, artificial, yet carefully controllable point-light displays
(PLDs) were investigated with ratings of valence, arousal, approachability, and dominance. PLDs
were varied by movement location (upper and lower) and intensity (10°, 20°, and 30° angular
change) for angular upward and downward movements. Half of participants (N=28) were told that
PLDs were related to face while to other half nothing was hinted. Results showed that 20° and 30°
angle lower location upward movements were rated as significantly more pleasant, relaxing, and approachable than corresponding upper location downward movements. Informed participants rated 20° and 30° angle lower movements as significantly less dominant than corresponding upper movements. Results are important from many perspectives, like for understanding human perceptual mechanisms and mediation of information with low bandwidth and computational requirements because only small amount of information is required for transition and presentation of information.
Author keywords: Point-light displays; biological movement; biological motion; emotions; face perception; human-computer interaction ACM classification keywords: Laboratory experiments; Information visualization; Psychology
- The aim of the experiment was to investigate emotional reactions to basic, artificial, yet carefully controllable point-light display (PLD) animations with the ratings of valence, arousal, approachability, and dominance.
- It was found that the PLDs convey emotional information.
- The knowledge of PLD processing can be valuable both for basic research and for humancomputer interaction applications.
This article has been accepted for publication in Interacting with Computers Published by Oxford University Press.
This is the author version prior the minor revision. The final draft will be available later, according the publisher's policy.
1. INTRODUCTION Point-light displays (PLDs) of biological movement patterns refer to the means of presenting complex information by relatively simple dot-based representations of original source of information. For example, a moving human body can be represented by a group of a few moving dots (Johansson, 1973). There is evidence that humans are able to distinguish biological movement patterns of human movements from other patterns of motion signals, like random movement of dots or movement of other mammals than humans. Johansson (1973) studied the perceptual capacity of recognising human movements using only PLDs of human walkers. The stimulus material of the experiments consisted of videos of moving persons, videotaped against a black background in black clothing so that only reflecting material attached on the limbs was visible while the actors moved.
The participants were able to recognise spontaneously that the moving group of dots was a human walker. Further, people often associate movements, like PLD movements, to be related to social communication. Bassili (1976) found that temporal and spatial changes in the movements of two circles had an effect on participants’ ratings on whether or not the circles were interacting with each other. For the explanations about the interaction participants used words like “chasing” and “following”.
PLDs have also been used to study the recognition of face related information, including gender recognition, facial expression recognition, emotion recognition development, narrowed recognition in disorders, etc. Gender recognition studies have been made to understand similarities and differences between children and adults in facial expression recognition (Berry, 1991), and also to compare differences between PLD face representations and real face presentations in the recognition tasks (Hill, Jinno, & Johnston, 2003). Emotional information processing has been studied by PLD in schizophrenia (Tomlinson et al., 2006), and infant development (Doi et al., 2008).
Further, PLDs have been used in some brain-imaging studies to understand which regions of the brains are responsible of processing the facial expression information (Atkinson, Vuong, & Smithson, 2012; Ichikawa et al., 2010).
In principle, PLDs offer means to investigate basic human perceptual processes. Bassili (1978; 1979) hypothesised that the movement of facial expressions is a key factor in evoking both the perception of a human face and the emotion related information of the face. In a series of studies by Bassili (1978; 1979), the usual features of the face, like eyes, nose, and mouth, were omitted, and the facial movement was shown with the PLD technique. PLDs showing emotional facial expressions were made by real human actors by blackening the face and attaching contrasting spots on the face so that the videotaped expressions appeared only as series of movements of the spots. The participants watched at the emotional facial expressions videotaped in a PLD condition and in a real face condition. The results showed that recognition errors of emotional facial expressions were similar between the PLD condition and the real face condition. Additionally, a series of non-human movements were constructed with otherwise the same technique as the PLDs of human facial expressions, but the spots were placed on a piece of foam and the movement was created by crushing the foam. The results showed that the participants distinguished the facial movements from the movements made with the foam. Further, the importance of different parts of the face in the facial expression recognition was investigated in both face-related conditions. It was found that visible lower part of the face was needed for the recognition of happiness, sadness, and disgust.
Visible upper part of the face was needed for the recognition of surprise and anger.
Pollick et al. (2003) studied the PLD visualisation of naturalistic emotional expressions of anger, happiness, sadness, and surprise. They varied the visualisation motion by spatial exaggeration and timing of the movements, and measured the effects of variations to the recognition and intensity ratings. They found that the manipulation of the spatial exaggeration had an effect on both the emotion recognition and the intensity ratings, whereas the manipulation of the movement timing had a small effect on the ratings of emotion intensity. Afzal et al. (2009) generated three different sets of animations from four datasets of facial emotional expressions to investigate how a certain set of feature points can convey affective content. In their animations, the same facial expressions were shown either by a PLD animation, a stick figure animation, or a 3D avatar. Interestingly, they found that the recognition rate of the expressions was higher with simple PLDs than with the 3D avatar.
Matsuzaki and Sato (2008) investigated the critical number of dots needed for the recognition of the expressions of anger, happiness, sadness, and surprise with stationary dot patterns. The stimuli were generated from facial images. They varied the amount of dots to be 10, 14, 18, or 34, and found that the recognition rate of anger, happiness, and surprise improved when the number of dots increased from 10 to 18 dots. Increasing the dots up to 34 did not improve the recognition any further. In the case of sad expression, the recognition rate improved throughout the whole range of the amount of dots. Further, they investigated the recognition of the same expressions with 18 or less dots in two conditions. In one condition, called as a repetitive condition, a group of dots in an expressive representation was shown twice. In another condition, called as an apparent motion condition, the participants firstly saw a group of dots in a neutral representation, following a group of dots in an expressive representation. In their study, Matsuzaki and Sato found that the recognition performance was improved as the amount of dots increased. The recognition rates of happy and surprised expressions were higher overall than the recognition rate of sad and angry expressions.
Additionally, they found that compared to the repetitive condition the apparent motion improved the recognition of angry, happy, and surprised facial expressions when the amount of dots was decreased. Still further, they investigated if the recognition of the expressions is affected by disturbing the apparent motion by placing a white field between the two frames, the neutral frame and the expressive frame. Here, the recognition rates were again compared between the apparent motion condition and the repetition condition. The result was that the recognition of an expression was disturbed when the white field was shown and the advantage of apparent motion compared to the repetition condition was lost. In sum, it seems that the recognition of an expression is possible even with minor spatial information as long as the motion of an expression is available. This was noticed in other experiment too (Cunningham & Wallrave, 2009).
The observed studies prove that simple facial PLDs have a serious potential to convey information about face presence, gender, and even emotional expressions with motion information. However, more research is needed with even more simplistic and systematically controlled representations of supposedly human facial movements in order to understand more deeply the effect of movement on facial information processing.
The knowledge of human perceptual capabilities could be utilised in human-computer interaction in various ways. For example, facial PLDs can be potentially useful in improving systems of artificial intelligence which actively and unobtrusively recognise facial expressions of a computer user from video input. Thus, simplistic PLDs can reveal spatial position and movement of facial points which are sufficient for conveying particular emotional information to the observer. These spatiotemporal patterns of facial PLDs can be used in building facial representations which are compact yet descriptive enough for the classification purposes. Furthermore, the knowledge of the minimal information needed to convey emotional information can be used to develop technology-mediated communication. With PLDs it is possible to provide information cues in instant messaging and internet telephony software, groupware applications, and social media services. When using PLDs only a small amount of information is needed to be transmitted. This enables low bandwidth requirements. As PLD visualisations are simple, there is no need for high definition displays. The presentation of PLDs is sparse, and might even be superimposed on something else without being too obtrusive.
As noted already by Bassili (1979), the human face can be divided into two parts in which the movements of facial expressions mainly take place, (1) to the upper part of the face which is the forehead area, including the eyes and eyebrows, and (2) the lower part of the face which includes the cheeks, the mouth area, and the jaw. Further, Bassili (1979) suggested that there appear certain directions of the facial expression movements. Simply put, the facial movements can be represented by upward and downward movements of the facial feature points, like the eyebrows and the mouth corners, including some angular and shape changes of the features. Based on these considerations we created highly basic, artificial, yet carefully controllable PLD animations, consisting of angular, upward and downward movements, in order to initially investigate both the movement information processing and the possible emotional response elicitation by the PLD animations. As many of the findings above deal with facial information, we were further interested in the role of the face related context when processing information that has basically no obvious relation to faces. This was investigated by informing one half of the participants of the face presence, while the other half did not have this information. By dividing the participants into two groups we expected to find out if the participants’ reactions were similar or not despite the prior knowledge about the stimuli.
There are two basic approaches in theories of emotions. One is the discrete emotion theory (e.g., Ekman, 1992) saying that human emotion system consists of certain specific emotion patterns that are differentially represented in the brain and body. Following this the emotional reactions include specific brain activations, physiological responses, emotional experiences, and facial expressions.
These specific sets of activations are frequently referred to as a set of basic emotions of happiness, sadness, surprise, anger, disgust, and fear. Many of the earlier studies investigating functionalities of PLDs have used this approach more or less explicitly. The other theory of emotions is called as the dimensional theory of emotions. According to this theory, emotions consist of a set of dimensions that reflect appetitive or directional motivational (approach-withdrawal) behaviour, the intensity of the behaviour (calm-aroused), and the level of control one has over a situation or stimulation. In respect to the measurement of these dimensions there is a long history of developing rating scales. A frequently used method has been to use nine-point numeric bi-polar dimensional rating scales or the Self-Assessment Manikin (SAM) which is a pictorial rating scale to make ratings of how one feels about various things (e.g., Osgood, 1952; Schlosberg, 1954; Bradley & Lang, 1994). The most used emotional dimensions are pleasantness (valence) and level of activation (arousal). SAM (Bradley & Lang, 1994) has three dimensions called valence (varying from unpleasant to pleasant), arousal (varying from calm / relaxed to aroused), and dominance (varying from a feeling of being controlled to being in control). These three rating scales have been often used in the field of human-computer interaction for measuring the ratings of emotional experiences of the users. For example, Pfister, Wollstädter, and Peter (2011) measured emotional experiences towards messages of a computer system with the ratings of valence, arousal, and dominance.