Ekman Faces

[Niobe Hennigan]

Thankyou for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Forty years ago, psychologist Paul Ekman took his camera to the island of New Guinea to photograph the faces of the South Fore people. He wanted to prove that the expressions on their faces did not mirror social convention but were universal displays of human emotion. A set of these photographs, which launched Ekman's long career deciphering the secrets of the human face, is now on display in the new Mind exhibition at San Francisco's Exploratorium.

Ekman went on to devise a system to classify facial expressions using the movements of 43 muscles in the face. He discovered that hidden emotions, such as those caused by lying, can be revealed by fleeting 'microexpressions'. His system is now used by computer animators to create realistic facial animations and by police officers interviewing suspects. Ekman is also working with the US Department of Homeland Security to train airport staff to identify potential hijackers by searching for suppressed fear and disgust in passengers' faces.

One item in the exhibition stands out. A tiny video screen shows Ekman's 1967 footage of a group of boys playing outside the window of his hut in New Guinea. After disappearing from view, a young boy suddenly sticks his head back into the frame and pulls faces at the camera. The scene is so familiar, yet, after one has paid such close attention to facial detail, it seems utterly foreign. It is a reminder of how flexible our faces are, and of how much we can convey when we know someone is watching.

The facial nerve receives impulses from multiple brain areas. Lower face muscles are represented more fully in the motor cortex than the upper face, allowing for more voluntary and learned control of the lower face; this provides the fine controls of that facial region required for speech articulation. The amount of bilateral v. contralateral fibers to the facial muscles differs depending on region, with the lower face being primarily contralateral and bilateral fibers increasing in the upper face (Matsumoto & Lee, 1993). (Note that there are large individual differences in this regard, and that involuntary expressions for the most part provide bilateral activation.) Voluntary and involuntary expressions are under the control of different neural tracts (Rinn, 1991), with voluntary expressions controlled by impulses from the motor strip through the pyramidal tract, and involuntary expressions controlled by impulses from subcortical areas through the extrapyramidal tract. The activation of facial movements that have become habitual, although acquired voluntarily, might resemble involuntary activation, but no research on this has been reported.

Table 1

Descriptions of Facial Muscles (and Other Nonverbal Behaviors) Involved in the Emotions Darwin Considered Universal and that Research has Shown to be Universally Expressed and Recognized

Still, opponents levied criticism against the research, arguing that all the previous studies involved judgments, not productions, of facial expressions. Ekman (1972) therefore conducted another study in which he videotaped American and Japanese participants alone in a room as each watched a strong emotion-provoking film. Analysis of the specific muscles in the face that moved in the participants showed that both Americans and Japanese produced the same expressions of emotion, which were the same as the expressions previous studies demonstrated were universally recognized. These data provided the fourth source of evidence for universality of facial expression of emotion.

Since then there have been many other judgment studies around the world that have demonstrated convincingly that a small set of facial expressions are universally recognized (Matsumoto, 2001). And, many other studies have demonstrated that when emotions are aroused, the same facial expressions of emotion are reliably produced by people all around the world and from all walks of life (Matsumoto, Keltner, O'Sullivan, & Frank, 2007). Subsequent research has also demonstrated the universal recognition of contempt (Ekman & Friesen, 1986), and that facial expressions of emotion are part of a coordinated response system that involves unique physiological signatures, specific cognitive activities, preparation for motor behaviors, and specific feelings. They are signals of a rich, complex response system all humans have. Of the literally thousands of expressions that can possibly be produced, the facial configurations associated with discrete emotional states represent a relatively small set of specific combinations of the available repertoire. Figure 2 presents examples of the prototypic, full-face expressions of each of the seven emotions that research has demonstrated to be universally recognized and expressed.

Documenting the universality of facial expressions of emotion led to a wide range of research on facial expressions, much but certainly not all of it focusing on emotion signaling. This led to the need to develop methodologies to measure facial behavior validly and reliably. While several ways to do so have been developed over the years, the Facial Action Coding System (FACS) (Ekman & Friesen, 1978) is widely acknowledged to be the most comprehensive and objective system available.

FACS coding is atheoretical and can describe any facial behavior, not only those related to emotion signals. Coding involves minimally the identification of which facial muscles are innervated in any expression based on the visible appearance changes that occur in the face when expressions occur. Coding can also involve the rating of the intensity, laterality, and timing (onset, apex, and offset) of individual AUs. Because any of the 44 AUs can occur independently of each other, and depending on whether each AU is coded for intensity, laterality, and timing characteristics, FACS coding is typically comprehensive, complex, and labor intensive. Table 1 lists the AU codes for the expressions Darwin considered universal, expressions subsequent research has shown to be universal, and analogous or homologous expressions of nonhuman primates. Figure 3 shows an example of a FACS coded expression of fear.

There are a number of derivatives of FACS. For instance, Ekman and Friesen (1982) developed an abbreviated version of it called Emotion FACS (EMFACS), which identifies only those AUs that were theoretically or empirically related to emotion. EMFACS coding, therefore, focuses only on the identification of a smaller set of AUs or AU combinations, resulting in a more time efficient coding procedure.

Although all of the facial musculature in the human adult exists and is fully functional at birth (Ekman & Oster, 1979), infant faces differ from adult faces in several ways, such as in the proportions and dimensions of the bony structures of the face, fat deposits and elastic skin, and in the presence of specialized features such as the sucking lip and buccal sucking pad (Oster, 2005). For these reasons, the appearance changes produced by facial movements in infants differ slightly than those that occur in adults. Thus, there is a modified version of FACS specialized for use with infants called BabyFACS (Oster, 2004).

For years ethologists have examined the facial behaviors of nonhuman primates, noting the similarities between human and nonhuman expressions displayed in similar contexts (de Waal, 2003). Previous research in this area, however, was limited because there was no method of objectively identifying nonhuman primate expressions; expressions that appeared alike were merely described and classified by researchers according to their overall appearance. Recently, Vick and colleagues (2007) developed a modified version of FACS for chimpanzees (Pan troglodytes) called ChimpFACS. The issues associated with the translation of human adult FACS to chimpanzees are similar to those associated with its translation to the study of infants; for example, the forehead musculature of chimps is less well developed than that of humans. Nevertheless, there are minimal differences in the underlying musculature for the AUs that are common between humans and chimps (Vick et al., 2007), and many muscles that are related to human facial expressions of emotion appear to have the same location and functional effect in chimpanzees, thus representing homologues.

Human social life requires expression regulation, because the non-regulated, unadulterated expression of emotion would lead to social chaos (Matsumoto, Yoo, Nakagawa et al., in press); that is, humans cannot just act automatically on their impulses whenever strong emotions are aroused if they are to live harmoniously with others. Fortunately, humans differ greatly from other animals in that they have been endowed with an elaborate set of neuroanatomical structures that allow the alteration of the linkage between the tendency to respond and the actual response (Levenson, 1999). With regard to facial expressions of emotion, these regulatory mechanisms are known as display rules (Ekman & Friesen, 1969).

Display rules are learned early in life and dictate the management and modification of facial expressions depending on social circumstances. There are seven ways in which expressions can be regulated (Ekman & Friesen, 1969). Emotions can be

One of the major functions of every human culture is to proscribe rules by how expression regulation should occur. One way this is accomplished is through the ascription of cultural meaning to different social situations (Matsumoto, 2007). For example, expressing emotions such as anger, contempt, or disgust to ingroup members or higher-status others may be dangerous because it disrupts ingroup harmony or dominance hierarchies. In fact, this display rule is universal to all human cultures (Matsumoto, Yoo, Fontaine et al., in press)

3a8082e126