Live Lecture: Reading Faces and Bodies

Eye gaze detection:
- Natural behavior in humans. But in the animal kingdom, it’s often perceived as something threatening or as showing dominance.
- To improve or initiate dyadic communication
- Social interaction and affective bonding

Eye gaze varieties:

Eye gaze detection in babies:
- Newborns preferentially look at faces with direct gaze. They pay attention more to somebody who is looking directly at them.
- This is also associated with increased oxytocin levels in mothers (and fathers); this facilitates direct gaze → first development of social skills of babies
- 6+ months: babies can follow moving head and eyes, preference towards the eyes by the age of 2

The watching eye effect:
- We intuitively know when somebody is looking at us. So when you are sitting in class and you feel that someone is looking at you
→ this enhances prosocial behavior (for example, fewer bicycle thefts, less littering, higher donations). Just because of the feeling that someone/people are watching you
- It also increases self-awareness (larger Stroop effect > larger cognitive interference, thus a harder time processing visual information)

The gaze orienting effect: it is easier with eyes as a cue (directing you to the place of the target, so in a congruent trial) to watch to the correct side of the target.
→ it is situation dependent: larger gaze orienting effect with…
- Fearful, surprised, or dominant faces.
- When the face and object are both positive
- Under joint attention
→ it is also person-dependent:
- Females show a larger gaze orienting effect than males
- Younger people show a larger effect than older people
- Low-esteem people show a lower effect than high-esteem people

But there are also cultural effects:
→ in a study (Cohen et al. 2017) about the comparison of gaze cueing effects in European Americans and East Asians:
- Gaze cueing effect for individuals from East Asia disappears for mismatched conditions. But this was the case for the European Americans. So there seems to be a culturally specific effect, where individuals that come from a collectivistic-holistic society are more likely to show the gaze effect if it's embedded in a specific context where everyone is looking in the same direction.
- Cultural differences in social orientation: collectivistic-holistic vs. individualistic-analytical

Neural substrates of gaze perception:
Two different networks:
- Yellow: social processing
- Blue: gaze perception
→ they, of course, show overlap because a lot of gaze perception has to do with social processing, and the other way around.

Superior temporal gyrus:
- Gaze perception
- Sensitive to social attention cues

Medial prefrontal cortex:
- Joint attention

Right temporal parietal junction:
- Live face-to-face interaction (when we have direct interaction with somebody, instead of looking at pictures or recordings)

Neural substrates in infants:
→ fNIRS study in 4-month-olds: engaging in direct gaze (as opposed to averted gaze) recruits the right superior posterior temporal cortex (cue region for social behavior) and right frontopolar cortex (active decision-making is taking place)
→ focus on the picture primarily on the red line
→ Dotted lines: averted gaze. Solid lines: direct gaze.
→ so there are distinct cortical differences in processing gaze (direct vs. averted)

Neural substrates of direct gaze:
- Fusiform gyrus: enhanced encoding when the face is unexpected. It’s the very primary and initial process when we first see someone and process their face. It’s not active anymore when we process the face further, for identity, for example.
- Right anterior STS: gaze detection is facilitated by instructed attention to the eyes, but not the gaze itself. So when individuals actively focus on the eyes, but it doesn’t matter what kind of gaze it is.
- Right posterior STS: during eye contact which is consciously perceived as social or communicative. So when we are aware of being in a social situation.
- Right middle prefrontal cortex and the orbitofrontal cortex: for dynamical facial expressions and when communicative intention needs to be decoded.
- Amygdala: emotional processing. So the processing of the emotional significance of the face (am I being threatened, flirted with, etc.)

Dual pathway of mutual gaze:
2 pathways:
- Fast pathway (subcortical structures): primarily processes visual information in a fast way, we don't get many details out of it.
- Slow pathway (cortical processing route): to process the details more, like the identities of the faces we look at, or if it's a directed or averted gaze, or if we know the people, etc. More of an analytical stage.
→ at the end is the task-relevant modulation: responsive to task demands in social context, so if it's a friendly or threatening situation, for example.

Take-home messages:
- Gaze detection fulfills important social function and is an innate human ability
- Gaze orienting is context, person and culture dependent
- Gaze is processed by medial, inferior frontal, superior temporal, temporoparietal, and subcortical brain regions
→ following a fast subcortical pathway and a more analytical cortical pathway