Neurocinematics: The Science of Watching Movies

Cinema has long been one of humanity’s shared pastimes, serving as a window into a plethora of different worlds that we can traverse as a mode of escapism. Whether it’s the high emotion and drama of Titanic, or the evergreen, incisive comedy of Mean Girls, movies capture nearly the entire spectrum of human experience and offer a sense of community. Alfred Hitchcock once said that “creation is based on an exact science of audience reactions”, and indeed, the emotional journey an audience undergoes while watching a film is a result of many carefully calibrated decisions by the director. This involves methods of filmmaking such as blocking, framing, and sequencing.

But what actually happens in the brain when we sit down in a movie theatre, armed with popcorn and soda, and the lights go down? How and where does our emotional response to film originate in the brain? In 2008, researcher Uri Hasson and his group set out to analyse how brain activity is affected by a given film, in a seminal paper that has come to define the study of neurocinematics. Hasson used functional MRI scanning to capture neural activity in his participants when viewing a series of films while lying flat in the scanner, and then used intersubject correlation (ISC) to analyse the similarity of brain and eye responses across the cohort. He also looked at the effects of sequencing and framing by showing participants jumbled up segments of Charlie Chaplin films.

"some coherency of shot sequencing was required to initiate responses from areas such as the fusiform gyrus, commonly known to respond to faces, and the parahippocampal place area, which is more receptive to scenery."

The results were fascinating. They were analysed using ISC, a correlational method that summarizes the synchronization of brain activity across a cohort over the course of an imaging session. When participants were shown Sergio Leone’s The Good, The Bad and The Ugly, a statistically significant correlation was seen in activity in visual areas of the brain located in the occipital and temporal lobes, auditory areas in Heschl’s gyrus, Wernicke’s area, and multi-sensory areas in the temporal and parietal lobes. Though participants were free to look anywhere they wished while in the scanner, recordings of their eye movements were highly similar, indicating they were looking at the same areas of the frame. In assessing the effect of a film’s editing by shuffling up sequences of shots from Charlie Chaplin films, it was found that the visual cortex and the visual motion area showed similar levels of activity for the scrambled films as the unscrambled ones. However, some coherency of shot sequencing was required to initiate responses from areas such as the fusiform gyrus, commonly known to respond to faces, and the parahippocampal place area, which is more receptive to scenery. Areas responsible for higher levels of cognitive processing such as the lateral sulcus and temporal parietal junction were activated only with higher levels of cohesion, indicating these areas may be important when analysing narrative and character motives. And finally, collective engagement in a variety of media (The Good, The Bad and The Ugly; Hitchcock’s Bang! You’re Dead; Curb Your Enthusiasm; and an unstructured reality segment taken at a park) was measured. Hitchcock’s film was found to have the highest ISC, indicating higher levels of “control”— defined by Hasson as the reliability and predictability of the neural state evoked by a film — in the brains of the viewers, no doubt as a result of directorial intent and authorial choices.

Hasson’s study was a landmark in elucidating how brain activity differs while viewing film, and how it is closely affected by techniques and the kind of content presented. Visual engagement varies based on framing and blocking, while cognitive engagement with the plot of a film relies on activation of higher cognitive areas. This presented a new perspective of how neuroscience and neural imaging could be used in cinema studies. That being said, the study did not attempt to mimic natural viewing conditions. More refined ISC analysis could also be conducted keeping in mind the genre of film. And, the study does not consider inter-cohort differences such as age, sex or cultural groups, instead analysing patterns holistically.

"For both negative and arousal clips, the prompt to express emotion did not activate as many brain regions [as the prompt to suppress emotion]"

Still, Hasson’s work provided a valuable foundation for this kind of study, and future research builds on it by using neuroimaging to assess emotional impact. In 2012, Shimamura et al. used film clips meant to stimulate negative (including blood and gore), aroused (including high-speed action), or neutral (cooking and fly-fishing) reactions. By asking them to either openly express or suppress their emotional reaction to the clips, and using fMRI, different regions of the brain were stimulated. Compared to the neutral clips, the negative and arousal clips activated the posterior parietal and occipital regions. When participants were asked to suppress their emotions while viewing negative clips, an extensive frontoparietal network was activated, including the lateral and medial prefrontal cortex, the angular gyrus and posterior cingulate gyrus. This signals towards the difficulty of suppressing negative responses to disturbing stimuli. These regions are also involved in the default mode network, activated when the brain is at rest. Suppression during arousal clips showed similar areas of activation, though not to the same extent. For both negative and arousal clips, the prompt to express emotion did not activate as many regions. These findings contribute towards understanding the neural correlates of emotional regulation, especially while viewing film rather than still images. 

Studies have also been conducted analysing the empathetic response to film. Vemuri and Surampudi in 2015 used independent component analysis (ICA) of fMRI data to study the neural correlates of empathy when participants were shown a diverse range of film clips. Segments from Hollywood film The Green Mile, Indian film Taare Zameen Par, and animated film Up! were shown, which depicted the actors in emotional states and narratives which could invoke empathy. They plotted the components of empathy networks activated by each film, and found that the Hollywood film prompted strong activity in the medial dorsal nucleus of the thalamus, responsible for emotional processing. The pulvinar, insula and superior temporal gyrus being activated also may suggest a higher empathetic response. These areas were also activated by the Indian film, along with a wider cluster of activity in the insula. In contrast, the animated film did not incite significant activity in the thalamus region, or areas associated with cognitive empathy such as the paracingulate cortex. While the Hollywood and Indian film activated parts of the emotional empathy circuit, the animated film did not, indicating a lack of emotional connection being attributed by viewers to computer-generated agents. This study adds to the research conducted on how empathy processing is affected by visual stimuli by identifying the activation of specific empathy networks.

"While the Hollywood and Indian film activated parts of the emotional empathy circuit, the animated film did not"

Overall, neuroimaging and fMRI have provided great insight into how the brain processes and reacts to visual and narrative stimuli, and what regions are involved in processing different components of a film. These studies illustrate the juxtaposition of neuroscience and cinema studies, and how the former can be used to augment the understanding of the latter. By demonstrating how eye movements are impacted by technical aspects, such as blocking and framing, the director’s intent and calibration of audience responses becomes more evident. How we analyse the narrative presented on screen, character motives, or the emotional state of the actor, is also heavily driven by activation of specific brain regions. However, the findings of these studies could be skewed by limited sample sizes, and not considering individual factors which could introduce nuance to the sample. Their applicability to real world scenarios could also be negated by how participants view the films (in segments, alone, in a controlled setting with a scanner). Analysing activity in smaller regions could also be complicated by poor signal/noise ratio and artifacting, as noted by Shimamura (2012). Filmmaking techniques have also grown and developed significantly with the advent of streaming services, so researching how audiences respond to modern film could show how contemporary media and methods affect retention and engagement. Further work in this area can also attempt to more closely resemble how a theater audience may see a film. After all, few things compare to the magic of the lights going down in a packed cinema, and the way our brains make sense of it to allow us to fully immerse ourselves is simply incredible.

References:

1. Hasson, Uri, et al. “Neurocinematics: The Neuroscience of Film.” Projections, 2, 1-26 (2008). https://doi.org/10.3167/proj.2008.020102

2. Shimamura, A.P., et al. “Neural correlates of emotional regulation while viewing films.” Brain Imaging and Behavior 7, 77–84 (2012). https://doi.org/10.1007/s11682-012-9195-y

3. Vemuri, Kavita & Surampudi, Bapi R. “Evidence of stimulus correlated empathy modes – Group ICA of fMRI data.” Brain and Cognition, 94, 32-43 (2015). https://doi.org/10.1016/j.bandc.2014.12.006

---

This article was written by Reeva Trivedi and edited by Julia Dabrowska, with graphics produced by Saba Keshan. If you enjoyed this article, be the first to be notified about new posts by signing up to become a WiNUK member (top right of this page)! Interested in writing for WiNUK yourself? Contact us through the blog page and the editors will be in touch.