“How can cinema have so powerful a reality effect when it is so manifestly unreal?”
Vittorio Gallese and I have started from this very simple assumption by Steven Shaviro to wonder at which levels a movie engages its viewer. The moving image is with no doubts the most sophisticated form of mediated intersubjectivity: the relationship we build with the characters, with the objects and the landscape which appear on the screen is very similar to the relationship we build in our daily life with people we interact with, objects we use and environments we inhabit.
The reason why we believe in a research capable of studying our film experience from a neuroscientific perspective lays on the very particular ecological approach to film we feel every time we watch a movie. We perfectly know, as J.J. Gibson pointed out in the last part of his The Ecological Approach to Visual Perception, that there is a director who controls our head movement and locomotion, and nonetheless, getting out from a movie theater, we would agree with Gibson saying that “the various kinds of filmic transition – zoom, dolly, pan, cut, fade, wipe, dissolve, and split-screen shot – could usefully be evaluated in the light of ecological optics instead of the snapshot optics that is currently accepted.” This aspect enters the debate on “real” and “reel” stimuli, recently discussed in a review by Evan F. Risko et alii.
From the very beginning of film history, filmmakers have created a style that – like a sort of universal language – enables viewers not only to perfectly understand the meaning of what they are watching, but also to move within the space and time this style is able to build and simulate. As Antonio Damasio wrote, who invented cinema should have kept in mind – more or less consciously – the functioning of the human brain. With their dissimulated segmentation, their strong impression of transparency, the appearance of a world they imply, movies have been compared to the uncanny mechanism of our consciousness by scholars like Damasio, Dennett, Metzinger, Koch (but also think of Bergson and his Creative Evolution). Taking this metaphor ad litteram could be a first step toward the comprehension of the complexity of the brain-body system in relation to cinema. Should we enter a lab to find a new direction for film theory? Was Deleuze right in affirming that the biology of the brain will be crucial for film theorists?
Take for instance a research appeared on the last issue of our journal: Tim Smith’s proposal for an Attentional Theory of Cinematic Continuity is a bright example of how a research rooted in psychology and visual cognition can help us to finally grasp the very meaning of a concept like “continuity”. The gaze behavior studied by Smith sheds new light on the most relevant and common film style, demonstrating the power of filmmakers in controlling our experience and guiding us across the frames. Eye-tracking is the new frontier for this kind of analysis, based on the study of the viewers’ gaze behavior and revitalizing researches that had had some fortunes during the season of French filmology.
The idea that we should test the behavior of many viewers in order to understand how cinema works, is also present in Uri Hasson’s works based on inter-subject correlation analysis. In this case fMRI replaces eye-tracking showing us that different brains “ticking together” in front of a well-structured movie like Sergio Leone’s Il buono, il brutto e il cattivo. The very idea is that we do not empathize just with the characters, but with a meta-character like the camera, whose behavior is generally “goal-oriented” and “action-packed”, that is, suitable to be interpreted by our brain-body system. Editing rules play a crucial role in regulating our projection into the film body. It would be interesting to relate these aspects to Hans Belting’s reflections on the relationship medium/body.
Thus it is important to question viewers’ gaze behavior as well as viewers’ brain responses, focusing more precisely on the impact our simulated motor behavior has on our film experience: fMRI or EEG experiments can offer a new and different perspective on the history of film style grounded on embodiment.
There are at least three different forms of embodiment implied by a movie. The first is tied to acting as embodiment, and it is strictly connected to the study of the “actor in the neuroscience era” (borrowing the title of a work by Vanda Monaco and Yang Wenting). The second form is tied to film style as embodiment, and it is based on the assumption that the style a director chooses to tell her story entails a more or less strong embodied relationship with the viewer. The third form is tied to viewers’ responses and it is mostly interested in finding what happens in our brain when we watch a particular film scene or film shot (and we could say that this third form is substantially implied by the other two).
Obviously, these three forms are strictly connected and often they work together within the same filmic situation or within the same experimental condition. We think that the second form is the most intriguing one, because it allows us to focus on the very specific of film art, that is, the way we are carried into the movie. Something mysterious happens: remaining still on my seat, I can move through the frame right on the screen in front of me. I occupy a double position, rooted in what Serge Daney would have described as the dialectic between a “vision bloquée” and a “vision libérée”. Erich Feldmann, in an article published on the “Revue Internationale de Filmologie” in 1956, observed how the film puts the viewer in an unbelievable condition: she is carried into an unreal situation without the help of drugs, nor of other “modifications psycho-physiologiques engendrées par la seule projection lumineuse”.
What happens then? It happens that we are able to bridge material and immaterial, virtual and real, and we do so by simulating a bodily action, an emotion, or sensation, recurring to nothing but our sensory-motor capacity. As Barbara Stafford put it, “the action-view of visual media corresponds to our innately performative perceptual processes, which are capable of jumping over great divides.” Motion pictures, because of their own essence, entail a body able to decipher the movement by simulating it internally. In his “Note preparatoires” for his course at the Collège de France in 1953 (recently published with the title “Le monde sensible et le monde de l’expression), Merleau-Ponty writes that we can understand the movement only through the movement, that is, thanks to our own body “possibilités motrices”.
It is not enough to believe that we can empathize with the world on the screen by means of our eyes and then of a cognitive elaboration of those data. There is a research we have to carry out at the subpersonal (neural) level in order to understand the connection between perception and action, which is displayed very well by the movie itself. This is the way to demonstrate that cognitive neuroscience and subpersonal neural data can shed new light on the personal level of description.
To us, what cognitive neuroscience can emphasize is the degree of our sensory-motor engagement during film experience, and we consider such a degree an useful index to rethinking the history of film style on an embodied basis, and to question our sensory-motor relationship with the moving image. In this regard our work is influenced in its own philosophy by some deleuzian suggestions.
We consider perceptions as embedded in the dynamics of actions, and according to the discoverers of mirror neurons, we think that “the acting brain is also and above all a brain that understands”. In other words, our brain serves primarily the purpose of moving us around, an activity that keeps a relevance in our conceptual life too. Münsterberg was really on something when he wrote that our body adjusts itself to the perception: “our head enters into the movement of listening for the sound, our eyes are fixating the point in the outer world. We hold all our muscle in tension in order to receive the fullest possible impression with our sense organs.”
The discovery of mirror neurons and mirror mechanisms (MMs) in the human brain has revolutionized many studies in phenomenology, philosophy of mind, simulation, and intersubjectivity, shedding new light on the role of our motor system both in our daily life and in the imaginary worlds we happen to cope with. Such a discovery has provided an empirically based notion of intersubjectivity, now to be considered as intercorporeality (“the mutual resonance of intentionally meaningful sensory-motor behaviors”, as Gallese wrote) as the main source of knowledge we directly gather about others. There is no better way to give a scientific basis to Siegfried Kracauer’s idea according to which the moving image engages the viewer “physiologically before he is in a position to respond intellectually”, eliciting “a resonance effect provoking in the spectator such kinesthetic responses as muscular reflexes, motor impulses, or the like.” Film literature is full of works that seems to anticipate the discovery of mirror neurons: from the first physiology of cinema by Dr. Edouard Toulouse (who wrote his first article in 1912!), to French filmology, from Eisenstein to Mitry, we have a huge array of papers and books worthy to be reconsidered according to the perspective we are proposing. At the same time, there is a recent literature on embodiment, tactility, and enactive cinema that offers a good basis for the neuroscientific bet. Maybe we are close to the solution of the gap between low- and high-level theories to which referred Joseph and Barbara Anderson in their essay in the book Post-Theory. What we need is a theoretical tool capable of connecting MMs and the vast horizon of our experience.
Embodied Simulation (ES), that I have briefly introduced in my response to David Bordwell’s last post, seems to be the right one for founding a motor theory of our filmic cognition. Gallese describes ES as “a common underlying functional mechanism [that] mediates our capacity to share the meaning of actions, intentions, feelings, and emotions with others, thus grounding our identification with and connectedness to others.”
So the question is: who acts in a movie? Who owns intentions? Who feels something? Who experience emotions? Obviously the actors act, own intentions (generally established by the screenplay), and feel emotions, but the real target of the movie is the viewer. She has to feel emotions, as well as she has to behave as if she were acting and owning intentions.
Some scenes elicit motor responses in a very elementary way, creating for instance a vertigo effect, or showing disgusting or painful images. Nonetheless the only way to assure a motor link between the viewer’s body and the camera behavior is by means of film style, matching film’s shots according to a corporeal schema that will trigger a mental one. Directors aiming to contrast the transparency of film style neutralize this schema and make our film cognition harder.
Our motor approach to film style, based on ES, emphasizes this particular relationship, allowing us to reflect on the “as if” component and also reframing matters like the suspension of disbelief and the so-called “segregation of spaces”. From this perspective, we have to evaluate the role of the “liberated” ES in film experience. When we read a novel, stare at a painting, or watch a movie, “our ES becomes liberated, that is, it is freed from the burden of modeling our actual presence in daily life. We look at art from a safe distance from which our being open to the world is magnified.” To put it in even simpler terms, by focusing on narrative “we can fully deploy our simulative resources, letting our defensive guard against daily reality slip for a while.”
We are testing our proposal both by applying our hypothesis to film sequences (mainly working on classical Hollywood cinema) and by filming clips that recreate optimal conditions for a neuroscientific experimental approach.
The naturalistic trend in film studies is promoting an interdisciplinary approach that reveals how subpersonal analyses can be usefully reinvested in the wide horizon of cognitive film studies.