Déjà vu, the eerie sense that something new has been experienced before, has confounded us for hundreds of years. Along with the public, philosophers, physicians, intellectuals and, more recently, scientists have tried to get to the bottom of the phenomenon. Potential explanations have ranged from double perception (the idea that an initial glance at something was only partially taken in, leading to déjà vu upon a second, fuller glance) to dissolution of perceptual boundaries (a brief blurring of boundaries between the self and the environment) to seizure activity to memory-based explanations (the idea that déjà vu results from a buried memory). Now, research emerging from my lab and others suggests that déjà vu is not just a spooky experience, but a possible mechanism for focusing attention – perhaps an adaptive mechanism for survival shaped by evolution itself.
I first became interested in the topic after reading the paper ‘A Review of the Déjà Vu Experience’ (2003) by the psychologist Alan S Brown – probably the first treatment ever to appear in a mainstream psychology journal. Writing in the Psychological Bulletin, Brown described survey studies, case reports and theoretical ideas culled from more than a century’s worth of writings on déjà vu. Much of the available literature on déjà vu at the time came from non-mainstream sources (and some were even of a paranormal flavour). Still, from this largely fragmented literature, Brown managed to winnow some important clues and presented them in a language that cognitive scientists could work with and act upon: data and theory. The data from the survey studies provided useful empirical starting points, and the very old theories of déjà vu that Brown reviewed provided a scaffolding for devising highly specified hypotheses that could be tested in a lab.
From the large collection of surveys conducted over the years, Brown determined that roughly two-thirds of people experience déjà vu at some point in their lives. He also reported that the likelihood of experiencing déjà vu decreases with age, and that physical settings (or places) are the most common trigger. The finding that déjà vu is most commonly elicited by scenes (as opposed to just speech or objects) was a particularly useful clue for scientists: a new theoretical approach to autobiographical and event memory emphasises a role of scenes in the ability to recollect past life events. Partly based on newer understandings that brain areas critical for first-person navigation through places may also underlie recollective memory ability, the idea is that the first-person perspective within a scene is a crucial facet of human memory. Consider the last dinner that you ate at a restaurant. What is this memory like? Can you ‘see’, in your mind’s eye, where everyone else is sitting relative to you at the table? This illustrates how our ability to process, navigate through and mentally reconstruct our place within past scenes may be central to our recollective memory ability.
The critical role of our place within scenes in memory may also be why the centuries-old memorisation technique known as the Method of Loci (also called the Memory Palace) is very effective and used by competitive memorisers; it involves envisioning your to-be-remembered information within particular scenes along a route that you regularly take, or within a building that you know well. For example, to remember his talking points in their correct order for his TED talk ‘Feats of Memory Anyone Can Do’ (2012), the science writer Joshua Foer created a visualisation of different points throughout his house, each with a visual-image cue attached to it so that, when he did a mental walk-through of his house starting at a mental image of the front door, he would ‘see’ in his mind’s eye an image cuing him for the next talking point.
In the foyer of his house, Foer had imagined Cookie Monster (the Muppet) on top of Mister Ed (the horse) as his cue to introduce his friend Ed Cooke at that point in the talk. Foer continued moving through various places within his image of his house to access his cues for the next talking points in the order in which he needed to raise them. For example, later on, when arriving at the kitchen in his mental walk-through of his home, he had imagined the characters from The Wizard of Oz along a Yellow Brick Road; this was his cue to describe how he had embarked on a journey and the many friends he met along the way. As Ulric Neisser, often considered the father of cognitive psychology, suggested decades ago, ‘a sense of where you are’ may provide a basis for recollective memory. Although déjà vu is more of a contentless sensation of memory than a recollection of autobiographical experience, the fact that it tends to be elicited by scenes hints at the possibility that it, too, emerges from the same basic scene-processing mechanisms that enable this ‘sense of where you are’.
An example is having a sense of recognition for a person’s face without being able to pinpoint just how you know them
Dovetailing with this useful clue about déjà vu, Brown’s 2003 review also mentioned the ‘Gestalt familiarity hypothesis’ – the theory that déjà vu results from a familiar Gestalt, a German word for the arrangement of elements within a space – such as when a new acquaintance’s living room happens to have the same spatial layout as a previously visited space that fails to come to mind. Brown linked this untested hypothesis of déjà vu to an ongoing approach for studying memory known as the ‘source-monitoring framework’, in which a person can recognise a situation as having been experienced before without pinpointing the source of the familiarity. In what seemed to be an invitation for cognitive scientists, Brown suggested that it would be straightforward to test such hypotheses in the lab.
At the time, I had been studying a phenomenon known as ‘recognition without identification’ and its sister phenomenon, ‘recognition without recall’. Both are thought to reflect the ability to sense that something was experienced before, even when no specific past instance comes to mind. A common example is having a sense of recognition for a person’s face without being able to pinpoint just how you know the person. I immediately saw a connection between my own work and what Brown presented in his review, and I set out to test his ideas.
One of my methods seemed particularly applicable. This was the recognition without recall method. In my original work, participants might receive a cue like ‘POTCHBORK’ that resembles an earlier viewed word, ‘PITCHFORK’. Although a person can successfully use the cue to recall the word it resembles, sometimes recall fails. Recognition without recall is the finding that people give higher familiarity ratings to cues that resemble unrecalled studied words than to cues that do not.
Applying this insight to déjà vu, my students and I developed a variant of the task using black-and-white line drawings. Each test image potentially shared an overall ‘Gestalt’ or arrangement, with an image that had been studied before. When presented with an image on the test, participants attempted to recall a previously viewed image having a similar arrangement of elements. They also rated how familiar the test image seemed and whether or not it provoked a sense of déjà vu. Images that fell into the déjà vu category tended to indeed match arrangements found in prior images, establishing evidence for the Gestalt familiarity hypothesis and setting the stage for what I would later facetiously refer to, in a TEDx talk, as a ‘déjà vu generator’– an implementation of the Gestalt familiarity idea in virtual reality (VR). Wearing a VR headset, participants would be sequentially immersed in different sets of visual surroundings throughout a study phase. In a later test phase, they would be immersed in new scenes, some of which share a spatial layout (ie, arrangement of elements) with scenes from the study phase. Here, the familiar Gestalt would be one’s visual surroundings within the VR environment, as might mimic real-life situations in which déjà vu occurs, and as might involve a sense of where you are.
I met Alan Brown in the summer of 2007 at the American Psychological Association annual convention in San Francisco after inviting him to give a talk on déjà vu. I told him how his 2003 review paper and later book, The Déjà Vu Experience (1st ed, 2004), inspired me to pick up the study of déjà vu myself. This formed the start of a long collaboration. Later over dinner at the 2007 annual meeting of the Psychonomic Society that November in Long Beach, California, we marvelled at how neat it would be to be able to test the Gestalt familiarity hypothesis in VR for an immersive experience in which the spatial layout is one’s visual surroundings within the VR environment. To the extent that VR allows for a simulation of life-like immersion within scenes, this approach might approximate the way in which an arrangement of elements in space (such as where a table, couch, floor lamp and artwork are placed relative to one another within a living room scene) might produce déjà vu in real life. It seemed like a castles-in-the-air idea.
But then in 2008, there I was with a group of students, down in a windowless cinderblock room in the basement of the old Clark Building on the Colorado State University campus, wrestling with a VR headset. Fifteen years ago, VR systems were quite crude. They lacked a user-friendly interface or any form of tech support, and required a lot of improvising in the form of makeshift workarounds. We were working with a set of eMagin Z800 VR goggles and were attempting to get The Sims 2 – a 2004 life-simulation game – to display within the goggles for an immersive experience with the game. This was not trivial. Fortunately, Ben Sawyer was among the tinkerers down in that basement. An undergraduate at the time, with a lot of technical savvy, Sawyer was a legend among the Clark A-wing basement-dwellers for having taken apart and reassembled the always-malfunctioning driving simulator, completely reprogramming it for functional operation in research.
Recognition without recall occurred in the form of higher familiarity ratings among VR test scenes that shared a spatial layout
The Sims 2 video game involves creating indoor and outdoor spaces by placing elements onto a grid from a bird’s eye perspective, and then zooming down into the scene from a first-person perspective to make adjustments and tour the scene. This provided a means by which a large set of scenes, each having an identically configured but otherwise distinct counterpart scene, could be created for viewing from a first-person perspective. For example, a clothing-store scene might have the same arrangement of elements on a grid (eg, the placement of hanging wall displays of clothing relative to a table with folded shirts) as a bedroom scene (eg, the placement of windows and end tables relative to a bed). So while Sawyer worked for months on getting the Sims 2 game engine to output in 3D to the Z800 goggles, I used a pad of graph paper to sketch out a bird’s eye view of dozens of pairs of distinct but identically configured scenes to then manually create within The Sims 2 game, soliciting scene ideas from other team members along the way and keeping a running list (eg, a clothing store configured the same way as a bedroom, a bowling alley configured the same way as a subway station, a museum configured the same way as a courtyard, etc). After many months of creating Sims scenes, and many remarkable improvisations that included having the machine output in 3D to the monitor or any attached display device, and creating short-cut keys to enable teleportation from one scene to the next within The Sims 2 structure (and with Sawyer at one point taking apart then soldering together a pair of non-functioning Z800 goggles that Brown had shipped to us), we eventually got the experiment to work in VR.
From within the goggles, which felt a bit like thick, heavy ski goggles edged with foam, a given cartoon-like Sims scene could be viewed through a square, straight ahead. The depth perception was comparable to that of a 3D movie viewed with 3D glasses, and turning your head enabled viewing differing aspects of the scene, such as looking up at the ceiling or down at the floor, or left or right.
The first VR experiment to examine the Gestalt familiarity hypothesis involved 24 college students. A short-cut key zapped the participant from a particular vantage point within one scene to the next and, from each pre-established vantage point, the participant was free to look around the scene by turning their head. After the first 16 scenes, the person viewed a new set of 32 test scenes, half having an identical spatial layout to one of the first 16. While viewing a test scene, the student rated how familiar the scene seemed, indicated if the current scene prompted any recollection of one of the earlier 16 scenes (and if so, which one), and indicated if déjà vu had occurred. After cycling through the 32 test scenes in this manner, the process started over with a new set of 16 study scenes followed by another 32 test scenes. Recognition without recall occurred in the form of higher familiarity ratings among VR test scenes that shared a spatial layout with an earlier viewed but unrecalled scene, and during recall failure, participants reported déjà vu 27 per cent of the time, compared with a baseline of 17 per cent of the time when there was no spatial layout resemblance to an earlier scene.
Although this study demonstrated interesting findings and represented a remarkable technological feat for its era, it was repeatedly rejected from journals before finally finding a home at Consciousness and Cognition in 2012. The topic of déjà vu, was, and still is, a tough sell in the world of science.
Still, the publication generated a great deal of media attention and public interest, and with that came a number of enquiries to me from the general public about the research, by phone, email and sometimes mailed letters. In many of these enquiries, people were reaching out to tell me that they thought the idea that déjà vu was grounded in memory could not be correct, or could not be complete, because, to them, déjà vu included a sense of knowing what will happen next. Some people even used the term ‘precognition’ to describe this. At first, I was not only sceptical, but also wary of venturing into what seemed like more of a topic for paranormal literature than mainstream scientific research, especially when déjà vu was already a tough sell as a topic within science. But the line of questioning kept happening, even in academic settings and, eventually, I started looking into it.
Was there a logical, scientific explanation for the sensation these people expressed? Perhaps if a situation was experienced before but failed to be recalled, the sense of how a similar situation would unfold might seem like a prediction? To test this with our spatial layout paradigm, we needed scenes to dynamically unfold over time. The Sims 2 platform was well suited to this because it was set up to easily create videos of virtual tours to publicise one’s Sims creations on YouTube. From this idea, the ‘virtual tour’ paradigm was born. Participants viewed video tours of the Sims scenes that had been used in the previous VR study, each taking a particular path with turns through the scene from a first-person perspective. In the test phase, the tours through scenes with identical spatial layouts also followed the same path as in the earlier-viewed counterpart scene, but only up to a point – the tour stopped short of a turn that happened in the earlier counterpart scene, and participants had to determine the direction of the next turn. If our hypothesis was correct, we thought, then we would find that, when participants experience déjà vu while viewing a tour of a scene with an identical layout as an earlier viewed but unrecalled scene, they should be more likely to successfully predict the next turn.
However, that was not what we found. Our new hypothesis was not supported and, deeming the study a failure, I let it sit for a couple of years.
Everything the doctor was saying was something she’d heard before, but also she knew what the doctor was going to say next
But the enquiries continued to come. One that stands out was when my office phone rang and it was a somewhat shaken man calling from Alaska. He’d had a strange déjà vu experience and was looking for answers. He found some of my research on déjà vu in an internet search. He had recently experienced a strong sense of déjà vu while on a hunting trip and was quite shaken by the fact that, during his déjà vu, he knew exactly what would happen next. ‘I am not a superstitious person,’ he said, ‘so I just don’t understand how this could be possible. I’m hoping maybe you have some answers that can explain this.’ He was distraught, looking for an explanation. I had no good explanation to offer.
Conversations like this continued to eat at me.
Then one day it occurred to me that perhaps the feeling of déjà vu is associated with an illusory sense of prediction. Digging back through old literature, there were some hints at this idea. For example, in a very old neurology case report from 1959, Sean Mullan and Wilder Penfield reported on a patient for whom electrical stimulation during awake brain surgery induced déjà vu. The patient reported feeling like everything the doctor was saying was something she had heard before, but also like she knew what the doctor was going to say next. Since the déjà vu was induced artificially through electrical stimulation to the brain, the accompanying sense of prediction must have been illusory in that case, rather than memory based.
So, I dusted off the old experiment from a couple of years earlier and ran it again with an additional prompt following the pause during each tour of a test scene: rate the feeling of being able to predict the direction of the next turn. And lo and behold, people felt pretty strongly that they knew the direction of the next turn when experiencing déjà vu, even though that was not the case. This finding persisted across many subsequent experiments, including in the original multi-experiment study that was the first to show it in 2018 and in the studies that followed it.
But this research still didn’t address the question of why people like the Alaska caller feel like they really did predict what was going to happen during déjà vu. So, we did a follow-up study, which suggested that not only is there a predictive bias associated with déjà vu, but a ‘postdictive’ bias (a feeling of having known all along how the situation was going to unfold) too.
As to what all of this means, it may be that déjà vu produces the feeling of being on the verge of retrieving a past experience from memory, leading to the belief that you can identify what will happen next (because it feels like how the situation unfolds is about to come to mind at any moment); then, as the situation does unfold in a certain way, its continued familiarity tricks the mind into believing that it knew it all along.
Although these research findings represent major steps toward understanding déjà vu, it wasn’t until I was able to experience déjà vu myself within the ‘déjà vu generator’ that I had what may be my most critical insight. It took a recreation of the VR déjà vu paradigm by someone else for me to have the experience myself. Because I had personally created most of the scenes in our previous work, and because I knew every scene and its counterpart, I could never experience déjà vu myself within our system. The scenes were just too familiar to me.
That changed when I donned an HTC Vive VR headset to personally run through a brand-new variant of the VR paradigm created by Noah Okada, then a computer science student at Emory University in Georgia.
I met Okada in 2019 while on a visit to Emory during my sabbatical. He was working with the neuroscientist Daniel Drane and the neurologist Nigel Pedersen – whom I was visiting – to create VR scenes for use in research. Pedersen and Drane work with people who have epilepsy. Our collaboration had formed a year earlier through Joe Neisser, a philosopher at Grinnell College in Iowa (who, somewhat serendipitously, happens to be the son of Ulric Neisser). Joe Neisser met Pedersen during his own sabbatical at Emory while attending a talk. Like most neurologists specialising in epileptology, Pedersen was familiar with seizure-related déjà vu, as neurologists have been writing about it for more than a century. Joe Neisser and I had met in Savannah, Georgia in 2012 during a symposium he moderated at the Southern Society for Philosophy and Psychology, where I gave a talk on our recent VR study of déjà vu. When Pedersen and Joe Neisser got to talking about seizure-related déjà vu, Neisser described the VR paradigm to Pedersen and suggested that we should all collaborate.
So there I was in 2019 on my own sabbatical, visiting Pedersen’s group at Emory to help get the video-based virtual tour experiment running on a portable computer that could be wheeled on a cart into a patient’s room. Patients with pharmacologically intractable seizures sometimes undergo pre-operative evaluation for surgical candidacy through the use of implanted electrodes with continuous monitoring. While hospitalised for the lengthy monitoring period, interested patients can participate in computerised cognitive tasks while their brain activity is being measured through the electrodes to better understand the function of different brain circuits. As many of the common seizure foci (and thus commonly implanted regions of the brain) happen to be implicated in seizure-related déjà vu, measuring neural activity while a patient completes the virtual tour task might shed light on the mechanisms involved in both familiarity-detection and déjà vu.
While I was there helping to prepare the computer cart for the task, I had a long conversation with Okada about it. An impressive and intellectually curious student, he had already read my publications on déjà vu, and already had several great ideas for extending the research using modern-day VR. He got to work re-creating the virtual tour task for use with present-day VR systems. Using the gaming engine Unity, he created new scenes with new layouts and programmed a VR implementation of the virtual tour paradigm for the Vive headset. The viewer is pulled on rails through a highly realistic scene on a particular path as if on a ride (similar to the It’s a Small World ride at Disney World). In a later identically configured but otherwise novel scene, that precise path is taken through the identical layout of that new scene.
It felt like my mind signalling to me to pause exploring the new scene and instead turn my attention inward to something in my memory
It happened as I sat there in 2022 looking through the Vive headset, in a VR lab room in the Behavioral Sciences Building at Colorado State, testing out Okada’s VR virtual tour program for the first time. I had been exploring the various scenes he had created, looking around while ‘riding’ through them and admiring the detail of the textures and the cleverness of the placement of various realistic-seeming objects within each scene. Then, as I was being pulled through a scene of straw huts along a boardwalk in an oceanside resort, I was suddenly overcome with an intense sense of familiarity. The sensation grabbed hold of my attention and I found myself no longer looking around and taking in the details of the scene but instead intensely focused on trying to figure out why it felt so familiar. It was déjà vu.
At first, I could not figure out exactly why I was experiencing it. That is, I could not identify a specific scene from earlier that might be responsible for the feeling. But my attention had now been fully devoted to trying to figure it out. So, as I continued to be pulled through the scene, I kept going through possibilities in my mind for what might be the reason behind the déjà vu. Eventually, by the time the tour of that scene came to an end and the prompts started appearing, I figured it out. It was the campground from earlier. The campground had an arrangement of tents along a dirt pathway and was identically configured to the layout of the huts along the oceanside boardwalk at the resort. And it happened several more times in several more test scenes as I continued through the program. What I noticed during these experiences was that, while thoroughly enjoying looking around a highly realistic, detailed scene that I had never seen before, I would be hit with a strong feeling of familiarity and would feel certain that the scene was reminding me of something I hadn’t quite placed yet. It felt like my mind signalling to me to pause exploring the novel and interesting scene and instead turn my attention inward to look for something in my memory. Then I would spend a lot of time going through possibilities in my mind. In many such instances, I would eventually figure it out: I would identify the previously viewed scene responsible for the familiarity.
This made me realise that there may be a component to déjà vu that we had been overlooking: it may prompt a flip of attention from outward to inward, to search one’s memory for potentially relevant information. For me, the déjà vu sensation in the VR environment was often a step along the way to eventual recall success, and this facet of the experience might be getting completely missed in our usual research approach of separating instances of recall success and recall failure. Instances of recall success may sometimes be preceded by a feeling of déjà vu – but our studies had not been set up to examine how the memory experience unfolds over time.
Perhaps déjà vu grabs attention and pulls it inward toward a search of memory for potentially relevant information? My students and I began to sift through some of our existing data sets in search of evidence that we might have previously missed. And we found some. For one thing, as reported in a recent article led by my former student Katherine McNeely-White, participants seem to guess more at earlier experienced scenarios when experiencing déjà vu than when not. That is, when experiencing déjà vu, they tend to type inaccurate information into the recall prompt rather than just leaving the recall prompt blank (leaving it blank more often when not experiencing déjà vu). This is consistent with the idea that, during déjà vu, people expend more effort searching their memory trying to conjure potentially relevant information, even if what they generate from the search is incorrect. For another thing, even when participants did leave the recall prompt blank during instances of déjà vu, they spent more time at the prompt before hitting Enter to move on, compared with when they were not experiencing déjà vu. This greater time spent at the recall prompt suggests that participants were likely trying a bit harder to recall an earlier scene when déjà vu was experienced than when it was not. Finally, participants were also more curious to discover whether a studied scene (and if so, which one) might map on to the current scene when experiencing déjà vu than when not.
There are other hints that déjà vu relates to attention. When it accompanies seizure activity, its pull on attention is so powerful that it may provoke some patients to confabulate memories – to invent recollections that help explain away the sensation of reliving something from the past. Much like the active search of memory I myself experienced, this kind of ‘recollective confabulation’ could represent an inward-directed accounting, and the information pulled up, real or not, could be a means of trying to provide oneself relief from the forced, prolonged, inward-directed attention that may ensue during seizure-related déjà vu.
Déjà vu may be an eerie shadow of the mind at work, and a window into the mind’s evolutionary past. Most of the time, our cognitive processing takes place smoothly and effortlessly – we just process the world around us and retrieve relevant information rapidly, without introspective access to how that occurs. It just does. Déjà vu occurs when there is a hiccup in the system, and we notice the pull on our attention; it grabs hold of our focus, allowing us to catch a quick glimpse of our memory’s operation occurring in slow motion. What would ordinarily take place quickly beneath the surface – the unfolding process of familiarity-detection followed by inward-directed attention and retrieval search effort leading to retrieval of relevant information – suddenly has a light shining on the spot where the halt occurred, where the retrieval piece was not successful, and we find ourselves in a heightened state of searching our memory, trying to find out why the situation feels so familiar. But rather than being an odd quirk of memory, this cognitive mechanism could be forcing us to retrieve the very memories we need to survive – and could be evolution’s way of forcing the mind inward, when it needs that insight most.
