Investigating the Neural Correlates of Deja vu
Team: Noah Okada, Katherine McNeely-White, Daniel Drane, Joseph Neisser, Thomas Parsons, Timothy McMahan, Nigel P.Pedersen, Anne M. Cleary
Since 2019, I have been working with Dr. Daniel Drane and Dr. Nigel Pedersen in the Epilepsy and Systems Neuroscience Laboratory at the Emory School of Medicine to develop virtual reality (VR) assessments to study the neural correlates of cognitive phenomena. Specifically, my project has focused on the development of a framework that will enable the simultaneous presentation of VR stimuli, paired with direct brain recording and stimulation using stereoelectroencephalography (SEEG) to study the neural activity associated with familiarity. SEEG consists of placing depth electrodes into the brains of patients with epilepsy to understand the brain network response for their seizures - a critical step in planning potentially curative surgery. Although memory and familiarity have been studied extensively using neuroimaging, there is a limited understanding of the neural circuitry underlying these phenomena. Researchers have struggled to replicate the multimodal complexities of memory “in the wild” using traditional cognitive and neuropsychological tests; therefore memory research has been limited to assessing specific measures to represent cognitive processes in artificial settings. Recent advancements in immersive VR offer the potential to change this by enabling researchers to immerse participants in ecologically valid virtual environments. Therefore, my project pairing the SEEG recordings with VR memory paradigms enables the study of memory circuitry that can substantially increase our understanding of the neural circuits of cognitive functions.
Regions of Interest
The Association between Deja Vu and the MTL
Our research group is particularly interested in using these new paradigms to investigate the role of the perirhinal area and the hippocampal formation in eliciting familiarity as has been implicated in the studies of déjà vu in people with epilepsy. Déjà vu is the phenomenon of experiencing a new situation with an overwhelming feeling of familiarity and sometimes prediction. It is an experience that has been recorded across the general population, with a modified form prominent in patients with temporal lobe epilepsy (TLE). Studies focusing on patients with TLE have established that alterations in electrical activity within the medial temporal lobe (MTL) can be correlated with the feeling of déjà vu. It has therefore been suggested that disturbances of the recollection and recognition systems within the MTL produce déjà vu.
Phenomena of Interest
How do you elicit Deja Vu?
This association has been difficult to study due to the rarity of spontaneous déjà vu and the difficulty of eliciting the distinct experience in a laboratory setting. Previous studies have found that a form of this phenomenon can be elicited when the configuration of elements within an environment maps onto a previously seen spatial configuration. Leveraging these insights, our research group established an interdisciplinary collaboration to develop a new generation of immersive VR paradigms to study déjà vu and the related familiarity circuitry within the MTL.
Regions of Interest
Building VR Environments that Elicit Deja Vu
Seeking to leverage the advancements in modern immersive VR, I developed a framework using the Unity Game Engine. This platform allowed me to design a repository of 128 categorically distinct virtual environments using open-source 3D objects from public repositories. To replicate previous studies of déjà vu, I designed a set of 64 primary scenes with a configurationally similar counterpart for each scene. As I developed the scenes I noticed that previous studies had measured the configurational similarity of the stimuli using only qualitative measures. Understanding that the configuration of objects in the scenes was pivotal to our hypothesis, I proposed a quantitative method to analyze spatial configuration of the objects in the virtual environments. To accomplish this I taught myself how to use the MATLAB Image Processing toolkits to segment the scenes into indices of spatial similarity. This resulted in the ability to quantitatively correlate the configurational similarity to the incidence of reported déjà vu. Understanding that these stimuli had the potential to be used in a variety of experimental settings, I used C# scripts to develop a dynamic framework with multi-platform compatibility. This feature allowed our research group to test our hypothesis at varying levels of immersion while also recording simultaneous neural activity (e.g. fMRI and intracranial EEG), as well as other data (e.g. videography of behavior, EKG, eye-tracking).
Experiments & Results
Across three experiments, each with varying levels of immersion (Experiment 1: N = 56 remote desktop-based, non-immersive VR; Experiment 2: N = 62 in-person laboratory desktop-based, non-immersive VR; Experiment 3: N = 20 in-person laboratory, immersive head-mounted displayed-based), participants were presented with test lists consisting of novel visual scenes that potentially shared spatial layouts with previously studied scenes. When participants failed to recall the spatially similar studied scene, they were more likely to find the test scene familiar, either through familiarity ratings (Experiment 1: p = .002; Experiment 2: p < .001) or through yes-no familiarity judgements (Experiment 3: p = .002) if it configurationally overlapped with a studied scene than if it did not. Furthermore, participants were significantly more likely to indicate a sense of déjà vu for test scenes that spatially corresponded to previously studied scenes (Experiment 1: p = .01; Experiment 2: p < .001; Experiment 3: p = .02). These findings collectively indicate that the virtual environment stimuli developed for this project can be used to study déjà vu experiences, as demonstrated by the increased probability of experiencing déjà vu during retrieval failure. This validated VR paradigm will enable the development of highly controlled ecologically valid experiments to further explore the neural mechanisms underlying familiarity and déjà vu.