Virtual Escape Room
How do every day stressors impact our capacity for insight and problem solving? Because problem solving is a complex ability that involves many aspects of cognition. Our group approaches this question by integrating clues from many sources, including human behavior, brain activity, and autonomic function. We are working to relate changes in self-reported stress and fatigue to variability in solution search strategies, allocation of visual attention, measures of arousal and neurocognitive engagement.
Our methods leverage recent advances in lightweight biometric sensors, virtual reality, eye tracking, EEG, as well as state of the art immersive Virtual Reality. This unique collaboration between UC San Diego’s Swartz Center for Computational Neuroscience and the Arthur C. Clarke Center for Human Imagination has given rise to an interdisciplinary team of neuroscientists, computer scientists and artists working together to pose problems in new ways and advance scientific methods and knowledge with full STEAM ahead.
Publication status: ONGOING
This project is funded by the National Science Foundation.
VR Escape Room Demo
Ying Wu
By simultaneously studying behavior — including the allocation of visual attention — brain activity, and autonomic arousal, we hope to obtain a more robust understanding of problem solving in realistic environments than would be possible by studying each modality individually.
Integrating Electrocardiographic Data with EEG
Ying Wu October 2, 2019
Electrocardiography (ECG) is an important bio-metric tool for estimating aspects of mental states related to stress, relaxation, and more. The HeartyPatch is a wireless ECG recording device with a low cost and easy setup.
Eye Tracking in VR
Ying Wu September 30, 2019
During Escape Room problem solving, eye gaze co-ordinates (x, y, z) detected by the Vive Pro Eye headset are converted into a stream of gaze direction vectors. These vectors are combined with information about the user’s head direction and eye position inside the game world in order to create a ray.
Escape Room Playback
Ying Wu September 19, 2019
It is useful to retain a complete record of all actions and events in each Escape Room problem solving session for downstream visualization and validation purposes. We created an interface that records the unique names and positions of all active game objects at the start of each session, and then records position changes on a frame-by-frame basis at a rate of 30 frames per second.
Non-Stationary Brain Network Dynamics
Ying Wu September 16, 2019
Segmenting high dimensional physiological or behavioral data during Escape Room problem solving can be challenge. Here, we propose to track changes in non-stationary brain network dynamics. While working to solve Escape Room puzzles, participants log their motivational state (happy, lost, ambivalent, angry, sad, thinking, curious), how close they feel to solving the puzzle, and how much they would like to continue searching at five-minute intervals by means of an in-game survey that appears in the virtual environment.
Creating Escape Room Scenarios
Josh Pallag November 17, 2018
Throughout this process, our main goal has been to design a series of puzzles of equal difficulty in a virtual reality space that can be used to test problem solving and insight. With the desired test length of roughly 25 minutes, we aimed to have approximately 3 puzzles per environment.