Below please find information in regards to BME PhD student Jason Ki’s dissertation defense, which is open to all and will take place on Tuesday, March 30th at 3:00pm via Zoom. Please email Jason at jki00@citymail.cuny.edu for the Zoom link. Jason’s abstract is also below.
When the Brain Plays a Game:
Neural responses to visual dynamics during naturalistic visual tasks.
Department of Biomedical Engineering
Jason Ki
Mentor: Lucas C Parra
ABSTRACT
Many day-to-day tasks involve processing of complex visual information in a continuous stream. While much of our knowledge on visual processing has been established from reductionist approaches in lab-controlled settings, very little is known about the processing of complex dynamic stimuli experienced in everyday scenarios. Traditional investigations employ event-related paradigms that involve presentation of simple stimuli at select locations in visual space or discrete moments in time. In contrast, visual stimuli in real-life are highly dynamic, spatially-heterogeneous, and semantically rich. Moreover, traditional experiments impose unnatural task constraints (e.g. inhibited saccades), thus, it is unclear whether theories developed under the reductionist approach apply in naturalistic settings. Given these limitations, alternative experimental paradigms and analysis methods are necessary. Here, we introduce a new approach for investigating visual processing, applying the system identification (SI) framework. We investigate the modulation of stimulus-evoked responses during a naturalistic task (i.e. kart race game) using non-invasive scalp recordings.
In recent years, multivariate modeling approaches have become increasingly popular for assessing neural response to naturalistic stimulus. Encoding models use stimulus patterns to predict brain responses and decoding models use patterns of brain responses to predict stimulus that drove these responses. In this dissertation, we employ a hybrid method that “encodes” the stimulus to predict “decoded” brain responses. With this approach, we measure the stimulus-response correlation (SRC, i.e. temporal correlation of neural response and dynamic stimulus) to assess the strength of stimulus-evoked activity to uniquely experienced naturalistic stimulus. To demonstrate this, we measured the SRC during a kart race videogame. We find that SRC increased with active play of the game, suggesting that stimulus-evoked activity is modulated by the visual task demands. Furthermore, we analyzed the selectivity of neural response across the visual space. While it is well-established that neural response is spatially selective to discrete stimulus, it is unclear whether this is true during naturalistic stimulus presentation. To assess this, we measured the correlation of neural response with optical flow magnitude at individual locations on the screen during the videogame. We find that the SRC is greater for locations in space that are task-relevant, enhancing during active play. Moreover, the spatial selectivity differs across scalp locations, which suggest that individual brain regions are spatially selective to different visual dynamics.
Overall, we leveraged the SI framework to investigate visual processing during a naturalistic stimulus presentation, extending visual research to ecologically valid paradigms. Our findings shed new insights about the stimulus-evoked neural response to visual dynamics during a uniquely experienced naturalistic visual task. We show that selectivity of neural response can be spatially-resolved at pixel-level from a low-SNR EEG. In the future, by further probing other spatial and temporal dimensions of the stimuli (beyond optical flow), we may gain new insights into how neural signals convey visual processing during dynamic natural visual experiences.
