Temporal decoding of naturalistic fMRI with computationally efficient deep neural networks

Abstract

Deep learning has demonstrated strong potential for decoding cognitive states from functional magnetic resonance imaging (fMRI) data, but its substantial computational requirements and limited capacity to capture temporal dynamics often hinder practical application. This study introduces a computationally efficient deep learning framework designed for the classification of cognitive states (in this case, face processing) from single events in naturalistic fMRI, where stimuli are temporally rich and dynamically evolving. To facilitate efficient processing while preserving temporal complexity, fMRI related to face onset and face offset events from 86 participants from the Naturalistic Neuroimaging Database version 2.0 (Aliko et al., 2021) were transformed from their original 4D spatiotemporal format into 2D voxel-by-time matrices that explicitly incorporate temporal information. We developed a lightweight convolutional neural network (CNN) that extracts both spatial and temporal patterns in brain activity, enabling efficient analysis with minimal computational cost. The model achieved 82% test accuracy and 89% validation AUC while preserving temporal information. Attribution analysis using Integrated Gradients (via DeepExplain) highlighted activation patterns in expected face-processing regions including Fusiform Face Area (FFA), Occipital Face Area (OFA), and posterior superior temporal sulcus (pSTS), validating that the predictions were driven by relevant neural activity. Therefore, this framework provides a solution for moment-to-moment brain decoding in ecologically valid, naturalistic environments. This work is positioned as a proof-of-concept framework rather than as a performance benchmark.