Evaluating deep transfer learning for whole-brain cognitive decoding |
| |
| Institution: | 1. Department of Electrical Engineering and Computer Science, Technische Universität Berlin, Berlin, Germany;2. Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany;3. Stanford Data Science, Stanford University, Stanford, CA, USA;4. Department of Psychology, Stanford University, Stanford, CA, USA;5. Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK;6. Department of Artificial Intelligence, Fraunhofer Heinrich Hertz Institute, Berlin, Germany;7. BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany;8. Department of Artificial Intelligence, Korea University, Seoul, South Korea;9. Max Planck Institute for Informatics, Saarbrücken, Germany;1. School of Automation, Nanjing University of Information Science and Technology, Nanjing, 210044, China;2. Collaborative Innovation Center of Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing, 210044, China;3. Jiangsu Province Engineering Research Center of Intelligent Meteorological Exploration Robot, Nanjing, 210044, China;1. Federal University of Minas Gerais, Graduate Program in Electrical Engineering, Av. Antonio Carlos, 6627, Belo Horizonte, MG 31270-901, Brazil;2. Federal University of Minas Gerais, Department of Electronics Engineering, Av. Antonio Carlos, 6627, Belo Horizonte, MG 31270-901, Brazil;3. Federal University of São João del-Rei, Department of Electrical Engineering, Praça Frei Orlando, 170, São João del-Rei, MG, Brazil;1. Chongqing Key Laboratory of Nonlinear Circuits and Intelligent Information Processing, College of Electronic and Information Engineering, Southwest University, Chongqing 400715, PR China;2. School of Computer Science and Engineering, Chongqing Three Gorges University, Chongqing 404120, PR China;1. College of Information Engineering, Shenyang University of Chemical Technology, Shenyang, 110142, China;2. State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, 110169, China;3. Institutes for Robotics and Intelligent Manufacturing, Chinese Academy of Sciences, Shenyang, 110169, China;4. University of Chinese Academy of Sciences, Beijing, 100049, China;5. The College of Automation, Shenyang Aerospace University, Shenyang, 110136, China;1. Technological Research Institute (IRT) of the Railway Sector RAILENIUM, France;2. Univ. Polytechnique Hauts-de-France, LAMIH, CNRS, UMR 8201, Valenciennes F-59313, France;3. INSA Hauts-de-France, Valenciennes F-59313, France |
| |
| Abstract: | Research in many fields has shown that transfer learning (TL) is well-suited to improve the performance of deep learning (DL) models in datasets with small numbers of samples. This empirical success has triggered interest in the application of TL to cognitive decoding analyses with functional neuroimaging data. Here, we systematically evaluate TL for the application of DL models to the decoding of cognitive states (e.g., viewing images of faces or houses) from whole-brain functional Magnetic Resonance Imaging (fMRI) data. We first pre-train two DL architectures on a large, public fMRI dataset and subsequently evaluate their performance in an independent experimental task and a fully independent dataset. The pre-trained DL models consistently achieve higher decoding accuracies and generally require less training time and data than model variants that were not pre-trained, while also outperforming linear baseline models trained from scratch, clearly underlining the benefits of pre-training. We demonstrate that these benefits arise from the ability of the pre-trained models to reuse many of their learned features when training with new data, providing deeper insights into the mechanisms giving rise to the benefits of pre-training. Yet, we also surface nuanced challenges for whole-brain cognitive decoding with DL models when interpreting the decoding decisions of the pre-trained models, as these have learned to utilize the fMRI data in unforeseen and counterintuitive ways to identify individual cognitive states. |
| |
| Keywords: | |
| 本文献已被 ScienceDirect 等数据库收录！ |
|
