*Result*: A Prompt-Guided Generative Language Model for Unifying Visual Neural Decoding Across Multiple Subjects and Tasks.

Title:
A Prompt-Guided Generative Language Model for Unifying Visual Neural Decoding Across Multiple Subjects and Tasks.
Authors:
Huang W; The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Brain-Computer Interface & Brain-Inspired Intelligence, Key Laboratory of Sichuan Province, Chengdu, P. R. China., Li H; The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Brain-Computer Interface & Brain-Inspired Intelligence, Key Laboratory of Sichuan Province, Chengdu, P. R. China., Qin F; The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Brain-Computer Interface & Brain-Inspired Intelligence, Key Laboratory of Sichuan Province, Chengdu, P. R. China., Wu D; The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Brain-Computer Interface & Brain-Inspired Intelligence, Key Laboratory of Sichuan Province, Chengdu, P. R. China., Cheng K; College of Language Intelligence, Sichuan International Studies University, Chongqing 400031, P. R. China., Chen H; The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, University of Electronic Science and Technology of China, Brain-Computer Interface & Brain-Inspired Intelligence, Key Laboratory of Sichuan Province, Chengdu 611731, P. R. China.
Source:
International journal of neural systems [Int J Neural Syst] 2026 Feb; Vol. 36 (2), pp. 2550068. Date of Electronic Publication: 2025 Sep 26.
Publication Type:
Journal Article
Language:
English
Journal Info:
Publisher: World Scientific Pub. Co Country of Publication: Singapore NLM ID: 9100527 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1793-6462 (Electronic) Linking ISSN: 01290657 NLM ISO Abbreviation: Int J Neural Syst Subsets: MEDLINE
Imprint Name(s):
Original Publication: Singapore ; Teaneck, N.J. : World Scientific Pub. Co., c1989-
Contributed Indexing:
Keywords: Visual decoding; functional magnetic resonance imaging; generative language model
Entry Date(s):
Date Created: 20250927 Date Completed: 20260101 Latest Revision: 20260101
Update Code:
20260130
DOI:
10.1142/S0129065725500686
PMID:
41006944
Database:
MEDLINE

*Further Information*

*Visual neural decoding not only aids in elucidating the neural mechanisms underlying the processing of visual information but also facilitates the advancement of brain-computer interface technologies. However, most current decoding studies focus on developing separate decoding models for individual subjects and specific tasks, an approach that escalates training costs and consumes a substantial amount of computational resources. This paper introduces a Prompt-Guided Generative Visual Language Decoding Model (PG-GVLDM), which uses prompt text that includes information about subjects and tasks to decode both primary categories and detailed textual descriptions from the visual response activities of multiple individuals. In addition to visual response activities, this study also incorporates a multi-head cross-attention module and feeds the model with whole-brain response activities to capture global semantic information in the brain. Experiments on the Natural Scenes Dataset (NSD) demonstrate that PG-GVLDM attains an average category decoding accuracy of 66.6% across four subjects, reflecting strong cross-subject generalization, and achieves text decoding scores of 0.342 (METEOR), 0.450 (Sentence-Transformer), 0.283 (ROUGE-1), and 0.262 (ROUGE-L), establishing state-of-the-art performance in text decoding. Furthermore, incorporating whole-brain response activities significantly enhances decoding performance by enabling the integration of distributed neural signals into coherent global semantic representations, underscoring its methodological importance for unified neural decoding. This research not only represents a breakthrough in visual neural decoding methodologies but also provides theoretical and technical support for the development of generalized brain-computer interfaces.*