Perception coordination network: A neuro framework for multimodal concept acquisition and binding

• Main Authors: XING, You-Lu, SHI, Xiao-Feng, SHEN, Fu-Rao, ZHAO, Jin-Xi, PAN, Jing-Xin, TAN, Ah-hwee
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2019
• Subjects: Concept acquisition and binding; multimodal learning; online incremental learning; perception coordination network (PCN); unsupervised learning; OS and Networks; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/5249; https://ink.library.smu.edu.sg/context/sis_research/article/6252/viewcontent/tnnls19_Perception_av.pdf
• Institution: Singapore Management University

To simulate the concept acquisition and binding of different senses in the brain, a biologically inspired neural network model named perception coordination network (PCN) is proposed. It is a hierarchical structure, which is functionally divided into the primary sensory area (PSA), the primary sensory association area (SAA), and the higher order association area (HAA). The PSA contains feature neurons which respond to many elementary features, e.g., colors, shapes, syllables, and basic flavors. The SAA contains primary concept neurons which combine the elementary features in the PSA to represent unimodal concept of objects, e.g., the image of an apple, the Chinese word “[píng guǒ]” which names the apple, and the taste of the apple. The HAA contains associated neurons which connect the primary concept neurons of several PSA, e.g., connects the image, the taste, and the name of an apple. It means that the associated neurons have a multimodal response mode. Therefore, this area executes multisensory integration. PCN is an online incremental learning system, it is able to continuously acquire and bind multimodality concepts in an online way. The experimental results suggest that PCN is able to handle the multimodal concept acquisition and binding effectively.