Multimedia information fusion.

• Main Author: Woon, Kia Yan.
• Other Authors: Tan Ah Hwee
• Format: Theses and Dissertations
• Language: English
• Published: 2010
• Subjects: DRNTU::Social sciences::Mass media
• Online Access: http://hdl.handle.net/10356/41506
• Institution: Nanyang Technological University

Information in the ubiquitous media age is typically fragmented and appears in various unstructured and unlabelled fonns as data, text, image, audio, and video. For transforming raw information content into knowledge, there is a need to develop various cross-media and media-specific technologies for modeling and working with text, audio, image, and video data as well as their unification and association at the semantic level. As part of the research endeavor of the 12R-SCE, NTU joint project, "Intelligent Technologies for Media Analysis, Representation and Fusion (Intelligent Media)", this dissertation aims to contribute the techniques for information fusion. Following a thorough research of the literature review on the related work, this dissertation presents a self-organizing network model known as fusion Adaptive Resonance Theory (fusion ART) for the fusion of multimedia infonnation. By synchronizing the encoding of infonnation across multiple media channels, the fusion ART model generates clusters that encode the associative mappings among multimedia information in a real-time and continuous manner. The fusion ART's functionalities are illustrated through experiments on two multimedia data sets, namely the terrorist domain data set and Corel data set. In the experiments using the terrorist domain data set, it demonstrates that by incorporating a semantic category channel, fusion ART further enables multi-media infonnation to be fused into predefined themes or semantic categories. In the experiments using the Corel data set, the results suggest the viability of the proposed approach in comparison with other prior work in image annotations, image classification and image-text fusion.