Semantic context modeling with maximal margin conditional random fields for automatic image annotation

Context modeling for Vision Recognition and Automatic Image Annotation (AIA) has attracted increasing attentions in recent years. For various contextual information and resources, semantic context has been exploited in AIA and brings promising results. However, previous works either casted the probl...

Full description

Saved in:
Bibliographic Details
Main Authors: XIANG, Yu, ZHOU, Xiangdong, LIU, Zuotao, CHUA, Tat-Seng, NGO, Chong-wah
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2010
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/6601
https://ink.library.smu.edu.sg/context/sis_research/article/7604/viewcontent/xiang_cvpr10.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-7604
record_format dspace
spelling sg-smu-ink.sis_research-76042022-01-13T08:20:00Z Semantic context modeling with maximal margin conditional random fields for automatic image annotation XIANG, Yu ZHOU, Xiangdong LIU, Zuotao CHUA, Tat-Seng NGO, Chong-wah Context modeling for Vision Recognition and Automatic Image Annotation (AIA) has attracted increasing attentions in recent years. For various contextual information and resources, semantic context has been exploited in AIA and brings promising results. However, previous works either casted the problem into structural classification or adopted multi-layer modeling, which suffer from the problems of scalability or model efficiency. In this paper, we propose a novel discriminative Conditional Random Field (CRF) model for semantic context modeling in AIA, which is built over semantic concepts and treats an image as a whole observation without segmentation. Our model captures the interactions between semantic concepts from both semantic level and visual level in an integrated manner. Specifically, we employ graph structure to model contextual relationships between semantic concepts. The potential functions are designed based on linear discriminative models, which enables us to propose a novel decoupled hinge loss function for maximal margin parameter estimation. We train the model by solving a set of independent quadratic programming problems with our derived contextual kernel. The experiments are conducted on commonly used benchmarks: Corel and TRECVID data sets for evaluation. The experimental results show that compared with the state-of-the-art methods, our method achieves significant improvement on annotation performance. 2010-06-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/6601 info:doi/10.1109/CVPR.2010.5540015 https://ink.library.smu.edu.sg/context/sis_research/article/7604/viewcontent/xiang_cvpr10.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Databases and Information Systems Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Databases and Information Systems
Graphics and Human Computer Interfaces
spellingShingle Databases and Information Systems
Graphics and Human Computer Interfaces
XIANG, Yu
ZHOU, Xiangdong
LIU, Zuotao
CHUA, Tat-Seng
NGO, Chong-wah
Semantic context modeling with maximal margin conditional random fields for automatic image annotation
description Context modeling for Vision Recognition and Automatic Image Annotation (AIA) has attracted increasing attentions in recent years. For various contextual information and resources, semantic context has been exploited in AIA and brings promising results. However, previous works either casted the problem into structural classification or adopted multi-layer modeling, which suffer from the problems of scalability or model efficiency. In this paper, we propose a novel discriminative Conditional Random Field (CRF) model for semantic context modeling in AIA, which is built over semantic concepts and treats an image as a whole observation without segmentation. Our model captures the interactions between semantic concepts from both semantic level and visual level in an integrated manner. Specifically, we employ graph structure to model contextual relationships between semantic concepts. The potential functions are designed based on linear discriminative models, which enables us to propose a novel decoupled hinge loss function for maximal margin parameter estimation. We train the model by solving a set of independent quadratic programming problems with our derived contextual kernel. The experiments are conducted on commonly used benchmarks: Corel and TRECVID data sets for evaluation. The experimental results show that compared with the state-of-the-art methods, our method achieves significant improvement on annotation performance.
format text
author XIANG, Yu
ZHOU, Xiangdong
LIU, Zuotao
CHUA, Tat-Seng
NGO, Chong-wah
author_facet XIANG, Yu
ZHOU, Xiangdong
LIU, Zuotao
CHUA, Tat-Seng
NGO, Chong-wah
author_sort XIANG, Yu
title Semantic context modeling with maximal margin conditional random fields for automatic image annotation
title_short Semantic context modeling with maximal margin conditional random fields for automatic image annotation
title_full Semantic context modeling with maximal margin conditional random fields for automatic image annotation
title_fullStr Semantic context modeling with maximal margin conditional random fields for automatic image annotation
title_full_unstemmed Semantic context modeling with maximal margin conditional random fields for automatic image annotation
title_sort semantic context modeling with maximal margin conditional random fields for automatic image annotation
publisher Institutional Knowledge at Singapore Management University
publishDate 2010
url https://ink.library.smu.edu.sg/sis_research/6601
https://ink.library.smu.edu.sg/context/sis_research/article/7604/viewcontent/xiang_cvpr10.pdf
_version_ 1770575999414566912