Distance based image classification: A solution to generative classification’s conundrum?

• Main Authors: LIN, Wen-yan, LIU, Siying, DAI, Bing Tian, LI, Hongdong
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: incremental learning; high dimensions; statistics; shell theory; generative classifiers; anomalydetection; nearest neighbor; distance; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7309; https://ink.library.smu.edu.sg/context/sis_research/article/8312/viewcontent/template173.pdf
• Institution: Singapore Management University

Most classifiers rely on discriminative boundaries that separate instances of each class from everything else. We argue that discriminative boundaries are counter-intuitive as they define semantics by what-they-are-not; and should be replaced by generative classifiers which define semantics by what-they-are. Unfortunately, generative classifiers are significantly less accurate. This may be caused by the tendency of generative models to focus on easy to model semantic generative factors and ignore non-semantic factors that are important but difficult to model. We propose a new generative model in which semantic factors are accommodated by shell theory’s [25] hierarchical generative process and non-semantic factors by an instance specific noise term. We use the model to develop a classification scheme which suppresses the impact of noise while preserving semantic cues. The result is a surprisingly accurate generative classifier, that takes the form of a modified nearest-neighbor algorithm; we term it distance classification. Unlike discriminative classifiers, a distance classifier: defines semantics by what-they are; is amenable to incremental updates; and scales well with the number of classes.