Compressed event sensing (CES) volumes for event cameras

• Main Authors: Lin, Songnan, Ma, Ye, Chen, Jing, Wen, Bihan
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2024
• Subjects: Engineering; Data representation; Compressed sensing
• Online Access: https://hdl.handle.net/10356/180781
• Institution: Nanyang Technological University

Deep learning has made significant progress in event-driven applications. But to match standard vision networks, most approaches rely on aggregating events into grid-like representations, which obscure crucial temporal information and limit overall performance. To address this issue, we propose a novel event representation called compressed event sensing (CES) volumes. CES volumes preserve the high temporal resolution of event streams by leveraging the sparsity property of events and the principles of compressed sensing theory. They effectively capture the frequency characteristics of events in low-dimensional representations, which can be accurately decoded to raw high-dimensional event signals. In addition, our theoretical analysis show that, when integrated with a neural network, CES volumes demonstrates greater expressive power under the neural tangent kernel approximation. Through synthetic phantom validation on dense frame regression and two downstream applications involving intensity-image reconstruction and object recognition tasks, we demonstrate the superior performance of CES volumes compared to state-of-the-art event representations.