A practical man-in-the-middle attack on deep learning edge device by sparse light strip injection into camera data lane

• Main Authors: Liu, Wenye, He, Weiyang, Hu, Bowen, Chang, Chip-Hong
• Other Authors: School of Electrical and Electronic Engineering
• Format: Conference or Workshop Item
• Language: English
• Published: 2023
• Subjects: Engineering::Computer science and engineering::Hardware::Input/output and data communications; Engineering::Electrical and electronic engineering::Computer hardware, software and systems; Cameras; Image Edge Detection
• Online Access: https://hdl.handle.net/10356/165204
• Institution: Nanyang Technological University

The vulnerability of deep neural networks (DNNs) has been exposed by adversarial examples. Although the adversarial perturbations can be made visually imperceptible or photorealistic on any image, they have to be added offline on pre-captured static input in order to accomplish the malicious goal. As opposed to subtle distortion, real-time misclassification on streaming images can be realized by manipulating the objects in physical world. Recently, object-contactless physical attacks, as exemplified by a translucent sticker affixed to the lens of a camera, show that a sensor-enabled edge computing platform can be an alluring target of adversarial attack. Nevertheless, success rates of reported camera-based patch attacks are not high enough to overshadow other forms of evasion attacks even when they are performed under the white-box scenario. In this paper, we present a practical and robust fault injection approach cooperated with a hardware-friendly sparse strip pattern to deceive the deployed DNN device on real-time streaming images. The strip perturbation is generated in a line-offset form by an optimization algorithm. It can be injected into camera data lane between the image sensor and the endpoint node stealthily without disturbing the data traffic through an interface bridge implemented by a tiny off-the-shelf FPGA device. We demonstrate our attack on the Raspberry Pi 4 platform with the Pi camera v2 and the Intel NCS2 inference stick. By evaluating 280 physically captured images from ten objects in 28 viewing angles, we show that the proposed attack on four ImageNet models including ResNet50, MobileNet-v2, Inception-v3 and EfficientNet-B0 can achieve 89.2% ∼ 96.1% success rates.