Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware
Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI. However, as an image usually contains much spatial redundancy, e.g., background pixels, directly shrinking the whole image will lose impo...
Saved in:
Main Authors: | , , , , , , , |
---|---|
Other Authors: | |
Format: | Conference or Workshop Item |
Language: | English |
Published: |
2023
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/164833 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-164833 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1648332023-12-15T02:45:49Z Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware Kong, Hao Liu, Di Huai, Shuo Luo, Xiangzhong Liu, Weichen Subramaniam, Ravi Makaya, Christian Lin, Qian School of Computer Science and Engineering 2022 IEEE/ACM International Conference On Computer Aided Design (ICCAD) HP-NTU Digital Manufacturing Corporate Lab Engineering::Computer science and engineering Convolutional Neural Networks Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI. However, as an image usually contains much spatial redundancy, e.g., background pixels, directly shrinking the whole image will lose important features of the foreground object and lead to severe accuracy degradation. In this paper, we propose a dynamic image cropping framework to reduce the spatial redundancy by accurately cropping the foreground object from images. To achieve the instance-aware fine cropping, we introduce a lightweight foreground predictor to efficiently localize and crop the foreground of an image. The finely cropped images can be correctly recognized even at a small resolution. Meanwhile, computational redundancy also exists in CNN architectures. To pursue higher execution efficiency on resource-constrained embedded devices, we also propose a compound shrinking strategy to coordinately compress the three dimensions (depth, width, resolution) of CNNs. Eventually, we seamlessly combine the proposed dynamic image cropping and compound shrinking into a unified compression framework, Smart Scissor, which is expected to significantly reduce the computational overhead of CNNs while still maintaining high accuracy. Experiments on ImageNet-1K demonstrate that our method reduces the computational cost of ResNet50 by 41.5% while improving the top-1 accuracy by 0.3%. Moreover, compared to HRank, the state-of-theart CNN compression framework, our method achieves 4.1% higher top-1 accuracy at the same computational cost. The codes and data are available at https://github.com/ntuliuteam/smart-scissor. Nanyang Technological University National Research Foundation (NRF) Submitted/Accepted version This study is partially supported under the RIE2020 Industry Alignment Fund – Industry Collaboration Projects (IAF-ICP) Funding Initiative, as well as cash and in-kind contribution from the industry partner, HP Inc., through the HP-NTU Digital Manufacturing Corporate Lab (I1801E0028). This work is also partially supported by Nanyang Technological University, Singapore, under its NAP (M4082282). 2023-02-23T08:09:55Z 2023-02-23T08:09:55Z 2022 Conference Paper Kong, H., Liu, D., Huai, S., Luo, X., Liu, W., Subramaniam, R., Makaya, C. & Lin, Q. (2022). Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware. 41st IEEE/ACM International Conference on Computer-Aided Design, 1-9. https://dx.doi.org/10.1145/3508352.3549397 9781450392174 https://hdl.handle.net/10356/164833 10.1145/3508352.3549397 2-s2.0-85145650156 1 9 en I1801E0028 M4082282 10.21979/N9/NB6FU2 © 2022 Association for Computing Machinery. All rights reserved. This paper was published in the Proceedings of IEEE/ACM International Conference On Computer Aided Design (ICCAD) and is made available with permission of Association for Computing Machinery. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Computer science and engineering Convolutional Neural Networks |
spellingShingle |
Engineering::Computer science and engineering Convolutional Neural Networks Kong, Hao Liu, Di Huai, Shuo Luo, Xiangzhong Liu, Weichen Subramaniam, Ravi Makaya, Christian Lin, Qian Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware |
description |
Scaling down the resolution of input images can greatly reduce the computational overhead of convolutional neural networks (CNNs), which is promising for edge AI. However, as an image usually contains much spatial redundancy, e.g., background pixels, directly shrinking the whole image will lose important features of the foreground object and lead to severe accuracy degradation. In this paper, we propose a dynamic image cropping framework to reduce the spatial redundancy by accurately cropping the foreground object from images. To achieve the instance-aware fine cropping, we introduce a lightweight foreground predictor to efficiently localize and crop the foreground of an image. The finely cropped images can be correctly recognized even at a small resolution. Meanwhile, computational redundancy also exists in CNN architectures. To pursue higher execution efficiency on resource-constrained embedded devices, we also propose a compound shrinking strategy to coordinately compress the three dimensions (depth, width, resolution) of CNNs. Eventually, we seamlessly combine the proposed dynamic image cropping and compound shrinking into a unified compression framework, Smart Scissor, which is expected to significantly reduce the computational overhead of CNNs while still maintaining high accuracy. Experiments on ImageNet-1K demonstrate that our method reduces the computational cost of ResNet50 by 41.5% while improving the top-1 accuracy by 0.3%. Moreover, compared to HRank, the state-of-theart CNN compression framework, our method achieves 4.1% higher top-1 accuracy at the same computational cost. The codes and data are available at https://github.com/ntuliuteam/smart-scissor. |
author2 |
School of Computer Science and Engineering |
author_facet |
School of Computer Science and Engineering Kong, Hao Liu, Di Huai, Shuo Luo, Xiangzhong Liu, Weichen Subramaniam, Ravi Makaya, Christian Lin, Qian |
format |
Conference or Workshop Item |
author |
Kong, Hao Liu, Di Huai, Shuo Luo, Xiangzhong Liu, Weichen Subramaniam, Ravi Makaya, Christian Lin, Qian |
author_sort |
Kong, Hao |
title |
Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware |
title_short |
Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware |
title_full |
Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware |
title_fullStr |
Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware |
title_full_unstemmed |
Smart scissor: coupling spatial redundancy reduction and CNN compression for embedded hardware |
title_sort |
smart scissor: coupling spatial redundancy reduction and cnn compression for embedded hardware |
publishDate |
2023 |
url |
https://hdl.handle.net/10356/164833 |
_version_ |
1787136740491788288 |