The BeMi Stardust: A structured ensemble of Binarized Neural Networks

• Main Authors: BERNARDELLI, Ambrogio Maria, GUALANDI, Stefano, LAU, Hoong Chuin, MILANESI, Simone
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Binarized neural networks; Mixed-integer linear programming; Structured ensemble of neural networks; Artificial Intelligence and Robotics; OS and Networks
• Online Access: https://ink.library.smu.edu.sg/sis_research/8310; https://ink.library.smu.edu.sg/context/sis_research/article/9313/viewcontent/BeMiStardust_av.pdf
• Institution: Singapore Management University

Binarized Neural Networks (BNNs) are receiving increasing attention due to their lightweight architecture and ability to run on low-power devices, given the fact that they can be implemented using Boolean operations. The state-of-the-art for training classification BNNs restricted to few-shot learning is based on a Mixed Integer Programming (MIP) approach. This paper proposes the BeMi ensemble, a structured architecture of classification-designed BNNs based on training a single BNN for each possible pair of classes and applying a majority voting scheme to predict the final output. The training of a single BNN discriminating between two classes is achieved by a MIP model that optimizes a lexicographic multi-objective function according to robustness and simplicity principles. This approach results in training networks whose output is not affected by small perturbations on the input and whose number of active weights is as small as possible, while good accuracy is preserved. We computationally validate our model using the MNIST and Fashion-MNIST datasets using up to 40 training images per class. Our structured ensemble outperforms both BNNs trained by stochastic gradient descent and state-of-the-art MIP-based approaches. While the previous approaches achieve an average accuracy of on the MNIST dataset, the BeMi ensemble achieves an average accuracy of when trained with 10 images per class and when trained with 40 images per class.