Dynamic point stochastic rounding algorithm for limited precision arithmetic in Deep Belief Network training
This paper reports how to train a Deep Belief Network (DBN) using only 8-bit fixed-point parameters. We propose a dynamic-point stochastic rounding algorithm that provides enhanced results compared to the existing stochastic rounding. We show that by using a variable scaling factor, the fixed-point...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Published: |
IEEE Computer Society
2017
|
| Online Access: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85028585216&doi=10.1109%2fNER.2017.8008430&partnerID=40&md5=3d565633d6b0ee148b8349a9b15f1d76 http://eprints.utp.edu.my/20033/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Institution: | Universiti Teknologi Petronas |
| Summary: | This paper reports how to train a Deep Belief Network (DBN) using only 8-bit fixed-point parameters. We propose a dynamic-point stochastic rounding algorithm that provides enhanced results compared to the existing stochastic rounding. We show that by using a variable scaling factor, the fixed-point parameter updates are enhanced. To be more hardware amenable, the use of common scaling factor at each layer of DBN is further proposed. Using publicly available MNIST database, we show that the proposed algorithm can train a 3-layer DBN with an average accuracy of 98.49, with a drop of 0.08 from the double floating-point average accuracy. © 2017 IEEE. |
|---|