Neural Network models most often exploit the SoftMax function in the classification stage for computing probabilities through exponentiation and division operations. To reduce the complexity and the energy consumption of such stage, several hardware-friendly approximation strategies have been disclosed in the recent past. This brief evaluates the effects of an aggressive approximation of the SoftMax layer on both classification accuracy and hardware characteristics. Experimental results demonstrate that the proposed circuit, when implemented in a 28 nm FDSOI technology, saves 65% of silicon area with respect to competitors, dissipating less than 1 pJ. FPGA implementation results confirm a massive energy dissipation reduction with respect to the conventional baseline architecture, without introducing penalties in the Top-1 accuracy.

Aggressive Approximation of the SoftMax Function for Power-Efficient Hardware Implementations

Spagnolo, Fanny;Perri, Stefania;Corsonello, Pasquale
2022-01-01

Abstract

Neural Network models most often exploit the SoftMax function in the classification stage for computing probabilities through exponentiation and division operations. To reduce the complexity and the energy consumption of such stage, several hardware-friendly approximation strategies have been disclosed in the recent past. This brief evaluates the effects of an aggressive approximation of the SoftMax layer on both classification accuracy and hardware characteristics. Experimental results demonstrate that the proposed circuit, when implemented in a 28 nm FDSOI technology, saves 65% of silicon area with respect to competitors, dissipating less than 1 pJ. FPGA implementation results confirm a massive energy dissipation reduction with respect to the conventional baseline architecture, without introducing penalties in the Top-1 accuracy.
2022
Approximate computing; Deep neural networks; Low-power hardware architectures; SoftMax
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.11770/324908
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 8
  • ???jsp.display-item.citation.isi??? 6
social impact