AutoFocus: Interpreting attention-based neural networks by code perturbation

Despite being adopted in software engineering tasks, deep neural networks are treated mostly as a black box due to the difficulty in interpreting how the networks infer the outputs from the inputs. To address this problem, we propose AutoFocus, an automated approach for rating and visualizing the im...

Full description

Saved in:
Bibliographic Details
Main Authors: BUI, Duy Quoc Nghi, YU, Yijun, JIANG, Lingxiao
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/4817
https://ink.library.smu.edu.sg/context/sis_research/article/5820/viewcontent/ase19autofocus.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-5820
record_format dspace
spelling sg-smu-ink.sis_research-58202024-05-31T08:17:50Z AutoFocus: Interpreting attention-based neural networks by code perturbation BUI, Duy Quoc Nghi YU, Yijun JIANG, Lingxiao Despite being adopted in software engineering tasks, deep neural networks are treated mostly as a black box due to the difficulty in interpreting how the networks infer the outputs from the inputs. To address this problem, we propose AutoFocus, an automated approach for rating and visualizing the importance of input elements based on their effects on the outputs of the networks. The approach is built on our hypotheses that (1) attention mechanisms incorporated into neural networks can generate discriminative scores for various input elements and (2) the discriminative scores reflect the effects of input elements on the outputs of the networks. This paper verifies the hypotheses by applying AutoFocus on the task of algorithm classification (i.e., given a program source code as input, determine the algorithm implemented by the program). AutoFocus identifies and perturbs code elements in a program systematically, and quantifies the effects of the perturbed elements on the network’s classification results. Based on evaluation on more than 1000 programs for 10 different sorting algorithms, we observe that the attention scores are highly correlated to the effects of the perturbed code elements. Such a correlation provides a strong basis for the uses of attention scores to interpret the relations between code elements and the algorithm classification results of a neural network, and we believe that visualizing code elements in an input program ranked according to their attention scores can facilitate faster program comprehension with reduced code. 2019-11-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/4817 info:doi/10.1109/ASE.2019.00014 https://ink.library.smu.edu.sg/context/sis_research/article/5820/viewcontent/ase19autofocus.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University attention mechanisms neural networks algorithm classification interpretability code perturbation programcomprehension Software Engineering
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic attention mechanisms
neural networks
algorithm classification
interpretability
code perturbation
programcomprehension
Software Engineering
spellingShingle attention mechanisms
neural networks
algorithm classification
interpretability
code perturbation
programcomprehension
Software Engineering
BUI, Duy Quoc Nghi
YU, Yijun
JIANG, Lingxiao
AutoFocus: Interpreting attention-based neural networks by code perturbation
description Despite being adopted in software engineering tasks, deep neural networks are treated mostly as a black box due to the difficulty in interpreting how the networks infer the outputs from the inputs. To address this problem, we propose AutoFocus, an automated approach for rating and visualizing the importance of input elements based on their effects on the outputs of the networks. The approach is built on our hypotheses that (1) attention mechanisms incorporated into neural networks can generate discriminative scores for various input elements and (2) the discriminative scores reflect the effects of input elements on the outputs of the networks. This paper verifies the hypotheses by applying AutoFocus on the task of algorithm classification (i.e., given a program source code as input, determine the algorithm implemented by the program). AutoFocus identifies and perturbs code elements in a program systematically, and quantifies the effects of the perturbed elements on the network’s classification results. Based on evaluation on more than 1000 programs for 10 different sorting algorithms, we observe that the attention scores are highly correlated to the effects of the perturbed code elements. Such a correlation provides a strong basis for the uses of attention scores to interpret the relations between code elements and the algorithm classification results of a neural network, and we believe that visualizing code elements in an input program ranked according to their attention scores can facilitate faster program comprehension with reduced code.
format text
author BUI, Duy Quoc Nghi
YU, Yijun
JIANG, Lingxiao
author_facet BUI, Duy Quoc Nghi
YU, Yijun
JIANG, Lingxiao
author_sort BUI, Duy Quoc Nghi
title AutoFocus: Interpreting attention-based neural networks by code perturbation
title_short AutoFocus: Interpreting attention-based neural networks by code perturbation
title_full AutoFocus: Interpreting attention-based neural networks by code perturbation
title_fullStr AutoFocus: Interpreting attention-based neural networks by code perturbation
title_full_unstemmed AutoFocus: Interpreting attention-based neural networks by code perturbation
title_sort autofocus: interpreting attention-based neural networks by code perturbation
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/4817
https://ink.library.smu.edu.sg/context/sis_research/article/5820/viewcontent/ase19autofocus.pdf
_version_ 1814047561522085888