AutoFocus: Interpreting attention-based neural networks by code perturbation
Despite being adopted in software engineering tasks, deep neural networks are treated mostly as a black box due to the difficulty in interpreting how the networks infer the outputs from the inputs. To address this problem, we propose AutoFocus, an automated approach for rating and visualizing the im...
Saved in:
Main Authors: | , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2019
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/4817 https://ink.library.smu.edu.sg/context/sis_research/article/5820/viewcontent/ase19autofocus.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-5820 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-58202024-05-31T08:17:50Z AutoFocus: Interpreting attention-based neural networks by code perturbation BUI, Duy Quoc Nghi YU, Yijun JIANG, Lingxiao Despite being adopted in software engineering tasks, deep neural networks are treated mostly as a black box due to the difficulty in interpreting how the networks infer the outputs from the inputs. To address this problem, we propose AutoFocus, an automated approach for rating and visualizing the importance of input elements based on their effects on the outputs of the networks. The approach is built on our hypotheses that (1) attention mechanisms incorporated into neural networks can generate discriminative scores for various input elements and (2) the discriminative scores reflect the effects of input elements on the outputs of the networks. This paper verifies the hypotheses by applying AutoFocus on the task of algorithm classification (i.e., given a program source code as input, determine the algorithm implemented by the program). AutoFocus identifies and perturbs code elements in a program systematically, and quantifies the effects of the perturbed elements on the network’s classification results. Based on evaluation on more than 1000 programs for 10 different sorting algorithms, we observe that the attention scores are highly correlated to the effects of the perturbed code elements. Such a correlation provides a strong basis for the uses of attention scores to interpret the relations between code elements and the algorithm classification results of a neural network, and we believe that visualizing code elements in an input program ranked according to their attention scores can facilitate faster program comprehension with reduced code. 2019-11-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/4817 info:doi/10.1109/ASE.2019.00014 https://ink.library.smu.edu.sg/context/sis_research/article/5820/viewcontent/ase19autofocus.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University attention mechanisms neural networks algorithm classification interpretability code perturbation programcomprehension Software Engineering |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
attention mechanisms neural networks algorithm classification interpretability code perturbation programcomprehension Software Engineering |
spellingShingle |
attention mechanisms neural networks algorithm classification interpretability code perturbation programcomprehension Software Engineering BUI, Duy Quoc Nghi YU, Yijun JIANG, Lingxiao AutoFocus: Interpreting attention-based neural networks by code perturbation |
description |
Despite being adopted in software engineering tasks, deep neural networks are treated mostly as a black box due to the difficulty in interpreting how the networks infer the outputs from the inputs. To address this problem, we propose AutoFocus, an automated approach for rating and visualizing the importance of input elements based on their effects on the outputs of the networks. The approach is built on our hypotheses that (1) attention mechanisms incorporated into neural networks can generate discriminative scores for various input elements and (2) the discriminative scores reflect the effects of input elements on the outputs of the networks. This paper verifies the hypotheses by applying AutoFocus on the task of algorithm classification (i.e., given a program source code as input, determine the algorithm implemented by the program). AutoFocus identifies and perturbs code elements in a program systematically, and quantifies the effects of the perturbed elements on the network’s classification results. Based on evaluation on more than 1000 programs for 10 different sorting algorithms, we observe that the attention scores are highly correlated to the effects of the perturbed code elements. Such a correlation provides a strong basis for the uses of attention scores to interpret the relations between code elements and the algorithm classification results of a neural network, and we believe that visualizing code elements in an input program ranked according to their attention scores can facilitate faster program comprehension with reduced code. |
format |
text |
author |
BUI, Duy Quoc Nghi YU, Yijun JIANG, Lingxiao |
author_facet |
BUI, Duy Quoc Nghi YU, Yijun JIANG, Lingxiao |
author_sort |
BUI, Duy Quoc Nghi |
title |
AutoFocus: Interpreting attention-based neural networks by code perturbation |
title_short |
AutoFocus: Interpreting attention-based neural networks by code perturbation |
title_full |
AutoFocus: Interpreting attention-based neural networks by code perturbation |
title_fullStr |
AutoFocus: Interpreting attention-based neural networks by code perturbation |
title_full_unstemmed |
AutoFocus: Interpreting attention-based neural networks by code perturbation |
title_sort |
autofocus: interpreting attention-based neural networks by code perturbation |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2019 |
url |
https://ink.library.smu.edu.sg/sis_research/4817 https://ink.library.smu.edu.sg/context/sis_research/article/5820/viewcontent/ase19autofocus.pdf |
_version_ |
1814047561522085888 |