DeepMutation++: A mutation testing framework for deep learning systems

Deep neural networks (DNNs) are increasingly expanding their real-world applications across domains, e.g., image processing, speech recognition and natural language processing. However, there is still limited tool support for DNN testing in terms of test data quality and model robustness. In this pa...

Full description

Saved in:
Bibliographic Details
Main Authors: HU, Qiang, MA, Lei, XIE, Xiaofei, YU, Bing, LIU, Yang, ZHAO, Jianjun
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2019
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7071
https://ink.library.smu.edu.sg/context/sis_research/article/8074/viewcontent/ASE.2019.00126.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8074
record_format dspace
spelling sg-smu-ink.sis_research-80742022-04-07T08:15:34Z DeepMutation++: A mutation testing framework for deep learning systems HU, Qiang MA, Lei XIE, Xiaofei YU, Bing LIU, Yang ZHAO, Jianjun Deep neural networks (DNNs) are increasingly expanding their real-world applications across domains, e.g., image processing, speech recognition and natural language processing. However, there is still limited tool support for DNN testing in terms of test data quality and model robustness. In this paper, we introduce a mutation testing-based tool for DNNs, DeepMutation++, which facilitates the DNN quality evaluation, supporting both feed-forward neural networks (FNNs) and stateful recurrent neural networks (RNNs). It not only enables static analysis of the robustness of a DNN model against the input as a whole, but also allows the identification of the vulnerable segments of a sequential input (e.g. audio input) by runtime analysis. It is worth noting that DeepMutation++ specially features the support of RNNs mutation testing. The tool demo video can be found on the project website https://sites.google.com/view/deepmutationpp. 2019-11-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7071 info:doi/10.1109/ASE.2019.00126 https://ink.library.smu.edu.sg/context/sis_research/article/8074/viewcontent/ASE.2019.00126.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University OS and Networks Software Engineering
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic OS and Networks
Software Engineering
spellingShingle OS and Networks
Software Engineering
HU, Qiang
MA, Lei
XIE, Xiaofei
YU, Bing
LIU, Yang
ZHAO, Jianjun
DeepMutation++: A mutation testing framework for deep learning systems
description Deep neural networks (DNNs) are increasingly expanding their real-world applications across domains, e.g., image processing, speech recognition and natural language processing. However, there is still limited tool support for DNN testing in terms of test data quality and model robustness. In this paper, we introduce a mutation testing-based tool for DNNs, DeepMutation++, which facilitates the DNN quality evaluation, supporting both feed-forward neural networks (FNNs) and stateful recurrent neural networks (RNNs). It not only enables static analysis of the robustness of a DNN model against the input as a whole, but also allows the identification of the vulnerable segments of a sequential input (e.g. audio input) by runtime analysis. It is worth noting that DeepMutation++ specially features the support of RNNs mutation testing. The tool demo video can be found on the project website https://sites.google.com/view/deepmutationpp.
format text
author HU, Qiang
MA, Lei
XIE, Xiaofei
YU, Bing
LIU, Yang
ZHAO, Jianjun
author_facet HU, Qiang
MA, Lei
XIE, Xiaofei
YU, Bing
LIU, Yang
ZHAO, Jianjun
author_sort HU, Qiang
title DeepMutation++: A mutation testing framework for deep learning systems
title_short DeepMutation++: A mutation testing framework for deep learning systems
title_full DeepMutation++: A mutation testing framework for deep learning systems
title_fullStr DeepMutation++: A mutation testing framework for deep learning systems
title_full_unstemmed DeepMutation++: A mutation testing framework for deep learning systems
title_sort deepmutation++: a mutation testing framework for deep learning systems
publisher Institutional Knowledge at Singapore Management University
publishDate 2019
url https://ink.library.smu.edu.sg/sis_research/7071
https://ink.library.smu.edu.sg/context/sis_research/article/8074/viewcontent/ASE.2019.00126.pdf
_version_ 1770576198832750592