A black-box attack on code models via representation nearest Neighbor search
Existing methods for generating adversarial code examples face several challenges: limted availability of substitute variables, high verification costs for these substitutes, and the creation of adversarial samples with noticeable perturbations. To address these concerns, our proposed approach, RNNS...
Saved in:
Main Authors: | , , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2023
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/8588 https://ink.library.smu.edu.sg/context/sis_research/article/9591/viewcontent/black_box.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-9591 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-95912024-01-25T08:52:34Z A black-box attack on code models via representation nearest Neighbor search ZHANG, Jie MA, Wei HU, Qiang Liu, Shangqing XIE, Xiaofei LE Traon, Yves LIU, Yang Existing methods for generating adversarial code examples face several challenges: limted availability of substitute variables, high verification costs for these substitutes, and the creation of adversarial samples with noticeable perturbations. To address these concerns, our proposed approach, RNNS, uses a search seed based on historical attacks to find potential adversarial substitutes. Rather than directly using the discrete substitutes, they are mapped to a continuous vector space using a pre-trained variable name encoder. Based on the vector representation, RNNS predicts and selects better substitutes for attacks. We evaluated the performance of RNNS across six coding tasks encompassing three programming languages: Java, Python, and C. We employed three pre-trained code models (CodeBERT, GraphCodeBERT, and CodeT5) that resulted in a cumulative of 18 victim models. The results demonstrate that RNNS outperforms baselines in terms of ASR and QT. Furthermore, the perturbation of adversarial examples introduced by RNNS is smaller compared to the baselines in terms of the number of replaced variables and the change in variable length. Lastly, our experiments indicate that RNNS is efficient in attacking defended models and can be employed for adversarial training. 2023-12-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/8588 info:doi/10.18653/v1/2023.findings-emnlp.649 https://ink.library.smu.edu.sg/context/sis_research/article/9591/viewcontent/black_box.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Databases and Information Systems Programming Languages and Compilers |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Databases and Information Systems Programming Languages and Compilers |
spellingShingle |
Databases and Information Systems Programming Languages and Compilers ZHANG, Jie MA, Wei HU, Qiang Liu, Shangqing XIE, Xiaofei LE Traon, Yves LIU, Yang A black-box attack on code models via representation nearest Neighbor search |
description |
Existing methods for generating adversarial code examples face several challenges: limted availability of substitute variables, high verification costs for these substitutes, and the creation of adversarial samples with noticeable perturbations. To address these concerns, our proposed approach, RNNS, uses a search seed based on historical attacks to find potential adversarial substitutes. Rather than directly using the discrete substitutes, they are mapped to a continuous vector space using a pre-trained variable name encoder. Based on the vector representation, RNNS predicts and selects better substitutes for attacks. We evaluated the performance of RNNS across six coding tasks encompassing three programming languages: Java, Python, and C. We employed three pre-trained code models (CodeBERT, GraphCodeBERT, and CodeT5) that resulted in a cumulative of 18 victim models. The results demonstrate that RNNS outperforms baselines in terms of ASR and QT. Furthermore, the perturbation of adversarial examples introduced by RNNS is smaller compared to the baselines in terms of the number of replaced variables and the change in variable length. Lastly, our experiments indicate that RNNS is efficient in attacking defended models and can be employed for adversarial training. |
format |
text |
author |
ZHANG, Jie MA, Wei HU, Qiang Liu, Shangqing XIE, Xiaofei LE Traon, Yves LIU, Yang |
author_facet |
ZHANG, Jie MA, Wei HU, Qiang Liu, Shangqing XIE, Xiaofei LE Traon, Yves LIU, Yang |
author_sort |
ZHANG, Jie |
title |
A black-box attack on code models via representation nearest Neighbor search |
title_short |
A black-box attack on code models via representation nearest Neighbor search |
title_full |
A black-box attack on code models via representation nearest Neighbor search |
title_fullStr |
A black-box attack on code models via representation nearest Neighbor search |
title_full_unstemmed |
A black-box attack on code models via representation nearest Neighbor search |
title_sort |
black-box attack on code models via representation nearest neighbor search |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2023 |
url |
https://ink.library.smu.edu.sg/sis_research/8588 https://ink.library.smu.edu.sg/context/sis_research/article/9591/viewcontent/black_box.pdf |
_version_ |
1789483281247371264 |