An empirical study of memorization in NLP

• Main Authors: ZHENG, Xiaosen, JIANG, Jing
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Natural language processing; Computer Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7705; https://ink.library.smu.edu.sg/context/sis_research/article/8708/viewcontent/2022.acl_long.434.pdf
• Institution: Singapore Management University

A recent study by Feldman (2020) proposed a long-tail theory to explain the memorization behavior of deep learning models. However, memorization has not been empirically verified in the context of NLP, a gap addressed by this work. In this paper, we use three different NLP tasks to check if the long...