Benchmarking library recognition in tweets

• Main Authors: ZHANG, Ting, CHANDRASEKARAN, Divya Prabha, THUNG, Ferdian, LO, David
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Software libraries; Tweets; Disambiguation; Benchmark study; Artificial Intelligence and Robotics; Computer and Systems Architecture; Data Storage Systems; Information Security; Software Engineering
• Online Access: https://ink.library.smu.edu.sg/sis_research/7632; https://ink.library.smu.edu.sg/context/sis_research/article/8635/viewcontent/Benchmarking_library_recognition_in_tweets.pdf
• Institution: Singapore Management University

Software developers often use social media (such as Twitter) to shareprogramming knowledge such as new tools, sample code snippets,and tips on programming. One of the topics they talk about is thesoftware library. The tweets may contain useful information abouta library. A good understanding of this information, e.g., on thedeveloper’s views regarding a library can be beneficial to weigh thepros and cons of using the library as well as the general sentimentstowards the library. However, it is not trivial to recognize whethera word actually refers to a library or other meanings. For example,a tweet mentioning the word “pandas" may refer to the Pythonpandas library or to the animal. In this work, we created the firstbenchmark dataset and investigated the task to distinguish whethera tweet refers to a programming library or something else. Recently,the pre-trained Transformer models (PTMs) have achieved greatsuccess in the fields of natural language processing and computervision. Therefore, we extensively evaluated a broad set of modernPTMs, including both general-purpose and domain-specific ones,to solve this programming library recognition task in tweets. Experimental results show that the use of PTM can outperform thebest-performing baseline methods by 5% - 12% in terms of F1-scoreunder within-, cross-, and mixed-library settings.