Injecting descriptive meta-information into pre-trained language models with hypernetworks

Injecting descriptive meta-information into pre-trained language models with hypernetworks

Pre-trained language models have been widely adopted as backbones in various natural language processing tasks. However, existing pre-trained language models ignore the descriptive meta-information in the text such as the distinction between the title and the mainbody, leading to over-weighted atten...

Full description

Saved in:
Bibliographic Details
Main Authors: DUAN, Wenying, HE, Xiaoxi, ZHOU, Zimu, RAO, Hong, THIELE, Lothar
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2021
Subjects:
descriptive meta-information
hypernetworks
pre-trained language mode
Programming Languages and Compilers
Software Engineering
Online Access:https://ink.library.smu.edu.sg/sis_research/6237
https://ink.library.smu.edu.sg/context/sis_research/article/7240/viewcontent/Injecting_Descriptive_Meta_Information_into_Pre_Trained_Language_Models_with_Hypernetworks.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English

Internet

https://ink.library.smu.edu.sg/sis_research/6237
https://ink.library.smu.edu.sg/context/sis_research/article/7240/viewcontent/Injecting_Descriptive_Meta_Information_into_Pre_Trained_Language_Models_with_Hypernetworks.pdf

Similar Items