Morphologically-aware vocabulary reduction of word embeddings

Morphologically-aware vocabulary reduction of word embeddings

We propose SubText, a compression mechanism via vocabulary reduction. The crux is to judiciously select a subset of word embeddings which support the reconstruction of the remaining word embeddings based on their form alone. The proposed algorithm considers the preservation of the original embedding...

Full description

Saved in:
Bibliographic Details
Main Authors: CHIA, Chong Cher, TKACHENKO, Maksim, LAUW, Hady Wirawan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2023
Subjects:
Word embeddings
compression
vocabulary reduction
Databases and Information Systems
Numerical Analysis and Scientific Computing
Theory and Algorithms
Online Access:https://ink.library.smu.edu.sg/sis_research/7608
https://ink.library.smu.edu.sg/context/sis_research/article/8611/viewcontent/wiiat22.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
Description
Summary:We propose SubText, a compression mechanism via vocabulary reduction. The crux is to judiciously select a subset of word embeddings which support the reconstruction of the remaining word embeddings based on their form alone. The proposed algorithm considers the preservation of the original embeddings, as well as a word’s relationship to other words that are morphologically or semantically similar. Comprehensive evaluation of the compressed vocabulary reveals SubText’s efficacy on diverse tasks over traditional vocabulary reduction techniques, as validated on English, as well as a collection of inflected languages.

Similar Items