Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks

Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, w...

Full description

Saved in:
Bibliographic Details
Main Authors: Lauren, Paula, Qu, Guangzhi, Yang, Jucheng, Watta, Paul, Huang, Guang-Bin, Lendasse, Amaury
Other Authors: School of Electrical and Electronic Engineering
Format: Article
Language:English
Published: 2020
Subjects:
Online Access:https://hdl.handle.net/10356/141680
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-141680
record_format dspace
spelling sg-ntu-dr.10356-1416802020-06-10T02:46:10Z Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks Lauren, Paula Qu, Guangzhi Yang, Jucheng Watta, Paul Huang, Guang-Bin Lendasse, Amaury School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Word Embeddings Extreme Learning Machine (ELM) Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, we proposed using an Extreme Learning Machine (ELM) for generating word embeddings. In this research, we apply the ELM-based Word Embeddings to the NLP task of Text Categorization, specifically Sentiment Analysis and Sequence Labeling. The ELM-based Word Embeddings utilizes a count-based approach similar to the Global Vectors (GloVe) model, where the word-context matrix is computed then matrix factorization is applied. A comparative study is done with Word2Vec and GloVe, which are the two popular state-of-the-art models. The results show that ELM-based Word Embeddings slightly outperforms the aforementioned two methods in the Sentiment Analysis and Sequence Labeling tasks.In addition, only one hyperparameter is needed using ELM whereas several are utilized for the other methods. ELM-based Word Embeddings are comparable to the state-of-the-art methods: Word2Vec and GloVe models. In addition, the count-based ELM model have word similarities to both the count-based GloVe and the predict-based Word2Vec models, with subtle differences. 2020-06-10T02:46:10Z 2020-06-10T02:46:10Z 2018 Journal Article Lauren, P., Qu, G., Yang, J., Watta, P., Huang, G.-B., & Lendasse, A. (2018). Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks. Cognitive Computation, 10(4), 625-638. doi:10.1007/s12559-018-9548-y 1866-9956 https://hdl.handle.net/10356/141680 10.1007/s12559-018-9548-y 2-s2.0-85045130357 4 10 625 638 en Cognitive Computation © 2018 Springer Science+Business Media, LLC, part of Springer Nature. All rights reserved.
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering
Word Embeddings
Extreme Learning Machine (ELM)
spellingShingle Engineering::Electrical and electronic engineering
Word Embeddings
Extreme Learning Machine (ELM)
Lauren, Paula
Qu, Guangzhi
Yang, Jucheng
Watta, Paul
Huang, Guang-Bin
Lendasse, Amaury
Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
description Word Embeddings are low-dimensional distributed representations that encompass a set of language modeling and feature learning techniques from Natural Language Processing (NLP). Words or phrases from the vocabulary are mapped to vectors of real numbers in a low-dimensional space. In previous work, we proposed using an Extreme Learning Machine (ELM) for generating word embeddings. In this research, we apply the ELM-based Word Embeddings to the NLP task of Text Categorization, specifically Sentiment Analysis and Sequence Labeling. The ELM-based Word Embeddings utilizes a count-based approach similar to the Global Vectors (GloVe) model, where the word-context matrix is computed then matrix factorization is applied. A comparative study is done with Word2Vec and GloVe, which are the two popular state-of-the-art models. The results show that ELM-based Word Embeddings slightly outperforms the aforementioned two methods in the Sentiment Analysis and Sequence Labeling tasks.In addition, only one hyperparameter is needed using ELM whereas several are utilized for the other methods. ELM-based Word Embeddings are comparable to the state-of-the-art methods: Word2Vec and GloVe models. In addition, the count-based ELM model have word similarities to both the count-based GloVe and the predict-based Word2Vec models, with subtle differences.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Lauren, Paula
Qu, Guangzhi
Yang, Jucheng
Watta, Paul
Huang, Guang-Bin
Lendasse, Amaury
format Article
author Lauren, Paula
Qu, Guangzhi
Yang, Jucheng
Watta, Paul
Huang, Guang-Bin
Lendasse, Amaury
author_sort Lauren, Paula
title Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
title_short Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
title_full Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
title_fullStr Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
title_full_unstemmed Generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
title_sort generating word embeddings from an extreme learning machine for sentiment analysis and sequence labeling tasks
publishDate 2020
url https://hdl.handle.net/10356/141680
_version_ 1681058898182668288