Mobile big data analytics using deep learning and Apache Spark

The proliferation of mobile devices, such as smartphones and Internet of Things (IoT) gadgets, results in the recent mobile big data (MBD) era. Collecting MBD is unprofitable unless suitable analytics and learning methods are utilized for extracting meaningful information and hidden patterns from da...

Full description

Saved in:
Bibliographic Details
Main Authors: Niyato, Dusit, Abu Alsheikh, Mohammad, Lin, Shaowei, Tan, Hwee-Pink, Han, Zhu
Other Authors: School of Computer Engineering
Format: Article
Language:English
Published: 2016
Subjects:
Online Access:https://hdl.handle.net/10356/84039
http://hdl.handle.net/10220/41585
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-84039
record_format dspace
spelling sg-ntu-dr.10356-840392020-05-28T07:18:12Z Mobile big data analytics using deep learning and Apache Spark Niyato, Dusit Abu Alsheikh, Mohammad Lin, Shaowei Tan, Hwee-Pink Han, Zhu School of Computer Engineering Distributed deep learning Big data The proliferation of mobile devices, such as smartphones and Internet of Things (IoT) gadgets, results in the recent mobile big data (MBD) era. Collecting MBD is unprofitable unless suitable analytics and learning methods are utilized for extracting meaningful information and hidden patterns from data. This article presents an overview and brief tutorial of deep learning in MBD analytics and discusses a scalable learning framework over Apache Spark. Specifically, the learning of deep models is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall MBD, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness. ASTAR (Agency for Sci., Tech. and Research, S’pore) Accepted version 2016-10-26T09:29:59Z 2019-12-06T15:37:01Z 2016-10-26T09:29:59Z 2019-12-06T15:37:01Z 2016 Journal Article Abu Alsheikh, M., Niyato, D., Lin, S., Tan, H.-P., & Han, Z. (2016). Mobile big data analytics using deep learning and Apache Spark. IEEE Network, 30(3), 22-29. 0890-8044 https://hdl.handle.net/10356/84039 http://hdl.handle.net/10220/41585 10.1109/MNET.2016.7474340 en IEEE Network © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: [http://dx.doi.org/10.1109/MNET.2016.7474340]. 9 p. application/pdf
institution Nanyang Technological University
building NTU Library
country Singapore
collection DR-NTU
language English
topic Distributed deep learning
Big data
spellingShingle Distributed deep learning
Big data
Niyato, Dusit
Abu Alsheikh, Mohammad
Lin, Shaowei
Tan, Hwee-Pink
Han, Zhu
Mobile big data analytics using deep learning and Apache Spark
description The proliferation of mobile devices, such as smartphones and Internet of Things (IoT) gadgets, results in the recent mobile big data (MBD) era. Collecting MBD is unprofitable unless suitable analytics and learning methods are utilized for extracting meaningful information and hidden patterns from data. This article presents an overview and brief tutorial of deep learning in MBD analytics and discusses a scalable learning framework over Apache Spark. Specifically, the learning of deep models is executed as an iterative MapReduce computing on many Spark workers. Each Spark worker learns a partial deep model on a partition of the overall MBD, and a master deep model is then built by averaging the parameters of all partial models. This Spark-based framework speeds up the learning of deep models consisting of many hidden layers and millions of parameters. We use a context-aware activity recognition application with a real-world dataset containing millions of samples to validate our framework and assess its speedup effectiveness.
author2 School of Computer Engineering
author_facet School of Computer Engineering
Niyato, Dusit
Abu Alsheikh, Mohammad
Lin, Shaowei
Tan, Hwee-Pink
Han, Zhu
format Article
author Niyato, Dusit
Abu Alsheikh, Mohammad
Lin, Shaowei
Tan, Hwee-Pink
Han, Zhu
author_sort Niyato, Dusit
title Mobile big data analytics using deep learning and Apache Spark
title_short Mobile big data analytics using deep learning and Apache Spark
title_full Mobile big data analytics using deep learning and Apache Spark
title_fullStr Mobile big data analytics using deep learning and Apache Spark
title_full_unstemmed Mobile big data analytics using deep learning and Apache Spark
title_sort mobile big data analytics using deep learning and apache spark
publishDate 2016
url https://hdl.handle.net/10356/84039
http://hdl.handle.net/10220/41585
_version_ 1681058835199950848