Using Hadoop and Cassandra for Taxi Data Analytics: A Feasibility Study

• Main Authors: KOH, Alvin Jun Yong, NGUYEN, Xuan Khoa, WOODARD, C. Jason
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2010
• Subjects: taxi fleet management; GPS data; Numerical Analysis and Scientific Computing; Operations and Supply Chain Management
• Online Access: https://ink.library.smu.edu.sg/sis_research_smu/15; https://ink.library.smu.edu.sg/cgi/viewcontent.cgi?article=1014&context=sis_research_smu
• Institution: Singapore Management University

This paper reports on a preliminary study to assess the feasibility of using the Open Cirrus Cloud Computing Research testbed to provide offline and online analytical support for taxi fleet operations. In the study, we benchmarked the performance gains from distributing the offline analysis of GPS location traces over multiple virtual machines using the Apache Hadoop implementation of the MapReduce paradigm. We also explored the use of the Apache Cassandra distributed database system for online retrieval of vehicle trace data. While configuring the testbed infrastructure was straightforward, we encountered severe I/O bottlenecks in running the benchmarks due to the lack of local disk storage on the compute nodes. This design limitation severely impedes the analysis of large data sets using cloud computing technologies.