Weather data analysis using hadoop : Applications and challenges

• Main Authors: Mohammed Adam Ibrahim, Fakharaldien, Khalid, Adam, Noor Akma, Abu Bakar, Mazlina, Abdul Majid
• Format: Conference or Workshop Item
• Language: English
• Published: IOP Publishing 2019
• Subjects: QA76 Computer software; QC Physics; TA Engineering (General). Civil engineering (General)
• Online Access: http://umpir.ump.edu.my/id/eprint/26182/1/Weather%20data%20analysis%20using%20hadoop.pdf; http://umpir.ump.edu.my/id/eprint/26182/; https://doi.org/10.1088/1757-899X/551/1/012044
• Institution: Universiti Malaysia Pahang

Weather data is very crucial in every aspect of human daily life. It plays an important role in many sectors such as agriculture, tourism, government planning, industry and so on. Weather has a variety of parameters like temperature, pressure, humidity and wind speed. The meteorological department deployed sensors for each weather parameter at different geographical locations to collect data. This data is stored mostly in the unstructured format. Thus, a big amount of data has been collected and archived. Therefore, storage and processing of this big data for accurate weather prediction is a huge challenge. Hadoop an apache product it used to support big data sets in a distributed environment. Hadoop has greatest advantages over scalable and fault-tolerant distributed processing technologies. This paper explains a system that uses the historical weather data of a region and apply the MapReduce and Hadoop techniques to analysis these historical data.