A data engineering solution for micrometeorology forecasting using cloud recognition

Micrometeorology forecasting stands at the forefront of accurate localized weather predictions, providing invaluable insights for individuals, communities and industries alike. Traditional weather forecast systems predominantly operate on mesoscale and synoptic meteorology, providing generalized...

Full description

Saved in:
Bibliographic Details
Main Author: Chai, Ai Jia
Format: Final Year Project / Dissertation / Thesis
Published: 2024
Subjects:
Online Access:http://eprints.utar.edu.my/6629/1/fyp_CS_2024_CAJ.pdf
http://eprints.utar.edu.my/6629/
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Universiti Tunku Abdul Rahman
id my-utar-eprints.6629
record_format eprints
spelling my-utar-eprints.66292024-10-23T05:47:26Z A data engineering solution for micrometeorology forecasting using cloud recognition Chai, Ai Jia T Technology (General) Micrometeorology forecasting stands at the forefront of accurate localized weather predictions, providing invaluable insights for individuals, communities and industries alike. Traditional weather forecast systems predominantly operate on mesoscale and synoptic meteorology, providing generalized information for large geographic regions. However, the micrometeorological conditions within smaller areas are often overlooked, leading to inaccuracies in local weather predictions. Apart from that, most of the existing methods rely on costly data collection and complex modelling. Thus, this has limited their accessibility and usability. Therefore, this project aims to address these pivotal challenges in weather forecasting. This project falls within the domain of micrometeorology and weather forecasting, focusing on developing a data engineering solution for the micrometeorology forecasting system and the recognition of cloud patterns to predict localized rainfall events. The proposed system employs a multi-faceted approach. It begins with the automatic collection of cloud images by capturing it with a camera module. After capturing, the images are automatically uploaded to AWS cloud storage platform. By leveraging machine learning techniques, the images will undergo a labelling process to categorize them as either rainy or not rainy. Manual labelling method will then be used to validate the accuracy of the previous annotation to create a ground truth dataset. Next, classification method will be utilized to classify the dataset into different classes. This approach empowers the system to label the new images automatically in future. Lastly, a predictive model will be trained to recognize the cloud formation patterns and predict the probability of the occurrence of the rainfall events. The novelties of this project encompass several groundbreaking aspects including the automated cloud recognition, localized and realtime predictions and the integration of data engineering methodologies to ensure the adaptability of the system to various geographical areas by facilitating the efficient collection, storage and labelling of cloud images. This streamlined process enables the generation of labeled datasets essential for training accurate predictive models, thus enhancing the system’s effectiveness across diverse environments. Moreover, this project also aims to offer a costeffective solution by using only a camera module to predict the rainfall events, making weather predictions more accessible to a broader audience. 2024-01 Final Year Project / Dissertation / Thesis NonPeerReviewed application/pdf http://eprints.utar.edu.my/6629/1/fyp_CS_2024_CAJ.pdf Chai, Ai Jia (2024) A data engineering solution for micrometeorology forecasting using cloud recognition. Final Year Project, UTAR. http://eprints.utar.edu.my/6629/
institution Universiti Tunku Abdul Rahman
building UTAR Library
collection Institutional Repository
continent Asia
country Malaysia
content_provider Universiti Tunku Abdul Rahman
content_source UTAR Institutional Repository
url_provider http://eprints.utar.edu.my
topic T Technology (General)
spellingShingle T Technology (General)
Chai, Ai Jia
A data engineering solution for micrometeorology forecasting using cloud recognition
description Micrometeorology forecasting stands at the forefront of accurate localized weather predictions, providing invaluable insights for individuals, communities and industries alike. Traditional weather forecast systems predominantly operate on mesoscale and synoptic meteorology, providing generalized information for large geographic regions. However, the micrometeorological conditions within smaller areas are often overlooked, leading to inaccuracies in local weather predictions. Apart from that, most of the existing methods rely on costly data collection and complex modelling. Thus, this has limited their accessibility and usability. Therefore, this project aims to address these pivotal challenges in weather forecasting. This project falls within the domain of micrometeorology and weather forecasting, focusing on developing a data engineering solution for the micrometeorology forecasting system and the recognition of cloud patterns to predict localized rainfall events. The proposed system employs a multi-faceted approach. It begins with the automatic collection of cloud images by capturing it with a camera module. After capturing, the images are automatically uploaded to AWS cloud storage platform. By leveraging machine learning techniques, the images will undergo a labelling process to categorize them as either rainy or not rainy. Manual labelling method will then be used to validate the accuracy of the previous annotation to create a ground truth dataset. Next, classification method will be utilized to classify the dataset into different classes. This approach empowers the system to label the new images automatically in future. Lastly, a predictive model will be trained to recognize the cloud formation patterns and predict the probability of the occurrence of the rainfall events. The novelties of this project encompass several groundbreaking aspects including the automated cloud recognition, localized and realtime predictions and the integration of data engineering methodologies to ensure the adaptability of the system to various geographical areas by facilitating the efficient collection, storage and labelling of cloud images. This streamlined process enables the generation of labeled datasets essential for training accurate predictive models, thus enhancing the system’s effectiveness across diverse environments. Moreover, this project also aims to offer a costeffective solution by using only a camera module to predict the rainfall events, making weather predictions more accessible to a broader audience.
format Final Year Project / Dissertation / Thesis
author Chai, Ai Jia
author_facet Chai, Ai Jia
author_sort Chai, Ai Jia
title A data engineering solution for micrometeorology forecasting using cloud recognition
title_short A data engineering solution for micrometeorology forecasting using cloud recognition
title_full A data engineering solution for micrometeorology forecasting using cloud recognition
title_fullStr A data engineering solution for micrometeorology forecasting using cloud recognition
title_full_unstemmed A data engineering solution for micrometeorology forecasting using cloud recognition
title_sort data engineering solution for micrometeorology forecasting using cloud recognition
publishDate 2024
url http://eprints.utar.edu.my/6629/1/fyp_CS_2024_CAJ.pdf
http://eprints.utar.edu.my/6629/
_version_ 1814061974360686592