AquaLens: advancing monitoring in aquaculture using autodistill and CLIP

• Main Author: Ng, Siew Juan
• Other Authors: Ng Yin Kwee
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Engineering; Machine vision; Aquaculture
• Online Access: https://hdl.handle.net/10356/177865
• Institution: Nanyang Technological University

Deformities and disease occurrence within aquaculture are the by-products of poor practices and inadequate monitoring. Such procedures are performed manually which is an exhaustive process that can cause harm to the fishes. A non-intrusive and robust system that can adaptively monitor and track every activity of the fishes will ease the traditional laborious practices and potentially increase aquaculture’s yield production overall. This project aims to develop an effective and robust monitoring system called ‘AquaLens’. It will precisely monitor all the fishes in the tank without human intervention. With the video captured from the tank, the detection algorithm passes the initial value of the target into the tracking algorithm. The fish’s features are then extracted for image classification and feature comparison to perform continuous identification. During final processing, all the pre-processed data are combined to generate the output of the overall monitoring system. The AquaLens system is developed using a detection algorithm, Autodistill, to create a custom model with a confidence score of 83% and a precision score of 97.1%. As for the multi-target tracker algorithm, BotSORT achieves a Muti-Object Tracking Accuracy (MOTA) of 77.7%. The re-identification process was modified to the image classification algorithm and the feature extraction model: ResNet50 and OpenAI CLIP, to achieve an accuracy score of 97.2% and 79.4% respectively. Overall, AquaLens performs with an average precision of 81.12%. AquaLens aims to provide farmers and stakeholders with the ability to continuously detect, track and re-identify every fish’s physical appearance and behavioural activities in the tank without human input and intervention. The long-term usage of the system will address the deteriorating health of the fish by providing comprehensive visual documentation of their activities, which can potentially replace existing manual monitoring methods.