Mobile deployment of stable diffusion models

• Main Author: Tran, Que An
• Other Authors: Dusit Niyato
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science
• Online Access: https://hdl.handle.net/10356/181081
• Institution: Nanyang Technological University

The rapid development of Generative Artificial Intelligence (Gen AI) in recent years has posed significant challenges in terms of both cost and power consumption since training and inference processes require intensive GPU computation. Moreover, the use of GPUs in cloud-based platforms for Gen AI inference raises privacy concerns such as the potential leakage of user prompts to external service providers, which might lead to legal complications afterwards. This project investigates the potential use of Gen AI by optimizing and benchmarking the performance of the Stable Diffusion model on Android devices. By using the NCNN framework, the project enables Text-to-Image and Image-to-Image generation on mobile platforms without relying on cloud-based resources, addressing concerns such as cost, power consumption, and privacy. The project aims to develop an Android application that is capable of efficiently running the Stable Diffusion model. The implementation focuses on minimizing computational load and enhancing inference speed. By reusing prompts with text embedding models, the project refines the Distributed Diffusion optimization technique, contributing significantly to the integration of Stable Diffusion on mobile devices. The project also explores using cloud-based infrastructure for multi-model text-to-video generation. These efforts showcase Stable Diffusion's capabilities and suggest future possibilities for AI applications on mobile devices. The detailed implementation for this project is available at GitHub.