Material recognition using machine learning

• Main Author: Zhang, Haosong
• Other Authors: Lap-Pui Chau
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/140257
• Institution: Nanyang Technological University

Material recognition is a useful technique which identifies materials by exploiting their detailed surface information, whereas traditional object recognition methods fail to lay emphasis on. Material recognition has been widely used in real-life applications such as autonomous agents, clothing industries and interior design. Equipment used to collect images of various materials can be classified into lab-based devices like light field cameras or robots [1], and non-lab-based devices like mobile phone cameras. Utilising convolutional neural network (CNN) as a solid extractor of features, existing methods have been successful in decoding texture cues and reflectance from taking different views of material surfaces using lab-based equipment, this is also known as differential angular images [2]. Stereos cameras such as cameras on some mobile phones are dual-lens systems, also capture differential angular images that expose a material’s latent characteristic. In addition, depth maps can be calculated from a pair of stereo images. Inspired by the ever-increasing mobilisation of stereo cameras, we look into the capability of designing algorithms that involve differential angular information or geometric information on a common dual-camera or multi-camera mobile phone. With a data collection tool outputting a pair of stereo images from stereo cameras, we collected a mini-dataset for training and testing [3]. Furthermore, to deal with the complex outdoor condition in non-lab scenes, and differentiate highly similar textures or patterns from different materials, we propose a novel method that is able to extract depth information from stereo images. The depth information serves to distinguish materials that have very similar appearance textures but different depth maps such as “grass” versus “a computer screen that displays grass”, or “wood” versus “wallpaper with printed wood”. With the depth information serving as a feature branch, the machine learning model will be able to provide a more robust material recognition capability.