Tattoos in forensics : retrieval, detection and synthesis

• Main Author: Xu, Xingpeng
• Other Authors: Kong Wai-Kin Adams
• Format: Thesis-Doctor of Philosophy
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Image processing and computer vision
• Online Access: https://hdl.handle.net/10356/151535
• Institution: Nanyang Technological University

A wide variety of biometric systems have been developed for forensic investigations based on biometric traits like fingerprints, faces, and iris, etc. Using those biometric traits as evidence in forensic investigations can be very challenging because they are not always visible. In the cases that those biometric traits are not available, tattoos provide us another opportunity to identify the criminals or the victims during the forensic investigation. Tattoos have been widely used by law enforcement agencies for forensic investigation, such as criminal and victim identification. Tattoo retrieval and tattoo detection are two important use cases of tattoos in forensic investigation. Many works have been done in these areas. However, there are still some unsolved problems that limit the performance of these works. For example, most tattoo retrieval systems require a query tattoo image which is not always available; tattoo detectors suffer from the lack of a large-scale tattoo dataset and their performance is limited by the architectures of the detection networks. In this research, we proposed some solutions to those problems. First, we developed a novel tattoo retrieval system that uses geometry information of tattoos to perform tattoo searching. This geometry-based tattoo retrieval (GBTR) system only requires a simple tattoo sketch as a query and avoids the requirement of a query tattoo image or a tattoo sketch with rich details. The location and shape information of the tattoo sketch is extracted first. Then, a location mapping procedure based on full-body coordinates finds all the images in the searching database that have tattoos in the corresponding location. The tattoo shapes are extracted via the Snake algorithm. Coherent Point Drift (CPD) algorithm is then employed to match the shape of the tattoo sketch and the shapes of tattoos in the searching database. The retrieved tattoos are sorted based on the matching scores. To evaluate the GBTR system, we established a full-body human image database. All the images in the database contain at least one tattoo. The evaluation results showed that the GBTR system achieves a promising tattoo retrieval accuracy. Second, we designed a prior knowledge-based attention mechanism (PKAM) to improve the performance of text tattoo detection networks. We concentrate on text tattoo detection because text tattoos provide vital clues in forensic investigations such as names and important dates. A text tattoo detection network with double PKAMs (TTDN-DA) was proposed. The first PKAM uses the features of human instances to enhance the generic tattoo images, and the second PKAM uses the features of general tattoos to enhance the text tattoo images. Two variants of TTDN-DA, TTDN-DA-V2, and TTDN-DA-V3, were also proposed for handling different training and deploying scenarios. To train and evaluate the proposed text tattoo detection networks, we established NTU Tattoo V2 dataset and NTU Text Tattoo V1 dataset. The evaluation results show that PKAM can improve text tattoo detection accuracy significantly. A large tattoo dataset is essential for training robust tattoo retrieval systems and tattoo detectors, therefore, we proposed digital tattooing (DT) approach to synthesize tattoo images. We focus on synthesizing portrait tattoo images because portrait photos contain sufficient texture details for us to evaluate the performance of the DT algorithm. DT algorithm takes a portrait photo, a real portrait tattoo image, and a skin image as input. The portrait tattoo image is used as a reference. DT algorithm extracts the facial landmarks of the portrait tattoo image and enhances it by re-weighting the landmark textures and adding shadow effect, etc. Then it calculates a color mapping from reference portrait tattoo image to the enhanced portrait photo and uses a novel tattoo needle model to simulate the physical procedure of tattooing. A set of parameters are provided for controlling the visual style of the synthetic portrait tattoo images. The experimental results showed that compared with other image synthesizing methods, the DT algorithm generates more realistic portrait tattoos. In this research, we provided new solutions to tattoo retrieval and tattoo detection tasks and showed their potential to address the unsolved problems of existing tattoo retrieval systems and tattoo detectors. Furthermore, we also proposed a tattoo synthesizing approach which can be very useful in generating a large number of tattoo images for training tattoo retrieval systems and tattoo detectors.