Assessing the accuracy of four popular face recognition tools for inferring gender, age, and race

• Main Authors: JUNG, Soon-Gyu, AN, Jisun, KWAK, Haewoon, SALMINEN, Joni, JANSEN, Bernard J.
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2018
• Subjects: Computational social science; Measurement study; Computational Engineering; Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/5338; https://ink.library.smu.edu.sg/context/sis_research/article/6342/viewcontent/17839_77973_1_PB.pdf
• Institution: Singapore Management University

In this research, we evaluate four widely used face detection tools, which are Face++, IBM Bluemix Visual Recognition, AWS Rekognition, and Microsoft Azure Face API, using multiple datasets to determine their accuracy in inferring user attributes, including gender, race, and age. Results show that the tools are generally proficient at determining gender, with accuracy rates greater than 90%, except for IBM Bluemix. Concerning race, only one of the four tools provides this capability, Face++, with an accuracy rate of greater than 90%, although the evaluation was performed on a high-quality dataset. Inferring age appears to be a challenging problem, as all four tools performed poorly. The findings of our quantitative evaluation are helpful for future computational social science research using these tools, as their accuracy needs to be taken into account when applied to classifying individuals on social media and other contexts. Triangulation and manual verification are suggested for researchers employing these tools.