Big data processing model for authorship identification

• Main Authors: Eng, T. C., Hasan, S., Shamsuddin, S. M., Wong, N. E., Jalil, I. A.
• Format: Article
• Published: International Center for Scientific Research and Studies 2017
• Subjects: QA75 Electronic computers. Computer science
• Online Access: http://eprints.utm.my/id/eprint/76314/; https://www.scopus.com/inward/record.uri?eid=2-s2.0-85033701418&partnerID=40&md5=48ba51a458663108253929e5316fcc55
• Institution: Universiti Teknologi Malaysia

The era of Big Data has arrived and an average of about quintillions of data is produced daily. Data can be in many forms such as image, document or movie. For document file, there are digitalized document and handwritten document that often relates to the issue of copyright or ownership. This is due to improper authentication that leads to unhealthy authorship claimed of that particular handwritten document. Authorship identification is a sub-area of Document Image Analysis and Identification (DIAR). DIAR aim is to analyze and identify document authorship. However, for big scale of documents text images, the issue of document processing time becomes crucial for better authorship identification. Therefore, in this study, we propose an alternative solution to solve the above problems dealing with massive amount of document text images by integrating Hadoop MapReduce and Spark's MLlib for authorship identification through data processing parallelization. MapReduce processing is used as the platform to pre- process these huge document text images in Hadoop Distributed File Systems (HDFS), follows by the authorship identification through Apache Spark machine learning library.The experiments show the integration is successfully implemented for big size of document text images. However, further improvement is needed for the post-analytics of the reduced document text images for better identification.