Open source intelligence gathering and topic modelling on cyber security incidents

• Main Author: Ang, Chuan Kai
• Other Authors: Anwitaman Datta
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Computer science and engineering::Computer applications
• Online Access: https://hdl.handle.net/10356/138004
• Institution: Nanyang Technological University

This paper aims to demonstrate the use of Open Source Intelligence (OSINT) techniques to gather publicly available data on cyber risks and to predict current attack trends. The paper will start off by first introducing the current threat landscape. It will talk about the current state of the threat landscape, how the landscape will continuously evolve and the use of OSINT techniques to combat these upcoming rising threats. The paper will then move on to showcase the approach and methodology of using OSINT techniques. Using the defined approach, implementations will be performed accordingly. The implementation will start off by generating a list of publicly available online data sources. These sources will then be crawled using web spiders. The crawled data will then undergo processing. Processing will cleanse the crawled data of invalid data and format the crawled data into a suitable format for storing. The stored data will be queried for analysis. Findings from the analysis will then be displayed using visualization tools. From these findings, inferences are made and discussed. Recommendations deriving from the inferences will be given to the affected industry and businesses to help fight against the threats.