Development of a forum search system

• Main Author: Gambani, Mohit Gurno
• Other Authors: Sun Aixin
• Format: Final Year Project
• Language: English
• Published: 2017
• Subjects: DRNTU::Engineering::Computer science and engineering::Information systems::Information storage and retrieval
• Online Access: http://hdl.handle.net/10356/70557
• Institution: Nanyang Technological University

For the past decade or two, social media has been growing at an enormous rate. The increase in social presence of users online has made Internet forums increasingly popular over the years. Social media platforms such as forums have encouraged people to communicate with each other on a vast variety of topics online. In a world largely driven by data, these Internet forums have become a huge warehouse of data comprising of people’s opinions, experiences, knowledge put in text and other forms of media. Search engines have existed since the onset of the Internet, however, the issue with search engines is that most of them operate on all the data of the World Wide Web – and that’s a huge amount of data. With online forums being widely prevalent on the Internet now, there is a need to build a search system specifically for a group of forums. This report analyzes various possible approaches to building a holistic search system catering only to online forums. The design and implementation of some of the core components of a search system for forums, such as a generic crawler and a search engine are elaborately described in this report. It further scrutinizes some of the main features and design considerations that go into the making of these components, namely data scraping, indexing, data processing, data storage and querying. It further identifies the limitations that exist in the application designed and paves way for improvements that can be made in the future.