Evaluation of CQAs

• Main Author: Banerjee Snehasish
• Other Authors: Alton Chua Yeow Kuan
• Format: Theses and Dissertations
• Language: English
• Published: 2012
• Subjects: DRNTU::Engineering::Computer science and engineering::Information systems
• Online Access: http://hdl.handle.net/10356/49019
• Institution: Nanyang Technological University

Search engines have long been recognized as the gateways for online information seeking, but the gradual sophistication of the online information seekers has led to the emergence of Community-based Question-Answering (CQA) sites as an alternative source of information on the Web. There are many CQAs around but their overall effectiveness has not been widely studied. So, the dissertation addresses this problem and attempts to evaluate the overall effectiveness of CQAs based on a three-pronged evaluation framework; consisting of (1) answer quality, (2) answer responsiveness, and (3) corpus comprehensiveness. A total of 500 questions were posted in each of the six CQA sites (Yahoo! Answers, WikiAnswers, Answerbag, Baidu Knows, Tencent Soso Wenwen, Sina iAsk) and the resulting 3000 transactions were examined to conduct a three-way comparison: (1) within languages – English and Chinese; (2) between languages- English versus Chinese; and (3) across all the six individual CQA sites. Results indicate that among the three English CQAs, Yahoo! Answers and WikiAnswers provide the highest answer quality; Yahoo! Answers provides the highest answer responsiveness; and WikiAnswers, the highest corpus comprehensiveness. On the other hand, among the three Chinese CQAs, Tencent Soso Wenwen and Sina iAsk provide the highest answer quality; Tencent Soso Wenwen provides the highest answer responsiveness; and Baidu Knows, the highest corpus comprehensiveness. The cross-cultural and inter-language comparison between the English CQAs and the Chinese CQAs reveal that there is hardly any difference between the two in terms of answer quality. However, the English sites outdo the Chinese sites in terms of answer responsiveness; and vice-versa in terms of corpus comprehensiveness. The comparison across all the six individual CQAs reveals that WikiAnswers and Sina iAsk provide the highest answer quality; Yahoo! Answers provides the highest answer responsiveness; while Baidu Knows provides the highest corpus comprehensiveness. The results serve as a prescriptive guideline to the online information seekers, highlighting which CQA to turn to, based on their requirements. Also, it gives useful insights on the desiderata of an ideal CQA.