Evaluation of response quality for heterogeneous question answering systems

• Main Authors: Wong, W., Sahib, S., Goh, Ong Sing
• Format: Conference or Workshop Item
• Language: English
• Published: IEEE 2005
• Subjects: QA75 Electronic computers. Computer science
• Online Access: http://eprints.utem.edu.my/id/eprint/12398/1/Evaluation_of_response_quality_for_heterogeneous_question_answering_systems_page_1-2.pdf; http://eprints.utem.edu.my/id/eprint/12398/; http://researchrepository.murdoch.edu.au/14622/
• Institution: Universiti Teknikal Malaysia Melaka

The research in this paper makes explicit why existing measures for response quality evaluation is not suitable for the ever-evolving field of question answering and following that, a short-term solution for evaluating response quality of heterogeneous systems is put forward. To demonstrate the challenges in evaluating systems of different nature, this research presents a black-box approach using a classification scheme and scoring mechanism to assess and rank three example systems.