FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation
In crowdsourced data aggregation task, there exist conflicts in the answers provided by large numbers of sources on the same set of questions. The most important challenge for this task is to estimate source reliability and select answers that are provided by high-quality sources. Existing work solv...
Saved in:
Main Authors: | , , , , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2015
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/3258 https://ink.library.smu.edu.sg/context/sis_research/article/4260/viewcontent/KDD2015Fenglong_1_.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-4260 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-42602020-07-15T08:46:50Z FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation MA, Fenglong LI, Yaliang LI, Qi QIU, Minghui GAO, Jing ZHI, Shi SU, Lu ZHAO, Bo HAN, Jiawei In crowdsourced data aggregation task, there exist conflicts in the answers provided by large numbers of sources on the same set of questions. The most important challenge for this task is to estimate source reliability and select answers that are provided by high-quality sources. Existing work solves this problem by simultaneously estimating sources' reliability and inferring questions' true answers (i.e., the truths). However, these methods assume that a source has the same reliability degree on all the questions, but ignore the fact that sources' reliability may vary significantly among different topics. To capture various expertise levels on different topics, we propose FaitCrowd, a fine grained truth discovery model for the task of aggregating conflicting data collected from multiple users/sources. FaitCrowd jointly models the process of generating question content and sources' provided answers in a probabilistic model to estimate both topical expertise and true answers simultaneously. This leads to a more precise estimation of source reliability. Therefore, FaitCrowd demonstrates better ability to obtain true answers for the questions compared with existing approaches. Experimental results on two real-world datasets show that FaitCrowd can significantly reduce the error rate of aggregation compared with the state-of-the-art multi-source aggregation approaches due to its ability of learning topical expertise from question content and collected answers. 2015-08-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/3258 info:doi/10.1145/2783258.2783314 https://ink.library.smu.edu.sg/context/sis_research/article/4260/viewcontent/KDD2015Fenglong_1_.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Truth Discovery Source Reliability Crowdsourcing Computer Sciences Databases and Information Systems |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
Truth Discovery Source Reliability Crowdsourcing Computer Sciences Databases and Information Systems |
spellingShingle |
Truth Discovery Source Reliability Crowdsourcing Computer Sciences Databases and Information Systems MA, Fenglong LI, Yaliang LI, Qi QIU, Minghui GAO, Jing ZHI, Shi SU, Lu ZHAO, Bo HAN, Jiawei FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation |
description |
In crowdsourced data aggregation task, there exist conflicts in the answers provided by large numbers of sources on the same set of questions. The most important challenge for this task is to estimate source reliability and select answers that are provided by high-quality sources. Existing work solves this problem by simultaneously estimating sources' reliability and inferring questions' true answers (i.e., the truths). However, these methods assume that a source has the same reliability degree on all the questions, but ignore the fact that sources' reliability may vary significantly among different topics. To capture various expertise levels on different topics, we propose FaitCrowd, a fine grained truth discovery model for the task of aggregating conflicting data collected from multiple users/sources. FaitCrowd jointly models the process of generating question content and sources' provided answers in a probabilistic model to estimate both topical expertise and true answers simultaneously. This leads to a more precise estimation of source reliability. Therefore, FaitCrowd demonstrates better ability to obtain true answers for the questions compared with existing approaches. Experimental results on two real-world datasets show that FaitCrowd can significantly reduce the error rate of aggregation compared with the state-of-the-art multi-source aggregation approaches due to its ability of learning topical expertise from question content and collected answers. |
format |
text |
author |
MA, Fenglong LI, Yaliang LI, Qi QIU, Minghui GAO, Jing ZHI, Shi SU, Lu ZHAO, Bo HAN, Jiawei |
author_facet |
MA, Fenglong LI, Yaliang LI, Qi QIU, Minghui GAO, Jing ZHI, Shi SU, Lu ZHAO, Bo HAN, Jiawei |
author_sort |
MA, Fenglong |
title |
FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation |
title_short |
FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation |
title_full |
FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation |
title_fullStr |
FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation |
title_full_unstemmed |
FaitCrowd: Fine grained truth discovery for crowdsourced data aggregation |
title_sort |
faitcrowd: fine grained truth discovery for crowdsourced data aggregation |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2015 |
url |
https://ink.library.smu.edu.sg/sis_research/3258 https://ink.library.smu.edu.sg/context/sis_research/article/4260/viewcontent/KDD2015Fenglong_1_.pdf |
_version_ |
1770573044440367104 |