Bias and Controversy: Beyond the Statistical Deviation

In this paper, we investigate how deviation in evaluation activities may reveal bias on the part of reviewers and controversy on the part of evaluated objects. We focus on a 'data-centric approach' where the evaluation data is assumed to represent the ground truth'. The standard stati...

Full description

Saved in:
Bibliographic Details
Main Authors: LAUW, Hady W., LIM, Ee Peng, WANG, Ke
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2006
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/892
https://ink.library.smu.edu.sg/context/sis_research/article/1891/viewcontent/biasNcontroversy.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-1891
record_format dspace
spelling sg-smu-ink.sis_research-18912017-12-26T06:38:53Z Bias and Controversy: Beyond the Statistical Deviation LAUW, Hady W. LIM, Ee Peng WANG, Ke In this paper, we investigate how deviation in evaluation activities may reveal bias on the part of reviewers and controversy on the part of evaluated objects. We focus on a 'data-centric approach' where the evaluation data is assumed to represent the ground truth'. The standard statistical approaches take evaluation and deviation at face value. We argue that attention should be paid to the subjectivity of evaluation, judging the evaluation score not just on 'what is being said' (deviation), but also on 'who says it' (reviewer) as well as on 'whom it is said about' (object). Furthermore, we observe that bias and controversy are mutually dependent, as there is more bias if there is higher deviation on a less controversial object. To address this mutual dependency, we propose a reinforcement model to identify bias and controversy. We test our model on real-life data to verify its applicability. 2006-08-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/892 info:doi/10.1145/1150402.1150478 https://ink.library.smu.edu.sg/context/sis_research/article/1891/viewcontent/biasNcontroversy.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Data centric Data models Modeling Probabilistic approach Statistical analysis Bias Ground truth Knowledge discovery Data mining Databases and Information Systems Numerical Analysis and Scientific Computing
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Data centric
Data models
Modeling
Probabilistic approach
Statistical analysis
Bias
Ground truth
Knowledge discovery
Data mining
Databases and Information Systems
Numerical Analysis and Scientific Computing
spellingShingle Data centric
Data models
Modeling
Probabilistic approach
Statistical analysis
Bias
Ground truth
Knowledge discovery
Data mining
Databases and Information Systems
Numerical Analysis and Scientific Computing
LAUW, Hady W.
LIM, Ee Peng
WANG, Ke
Bias and Controversy: Beyond the Statistical Deviation
description In this paper, we investigate how deviation in evaluation activities may reveal bias on the part of reviewers and controversy on the part of evaluated objects. We focus on a 'data-centric approach' where the evaluation data is assumed to represent the ground truth'. The standard statistical approaches take evaluation and deviation at face value. We argue that attention should be paid to the subjectivity of evaluation, judging the evaluation score not just on 'what is being said' (deviation), but also on 'who says it' (reviewer) as well as on 'whom it is said about' (object). Furthermore, we observe that bias and controversy are mutually dependent, as there is more bias if there is higher deviation on a less controversial object. To address this mutual dependency, we propose a reinforcement model to identify bias and controversy. We test our model on real-life data to verify its applicability.
format text
author LAUW, Hady W.
LIM, Ee Peng
WANG, Ke
author_facet LAUW, Hady W.
LIM, Ee Peng
WANG, Ke
author_sort LAUW, Hady W.
title Bias and Controversy: Beyond the Statistical Deviation
title_short Bias and Controversy: Beyond the Statistical Deviation
title_full Bias and Controversy: Beyond the Statistical Deviation
title_fullStr Bias and Controversy: Beyond the Statistical Deviation
title_full_unstemmed Bias and Controversy: Beyond the Statistical Deviation
title_sort bias and controversy: beyond the statistical deviation
publisher Institutional Knowledge at Singapore Management University
publishDate 2006
url https://ink.library.smu.edu.sg/sis_research/892
https://ink.library.smu.edu.sg/context/sis_research/article/1891/viewcontent/biasNcontroversy.pdf
_version_ 1770570760358723584