How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs
An outstanding fact (OF) is a striking claim by which some entities stand out from their peers on someattribute. OFs serve data journalism, fact checking, and recommendation. However, one could jump to conclusions by selecting truthful OFs while intentionally or inadvertently ignoring lateral contex...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | text |
Language: | English |
Published: |
Institutional Knowledge at Singapore Management University
2024
|
Subjects: | |
Online Access: | https://ink.library.smu.edu.sg/sis_research/9667 https://ink.library.smu.edu.sg/context/sis_research/article/10667/viewcontent/KDD24_KGPerturbationAnalysis.pdf |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Singapore Management University |
Language: | English |
id |
sg-smu-ink.sis_research-10667 |
---|---|
record_format |
dspace |
spelling |
sg-smu-ink.sis_research-106672024-11-28T09:20:59Z How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs XIAO, Hanhua LI, Yuchen WANG, Yanhao KARRAS, Panagiotis MOURATIDIS, Kyriakos AVLONA, Natalia Rozalia An outstanding fact (OF) is a striking claim by which some entities stand out from their peers on someattribute. OFs serve data journalism, fact checking, and recommendation. However, one could jump to conclusions by selecting truthful OFs while intentionally or inadvertently ignoring lateral contexts and data that render them less striking. This jumping conclusion bias from unstable OFs may disorient the public, including voters and consumers, raising concerns about fairness and transparency in political and business competition. It is thus ethically imperative for several stakeholders to measure the robustness of OFs with respect to lateral contexts and data. Unfortunately, a capacity for such inspection of OFs mined from knowledge graphs (KGs) is missing. In this paper, we propose a methodology that inspects the robustness of OFs in KGs by perturbation analysis. We define (1) entity perturbation, which detects outlying contexts by perturbing context entities in the OF; and (2) data perturbation, which considers plausible data that render an OFless striking. We compute the expected strikingness scores of OFs over perturbation relevance distributions and assess an OF as robust if its measured strikingness does not deviate significantly from the expected. We devise a suite of exact and sampling algorithms for perturbation analysis on large KGs. Extensive experiments reveal that our methodology accurately and efficiently detects frail OFs generated by existing mining approaches on KGs. We also show the effectiveness of our approaches through case and user studies. 2024-08-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/9667 info:doi/10.1145/3637528.3671763 https://ink.library.smu.edu.sg/context/sis_research/article/10667/viewcontent/KDD24_KGPerturbationAnalysis.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University outstanding facts robustness measurement perturbation analysis knowledge graphs Databases and Information Systems Graphics and Human Computer Interfaces |
institution |
Singapore Management University |
building |
SMU Libraries |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
SMU Libraries |
collection |
InK@SMU |
language |
English |
topic |
outstanding facts robustness measurement perturbation analysis knowledge graphs Databases and Information Systems Graphics and Human Computer Interfaces |
spellingShingle |
outstanding facts robustness measurement perturbation analysis knowledge graphs Databases and Information Systems Graphics and Human Computer Interfaces XIAO, Hanhua LI, Yuchen WANG, Yanhao KARRAS, Panagiotis MOURATIDIS, Kyriakos AVLONA, Natalia Rozalia How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs |
description |
An outstanding fact (OF) is a striking claim by which some entities stand out from their peers on someattribute. OFs serve data journalism, fact checking, and recommendation. However, one could jump to conclusions by selecting truthful OFs while intentionally or inadvertently ignoring lateral contexts and data that render them less striking. This jumping conclusion bias from unstable OFs may disorient the public, including voters and consumers, raising concerns about fairness and transparency in political and business competition. It is thus ethically imperative for several stakeholders to measure the robustness of OFs with respect to lateral contexts and data. Unfortunately, a capacity for such inspection of OFs mined from knowledge graphs (KGs) is missing. In this paper, we propose a methodology that inspects the robustness of OFs in KGs by perturbation analysis. We define (1) entity perturbation, which detects outlying contexts by perturbing context entities in the OF; and (2) data perturbation, which considers plausible data that render an OFless striking. We compute the expected strikingness scores of OFs over perturbation relevance distributions and assess an OF as robust if its measured strikingness does not deviate significantly from the expected. We devise a suite of exact and sampling algorithms for perturbation analysis on large KGs. Extensive experiments reveal that our methodology accurately and efficiently detects frail OFs generated by existing mining approaches on KGs. We also show the effectiveness of our approaches through case and user studies. |
format |
text |
author |
XIAO, Hanhua LI, Yuchen WANG, Yanhao KARRAS, Panagiotis MOURATIDIS, Kyriakos AVLONA, Natalia Rozalia |
author_facet |
XIAO, Hanhua LI, Yuchen WANG, Yanhao KARRAS, Panagiotis MOURATIDIS, Kyriakos AVLONA, Natalia Rozalia |
author_sort |
XIAO, Hanhua |
title |
How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs |
title_short |
How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs |
title_full |
How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs |
title_fullStr |
How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs |
title_full_unstemmed |
How to avoid jumping to conclusions: Measuring the robustness of outstanding facts in knowledge graphs |
title_sort |
how to avoid jumping to conclusions: measuring the robustness of outstanding facts in knowledge graphs |
publisher |
Institutional Knowledge at Singapore Management University |
publishDate |
2024 |
url |
https://ink.library.smu.edu.sg/sis_research/9667 https://ink.library.smu.edu.sg/context/sis_research/article/10667/viewcontent/KDD24_KGPerturbationAnalysis.pdf |
_version_ |
1819113097213771776 |