A head-to-head comparison of situational judgment tests and assessment centers for measuring and predicting the same performance dimensions

• Main Authors: SHAKERI, Iman, LIEVENS, Filip
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2025
• Subjects: assessment centers; general mental ability; head-to-head comparison; personality; simulation; situational judgment tests; validity; Industrial and Organizational Psychology; Organizational Behavior and Theory
• Online Access: https://ink.library.smu.edu.sg/lkcsb_research/7634
• Institution: Singapore Management University

To date, a limited set of studies have compared the criterion-related validity of low-fidelity (SJT) versus high-fidelity (AC) simulations for predicting job performance. Unfortunately, these studies validated these simulations through the overall assessment rating (OAR) instead of on the basis of specific dimensions. Given SJTs and ACs were compared that measured different dimensions, our understanding of the relative and comparative validity of these assessment approaches in measuring the same set of dimensions is still limited. Therefore, this study aims to conduct a head-to-head comparison of the criterion-related validity of the AC and the SJT (and their incremental validity) while keeping the performance dimensions under investigation constant. Data were collected from 406 applicants for supervisory and management positions in a large Iranian steel industry company. In this process, a general mental ability test, a personality inventory, an SJT, and an AC were used as predictors, and supervisory ratings of job performance dimensions (Thinking, Feeling, and Power) served as criteria. The AC had relatively high validity for all three dimensions, whereas the SJT had a similar validity only for the Thinking dimension. So, the SJT was significantly weaker in assessing the Feeling and Power dimensions. These results were confirmed by incremental validity analyses. Overall, this study shows that understanding the relationships between predictor and criterion dimensions plays a critical role in developing valid selection systems.