Threats to the operational use of situational judgment tests in the college admission process

• Main Authors: CULLEN, Michael J., SACKETT, Paul R., LIEVENS, Filip
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2006
• Subjects: Educational Psychology; Higher Education; Organizational Behavior and Theory
• Online Access: https://ink.library.smu.edu.sg/lkcsb_research/5565; https://ink.library.smu.edu.sg/context/lkcsb_research/article/6564/viewcontent/coaching.pdf
• Institution: Singapore Management University

This study examined the coachability of two situational judgment tests, the College Student Questionnaire (CSQ) and the Situational Judgment Inventory (SJI), developed for consideration as selection instruments in the college admission process. Strategies for raising scores on each test were generated, and undergraduates were trained in the use of the strategies using a video-based training program. Results indicated that the CSQ was susceptible to coaching. In addition, the scoring format of the CSQ was found to be easily exploited, such that trainees could increase their scores by greater than 1 SD simply by avoiding extreme responses on that test. The results as a whole sounded a note of caution for the potential use of the CSQ in the college admission process.