Weighting admission scores to balance predictiveness-diversity: The Pareto-optimization approach

• Main Authors: LIEVENS, Filip, SACKETT, Paul R., DE CORTE, Wilfried
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: Medical school; theoretical study; algorithm; Educational Assessment, Evaluation, and Research; Medical Education; Organizational Behavior and Theory
• Online Access: https://ink.library.smu.edu.sg/lkcsb_research/6859; https://ink.library.smu.edu.sg/context/lkcsb_research/article/7858/viewcontent/ParetoMed.pdf
• Institution: Singapore Management University

Context: Although many medical schools seek to improve diversity, they grapple with the challenge of how to weight the scores of different admission methods to achieve a balance between obtaining high predictiveness and ensuring diversity in the selected student pool. Yet, in large-scale employment settings, substantial progress has been made on this front: Pareto-optimization has been introduced as an elegant statistical tool to assist decision makers in determining the weights assigned to selection methods in advance (before the selection has taken place) so that a selection system is designed to achieve an optimal balance as reflected by the trade-off that one outcome (e.g., predictiveness) cannot be improved without harm to the other outcome (e.g., diversity). Aims: This paper reviews the theory and research evidence about Pareto-optimization and explains how Pareto-optimization permits medical schools to better balance predictiveness and diversity in medical admission systems. Methods: After reviewing common weighting schemes (unit, regression-based and ad hoc weighting) and their drawbacks, we introduce the theory and logic of Pareto-optimization for better balancing predictiveness and diversity. To this end, we also offer an illustrative example. Next, we review the mathematical basis and available research evidence regarding Pareto-optimization. Finally, we discuss potential criticisms (i.e., complexity and legal concerns). Conclusions: Compared to traditional unit weighting, regression-based weighting and ad hoc weighting, Pareto-optimization leads to substantial increases in diversity intake (up to three times more), while keeping the predictiveness of the selection methods at the same level. Moreover, the Pareto-optimization is robust to sampling variability and variability of the input selection parameters.