Reasoning through Quantitative Research Evidence: A Group Decision Experiment with Civil Servants in Pakistan

We examine whether small-group deliberation enhances policymakers’ collective ability to reason through quantitative research evidence. In a lab-in-the-field experiment with cadre civil servants in Pakistan, participants complete a stylized reasoning task either individually or in pairs. The task requires interpreting causal impact evaluations to assess the likely effects of a proposed intervention, reflecting the analytical demands of evidence use in a policy context. Despite access to a larger pool of knowledge, groups do not outperform individuals on average. However, treatment-group analysis reveals substantial unrealized potential: groups possessed sufficient pooled knowledge to achieve significantly higher scores but frequently failed to retain and integrate the correct answers initially available to their members. Groups characterized by a stronger top performer perform better, whereas groups in which correct answers are more dispersed across members tend to perform worse and retain less of their available knowledge. Our study contributes to the literature by providing rare experimental evidence on group deliberation among active civil servants and by identifying conditions under which deliberation may fail to enhance evidence use in bureaucratic decision-making.