Things you Wanted to Know about Bias in Evaluations but Never Dared to Think

Volume 45 Number 6
Published: January 21, 2016
https://doi.org/10.1111/1759-5436.12112
The thrust for evidence‐based policymaking has paid little attention to problems of bias. Statistical evidence is more fragile than generally understood, and false positives are all too likely given the incentives of policymakers and academic and professional evaluators. Well‐known cognitive biases make bias likely for not dissimilar reasons in qualitative and mixed methods evaluations. What we term delinquent organisational isomorphism promotes purportedly scientific evaluations in inappropriate institutional contexts, intensifying motivated reasoning and avoidance of cognitive dissonance. This leads to states of denial with regard to the validity of much evaluation activity. Independent replications, revisits and restudies, together with codes of ethics that relate to professional integrity, may mitigate these problems.

Keywords:

  • Impact
  • Evaluation
  • M&E
From Issue: Vol. 45 No. 6 (2014) | Rethinking Impact Evaluation for Development