New research from Bent Flyvbjerg, Senior Research Fellow of St Anne’s and First BT Professor and Inaugural Chair of Major Programme Management at Saïd Business School, and Dirk W. Bester, professional statistician, finds that cost-benefit analyses of public investments are highly inaccurate and biased.
The bad news is that this means the many cost-benefit analyses underpinning most public investments — from HS2 in the UK to California’s high-speed rail line — cannot be trusted as currently done. The good news is that Flyvbjerg and Bester have developed better methods and data to overcome this. Professor Flyvbjerg found that amongst forecasters, policymakers and scholars there tends to be a misguided assumption that cost-benefit forecasts are more or less accurate, when in fact they are highly inaccurate and biased. This fallacy, called the “cost-benefit fallacy” results in overestimates of benefit-cost ratios of 50 to 200 per cent, with individual estimates often exceeding these averages. The fallacy makes investments look profitable on paper, when in reality they are often unprofitable.
The cost-benefit fallacy is shown to be fundamental and pervasive, which has never been documented before.
Professor Flyvbjerg summarises: “Most cost-benefit analyses assume that the estimates of costs and benefits are more or less accurate and unbiased. But what if, in reality, estimates are highly inaccurate and biased? Then the assumption that cost-benefit analysis is a rational way to improve resource allocation would be a fallacy. This study includes the most comprehensive test ever conducted of the validity of CBA. Based on the largest dataset of its kind, we test the assumption that cost and benefit estimates of public investments are accurate and unbiased. We find this is not the case with overwhelming statistical significance.”
2,062 public capital investments were considered during the study, these consisted of dams, rail developments, tunnels and bridges amongst other major infrastructure projects. The sample included investments from 104 countries dating between 1927 and 2018, the older investments allowing historical trends to be considered.
Professor Flyvbjerg recommends that to redress these cost-benefit failings the following method should be followed:
(a) Behavioural science should be used to address biases in cost-benefit analysis
(b) Incentives should be used to encourage this
(c) Independent audits should be used to verify results
(d) Other real-life concerns that need consideration alongside cost-benefit analysis should be integrated through normal democratic processes
Professor Flyvbjerg commented: ‘The findings are crucial for all policymaking based on cost-benefit analysis. Policymakers will make the wrong decisions if they trust cost-benefit analysis as it is practiced today. We don’t stop with pointing out the problem, however, we also present solutions. This research used an incredibly large dataset, the biggest of its kind in the world. But precisely because of the many data our conclusions are firm: the cost-benefit fallacy is a fact that must be overcome if good policymaking is the goal.’
This new research has been covered by the Financial Times.
The full paper can be read here.