Is Economics Finally Becoming Trustworthy?

The field of empirical research has faced significant scrutiny over the years, with experts like economist Ed Leamer criticizing the lack of trustworthiness in data analyses. In his 1983 article “Lets Take the Con Out of Econometrics,” Leamer highlighted the arbitrary choices made in research processes that could impact the credibility of results. Despite this, peer-reviewed studies were generally taken seriously by the educated public for many years.

However, a shift began with physician John Ioannidis’ 2005 article “Why Most Published Research Findings Are False,” which raised concerns about the reliability of research findings. The replication crisis of the 2010s further fueled doubts, particularly in fields like psychology and economics. The core premise of science is replicability, where other researchers should be able to reproduce the same results if an experiment is well-documented.

Social science research, including economics, has faced challenges in replicating findings. Studies have shown that a significant number of published papers do not share their data or code, making it difficult for others to verify their results. However, recent efforts to improve transparency and replicability in research have shown some progress.

A collaborative effort led by the Center for Open Science, funded by the Defense Advanced Projects Research Agency, conducted a large-scale replication study across the social sciences. The results, published in a special issue of Nature, revealed improvements in data and code sharing practices. Economics and political science were highlighted for their relatively good reproducibility compared to other fields.

While the replication of results has shown some promise, there are still concerns about the exaggeration of effects in published research. Robustness tests have indicated that many studies may overstate the size of effects or claim effects that may not be significant. Despite these challenges, relying on research findings is still considered more reliable than chance.

Consumers of research are advised to trust entire bodies of literature rather than individual papers. Additionally, a rule of thumb inspired by the Nature papers suggests cutting estimated effect sizes in half to account for potential exaggeration. While the field of empirical research continues to face challenges, ongoing efforts to improve transparency and replicability are essential for building trust in scientific findings.