Camerer, C. F. (2016) replicated two experiments, namely, “Happiness and Time Preference: The Effect of Positive Affect in a Random-Assignment Experiment”by Ifcher, J. and Zarghamee, H. (2011) and “Deferred Compensation in Multiperiod Labor Contracts: An Experimental Test of Lazear’s Model”by Huck, S., Seltzer, A. J., and Wallace, B. (2011). The replication results were described in Camerer’s et al. “Evaluating replicability of laboratory experiments in economics” in Science First Release.
This study calls into question reproducibility of scientific findings. To contribute data about reproducibility in economics, 18 studies published in the American Economic Review and the Quarterly Journal of Economics in 2011-2014 were replicated, among them, the abovementioned experiments run at CESS. The findings showed a significant effect in the same direction as the original study for 11 replications (61.1%). This is notably lower than the replication rate of 92% (mean power) that would be expected if all original effects were true and accurately estimated. The results also show that there is some information in post-publication peer beliefs (revealed in both markets and surveys), and in simple statistics from published results, about whether studies are likely to replicate. All these developments suggest that cultivation of good professional norms, weeding out bad norms, disclosure requirements policed by journals, and simple evidence-based editorial policies can improve reproducibility of science, perhaps very quickly.