I lead the Rationality Enhancement Group at the MPI for Intelligent Systems in Tübingen and will join the psychology department of UCLA as an Assistant Professor in July 2023.
I completed my PhD in the Computational Cognitive Science Lab at UC Berkeley in 2013, obtained a master’s degree in Neural Systems and Computation from ETH Zurich, and completed two simultaneous bachelor's degrees in Cognitive Science and Mathematics/Computer Science from the University of Osnabrück.
Thank you for your feedback, Stan!
I think the appropriateness of E[CE] as a prioritization criterion depends on the nature of the decision problem.
I think the expected value of the cost-effectiveness ratio is the appropriate prioritization criterion for the following scenario: i) a decision-maker is considering which organization should receive a given fixed amount of money (m), and ii) each organization (i) turns every dollar it receives into some uncertain amount of value (CE_i). In that case, the expected utility of giving the money to organization i is E[U_i]= m*E[CE_i]. Therefore, the way to maximize expected utility is to give the money to the organization with the highest expected cost-effectiveness. In this scenario, the consequences of contributing $1 to a project with an expected cost-effectiveness of 1 WELLBY/$ are almost identical in both scenarios. Most of the expected utility comes from the possibility that the project might be highly cost-effective. If the project is not highly cost-effective, then the $1 contribution accomplishes very little, regardless of whether the project costs $10,000, $100,000, or $1,000,000.
In my view, your example illustrates that the expected cost-effectiveness ratio is an inappropriate prioritization criterion if the funder has to decide whether to pay 100% of the project's costs without knowing how much that will be. In that scenario, I think the appropriate prioritization criterion would be E[B]-E[CE_alt]*E[C], where E[CE_alt] is the expected cost-effectiveness of the most promising project that the funder could fund instead.
I think the second decision problem describes the situation of a researcher or funder who is committed to seeing their project through until the end. By contrast, the first decision problem corresponds to a researcher/funder intending to allocate a fixed amount of time/money to one project or another (e.g., 3 years of personal time or 1 million dollars) and then move on to another project after that.
I think the best-known study on the subject is
Oliner, S. P. (1992). Altruistic personality: rescuers of Jews in Nazi Europe. Simon and Schuster.
Two other good articles on this subject are
Rachel Baumsteiger continues to conduct research on the intervention for promoting prosocial behavior. The intervention is currently being deployed by the University of California as a mental health service for students with adverse childhood experiences and toxic stress. This will yield some additional data on its benefits and effectiveness. However, because this deployment is funded as a mental health service, it doesn't include a control group. Running a rigorous, large-scale RCT will require additional funding. In a later post, I will show that doing so would be highly cost-effective. I think if funding became available, Rachel Baumsteiger would be happy to run the RCT. And I know that I would be excited to collaborate on that project.
"Anti-Malaria Foundation" was a typo. I have corrected it to "Against Malaria Foundation".
Thank you for your feedback, Michael, and thank you very much for making me aware of those specialized prediction platforms. I really like your suggestion. I think making predictions about the likely results of replication studies would be helpful for me. It would push me to critically examine and quantify how much confidence I should put in the studies my models rely on. Obtaining the predictions of other people would be a good way to make that assessment more objective. We could then incorporate the aggregate prediction into the model. Moreover, we could use prediction markets to obtain estimates or forecasts for quantities for which no published studies are available yet. I think it might be a good idea to incorporate those steps into our methodology. I will discuss that with our team today.
I have investigated the issues you highlighted, diagnosed the underlying errors, and revised the model accordingly. The root of the problem was that I had sourced some of the estimates of the frequency of prosocial behavior from studies on social behavior under special, unrepresentative conditions, such as infants interacting with adults for 10 min while being observed by researchers and prosocial behavior in TV series. I have removed those biased estimates of the frequency of prosocial behavior in the real world. As a consequence, the predicted lifetime increase in the number of kind acts per person reached by the intervention dropped from 1600 to 64. The predicted cost-effectiveness of the research dropped from 110 times the cost-effectiveness of StrongMinds to 7.5 times the cost-effectiveness of StrongMinds.
In producing this revised version, I also made a few additional improvements. The most consequential of those was to base the estimated cost of deploying the intervention on empirical data on the effectiveness of online advertising in $ per install.
I am currently using Squiggle to program a much more rigorous version of this analysis. That version will include additional improvements and rigorously document and justify each of the model’s assumptions.
Regardless thereof, I can rerun the analyses for E[B]/E[C] as a robustness check and let you know what I find.