I don't think that high x-risk implies that we should focus more on x-risk all else equal - high x-risk means that the value of the future is lower. I think what we should care about is high tractability of x-risk, which sometimes but doesn't necessarily correspond to a high probability of x-risk. 

I pretty strongly disagree with the thrust of this post. As other commenters have pointed out, public outreach projects are inevitably heavy-tailed ex-ante, and if we think that public outreach projects are worth funding then most outputs will be duds. One could make the argument that public outreach projects aren't worth it in expectation, or that there's some good reason to think that it's silly to fund video games for public outreach, but generically picking on grants that look bad ex-post seems like a very bad way to evaluate funding strategies to me. 

Thanks Charlie, enjoyed this. 

I'd be very surprised if this burnt your ability to speak with EAs. 

I essentially agree with the basic point of this post - and think it was a great post!

I have some what feel like nitpicks about the specific story that you told that and I'm sort of confused about how much they matter. My guess is that this actually is a counterargument to the point being made in the post and imply that trapped priors are less of a problem than the example used in the post would imply. 

I think that the broadly libertarian view and Scandinavian-style social democracy views are much more similar than this post gives them credit for. In particular, they agree on the crucial importance of liberal democracy that prevents elites (in the 19th-century traditional agricultural elites) from using the state to engage in rent-seeking. I remember reading a list of demands of the German Social Democratic party in the 1870s (before it had moderated) that read a list of liberal democratic demands - secret ballot, free speech, expansion of the power of democratically elected Reichstag etc.  These two strands of modern liberal thought also agreed on a liberal epistemology that should be used to try to systematically improve society from a broadly utilitarian perspective - the London School of Economics was founded by 4 Fabian Society members to further this aim! 

I think this cashes out in the Effective Samaritans and the Libertarian side of EA (although the Libertains side of EA is pretty unusually libertarian) pursuing pretty similar projects when trying to use non-randomista means for development. For instance, my guess is that both would support increasing state capacity in low-income countries to improve the basic nightwatchman functions of the state, reducing corruption, protecting liberties and the integrity of elections, and removing regulation that that represent elite rent-seeking. Of course they'll be some differences in emphasis - the Effective Samarations might have a particular theory of change around using unions to coordinate labour to push for political change - but these seem relatively minor compared to the core things both agree are important. Byran Caplan and Robin Hanson are genuinely unusually libertarian even amongst broadly free-market economists, but typically both utilitarian-motivated libertarians and social democrats would be interested in building at least a basic welfare state in low-income countries. 

I think we actually in practice see this convergence between liberal social democrats and broadly utilitarian libertarians in the broadly unified policy agendas of Ezra Klien's abundance agenda and lots of EA-Rationalist adjacent Libertarians like a focus on making it easier to build houses in highly productive cities, reducing barrier to immigration to rich countries, increased public funding of R&D and improving state capacity, particularly around extremely ambitious projects like operation warp speed. 

This is really useful context!

This looks fantastic, thanks for putting it together!

Thanks for your comment Ceb. 

I think my case for more focus on good statistical work when looking at governance is that when doing good statistical work on interventions, we often find very high degrees of variation in effect sizes that are enough to justify the extra work of the intervention. I'm personally very unsure of the effect size of changes in liability law, soft law, and various corporate governance interventions on accident rates. 

There's been lots of great case study/best guess/expert consensus work on these questions which I think is often great and I'm very happy exists, but leaves me with large uncertainties about effect sizes, and so on the current margin, I want more good statistical work. 

I think the case for good statistical work on areas like degree of misuse risk and forecasting is stronger because I think that people's takes on these questions are pretty grounded in quite theoretical arguments (unlike governance interventions which are much more empirically grounded) that I think would benefit a lot from more grounded statistical work. 

Thanks for writing this Charlie, this was great

This is great Matt! I think I'd be also be interested in work trying to estimate the effect sizes of this stuff, as well as research on optimal design.