Great to see attempts to measure impact in such difficult areas. I'm wondering if there's a problem of attribution that looks like this (I'm not up to date on this discussion):

1. An organisation like the Future Academy or 80,000 hours or someone says "look, we probably got this person into a career in AI safety, which has a higher impact, and cost us $x, so our impact per dollar is $x per probable career spent on AI safety".
2. The person goes to do a training program, and they say "we trained this person to do good work in AI safety, which allows them to have an impact, and it only cost us $y to run the program, so our impact is $y per impactful career in AI safety"
3. The person then goes on to work at a research organisation, who says "we spent $z including salary and overheads on this researcher, and they produced a crucial seeming alignment paper, so our impact is $z per crucial seeming alignment paper".

When you account for this properly, it's clear that each of these estimates is too high, because part of the impact and cost has to be attributed elsewhere.

A few off the cuff thoughts:

It seems there should be a more complicated discounted measure of impact here for each organisation, that takes into account additional costs.

It certainly could be the case that at each stage the impact is high enough to justify the program at the discounted rate.

This might be a misunderstanding of what you're actually doing, in which case I would be excited to learn that you (and similar organisations) already accounted for this!

I don't mean to pick on any organisation in particular if no one is doing this, it's just a thought about how these measures could be improved in general.

I really appreciate the feedback!

Thanks for writing such a thoughtful comment. The post has to reflect the content of the paper, so I'm glad your comment can provide extra context. The post now reflects that the paper was written in 2019, and I plan to address the 30x figure soon.

Thanks for pointing this out, the version on the GPI website has been corrected.

Thanks, this is really helpful information about trusts and the 4% rule! 

On self trust: I feel that a common pattern might be that when you're young, you're 'idealistic' and want to do things like donate. When you're older, you feel like spending your money (if you have it) in ways that might not make you particularly happy. I might even decide I would rather give it all to my kids (if I have some). This makes me think there's a good chance I won't donate it later if I haven't pre-committed. 

On safety: I am from Australia, and to some extent my context is probably quite different to many others. (On the whole, Australia tends to look after you if you get severely injured or run entirely out of money. This makes quick access less of a pressing consideration for me). But to the extent that it is an important consideration, why not have a little money that's easily accessible and most of it in a trust?

Here are some articles I think would make good scripts (I'll also be submitting one script of my own). 

Summaries of the following papers:

• The Epistemic Challenge to Longtermism 
• A Paradox for Tiny Probabilities and Enormous Values 
• The Case for Strong Longtermism
• Forecasting transformative AI: the "biological anchors" method in a nutshell
• Are we living at the hinge of History?^[1]
• In defence of fanaticism^[1]
• Longtermist Institutional Reform^[1]
• Doomsday Rings Twice^[1]
• Asymmetry, Uncertainty, and the Longterm^[1]
• Simulation in Expectation^[1]
• Moral Uncertainty about population axiology^[1]
• Existential risk pessimism and the time of perils^[1]

I'd also suggest the following papers which I haven't seen a summary of:

• The Potato's Contribution to Population and Urbanization: Evidence from a Historical Experiment
• A Bayesian Truth Serum for Subjective Data
• Improving Judgments of Existential Risk: Better Forecasts, Questions, Explanations, Policies
• The Parliamentary Approach to Moral Uncertainty
• Is Power-Seeking AI an Existential Risk?
• I'd also suggest all of WWOTF's supplementary materials, especially Significance, Persistence and Contingency.

(Edit: Spacing)

1. ^{^}

    I am writing these 8 summaries, message me if you want to see them early.  

Recently, I was reading David Thorstad’s new paper “Existential risk pessimism and the time of perils”. In it, he models the value of reducing existential risk on a range of different assumptions.

The headline result is that 1) most plausibly, existential risk reduction is not overwhelmingly valuable–though it may still be quite valuable, it doesn’t probably swamp all other cause areas. And 2) thinking that extinction is more likely tends to weaken the case for existential risk reduction rather than strengthen it.

It struck me that one of the results is particularly interesting, I call it the repugnant solution:

If we can reduce existential risk to 0% per century across all future centuries, this act is infinitely valuable, even if the initial risk was absolutely tiny and each century is only just of positive value. This act is therefore, better than basically anything else we could do.

Perhaps, in a Pascalian way, if we think there is a tiny chance that some particular action will lead to a permanent reduction in existential risk, that act too is infinitely valuable, and everything breaks.

This is also true even if we decrease the value of each century from “really amazingly great” to “only just net positive”.