Some quick observations:

• Eastern Europe comes out well when economy, workforce and law are given equal weighting. They are helped by the scaling on the GBP and income indicators - in the “rankings” version, Western & Northern European start to compete
• India and Philippines look better as the emphasis on economy increases
• Weighting only on economy and law (cheap, stable countries) gives a top 5 of India, Rwanda, Georgia, Indonesia, Uzbekistan
• With a double weighting on workforce, the top 5 are New Zealand, Australia, US, UK, Georgia
• Georgia comes out looking surprisingly good. It appears to have flukey scores on education and crime.
• I did some top 10 rankings with different ratings

Any update on what you are doing/thinking now, 11 months later?

As far as I can tell, ex-ante cost-effectiveness is not the most important figure for someone considering whether to fund a future project. I think the expected benefit per unit cost is more important.

For reference, you give this definition:

We define the ex-ante cost-effectiveness of an R&D project as the expected value of its ex-post cost-effectiveness, given only the information that is available before the project is conducted. 

I think I understand what this means, but I am going to attempt to show why it's not that useful using a simple example.

Scenario 1: Suppose we know that a project will have benefit $B=1$ and that the projected cost $C$ has distribution

$P(C=1)=\frac{1}{2}$

$P(C=10)=\frac{1}{2}$

Then the ex-post cost-effectiveness $CE$ of the project will have distribution

$P(CE=1)=\frac{1}{2}$

$P(CE=\frac{1}{10})=\frac{1}{2}$

and thus has expected value

$E\left(CE\right)=\frac{1}{2}\cdot 1+\frac{1}{2}\cdot \frac{1}{10}=0.505$

Why is this not useful? It does not reflect the expected return-on-investment, and is not sensitive to high-cost scenarios. Consider Scenario 2, a similar project with known benefit $B=1$ and cost with distribution

$P(C=1)=\frac{1}{2}$

$P(C=10000)=\frac{1}{2}$

Scenario 2 is clearly much less cost-effective than Scenario 1. But the ex-ante cost-effectiveness is $0.500005$, very close to $0.505$.

What a decision-maker really wants to know is the amount of benefit they can expect from each unit of investment. This can be given by$\frac{E\left(B\right)}{E\left(C\right)}$.

Scenario 1: $\frac{E\left(B\right)}{E\left(C\right)}=\frac{1}{5.5}\approx 0.18$

Scenario 2: $\frac{E\left(B\right)}{E\left(C\right)}=\frac{1}{5000.5}\approx 0.00020$

We can see that this does appropriately reflect the difference in cost-effectiveness between the two scenarios. What I'm not so sure about is how we might give the expected benefit per unit cost as a distribution, rather than just a point-estimate.

It seems likely that I'm missing something.

• What is your rationale for focusing on expected value of ex-post cost-effectiveness ?
• Could you use an adapted method to make an ex-ante prediction of the benefit per unit cost of Baumsteiger’s R&D project?

Can we see it yet?

I think Ghandi's point nods to the British Empire's policy of heavily taxing salt as a way of extracting wealth from the Indian population. For a time this meant that salt became very expensive for poor people and many probably died early deaths linked to lack of salt.

However, I don't think anyone would suggest taxing salt at that level again! Like any food tax, the health benefits of a salt tax would have to be weighed against the costs of making food more expensive. You certainly wouldn't want it so high that poor people don't get enough of it.

Thanks again!

I think I have been trying to portray the point-estimate/interval-estimate trade-off as a difficult decision, but probably interval estimates are the obvious choice in most cases.

So I've re-done the "Should we always use interval estimates?" section to be less about pros/cons and more about exploring the importance of communicating uncertainty in your results. I have used the Ord example you mentioned.

Thanks for your feedback, Vasco. It's led me to make extensive changes to the post:

• More analysis on the pros/cons of modelling with distributions. I argue that sometimes it's good that the crudeness of point-estimate work reflects the crudeness of the evidence available. Interval-estimate work is more honest about uncertainty, but runs the risk of encouraging overconfidence in the final distribution.
• I include the lognormal mean in my analysis of means. You have convinced me that the sensitivity of lognormal means to heavy right tails is a strength, not a weakness! But the lognormal mean appears to be sensitive to the size of the confidence interval you use to calculate it - which means subjective methods are required to pick the size, introducing bias.

Overall I agree that interval estimation is better suited to the Drake equation than to GiveWell CEAs. But I'd summarise my reasons as follows:

• The Drake Equation really seeks to ask "how likely is it that we have intelligent alien neighbours?", but point-estimate methods answer the question "what is the expected number of intelligent alien neighbours?". With such high variability the expected number is virtually useless, but the distribution of this number allows us to estimate the number of alien neighbours. GiveWell CEAs probably have much less variation and hence a point-estimate answer is relatively more useful
• Reliable research on the numbers that go into the Drake equation often doesn't exist, so it's not too bad to "make up" interval estimates to go into it. We know much more about the charities GiveWell studies, so made-up distributions (even those informed by reliable point-estimates) are much less permissible.

Thanks again, and do let me know what you think!

My attempt to summarize why the model predicts that preventing famine in China and other countries will have a negative effect on the future:

• Much of the value of the future hinges on whether values become predominantly democratic or antidemocratic
• The more prevalent antidemocratic values (or populations) are after a global disaster, the likelier it is that such values will become predominant
• Hence preventing deaths in antidemocratic countries can have a negative effect on the future.

Or as the author puts it in a discussion linked above:

To be blunt for the sake of transparency, in this model, the future would improve if the real GDP of China, Egypt, India, Iran, and Russia dropped to 0, as long as that did not significantly affect the level of democracy and real GDP of democratic countries. However, null real GDP would imply widespread starvation, which is obviously pretty bad! I am confused about this, because I also believe worse values are associated with a worse future. For example, they arguably lead to higher chances of global totalitarianism or great power war.

I agree with the author that the conclusion is confusing. Even concerning.

I'd suggest that the conclusion is out-of-sync with how most people feel about saving lives in poor, undemocratic countries. We typically don't hesitate to tackle neglected tropical diseases on the basis that doing so boosts the populations of dictatorships.

Perhaps it can be captured by ensuring we compare counterfactual impacts.

For an urgent, "now or never" cause, we can be confident that any impact we make wouldn't have happened otherwise.

For something non-urgent, there is a chance that if we leave it, somebody else could solve it or it could go away naturally. Hence we should discount the expected value of working on this (or in other words we should recognise that the counterfactual impact of working on non-urgent causes, which is what really matters, is lower than the apparent impact).