Thank you for presenting these views! This was very interesting.
I have some questions of interest to me - apologies if I've missed something and for these being slightly outside of scope.
Excited to see more work on mental health charities! Thank you for this. I will need a bit of time to read before I comment I could comment in more detail.
What's stopping me from have a good overview of your results is that the cost-effectiveness of each proposed intervention is on a different mental-health outcome. If I am not mistaken, these have different scale sizes. Do you have results converted in effect sizes (Cohen's d)? This would mean all the outcomes are converted to the same unit, standard deviations. This makes it easier to compare and allows you to compare them to other interventions that are also evaluated in affect/wellbeing (e.g., McGuire et al., 2022).
As my colleagues have mentioned in their responses (Michael's general response, Joel's technical response), the WELLBYs per $1000 that GiveWell put forward for AMF are dependent on philosophical choices about the badness of death and the neutral point. There are a range of plausible possible choices and these can affect the results. HLI does not hold a view.
We've whipped up an R Shiny app so that you, the reader, can play around with these choices and see how your views affect the comparison between StrongMinds and AMF.
Please note that this is a work in progress and was done very quickly. Also, I'm using the free plan for hosting the app so it might be a bit slow/limited in monthly bandwidth.
Hi Nick,
Thanks for pointing out both kinds of biases. These biases can cause a failure of comparability. Concretely, if an intervention causes you to give counterfactually higher scores as a matter of ‘courtesy’ to the researcher, then the intervention changed the meaning of each given response category.
I therefore take it that you don’t think that our particular tests of comparability will cover the two biases you mention. If so, I agree. However, my colleague has given reasons for why we might not be as worried about these sorts of biases.
I don’t think this can be tested in our current survey format, but it might be testable in a different design. We are open to suggestions!
Hello Henry,
Thank you for presenting this thought experiment.
The core here is about whether groups like the Sentinelese who do not have the same levels of development as others would give similar levels of SWB. I think the other comments here have done a great job at pointing out possible explanations.
Some briefs answers / pointers. Many of these things have been discussed in more details elsewhere.
Hi Nick, A quick comment to thank you for engaging with our work and for your insights. This is super interesting.
Arthritis - same as treating most other pain - large amounts of paractamol, ibuprofen (and other nsaids) and diclofenac gel is what we do for arthritis.
This suggests that this could be really cost-effective, considering the price of NSAIDs! However, wouldn't issues of side effects also occur here? Or is this less of an issue because the gains would be higher?
Thank you for your engagement with our research. Our research team is going on winter break so we might not respond to additional comments until after the holidays.
The point is to form a view on whether we recommend deworming. What matters to us are effects on subjective wellbeing. We would like to restate that this is the only SWB data on deworming that we could find, and this the only SWB analysis that we know of. The effects are non-significant, so this data does not give us good grounds to recommend deworming (see Section 4). The broader literature on outcomes other than SWB (reviewed in Section 1.3) is so mixed that our results do not strike us as particularly surprising – it’s just one null result amongst many.
We could just stop there, but because this is a novel analysis and dealing with non-significant findings isn’t easy, we aired on the side of being overly-thorough. For completeness, we discuss many other considerations in Section 2.3 (alternative analyses, Bayes factors, cost-effectiveness, converting GiveWell’s analysis to WELLBYs, etc. - cost-effectiveness being one of them). These converging lines of evidence all fail to provide good grounds to recommend deworming. Of course, non-significant findings do not prove there is no effect, and we discuss limitations of the data in section 5 that highlight key uncertainties about the results. In section 6, we discuss future research that could help address these limitations.
If strong evidence is produced showing that deworming is more cost-effective than StrongMinds for SWB, then we would change our recommendation. Collecting this data could be expensive, but given the insufficient evidence we think it is necessary for proponents to clarify the effect of deworming on SWB (which we think is what ultimately matters).
Have a nice end of the year and keep doing good folks.
Thank you for taking the time to reply. Your responses to 1a and 1b make sense to me. 2 I'm still exploring and turning these ideas around in my mind - thank you for the paper. I wonder if some of this can be tested by asking people about their number of desires, general life satisfaction, % life satisfaction/desires fulfilled.
If I may, I'd like to expand a bit on number 1.