Related (and classic) post in case others aren't aware: EA should taboo "EA should".

Lizka makes a slightly different argument, but a similar conclusion

I guess that introduces another potential confusion though, because the scores (e.g. 0.78 for the Czech Republic) is 'Longtermist score' minus 'Neartermist score'. "Neartermism - Longtermism" might lead people to think high scores mean strong interest in neartermist cause areas.

Thanks so much for this - really clear, accessible and valuable data. I'm happy to see that the community is becoming more geographically diverse.

One minor nitpick: I found this graph confusing. The title is 'Longtermism - Neartermism' but the bars below are oriented 'Neartermism <> Longtermism'. At a glance, especially at the top, it looks like the Czechs strongly favour Neartermist causes but it's the opposite.

 

Hey Anon,

I ran this analysis, beheading (?) the data to remove all responses which I scored over 5 and changed them to 5 (18 responses).

I realised that all of these responses were from EAGx attendees - that's not that surprising since 90% of survey responses were from EAGx attendees but that highlights a limitation of my analysis. Plausibly, if I had data from CEP retreats, we might find more high-scoring impact stories.

Previously:

This means the cost per counterfactual impact point at EAGx events is ~$1,219, while the cost per counterfactual impact point at CEP retreats is ~$1,711

Post-beheading:

The cost per counterfactual impact point at EAGx events is ~$1,639, while the cost per counterfactual impact point at CEP retreats is ~$1,724 (due to GBP/USD fluctuation).

So, EAGx still performs slightly better but it's much closer, probably not a significant difference.

Thanks for proposing this - it does suggest the results are sensitive to my scoring system. I'm not sure where this leaves me; that isn't ideal and I'd like something more robust but, on the other hand, I think these high-scoring results (people securing jobs, teams being formed) are exactly the kind of things we want to happen at our events so I think it's reasonable to put significant weight on them. 

Even if you don’t want to use an equal-weighted scoring system, you could see if the results change materially if you use a much less fat-tailed rubric (e.g. have scores ranging from 1-5 vs. 1-50).

Thanks, I like this idea, and it would be easy to implement! There were only a handful of cases with scores above 5 across the dataset and I think they weren't bunched anywhere. I'll report back by the end of next week (probably just in this thread), please DM me if I drop this.

Yes, I expect they'll do worse on fostering great mentor-mentee relationships and I'm not proposing this as a fix-all. That said, I reckon the casual setting might mean the bar for attending is lower, especially if it's in an EA-dense city e.g. I can imagine a good mentor might not want to give a talk at a uni group, but might swing by a picnic. 

Thanks, Anon!

I’d be curious in seeing an equal-weighted “valuable outcomes” measure (i.e. every outcome that passes a bar gets one point vs. different scores for different outcomes) and whether that changes any results

Sounds interesting, but what's the case for doing this? To see if some events score better at producing outcomes in general? I agree outcomes are difficult to score, but I feel fairly confident that some are better than others, so some adjustment seems more appropriate to me.

I think the finding that “EAGxVirtual is unusually cost-effective” belongs in the “bottom line up front” section, and could conceivably be the most important finding of this analysis.

Thanks for this feedback. In fact, this was the case in an earlier draft, but I wanted to make the BLUF a clear outline of my core claim. Perhaps that was a mistake!

I agree with your points, thanks for adding to the case here.

Thanks, Rocky! 

Dogs descending on our snacks

I think you mistakenly listed this as an "issue", FYI

I thought these goals were reasonable!

The guidance I give to EAGx organisers about goals is:

--

We support EAGx events primarily to connect the community. Deep collaborations, within and between community members, seem key to achieving many of the goals the community has. We think that we can form more of these collaborations by helping people meet others who might be useful collaborators.

However, there are other important goals that EAGx events are particularly well-placed to serve:

• Introducing attendees to ideas which can help them find their path to solving pressing problems. Broadly speaking, events help people navigate both the people and the ideas in the community;
• Building the social network of attendees, particularly those who are new to the community; and
• Establishing new EA hubs - EAGx events often serve as the seed for future community-building in that country or city.

Feel free to set further goals specific to your conference (“e.g. re-animate the national EA group”).