Introduction

The Giving What We Can (GWWC) Pledge asks people to donate 10% of their income to the organizations they believe can most effectively use it to improve the lives of others. It was once used as a benchmark for the EA movement in determining growth and allocating grant money, however it has been de-emphasized over the past few years. The pledge is notably not intended for everyone, and taking the pledge can result in a loss of flexibility^[1]. It does, however, seem that taking the pledge, or convincing someone else to take the pledge, is of at least some value, since donations are generally a good thing and the pledge plausibly convinces people to donate more than they otherwise would. But there’s a question of how much value pledging has and how long people stick with it.

To look at this in depth, we turn to data from the 2017 and 2018 EA Surveys. Looking at our sample, we find 476 people self-identifying as Giving What We Can members^[2] in the 2017 data and 843 people self-identifying as Giving What We Can members in the 2018 data. However, most of this analysis will focus on the ~400 GWWC members that are non-students and have self-reported income and donation information.

Summary

Our analysis finds that:

• GWWC members donate more than non-GWWC members, both absolutely and as a percentage of income.

• ~40% of self-reported GWWC members are not reporting donation data that is consistent with keeping their pledge -- far more pledgers than GWWC originally reported based on data ending in 2014.

• This lack of keeping the pledge cannot be fully or adequately explained by appealing to people bunching donations in a particular year, discrepancies in when people took the survey versus joined GWWC, or people being non-earners (e.g., students, retired people, or homemakers).

• No relationship is found between when someone takes the pledge, joins EA, or someone’s age and whether or not they keep their GWWC pledge, frustrating arguments that this analysis is biased by timing issues. However, a significant relationship between pledge-keeping and income is found. Predictably, for example, people earning more than $100K are far more likely to keep the pledge than people earning under $30K.

• This trend is most likely the result of attrition over time, but could include methodological shortcomings, such as sampling bias or inaccurate self-reporting (either to us, GWWC, or both).

How Much Money is Giving What We Can Influencing?

Looking at EA Survey data from 2018, self-identified GWWC members in our sample reported donating an average of $14,847.77 and a median of $2,890.56 during 2017, whereas non-members reported donating an average of $9,162.14 and a median of $520.12. Looking at non-students only^[3], self-identified GWWC members in our sample report donating an average of $20,327.76 and a median of $4,199 in 2017, whereas non-members report donating an average of $12,287.59 and a median of $901.04 (Details are in Table 1).

GWWC’s influence is also clear when looking at the percentage of income donated. When looking at GWWC members, excluding students, we see they donated a mean of 13.8% and a median of 10% in 2017, compared to a mean of 7.64% and a median of 2.2% among non-members. Even after the high variability in percentage of income donation is considered, the difference is still considered statistically significant.

Another possibility for impact that is harder to analyze is the influence GWWC has on the moral norms of EA. For example, there may be EAs who choose not to take the pledge itself for concerns of flexibility, but still donate more than they otherwise would because of the influence of GWWC.

Indeed, in the distribution chart above we do see a peak at 10% even for non-GWWC members, suggesting that this target has some pull. However, the 10% target has cultural and historical precedent before GWWC existed, so it’s not easy to tell what part of this effect is attributable to GWWC or not.

How Many Giving What We Can Members Are Keeping Their Pledge?

While the GWWC pledge is meant to be taken seriously as a lifelong commitment, not everyone who takes the pledge ends up following through. As of Giving What We Can’s 2014 Impact Analysis, they had noted that 1.7% of members leave each year and an additional 4.7% of members “go silent” each year, meaning that GWWC has not been able to hear a response from them after two years (doing roughly annual check-ins), with a total number of people ceasing reporting donations (and very likely ceasing keeping the pledge) at 5.8%.

It was encouraging that Table 1 found the median donation of GWWC members to be 10%, showing that the median GWWC member is on track. However, when we bin our EA survey respondents into donating 10% or more or not doing so and exclude students and people earning $10K or under, we find that only 62% of GWWC members reached their pledge in 2015 (see Table 4), only 54% did so in 2016 (see Table 5), and only 52% did so in 2017 (see Table 3).

Are They “Bunching” Donations?

How do we explain that many GWWC members are reporting EA Survey data inconsistent with GWWC pledge keeping? One potential explanation is that perhaps GWWC members are “bunching” their donations, grouping two years of donations into a single year. Luckily, while the 2018 EA Survey only has us look at 2017 donation data, the 2017 EA Survey captured both 2016 and 2015 donation data for the same individuals, letting us look at a snapshot of two years of donation data for the same individual, allowing us to look to see if people meet the pledge over a two-year period instead of one^[4].

We try to capture this in Table 6 and find that when you average individuals’ donations over both 2015 and 2016, 69% of GWWC members (and 17.6% of non-members) are on track to keep the pledge. There could be additional bunching not in the 2015/2016 time horizon of this survey (such as 2014-2015 bunching or 2016-2017 bunching) that might create false negatives. However, looking at data in 2018 did not find any large population of people donating in the region of 20%, which makes bunching uncaptured by Table 6 fairly unlikely.

Is it a Matter of When People Joined?

In the 2018 EA Survey, we captured data directly about when people joined GWWC (whereas in the 2017 survey we only had data about when people joined EA generally, which was also recorded in the 2018 survey). Table 7 shows that 13.64% of those who took the pledge in 2018 were already donating 10% or more in 2017. Going beyond that we can see that around half of those who took the pledge in a year before 2017 were donating 10% or more in 2017, though there are fewer people in the earlier year groups. We can also see a discernible trend of people being more likely to keep their pledge the longer they’ve been involved in GWWC. The results of two Cochran-Armitage trend tests (approximated, and simulations based) are close and lead us to conclude that there is a linear trend in the proportions^[5].

It’s harder to see how this trend holds up in our prior data. For previous years when we did not ask specifically when people joined GWWC, we can try to make up for this by filtering on when people claim they first started self identifying as an EA. Notably, this is not the same as the year they joined GWWC^[6], and they may report being a GWWC member as of the time of the survey, and being an EA before the time of their donations, despite not being a GWWC member at the time of their donations. Thus not donating 10% in that year would not be breaking the pledge, because they had not taken the pledge yet.

Another factor would be that while we filter out people who report being students as of the time of them taking the survey, it’s possible (and, given the age of EAs, quite likely in many cases) that they were a student at the time of the donations. Thus they would only need to meet a lower pledge of 1% of spending money, rather than 10% of income.

Table 8 shows us that, predictably, those joining EA in 2018 have very low pledge adherence as they were overwhelmingly likely not to be GWWC members at the time of their reported donations. This is why we have excluded them from our analyses elsewhere. Beyond that, we see a similar trend as before, and again trend tests lead us to conclude, with a very low risk of being wrong (lower than 1.44%), that there is a linear trend in the proportions.

This trend is likely the case due to a variety of potential factors, including but not limited to:

1. First we can’t rule out mere survivorship bias - people less likely to keep their pledge are also less likely to keep showing up in our survey data.

2. People recruited into EA and GWWC earlier are less likely to be the result of mass media pushes and thus are more likely to have discovered GWWC organically and be more committed. (However, GWWC media pushes have now died down again, which could complicate this trend.) Additionally, early GWWC had a higher barrier to entry (having to mail in a physical form as opposed to join online) which screened for commitment.

3. People who have been in EA longer have also generally have higher earning potential, thus making it easier to keep a 10% pledge.

Looking specifically at that third factor (confounding by age), we look to Table 10 and Table 11, which seems to show a discernible trend of pledge keeping increasing as age increases, and trend tests lead us to conclude that there is a linear trend in the proportions^[7]. However, while statistically significant, it does not hold up to a stricter multiple comparisons check^[8].

Are People Mistaken about Being GWWC Members?

One question is whether people who are not reporting keeping the pledge are just mistaken when they self-report being a GWWC member - maybe they took the smaller “Try Giving” pledge which does not have a 10% commitment or maybe they didn’t pledge at all. To check this, I cross-referenced the list of names I have of GWWC members in the EA Survey with GWWC’s member roster. In the 2018 EA Survey data, I found 190 self-reported GWWC members who joined before 2017 and gave their name^[9] plus enough information to determine if they kept the pledge in 2017. Of these people, 69% of them were keeping the pledge. Of the 59 identifiable non-pledge keepers, 48 (81%) of them were identifiable in the roster. I also selected a random subset of 50 identifiable pledge keepers and found 46 (92%) of them in the roster. Based on this, people being mistaken about whether they are GWWC members is a possible explanation for a small fraction of people missing the pledge but still leaves a substantial fraction unexplained.

What Predicts Keeping the Pledge?

When looking at factors that best predict whether one will keep the GWWC pledge, the most important factor that emerges is income. The EA Survey looked at both individual and household income (e.g., a couple filing taxes jointly). Here, we chose to look at individual income. As seen in Table 12 and Table 13, there was a strong relationship (logistic regression average marginal effect of 11% p<0.0001) between income brackets and pledge adherence. This is also the case when looking at continuous income data where individual income is correlated with giving 10% or more (Point-Biserial r = 0.1480, p<0.03).

While certainly correlated with income, one’s area of employment could also be associated with pledge keeping. As seen in Table 14 and 15, those in Earning-to-Give careers are more likely to keep their pledge than those in direct charity, non-profit work, or research. Additionally, those working in academia and for-profit industries were a lot more likely to keep their pledge than those who were unemployed or worked for the government^[10].

However, this may be the result of some confounding where people in some job categories are more likely to be engaged EAs. Additionally, one would think that those self-identifying as taking an “Earning to Give” career are basically defining themselves by donating money, so they would be more likely to keep the pledge.

Are GWWC Members Who Don’t Meet Their Pledge Still Donating More?

It is important to note that members who are not meeting their pledge are still, on average, donating more than their non-member counterparts. We find that of the non-students who donate less than 10%, those reporting GWWC membership donate 4.5% on average, compared to a 2.4% average of non-GWWC members who are not students and donate less than 10%. This difference is statistically significant (t-test, p < 0.0001).

Conclusion

Based on data across multiple EA Surveys, it does appear that a proportion of GWWC members are not reporting donation data that is consistent with keeping their pledge, and that this proportion is much larger than would’ve been expected. GWWC originally reported in 2014 an annual membership attrition rate of ~4.8%, however we record ~40% of the sampled GWWC membership population as not keeping their pledge in any particular year (not to mention EAs who drop out of the EA Survey altogether^[11]). A variety of attempts to explain away this trend do not appear adequate enough, though we cannot fully rule out some sampling or other methodological issue in calculating this conclusion.

Despite this, GWWC members do donate more than non-GWWC members (though this difference is only statistically significant in 2017 and not in 2015 or 2016 because there is very high variability in donation amounts) and GWWC members donate more as a percentage of income than non-GWWC members (this is statistically significant). Together with provisional data on EA retention and EA growth rate, this could be taken to be a springboard for further discussion and analysis of how people join and leave the effective altruism movement. We hope to continue to explore this in the forthcoming 2019 EA Survey and we’d appreciate your feedback.

Coda

This post is part of the supplementary posts for the EA Survey 2018 Series. The annual EA Survey is a project of Rethink Charity with analysis and commentary from researchers at Rethink Priorities.

This post was written by Peter Hurford and Neil Dullaghan, with graphs by Neil Dullaghan, and analysis by Peter Hurford, Neil Dullaghan, David Moss, and Tee Barnett. Thanks to Julia Wise and Greg Lewis for providing very helpful feedback.

Previous articles in the EA Survey 2018 Series include:

I - Community Demographics & Characteristics

II - Distribution & Analysis Methodology

III - How do people get involved in EA?

IV - Subscribers and Identifiers

V - Donation Data

VI - Cause Selection

VII- Group Membership

VIII- Where People First Hear About EA and Influences on Involvement

IX- Geographic Differences in EA

X- Welcomingness- How Welcoming is EA?

XI - How Long Do EAs Stay in EA?

Prior EA Surveys include:

The 2017 Survey of Effective Altruists

The 2015 Survey of Effective Altruists: Results and Analysis

The 2014 Survey of Effective Altruists: Results and Analysis

If you like work from Rethink Priorities, please consider subscribing to our newsletter. You can see all our work to date here.

---

1. See also “Considerations against pledging donations for the rest of your life”. ↩︎

2. These are people who answered “Yes” when we asked "Have you have taken the Giving What We Can pledge?" ↩︎

3. When comparing these figures, it is more instructive to exclude students, since students have both a significantly lower income and are bound by a lower “1% of spending money” pledge, rather than a 10% pledge. ↩︎

4. We can compare entries across different EA Surveys for those who (a) give an email address and (b) give the same email address for each survey. This is done by comparing a hashed version of the email address for anonymity. However, few people fulfill these two criteria and thus the population for our longitudinal sample is frequently not large enough to be useful. See the provisional data on EA retention for yet more information. ↩︎

5. The Cochran–Armitage test for trend shows a very low risk of being wrong (lower than 4.65%). ↩︎

6. There is a mean lag of 1.07 years between when someone joins EA and when they take the GWWC Pledge, for those people who have actually taken the GWWC Pledge so far. Notably, this lag calculation excludes people who have so far never taken the GWWC pledge but might in the future. ↩︎

7. Cochran-Armitage trend test (Asymptotic p-value) / (Monte Carlo method - Number of simulations = 5000)/Two-tailed test: p<0.038 p<0.035 (however, this fails to meet a stricter multiple comparisons threshold). Test shows a very low risk of being wrong (lower than 3.83%). ↩︎

8. Given the number of hypothesis tests we do in this piece, we may run into problems of multiple comparisons, so it is also helpful to consider Bonferroni correction - that is count the number of hypothesis tests in the analysis and note whether any p-values significant at the p<0.05 level remain significant at the p<0.05/(N of tested hypotheses) level. We ran ~12 such tests, making our adjusted threshold p<0.00455. We admit that this correction method may now err on the side of being too strict and that more balanced correction methods are available in the literature, but we decided not to look into this more as it did not change our endline conclusions. ↩︎

9. Names are available in confidential, non-shared, non-public EA Survey data for those who gave it. Consistent with our privacy policy, we do not provide any individual-level data unless explicit permission is given. This analysis was aggregated and the identities and pledge-keeping status of individuals was kept strictly confidential. ↩︎

10. Also note that while a huge majority of those who reported being homemakers are donating more than 10%, this comes from only 5 individuals who span the individual income spectrum from below $10K to above $1M. ↩︎

11. The data on EA retention shows a ~40% rate of people leaving EA over a five year time span (though this number is very preliminary and can be dramatically different based on the chosen methodology used). Note, however, this number cannot be fairly compared to the rate of people not keeping their pledge, since that is on the basis of looking at one particular year, not a five year window. ↩︎