Effective Altruism Forum web traffic from Google Analytics

vipulnaik

Note: If you are using uBlock you may not be able to see the images, because they have "/google-analytics" in the name and uBlock is a bit ... paranoid ... about blocking file names with that substring. Please disable uBlock for effective-altruism.com or temporarily disable it to see images.

As 2016 comes to a close, I thought it'd be interesting to look at traffic trends on the Effective Altruism Forum (which I'll call EAF for this article) since its launch around September 7, 2014. Some key takeaways follow, after which I discuss more of the data.

Key takeaways

Annual cycle and long-term growth trends (more):

December and January see the highest traffic in the year, as expected considering it is Giving Season (though the relative role of demand for content and supply of content is unclear). The summer months see the lowest traffic. This seasonal fluctuation is more significant than year-over-year growth.
Overall, year-over-year growth in total pageviews has been about 10% per year. It is swamped by other sources of fluctuation (so for a given month it could even be negative, and for some months it could be as high as 30%). From late 2014 to late 2015, there was a significant increase in sessions accompanied by a decrease in pages/session that roughly canceled. In 2016, sessions and pages/session have remained roughly constant.
The year-over-year growth in unique visitors is comparable to that of GiveWell's website, but much less than that of 80,000 Hours. GiveWell and 80,000 Hours are the only two organizations I know of that cater to a similar audience and release web traffic data.

Impact of EA events (more): EA Global and other similar events have negligible traffic impact, whether we look at impact during the event, shortly before or after the event, or long-term before versus after changes.
Post traffic lifecycle (norms and exceptions) (more):

The majority of posts get almost all their traffic in the first few days after they are published. The traffic may come from social media and other referrals, or people discovering the content from the home page.
A few pieces of content get sustained traffic over a long period of time. This could be steady traffic or traffic resulting from multiple traffic spikes. Some of these pieces are generic EA classics, and others are about tangentially related subjects of wider interest. No articles exclusively about EA organizations or personal donation decisions have made it to this list.
The size of the initial traffic to a post correlates far more closely with other indicators of post traction (such as comments and net upvotes on the post) than the long-term traffic to the post. On the other hand, Facebook likes + comments + shares correlate more with long-term traffic than initial traffic, though I have less confidence about that conclusion (due to more limited data).

I close with some speculative thoughts on implications.

Some additional graphs and data (that weren't important enough to make it to the main post) are in the Miscellanea section.

Annual cycle and long-term growth trends

Here's a graph of monthly pageviews in Google Analytics. Note that since the site was launched after a few days of September 2014 were already over, the September 2014 number represents a period shorter than a month.

You can see that December and January have higher traffic than the surrounding months for the end of 2014 end-of-year Giving Season. For the 2015 end-of-year Giving Season, February (so February 2016) actually has slightly higher traffic than January, but you can still see the same qualitative picture: a sharp rise in traffic in December, a decline in January, and a return to normalcy soon. For the 2016 end-of-year Giving Season, we only have December data, and we see a pretty huge rise from November to December.

An year-over-year comparison between EAF's first and second year (September 7, 2014 to September 6, 2015, compared with September 7, 2015 to September 6, 2016) shows a 9.86% increase in pageviews (from 314,789 to 345,832), a 21.53% increase in sessions, and a 26.97% increase in users. Pages/session and average session duration fell.

Looking at some specific months shows how noise overwhelms this slight long-term increase. For instance, in September 2014 (the month of launch, where a few days were missing since those were pre-launch), there were 26,256 pageviews. The corresponding numbers in September 2015 and September 2016 were 30,467 (+16.0% year-over-year) and 26,347 (-14.5% year-over-year). For October, the numbers in 2014, 2015, and 2016 were respectively 25,237, 26,259 (+4.0% year-over-year), and 32,880 (+25.2% year-over-year).

With that said, there is clearer evidence of year-over-year growth for November and December. As a result, the gap between Giving Season and other months has been increasing from 2014 to 2015 to 2016.

Data is very limited but one tentative theory that fits the evidence so far is that the growth of the EA movement has been strongest among readers whose interest in EAF content is high only during Giving Season. Another is that content creators are focusing their content creation and publishing efforts more during Giving Season and less during other months, and this is increasing the traffic difference between Giving Season and other months.

One interesting trend between late 2014 and late 2015: although the increase in number of pageviews was small, sessions increased quite a bit. Pages/session dropped to compensate. This could be because people began using EAF more on their phones (thus increasing the number of sessions and users recorded by GA). It could be because the number of people accessing EAF increased, but the interest and dedication of the early adopters declined.

I also compared this growth rate against that of other websites that cater to similar audiences. I was limited to the ones that have released some public data, namely, 80,000 Hours and GiveWell. Here's the comparison:

80,000 Hours does not release pageview data but it does release an estimate of unique visitors to the site. You can get the numbers here. The number grew 244% year-over-year from 149,164 in 2014 to 513,697 in 2015, and then again by 62% to 834,310 in 2016. The number for 2016 is an underestimate since it only includes data till November, when the blog post was prepared. The corresponding increase in visitors from 2015 to 2016 for the Effective Altruism Forum was 15.6% from 66,384 to 76,712. Thus, 80,000 Hours had more unique visitors and a faster growth rate.
GiveWell's web traffic numbers. The latest numbers released by GiveWell are in this spreadsheet and go up to June 2016. You can also see the PDF report that goes up to the end of 2015 (including January 2016, since GiveWell takes the year as starting in February and ending in January). The report shows an year-over-year increase of 17% from 737,592 in 2014 to 863,883 in 2015. Those are about ten times the numbers we see for EAF, but a similar growth rate.

Impact of EA events

The main story regarding EA events is the non-story: there is no increase in traffic to EAF in or around EA Global or major events. I got a list of events from Effects of major events on EA activity by Eric Yu. His post didn't discuss effects on EAF traffic, although the original post by Peter Hurford that his post answered had asked for that data. So I thought this would be a good set of events to look at:

GWWC pledge drive 2014–2015: a few weeks around 1/1/2015: Traffic was lower in January than December, but it's hard to isolate the effects of the GWWC Pledge Drive from the general Giving Season cycle.
GWWC pledge drive 2015–2016: a few weeks around 1/1/2016: Same comment as for the previous pledge drive.
EA Global 2015: 7/31-8/2, 2015 (SF), 8/14-8/16 (Melbourne), 8/28-8/30 (Oxford): Traffic was lower during the days of the summits than the surrounding days. This could be due to it being a weekend. It could be because the in-person event substituted for website visits and reduced the amount of content being posted. Overall, traffic in August was the same as July, so it doesn't seem like EA Global increased traffic to EAF in the surrounding days. Moreover, most of the pages that were visited during these events were qualitatively similar to the pages that were visited before and after them. The main exception was the August open thread, that got a total of 304 views, most of them in the days right after the event ended. The view count for this was too small to meaningfully affect overall traffic.
Doing Good Better published: 7/28/2015 (around the same time as EA Global): Traffic was higher on this day but the qualitative pattern of posts viewed wasn't different from other days, suggesting that the publication of the book wasn't the driving factor for the traffic increase.
The Most Good You Can Do published: 3/15/2015 (published on Amazon 4/7/2015): Traffic moved up and down around this time and there wasn't a clear effect on overall traffic. The qualitative nature of articles visited also didn't change. Note that this is in contrast with the Wikipedia page for effective altruism, where the publication of the book approximately marked a permanent increase in the traffic level to the article.
GiveWell top charities announced (2015–2016): 11/20/2015 (corrected date from source): Also hard to isolate the effect of this relative to Giving Season changes. Looking at traffic to posts with GiveWell in the title, we see spikes at various points in time related to posts about GiveWell, but these don't correspond to the timing of GiveWell's recommendation release.

Post traffic lifecycle: norms and exceptions

The vast majority of EAF posts get the vast majority of their traffic within a few days of publication, and all the associated engagement activity (post upvotes and comments) also occurs within that time. This time can vary from a day to a little over a week. Typical examples:

President Trump as a Global Catastrophic Risk (12 net upvotes, 15 comments, 89 Facebook likes + comments + shares)
Introducing Effective Altruism Policy Analytics: Request for help and expert consultation (10 net upvotes, 10 comments, 94 Facebook likes + comments + shares)
EA is elitist. Should it stay that way? (11 net upvotes, 34 comments, 2 Facebook likes + comments + shares)

The fact that most posts have short lifespans is part of the explanation for why, despite the cumulative content on EAF being much much more now than it was during launch in September 2014, total pageviews aren't up that much. If the posts had longer lifespans, and continued to get steady trickles of views, these steady trickles would add up to a lot of total pageviews, and that number would increase as the total number of published posts went up. However, since posts typically die quickly after being published, we don't see the steady accumulation of pageviews. So pageviews in a given month are determined more by new content created at the margin rather than the total corpus of content so far.

However, the top viewed items of all time display a different pattern. Like the other posts, they have a spike around the time of initial publication (or initial import into EAF), and much of the engagement activity (upvotes, comments) occurs within that time. However, unlike other posts, the top viewed items continue to get a steady trickle of views later in life, or additional later spikes in traffic, long after active engagement with them (in the form of comments and upvotes) has died down.

Another noteworthy feature of the top posts is that many of them were imported into EAF, or cross-posted to EAF, from other places (such as LessWrong, or the old Effective Altruism blog that was hosted on the same domain).

The situation with Facebook likes + comments + shares seems similar to long-term traffic (and therefore different from comments and upvotes on EAF): posts that continue to get a steady trickle of views also continue to get a steady trickle of Facebook likes, comments, and shares. However, I don't have good time series data on Facebook likes, comments, and shares, so this is just a rough estimate based on what i do have.

Here are the top five posts:

Introduction to Effective Altruism (20 net upvotes, 5 comments, 292 Facebook likes + comments + shares), averaged about 50 pageviews a day from mid-2015 to August 2016, and then dropped to about 10 pageviews a day ((probably due to competition from Introduction to Effective Altruism on effectivealtruism.org).
The Drowning Child and the Expanding Circle (8 net upvotes, 8 comments, 138 Facebook likes + comments + shares) sees traffic in waves, with sudden spikes after some sort of media publicity and then gradual decay over time, till the next spike.
What is Effective Altruism? (7 net upvotes, 7 comments, 16 Facebook likes + comments + shares) had a traffic spike in July 2015, after which it saw 20 pageviews a day for a while. The number has decayed gradually and is now about 5 a day. Search query competition could explain the decline.
Efficient charity: do unto others... (13 net upvotes, 5 comments, 87 Facebook likes + comments + shares) spiked in late January 2016, and has since fluctuated, averaging between 2 and 15 pageviews a day.
Should you donate to the Wikimedia Foundation? (10 net upvotes, 3 comments, 126 Facebook likes + comments + shares) had three spikes: one at the time of initial writing (in late March 2015), and two more during the Giving Seasons of 2015 and 2016, coinciding with the Wikimedia Foundation fundraisers through Wikipedia banners. Disclosure: I wrote this post, and have spent some time understanding its view statistics.

Beyond the top five, the picture becomes more mixed. The sixth and seventh article, namely Philosophical Critiques of Effective Altruism by Prof Jeff McMahan and Cheerfully, had only one spike at time of publication, but do still get a very slow but still steady trickle of traffic. In eighth place is the wide-interest article Six Ways To Get Along With People Who Are Totally Wrong*. This has had three big spikes and a much much smaller steady stream of views.

The most-commented article I could find was Concerns with Intentional Insights, with 39 net upvotes and 182 comments, and 60 Facebook likes + comments + shares. This placed ninth in terms of lifetime pageviews. The traffic to this article had a single, really huge spike at the time of launch, and has since been pretty low.

Another way of trying to gauge the relative importance of steady views to top-performing content versus views of new content is to compare total pageviews against pageviews of the top five articles of all time over time. I have included a graph below with this data. All time, the top five articles account for 52,872 pageviews out of 467,380. In December 2016 (after any of them was published, so it misses the initial peak of all five) the top five accounted for 2,703 out of 28,284 views. Overall, then, the top five of all time account for something like 10% of pageviews in a given month.

Speculative thoughts on implications

The slow increase in total traffic to EAF suggests some tempering of the widespread narrative of rapid EA movement growth. It would be interesting to understand why EAF traffic lags other widely cited indicators of movement growth, such as GWWC pledge sign-ups and GiveWell money moved. A number of plausible theories can be proposed. For instance, it could be that EAF appeals only to a segment of EAs (and EA-penumbra people) and this segment is not represented well in new recruits. On the other hand, it could be that EAF has huge churn, so that even though the EA movement is growing, EAF membership at any given time isn't growing that fast because people leave quickly. It's also possible that the problems are on the supply side (not enough quality content on EAF, or more competition for readers' attention).

More interesting (to me) than the overall traffic trend is the question of what the pageviews and other metrics for specific posts can tell us about the impact of those posts. Content creators get feedback on posts based on initial traction in the form of comments and upvotes, which correlate well with initial pageviews. But for the best-performing posts in the long term, a lot of the pageviews happen over time, and the long-term pageview trend is not as correlated with the initial spike in pageviews as we might think.

One possible implication is that content creators on EAF may be overproducing content that gets a lot of initial traction, at the expense of creating evergreen content with long-term value. I mean this more in the sense that people are reluctant to produce evergreen content because they don't see it getting that much traction, rather than that they are cynically generating content of short-term value to increase their karma. Of course, this implication is far from proven, as it's not clear that long-term pageviews are a good indicator of long-term value. But even if it's not, analogous reasoning might apply for long-term value.

Take a look at the top five posts. Four of them are generic, evergreen content whose relevance declines very slowly over time. The fifth one (on donating to the Wikimedia Foundation) is more specific, but the topic it touches on, namely donations to Wikipedia, is something that a sizable fraction of the Internet-using population comes across during Giving Season. Contrast this with a post like Concerns with Intentional Insights that gets a lot of interest when it is published but does not address a recurring, long-term human need, and therefore dies down quickly. Note that it's still possible (and many have argued) that the Intentional Insights post did have a huge indirect long-term impact not reflected in recurring pageviews, by changing norms around fundraising and accountability that would serve as cautionary lessons for future donors and organizations. My point of using it as an example was mostly to simply highlight it as a case where direct interest (as measured by pageviews) dropped sharply after an initial period. There is also a possibility that the post will see another traffic surge when similar issues crop up in evaluating other organizations, just like the Wikimedia Foundation post has had traffic surges triggered by Wikipedia banner fundraising.

Another implication is around the way we link to older material. A lot of content is high-quality, but fails to get visited after a huge amount of traffic in the first week. Regulars forget about the content, newcomers never hear of it. Better, stronger linking practices to older content, and more high-quality followup posts that help revive older content, could help address this. Discussions and debates advance when people build on top of past material, rather than regenerate or rehash it, so a strong memory of past content, reinforced through extensive linking to it, can help.

Miscellanea

Prior to the Effective Altruism Forum, the same domain name hosted an invite-only blog that had a handful of posts. These ports were ported over to EAF when it started (along with some more seed content). Below is the Google Analytics for the pre-EAF site. You can see it was significantly less than traffic to EAF. Data is available only starting October 2013. The drop in September 2014 coincides with the launch of EAF during the month.

Below, I paste the pageview data for the new Effective Altruism Forum at daily and weekly granularity respectively for the period September 1, 2014 to December 31, 2016, and then the hourly pageview data for December 2016. By looking at this data you can verify that there is a clear daily cycle in traffic, but no weekly cycle.

Data exports

Although I have Read&Analyze access to the EAF's Google Analytics, I don't have the authority to grant others access. You can contact Ryan Carey to give you Read&Analyze access if you are interested.

If there are specific EAF posts that you think are important enough to have their traffic patterns covered in this post, please list them in the comments and I'll add their traffic patterns to the post when I get time.

Effective Altruism Forum
EA Forum