We’re the Forum team — we run this platform. You’re free to ask us anything (just post it as a comment)![1] We’ll start answering questions tomorrow, Thursday, July 6, and will continue answering on Friday (and possibly through the weekend).
We might not be able to get to all the questions, but we’ll try to get to as many as we can. (Vote for the questions you’re most interested in!)
About us
Tl;dr: we run the EA Forum. More below!
➡️ We’re hiring for a Content Specialist! (Deadline extended: apply by July 26)
We’re looking for someone to join the team full-time as a Content Specialist, to work with me (Lizka), probably taking on most or all of my current responsibilities in the next six months.
We just updated the deadline (moved from July 19 to July 26) to make sure that we have time to reach people who we think should apply. (The deadline on the website is outdated.) We had pre-announced this role here.
➡️ Apply here.
If you’re hired, your core responsibilities would include:
- Encouraging engagement with important and interesting content
- Improving the epistemics, safety, and trust levels on the Forum
- Helping the Online Team (or the EA network, or CEA) to prepare for future challenges and opportunities (e.g. by seeking out some information, helping shape the team’s strategy, etc).
This would likely involve taking over running the moderation team, seeking out, curating, and organizing content on the Forum, and running or working on things like the monthly EA Newsletter (59K subscribers) and the weekly EA Forum Digest (10K subscribers)— and other things!
You can ask questions about the role — but not only.
More on who we are and what we do
Who we are
This AMA is with the “Forum Team”[2] — the people listed here (and hopefully another person soon). The team includes:
- Engineers (Sarah, Will, Ollie, JP (who is also the interim head of the team — this was previously Ben)): building and maintaining the site
- The content side (currently just me — Lizka): running the moderation/facilitation teams, the EA Forum Digest (and the EA Newsletter), and assorted other content-related responsibilities
- The Product side (Agnes, Sharang): noticing opportunities and issues, and working with engineers to develop new features and solutions to users’ problems
Different people are able to commit different amounts of time to the AMA, but the following people will definitely answer some questions if there are questions about their work:
- JP (interim head of the team)
- Sarah and Will (engineers)
- Sharang (the Product Manager)
- Agnes (Product designer)
- Lizka (me, content side)
Our work
Some of our current & recent projects include:
- Features (you can see our past feature updates here)
- Building better recommendations for further reading
- Improving what’s currently the Library page
- Working with Type III Audio on AI narrations
- Putting “shortform” on the Frontpage and renaming it “Quick Takes”
- Making the “new post” page easier to use
- Exploring adding the ability to “react” to comments
- Improvements that are less visible: speeding up the site, improving our analytics database, etc.
- In general:
- Our codebase is open source and shared with LessWrong — you can submit issues here (although that list is not super well maintained)
- You can also share your thoughts and feedback in the Forum feature suggestion thread
- Strategy and team
- Developing better metrics to track the Forum’s impact and health
- Exploring the space of communication/coordination channels in AI safety and AI governance, and trying to understand if there are gaps/bottlenecks/issues that we can address with improved or new platforms/tools
- Hiring for a Content Specialist — see above! — and moderators (that round is near its final stage)
- Continuing our ongoing projects, like:
- Forum support, making sure that people can use their accounts, etc.[3]
- Non-technical support/improvements: Forum facilitation and moderation
- The weekly EA Forum Digest (and the monthly EA Newsletter)
- Helping with other sites and projects (and exploring new ones), including effectivealtruism.org
We’ve been trying to share more about moderation changes, how the team works and thinks about the Forum, etc., but we haven’t managed to make as much time for this as some of us have hoped — I hope this AMA helps!
- ^
You can explore other AMAs here.
- ^
There are more groups of people who work on the Forum. You could picture this in terms of the following diagram:
The lines sometimes get blurry:
- Facilitators do some things that look more like moderation
- Moderators do, too: they help new users, help with the wiki
- The Online Team supports moderation (e.g. by building tools, helping with investigations, etc.) — and JP and I (Lizka) are also active moderators, while Dane Magaway (the executive assistant on the Online Team) also helps a lot with moderation
But the diagram is roughly accurate, except for a glaring omission — the Forum wouldn’t happen without people who post on it and people who read it. So a more accurate picture might be a bit more like:
- ^
When something breaks, who ya gonna call? Bug-busters!
What is the theory of change for the EA forum?
How much is spent, total, on the EA forum each year?
Related: What else would you do with twice the resources? What would you forego if you had only half of them? What do you think the effect on impact of either funding effect would be?
(The motivation here is that I think transparency around funding tradeoffs is particularly important for meta work, both optically and epistemically. It also promotes a culture in which inappropriate spending, like fancy trips to the Bahamas, is less likely to occur.)
The Online team overall spends ~$2M per year. 75% of that is on “people costs” for our 8 full-time employees, all-in.[1] Is that all on the Forum? Kinda. Our main focus right now, and for the past 6 months, has been the core Forum product, due to, basically, the community spike here. However, during normal times, including starting ~now, we spend 30-60% of our effort on expansionary projects, such as this major project from last year. I would model us like a tech startup, where most of our money is going to “R&D” that should pay off in future impact.
Which then nicely answers part of Jason’s question. If we had less money, then, given that our costs are so staff-driven, we would need to have fewer staff, and then we would be investing less in future impact. As for what we would do if we had double the funding, we are currently more bottlenecked by trying to remain a two-pizza team. It’s possible that we would make a hire on the margin, but we wouldn’t accept double the funding.
Including taxes, travel, team retreats, professional development budgets, etc.
It seems that we're spending 2 million a year on a glorified subreddit. What's the donors case for this being a good use of funds? Isn't the forum good enough as is? Some reddit mods handle ~100x the content, with 100x more traffic, for 0% of the cost.
What have you accomplished that justifies the spend on the forum to date? Couldn't 99% of this have been accomplished by forking lesswrong and making minor tweaks?
It’s worth clarifying that the Online team’s budget was much less than $2M budget before 2022, and a lot of the development of the Forum happened between 2018 and 2021 with one developer (JP). The budget and team grew in the last 12 months with FTX funding and significant growth in the EA movement, and now we’re evaluating what portion of the $2M budget to put towards continuing to develop the Forum versus new projects we’re exploring. It's also worth noting that a portion of the $2M budget goes towards the content team (Lizka), and moderation and support contractors, who I think are super valuable but not the bulk of our budget, so I'll instead focus on the product/engineering team and the platform in the next part of my response.
When I think of how the Forum compares to other platforms, I see some major tradeoffs. Compared to social media sites like reddit, twitter, and facebook groups, the EA Forum encourages long form content and high quality discussion much better, is a more focused space (with no ads or non-EA content), and indexes and organizes content better. Compared to individual or group blogs, the Forum is much more open (anyone can post), and prioritizes discussion better (commenting features are more hidden on many blogs).
Put together, I think these mean that the EA Forum is a more attractive space for EAs, and ultimately the network of users is a key factor in determining the value of the site.
This isn’t to say that the Forum can claim 100% counterfactual value for every interaction that happens in this space (compared to a world where it didn’t exist and we had a subreddit), but I do think the Forum has been valuable. I wish we could measure this value in a really clean way without too much effort (and we have near term plans to do more work here), but on the other hand I think it’s important we avoid the trap of focusing too much on measurement and evaluation for a public good that has fairly diffuse impact.
Hope that was helpful.
This isn't a convincing less of analysis to me, as these two things can both be true at the same time:
i.e., you don't seem to be thinking on the margin.
While I'm sympathetic to the view that ~$2M is too much to spend, the quality of moderation here is much higher than in any open-access, high-volume space I am aware of on Reddit.[1] So I don't think it is helpful to compare the mod workload here with what "reddit mods handle" (usually large mod teams on the major subreddits).
Curating higher-quality content by moderating with an iron fist is easier, but would destroy a significant portion of the Forum's value in my opinion.
I think some subreddits do a good job of moderating to create a culture which is different from the default reddit culture, e.g. /r/askhistorians. See this post for an example, where there are a bunch of comments deleted, including one answer which didn't cite enough sources. Maybe this is what you have in mind when you refer to "moderating with an iron fist" though, which you mention might be destructive!
Seems like the challenge with reddit moderation is that users are travelling between subreddits all the time, and most have low quality/effort discussion norms. Whereas on the Forum, the userbase is more siloed, which I guess would make good quality moderation easier.
I think benchmarking at reddit moderation is probably the wrong benchmark. Firstly, because the tail risk of unpaid moderation is really bad (e.g. the base rate of moderator driven meltdowns in big subreddits is really high). Secondly, I just don't think we should underpay people in EA because (a) it creates financial barriers to entry to EA that have long-term effects (e.g. publishing unpaid internships have made the wider labour market for journalism terrible) (b) it'll create huge amounts of more informal barriers that mean we lean on more informal relationships in EA even more.
There is already an effective altruism subreddit. I think we should post in both places and see if the difference is worth the cost
What is the average salary of the forum team?
~$120,000 (sans benefits). It varies greatly by role and location. You can get a sense for roughly what a given role might pay by looking at our job postings. As mentioned elsethread, these salaries are aimed at not being huge sacrifices for tech workers living in expensive american cities, while also not being egregiously luxurious in lower salary places like Oxford. I imagine some engineers might look at that number and think it’s low compared to their expectations, and some non-engineer Brits might think it’s quite high. I encourage you to look at the job postings for the ranges.
The salary of the content specialist is here.
It sounds like the 1.5 million does not include $ on the forum team or facilitators - what does the total spending/average salary look like if you include these groups?
Ah, thanks for finding that, I posted my reply before seeing this comment.
By Forum team do you mean non-CEA moderators? We have volunteers on our team, which throws off our actual spend, but I've budgeted as if we have to pay for all of our moderation contracting, and have budgeted $20,000/yr. We pay moderators $40/hr, and facilitators $30/hr.
Cool to see that AMA, big fan of that team!
Apart from LW, what are some other forums you take inspiration from? Curious about features you implemented inspired by those forums (or you still want to implement), culture, etc
What makes you think we take inspiration from LessWrong? 😛
The biggest influence is Reddit. Did you know that we once were a fork of reddit? The basic primitives of posts that get comments and karma are taken from reddit via that history. We still look over their work for inspiration sometimes.
I have also looked at Medium, Substack, Quora and Facebook (for Events).
Just to add a few: when I think about Forum features/structure/culture, I often think of other sites that have similar-ish functions that I've appreciated and/or used, including:
Hey :)
Would working as a content specialist require being available for meetings similarly as working on the online team? I'm asking because these hours generally don't work for people who live in Israel
We do require timezone overlap. We’re a remote team, but we are not an asynchronous team. From our role advertisement:
We’re targeting Pacific time through British time, though anyone who can make those hours is welcome to apply. Note that while we want people to be available during those times, we won’t slam them chock full of meetings.
How much time/effort/resources are spent managing trolls, spam accounts, people looking for Electronic Arts, and similar "totally off base" accounts/posts/comments?
For all the other categories besides trolls, the answer is, “quite small.” We are blessed by having a relatively light troll burden, but there are some situations where trolls can get past a tired moderator or facilitator, and cause us to spend too much time on them. If our systems were better developed, this would be less of an issue, but thankfully the costs have been low enough that we have not needed to invest in “shoot first, ask questions later” policies.
Adding a bit of info:
It was around 806 users in 2021.
What is the future of the EA Forum?
Hi Peter,
It’s hard to ask an agile team without an executive director about the future. I can tease that we’ve been exploring how different users find information on the Forum. Watch this space.[1]
I also hear that we’re about to hire a great Content Specialist who’s going to have a transformative effect on the Forum’s content and culture. Maybe you know them and can tell me who they are?
Longer term, I think it depends on who the ED is and what their vision is. The Forum could be instrumental in their vision. A sketch as to how: The Forum is, according to me, the largest lever we have over EA culture. Or, the Forum is the place where the most EAs gather which EAs are able to write software for, so any future where you’d want an EA-facing software project with a network effect probably would want to start on top of the Forum.
Or perhaps the Online Team will focus on project(s) that are not based on the Forum, in which case, the Forum will continue to be its great self.
And opt into experimental features to see something sooner.
"Maybe you know them and can tell me who they are?"
Indeed, I do: They are me!
I'm sorry I missed this AMA. It's been enlightening; as much for the questions as the responses. I'll be reaching out to Lizka this week (and not just because our last names mean the same thing).
How do you feel retrospectively about the change to lower community engagement on the EA forum? Do you have any numbers about activity/engagement since then?
Just a quick clarification: I don't think this was a "change to lower community engagement." Adding the community section was a change, and it did (probably[1]) lower engagement with community posts, but that wasn't the actual point (which is a distinction I think is worth making, although maybe some would say it's the same). In my view, the point was to avoid suffocating other discussions and to make the Forum feel less on-edge/anxiety-inducing (which we were hearing was at least some people's experience). In case it helps, this outlines our (or at least my) thinking about it.
And before I get into my personal takes, here's a graph of engagement hours on Community posts vs. other content:
and here's a graph of the percent of overall engagement that's spent on Community posts (the blue line is a running 30-day average):
And another graph that might be interesting is this one — percent of karma (orange) given that comes from downvotes (the black line is the total number of votes in a period):
On to how I feel about the change:
I'm pretty happy with it, with some caveats.
I was quite worried about it initially,[2] but eventually got convinced/decided that it was a good thing to try. I'm much more positive about it now, although I still think that we might be able to find a better and more elegant solution at some point.
Some reasons I feel positive about it:
(1) We get a lot of positive feedback on the Community Section, from a broad range of people.
For the initial test, we surveyed people on the Forum (more here), and got the following results:
(45 responses. On a scale of 1-7 where "1" was negative and "7" was positive, 38% of responses put 6, 22% put 7, 13% put 5, 11% put 4, 7% put 3 and 2 (each), and 2% put 1.)
We'd also done some user interviews and gotten quick takes from people (again, a bit more here).
Since then, we've continued hearing from people who really appreciate it. At EAG and in other ~EA spaces, I've been, unprompted, told things like "I'm going on the Forum a lot more now because I see interesting and useful content" and "The Forum seems much better now."
A number of people have also asked for ways to hide the section for themselves entirely.
(2) I feel that the Forum has gotten healthier.
I could try to point you to discussions that, before the change, I'd have expected would blow up in some unpleasant ways, I could pull out the number for how often moderators have to ban or warn someone per engagement hour in different periods, and I can point you to the "percent of karma that comes from downvotes" that I shared above, but I think this is mostly a subjective judgement, and factors in things that others might not agree with me on, like "how much does karma track what I think is more valuable?"
I also find it easier to read what I want to read on the Forum right now. I hadn't expected a huge change in my own experience — I read a lot of Community posts! — but I actually quite like having the section.
(3) I was worried about various downsides, and they haven't seemed to have gotten very bad — although that's my main uncertainty.
My personal approach here is to keep an eye on this, see if people are engaging to the extent that they endorse, and continue looking for better solutions.
I think this is almost certainly true — engagement with community posts went down —but there was also less big news to discuss, and that might have been the actual cause of the decrease.
Initially I was particularly worried that we would miss out on important conversations, and worried about our reasoning (I was worried that we were getting feedback from a heavily biased sample of people). I think I was also worried that there was something "special" about the Forum as it was at the time — I've now articulated this in terms of "we're at relatively ~stable eliquibrium points on some important parameters and structural/external changes can mess with that stability" — that we would mess with by doing something big like this.
Here's a bad baseline sanity check that I did at some point: (1) Take the posts shared in the Digest — which gives me a baseline of ~quality/value-to-readers (according to my own judgement). Note that I don't filter strongly for karma in the Digest; if something gets a lot of karma and I'm not too sure about it, I'd maybe err on the side of including, but I don't really use karma as a big factor. (I might also be more likely to miss low-karma good posts, and there are other reasons to share or not share something in the Digest, but let's put that aside for now.) (2) I compared how much karma different types of posts were getting. The posts that I was putting as ~top 3 in the list order for the Digets had noticeably more karma, on average. Setting those aside, though, Community posts in the Digest got ~100 karma on average, and non-Community posts got ~80 karma on average. (Note that there's a lot of variation, though.)
Regarding "I feel that the Forum has gotten healthier," is that statement in comparison to pre-FTX levels of health?
It doesn't surprise me that a series of scandals, as well as the various pro-reform/anti-reform posts, would trigger a rise in downvote percentage and in warnable events per hour in comparison to a calmer mix of topics. Without more, I would not be inclined to view the 4Q 22 / 1Q 23 trends as strong evidence of a generalized health decline rather than a reflection of the topic mix that needed to be discussed because of events that happened external to the Forum.
This is a good question. I was mostly not thinking of pre-FTX levels when I was writing that part of my response. I'm not sure if I think my claim holds on longer time scales! Things in the community have been calmer recently (outside of the Forum); I don't know what would have happened had we not added the section (i.e on the Forum. whether things would have calmed down by themselves), nor what would happen if we removed the section now (and unfortunately we can't e.g. A/B test this kind of thing, but maybe we should test in some other way at some point).
Aside (not related to what I was talking about in the "Forum has gotten healthier" section above): I agree that 4Q 22 / 1Q 23 aren't representative in many ways, but a key underlying motivation for the Community section is giving posts that have a more niche audience a better chance by separating out posts that tend to get a lot of karma so they're not "fighting for the same spots" (which are determined by ~karma). That phenomenon (and goal) is unaffected by recent events; ie. what we described in Karma overrates some topics; resulting issues and potential solutions should hold at all times. (On a skim, the top-by-karma posts of 2023, 2022, to some extent 2021 are Community posts. Curiously, this seems less true of 2020 but again more true of 2019.)
Wow this is a spectacularly detailed answer thanks!
Do you see most of the expected value of the forum coming from value/year similar to the current value/year but continuing into the future (maintaining what you currently have)? Or is most of the value coming from the possibility of producing much more value per year (going big)?
If you imagine going big as most of expected value, then how do you anticipate going big?
Thanks for this great question. It’s hard to answer this both because our model of how the Forum creates value includes lots of diffuse second and third order effects, as well as because well, predicting the future is hard! My guess is that the biggest factor here is the size of the EA movement. I would guess that the value of the Forum will scale as the EA movement and community scales.
Here’s a summary of how I think about the value of the Forum:
Often when we try to ask individual users how the Forum has been impactful for them, we hear that they enjoy reading and have learned about important ideas or made useful connections on the Forum. EA organizations tell us the Forum helps them establish their brand and ideas and later on recruit funding or employees.
Based on this, my guess is that the network of users that participate in the Forum is a huge factor in its value - a bigger EA movement would mean coordination and education would simply have a bigger audience and more potential for impact.
In terms of going big or maintaining what we have, I think we're probably near to the edge of the impact we can get with tech improvements given the current size of the EA community, though there's disagreement about this. A content specialist could add a lot of value in ways not through the tech, through content curation, special events, etc. That said, we do plan to take big swings that are outside of the core product of the EA forum, as JP mentions elsewhere.
There used to be a sort of best posts on the Forum that you haven't read yet section (although I don't remember what the actual name was). I liked that a lot and I miss it. Is there any chance of bringing that back, or did it perhaps get moved to a different place on the forum that I haven't seen? Or perhaps adding a filter on the All posts page that allows me to filter out posts that I have already read?
Hi Joseph, we removed that section because we could see that it wasn't getting many clicks, and we didn't think the algorithm for selecting the posts was that good[1]. You can still find a page with the remnants of it here though if you want. The list at the bottom there uses the same algorithm as the old Frontpage section, including filtering out things you have already read.
We have started experimenting with better algorithmic recommendations recently, currently these are just at the bottom of posts but we're thinking of adding them back into the Frontpage. We're also thinking about adding a "best of" page (possibly algorithmic, possibly not) à la slatestarcodex, so hopefully your wishes will be answered soon. The filter on the All posts page is a good idea too, we'll consider adding that, it might be tricky to do though for annoying technical reasons.
Because it would always show the same few high karma posts until the user clicked on them, so if there were none that you wanted to click on then the list would become useless
Thanks for taking the time to respond! :)
Is there another forum/discussion board on the internet which is as good or better than the EA forum? If so can you point me there so I can have a quick look? Obviously this is subjective but areas where I think the forum seems better than any other I have seen. I know the question was asked about inspiration, but this question is about quality.
- Quality of back and forth conversation, without personal abuse or other derailing of the argument.
- High level of daily activity
- Percentage of "front page" posts which are well thought through, well written and interesting
- High engagement from thought leaders and big names in the field
Perhaps some subreddit?
Hi there!
The hiring page for the Content Specialist role states that CEA would also consider hiring for a more junior or more senior version of the position; would you be able to elaborate at all about what a more junior version of this role might look like?
The short answer is that both the "default" version of the role and the junior version would be, in large part, crafted around the candidate we want to hire. This role is very flexible; there are some core responsibilities, but those are vague and there could be a lot of ways to approach them.
But here's an attempt at some more general things:
It's hard to pin down the exact qualifications that would be useful for the role. The position is currently framed as a "Specialist" role, which might be more senior than what some are prepared to start as, and we didn't want to rule those people out (and we didn't want them to rule themselves out). A more junior version of the role would probably be less independent (or at least start that way); instead of taking on large projects, the person might work on more specific tasks or co-run projects.
I also think that some people tend to underestimate themselves and how quickly they could grow into some responsibilities, and I don't want them to select themselves out. If someone looks like a really promising candidate, but is less prepared than others to start, I want to be open to hiring them.
My current best guess is that we'll end up with something closer to the default version, but the first round of the application is really short to fill out, so if someone is in doubt because of this uncertainty specifically, I'd just nudge them to apply.
Do you sometimes think of yourself as the potential beginning of "the best place on the internet for high quality conversations/disagreements/truth-seeking"?
Like a social network - but better
?
<3
I think our Forum software (ForumMagnum) is the best in the world — show me one better. It is in some ways a shame then, that I think our impact case is directed mostly through the EA and related communities right now, and my guess is that we should be spending our effort actively trying to expand beyond them.
Question: Are you still coming out with the new reactions?
Comment/Feedback: It seems the AI narration only captures the initial post and not the edited sections. This only plays for 55s until the end of the first set of bullet points and before it says that it was edited to add something.
Comment/Feedback: Are you able to make it so that clicking the top right link icon on comments automatically copies the URL to that comment instead of users needing to navigate to the new page and then copying the URL from the address bar?
Yes we are working on reactions now, and hope to roll something out within a week or two!
Thanks for flagging the issue with AI Narration. We've known that edited posts were a possibility, but hadn't prioritized recreating the narration. My guess is this still doesn't quite come to the top of the priority list since most posts do not change after the first publish, but we're going to be tracking this and prioritizing over time.
Thanks for the suggestion on making the link icon copy the url! Someone actually suggested this just this week, and we've been working on it!
Hi there :) I have a couple of questions about the Content Specialist role:
Are you open to remote applicants from any location with a compatible timezone, or do you strongly prefer remote candidates based in certain countries (UK, US, etc.)? I ask because I'm based in South Africa, and would be hesitant to apply if there's a chance my location may ultimately count against me.
Would work trials be conducted remotely or would top candidates be required to travel? And related to that – how important is flexibility around trial and/or travel dates?
Can you offer any more insights on how you determine salaries and benefits for candidates based outside of the US and UK?
Thanks for taking the time to consider my questions!
Hi!
So I am planning to join EA in the next 15-18 months, what strategy, and preparation tips will you recommend? Please note that I am in DEI (Diversity, Equity and Inclusion) with a focus on Disability and Neurodiversity.
I am also a workshop facilitator and designer of learning spaces, advocacy, and activist movements.
Considering the given info, please suggest actionable steps. Thanks.
Hey! I noticed this is your first comment and wanted to try to explain why you got some negative karma there: I think this might not be the best place for you to ask those questions, since they're the online team rather than introductory-type community builders.
Here are a couple of links that might help you with your question: the "Learn" section on effectivealtruism.org and the EA virtual intro program.
The Open Thread pinned to the top of the frontpage would also be an appropriate place to ask this kind of question.