The CEA Online Team (which runs this Forum) has finalized our OKRs for this new half-quarter, and I've updated the public doc, so I'm reposting the link to the doc here.

Interesting — I feel like I don't have good intuitions for how to compare an hour of coding with an hour of writing. But I believe you're suggesting something like, convert both of these into "hours of Forum engagement" per hour of my work (like, comparing how much engagement my post gets, divided by the hours I spent writing it, to the hours of engagement I add to the Forum for adding a feature, divided by the hours I spent building it).

If I were to do that comparison, I'm guessing that coding looks way better. My best post got ~23 hours of engagement total, and I don't expect to get significantly more than that. One of our best features wrt engagement is adding AI-narrated audio for posts. This gets us very approximately 8 hours of engagement per day, was relatively quick to do, and we expect it will last for years. If it lasts 5 years that would be 14,600 hours of engagement. Even if it took 10x as long to build as the post did to write, building the feature seems clearly better.

Perhaps this is some evidence that I should not spend more time writing on the Forum, though I think there's some value from posting that is not captured by this, so maybe converting everything into dollars is still better if it allows me to account for more factors.

I'm glad that you like the wiki! ^^ I agree that it's a nice way for people in the community to contribute.

I believe no one on the team has focused on the wiki in a while, and I think before we invest time into it we should have a more specific vision for it. But I do like the idea of collaborative wiki editing events, so thanks for the nudge! I'll have a chat with @Toby Tremlett🔹 to see what he thinks. For reference, we do have a Wiki FAQ page, which is a good starting point for people who want to contribute.

About your specific suggestion, thank you for surfacing it and including detailed context — that's quite helpful. I agree that ideally people could contribute to the wiki with lower karma. I'll check if we can lower the minimum at least. Any more substantive changes (like making a "draft" change and getting it approved by someone else) would take more technical work, so I'm not sure when we would prioritize it.

(It looks like your link to a specific quick take did work, but if you think there's a bug then let me know!)

Yeah I agree, it does feel like a thing that should exist, like there's some obvious value to it even though I got some evidence that there was low demand for it on the Forum. I think it would be faster to add to EA.org instead so perhaps we should just add a static page there.

I like that we have a list in the wiki, so that people in the EA community can help us keep the info up-to-date by editing it, but practically speaking people don't spend much time doing that.

Thanks for the suggestion Chris! I'd be really excited for the Forum (or for EA.org) to have a nice page like that, and I think others at CEA agree. We did a quick experiment in the past by adding the "Take action" sidebar link that goes to the Opportunities to take action topic page, and the link got very few clicks. We try not to add clutter to the site without good reason so we removed that link for logged in users (it's still visible for logged out users since they're more likely to get value from it). Since then we've generally deprioritized it. I would like us to pick it back up at some point, though first we'd need to decide where it should live (EA.org or here) and what it should look like, design-wise.

For now, I recommend people make updates to the Opportunities to take action wiki text to help keep it up-to-date! I've done so myself a couple times but I think it would be better as a team effort. :)

Thanks Will! I really appreciate this comment, and I think this is a great suggestion. I buy something in this general direction (the Forum produces a certain amount of value per year, and so contributing on the Forum is worth some value relative to the total). I think my Forum-related posts/comments tend not to be very actionable — like, I would guess a post from me is actually worth less (in value to the world) than a post about a new job opening, and probably worth less than an object-level post with the same amount of karma. But I also think that coordination/information-sharing is a key source of value for the Forum, so I'm convinced enough to dig into this further and value my comms at more than $0. (The value per hour is likely going to look less promising though, since I am pretty slow at writing...)

As a bonus, I think this is a good general pitch for everyone to write more on the Forum! ^^

Some quick responses to Nuño’s article about EA Forum stewardship

I work on the CEA Online Team, which runs the Forum, but I am only speaking for myself. Others on my team may disagree with me. I wrote this relatively quickly so I wouldn’t be surprised if I changed my mind on things upon reflection.

Overall, I really appreciated Nuño’s article, and did not find it to be harsh, overly confrontational, or unpleasant to read. I appreciated the nice messages that he included to me, a person working on the Forum, at the start and end of the piece.

On the design change and addition of features:

People have different aesthetic preferences, and I personally think the current Forum design looks nicer than the 2018 version, plus I think it has better usability in various ways. I like minimalism in some contexts, but I care more about doing good than about making the Forum visually pleasing to me. To that end, I think it is correct for the Forum to have more than just a simple frontpage list of posts plus a “recent discussion” feed (which seems to be the entirety of the 2018 version).

For example, I think adding the “quick takes” and “popular comments” sections to the home page have been really successful. By making quick takes more salient, we’ve encouraged additional discussions on the site (since quick takes are intended for less polished posts). “Popular comments” helps to highlight ongoing discussions in posts that may not be visible elsewhere. I take the fact that LessWrong borrowed these sections for their site as further evidence of their value, and in fact LessWrong has had some impactful discussions happen in their quick takes section. As another example, features like the “Groups directory” and the “People directory” are not available anywhere else online, and I view them as more like “essential infrastructure for the EA community”. I think it’s reasonable for those to live on the EA Forum, where people already gather to talk about EA things and look for some EA resources. 

On the costs of the Forum:

Here, I want to clarify what the actual costs are. I will focus on budgeted costs over true costs for privacy reasons.

• I believe that the $2M/year was the highest ever (overall, budgeted) cost of the CEA Online Team, when we were a team of 8 people, and it was only the case for approximately 6 months (between Jan 2023 and July 2023). We were probably 7 people for another ~6 months total, and less than 7 people for the rest of the time.
• We are currently at 6 people, and our team’s budget for the year is close to $1.6M/year. I joined in late 2021, when the team was 5 people (including me). 6 is a more representative number for how many people have been on the Online Team in the past few years.
• My understanding is that about $960K/year of the budget is staff costs, broadly construed.^[1] The rest are things such as SaaS costs (which we have made significant strides in reducing in the past year).
• Our true staff costs are lower, because multiple people on the Online Team choose to reduce their take-home pay. We budget for their full salaries because CEA needs to be able to pay them that salary if they choose to take it. I think it’s reasonable to allow people to keep this information private, so we do not report the true staff costs.
• The Online Team does not spend all of its time working on the Forum (although it is a majority of our time). The actual time split depends on when other projects arise, either via external partnerships or internal initiatives (such as the big EA.org redesign in 2022). I would broadly estimate that we spent 10% of Online Team staff time on other projects over the past 3 years.
• Therefore, the actual average yearly cost of Forum staff time has been roughly $860K minus voluntary salary reductions.
• In the future, I expect more Online Team staff time to be used for non-Forum projects, so I expect that cost to naturally decrease over time. (We spent roughly 20% of staff time on non-Forum projects in the past 3 months, for example.)

EDIT: I will also note that there are additional overheads to staff that are not captured even in the "broadly construed" figures above. For example, we have a People Ops team whose job is focused on CEA staff, so one could argue that part of the People Ops budget goes towards the Forum team. Part of the rest of the $1.6M/year is also ops overhead related to having staff. You could even argue that some of the SaaS costs are overhead on staff. I didn't want to go line by line and delay posting this quick take, but I'm adding this here to highlight that "staff costs" can vary a lot depending on what you consider part of that cost. I just picked a "staff cost" value that I thought would generally match the definition that readers would have in mind (i.e. the most direct costs to the company, which is mostly salary plus benefits). I would guess that from the CEA perspective (like if we were considering hiring another person on our team), we would take into account other overhead costs to some extent.

[I will skip responding to the moderation section because I have not been involved with that.]

On Forum culture evolution:

I guess I generally think things like, making the Forum more accessible and decreasing barriers to entry, are good. There are many people in the world who can do good via the broad project of effective altruism, and I want the Forum to work for them as much as is reasonable. Of course there are tradeoffs, and we do need to keep the interests of more experienced users in mind and be mindful not to lose parts of the Forum culture that are valuable, but I don’t see why that should mean we stop making the Forum more accessible. In my opinion, new Forum users are some of the most valuable contributors, and it would be to our detriment to ignore them.

On cost-effectiveness:

I think we agree that what matters is marginal cost-effectiveness. I believe that the marginal expense of the Online Team is a software engineer. Our team has done an internal marginal impact analysis to evaluate how much value the marginal engineer produces via the Forum (broadly, by looking at the software engineering work we have done in a year, BOTEC-ing how much counterfactual value it has produced, then attributing an appropriate fraction to the marginal engineer), and the resulting 90% confidence interval was above break even.^[2]

1. ^{^}

   Inclusive of benefits, travel, allowances for equipment, etc. (Not all of which is spent.)

2. ^{^}

   Multiple people within CEA red-teammed it (and there is not full agreement on the current version). The last time I looked at the Squiggle model, the analysis had over 100 factors. That doesn’t prove its accuracy but hopefully it demonstrates that we take this seriously. I hope to publish more about this publicly, but it will take a good chunk of time to extract the private information and write a version that is clear for a public audience. And, as I mentioned in another comment, I actually don’t know how to incorporate the time I spend on writing it up into the marginal impact analysis, so by default I feel like it’s a poor use of my work time (like, by spending time writing a public version, I would actually be decreasing the value of the marginal engineer, though this is slightly complicated by the fact that I’m currently technically in a non-engineering interim role).

(I work on the Forum but I am only speaking for myself.)

To respond to some bits related to the Forum:

In the actual world, the community doesn't really know... why some posts get tagged as ‘community’ on the forum, and therefore effectively suppressed while similar ones stay at the top level

If you're referring to "why" as in, what criteria is used for determining when to tag a post as "Community", that is listed in the Community topic page. If you're referring to "why" as in, how does that judgement happen, this is done by either the post author or a Forum Facilitator (as described here).

In the actual world, the community doesn't really know... why the ‘community’ tag has been made admin-editable-only

We provided a brief explanation in this Forum update post. The gist is that we would like to prevent misuse (i.e. people applying it to posts because they wanted to move them down, or people removing it from posts because they wanted to move them up).

Thank you for flagging your interest in this information! In general we don't publicly post about every small technical change we make on the Forum, as it's hard to know what people are interested in reading about. If you have additional questions about the Forum, please feel free to contact us.

In general, our codebase is open source so you're welcome to look at our PRs descriptions. It's true that those can be sparse sometimes — feel free to comment on the PR if you have questions about it.

we don't have any regular venue for being able to discuss such questions and community-facing CEA policies and metrics with some non-negligible chance of CEA responding - a simple weekly office hours policy could fix this.

If you have questions for the Forum team, you're welcome to contact us at any time. I know that we have not been perfect at responding but we do care about being responsive and do try to improve. You can DM me directly if you don't get a response; I am happy to answer questions about the Forum. I also attend multiple EAG(x) conferences each year and am generally easy to talk to there - I take a shift at the CEA org fair booth (if I am not too busy volunteering), and fill my 1:1 slots with user interviews asking people for feedback on the Forum. I think most people are excited for others to show an interest in their work, and that applies to me as well! :)

> “While we often strive to collaborate and to support people in their engagement with EA, our primary goal is having a positive impact on the world, not satisfying community members (though oftentimes the two are intertwined).”

I think this politicianspeak. If AMF said 'our primary goal is having a positive impact on the world rather than distributing bednets' and used that as a rationale to remove their hyperfocus on bednets, I'm confident a) that they would become much less positive on the world, and b) that Givewell would stop recommending them for that reason. Taking a risk on choosing your focus and core competencies is essential to actually doing something useful - if you later find out that your core competencies aren't that valuable then you can either disband the organisation, or attempt a radical pivot (as Charity Science's founders did on multiple occasions!). 

I personally disagree that it would be better for CEA to have a goal that includes a specific solution to their overarching goal. I think it is often the case that it's better to focus on outcomes rather than specific solutions. In the specific case of the Forum team, having an overarching goal that is about having a positive impact means that we feel free to do work that is unrelated to the Forum if we believe that it will be impactful. This can take the shape of, for example, a month-long technical project for another organization that has no tech team. I think if our goal were more like "have a positive impact by improving the EA Forum" that would be severely limiting.

I also personally disagree that this is "politicianspeak", in the sense that I believe the quoted text is accurate, will help you predict our future actions, and highlights a meaningful distinction. I'll refer back to an example from my other long comment: when we released the big Forum redesign, the feedback from the community was mostly negative, and yet I believe it was the right thing to do from an impact perspective (as it gave the site a better UX for new users). I think there are very few examples of us making a change to the Forum that the community overall disagrees with, but I think it is both more accurate for us to say that "our primary goal is having a positive impact on the world", and better for the world that that is our primary goal (rather than "community satisfaction").

Nice, I like that idea, and I think it would be good to make it easier for writers to understand what demand exists for topics. It reminds me of the What posts would you like someone to write? threads - I'm glad we experimented with those. However, I don't know if they actually led to any valuable outcomes, so I'd like to think more about how much user attention we should aim to put on this (for example, right now I feel hesitant to make a new thread pinned to the frontpage). Perhaps it would be worth experimenting with bounties, although I'm not sure if people would actually offer to pay for posts.

In the meantime, you can feel free to respond to one of the old threads (which will still appear in the "Recent discussion" feed), or my suggestion is to write a quick take about it (the rate of quick takes is currently low enough that you'll get some attention on the frontpage).

Thanks! For that kind of thing, I would suggest posting it as a quick take or a comment in the open thread. :)