TL;DR: Karma overrates “lowest-common-denominator” posts that interest a large fraction of the community, leading to some issues. We list some potential solutions at the bottom.
Please see the disclaimer at the bottom of the post.
Posts that interest everyone — or discussions where everyone has an opinion — tend to get a lot more Forum karma (and attention) than niche posts.
These posts tend to be
- about the EA community
- accessible to everyone, or on topics where everyone has an opinion
Why does this happen?
There are different groups with different niche interests, but an overlapping interest in the EA community:
When a post about the EA community is published, many people might have opinions, and many people feel that they can vote on the post. Most people upvote, so more people voting usually means that a post will get higher karma.
Similarly, if the topic of the post is something that doesn’t require particular expertise to have an opinion about, lots of people feel like they can weigh in. You can think of these as “lowest-common-denominator posts.” This is related to bike-shedding.
This leads to some issues
- This misleads people about what the Forum — and the EA community — cares about
- 10 of the 10 highest karma posts from 2022 were community posts, even though less than ⅓ of total karma went to community posts.
- When someone is trying to evaluate the quality of the Forum, they often go to the list of top posts and evaluate those. This seems like a very reasonable thing to do, but it's actually giving a very skewed picture of what happens on the Forum.
- Because discussions about the community seem to be so highly valued by Forum readers, people might accidentally start to value community-oriented topics more themselves, and drift away from real-world issues
- Imagine an author posting about some issue with RCTs that’s relevant to their work — they’ll get a bit of engagement, some appreciation, and maybe some questions. Then they write a quick post about the font on the Forum — suddenly everyone has an opinion and they get loads of karma. Unconsciously, they might view this as an indicator that the community values the second post more than the first. If this happens repeatedly or they see this happening, they might shift towards that view themselves if they defer even a bit to the community’s view.
- Now imagine this happening on the scale of the thousands of people who use the Forum; these small updates add up.
- This directs even more attention to community-oriented, low-barrier topics, and away from niche topics and topics that are more complex, which might be more valuable to discuss
- Karma is used for sorting the Frontpage: higher-rated posts stay on the Frontpage for longer. This is useful, as it tends to hide the most irrelevant posts, and generally boosts higher quality content — more people see the better posts.
- But because posts that hit the middle sections in the Venn diagrams above get more karma, they tend to stick around for longer, which then gets them more karma, etc.
(We didn't try to make this list of issues as exhaustive as possible.)
Note that karma is not perfect even within a much more specific topic — pretty random factors can affect a Forum post’s karma, and readers aren’t always great at voting, but that is a separate issue. (We might write a post about it later.)
Solutions we’re considering or exploring
- Create something like a subforum or separate tab for “community opinion” posts, and filter them out from the Frontpage by default
- Or otherwise move in this direction
- Rename “Top” sorting to more clearly indicate what karma actually measures
- We tend to have a somewhat higher bar for sharing “community” posts in places like the Digest, largely for these reasons
Note: We (Lizka and Ben) think most of our coworkers on the CEA Online Team more-or-less agree with the post, but there are a variety of opinions. We’re currently at 90%+ that we will do something to address this phenomenon, but at much lower confidence about what specific thing we will do.
Thanks for trying to fix this, I agree it's a significant problem with the status quo.
I think you've identified two issues, one of which is significantly easier to fix than the other. For community topics being too prominent on the frontpage, you could (e.g.) impose a cap on how many can appear, and then list them in a "Community" section of the frontpage after general topics.
To the extent you think that the showering of karma on Community posts skews poster/commenter incentives, I think that is much harder to fix. Karma is designed to measure the degree of appeal to the entire audience, and so posts that only can engage a fraction of the audience aren't going to score as well without some "help" from the formula.
There are algorithms that would implement this elegantly. One of these is inspired by the Borda count:[1]
Essentially, this simulates having multiple sub-forums "vote" on the ranking of items on the frontpage. This would help posts in categories that are underrepresented on the frontpage get into a higher rank, so it might help with this. However, it would still disproportionately favor posts that are in multiple top-level categories (e.g. Existential risk and Community) and have a lot of karma, because these posts would be ranked far and away the highest in the non-Community categories. To mitigate this, you could exclude Community posts when ranking for other categories.
Section 4.1, "Borda's method", of "Rank Aggregation Methods for the Web" (Dwork et al. 2001)
Thanks!
FYI this is already how Community works (more or less).
I meant a deflator or multiplier on the calculation of karma itself. For example, a user with a +2 upvote and +6 upvote could count as +1.33/+4 on community topics (a deflator) and/or could count as +3/+9 on undervoted topics (a multiplier). The current system, if I understand correctly, only addresses Frontpage placement.
To the extent people experience receiving karma as reinforcing and/or as feedback on what contributions are valued, one might think the relative distribution of karma on Community vs. non-Community posts creates undesirable incentives that are independent from Frontpage placement. If that is so, tinkering with the Frontpage criteria will only have a limited effect on the incentive problem.
Ah cool, makes sense.
I think this does a good job of describing the problem.
The solution is hard. I've certainly found myself getting sucked into reading EA Forum posts about community topics and felt that my time was used poorly.
On the other hand, some of the posts were really valuable (George's post on big-spending EA and some of the very posts in the aftermath of the FTX crisis spring to mind).
I think that means I want a UX which does allow me to see community posts, but somehow gives posts which have more substantive/subject-matter content more prominence.
I'm really very unclear about exactly what this looks like, which is why this seems hard.
I think one of the more underused features of the forum is the ability to adjust the visibility of posts according to their tags. If everybody used this feature and wanted to see fewer community-tagged posts, they could downweight their visibility. The fact that they don't could mean that most people do want to see these posts, but it could also mean that they're unaware of this feature, haven't had the idea of turning it on for community posts, or just haven't gotten around to it.
I could see you doing either of these:
A subforum also seems fine, it just seems like a potentially more expensive and buggy option that adds complexity to the forum structure.
Thanks for this comment — I agree that tag filtering/following is underused, and we're working on some things that we hope will make it a bit more intuitive and obvious. I like a lot of your suggestions.
We also just deployed something that makes them somewhat more prominent. (For example, blue instead of grey.)
Somewhat ironically, given all the strong upvotes that this is getting, I want to (gently) push back a bit. I would ask those who disagree with me to not downvote me too strongly[1] or at least explain why (disagree vote away though).
I feel like there's a mostly unstated opinion in this post that the posts getting high karma are actually not the highest quality ones, or the ones that the community cares about, or the ones that Forum readers should care about. I think terms like "lowest common denominator" and "bikeshedding" imply this quite heavily. But my fear is that this could be used, perhaps unintentionally, to shut down debates on issues, especially from newer users or those new to EA.
For example, the top-voted comment on the Wytham Abbey post writes the whole community response off as bikeshedding[2]. This comment seems linked to the use of 'bad epistemics' which I feel often assumes a conclusion. See this post, which I mostly agree with, for further discussion on this issue. My emotional reaction is that it all feels a bit like subtweeting[3], unless there are examples given of "this specific post had too much karma and was over-rated by the forum" and "this specific post was undervalued and dropped from the frontpage too quickly".
Furthermore, it's not clear that valuing community-oriented topics is bad in-and-of-itself. "Building effective altruism" is currently number 3 on 80,000 Hour's list of most pressing problems. To the extent that these posts involving the whole community lead to community improvement, then it might be positive. I'm not necessarily arguing that this is true, but I do dispute that the issues discussed in community posts are not 'real-world' issues.
As for the practical solutions proposed, I think 2 & 3 sound fine. Potentially you could add a section similar to 'Recommendations' that might be something like 'CEA Online Choice' or 'Curated' or something - where the Online Team selects high-quality, valuable niche posts that aren't getting viewed and displays them more prominently?
Finally, this reminds me of the discourse around 'Open EA Global' and the CEA response. There seemed to be a misunderstanding between a large part of the community and CEA regarding what the purpose of EAG was, and perhaps this is true of the forum as well? I would welcome thoughts on this or any of the above, or people providing more information and corrections to what I've written
Yeah, I hate how much this sounds like "I know I'll get downvoted for this but..." too smh
Though I was pleased to see that there was strong pushback to this characterisation in the replies
Personal to me. I don't want to imply that 'subtweeting' is actually happening
I can see how all of this can feel related to the discussion about "bad epistemics" or a claim that the community as a whole is overly navel-gazing, etc. Thanks for flagging that you're concerned about this.
To be clear, though, one of the issues here (and use of the term "bike-shedding") is more specific than those broader discussions. I think, given whatever it is that the community cares about (without opining about whether that prioritization is "correct"), the issues described in the post will appear.
Take the example of the Forum itself as a topic that's relevant to building EA and a topic of interest to the EA community.
Within that broad topic, some sub-topics will get more attention than others for reasons that don't track how much the community actually values them (in ~total). Suppose there are two discussions that could (and potentially should) happen: a discussion about the fonts on the site, and a discussion on how to improve fact-checking (or how to improve the Forum experience for newcomers, or how to nurture a culture that welcomes criticism, or something like that). I'd claim that the latter (sub)topic(s) is likely more important to discuss and get right than the former, but, because it's harder, and harder to participate in than a discussion about the font — something everyone interacts with all the time — it might get less attention.
Moreover, posts that are more like "I dislike the font, do you?" will often get more engagement than posts like "the font is bad for people with dyslexia, based on these 5 studies — here are some suggestions and some reasons to doubt the studies," because (likely) fewer people will feel like they can weigh in on the latter kind of post. This is where bike-shedding comes in. I think we can probably do better, but it'll require a bit of testing and tweaking.
[Writing just for myself, not my employer or even my team. I am working on the Forum, and that's probably hard to separate from my views on this topic— but this is a quickly-written comment, not something that I feedback on from the rest of the team, etc.]
Thanks for this!
My take on this is: maybe this is fine actually, because, for precisely the reasons you said, high karma is a sign of high accessibility and high popularity...which is useful for users! If I see that a post has high karma, that's a reliable signal to me that it's both interesting and accessible to the general reader (i.e., to me). If all the highest-rated posts were highly-technical, long treatises on niche topics, even if they were very good quality, then high-karma wouldn't be such a good signal that I would get something out of reading it, if that makes sense? So karma would then be a less good tool at nudging people to read stuff that they might actually enjoy/get something out of.
I do take your point that there can be a snowball effect where high-quality but high-effort-to-read posts can just get completely pushed off the frontpage before anyone has even seen them, while middling community posts just hang around forever, accumulating more karma. That is a problem.
I guess a question underlying all of this is 'what is karma for?' An implication of this post seems to be that karma should reflect quality, or how serious people think the issues are, all things considered. But I think that's too big a responsibility to place on upvotes and downvotes. I don't think the Forum norms say that you should use them this way (they say you can upvote if 'you want others to see it' and 'generally like it', not only if you think it's objectively really important), and even if they did, I don't think it's reasonable to expect that people really would use them that way, because people don't have that much brainspace to devote to "is this post really impactful/serious?" And the majority of EA Forum readers are never going to be qualified to say whether a niche post is high-quality or not, because they don't have the expertise (they can say if they found it interesting, but things can be interesting, accessible, and also wrong).
Amber -- I agree. EA Forum is for the EA community. Community-related posts that interest a high proportion of the community should get plenty of karma.
Treated 'community-related posts' as if they're inferior to more specialist, technical posts seems to kinda miss the whole point of EA Forum -- to be a nexus for discussion of EA topics of broad interest to the EA community.
My ideal self spends most of my EA Forum time reading technical posts about various cause areas, both to stay up to date on the ones I know a lot about and to learn more about the ones I'm less familiar with.
My actual self disproportionately reads Community posts because they take a lot less energy to read.
But I reserve almost all my upvotes for more technical posts to help nudge myself and others toward reading those ones more.
PS - as always, for folks who disagree-voted on this, I'd appreciate seeing why specifically you disagree.
I didn't vote, but your assertion that the "whole point" of the Forum is to discuss "topics of broad interest" seems overstated to me. That's a purpose, but more technical/specialized discussion is also a purpose, and there are legitimate concerns that some of that is being drowned out.
Ok, fair enough. Imagine I said 'a major point' rather than 'the whole point'.
I don't think there really is a regular reader, there are just a bunch of different readers with different interests. If I had to choose between these two scenarios --
I would personally go with (B). Would you prefer (A), or do you think quality wouldn't be compromised in that way, i.e. I'm presenting a false choice? (By "quality" I mean some combination of useful/interesting takeaways, good reasoning transparency, engages with previous work, treats an important subject matter, etc.)
I think this is a false choice, because I don't think the top karma posts are usually mediocre. I think high karma is a good proxy for high quality, but low karma isn't a good proxy for poor quality, because some low karma posts are (as OP said) good, but too technical or niche for general readership, or perhaps just not many people have seen it. In other words, I think there are lots of false negatives with karma but few false positives (is that metaphor at all clarifying, lol).
I do think it's a shame if good non-community-drama posts never even get seen; on going onto the Forum, I'd love to see a front page featuring articles on a range of topics.
"Mediocre" was too strong -- I should've written "high quality" in scenario A versus "excellent quality" in scenario B.
I mostly agree, but I think there's some tendency for some of the best, most front-page-worthy posts to get stuck at "quite a lot of karma, but not the most" due to some combination of being (necessarily) long, specialized, technical, rigorous, difficult and/or dealing with a non-topical subject matter.
I agree!
Thanks for this comment, Amber!
I'll try to engage with the other things that you said, but I just want to clarify a specific claim first. You write:
I actually do not believe this. I think the primary/key point of karma is ordering the Frontpage & providing a signal of what to read (and ordering other pages, like when you're exploring posts on a given topic). We don't need to use only karma for ordering the Frontpage — and I really wish that more people used topic filters to customize their Frontpages, etc. — but I do think that's a really important function of karma. This means that karma needs to reflect usefulness-of-reading-something to a certain extent. This post is about correcting one type of issue that arises given this use.
Note that we also correct in other ways. The Frontpage isn't just a list of posts from all time sorted by (inflation-adjusted) karma, largely because people find it useful to read newer content (although not always), we have topic tags, etc.
So I don't directly care about whether a post that's 1000x more net useful than another post has 1000x (or even simply more) karma; I just want people to see the posts that will be most useful for them to engage with. (I think some people care quite a bit about karma correlating strongly with the impact of posts, and don't think this is unreasonable as a desire, but I personally don’t think it’s that important. I do think there are other purposes to karma, like being a feedback mechanism to the authors, a sign of appreciation, etc.)
[Writing just for myself, not my employer or even my team. I am working on the Forum, and that's probably hard to separate from my views on this topic— but this is a quickly-written comment, not something that I feedback on from the rest of the team, etc.]
Interestingly this very post is probably an example of what you describe - a post that everyone in EA can find interest in, have an opinion on, and weigh in!
I'm pretty in favor of this. I think it's also really weird for newcomers seeing the EA Forum for the first time to immediately be exposed to all the community discussion at the expense of object-level issues. It seems like a weird community experience which is best handled by a subforum.
I do think, though, that Community posts should not be banished from the Frontpage altogether. From the newcomer perspective, I think there's value to having some threads on the Frontpage that are more accessible to a broad and/or less experienced audience. Moreover, if someone is visiting for the first time soon after something like the Bostrom statement or FTX situation, there is a decent chance they are specifically looking for that content and may be confused if it is more difficult to find.
There may, however, be some subset of community topics that should stay off the Frontpage.
Thanks for these flags about the newcomer experience, both. I agree that these are important considerations.
[Writing just for myself, not my employer or even my team. I am working on the Forum, and that's probably hard to separate from my views on this topic— but this is a quickly-written comment, not something that I feedback on from the rest of the team, etc.]
I love that you have text describing the images, making this post more accessible to people that use screen readers. It makes me happy to see that accessibility was a considering for you in writing this post.
Another angle on this (I think this is implied by the OP but didn't quite state outright?)
All the community-norm posts are an input into effective altruism. The gritty technical posts are an output. If you sit around having really good community norms, but you never push forward the frontier of human knowledge relevant to optimizing the world, I think you're not really succeeding at effective altruism.
It is possible that frontier-of-human-knowledge posts should be paid for with money rather than karma, since karma just isn't well suited for rewarding it. But, yeah it seems like it distorts the onboarding experience of what people learn to do on the forum.
Worth pointing out a potential benefit of this imbalance:
Most work in 3 of EA's main cause areas, development, animal welfare and pandemic preparedness, takes place outside of EA, and it may be good for object level discussions in these areas to take place in places other than the EA Forum, to benefit from external expertise and intellectual diversity.
This is probably true for engineered pandemics and AI safety too but to a lesser extent because a high proportion of the work in these areas is done by EAs.
I think it is overall a good thing for the EA Forum to focus on the community, and for the community to act as a co-ordinating forum for people working in different high impact cause areas. I think it's better for an object level discussion on development, for example, to take place somewhere where feedback can be obtained from non-EA development economists, over somewhere like the EA Forum, where a lot of feedback will be from students, animal welfare activists, AI researchers, etc.
I really like this post and would support a change in the direction stated. But when this post goes high karma, there's at least a smidgen of irony :D.
If this post goes high karma, maybe it's a little ironic. But it also means a big net reduction in karma misallocated to community posts, and it will probably more than compensate for this one additional community post getting a ton of karma.
I was just making a wee joke, completely agree with you!
What about the ability to "pass vote" ie say this is neutral. Then you could look at the % of upvotes out of all votes. Feels like that would be more accurate.
Or just look at the ratio of karma to views/reads. A high karma-to-view ratio suggests a good post with a boring title which deserves more visibility.
It looks like Hacker News uses the comment-to-score ratio for flame-war detection.
We were recently asked about the posts we found most valuable over the course of 2022. I wonder what a machine learning algorithm tasked with predicting "most valuable" status from a few simple features like karma-to-view ratio or upvote/downvote ratio would find. (Presumably, the majority of posts were not marked as "most valuable", so you'd need a solution to the class imbalance problem -- I suggest increasing the weight of posts marked as "most valuable" in the loss function, to reflect the fact that false negatives are costly. Also, you might want to Bayes-adjust your features / have a prior that needs to be overcome, to avoid over-updating on the first few data points which come in regarding a new post.)
Maybe an upvote of a post could instead become an upvote of each (post, tag) tuple – e.g., (
dDudLPHv7AgPLrzef,Building effective altruism), (dDudLPHv7AgPLrzef,Community), (dDudLPHv7AgPLrzef,Software engineering), (dDudLPHv7AgPLrzef,Public interest technology), etc. The final score of the post could then be calculated as (say) median of the quantiles of the scores of the (post, tag) tuples of the post.This would make it hard for a post to reach any but a low quantile among Community-tagged posts but should make it easy for posts to reach a high quantile among niche posts.
It could introduce a bit of an incentive for people to mistag their posts with obscure tags and leave out common tags, but using the median should make this effect relatively mild. Plus anyone can add the common tags. One could also exclude tags with < 10 posts from the calculation, so some other suitable threshold.
I haven’t tested this, so it might not work at all!
Maybe we should have a forum ranking algorithm hackathon 😅
Sure! If folks want to pick this up at the upcoming Hackathon, that'd be stellar.
I would be in favor of karma given in community posts not counting towards a user's overall karma, but I expect it to be at the very least technically annoying to implement (and I'm not that strongly in favor of it).
A downside of both this approach and other ways of separating "community" and "object-level" posts is that it might require some tricky decisions about which "community building" posts count as "community".
I think this gets a bit complicated if e.g. someone makes a really good critique of an existing meta org or something - I think that person should get karma for doing good work and improving our meta efforts (e.g. the post red-teaming CEA was very high quality and took a lot of effort, george rosenfeld's post about spending was similarly well written / made good points). I think it would make sense if the karma were down-weighted or something, but not having it count at all seems bad to me.
(disclaimer: i write / read a lot of meta / community things and find them valuable so might be a bit biased here)
I agree that it would be a downside, but not as big as the implementation, and IMHO the change would still be net positive in the current situation (and maybe things tagged Criticism and Red Teaming Contest, or "building effective altruism" would still count)
In any case, excited to see if the new tab test improves things
FWIW I (as a software developer unaffiliated with the forum) don't expect the technical implementation of this to be difficult, at least not more than, say, the agreevote split.
(Not taking a position on whether the change would be desirable.)
When I first started using the forum in 2019, "Community" was a separate tab, and posts could be on either Frontpage or Community. Why was this changed?
Back in 2019 it was impossible to do "soft-filtering" (aka tag weighting). So it was an all or nothing approach.
It seemed a natural fit for community to use the tagging system instead of re-using the 'meta' flag which (because LessWrong used the 'meta' flag to mean something else) was an absolute mess engineering-wise. Then, once it was a tag, we could totally continue to do the thing where it was a separate page and set the tag to hidden. However, now that we had the option to use tag-weighting, Aaron and I decided to go with that instead of a completely separate page.
This makes sense. Upvotes are fundamentally anonymous, and we have no idea what kinds of people are upvoting what things. I'm pretty surprised at how mathematically obvious and explanatory your findings are in hindsight, and yet it never occurred to me or anyone else until now.
I'd like to add that, just like how auctions tend to be won by bidders who got carried away and accidentally bid more than what the object was worth to them, it makes sense to think that 80% of the upvotes could potentially be coming from 20% of the forum readers, and some of those people might spend a little too much time getting invested into the forum instead of feeling obligated to go to events and connect with lots of people and see what they're spending their time working on.
I hope that sharing papers and getting feedback still works well or even better with the new solution, e.g. I'm really glad I chanced across Akhil's research and can now share it with all sorts of people I meet in my line of work, even though my own priority is AI and AGI policy and I would never have encountered it if not for the forum.
May I upvote this post, or would that be counter productive? 🫣
This subthread here, with heavy involvement by Kelsey Piper, exemplifies a lot of things wrong with the forum in one place. It is a good note that is relevant for this post.
https://forum.effectivealtruism.org/posts/JCyX29F77Jak5gbwq/ea-sexual-harassment-and-abuse?commentId=cDnhvShDFhPfFBEpJ
Kelsey Piper is making a valid point that it's wrong, evil and bad to try to police individual conduct and impugn her valid choices, but everything else about the thread is cursed:
Yeah I agree this is a problem. Thanks for looking at it.
What if there were a way to say "I know about this and it's good". That seems the core issue here. Even if all the forecasters say "this is worth reading" that forecasting post still won't get as much karma as a community post that 1/3rd of people think is worth reading.
This shortform from two years ago might be of interest.
If this is an issue at all, I don't think this is an issue with karma, I think it's an issue with mass communication in general.
You argue that this brings attention to posts in a way that does not correspond to importance. For example, you talk about it misrepresenting the quality of the forum, and leading authors to disengage from real world issues. To this I reply - who are you to decide which posts are actually more important?
There are actually very few posts on the forum dealing with real world issues - they are mostly those looking at cause areas and interventions in global health and in policy, plus maybe some biosecurity org updates. The rest, are about the community and its priorities (even those about AI for example, because they deal with how to manage work by community members, and only deal with hypothetical reality). I don't think it can be said that among those there's inherently more value in a post about a specific cause area than in a meta post.