Quick takes
I sometimes say, in a provocative/hyperbolic sense, that the concept of "neglectedness" has been a disaster for EA. I do think the concept is significantly over-used (ironically, it's not neglected!), and people should just look directly at the importance and tractability of a cause at current margins.
Maybe neglectedness useful as a heuristic for scanning thousands of potential cause areas. But ultimately, it's just a heuristic for tractability: how many resources are going towards something is evidence about whether additional resources are likely to be i...
But neglectedness as a heuristic is very good precisely for narrowing down what you think the good opportunity is. Every neglected field is a subset of a non-neglected field. So pointing out that great grants have come in some subset of a non neglected field doesn't tell us anything.
To be specific, it's really important that EA identifies the area within that neglected field where resources aren't flowing, to minimize funging risk. Imagine that AI safety polling had not been neglected and that in fact there were tons of think tanks who planned to do AI saf...
I wish more work focused on digital minds really focused on answering the following questions, rather than merely investigating how plausible it is that digital minds similar to current day AI's could be sentient:
-
What does good sets of scenarios for post-AGI governance need to look like to create good/avoid terrible (or whatever normative focus we want) futures, assuming digital minds are the dominant moral patients going into the future 1a) How does this differ dependent on what sorts of things can be digital minds eg whether sentient AIs are likely to
You're sort of right on the first point, and I've definitely counted that work in my views on the area. I generally prefer to refer to it as 'making sure the future goes well for non-humans' - but I've had that misinterpreted as just focused on animals. I
I think for me the fact that the minds will be non-human, and probably digital, matter a lot. Firstly, I think arguments for longtermism probably don't work if the future is mostly just humans. Secondly, the fact that these beings are digital minds, and maybe digital minds very different to us, means a lot...
Anyone else get a pig butchering scam attempt lately via DM on the forun?
I just got the following message
> Happy day to you, I am [X] i saw your profile today and i like it very much,which makes me to write to you to let you know that i am interested in you,therefore i will like you to write me back so that i will tell you further about myself and send you also my picture for you to know me physically.
[EMAIL]
I reported the user on their profile and opened a support request but just FYI
Single examples almost never provides overwhelming evidence. They can provide strong evidence, but not overwhelming.
Imagine someone arguing the following:
1. You make a superficially compelling argument for invading Iraq
2. A similar argument, if you squint, can be used to support invading Vietnam
3. It was wrong to invade Vietnam
4. Therefore, your argument can be ignored, and it provides ~0 evidence for the invasion of Iraq.
In my opinion, 1-4 is not reasonable. I think it's just not a good line of reasoning. Regardless of whether you'...
1-4 is only unreasonable because you've written a strawman version of 4. Here is a version that makes total sense:
1. You make a superficially compelling argument for invading Iraq
2. A similar argument, if you squint, can be used to support invading Vietnam
3. This argument for invading vietnam was wrong because it made mistakes X, Y, and Z
4. Your argument for invading Iraq also makes mistakes X, Y and Z
5. Therefore, your argument is also wrong.
Steps 1-3 are not strictly necessary here, but they add supporting evidence to the claims.
As far as I c...
So long and thanks for all the fish.
I am deactivating my account.[1] My unfortunate best guess is that at this point there is little point and at least a bit of harm caused by me commenting more on the EA Forum. I am sad to leave behind so much that I have helped build and create, and even sadder to see my own actions indirectly contribute to much harm.
I think many people on the forum are great, and at many points in time this forum was one of the best places for thinking and talking and learning about many of the world's most important top...
After nearly 7 years, I intend to soon step down as Executive Director of CEEALAR, founded by me as the EA Hotel in 2018. I will remain a Trustee, but take more of a back seat role. This is in order to focus more of my efforts on slowing down/pausing/stoping AGI/ASI, which for some time now I've thought of as being the most important, neglected and urgent cause.
We are hiring for my replacement. Please apply if you think you'd be good in the role! Or send on to others you'd like to see in the role. I'm hoping that we find someone who is highly passionate ab...
I haven't visited CEELAR and I don't know how impactful it has been, but one thing I've always admired about you via your work on this project is your grit and agency. When you thought it was a good idea back in 2018, you went ahead and bought the place. When you needed funding, you asked and wrote a lot about what was needed. You clearly care a lot about this project, and that really shows. I hope your successor will too.
I'm reminded of Lizka's Invisible Impact post. It's easy to spot flaws in projects that actually materialise but hard/impossible to crit...
Instead of "Goodharting", I like the potential names "Positive Alignment" and "Negative Alignment."
"Positive Alignment" means that the motivated party changes their actions in ways the incentive creator likes. "Negative Alignment" means the opposite.
Whenever there are incentives offered to certain people/agents, there are likely to be cases of both Positive Alignment and Negative Alignment. The net effect will likely be either positive or negative.
"Goodharting" is fairly vague and typically just refers to just the "Negative Alignment" portion.&n...
I think the term "goodharting" is great. All you have to do is look up goodharts law to understand what is talked about: the AI is optimising for the metric you evaluated it on, rather than the thing you actually want it to do.
Your suggestions would rob this term of the specific technical meaning, which makes thing much vaguer and harder to talk about.
I imagine that scientists will soon have the ability to be unusually transparent and provide incredibly low rates of fraud/bias, using AI. (This assumes strong AI progress in the next 5-20 years)
- AI auditors could track everything (starting with some key things) done for an experiment, then flag if there was significant evidence of deception / stats gaming / etc. For example, maybe a scientist has an AI screen-recording their screen whenever it's on, but able to preserve necessary privacy and throw out the irrelevant data.
- AI auditors could review any experi
This context is useful, thanks.
Looking back, I think this part of my first comment was poorly worded:
> I imagine that scientists will soon have the ability to be unusually transparent and provide incredibly low rates of fraud/bias, using AI.
I meant
> I imagine that scientists will [soon have the ability to] be unusually transparent and provide incredibly low rates of fraud/bias], using AI.
So it's not that this will lead to low rates of fraud/bias, but that AI will help enable that for scientists willing to go along with it - but at the same time...
If antinatal advocacy was effective, wouldn't it make sense to pursue on animal welfare grounds? Aren't most new humans extremely net negative?
I have a 3YO so hold fire!
- Most new humans will likely consume hundreds (thousands?) of factory farmed animals over their lifetime, creating a substantial negative impact that might outweigh the positive contributions of that human life
- Probably of far less consequence, the environmental footprint of each new human also indirectly harms wild animals through habitat destruction, pollution, and climate change (TBH I am being very speculative on this point).
As AI improves, there's a window for people to get involved and make changes regarding AI alignment and policy.
The window arguably starts small, then widens as it becomes clearer what to do.
But at some point it gets too close to TAI, I expect that the window narrows. The key decisions get made by a smaller and smaller group of people, and these people have less ability get help from others, given the quickening pace of things.
For example, at T minus 1 month, there might ultimately be a group of 10 people with key decision-making authority on the most power...
Mini EA Forum Update
We've updated the user menu in the site header! 🎉 I'm really excited, since I think it looks way better and is much easier to use.
We've pulled out all the "New ___" items to a submenu, except for "New question" which you can still do from the "New post" page (it's still a tab there, as is linkpost). And you can see your quick takes via your profile page. See more discussion in the relevant PR.
Let us know what you think! 😊
Bonus: we've also added Bluesky to the list of profile links, feel free to add yours!
If we could have LLM agents that could inspect other software applications (including LLM agents) and make strong claims about them, that could open up a bunch of neat possibilities.
- There could be assurances that apps won't share/store information.
- There could be assurances that apps won't be controlled by any actor.
- There could be assurances that apps can't be changed in certain ways (eventually).
I assume that all of this should provide most of the benefits people ascribe to blockchain benefits, but without the costs of being on the blockchain.
Some neat opt...
Reading the Emergent Misalignment paper and comments on the associated Twitter thread has helped me clarify the distinction[1] between what companies call "aligned" vs "jailbroken" models.
"Aligned" in the sense that AI companies like DeepMind, Anthropic and OpenAI mean it = aligned to the purposes of the AI company that made the model. Or as Eliezer puts it, "corporate alignment." For example, a user may want the model to help edit racist text or the press release of an asteroid impact startup but this may go against the desired morals and/or co...
That sounds like [Cooperative AI](https://www.cooperativeai.com/post/new-report-multi-agent-risks-from-advanced-ai)
https://www.cooperativeai.com/post/new-report-multi-agent-risks-from-advanced-ai
In addition to wreaking havoc with USAID, the rule of law, whatever little had been started in Washington about AI safety, etc., the US government has, as you all know, decided to go after trans people. I'm neither trans nor an American, but I think it's really not nice of them to do that, and I'd like to do something about it, if I can.
To some extent, of course, it's the inner deontologist within me speaking here: trans people are relatively few, arguably in less immediate danger than African children dying of AIDS, and the main reason why I feel an urge ...
Thinking about the idea of an "Evaluation Consent Policy" for charitable projects.
For example, for a certain charitable project I produce, I'd explicitly consent to allow anyone online, including friends and enemies, to candidly review it to their heart's content. They're free to use methods like LLMs to do this.
Such a policy can give limited consent. For example:
- You can't break laws when doing this evaluation
- You can't lie/cheat/steal to get information for this evaluation
- Consent is only provided for under 3 years
- Consent is only provided starting in
A quick annoucement that Magnify mentee applications are now open!
Magnify mentee applications are currently open!
We would love to hear from you if you are a woman, non-binary person, or trans person of any gender who is enthusiastic about pursuing a high-impact career using evidence-based approaches. Please apply here by the 18th March.
Past mentees have been most successful when they have a clear sense of what they would like to achieve through the 6-month mentorship program. We look to match pairings based on the needs and availabil...
Does anyone want to develop a version of Talos/Tarbell/AIM/Horizon but for comms roles?
Last year, we at EA Netherlands spent considerable time developing such a programme idea in collaboration with someone with significant experience in science and technology communications. Our funding application wasn't successful but I think the idea still has legs, especially given the results of the MCF talent needs survey. If you're interested, let me know and perhaps we can hand over what we developed and you can give it another go.
The USDA secretary released a strategy yesterday on lowering egg prices. Explained originally as a WSJ opinion (paywall). Summarized here without the paywall.
Five points to the strategy:
- $500 million for a biosecurity program to limit transmission of avian flu
- $400 million to farmers to recover after an outbreak
- $100 million for vaccines
- Look to ease regulations, especially overriding California Proposition 12 that banned the sale of eggs from caged hens.
- Look to allow temporary imports of eggs
Key concerns:
- Enacting a key-goal of the EATS act to overri
If you've liked my writing in the past, I wanted to share that I've started a Substack: https://peterwildeford.substack.com/
Ever wanted a top forecaster to help you navigate the news? Want to know the latest in AI? I'm doing all that in my Substack -- forecast-driven analysis about AI, national security, innovation, and emerging technology!
Something that I personally would find super valuable is to see you work through a forecasting problem "live" (in text). Take an AI question that you would like to forecast, and then describe how you actually go about making that forecast. The information you seek out, how you analyze it, and especially how you make it quantitative. That would
- make the forecast process more transparent for someone who wanted to apply skepticism to your bottom line
- help me "compare notes", ie work through the same forecasting question that you pose, come to a conclusion, a
... (read more)