This is a linkpost for https://openai.com/blog/superalignment-fast-grants
From their website:
We are offering $100K–$2M grants for academic labs, nonprofits, and individual researchers. For graduate students, we are sponsoring a one-year $150K OpenAI Superalignment Fellowship: $75K in stipend and $75K in compute and research funding.
Things they're interested in funding:
With these grants, we are particularly interested in funding the following research directions:
- Weak-to-strong generalization: Humans will be weak supervisors relative to superhuman models. Can we understand and control how strong models generalize from weak supervision?
- Interpretability: How can we understand model internals? And can we use this to e.g. build an AI lie detector?
- Scalable oversight: How can we use AI systems to assist humans in evaluating the outputs of other AI systems on complex tasks?
- Many other research directions, including but not limited to: honesty, chain-of-thought faithfulness, adversarial robustness, evals and testbeds, and more.
I'd encourage a lot of EA orgs and independent researchers to apply to these. One of the ways it will be necessary for EA to fund itself as it grows and potentially becomes more funding constrained is to get funding from outside the direct EA community.
Point of information: at least half the funding comes from Schmidt futures (not OpenAI), though OpenAI are publicising and administrating it.