Overview of introductory resources in AI Governance

This post was created as part of the Supervised Program for Alignment Research, Spring 2024. This work would not have happened without the encouragement and accountability of my supervisor, Peter Gebauer.

Introduction

The AI Governance ecosystem is large and difficult to apprehend. There are tons of content, relevant organizations, introductory resources, newsletters and more. As a newcomer to this field, I found it hard to navigate the ecosystem and find the information I needed. I discovered lots of resources purely by chance, months after they could have been useful for me.

What I felt was missing was introductory resources that would not only introduce the “type of work” that is AI Governance, but also direct me towards the resources I needed at different times.

Desiderata: The perfect entry point to the ecosystem would allow me, no matter my background and my intentions, to find the resources I need.

The technical alignment ecosystem on the other hand seems far more organized, in no small part thanks to the Alignment Ecosystem Development team, which created tons of introductory resources, indexes, and other variations on the theme “list of links to useful stuff”. Since I discovered AI alignment two years ago, the resources AED created have helped me navigate the ecosystem and find opportunities I would have missed. I expect similar resources to also be valuable for AI governance.

I decided to investigate thoroughly what were the various introductory resources to AI Governance. Maybe the resources actually existed, and I just did not know where to find them? I compiled my findings below, to help newcomers find those resources faster, and hopefully to motivate others to fill in the gaps where resources are lacking. Hopefully, someone will get motivated to build the ultimate entry point to AI Governance!

Index of AI governance introductory resources

There are various kinds of resources which could be labeled as an “introduction to AI Governance”. My main criteria was whether a specific resource introduced an area of AI Governance, or allowed discovering more parts of the AI Governance ecosystem. I categorize below every resource I found, and give my recommendations for when to use each of those.

Disclaimer: I’m coming at AI Governance from a catastrophic risk perspective, so by AI Governance resources, I mean AI governance resources which could help someone like me who wants to reduce catastrophic risks due to AI.

Learning what AI Governance is

The following resources give a general overview of what AI Governance is. As they all target different audiences, I indicated when to use each specific resource.

• AISF Governance course by BlueDot Impact: Best for someone with a policy/technical AI background who wants to dive deep into AI governance or someone who has already looked at more intro resources like the 80k guide.
• The 80,000 Hours career guide to AI governance and policy: Best for EAs and other impact driven individuals who are considering whether to work in AI governance.
• The Governance Section of the CAIS AI Safety textbook: Best for technically inclined people who want a full overview of the AI Safety field, including learning about AI Governance
• The Center for AI Safety AI Risk introduction: Best for a quick introduction to AI risks and which governance interventions are possible.
• The AI Governance page on aisafety.info: Useful for introducing laypeople to AI Governance. However, it is incomplete, so I would suggest directing people to another resource as well.

Overviews of AI Governance research

Those resources give a broad overview of the various research directions in AI governance.

• A Map to Navigate AI Governance: This post divides AI governance into 17 different activities, and links to some of the actors involved.
• Policy Papers — AISST: Broad overview of AI Policy papers, categorized by research direction.

Generalist collections of resources

AKA “big lists of links”. Those resources are lists of links to various kinds of other resources, serving mainly as indexes. As there is no canonical resource in this category, they all have significant overlap and differences.

• AI governance and policy - Career review from 80,000 Hours: The section “Learn more” links to various other resources from 80,000 Hours and beyond. Seems best if you want a broad overview of what a career in the space might look like.
• AI policy | Emerging Technology Policy Careers: A collection of resources for people who enter US AI Policy.
• “What are some helpful AI policy resources?” on aisafety.info: A few resources of varied types, some of which were new to me and quite interesting.
• The AI Governance posts on the EA Forum and those on LessWrong cover a vast array of topics. Seems best if you want a broad overview of the discussions that have shaped the x-risk focused AI Governance community.

Those resources are mostly linked to each other, so all of them seem like reasonable starting points to explore the breadth of resources in AI Governance.

Thematic collections of resources

AKA “lists of every X”. Those resources focus on listing all the instances of something related to AI governance, be it organizations, training programs, research directions and more. I expect that some of them are not sufficiently well known for the value they provide.

Some of those were initially focused on technical AI safety and expanded to AI governance. They sometimes necessitate some filtering to find those relevant to AI governance.

• List of projects that seem impactful for AI Governance: Comprehensive list of useful projects to do in AI governance, sorted and categorized.
• https://aigov.world (Announcement): List of every organization working on AI governance, including forecasting, regulation, policymaking, research and advocacy.
• https://aisafetyideas.com: List of project and research ideas in AI safety, which can be filtered for AI governance.
• AI Safety Training: List of training programs in AI Safety. Some are relevant to AI Governance
• AI Safety Events: List of events related to AI Safety. Some are relevant to AI Governance, but there is no possibility to filter for them.
• Alignment Ecosystem Development: List of projects to improve the AI Safety ecosystem and who are potentially looking for volunteers. Some are relevant to AI Governance, but there is no filter.
• Communities – AISafety.com: List of online communities more or less focused on AI Safety. No filter for AI Governance.
• “What is everyone working on in AI governance?” on aisafety.info: List of organizations working fully or in part on AI Governance.

Newsletters

There are a myriad of newsletters relevant to AI Governance. Newsletters are probably the best aggregators of AI Governance content, but they’re not as searchable as living lists.

• AI Safety Newsletter: The Center for AI Safety newsletter. It gives general updates on AI Safety including major AI Governance news.
• Import AI: A weekly newsletter from Jack Clark, co-founder of Anthropic, where he analyzes recent developments of AI including governance.
• Newsletters | Center for Security and Emerging Technology: A monthly newsletter on artificial intelligence, emerging technology and security policy.
• AI Policy Weekly: The Center for AI Policy's weekly newsletter.
• Newsletters - AI Standards Hub: Monthly newsletter on AI standards and regulation
• The EU AI Act Newsletter: Newsletter describing updates to the creation and implementation of the EU AI Act.
• My favorite: Don't Worry About the Vase, Zvi Mowshowitz weekly AI roundup contains basically every possible news about AI and is fun to read.

Lists of people

A brief note on the history of the project: Initially, the project’s goal was to make a list/map/visualization of what every person in AI Governance is working on. After discussing with different people working in AI policy, we concluded that creating such a public resource would potentially be negative. I’m not sure how positive it is to share existing ones, but included them for completeness.

The following resources are indexes of people who voluntarily added themselves. They can be filtered to find people linked to AI Governance.

• Profiles Directory - EA Hub: EA affiliated index. Can be filtered for interest in AI policy, but not for experience in the field. After a spot check, it seems like most profiles are outdated.
• The European Network for AI Safety: Index of Europeans interested in working on AI Safety. Can be filtered for interest in AI policy, but not for experience in the field. It’s a more recent resource, so the profiles are more likely to be up-to-date.

A better entry point to the AI Governance ecosystem is possible

Creating this post made me discover multiple resources which I could have used if I had known about them earlier. I expect that making it easier for the right person to find the right information at the right time could have a large impact and be tractable.

Desiderata: The perfect entry point to the ecosystem would allow me, no matter my background and my intentions, to find the resources I need.

This could either be a new resource or an extension of an existing one. I expect that improving an existing resource would be faster and require lower maintenance.

My suggestion would be to improve the AI Governance section of aisafety.info.

Pros:

• Its branching structure and follow-up questions guide the reader towards the information they need. This seems especially well-suited to an entry point used by a variety of people.
• It already has answers to lots of questions people might think to ask and which are hard to find elsewhere.
• The wiki structure, where every page can be edited easily, makes it more likely that mistakes will be fixed and new information added.
• The platform is a project supported by multiple stakeholders, like Rob Miles and AED. It seems unlikely to go unmaintained soon.

Cons

• The AI Governance article seems less maintained (e.g. broken links, outdated list of orgs). This may mean that the team does not have the capacity to maintain a comprehensive AI Governance section.
• The platform hosts articles on both technical AI safety and AI governance., This could make the platform have an overwhelming amount of information, which would confuse rather than help newcomers.
• The maintainers of the website have views of AI risks that differ from other parts of the AI Governance ecosystem.

Conclusion

I hope this list will help some of you find resources you did not know existed, and inspire someone to improve the status quo.

Feel free to suggest additional resources I might have missed!