TL;DR

• Rethink Priorities’ AI Governance & Strategy team works to reduce long-term/extreme AI risks. We’re seeking a Compute Governance Researcher or Research Assistant to tackle questions such as:
  • how hardware security features could be used to facilitate AI governance
  • how recent US export controls will affect compute availability to different actors
  • the details of governance proposals such as ideas 2, 3, and 12 mentioned here
• This role does not require prior governance-related experience.
• Deadline: June 12, end of day in US/EST time zone
• Type of Role: Permanent role, full- or-part-time (min. 20h / week)
• Location: Remote & able to hire in most countries
• Compensation if full-time: $69,000 - $114,000 / year or equivalent in local currency (the amount is calculated using RP’s salary algorithm and is dependent on prior relevant experience and corresponding title level)
• A brief pitch for applying:
  • Important compute-related policy windows for reducing AI risk are open now or likely to open soon.
  • This team's compute governance work is in-demand from decision-makers at various key institutions.
  • The team is able to simply work on what we actually think is most impactful and do so in whatever way we think is most efficient.
  • We actively value and work to support professional development, employee wellbeing & satisfaction, and work-life balance. (Details below.)

About the Position 

We are seeking a Compute Governance Researcher or Research Assistant (RA) to join our AI Governance and Strategy (AIGS) team. This is an opportunity for technically inclined people to contribute to compute governance (see also), and does not require prior governance-related experience. We will determine whether to offer the successful candidate a Researcher or RA role based on the candidate’s prior experience and their performance in our hiring process. (In any case, RAs can potentially get promoted to researchers later on, and Rethink Priorities puts significant emphasis on professional development, such as by allowing staff to dedicate 10% of their work time to that.)

This role is fully remote, and we are able to legally hire in most countries. We welcome applicants from all time zones, although you may be expected to attend meetings during working hours between UTC-8 and UTC+3 time zones. This role is equally open to candidates who are available for either full-time or part-time work, as long as you’re available to work at least 20 hours per week.

About the Team

Our AIGS team tackles a diverse set of questions related to (1) what AI development and deployment scenarios may occur over the next few decades, and (2) how governments, firms, and other actors should prepare for, steer, and respond to various scenarios to reduce long-term/extreme risks. We engage closely with decision-makers (e.g., in labs, foundations, and policy communities) to increase the relevance and impact of our work.

Currently, our team is organized into four main workstreams: China-West relations, compute governance, corporate labs, and US regulation & legislation. Compute governance essentially means governing access to significant, concentrated computing resources. This could be a uniquely feasible way to create guarantees that all of the most powerful AI systems are developed and deployed safely, and to thereby alleviate dangerous race dynamics and risks of both accidents and misuse. This workstream’s current projects include research on how hardware security features could be used to facilitate compute (and thereby AI) governance, and how recent US export controls will affect compute availability to different actors. Future projects will likely include researching the details of governance proposals such as ideas 2, 3, and 12 mentioned here.

Key Responsibilities

If hired as a researcher, your responsibilities would likely include:

• Planning and conducting extended independent research projects
• Collaborating with other team members on projects
• Reviewing others’ research

If hired as a research assistant, your responsibilities would likely include:

• Supporting researchers by conducting various research tasks, including:
  • Searching and writing up answers to various questions related to e.g. AI hardware and cybersecurity
  • Seeking out and reaching out to various experts to find answers to these questions
• Completing short (e.g., less than one month) research projects and write-ups, semi-independently
• Providing support with other research-related tasks

More specific examples of what you might do:

• Help flesh out details of compute governance proposals. 
  • For examples of what we mean by a compute governance proposal, see Shavit (2023).
• Read about relevant technical and security standards, and assess e.g., whether a product meeting a given standard could be used in a particular governance application, and what changes would make it more applicable. 
  • For example, what can and can’t be done with systems that implement confidential computing support. More specifically, how could remotely attested trusted execution environments be used to verify how a given AI system was trained, or what its properties are, without revealing private or proprietary information?

What We Are Looking For

The following attributes are each desired but not essential; we are open to hiring someone who currently lacks some of these attributes, and then adjusting the role around that and/or helping that person develop those attributes. 

Skills and Competencies

• Ability to write clearly, concisely, and with reasoning transparency
• Intellectual curiosity and open-mindedness, including willingness to carefully consider ideas
• Ability to find, read, critically assess, and apply research from various disciplines and on various topics
• Resourcefulness and problem-solving ability 
• Attention to detail, and a commitment to maintaining high quality and accuracy in all research outputs
• Good interpersonal skills and comfort with reaching out to various people outside of Rethink Priorities
• Comfort with remote work (the AIGS team is fully remote, with staff in multiple time zones)
• Ability to prioritize well, meet deadlines, and work productively

Knowledge and Experience

Below is a list of topics about which knowledge would be helpful, in roughly decreasing order of priority. That said, we do not expect any applicant to have extensive knowledge of all or even most of these topics. 

• Computer hardware, particularly data center hardware used for large-scale AI training and inference
• Cybersecurity, especially hardware security
• Distributed computing, especially as applied to large-scale AI systems
• The semiconductor supply chain, especially for high-end chips
• AI and machine learning
• Technical, safety, and security standards and the processes by which they are set
• The US government and policymaking processes, both executive and legislative
• Arguments for existential risks from AI and proposed ways to reduce said risks through governance
• Relevant players in the AI industry
• International relations between – and technology policy in – any of the US, China, South Korea, Taiwan, Japan, and/or the Netherlands

What We Offer

Compensation

• Annual salary between the following ranges for a full-time research assistant or researcher position, prorated for part-time work: 
  • USD: $69,000 - $114,000 pre-tax
  • GBP: £54,000 - £88,000 pre-tax
  • EUR: €61,000 - €101,000 pre-tax
• The exact salary will be based on the candidate’s prior relevant experience and corresponding title level, and calculated using RP’s salary algorithm. To ensure fairness, RP does not negotiate salaries.  
• Compensation is not restricted to the currencies listed above. Payments may be made in different currencies and payment intervals depending on the location of applicants and legal requirement

Other Benefits

• Opportunity to contribute to a fast-growing, high-impact organization — our research is used by key decision-makers who influence the distribution of hundreds of millions of charitable dollars
• Flexible work hours and location
• Comprehensive global benefits package (while they vary by country, we make every effort to ensure that our benefits package is equitable and high-quality for all staff) 
• Generous paid time off leave, including, but not limited to:
  • Unlimited vacation with a minimum of 30 days off per year (including public local holidays, vacation time, and “mandated” 3-weeks total mid- and end-year organization-wide breaks)
  • Unlimited (within reason) personal and sick leave
  • Parental leave - up to 6 months of parental leave during the first 2 years after a child’s birth or adoption, for parents of all genders
• For more details about our benefits, please see Benefit Package for Permanent Roles
• A caring team that values respectful work relations and a healthy work-life balance
• Opportunities to grow/advance your career and engage in professional development
• Low administrative bureaucracy
• We don’t provide snacks but we could mail you a box of Oreos if you want!

Additional Information  

• Extension requests: We will try to accommodate extension requests that are made before the deadline and are up to five (5) days. To ensure fairness to other applicants, we generally cannot accommodate extension requests that are made on or after the application deadline or are longer than 5 days, and cannot accept late submissions.
• Inclusivity and fairness:  RP is committed to finding the best people for our team and to building an inclusive, equitable, and supportive community for you to thrive and do your best work. So please don’t hesitate to apply for a role regardless of your age, gender identity/expression, political identity, personal preferences, physical abilities, veteran status, neurodiversity, or any other background. We provide reasonable accommodations and benefits, including flexible work schedules and locations, mental health coverage in medical benefits (as available), and technology budgets and professional development time that can be used to purchase assistive technology or engage in job coaching. 
• Accessibility: We’re committed to running an inclusive and accessible application process. We warmly invite you to reach out to careers@rethinkpriorities.org with any questions or accessibility requests such as chat box use during interviews or time extension requests for any assessments that impose a time limit.
• Language: Please submit all of your application materials in English, and note that we require professional level English proficiency.
• Travel: Travel is not a requirement for this position. A majority of our staff travel a few times per year for conferences, team and all-staff retreats, and other work-related purposes, and we prefer if staff can travel for at least one retreat per year. But this won’t be taken into account in the hiring process, and we likely can and often do make accommodations such as allowing virtual participation for at least parts of retreats.
• Other: 
  • Visit our Career Opportunities webpage if you’d like to know more about our hiring process, culture, and what working at RP is like.
  • Please do not include a cover letter, photograph, or headshot of yourself, or any personal information that is not relevant to the role for which you’re applying (including marital status, age, identity traits, etc.). 
  • Please do not ask our staff members involved in the hiring process to meet with you – to ensure fairness, we try to minimize such interactions.

About Rethink Priorities

Founded in 2018, Rethink Priorities (RP) is a nonprofit organization that addresses global priorities—important and neglected issues—by researching solutions and strategies, mobilizing resources, and empowering our team and others. RP’s mission is to generate the most significant possible impact for others in the present and the long-term future. 

Our cause areas include animal welfare, global health and development, climate change, artificial intelligence, and other work to safeguard a flourishing long-term future. RP also aims to understand and support the professional communities working on these issues. Each researcher tends to focus on one particular cause area.

Rethink Priorities works as all of the following:

• A consultancy doing commissioned work in response to demands from organizations doing high-impact work 
• A research institute driven by research agendas we set according to our own priorities.
• A think tank aiming to inform public policy to improve the world.
• An accelerator, incubator, and base for entrepreneurial projects.

Some of RP’s recent accomplishments include: 

• Publishing a nine-post sequence on understanding the diffusion of large language models which presents key findings from case studies on the diffusion of eight language models that are similar to GPT-3. 
• Conducting and writing up results from an expert survey on AI strategy, which has informed key decision-makers and been included in reading lists for people entering this field.
• Organizing a well-received summit for 35 leading members of the existential-risk-focused AI strategy and policy field. 
• Producing public and nonpublic reports on various topics, including prospects for AI safety agreements between countries.
• Helping major foundations to answer their questions on climate change solutions, weather forecasting in lower- and middle-income countries, increasing access to medicine, and the effectiveness of prizes and other interventions. 
• Comparing the capacity of different animal species to experience pleasure and pain to help philanthropists decide how to allocate funding.
• Investigating various animal welfare interventions, as well as bringing to light the neglected areas of invertebrate and insect welfare.
• Publishing pieces on nanotechnology and ways to use forecasting to improve the long-term future, as well as supporting those interested in these types of topics.
• Launching a Special Projects Team to incubate promising new initiatives, such as Epoch (a new AI research organization) and Condor Camp (longtermism movement-building in Brazil and Latin America).
• Conducting surveys to better understand the Effective Altruism community

We welcome you to review our database of published work here. 

We’re supported by Open Philanthropy, the Survival and Flourishing Fund, and additional institutional and individual donors. 

Information on applying

To apply, please respond to the prompts in the application form. We ask that you spend no more than one (1) hour preparing your responses to the knowledge and experience questions.

Application Deadline: June 11, 2023, at the end of the day in US/Eastern (EST) time zone. 

Q&A Webinar: You can find the recording of the Q&A webinar held on May 26, Friday here and the chat history here.

Contact: Please email careers@rethinkpriorities.org  if you have any questions.

Resume/CV: Feel free to upload your CV if you want on the application page. But this is optional and will not be used in our evaluation process. We will use CVs only for purposes like later considering whether to refer you to other future roles within RP or at other organizations if you have consented for us to do so.

We invite anyone to apply and will evaluate applications based on anonymized prompt answers, so please ensure they represent your fit for the position well. We aim to select more for revealed knowledge and skills than for experience in itself. We also want to note that significant governance/policy knowledge is not required.

Rethink Priorities is a think tank dedicated to informing decisions made by high-impact organizations and funders across various cause areas. If you are interested in RP’s work, please visit our research database and subscribe to our newsletter.