Introduction

TL;DR: It went well.  We skimmed the readings with about a 2:1 ratio of reading time to discussion time, which didn’t allow us to understand everything deeply but generated lively discussions.  All participants said they got what they wanted out of the weekend, and we think short, high-intensity reading groups are a promising alternative to multi-week reading groups in some cases.

Over the weekend of June 17-19, 2022, we covered weeks 1-7 of Cambridge EA’s AGI Safety Fundamentals Technical Alignment Curriculum.  We had 3-6 people in the group at a given time, usually five people.  All participants had prior exposure to AI safety as a cause area, with experience ranging from just having completed Harvard EA’s Intro Fellowship to contributing to AI safety research.

The five participants who stayed in the program until the end thought that this experiment was a success.  We’ll describe the program in more detail, summarize participants’ feedback, and share takeaways for future programming.

The Schedule

We read weeks 1-3 on Friday, 4-6 on Saturday, and 7 on Sunday.  We also spent time Sunday completing additional reading, clearing up confusions, and presenting/discussing other topics.  We started each meeting around 9:15am and ended around 3:30pm each day.  We took a lunch break to socialize and discuss each day around 12-1pm.

For each week on the Cambridge curriculum, we used the following schedule:

1. Skim the overview of the week on the Cambridge curriculum.
2. For each reading, if the curriculum suggests 5t minutes for the reading:
   1. Spend t minutes skimming the reading, capturing the core ideas and implications needed to participate in the discussion.  Add more time if the reading is particularly dense.^[1]
   2. Spend 2t + 1 minutes discussing the reading in small groups of 2-4 people.
3. Discuss concepts and ideas from all the week’s readings for ten minutes.
4. Break for five minutes.

We discussed the readings for about twice the time we read them because we expected that most learning from this style of reading group comes from discussion.  First, discussions can quickly clarify confusions and misunderstandings from the readings.  Second, discussions can go beyond the readings and are naturally tailored to participants’ interests.

After completing week 7 on Sunday, we spent 1.75 hours reading anything we found interesting to shore up our knowledge.  Given that it was impossible to explore the readings in-depth while skimming, this time provided people a chance to more deeply understand the concepts they found most interesting.  We also spent time after lunch presenting and discussing what we found and any disagreements or questions we had.

Evaluation

Everyone who stayed through Sunday thought that the program was a success and that they got what they wanted from the weekend.  Most participants agreed upon the following:^[2]

• It was good to spend about twice the time discussing as reading.  One participant disagreed and thought there should be more reading time than we allotted.  Here are three ways to improve the reading experience:
  • Better tailor discussion time to the readings. Basing the amount of time allocated to discussing the reading off the time it takes to complete the reading is a decent, but not ideal, proxy.  Thus, some readings were short but sparked long and interesting discussions, while others were long but didn’t generate as lively discussion.
  • Group the readings by common themes and discuss them together.  Some readings made more sense when grouped with others and our discussion could have been better by reading both of the readings together, then discussing them together after.  For example, “Recursively summarizing books with human feedback” provided a concrete example of factored cognition, which enriched our discussion of factored cognition in Week 5.  Discussing related readings together thus can clear confusions and could decrease the total amount of discussion time needed to understand a given reading.^[3]
  • Explicitly state we’re skimming and suggest how to do that. For some, a default metric for success is “read every word” which doesn’t work when you’re reading for only 3-6 minutes. The participant who believed we should’ve had more time to read agreed that if we were explicit that our goal was to skim, they would’ve had a much better time. Making explicit “for this academic paper, read the abstract and pretty pictures first” would’ve been useful.
• The length of the day and intensity was exhausting but created focus.  Given that they had barely enough time to skim some of the readings, participants needed to be focused throughout the day (except at meals and breaks).  This created a level of intense focus that people enjoyed.  On the other hand, maintaining this level of intensity is tiring, so participants thought that ending by 3:30pm made sense (we originally planned for 6pm but our brains felt like slosh around 3pm).
• The average density of readings varied by week, so it would have made more sense to split the weeks 3/2/2 (Fri/Sat/Sun) rather than 3/3/1.  Because all participants were involved in EA already, they were already somewhat familiar with the first three weeks.  Thus, we needed less time to complete them than the later weeks, which involved more novel and technical content. 
• This level of intensity was only possible because people had already been exposed to the basic ideas of AGI risk.  However, we also thought that such a program could work for people without prior exposure but with a strong technical background.  We covered content quickly enough that it would have been impossible to understand most concepts without some other foundation or familiarity with the terms or arguments.
• Skimming produced a good balance between exploration and exploitation. Skimming helps quickly explore ideas, marking the ones you already understand (or aren’t interesting) and marking ones to dig into more detail later. One possible failure mode is feeling the need to read every single word in order to truly understand the thing, so you don’t start at all because you don’t have time to commit a full e.g. 30 minutes to get it. Another failure mode is spending lots of time on things you already get, when we’re kind of in a time crunch here.^[4]
• It would be cool to learn about more researchers’ agendas.  The AGISF curriculum covers some researchers’ agendas (notably Chris Olah’s interpretability work), but some participants were curious to learn more about the details of MIRI’s work, Redwood’s work, etc.  Diving into researchers’ agendas could be a good replacement for week 8.

Probably the most important lesson and improvement is to be familiar with your curriculum. If we had been more familiar with the readings and goals of each week, we could have better structured the time and discussions.

Concerns

The major concern we have is that participants will not retain the knowledge they learned from the intensive program.  Cramming material is a poor method to learn things for the long run.  We encouraged participants to apply the knowledge from the intensive or to review/recall it in the future, but our exhortation doesn’t guarantee that they will.

Implications

Short, intensive reading programs are a promising form of EA programming.  We’ll focus our discussion on college programming, although much of it probably generalizes.  From my (Andrew’s) experience running Harvard EA’s Intro Fellowship, Harvard students are flakey and might be afraid to commit to weeks-long programming.  Weekend-long reading programs might mitigate this problem because students have a better sense of their time commitments over week-long or two-weeks-long windows.  One exciting way to structure such programming is as a retreat.  Given that from our experience it’s only possible to maintain the level of focus we have for 5-6 hours, it could be fun to spend the remaining time doing social or other retreat activities.

There is another question of what sort of content would be best suited for this style of reading group.

• Our experience suggests that AGISF is well suited for this format.  More generally, we suspect that intense reading groups for existing EAs could be fruitful.  Existing EAs are usually value-aligned, familiar with the basics of many cause areas, and at universities could benefit from the additional programming.  In particular, we believe that AI safety university groups should consider running AGISF as a weekend program for people already exposed to EA (probably at the level of an intro fellowship) as a quick onboarding.^[5]
• We are ambivalent about the prospects of these intensives for EA newcomers.  Some ideas take a while to sink in, so running a full intro fellowship over a weekend could cause some whiplash among participants.  On the other hand, retreats are an exciting way to quickly learn about EA with fun social activities.  At the very least, we think it’s especially important to outline clear next steps for participants, so that their enthusiasm from the intensive translates into continued involvement with EA.  With traditional reading groups, this might be less of a concern since participants are more likely to feel part of EA after 8 weeks of learning than after just a few days.

Finally, there should be at least one person who’s experienced with the material.  Without having an experienced person to correct basic misconceptions, minor misconceptions or confusions could quickly compound, leaving the whole experience unsatisfactory.

Thank you to Nikola Jurkovic and the participants of the intensive for feedback and to the EA Infrastructure Fund for funding.

1. ^{^}

   Examples: “Risks from Learned Optimisation: Deceptive alignment,” the inverse reinforcement learning readings, “Zoom In,” and Christiano’s posts on IDA/Debate.

2. ^{^}

   We solicited these at the end of the program.  You can find the raw notes here.

3. ^{^}

   One obvious way to group the readings is to complete all the readings for a given week at once, then discuss them together (as in a seminar).  Participants, however, reported that they preferred the format of discussing immediately after having completed the readings rather than grouping them all together.

4. ^{^}

   Additionally, we're curious about people’s skimming habits here for LW/EAF.

5. ^{^}

   Indeed, something like this was one participant’s motivation for joining the intensive.