Hide table of contents

A review of criticisms and an alternative estimate based on the thermodynamic approach

This is a Draft Amnesty Week draft. It may not be polished, up to my usual standards, fully thought through, or fully fact-checked. 
This is a Forum post that I wouldn't have posted without the nudge of Draft Amnesty Week (kudos!). I'd love to see comments and criticisms that take the ideas forward, but it's unlikely that I will spend much more time on this project. 

The following is just the executive summary. The full (draft) report is available here

This report is the work of Janvi Ahuja and Victoria Schmidt as part of the Epoch FRI Mentorship Programme 2023. We worked on for ~10 hours a week for two months. Tegan McCaslin mentored the project, and Rose Hadshar and Angelina Li provided significant feedback and advice as our peer reviewers. 

The Forecasting TAI timelines with biological anchors report produces an estimate of the compute needed to develop a transformative model using 2020 architectures and algorithms. It uses six different biological frameworks to estimate the compute needed to develop a transformative model, one of which is an evolution-based framework. The evolution anchor estimates the amount of computation done by all animals throughout evolution, from the earliest animals with neurons to modern-day humans. In this report, we look into criticisms of the evolution anchor and summarise their effect sizes. We then expand on one such criticism and discuss some of our own criticisms with the biological anchors framework as a whole. 

This report may be useful to you if you: 

  • Are interested in biological anchors and defer to the report to determine your AI timelines (and put some weight on the evolutionary anchor). In this case, I would recommend reading the executive summary and reading further on areas of interest. Note, Cotra has posted an update to her original draft and has been interviewed more recently on her timelines.
  • Are interested in the upper bound estimate of the biological anchors report and want to investigate the most conservative anchor, or otherwise particularly interested in evolutionary anchor. In this case, I might recommend reading the whole report.

Executive summary

Motivation statement

  • The biological anchors report has influenced many views on when TAI will be developed. A survey by Clarke and McCaffary (March 2023) found it was the second most cited source deferred to on TAI timelines (where the first is “inside view”). 
  • At the outset of the fellowship, our goal was to expand on Nuño Sempere’s criticism pertaining to the cost of simulating the environment, but we found this difficult to make traction on (see footnote here for more information).
  • Instead, we decided to collate all the criticisms on the evolutionary anchor and their effect sizes.
  • We also decided to expand on one of the alternative approaches to the evolutionary anchor.

What we did

  • Summarised critiques of the evolution anchor 
  • Proposed a best guess for an upper bound based on the thermodynamic approach 
  • Proposed reasons we think you should be sceptical of the evolution anchor framework and our results
  • Suggested how this might affect your TAI timelines

Summarised critiques of the evolution anchor 

Critique and approach to incorporate this into the evolution anchorExpected effect sizeUpdated evolution anchor Reference
Original estimateN/A

1E41 FLOP

Ajeya Cotra

Environment simulation: 

Add costs of simulating an environment and coupling architectures with that environment

Upwards: not quantified

N/A

Jennifer Lin

Environment simulation:

Add environmental simulation cost to the original estimate

+5E27 - ≥4E29 FLOP 

1E41 FLOP

Nuno Sempere

Environment simulation:

Simulate whole Earth, molecular simulation

1E60 FLOP 


 

1E60 FLOP

meanderingmoose

Environment simulation:

Simulate whole Earth, thermodynamic approach

1E45 FLOP 

1E45 FLOP

Ege Erdil
Anthropic critiques 

+up to 6 OOM 

1E41 - 1E47 FLOP

Ege Erdil
Paradigm shift

possibly +>>30 years

N/A

Jennifer Lin


 

Drastically shortened timelines; not quantified here

Discard biological anchors completely

Elizier Yudkowsky
Missing architecture search space

Upwards; not quantified

N/A

Jennifer Lin
Evolutionary algorithms are inefficient

Downwards; not quantified

Decrease the weight of the evolution anchor to 3%

Marius Hobbhahn


 

Proposed a best guess for an upper bound based on the thermodynamic approach 

  • The thermodynamic approach estimates the total amount of energy received by the earth from the Sun and converts this into FLOP. Erdil used the Landauer principle for this conversion, which is the theoretical lower limit of energy consumption of computation. As we expect that most energy was not converted as efficiently as the theoretical lower limit we propose two alternatives: 
    • Using the conversion rate it takes for the brain to convert joules into FLOP
    • Using the conversion rate it takes for the human body to convert joules into FLOP
  • As we can expect the average energy-to-information processor to be less efficient than the human brain or body, we expect this is still a conservative upper limit. Our model in the form of a Google sheet is available here.
  • These approaches result in upper bound estimates which are 4-6 OOMs smaller than the Landauer’s principle approach. 
  • In addition, the upper bounds for both of these estimates are lower than Cotra’s estimate (1E41), at 2E40 for the caloric approach and 6E40 for the brain energy consumption approach. This provides a more informative estimate for the evolution anchor, narrowing down the range to 2E40 FLOP as an upper bound.

Proposed reasons we think you should be sceptical of the evolution anchor framework and our results

  • We expand on fundamental issues with the evolution anchor and how it is derived in Cotra’s report. These include:
    • Weighing FLOP estimates against developments in compute capacity
    • Noting that bounding parameter estimates is difficult and sometimes arbitrary
    • Noting that some of the parameter ranges vary by many orders of magnitude
    • Noting that FLOP conversion to intelligence is abstract and weird

Suggested how this might affect your TAI timelines

  • Finally, we provide an overview of what our work could mean for TAI timeline estimates. This is shown below: 
    • If you: 
      • Believe that Cotra’s model and framework for calculating TAI is reasonable
      • Believe that the thermodynamic estimate for the evolution anchor is better than the brain computation method Cotra uses, and 
      • Believe that our best guess proposal for an improvement upon the thermodynamic estimate is better than the original Landauer approach 
        • You might change your best guess for a FLOP estimate for an upper bound to be 1.79E40 FLOP instead of 1e41 FLOP. 
    • If you:
      • Believe one of the other criticisms/adjustments noted in section two is legitimate, you might 
        • Use the naive estimated effect sizes to update your estimate of FLOP needed to develop TAI
        • Choose to investigate it further and update accordingly
        • Consider Nuño Sempere’s propagation of beliefs given additional uncertainty surrounding the evolution anchor
    • If you:
      • Believe any of the major reasons to be sceptical listed above you might
        • Downweight on the evolution anchor and the entire biological anchors report 
        • Consider other ways to assess AI timelines 
        • Consider Nuño Sempere’s propagation of beliefs given additional uncertainty surrounding the evolution anchor



 

13

0
0

Reactions

0
0

More posts like this

Comments1
Sorted by Click to highlight new comments since:

Thanks for writing this! I agree that bioanchors is still worth engaging with and revisiting given how important it has been and is.

I like the overall approach of trying to quantify how much different criticism would update the 1e41 estimate. I don't feel well-placed to comment on the thermodynamic approach part, but if it works roughly as you outline this seems like an important robustness check for the evolution anchor.

I left a bunch of more minor comments in the report.

More from Janvi
Curated and popular this week
Relevant opportunities