Defense in depth
A recent paper by Owen Cotton-Barratt, Max Daniel and Anders Sandberg proposes a defense in depth approach for handling risks of human extinction.[2] Such risks may be analyzed as involving three successive stages in the unfolding of a terminal event: the origination of the catastrophe, its attainment of global scale, and the destruction of everyone alive. To each of these three threats (origin, scaling and endgame) corresponds a particular type of defense layer: prevention, response and resilience. Besides serving to clarify the nature of the risks involved, this analysis can assist efforts to prioritize the allocation of resources among the different defense layers. In particular, if the threats that the layers protect against interact multiplicatively with one another, the impact on overall risk reduction of strengthening a particular layer will depend on the relative size of the change to the probability that the associated threat will get past it: halving the risk posed by a threat reduces total risk by the same degree regardless of the absolute magnitude of the change.
Defense in depth (DiD) (sometimes spelled defense-in-depth andor defense in-depth) is the use of multiple defense layers to protect against an external threat. The concept was originally developed in the context of military strategy, but has since been applied to many other domains.[1]
Further reading
Cotton-Barratt, Owen, Max Daniel & Anders Sandberg (2020) Defence in depth against human extinction: Prevention, response, resilience, and why they all matter, Global Policy, vol. 11, pp. 271–282.
Defense in depth (DiD) (sometimes spelled defense-in-depth and defense in-depth) is the use of multiple defense layers to protect against an external threat. The concept was originally developed in the context of military strategy, but has since been applied to many other domains (Muehlhauser 2021: fn. 19).domains.[1]
A recent paper by Owen Cotton-Barratt, Max Daniel and Anders Sandberg proposes a defense in depth approach for handling risks of human extinction (Cotton-Barratt, Daniel & Sandberg 2020).[2] Such risks may be analyzed as involving three successive stages in the unfolding of a terminal event: the origination of the catastrophe, its attainment of global scale, and the destruction of everyone alive. To each of these three threats (origin, scaling and endgame) corresponds a particular type of defense layer: prevention, response and resilience. Besides serving to clarify the nature of the risks involved, this analysis can assist efforts to prioritize the allocation of resources among the different defense layers. In particular, if the threats that the layers protect against interact multiplicatively with one another, the impact on overall risk reduction of strengthening a particular layer will depend on the relative size of the change to the probability that the associated threat will get past it: halving the risk posed by a threat reduces total risk by the same degree regardless of the absolute magnitude of the change.
Bibliography
Cotton-Barratt, Owen, Max Daniel & Anders Sandberg (2020) Defence in depth against human extinction: Prevention, response, resilience, and why they all matter, Global Policy, vol. 11, pp. 271–282.
Muehlhauser, Luke (2021) A personal take on longtermist AI governance, Effective Altruism Forum, July 21.
- ^
Muehlhauser, Luke (2021) A personal take on longtermist AI governance, Effective Altruism Forum, July 21, fn. 19.
- ^
Cotton-Barratt, Owen, Max Daniel & Anders Sandberg (2020) Defence in depth against human extinction: Prevention, response, resilience, and why they all matter, Global Policy, vol. 11, pp. 271–282.
Defense in depth (DiD) (sometimes spelled defense-in-depth and defense in-depth) is the use of multiple defense layers to protect against an external threat. The concept was originally developed in the context of military strategy, but has since been applied to many other domains (Muehlhauser 2021: fn. 19).
Defense in depth against human extinction
A recent paper by Owen Cotton-Barratt, Max Daniel and Anders Sandberg proposes a defense in depth approach for handling risks of human extinction (Cotton-Barratt, Daniel & Sandberg 2020). Such risks may be analyzed as involving three successive stages in the unfolding of a terminal event: the origination of the catastrophe, its attainment of global scale, and the destruction of everyone alive. To each of these three threats (origin, scaling and endgame) corresponds a particular type of defense layer: prevention, response and resilience. Besides serving to clarify the nature of the risks involved, this analysis can assist efforts to prioritize the allocation of resources among the different defense layers. In particular, if the threats that the layers protect against interact multiplicatively with one another, the impact on overall risk reduction of strengthening a particular layer will depend on the relative size of the change to the probability that the associated threat will get past it: halving the risk posed by a threat reduces total risk by the same degree regardless of the absolute magnitude of the change.
Muehlhauser, Luke (2021) A personal take on longtermist AI governance, Effective Altruism Forum, July 21.
Cotton-Barratt, Owen, Max Daniel & Anders Sandberg (2020) https://onlinelibrary.wiley.com/doi/10.1111/1758-5899.12786Defence in depth against human extinction: Prevention, response, resilience, and why they all matter, Global Policy, vol. 11, pp. 271–282.
Wishlist for a heroic editor to someday tick off: