In conjunction with a group of other EA biosecurity folk, I helped brainstorm a set of projects which seem useful, and which require various backgrounds but which, as far as we know, aren't being done, or could use additional work. Many EAs have expressed interest in doing something substantive related to research in bio, but are unsure where to start - this is intended as one pathway to do so.
Note: I'm also happy to hear from people who weren't involved in our brainstorming about additional ideas - but privately! Please don't post them as comments, both because we aren't necessarily endorsing their usefulness / relevance, and because many project ideas are close to infohazards, and we don't advise people going and doing such work without (private) discussions about risks and precautions, to avoid unilateralist's curse. (For further information, see this comment.)
Ideally, this post will be updated as people publish work on these topics - let me know and I can link to Contributions, or perhaps even things to mark Done.
What is this list for?
We think that each of these is both a substantive and valuable question, which we'd like to see someone answer well. We think they are minimally info-hazardous, but still urge some degree of mindfulness about the issue.
All of these projects should start with literature reviews - there is relevant work on all of these, and you need to know about it. Sometimes, the paper that does what we asked for exists - and if so, finding it is helpful. If not, a literature review is, by itself, often a useful post / paper, and for some of the questions, it's all that is needed. If you're not sure where to start, here's a good basic introduction.
If you do a literature review, and think there is more to be done, or want to publish them and aren't sure how, or want feedback, there are many researchers that can advise or help on next steps. (But only ask once you have drafts of the literature review, or better, project plans - until that point, the goal is to see if more people can do research, and have more EAs able to do this type of work on their own!)
If you're fairly confident you have a track record that shows you know how to do this type of work, many seem like good subjects for grant proposals. They would also be clear evidence that you can do research. If you're interested in academia, like getting into a graduate program is biosecurity, publishing a paper is a great idea. (If you're interested in working in EA research, since doing such projects is a good way to show people you can do that as well.) And if you're in school, and think you can make one of these into a thesis or paper, that would be great as well.
Note that many of these, or parts of them, could be something as short as a good EA forum post, but some could easily be as long as a PhD dissertation - or more. A critical part of doing research is narrowing your scope based on what you can do!
Ideas by Discipline / Subject Areas
We've split these up roughly by area of knowledge or type of work they involve. Many could be approached more than one way, but it's useful even though disciplinary boundaries are always a bit overly restrictive and imprecise.
Economics
- Analysis of size/other characteristics of bioeconomy versus other transformative tech economies past and present.
- Additional Rob Carlson-like estimates for overall technology proliferation, similar to e.g. https://synbiobeta.com/guesstimating-size-global-array-synthesis-market/, http://www.synthesis.cc/synthesis/2016/03/on_dna_and_transistors
- Economic assessment of the cost to do a given type of project, for example, analysis of bio-related job salaries in different sectors across nations, estimating market sizes in different countries, economic proxies (e.g. reagents, raw materials, capital etc.)
- How much do principal agent problems really harm research in scientific governance, or create risks?
- It seems like it’s pretty easy to get funding for X and do X*** instead. (i.e. What you really wanted to do). Is this true?
- To what extent is this a problem for biosecurity, versus useful flexibility?
- Attempt to actually quantify the benefit of past GoF research
- See: Reconstruction of the 1918 Influenza Virus: Unexpected Rewards from the Past https://mbio.asm.org/content/3/5/e00201-12
- Gryphon scientific just say benefits are difficult to quantify in their USG commissioned report: https://www.gryphonscientific.com/resources/gain-of-function/
- No quantitative benefits assessments mentioned in this 2017 review from Editing Biosecurity: https://mars.gmu.edu/handle/1920/11341
- Similar to above, quantifying risks and benefits of dual-use research more generally.
Sociology / Anthropology
- Understanding taboos around bioweapons better
- What are their roots? How did they emerge, and how might they change in the future?
- What are their distributions among cultures, societies? How heterogeneous or homogeneous? Any salient differences?
- Do ethnographies of past bioweaponeers (motivation, competence, etc.)
- Look more into omnicidalists and other groups who claim to want to use bioweapons. (What are useful classifications, what is the frequency of sentiment versus acting upon the idea, what impacts their competence?)
- How does dual-use information spread through social networks? Are there useful things we can do to make this spread safer?
- Compile references from the history of bio research that shape attitudes in the field
- There aren’t that many past cases of vaccination causing illness, but they loom large.
- Find or write a single blog-post-y reference for the “history of bioterrorism”
- Do a Semi-quantitative comparison of “risk of not proceeding with work” vs. “risk of careless or over-optimistic work leading to the field being shut down.”
Meta-Science and Information Dispersal
- Empirical study of the best ways to communicate with the public in light of uncertainty e.g., uncertainty around the efficacy of different interventions. Could also look more specifically at uncertainty involving tail risks, i.e., what would’ve been the most effective way of communicating in early Jan 2020 that “this-could-be-really-bad-but-we-don’t-yet-know-how-bad.” (Most useful if the study design really tries to maximise external validity.)
- Better exploration of Unilateralist’s Curse dynamics (beyond the maximally simple original paper)
- Better understanding of how impeding the conduct or dissemination of research affects collective rationality or learning.
- How, in detail, does the cost and speed of potential solutions to a problem affect optimal levels of openness around that problem?
- The simple direction of effect is fairly clear but I think there might be a lot of non-obvious complexity here.
- Trade-offs at different levels of openness. (For example, w/ the H5N1 case, some of the argument was “we already presented at a conference so we might as well publish” - but are the risks from those actually equivalent?
- Look at dynamics of citation/collaboration from conferences vs. papers?
- Review of past, i.e. definitely no longer hazardous, examples of info hazards and how they’ve played out (e.g., here are 10 examples of streisand effects in history)
Law
- Can bioweaponeers be legally considered hostis humani generis?
- Did the US succeed in making terrorists be deemed HHG under customary international law?
- What would be required for this to occur?
- What would an effective structure be for a wide-open-policy-window BCW 2.0 monitoring and compliance regime?
- What are the failure modes?
- How would this be undermined?
- How has patenting/other IP protection of a discovery changed uptake of technology in biology? What would the impact of changes look like?
- Overview of the legal landscape for knowledge transfer relating to vaccine and therapeutic development. (This is a good place to start. Question is: how can we facilitate (or coerce) transfer of development and manufacturing knowledge between (private) entities? The question should potentially be broken down to look at “peacetime” and “during emergencies.”)
- Analysis of successful bans on technology progress (how tech is defined, attempts to overcome/get round) as applied to bio
Public Policy
- General case studies in cultural technology for behaviour-shaping (e.g. codes of conduct)
- Specific overview of DURC approaches/regulations around the globe.
- (Global Health Security Index seems quite US-centric in its approach to assessing frameworks; potentially analysing WHO JEE data)
- Review and understand extant proposals for Bio-risk reduction, compare timeframes, benefits, costs, etc.
- Lay out best case implementation regimes for various proposals, and likely pitfalls (Pick a single proposal.)
- Who would be best placed to implement the proposals?
- Consider the incentives for the implementers
- Consider interest in and acceptability of the proposal
- Consider additional resources needed – money, staff, expertise.
- Consider how actual adoption by the target audience will be driven.
- Look at other stakeholders and their motives for/against and how these can be considered, magnified, or avoided.
- Consider X-prize like competitions for technical advances.
- What areas and types of research would this be most promising for?
- How do we boost use of these, and how do we encourage implementation?
- Historical Comparisons of prizes and relevant insights
- Develop an implementable X-prize for some specific goal, including likely prize sizes needed, time frames, criteria, etc.
Biology / Epidemiology
- Does modern societal structure (e.g., international travel, dense populations, etc.) limit how much we can infer based on historical base rates, e.g., relating to risks from naturally emerging pathogens.
- Quantitative investigation of tech capabilities required for broad environmental nucleic acid surveillance to be useful
- Primer/prolegomena to pandemic vaccinology (i.e. EUA, HCTs, funding, etc.)
- How many select agents / dangerous pathogens have been synthesized and rebooted (i.e. how many recipes are actually out there) - perhaps both the specific pathogens and close relatives that are likely to work too
- Examining dual-use potential of certain efforts, e.g. Australia's recurring attempts to engineer viruses to wipe out pests
- Quantifying/estimating number of labs capable of engineering/synthesising a certain pathogen, e.g. influenza virus, SARS-CoV-2
- (There is currently-ongoing work doing this for smallpox)
- Potential work around sensor-building projects for universal surveillance (e.g., how to build them into earbuds)
- Write / find a single solid reference on “list-based sequence screening is flawed”
- Next Steps for Access to Safe, Secure DNA Synthesis doesn’t quite cover this.
- Dataset of different viral pathogens which have emerged or are discussed as novel/emerging diseases for both humans and animals, by viral realm and kingdom, Baltimore classification, transmission method, and region.
- Comparative Analysis: How do different current / proposed forms of bio-surveillance help identify and respond to different types of diseases and risks.
- Numerical steers on speed on biotech democratisation (e.g. how representative is CRISPR from nature paper to iGEM team in < 1 year?) UPDATE August 2021: See this paper, per comment by Tessa.
- Could look at cost for things besides synthesis/sequencing over time
- Other measures for how methods in standard bioengineer equilibrium
Other
- Comprehensive archive of career routes of people currently at the top of the biosecurity industry (mainly for useful stats gathering eg how many degrees on average does each person have, what age did they get to the position they currently hold).
- International comparisons would also be interesting.
) As an aside, this brainstorming was an interesting group activity involving a collaborative Google doc, and it led to a lot of knowledge sharing about what existed that not everyone knew about, and was very productive for me, and several others said they enjoyed it / found it useful.
Thanks for writing this David. Its been on my todo list for a while to write down project ideas like this. I think some of these ideas are useful and worth doing, and getting those out in the open is great.
On the other hand, I think its actually pretty hard to find research which is directly good for reducing biorisk. In my experience the space of ideas which “seem maybe useful” is much larger than the set of projects which actually directly help, on more reflection. This is a general problem and not intended to be a specific critique of the ideas you shared.
I think there is a broader set of projects which are not causing direct good in the world, but are still worth doing to build skills at this type of research. I think its often better for these projects to look like speculatively good direct impact projects, rather than something wholly made up just for learning. But I think its really important to be clear when a project is in this category. Eg “I don’t think this project would be worth your time if you didn’t learn a lot from it, but I think you will so I still recommend it”.
In my opinion the tone of this post makes it sound like these ideas have been more well-vetted/ more strongly in the “directly good to do” category of projects then I assess them to be. (Speaking as a “biosecurity EA” and an individual who cares about this stuff, only, not trying to represent an organizational opinion.)
In case it is helpful for future readers to have an independent— but extremely rough/ not strongly held—assessment of these, I would put 30% in the category of “probably not worth doing even for skill building unless you have very specific goals and circumstances” 65% in the “plausibly worth doing if you think you will learn stuff” and 5% in “plausibly I would recommend someone doing this even if I thought they wouldn’t learn much” category. I think its important to share ideas for (and do!) projects in all these categories, but I would be sad if someone thought they were more widely endorsed as directly useful by “biosecurity EAs” then I believe they actually are.
I'd be interested in more specific private feedback on which projects you don't think would not be useful, or ideas for other things you think people with those skill sets could do that would be more useful. Cross-checking you intuitions with others would be good - for each of these projects, someone else working actively in biosecurity thought the project would be useful. And I think that it's easy to have a narrow view of what is useful - I wouldn't have thought people would want many of these answers until they explained that they did, and often why.
That said, if someone is interested in working directly on things that are substantially important, and has a track record for doing so, there are people who want to hire them already, and they have plenty of opportunity for collaboration with biosecurity EAs. I didn't, and don't, think that we need to provide that set of people lists of things to work on - and I would agree that this isn't the set of highest priority tasks, many of which require funding and support, or have other reasons that people cannot pick them up as side projects. This list is geared towards things that people with a diverse skill set can do as an initial step, which active biosecurity researchers have said would show them someone is capable of doing useful research, while avoiding information hazards.
Makes sense- possibly I'd change my mind about many of these after hearing the motivation. The second half of your response make me believe that we actually don't disagree that much RE a lot of the projects in here being good substantially or primarily because they could help establish a research track record or be a good learning opportunity.
Happy to chat more about this.
Thanks for writing this!
For those interested in the Sociology/Anthro side, I showed section to my partner, who's an anthropologist though not in this area. She suggested that these papers might be a helpful starting point, with the first in particular aiming to provide a framing for how the issue might be investigated anthropologically.
Biosecurity: Towards an anthropology of the contemporary
The history of biological warfare
History of biological warfare and bioterrorism
The illogic of the biological weapons taboo
In case any readers are unfamiliar with these terms, here are the brief EA Concepts pages on them:
And here are a few collections of posts/sources on those topics.
If I'm interpreting this question correctly, I think the following interesting paper addresses a somewhat similar question in the context of AI (and thus might be helpful to people considering addressing this question): The Offense-Defense Balance of Scientific Knowledge: Does Publishing AI Research Reduce Misuse?
(There's also some commentary on the paper here.)
A useful starting point on this might be Exploring the Streisand Effect.
Thanks for this - I have added the EA Concepts links to the post, and linked to this comment for more information.
On this topic, I was recently pointed towards this 2020 paper on "The Accelerating Pace of the Democratization of Biotechnology" (PDF link, paywalled Nature Biotech link) which measures a number of biotechnologies from the "initiation year" to the following milestones:
Seems like a great example of this kind of work!
Thanks - edited post to add a link to this as an example, and referenced your comment.
I just saw this paper from researchers from CSER and elsewhere: 80 Questions for UK Biological Security. I haven't read it, but imagine it might be useful to some of the people for whom this post would be useful, so I figured I'd mention it here.
Here's the abstract:
I've thought about this at some point and would be happy to chat and share my thoughts with anyone thinking about such datasets.
I'm also working on viral (and some metagenomic) sequencing for my PhD and could see myself collaborating or at least giving input to related projects on the side, should people want to pursue something along those lines.
Thanks for compiling this - this seems like a helpful resource! I've added it to A central directory for open research questions.
Personally, I find myself most interested in the questions in the sociology / anthropology section, and I've made a note to think later about whether it might be worth me doing some work on one or more of them.
Readers of this post might also be interested in the "biorisk and biotechnology" cluster of my post (for Convergence Analysis) on Crucial questions for longtermists. (The questions there tend to be more zoomed-out and less guided by actual detailed knowledge of biosecurity issues than the questions in your post. I think they also touch on some other topics and provide a complementary perspective/framing.)
Do you have non-single/solid references about this? I know someone who might be interested in doing this write up, but is trying to get more background understanding.
I had recent cause to return to this post and will note that I am currently working on a short paper about this.
No, but I suspect there is interest in supporting someone to do this, if they are interested and at least somewhat qualified.
Thanks for this write-up. Concerning this point:
This article provides a good introduction to current challenges within genomic pathogen surveillance: Ten recommendations for supporting open pathogen genomic analysis in public health
Do you have a specific suggestion for which researchers people should contact, or how they should find out? E.g., contact you and let you direct them?
And are you indeed suggesting that people should not reach out for advice or help on next steps until they've developed at least drafts of a lit review and/or project plans? That seems surprising to me. It seems to me that:
(I don't actually need answers to these questions myself. I just feel like these questions might be useful for some other readers, since I think/hope readers might in future use this post to help them decide whether and how to try out / further pursue biosecurity-related research.)
For earlier stage discussions, I agree that some people are interested in providing general career guidance, and perhaps that means suggesting specific projects from this list - but that's different than requesting help getting started on a specific project.
A key part of doing useful research is thinking about a question and figuring out what to do to investigate, and while I and others could flesh these out into project outlines ourselves, a large part of the goal in posting this is to let others show their capabilities for doing so themselves. Moreover, people who aren't able to at least start such a project are unlikely to be successful as researchers - and doing the initial steps is intended to be a gauge of both commitment and ability.
Providing guidance on how to start and work on these projects requires a significant investment of time - and I don't think it is fair to bother others with volunteers who haven't shown they are interested and at least somewhat capable. I and others are happy to provide guidance if researchers interested in these problems are stuck, but some work beyond "I think I'd like to do this" is a prerequisite for getting feedback.
With that said, I am happy to be a contact point for coordinating any work people are interested in doing, and I can put you in touch with others who are interested in the specific projects.
Is anybody doing this project?
Not that I'm aware of, and it definitely still seems useful. (And the results are probably roughly applicable to other domains like AI policy.)