TLDR:

• Governance Design and Formation (GDF) is about using theory and empirical experiments to design and form governments (at multiple levels of society in public and private spheres) which work better, i.e, which have high legitimacy and make high-quality decisions in pursuit of human flourishing in the short and long run.
• It seems more tractable than one might first imagine.

Notes:

• This is a submission for the Open Philanthropy Cause Exploration Prize (https://www.causeexplorationprizes.com/)
• This is a rough draft full of bullet points, because I was short of time (scrabbled this together in a day). But have been thinking about this stuff for years.
• There’s obviously prior work on political and institutional governance. However, as Holden noted in his piece on Ideal Governance, this topic doesn’t really exist as a field. (“Ideal governance seems like the sort of topic for which there should be a ‘field’ of ‘experts,’ studying it.”). Below I lay out a potential definition of this field, sketch its importance/neglectedness/tractability, and discuss how it could be launched.

What's Governance Design & Formation?

• Here's a concrete example: James Madison designed a possible means of governance for the US state of Virginia. It was called the Virginia Plan and it looked like this (don't worry about the details in the picture):
• The plan ended up forming a major part of the US Constitution.
• In short: GDF is figuring out how a governance mechanism should work, and then getting it implemented.
• We're talking here about governance mechanisms that:
  • hold some plenary power (or eventually will) that people actually care about
  • ultimately have to contend with the messy stuff like politics, capture, manipulation, etc
  • act in service of a large number of constituents
• By analogy to economic mechanism design:
  • Economic mechanism design asks the question, "What market rules would result in desirable market outcomes?"
  • Likewise, we ask, "What governance rules would result in desirable governance outcomes?"
• Overall goal of GDF is to use theory and empirical experiments to design and form governance (at multiple levels of society in public and private spheres) which works better, i.e, which has high legitimacy and makes higher-quality decisions in pursuit of human flourishing in the short and long run.
  • Really want to emphasize the "empirical experiments" bit. There's some room for pure research/theory here, but in order to be productive I think it quickly needs to meet reality in the form of experiments, both to iterate on design and to gain legitimacy and adoption.
  • In this day and age, it probably ultimately look like product and software engineering (open source) to make tools that can be grabbed and used.
• Note: there's large overlap with Holden's https://www.cold-takes.com/ideal-governance-for-companies-countries-and-more/ — I loved that piece. I arrived at this stuff independently of Holden's piece, so we should take this submission as a piece of consilience rather than a dependent phenomenon. (Have been thinking about this stuff in a mental background thread since ~2014.)
• GDF seems sufficiently specific and different approach from IIDM (Improving Institutional Decision-Making) that it's probably helpful to think of it as its own thing, even if there's lots of overlap and GDF is technically a subset of IIDM or something like that. I don't know enough about the IIDM area to comment.
  • Note that GDF isn't focused on reforming existing institutions (at least at first) — there's some truth to the whole "The best way to predict the future is to design it" thing.

Importance, in brief

• I'm going to handwave here for now. Do have a rough more involved analysis, but a handwave feels sufficient for now.
• In short:
  • As blast radius of tech increases, we're increasingly all in the same boat together, so we need powerful actors to make high-quality decisions or we're in trouble.
  • If we could somehow get a large portion of government-type power at various levels in various societies to make very high-quality decisions, it would be very important, in ways that are kind of obvious — huge mitigation of various x-risks, etc. (Not to mention quality of life)
  • This would be robustly good — i.e,. even if we're wrong about which challenges will arise or the ways in which they will come or how they might need to be handled, it would be good to have the powerful actors making high-quality decisions.

(Note: there's a slightly more quantitative take on this in the "Cost Efficacy" section towards the end)

Tractability

This submission will mainly focus on tractability because neglectedness and importance are easier to think about.

What I'm claiming about tractability

• First of all: I have a lot of uncertainty here.
• At first blush, tractability might seem quite low. (How, one might ask, will you institute your fancy new governance mechanisms? You and which army, exactly?)
• Surprisingly, though, I think tractability is actually lowish-to-mediumish. (And that it only recently became so: I think tractability was a lot lower in 2010.)
• Some causes seem less tractable the more you dig into them and think; this one seems to go the other way.
• This area is so neglected that we haven't seen many attempts at it. As a result, the evidence for tractability looks more inside-view-y sketches of what might be possible rather than outside-view-y "here's a list of past initiatives in this area and their base success rate".

Summary of the case for lowish-to-mediumish tractability

1. There's tons of stuff that would be good to try and might work
2. There are ways to try things, i.e. to run governance experiments
3. If something tried works well, it can quickly lead to greatly increased adoption amongst new orgs (there's some empirical evidence of this!)
4. There are plausible paths to significant worldwide adoption

1. There's lots of stuff that would be good to try and might work

Structural advantages of being new

• Greater knowledge
  • We simply have more history now; more democracy-years under our belt. We have more examples of what can go right or wrong.
  • We have better tools than the governance designers of yore (e.g. James Madison didn't have Google Search)
  • Can stand on the shoulders of giants
• Changed world
  • The world changed in scale (an order of magnitude more people than 400 yrs ago). Often when you scale something by 10x, things break unless you redesign them (existing govts mostly haven't been redesigned).
  • There’s far more complexity in the world now than before, and things move faster.
  • Technology changes things
    • it changes how governance can work
    • it changes what needs to be governed — governing technology just seems pretty different from governing other stuff, and also seems very important.
• Advantages of youth
  • Not being sclerotic/bureaucratic (can thus attract a different type of person)
  • Doesn't start out captured
  • Legitimacy of many existing things is so low that many might actually find that new alternatives could have more legitimacy in limited circumstances, even if not battle-tested yet

There's a rich design space of governance "lego blocks"

Note: I'm not being a technosolutionist here. I don't think technology alone is the answer. Nor do I think people alone are the answer. I think you need both great governance design (which may involve some tech, this is 2022 after all) and great people operating within that government.

Here's a list of some "lego" blocks (there are more in the appendix):

• Group-informed consensus (e.g. Polis https://pol.is/): use computational methods to discover clusters of people and their views, in order to grok people's complex multifaceted values and discover points of consensus. Avoids tyranny of the majority/minority type issues.
• Computational value pluralism TODO
• Representative deliberation: bring together a representative "mini-public" for a fixed period (h/t Aviv Ovadya); pay them; educate them on a topic; learn from their conclusions. Has been used globally to positive effect I think (confidence: low)
• Machine learning (applied to governance)
  • Machine learning is quickly becoming powerful and versatile, a trend which seems likely to continue swiftly
  • Mental model is that ~ML will be able to greatly improve whatever it is applied to
  • There's power in deciding which things to really seriously apply it to first
  • One thing we should seriously apply it to first is improving governance.
    • We're running an AI capabilities vs. AI alignment race — we should keep running that race.
    • But there's also an AI capabilities vs. General Collective Decision Quality race — we should start running that race, and (ironically) use AI to help us win it.
  • Have some specific thoughts around how ML could be used to improve governance
• Ultra-fast courts (very excited about this one, actually)
  • Existing courts were designed pre-internet and proceed at a weird paper snail mail "this judge has to ride their horse to the next district" type pace
  • Can make sloppy decisions in lower courts and have really quick/fast appeals processes instead
  • Works well digitally. Can be used both inside and outside the government.
• Epistemic networks:
  • everyone has some secret opinions stored away in their mind about how good/accurate/effective/ethical other people are.
  • These almost never become public; they're sometimes shared in limited form in backchannels during job interviews or the like. People keep this information private to keep the peace, avoid conflict, maintain their own reputation as a kind person, etc.
  • However this information is actually extremely valuable to the world. It might be possible to use cryptography to build a way for people to safely share their views about others such that no one's feelings get hurt by anyone else in particular, and also globally the government can learn about its workers.
  • There's about a million ways this could go wrong, but might be good to experiment with.
• Guard against narcissists/sociopaths/power-maximizers
  • These people really spoil the world for everyone else (https://forum.effectivealtruism.org/posts/LpkXtFXdsRd4rG8Kb/reducing-long-term-risks-from-malevolent-actors)
  • I think it's deeply under-appreciated how big a problem these actors are. Setting aside the obvious problems with them, they manipulate people en masse and cause a ton of flow-on problems.
  • There are people that can reliably detect these kinds of folks; could have a nexus of them which can ban problematic people. (Would need to be designed carefully, since that'd be quite a power)

There are lots of possible governance design lego blocks. I'm going to put some more in an appendix. It's probably the most fun part of the submission — do check it out!

Info rich vs. info poor

New governance mechanisms can be info-rich, that is, they could drag waaay more information out of people. The world and value is insanely complex, so it's important to work with lots of information.

• An example of an info-rich mechanism: market economies in which people reveal information through their purchasing decisions which flow through price signals)
• An example of an info-poor mechanism: traditional democracy, which pulls a tiny amount of information from people (votes) and has narrow power structures at the top w/ few inputs
  • Traditional government has informal means of pulling information such as surveys/polls, but these are ad-hoc and haphazard and not necessarily integrated into an actual governance process.

2. There're ways to run governance experiments

• There are ways to run meaningful governance experiments without getting anyone's permission first
• There's demand from new and nascent orgs for governance experiments, even for mechanisms that are new and untested
• There's demand from larger players for good well-tested governance
• There's demand from massive players (e.g. even nation states!) for seriously battle-tested governance mechanisms
  • There are people whose job it is to help nations write new constitutions!

Ways to try stuff out without needing anyone's permission

• City council ballot initiatives
  • Can set up a new governance mechanism whose purpose is to decide what ballot prop to propose.
    • could do as a one-off, or as an ongoing parallel innovative governance structure that plugs into the traditional one
    • could involve randomly sampling voter roles and sending invitations to participate in the mail
  • only a vanishingly small number of people realize this is a possibility (possibly just me and people I've discussed it with)
• Omnibus state ballot initiatives
  • Like city ballot initiatives, except at the state level. State props are expensive, but some states allow more than one topic in a single ballot prop, so the innovative governance mechanism could propose an omnibus prop.
  • Haven't really investigated the technicalities to know if this would work
• Creating new DAOs
• Creating civic non-profits that execute atraditionally
  • E.g. could make a city org that aims to beautify a city by keeping its sidewalks clean, generating more gardens, murals, etc in a way that is pleasing to its residents, and which is governed in new kinds of ways.

Demand from new/nascent institutions

• There's demand for less well-tested stuff that might work well. In particular, there are nascent that would to try new stuff and experiment with governance, but whose main focus is elsewhere, so they don't really "get" to. They'd love a high-quality option they could grab off the shelf.
  • DAOs (https://en.wikipedia.org/wiki/Decentralized_autonomous_organization) — they're super into experimenting with this stuff
  • Charter cities (total speculation that they'd want this)
• (Coming soon) Games/VR — this is probably going to be much bigger much more influential and happen much faster than people are expecting — VR products are nearing a point where there will be mass adoption for various reasons. Governance of virtual worlds is an ideal spot for experiments.

Demand from existing institutions

• Platforms like Facebook, Twitter etc actually want this stuff
  • Twitter is experimenting with group-informed consensus inspired by Polis (https://twitter.github.io/birdwatch/diversity-of-perspectives/)
  • Know of one person (https://aviv.me/), who in less than a year of part-time work, got multiple digital platforms (which govern communications of more people than most countries) putting resources towards governance experiments and some even running successful pilots (nothing public yet).
• Civil servants in government sometimes are excited to delegate an area to a new governance method
• Partial delegation from regulatory agencies or legislatures
  • Especially power vacuums due to hot-button topics no one wants to touch
• EA itself? — wouldn't want to consign the whole movement over to an untested gov structure, but could assign a chunk of money to be directed according to the decisions of an experimental governance mechanism
• There's demand for mature, battle-tested, well-designed governance
  • AI companies that need to govern their tech
  • New constitutions (there are actually people whose job it is to help design new national constitutions — it's a thing!)

Note: Running a governance experiment does require someone who "gets it" and knows how to execute. These entrepreneurial/builder-y/organizer-y folks might be in short supply, which could be a bottleneck in the space. (Not a positive for tractability)

3. If something tried works well, it can quickly lead to greatly increased adoption amongst new orgs

• The world actually seems fairly hungry for better governance mechanisms — there's demand out there.
• And there's an empirical example: at one point in the web3/DAO space, there was a protocol called Compound (https://compound.finance/governance) which made an easy off-the-shelf governance solution.
  • It was actually an extremely problematic design (it worked on coin voting aka plutocracy) and was/is vulnerable to various bribery attacks.
  • But it stood out by being in Actual Usage, and was easy for others to adopt. Many other protocols ended up adopting it; it was kind of the go-to for a while.
  • Maybe we can pull off the same except with a way better design!

4. There are plausible paths to significant worldwide adoption

• New and nascent orgs (which have adopted new governance) may end up holding a large portion of world power
  • technology shifts are generally arriving faster, and existing institutions that fail to navigate them may become obselete faster than in the past. As examples, Facebook and Twitter rose to significant power pretty quickly. I'd say there's maybe a 25% chance that this trend does or will apply to power and institutions broadly (low confidence here)
  • Great governance might help a new entity grow faster, and thus end up with more power
• There may "stochastic" adoption from large incumbents due to particular unexpected moments, crises in legitimacy, etc where there may be sudden/deep adoption if there are high-quality battle-tested governance mechanisms lying around already

Neglectedness

• As best I can tell, it's almost no one's job to do GDF.
• It's the consummate public good, so is under-provided by market economy
• Many nascent orgs are probably in a frustrated state of, "ahhhh I'd really like to try out better governance, but:"
  • "we can't afford the split focus / resources to put together a really cool experiment"
  • "there isn't anything off the shelf we can trust enough"
• There are a couple of meaningful experiments, e.g. https://community.optimism.io/docs/governance/
• There's a fair bit of work going into reforming existing actors/governments by slipping in some new methods, but very little in trying out radically new stuff.

Cost efficacy

Blunt estimate

• This is perhaps a cliche, but:
  • (smallish odds of getting great governance widely adopted) * (absurdly massive impact from having that happen) --> great expected value
  • I'd put P(our efforts could be instrumental in getting great governance adopted widely) = maybe 0.5% or 1%
  • And I'd put the impact if that occurred at, ummm, err... something so huge that this whole endeavor becomes kind of obviously worth it.
• This 'simple' gut-ish analysis is actually pretty valid here I think, given reasonable wide/well-calibrated estimates of the odds and impact. This is the framing I'm most comfortable with.
  • I've gotten a lot of mileage out of thinking this way in general
• Often in an area like this, the first dollars & attention make the biggest difference, so I think initial cost effectiveness is likely to be quite high.

BOTEC

Here's a BOTEC which is broken down a bit more (with very sketchy estimates. to emphasize, I really would not stand by these numbers without thinking about them a bunch more):

• P(average decision quality of new/nascent institutions increases significantly contingent on investing $100m + attention + people in this cause area) = 2%
• P(within 20 years, >1/3rd of world power is held by institutions which today don't exist or are nascent) = 25%
• P(within 40 years, there is an x-risk which would clobber us but for >1/3rd of world power having significantly higher decision quality) = P(there is a dodgeable x-risk) * P(we dodge it thanks to the increased decision quality) = 20% * 3% = 0.6%
• Multiplying through, we discover a ~0.3% chance of averting an x-risk in the next 40 years.
• If accurate, seems fairly absurdly good — to make this outcome more intuitive, if we value each of the 8B people at $10k/person (~amount it costs to save a life), works out to 500B, or a 5,000x return (exclusive of future people, which is where some would argue most of the value lies)
• I'm a little uncomfortable focusing this BOTEC so much on x-risk, because I think improved government would be robustly good in ways not captured by this analysis.

Are there grants to make? What's a grantmaker to do?

• Because a modern version of the field is so nascent, I doubt there are a ton of low-hanging passive funding targets. It might require the grantmaker to take on a bit of an active role in spurring things. (Sidebar: I actually think this is a powerful role for grantmakers to play)
• Some off-the-cuff grantmaking possibilities, in rough order from passive to active:
  • Fund people who are already working in this area, e.g. Aviv Ovadya https://www.belfercenter.org/publication/towards-platform-democracy-policymaking-beyond-corporate-ceos-and-partisan-pressure (He just told me he'd be up for being funded for a year to explore the cause area and potential grantmaking opportunities)
  • Fund organizations that are already working in this area, e.g. Polis https://pol.is/
  • Ninja edit after OP submission deadline: adding this item: Aaron Goldzimer is an EA who has also been thinking about this for years and envisions something similar to what the Hewlett Foundation is doing around rethinking economics - but instead for democracy/government. He imagines funding workshops, deep research, working groups, conferences, empirical work, etc., all around re-thinking human self-government for the future, with a special eye towards catastrophic/existential risk and explicitly trying to counteract human cognitive biases like short-termism; tribalism; and susceptibility to disinformation, propaganda, conspiracy theories, and demagoguery.
  • Crawl social graph to find other such examples of people already working on this, make sure they have the funding to focus on it.
  • Reach out to people who are interested in this space on Twitter and recruit them to the conference, to submit ideas, to collaborate with each other, offer grant money
  • Try to start the field 'for real': fund someone to run a Governance Design & Formation conference that pulls together everyone interested in this area. I think there are actually probably a fair number of such folks, some of them quite talented.
    • Invite people like this https://twitter.com/divyasiddarth/status/1556349384425705473
    • This one feels high-leverage, actually — I suspect there's a lot of energy for GDF but pluralistic ignorance about the existence of that energy/desire to go for it.
  • Crawl the social graph to see who is already thinking about this stuff, but not working on it directly. A bunch of them probably actually want to work on this. Fund them to work on it directly.
  • Ask https://en.wikipedia.org/wiki/Audrey_Tang what to do
  • Ask Vitalik Buterin what to do (he's thought about stuff, e.g. https://vitalik.ca/general/2021/08/16/voting3.html)
  • Ask Colin Megill what to do
  • Ask radicalxchange what to do
  • Fund someone to design a novel governance mechanism for directing an effective altruist initiative, and give that governance mechanism a bag of money to work with.
  • Fund a DAO governance shop
    • A non-profit that consults with new web3 protocols, figures out governance for them specifically, builds up knowledge over time from diff experiments, tries new things
  • Potential pitfall: getting rabbitholed in the academic/ the theory side of things. To avoid, consider things like:
    • Run a call for empirical experiments, give big awards for it to try to pull in interested entrepreneurs.
    • Matchmake practitioners who have experience running governance exercises(e.g. deliberative democracy facilitation people) w/ governance designers who want to test out new methods
    • Matchmake good governance design theoriests with with innovation-minded cities; fund them to run an experiment
  • Open a fund that supports governance experiments that plan to plug into city/state ballot initiative processes in order to be meaningfully connected to power
    • Can offer money to get initiatives on ballots

Appendix

About the author, in case relevant to evaluating his epistemics

• Name = Ted Suzman
• YC Founder. In 2018 took on a project to protect US democracy by defeating Trump; ended up accounting for 5% of the vote margin in the most pivotal swing states (the states in which you have to flip the least number of votes to flip the election outcome). Partial writeup: https://ted.suzman.net/2020/06/08/yc-methods-applied
• Have a verifiable track record of making very early weird / "out of the money" predictions that end up coming true, across many domains
• Good at strategy (was top 100 US Starcraft player during beta), reached 1 dan in Go as an adult in two years
• Was at the first ever EA Global in 2013 (was called the EA Summit back then!)
• Don't really have a resume or website that's accurately reflective of me. The open phil contest lured me out of lurker mode to write this up! Hoping the contest gets some attention/eyes on the cause area.

Fun stuff: more governance design lego blocks

Apologies for the TODOs in this section — ran out of time! If you're curious about any of them just ask.

• Open participation
  • Some stuff might need to be private
  • Could have internal bounties for any information marked private which doesn't actually need to be private.
  • Beyond that, there are huge benefits to running things in the open, and letting anyone who wants to wander in and start participating in some way. (DAOs in web3 sometimes work this way, and it's a great means of recruitment and building trust)
• Well-being/flourishing surveys
  • Actually design these well; run them consistently on a wide basis
  • Actively ferret out problems early; actively work to solve; actively measure to see if solutions are working
• Forecasting TODO
  • Futarchy/pseudo-futarchy type stuff
  • Forecasting outcomes of policies during policy formation
• Sortition: randomly sampling the population and engaging with the sample
• Economic mechanism design TODO
  • This is a super powerful toolkit
• Capture resistance (Designed for continual legibility / swappability of people)
  • Defeating the https://en.m.wikipedia.org/wiki/Iron_law_of_oligarchy TODO
  • Chaos monkey TODO
• Complexity limits (e.g. maximum line counts of all laws)
• Productive means of combining regular folks with experts (you want/need input from both)
• Understandability requirements (e.g. randomly selected folks of a certain reading level must be able to read and then accurately re-articulate each policy)
• Pay government workers better (hard to reform existing govts on this front, easier in new ones)
• Epistemic escalation games: make it so anyone can purport claims like, "That line of that strategy doc isn't true", or "This person isn't acting with the public interest at heart" or the like. Forecasters can anticipate which claim will win; things can be appealed up if there are disagreements (people might have to crowdfund bounties to get stuff appealed). Inspiration: https://kleros.io/
• Process control (PID controllers — think home thermostats that try to keep the temperature at the right level)
  • Lots of things to process control. For example, want some minimum % of your smart/empathetic/ambitious people to be come teachers, or you're screwed long term. As a result, continually automatically adjust teacher salaries / teaching school subsidies (up or down) until you're on track to hit your desired percentages
• Actually model out different groups' directed "goal graphs" / theories of change; seek out a shared graph on which there is group-informed consensus (h/t Colin Megill)
  • Plus deliberate model combination on varying approaches
• Quadratic mechanisms (a means of value pluralism) TODO
• Various solutions to people not being engaged TODO
• Solutions to the "it's worked for so long we started taking it for granted" problem: Competitive simulations / redteaming / etc where people try to capture, take power, seek misaligned goals, etc. Could give everyone involved in the government direct hands-on experience with all the known ways things can go wrong; working knowledge of the purpose of everything.
• Open state/execution TODO
  • Can also systematically prevent certain kinds of corruption, e.g. https://twitter.com/LisaBraganca/status/1488903119224328192
• Proofs of computational integrity TODO
  • Closes some loopholes, prevents certain kinds of corruption. like Jan 6 was literally interrupting a sum() process.
  • Doesn't close every angle of attack, of course, but the fewer the angles of attack, the more eyes/awareness/attention on each one
  • Magical tech, feels like it shouldn't exist https://eprint.iacr.org/2018/046
• Anti-goodharting module — could have people predict whether, after the fact, people in the future will decide that some important metric was goodhardted. If these predictions go over a certain bound, it could set off various alarms that pull in people to check out what's going on. The hope would be for people to use their common sense to avoid hacking metrics.

Examples of some lego pieces snapped together

• Forecasting + Economic escalation game + Polis:
  • Could be a cheap way to discover areas of consensus in many different areas.
  • People could predict (and stake some money on) what the consensus outcomes of a Polis conversation would be if the conversation were actually run.
  • Others could challenge those predictions, or let them stand.
  • If challenged and ultimately appealed enough, the actual Polis conversation could be run
  • If the original prediction goes unchallenged, then after a time it could earn a reward.
• Computational value pluralism + Machine learning
  • Human value is really complex and remarkably easy to crush by accident.
  • Ran out of time to write this one up :D
• Epistemic networks + Guards against sociopaths
  • Could discover people who are good at predicting which other people are good at a skill (such as detecting malevolent actors). That way, you can maintain and grow your network of sociopath-detectors.

---

Thanks to David Slifka for giving feedback an early outline of this, and a few other people for last-second comments. Wish I had time to run it by more people!