Why does DARPA work?

Why does DARPA work? Ben Reinhardt. June 2020. https://benjaminreinhardt.com/wddw


How can we enable more science fiction to become reality?

If you want to do something, it usually pays to study those who have done that thing successfully in the past. Asking ‘what is this outlier’s production function?’ can provide a starting point.

DARPA is an outlier organization in the world of turning science fiction into reality. Since 1958, it has been a driving force in the creation of weather satellites, GPS, personal computers, modern robotics, the Internet, autonomous cars, and voice interfaces, to name a few. However, it is primarily limited to the domain of defense technology – there are DARPA-style ideas that are out of its scope.  Which emulatable attributes contributed to DARPA’s outlier results? What does a domain-independent “ARPA Model” look like? Is it possible to build other organizations that generate equally huge results in other domains by riffing on that model?

Gallons of ink have been spilled describing how DARPA works1, but in a nutshell here is how DARPA works. Around 100 program managers (PMs) with ~5 year appointments create and run programs to pursue high-level visions like “actualize the idea of man-computer symbiosis.” In these programs they fund researchers at universities and both big and small companies to do research projects of different sizes. Collectively, groups working on projects are called performers. Top-level authority lies with a Director who ultimately reports to the Secretary of Defense.

DARPA has an incredibly powerful model for innovation in defense research, and I believe an abstract ‘ARPA Model’ could yield similar results in other domains. In this piece I’ll explain in detail why DARPA works. I’ll use that description to feel out and describe to the best of my ability a platonic ARPA Model.  I’ll also distill some of the model’s implications for potential riffs on the model. Incidentally, I’m working on just such an imitator, and in future essays, I’ll explain why this model could be incredibly powerful when executed in a privately-funded context.

How to use this document
This document acts more like a collection of atomic notes than a tight essay – a DARPA-themed tapas if you will. The order of the sections is more of a guideline than a law so feel free to skip around. Throughout you will come across internal links that look like this. These links are an attempt to illustrate the interconnectedness of the ARPA Model.

There are two stand-alone pieces to accomodate your time and interest: a distillation, and the full work. The distillation is meant to capture and compress the main points of the full work. Each section of the distillation internally links to the corresponding section one level deeper so if you want more info and nuance you can get it.

I would rather this be read by a few people motivated to take action than by a broad audience who will find it merely interesting. In that vein, if you find yourself wanting to share this on Twitter or Hacker News, consider instead sharing it with one or two friends who will take action on it. Thank you for indulging me!

Distillation
Program Managers
At the end of the day the ARPA Model depends on badass program managers. Why is this the case? PMs need to think for themselves and go up and down the ladder of abstraction in an unstructured environment. On top of that they need to be effective communicators and coordinators because so much of their jobs is building networks. There’s a pattern that the abstract qualities that make “great talent” in different high-variance industries boils down to the ability to successfully make things happen under a lot of uncertainty. Given that pattern, the people who would make good DARPA PMs would also make good hedge fund analysts, first employees at startups, etc. so digging into people’s motivations for becoming a PM is important. More precise details about what makes a PM good prevent you from going after the exact same people as every other high-variance industry. When ‘talent’ isn’t code for ‘specialized training’ it means the role or industry has not been systematized. Therefore, despite all the talk here and elsewhere about ‘the ARPA Model’ we must keep in mind that we may be attributing more structure to the process than actually exists.

DARPA program managers pull control and risk away from both researchers and directors. PMs pull control away from directors by having only one official checkpoint before launching programs and pull control away from performers through their ability to move money around quickly. PMs design programs to be high-risk aggregations of lower-risk projects. Only 5–10 out of every 100 programs successfully produce transformative research, while only 10% of projects are terminated early. Shifting the risk from the performers to the program managers enables DARPA to tackle systemic problems where other models cannot.

The best program managers notice systemic biases and attack them. For example, noticing that all of the finite element modeling literature assumes a locally static situation and asking ‘what if it was dynamic?’ “The best program managers can get into the trees and still see the forest.” Obviously, this quality is rather fuzzy but leads to two precise questions:

How do you find people who can uncover systemic biases in a discipline?
How could you systematize finding systemic biases in a discipline?
The first question suggests that you should seek out heretics and people with expertise who are not experts. The second question suggests building structured frameworks for mapping a discipline and its assumptions.

A large part of a DARPA program manager’s job is focused network building. DARPA PMs network in the literal sense of creating networks, not just plugging into them. PMs meet disparate people working on ideas adjacent to the area in which they want to have an impact and bring them together in small workshops to dig into which possibilities are not impossible and what it would take to make them possible. The PMs host performer days — small private conferences for all the people working on different pieces of the program where performers can exchange ideas on what is working, what isn’t working, and build connections that don’t depend on the PM. J.C.R. Licklider2 is a paragon here. He brought together all the crazy people interested in human-focused computing. On top of that,  he also helped create the first computer science lab groups. PMs also build networks of people in different classes of organizations – government, academia, startups, and large companies. These connections smooth the path for technologies to go from the lab to the shelf.

DARPA PMs need to think for themselves, be curious, and have low ego. Why does this matter? When you are surrounded by smart, opinionated people the easy option is to either 100% accept what they’re saying because it’s eloquent and well-thought through or reject it outright because it sounds crazy or goes against your priors. Thinking for yourself allows you to avoid these traps. PMs need to be curious because building a complete picture of a discipline requires genuine curiosity to ask questions nobody else is asking. A large ego would lead to a program manager imposing their will on every piece of the program, killing curiosity and the benefits of top down problems and bottom up solutions.

DARPA is incredibly flexible with who it hires to be program managers. There are legal provisions in place that let DARPA bypass normal government hiring rules and procedures. Hiring flexibility is important because PMs are the sort of people who are in high demand, so they may be unwilling to jump through hoops. Bureaucracies ensure consistency through rules – minimal bureaucracy means there are no safeguards against hiring a terrible program manager so the principle that ‘A players hire A players and B players hire C players’ is incredibly important. 

DARPA Program managers have a tenure of four to five years. This transience is important for many reasons. Transience can inculcate PMs against the temptation to play it safe or play power games because there’s only one clear objective – make the program work. You’re out regardless of success or failure. Explicitly temporary roles can incentivize people with many options to join because they can have a huge impact, and then do something else. There’s no implicit tension between the knowledge that most people will leave eventually and the uncertainty about when that will be. Regular program manager turnover means that there is also turnover in ideas.

Why do people become DARPA Program managers? From a career and money standpoint, being a program manager seems pretty rough. There are unique benefits though. It offers an outlet for people frustrated with the conservative nature of academia. The prospect of getting to control a lot of money without a ton of oversight appeals to some people. Patriotism is definitely a factor, and hard to replicate outside of a government. Being a PM can gain you the respect of a small, elite group of peers who will know what you did. Finally, there may be a particular technological vision they want to see out in the world and DARPA gives them the agency to make it happen in unique ways.

[complete text, lots of links, at the URL above]