How hard is it for an AGI to develop powerful nanotechnology?

  • For one AI to do this requires it to strongly outperform human R&D, which seems possible, but more like a superintelligence than an AGI, so we’ll assume that in the rest of the question [but maybe just change the question instead]

  • Molecular nanotechnology / atomically precise manufacturing is not a well-understood topic, and there’s not much reliable information about what’s possible in this space https://www.openphilanthropy.org/research/atomically-precise-manufacturing/

  • Drexler has analyzed a lot of designs in Nanosystems (and his PhD thesis, available online, which it originated in https://dspace.mit.edu/handle/1721.1/27999) and anticipated a lot of criticisms. Others have tried roadmaps https://www.researchgate.net/publication/266583736_A_Minimal_Toolset_for_Positional_Diamond_Mechanosynthesis

  • There’s been controversy, e.g. the Drexler-Smalley debate https://en.wikipedia.org/wiki/Drexler%E2%80%93Smalley_debate_on_molecular_nanotechnology, http://www.softmachines.org/PDFs/PhoenixMoriartyI.pdf

  • These discussions tend to focus on what it’s feasible for humans to do in decades, so they potentially both underestimate and overestimate what a superintelligence could do in much less time. But see other question about whether superintelligence would be slowed down by experiment time

  • At minimum, everything that exists in biology can be built by a superintelligence. https://twitter.com/ESYudkowsky/status/1637466309142401024 “Engineered superpathogens and classical shoggoth servitor cyborgs” AI already does protein folding. It’s a priori unlikely that evolution is at the limits of what’s possible.

  • Yudkowsky proposes an AI could use proteins to build something like a better ribosome https://twitter.com/ESYudkowsky/status/1627061752629456898

as a step along the way to even more powerful “hard” nanotechnology https://twitter.com/ESYudkowsky/status/1627109382474579968 that can build diamondoid nanoscale robots

  • Bostrom’s Superintelligence has a section based on EY’s ideas on how an AI could potentially go about gaining access to powerful nanotechnology in this way https://publicism.info/philosophy/superintelligence/7.html

  • https://intelligence.org/files/AIPosNegFactor.pdf “The elapsed turnaround time would be, imaginably, on the order of a week from when the fast intelligence first became able to solve the protein folding problem” https://twitter.com/perrymetzger/status/1627093356559106051

  • It’s possible that a superintelligence could come up with something much faster and more effective still

Prose version

Molecular nanotechnology and atomically precise manufacturing are speculative technologies, so we don't have a full understanding of what’s possible in this space. Eric Drexler's Nanosystems (and his PhD thesis, available online) considers a lot of potential designs and anticipates some criticisms. Others have tried to produce roadmaps to nanotech.

Discussions and disagreements around nanotechnology (e.g. the Drexler-Smalley debate, the Phoenix-Moriarty debate) tend to focus on what it’s feasible for humans to accomplish within a few decades, so they potentially both underestimate and overestimate what a superintelligence could do in much less time. (See this page for discussion of whether a superintelligence would be slowed down by "experiment time".)

At minimum, it seems very plausible that any existing biological system could be built by a superintelligence. AI systems already exist which can do protein folding, and it seems unlikely a priori that naturally-occurring biological systems represent the limits of what it's possible for biological systems to be or do.

Nick Bostrom’s Superintelligence includes some discussion of how a superintelligent AI might gain access to and use powerful nanotechnology. Eliezer Yudkowsky has argued that, for a "fast" intelligence (i.e. one that can think much faster than a human, and so pack potentially millions of times as much cognitive work into a relative short amount of "real time"), "the elapsed turnaround time [to developing molecular nanotechnology] would be, imaginably, on the order of a week from when the fast intelligence first became able to solve the protein folding problem."

Of course, it’s possible that a superintelligence could come up with something much faster and more effective than we can anticipate ahead of time.