How might AGI interface with cybersecurity?
When we talk about the interface between AGI and cybersecurity we could have 2 things in mind.
-
Cybersecurity to protect an AGI research project from human adversaries
-
Cybersecurity to protect against an adversarial AGI
First, an AGI will have great economic and strategic value, therefore governments and other companies have an incentive to try and steal research findings, including model weights, from organizations developing AGI.
If AGI is being developed by a safety conscious company or team, it would be essential for them to prevent others from using their models without the safety features or evaluations which they would implement. In order to do that they would need top cybersecurity since any leak of things like their model weights could allow less safety concerned teams to release a dangerous AGI.
Second, if AGI is developed cybersecurity could be a key failure which allows it to gain access to sensitive systems. There are many hacks every year, and most software has potential holes that a skillful adversary could hack into. AN AGI which has the ability of discovering these holes could use that to great effect.
For example, any plan which makes use of boxing an AGI needs to be sure that the box is actually secure.