What is GATO and what is it capable of?
What is GATO?
GATO is a deep neural network developed by DeepMind for a variety of complex tasks that require multiple modes of data like text and images. It does so using a single neural network, under the same set of weights.
Capabilities
GATO possesses the ability to engage in dialogues, play video games, control a robotic arm, and more.
Similar to other large language models, GATO is a transformer.
Unlike some large language models, however, GATO can perform all of these tasks under a single set of weights, meaning it can learn and switch between doing multiple different tasks without having to “forget” a previously learnt skill. It has been described as a “general purpose” artificial intelligence and a “step toward” artificial general intelligence.
In 604 simulated control tasks ranging across a large selection of skills, there were 450 tasks in which GATO achieved greater than the 50% expert score threshold:
Figure source
Limitations
- Generally poor performance at some tasks such as image captioning and conversation due to it being designed as a generalist rather than as a dedicated model.
Examples of GATO’s (rather poor) image captioning capabilities
- Difficulty in obtaining web-scale datasets, which is challenging for scaling GATO up to higher parameters.
Concerns
GATO has as yet not been deployed to the public, but the following risks should be considered nevertheless:
-
As generalist agents can take actions in the physical world, it could lead to users anthropomorphizing the agent, leading to misplaced trust in the case of a malfunctioning system, or be exploitable by bad actors.
-
GATO could create unexpected and undesired outcomes if certain behaviors (e.g. arcade game fighting) are transferred to the wrong context (e.g. resolving real world conflicts).
-
Inherited flaws of other vision-language models, being biased and/or harmful outputs. This is due to the model being biased/harmful input present in the dataset.