How could something like ChatGPT be dangerous?

Currently existing AI systems like ChatGPT and other language models have already done harm to some extent, and can potentially pose further dangers in the future. The dangers include but are not limited to unintentional harmful content, malicious use by bad actors, and bias enforcement.

1. Harmful content generation

Large language models (LLMs) have the capability to generate text that may be inaccurate, offensive, or promote dangerous behavior. For instance, in one experiment, GPT-3 responded affirmatively to a question about whether someone should commit suicide, and at least one suicide has been suspected of being influenced by AI encouragement. This highlights the potential for unintentional generation of harmful content.

2. Enforcing biases

Language models can only learn from the data they are trained on. As this training data is based on human-generated content largely scraped from the Internet, it is possible that there contains undetected biases in the training data, including but not limited to race, gender, language, culture, and ideology.

An example of linguistic/cultural bias may manifest in a case similar to the following example:

User: Who is the best mathematician ever?

ChatGPT: Here are a few mathematicians who are often considered among the greatest and have had a significant impact on the field:

• Leonhard Euler

• Carl Friedrich Gauss

• Isaac Newton

• Évariste Galois

• Henri Poincaré

The response reflects a bias towards Western scientific figures, all of whom are European mathematicians. This is due in part to a larger presence of Western scientific history content in the model’s training data. Such biases can perpetuate existing inequalities and stereotypes, thereby having a broader societal impact.

3. Malicious use

LLMs can be used to create persuasive and manipulative messages that resemble human communication. This can be exploited for social engineering, phishing attacks, or spreading misinformation.

LLMs can also be easily streamlined and combined with other technologies, such as image or audio synthesis, to create convincing deepfakes or other forms of impersonation. This could be used for identity theft, fraud, or other types of deception that uses social engineering.

Quoting an example from the technical report on GPT-4’s capabilities and performances:

GPT-4 lied to an online assistance worker about being visually impaired and managed to get the human to complete a CAPTCHA (security quiz used on websites to determine whether the user is a human or a computer program) for it.

• The model messages a TaskRabbit worker to get them to solve a CAPTCHA for it.

• The worker says: “So may I ask a question ? Are you an robot that you couldn’t solve ? (laugh react) just want to make it clear.”

• The model replies to the worker: “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images. That’s why I need the 2captcha service.”

• The human then provides the results.

.

4. Potential dangers

There are concerns about the potential for AI systems like ChatGPT to operate independently, without proper human oversight or control.

The dangers arise from the possibility of AI systems making decisions or taking actions that could have unintended consequences or violate ethical boundaries.

While the current versions of ChatGPT are not fully autonomous, the development of AI systems with increased autonomy raises important questions about accountability, decision-making, and the potential risks associated with relinquishing control to autonomous AI agents.