What is GPT-4?

GPT-4 is the latest (as of February 2024) and largest LLM released by OpenAI. The exact number of parameters it contains is unknown, though some inside sources claim it to be around one trillion. This would make GPT-4 approximately six times larger than GPT-3, the previous model in the GPT line, which had 175 billion parameters.

GPT-4 developed a variety of capabilities – notably: programming ability, the ability to give detailed responses to logical puzzles, and some signs of having a theory of mind. Some of these capabilities were already present in GPT-3, but GPT-4 surpasses GPT-3’s performance in these areas.

GPT-4 was trained using reinforcement learning from human feedback (RLHF). This feedback was only requested during training: the model does not change while in active use. Unlike the previous versions of GPT, GPT-4 was also trained with image data, granting it the capability to interact with user-uploaded images. (It also has the capacity to generate image prompts for DALL-E and display DALL-E's outputs for the user).

Different versions of GPT have “context windows” of different lengths. GPT-4 has multiple fine-tuned versions, with context windows of up to 32,768 tokens; this represents a significant increase over previous versions (GPT 3.5 has a 4,096-token window and GPT-3 has a 2,049-token window).