What is GPT-4?

GPT-4 is the latest (as of May 2024) and largest LLM released by OpenAI. The number of parameters it contains is unknown, but a leaked report claims it to be 1.76 trillion, or ten times the size of GPT-3.

GPT-4 developed a variety of capabilities, notably the ability to program and to give detailed responses to logical puzzles. Furthermore, it has shown some signs of having a theory of mind.

GPT-4 was trained using reinforcement learning from human feedback (RLHF). This feedback was only given during training: the model does not change while in active use. Unlike the previous versions of GPT, GPT-4 was also trained with image data, allowing it to interact with user-uploaded images. (It can also prompt DALL-E and display the resulting images.)

Different versions of GPT have “context windows” of different lengths. GPT-4 initially had multiple fine-tuned versions with context windows of up to 32,768 tokens, much longer than GPT-3.5’s 4,096 and GPT-3’s 2,048 tokens. Newer versions called GPT-4 Turbo and GPT-4o have a 128,000-token window.