GPU

What is a GPU?

A GPU — graphics processing unit — started life as the chip on your graphics card that renders frames for games. Its superpower is doing huge numbers of simple calculations at once. That happens to be exactly what AI training needs, so the humble game chip became the beating heart of modern AI compute.

How's it different from a CPU?

CPU: a few elite cores
A CPU has a small number of powerful cores, great for complex, sequential tasks.
GPU: strength in numbers
A GPU has thousands of cores. Each one isn't that fast, but together they can split one big job into a million little pieces and chew through them in parallel.

Why does AI training need GPUs?

Training a large model is, underneath it all, an astronomical amount of matrix math — and those operations are largely independent, so they can run in parallel. That's a GPU's home turf. A job that might take a CPU a year can finish in weeks on a cluster of GPUs.

How to read a GPU's specs

VRAM
This decides how big a model (or batch of data) you can load at once. More VRAM means fewer "out of memory" crashes.
Compute
Measured in FLOPs — the bigger the number, the more operations per second.
Bandwidth
How fast data moves between the chip and its memory. If it's slow, everything else chokes.

What GPUs are changing

ChatGPT, self-driving cars, drug discovery — they all run on fleets of GPUs. They also turned NVIDIA from a graphics-card maker into one of the world's most valuable companies. Compute has become the new oil, and GPUs are the refineries.

Bottom line: a GPU is the parallel-computing champ that trades individual speed for massive numbers — and it's what makes modern AI possible.