IB Computer Science Topic A1.1.2: Role of a GPU Notes

The GPU: Thousands of Small Cores Doing the Same Job at Once

1. GPU stands for Graphics Processing Unit.
2. It is a specialised processor for graphics rendering and for any task that breaks into many identical small calculations.
3. It was built to speed up image and video work, and the same architecture now serves machine learning, scientific computing and data analysis.
4. The GPU works alongside the CPU rather than replacing it, taking on the work its design suits and leaving the CPU free for everything else.

The Architecture Behind It: Many Cores, Wide Memory, One Instruction

1. Thousands of Simple Cores
   1. Each core handles a narrow range of simple operations, and there are enough of them to work on thousands of data elements in the same instant.
   2. This is parallel processing, and it only pays off when the task can be split into independent pieces.
2. High Throughput Rather Than Low Latency
   1. A GPU is tuned to finish a huge batch of work quickly, not to finish any single item quickly.
3. SIMD Execution
   1. Single Instruction, Multiple Data means one decoded instruction is applied to many data elements at once, so the cores do not each need their own control logic.
4. Dedicated High-Bandwidth Memory
   1. The GPU has its own VRAM on the graphics card, holding large textures and data sets.
   2. Thousands of cores are useless if data cannot reach them fast enough, so the wide memory path is as important as the core count.

Why That Architecture Suits Complex Computations

1. Rendering a frame, multiplying two matrices and applying a filter to an image are all the same small calculation repeated across millions of values.
2. Because each value can be handled without knowing the answer for any other value, no core has to wait for another to finish.
3. Software hands this work over through a framework such as CUDA or OpenCL, which is how a machine learning library ends up running on the graphics card.
4. A task with a long chain of dependent steps and lots of branching gains nothing from a GPU, because the cores would spend their time idle.

Where GPUs Are Used in the Real World

1. Graphics Rendering
   1. Video games need high-resolution textures, lighting and a steady frame rate, which means recomputing every pixel dozens of times a second.
2. Machine Learning
   1. Training a neural network is mostly matrix multiplication, so the same weights update across thousands of values in parallel.
   2. This is what makes image recognition and natural language processing practical to train at all.
3. Large Simulations
   1. Climate models, physics simulations and bioinformatics apply the same equation to every cell of a very large grid.
4. Cryptocurrency Mining
   1. Mining repeats one hash function over and over with different inputs, which is parallel work by definition.
5. Video Editing and Design
   1. Effects and 3D models can be rendered as the editor scrubs the timeline rather than being exported and waited for.

VRAM: Memory Built for the GPU

1. The CPU works out of system RAM on the motherboard, while the GPU works out of VRAM on the graphics card.
2. VRAM is built for wide, high-bandwidth transfers of visual data, so it can keep every core supplied at the same time.
3. When a game or model needs more than the card's VRAM, data has to be shuffled back from system RAM and performance drops sharply.