CPU pipelining increases processor throughput by overlapping the fetch, decode, and execute stages of the fetch–decode–execute cycle, so that different instructions are at different stages simultaneously. Instead of finishing one instruction completely before starting the next, the CPU works like a production line — always busy at every stage.
What problem does pipelining solve?
Without pipelining, the CPU processes one instruction at a time through three stages:
- Fetch: read the next instruction from memory into the instruction register
- Decode: interpret the instruction (what operation? which operands?)
- Execute: carry out the operation
In a non-pipelined CPU, stage 2 cannot start until stage 1 is completely finished, and stage 3 must wait for stage 2. If each stage takes one clock cycle, each instruction takes three clock cycles to complete:
Clock cycle: 1 2 3 4 5 6 7 8 9
Instruction 1: [Fetch] [Decode] [Execute]
Instruction 2: [Fetch] [Decode] [Execute]
Instruction 3: [Fetch] ...
Three instructions take 9 clock cycles — equivalent to 3 cycles per instruction (CPI = 3).
How does pipelining increase throughput?
A pipelined CPU starts fetching the next instruction as soon as the current one moves to decode, and starts fetching a third as soon as the second moves to decode — all three stages are active simultaneously:
Clock cycle: 1 2 3 4 5 6
Instruction 1: [Fetch] [Decode] [Execute]
Instruction 2: [Fetch] [Decode] [Execute]
Instruction 3: [Fetch] [Decode] [Execute]
Three instructions now complete in 5 clock cycles, not 9. More importantly, from clock cycle 3 onwards, one instruction completes every clock cycle — CPI approaches 1.
The throughput (instructions completed per unit time) has tripled without changing the clock speed. This is pipelining's power: it uses parallelism within a single core.
A simple analogy: the car wash
Imagine a car wash with three sections: soap, rinse, dry. Without pipelining, car 1 goes through all three sections before car 2 enters the soap section — the rinse and dry sections sit idle whilst car 1 is being soaped. With pipelining, as soon as car 1 moves to rinse, car 2 enters soap and car 3 waits. All three sections are active simultaneously. The total time for each car is unchanged, but the throughput — cars completed per hour — is tripled.
What are pipeline hazards?
Pipelining assumes that each stage can always proceed — but real programs introduce complications called hazards that can stall the pipeline:
| Hazard type | Cause | Example |
|---|---|---|
| Data hazard | Instruction needs the result of an instruction still in the pipeline | ADD R1, R2, R3 followed immediately by MOV R4, R1 — R1 not yet written |
| Control hazard | A branch instruction means the next instruction to fetch is unknown | IF x > 5 THEN GOTO label — which instruction is next? |
| Structural hazard | Two instructions need the same hardware resource simultaneously | Both fetch and execute need memory access in the same cycle |
Dealing with data hazards: Modern CPUs use forwarding (passing a result directly from the execute stage back to the decode stage without writing it to a register first) or insert pipeline stalls (NOP — no operation — instructions that waste a cycle until the needed value is available).
Dealing with control hazards: CPUs use branch prediction — the processor guesses which branch will be taken and prefetches along that path. If the guess is wrong, the prefetched instructions are discarded (pipeline flush) and the correct path is fetched, wasting several cycles. Modern processors predict correctly more than 95% of the time.
How does pipelining relate to multi-core processors?
Pipelining and multiple cores are complementary techniques:
- Pipelining increases throughput within a single core by overlapping instruction stages.
- Multiple cores allow genuinely parallel execution of separate instruction streams.
A modern CPU might have 8 cores (running 8 instruction streams simultaneously) each with a 15-stage pipeline (overlapping 15 instructions at different stages within each stream). Both techniques together give enormous throughput improvements over a single-core, non-pipelined design.
Frequently asked questions
Is CPU pipelining on the GCSE Computer Science syllabus?
Pipelining is explicitly listed in the OCR GCSE Computer Science specification (J277) as part of the CPU architecture topic. AQA GCSE also addresses it in the context of factors affecting CPU performance. You should be able to describe what pipelining is, explain how it improves throughput, and identify at least one hazard that can reduce its effectiveness.
Does pipelining make individual instructions faster?
No — pipelining does not reduce the time any single instruction takes to complete (latency). The instruction still passes through every stage. What pipelining improves is throughput — the number of instructions completed per unit time. The car wash analogy makes this clear: each car still takes the same total time to wash, but more cars are processed per hour.
What is a pipeline stall?
A pipeline stall (also called a bubble) occurs when a hazard prevents the next stage from proceeding. The CPU inserts one or more NOP (no operation) cycles — effectively doing nothing — to wait until the hazard is resolved. Stalls reduce throughput because the pipeline is not fully occupied during those cycles. Compilers and hardware engineers work to minimise stalls through instruction reordering and forwarding.
How does branch prediction work?
Branch prediction is a technique where the CPU guesses which path will be taken at a conditional branch (IF statement) before the condition has been evaluated. The simplest predictor always predicts "not taken" (continue sequentially). More sophisticated predictors track the history of recent branch outcomes to improve accuracy. When the prediction is correct, execution continues without a stall. When it is wrong, the incorrectly prefetched instructions are flushed and the correct path is loaded — wasting several clock cycles.
For help with CPU architecture questions — from pipelining to the fetch–decode–execute cycle — Professor Turing at aitutors.me will build your understanding systematically.