A computer's memory is not a single uniform store — it is a hierarchy of different types, each trading speed against capacity and cost. At the top sit the CPU's registers: tiny, blindingly fast, and expensive per byte. Below them is cache, then RAM, and finally secondary storage — vast and cheap but thousands of times slower than registers.
Why does a memory hierarchy exist?
In an ideal world, all of a computer's memory would be as fast as its processor. In reality, fast memory is expensive and physically large, while cheap memory is slow. A 1 TB solid-state drive costs around £70 but takes milliseconds to access; 1 TB of SRAM (the technology used in cache) would cost millions of pounds and physically fill a room.
The memory hierarchy is a pragmatic solution: keep a tiny amount of very fast memory close to the CPU for data it is using right now, while holding large amounts of data in cheaper, slower storage further away. The system works because of locality of reference — programs tend to use the same data and nearby data repeatedly in a short time, so recently used data is likely to be needed again soon.
What are the four levels of the hierarchy?
| Level | Type | Speed | Typical size | Cost per GB | Volatile? |
|---|---|---|---|---|---|
| 1 (fastest) | CPU Registers | < 1 ns | < 1 KB | Extremely high | Yes |
| 2 | Cache (L1/L2/L3) | 1–10 ns | 32 KB – 64 MB | Very high | Yes |
| 3 | RAM | 50–100 ns | 4–64 GB | Moderate (~£5/GB) | Yes |
| 4 (slowest) | Secondary storage (HDD/SSD) | 0.1–10 ms | 256 GB – 20 TB | Low (< £0.05/GB) | No |
Volatile means the memory loses its contents when power is removed. Registers, cache, and RAM are all volatile; secondary storage is non-volatile.
What are CPU registers?
Registers are the CPU's own internal storage locations — a handful of tiny, extremely fast memory cells built directly into the processor chip. The CPU can only operate on data that is currently in a register; all computation happens here.
Key registers for GCSE include:
- Program Counter (PC): holds the memory address of the next instruction to fetch.
- Memory Address Register (MAR): holds the address in RAM that the CPU wants to read from or write to.
- Memory Data Register (MDR): holds the data being transferred between the CPU and RAM.
- Accumulator (ACC): stores the result of the most recent arithmetic or logic operation.
Registers are measured in bits (e.g. 64-bit registers in modern CPUs) and there are typically fewer than 32 of them in a general-purpose processor.
What is cache memory?
Cache is a small, fast memory that sits between the CPU registers and RAM. It stores copies of data and instructions that the CPU has recently used or is likely to need soon. When the CPU requests data, it first checks the cache; if the data is there (a cache hit), it is delivered in nanoseconds. If not (a cache miss), the CPU must wait for a slower RAM access.
Modern CPUs have multiple cache levels:
- L1 cache: smallest (32–64 KB per core) and fastest; built directly into each CPU core.
- L2 cache: larger (256 KB – 1 MB per core) and slightly slower.
- L3 cache: shared among all cores; larger (8–64 MB) but slower than L2.
Cache operates automatically — the programmer does not control which data is cached. The CPU uses algorithms (like LRU — Least Recently Used) to decide which data to keep in cache and which to evict when space is needed.
What is RAM and what does "volatile" mean?
RAM (Random Access Memory) is the computer's main working memory — where programs and their data live while they are running. When you open a document, it is loaded from the hard drive into RAM, where the processor can access it quickly. Closing the document (or switching off the computer) removes it from RAM.
RAM is volatile: it loses its contents without a power supply. This is why unsaved work is lost when the computer crashes or power is cut.
Modern computers typically have 8–32 GB of RAM, at access speeds of 50–100 nanoseconds. This is fast enough for most tasks, but still 10–100× slower than L1 cache.
What is secondary storage and why is it at the bottom?
Secondary storage includes hard disc drives (HDDs), solid-state drives (SSDs), USB flash drives, and optical discs (CD/DVD). It is non-volatile — data persists without power — making it suitable for long-term storage of the operating system, programs, and user files.
The trade-off is access speed: an HDD takes 5–10 ms per access (involving mechanical movement of read/write heads); even a fast NVMe SSD takes 0.1–0.2 ms — still 1,000× slower than RAM. This is why loading a large program from disc into RAM before running it is essential — running directly from a hard drive would be intolerably slow.
What is locality of reference?
The memory hierarchy works because of two types of locality:
- Temporal locality: recently used data is likely to be used again soon (a variable in a loop is accessed many times).
- Spatial locality: data near recently used data is likely to be needed soon (the next instruction in a sequence; the next element of an array).
Cache exploits both: when a cache miss occurs, the CPU fetches not just the one needed byte but an entire cache line (typically 64 bytes) of surrounding memory — anticipating spatial locality.
Frequently asked questions
Why does adding more RAM usually speed up a computer?
When RAM is full, the operating system uses virtual memory — it swaps less-used pages of RAM out to a disc file and brings them back when needed. Disc access is thousands of times slower than RAM, so heavy use of virtual memory (called thrashing) makes a computer feel very slow. More RAM means less reliance on virtual memory, keeping more data at RAM speed.
Is ROM (Read-Only Memory) part of the memory hierarchy?
ROM is not typically part of the main memory hierarchy diagram. ROM stores the firmware (such as the BIOS/UEFI), which must be available immediately on power-up before the OS loads. It is non-volatile and non-writable (in its classic form). Flash memory used in modern BIOS chips is technically rewritable, but ROM still sits outside the main hierarchy because its role is bootstrap, not runtime data storage.
What is the difference between L1, L2, and L3 cache?
The numbers indicate proximity to the CPU core and speed. L1 is per-core, smallest, fastest, and checked first. On a miss, the CPU checks L2 (also per-core, larger, slightly slower). On an L2 miss, it checks L3 (shared across all cores, larger still, slower). On an L3 miss, the CPU fetches from RAM. Modern CPUs check all three levels in nanoseconds; only the RAM access takes tens of nanoseconds. Having multiple levels gives a useful balance between the cost of very fast small memory (L1) and the speed advantage of having more data cached (L3).
How does the memory hierarchy relate to the fetch-decode-execute cycle?
During the fetch phase, the CPU reads the address in the Program Counter and fetches the instruction — first checking cache, then RAM if needed. During execute, operands are fetched from registers (if already there) or loaded from cache/RAM into registers. The hierarchy exists precisely to keep the FDE cycle supplied with data as quickly as possible: if every instruction required a RAM access, the CPU would spend most of its time waiting.
Master the memory hierarchy, registers, and the FDE cycle with Professor Turing's structured hints at aitutors.me.