A high processing speed is the key factor for a powerful computer. Many people might think that a mere increase in the circuit speed, or the clock speed, adds power to a computer, but it is only one of many ways of doing so. In fact, if the system is bottlenecked by other factors, increasing clock speed may not help much. That means, a well designed Pentium 133 MHz machine can be faster than a poorly designed Pentium 1 66 MHz. One of such a bottle-neck is caused by memory latency, the time that elapses between a request by a processor to a memory unit for a block of information and the receipt of the requested item by the processor. In the last few years, the processor speed has tremendously increased, but the memory technology could not keep the equivalent pace. This disparity in the processor speed and memory speed naturally imposes a technology induced latency, affecting the system performance. Therefore, the designers use a memory hierarchy so as to minimize memory latency. The objective of this memory hierarchy is to establish memory sub-system that appears to have the capacity of the secondary storage (virtually unlimited capacity), and the speed of the fastest component. This is achieved by using a combination of memory system components as shown in figure,

In digital systems, the memory essentially means Random Access Memory, or the RAM. In fact, the Read Only Memory, or the ROM falls in a category of storage. With a RAM, the contents of any one location can be accessed in essentially the same time as can the contents of any other location chosen at random. Technically, there are two types of RAM cell, the Static RAM and the Dynamic RAM. Static RAM is comparatively faster than Dynamic RAM, but it has high power dissipation and low packaging density. Since the same amount of SRAM occupies larger chip area than DRAM does; SRAM is comparatively expensive too, and are unaffordable to be used as the main memory in the computer. Therefore, DRAM is used as the main memory; SIMM, DIMM, FPM, EDO memory, SDRAM, etc. are all different varieties of DRAM.

But as we learnt earlier, DRAMs are very slow for the processor, and do impose memory latency if used directly with the processor. To reduce memory latency, SRAM' can be used one level north of DRAM. Cache memory, which is used to store information that is frequently access by the processor, is made of SRAM. Therefore, when the processor requests for some information and finds it in the cache memory, it doesn't have to access the slower DRAM, and the entire process becomes faster. In this way cache memory accelerates the operating speed of the system. A PC without cache can never match the performance of a PC with cache due to this reason. But because of the price constraint, a PC with more than 256-512 kB of off-chip, or L2, cache is rarely seen.

    In the last few years, processor manufacturers have started integrating cache memory on the chip itself too, and is termed as on-chip, or L1, cache. Since the processor does not have to cross its boundary to access L1. cache, information can ~ retrieved faster from the L1 cache than from the L2 cache. But even L1 cache doesn't operate at the same speed as the processor does, Recently, processor with on-chip L2 cache, such as the Pentium Pro, has also shown up in the market.

Unlike the main memory and cache memory, there are memory components called registers inside the processor that do operate at the processor speed. This means the performance of the processor will be optimum if all information that is requested by the processor can be supplied by the registers themselves. But the number of registers within a processor is limited because of the chip area available. A Pentium processor, for example, has 16 registers only.
With the implementation of memory hierarchy, whenever any information is required, the processor first looks into the registers, which operate at the same speed as the processor. If the information is not found in the-registers, only then it looks into the on-chip cache, which is slightly slower, then into off-chip cache, that is more slower (typically, 10-15 nS access time). If the information required is not found even there, only then the main memory is accessed, which has typically 60-70 nS access time.

If the information is not even found in the main memory, an access to the storage device is unavoidable. Even the fastest storage device is too slow as compared to main memory; Hence accessing storage for a processor request severely degrades the system performance. As a remedy to this problem, virtual memory, which is the hard disk space used as memory by the operating system, is used. Due to intelligent organization, accessing the same hard-disk as virtual memory is comparatively efficient than accessing it as a storage device.

In this way, the use of memory hierarchy provides a computer system with a memory sub-system that has a maximum speed as fast as the speed of the processor itself, as well as the capacity comparable to the storage device. The memory hierarchy improves the overall access time, at the same time, improving the overall capacity of the memory sub-system.