Effective On Chip Cache Utilization In Chip Multiprocessors

"CMOS scaling trends allow increasing numbers of transistors on a single chip but with a limited power budget. Processor designers are increasingly turning toward multicore architectures- often chip multiprocessor (CMP) of simultaneous multithreaded (SMT) cores- in order to leverage these trends. However, increasing the number of cores on a single chip leads to higher demand on the on-chip cache capacity as well as on both on-chip and off-chip bandwidth due to coherence and capacity-related misses, respectively. Cache access latencies are also often a function of distance on the chip. Directory-based cache coherence protocols can support a large number of cores by reducing coherence bandwidth requirements but they introduce a level of indirection on the critical path of cache misses, resulting in increased communication latency depending on where data and coherence information are mapped. Many multithreaded commercial, scientific, and data mining workloads exhibit finegrain (both temporal and spatial) data sharing patterns due to data communication and synchronization. In addition, multiprogrammed and single-threaded applications, while exhibiting limited sharing behavior, may have working sets that well exceed the onchip cache capacity. On-chip caches must therefore adapt to these varying needs in order to reduce L1 miss penalties and both on-chip and off-chip bandwidth needs for all application domains. In this dissertation, we propose and evaluate cache coherence protocols that (1) exploit the low-latency on-chip interconnect to solve the directory-based indirection problem by using prediction to directly access the most up-to-date copy of the data, (2) support fine-grain sharing by localizing communication between the closest sharing nodes, (3) reduce access latency by bringing both data and metadata as close to the accesser as possible, and (4) increase effective cache capacity by reducing the number of copies of data in the caches and using access pattern aware adaptive replacement policies. We show that our techniques are effective at improving cache utilization and at reducing both on- and off-chip traffic and energy consumption. These properties are essential to ensure the continued scaling of future multi-core platforms."--Leaves vi-vii.