(This section shows how the program representation and its use of architectural registers of the system. Tools like ftrace can be used to verify consolidate the understanding.) Program Encodings Data Formats Accessing Information Arithmetic and Logical Operations Control Procedures Array Allocation and Access Heterogeneous Data Structures Combining Control and Data in Machine-Level Programs Floating-Point Code Tool: ftrace to trace program flow

(This section establishes the core concepts of the course. Focuses on the internals of a microprocessor core, especially the flow of instructions and code. This will be the most involving and eventually the most satisfying section. Post this section, one will be able to visualize program execution on a CPU quite clearly. ) Von Neumann architectures Modern processors Instruction Set Architecture Assembly View Layers of Abstraction CISC vs RISC Hardware Structure Hardware Stages PipeLines Real World Pipelines Computational Example 3-Way Pipeline Operating a Pipeline Non Uniform Delays Register Overhead Data Dependencies Data Hazards Data Dependencies in Processors Pipeline Demonstration Nops Stalling for Data Dependency Stall Conditions Detecting Stall Conditions What Happens When Stalling? Data Forwarding Data Forwarding Example Forwarding Priority Limitation of Priority Avoiding Load/Use Hazard Detecting Load/Use Hazard Control of Load/Use Hazard Modern CPU Design Instruction Control Execution units Superscalar Units Superscalar Execution In-Order Superscalar Processor Example Superscalar Performance with Dependencies Superscalar Execution Tradeoffs Branch Prediction The Branch Problem Importance of The Branch Problem Branch Prediction Branch Prediction: Guess the Next Instruction to Fetch Misprediction Penalty Simplest: Always Guess NextPC = PC + 4 Pipeline Flush on a Misprediction Performance Analysis Reducing Branch Misprediction Penalty Two-Level Prediction Global Branch Correlation Hybrid Branch Predictors Case Study Sandy Bridge Haswell Tools: Perf for event based hardware profiling and an even more fingrained tool (overseer) that can be integrated into the application.

(Microoptimizing Code for Performance, based on Section-2. This is specifically to verify the theories and assumptions in Section-2. For e.g. utilization of one’s knowledge of CPU pipeline and superscalar architecture to gain performance.) Optimization Realities Optimizing Compilers Limitations of Optimizing Compilers Optimization Blockers Memory Aliasing Procedure Calls Cycles Per Element (CPE) Optimization examples Removing loop inefficiency Procedure Calls Lower Case Conversion Performance Convert Loop To Goto Form Understanding Modern CPU:Haswell Loop Unrolling Going Superscalar Re-association SSE and Friends Limiting Factors Branch Prediction RegisterSpilling

(Solid idea of RAM, Cache, and Disk. More importantly, exploit the cache with cache-aware data structures and hide the CPU-memory GAP. Similarly, the OS page cache to DISK read/write gap.) (This section is to understand "CPU Memory Gap". This can be plugged with cache.) Memory hierarchies RAM SRAM vs DRAM Non Volatile Memory Traditional Bus Structure Connecting CPU and Memory Memory Read Transaction Memory Write Transaction CPU Memory Gap Locality to the Rescue! Qualitative Estimates of Locality Memory Hierarchies Example Memory Hierarchy Conventional DRAM Organization Reading DRAM Supercell Memory Modules Enhanced DRAMs Storage Trends Clock Rates Caches General Cache Concepts Types of Cache Misses Examples of Caching in the Mem. Hierarchy General Cache Organization Cache Reads Direct Mapped Cache Direct-Mapped Cache Simulation E-way Set Associative Cache What about writes? Intel Core i7 Cache Hierarchy Cache Performance Metrics Writing Cache Friendly Code The Memory Mountain Memory Mountain Test Matrix Multiplication Example Miss Rate Analysis for Matrix Multiply Core i7 Matrix Multiply Performance Layout of C Arrays in Memory (review) Cache Miss Analysis Blocked Matrix Multiplication Disk What’s Inside A Disk Drive? Disk Geometry Disk Geometry (Muliple-Platter View) Disk Capacity Recording zones Tools: Perf hardware profiling to measure various cache/memory/disk metrics. This is done to check if a particular design is taking effect. EBPF based customized tools can be built to show the impact of the above.

(Prefetchers can optimize the memory reads if the read patterns can be understood.) Tolerating Memory Latency Caching Prefetching Multithreading Out-Of-Order Execution Prefetching and Correctness How a HW Prefetcher Fits in the Memory System Prefetching: The Four Questions Software Prefetching X86 PREFETCH Instruction Next-Line Prefetchers Stride Prefetchers Instruction Based Stride Prefetching Prefetcher Performance Tool:Perf hardware profiling to measure various cache/memory/disk metrics. This is done to check if a particular design is taking effect. EBPF based customized tools can be built to show the impact of the above. *exact metrics can differ.

(Virtual memory has a lot of uses but here we explore the optimizations done for disk reads and writes using OS page cache.) A System Using Physical Addressing Address Spaces Why Virtual Memory (VM)? VM as a Tool for Caching DRAM Cache Organization Enabling Data Structure: Page Table Page Hit Page Fault Handling Page Fault Allocating Pages Locality to the Rescue Again! VM as a Tool for Memory Management Simplifying Linking and Loading VM as a Tool for Memory Protection VM Address Translation Address Translation: Page Hit Address Translation: Page Fault Integrating VM and Cache Speeding up Translation with a TLB Accessing the TLB TLB Hit TLB Miss Programmers View of Virtual Memory System's View of Virtual Memor Tool: Same as section-4 but exact metrics can differ.

(Linux interface to CPU and how can we use it to improve performance(Latency and throughput). Understanding the CPU driver in Linux. Understanding the process of OS interfacing with the CPU.) Core Sandy Bridge Pipeline:Frontend(Instruction load, decode, cache) Sandy Bridge Pipeline:Execution Sandy Bridge Pipeline:Backend (Data load and Store) Haswell Pipeline Uncore L3 Cache Integrated Graphics Integrated memory controller QuickPath Interconnect Linux Inteface to CPUIDLE CPUIDLE subsystem CPUIDLE subsystem:Driver load CPUIDLE subsystem:Call the Driver CPUIDLE subsystem:Governor CPUIDLE subsystem:Gathering and undertanding latency data

(This is the practical part of section-7. Power and performance (Latency and throughput) are interrelated. This section explores the trade-off with examples.) Power Power:Turn things off Power:c-states Power:Tuned Power:Tuned:c-states requests Power:Tuned:Hardware State Residency Power:Tuned:Influx:Grafana:c-states Power:Tuned:Measuring Latency Power:Tuned:Hardware Latency Power:Tuned:Wakeup Latency Power:Tuned:Influx:Grafana:latency Power:PMQOS Power:Turn things down Power:Turn things down:P-states:Hardware Latency Core and Uncore:Uncore Core and Uncore:Uncore:montioring and Tuning Core and Uncore:Uncore:montioring and Tuning:Hardware Latency Core and Uncore:Uncore:montioring and Tuning:Wakeup Latency Core and Uncore:Uncore:montioring and Tuning:Application Latency Tools: The primary tool here is ftrace but it majorly used to gather data as it is the lowest latency flow-based tracer. This data is then post-processed to visualize it.

(Multiprocessor systems and the effects of cache coherency. Multithreaded programming requires the idea of shared data in the presence of multicore systems. ) Introduction Multiprocessing Cache Coherence Flynn’s Taxonomy of Computers Why Parallel Computers? Types of Parallelism and How to Exploit Them Task-Level Parallelism: Creating Tasks Multiprocessing Fundamentals Multiprocessor Types Main Issues in Tightly-Coupled MP Hardware-based Multithreading Parallel Speedup Example Speedup with N Processors Revisiting the Single-Processor Algorithm Superlinear Speedup Utilization, Redundancy, Efficiency Utilization of a Multiprocessor Caveats of Parallelism Amdahl’s Law Sequential Bottleneck Why the Sequential Bottleneck? Bottlenecks in Parallel Portion Cache Coherence Introduction Multi-Core Cache Coherence Memory Ordering in Multiprocessors Ordering of Operations Memory Ordering in a Single Processor Memory Ordering in a Dataflow Processor Memory Ordering in a MIMD Processor Why Does This Even Matter? Protecting Shared Data Supporting Mutual Exclusion Sequential Consistency Programmer’s Abstraction Issues with Sequential Consistency? Weaker Memory Consistency Tradeoffs: Weaker Consistency Cache Coherence Shared Memory Model The Cache Coherence Problem Hardware Cache Coherence Two Cache Coherence Methods Snoopy Cache Coherence MESI Tools: Measure the effects of cache coherency with tools like perf, Solaris Analyzer, and overseer e.t.c.

Tools:EBPF (One of the best tool for customizing tracing of kernel code. This gives Linux tracing superpowers beyond even Solaris.) Tools:Ftrace (The best (least overhead) software flow-based tracer for Linux. It is a part of Linux Kernel and its official tracer.) Tools:Perf (One of the best event-based profilers in the business.) Tools:Solaris Analyzer Tools:Overseer Tools:pcm-master Tools:Core-freq Tools:Powertop Tools:Turbostat

(In this section we change the approach a bit. We take an open-source, high performance messaging framework and dissect it. We see an industry-standard implementation of the majority of the concepts below, including queuing theory, event-based architecture, etc. On one hand, we explore the adaptation of the framework on Solarflare like hardware, and on the other hand, we explore making the framework reactive.) Introduction Hardware Networking stack Software Networking stack Modes of parallelism Distributed Memory Models Understanding the cost of Communication vs Computation OpenMP, MPI overview Scheduling considerations for distributed computing Intro to Queueing Theory Latency vs Throughput Scheduling to meet SLAs Fault-Tolerant execution Approaches to distributed computing Message Passing Event-driven approach to grid computing