CISC 811 - High Performance Computing

PREREQUISITE: Permission of the Instructor for students not registered in the CSE Master's specialization.

This course covers the principles and practices of current high performance computing architectures and software, with a focus on applications and performance. Shared-memory parallel, distributed-memory parallel, and grid computing architectures, and their associated programming paradigms will be reviewed. The HPC productivity crisis and possible solutions will be examined. Contemporary tools for analyzing, debugging and optimizing the performance of algorithms relevant to computational science will be introduced. High performance libraries and development frameworks will also be covered.

Syllabus:

1. Performance issues and measurement: Profiling and development tools, Sustained versus peak performance

2. High performance sequential computing: Effects of the memory hierarchy, Out-of-order execution, superscalar processors

3. Vector processing

4. Shared-memory processing: Architectures (extensions of the memory hierarchy), Programming paradigms, OpenMP

5. Distributed-memory processing: Architectural issues (networks and interconnects), Programming paradigms, MPI (+MPI2)

6. Grids: Computational grids, Data grids

7. Performance libraries and packages

8. The productivity crisis & future directions: Development overheads, Petaflops programming, New parallel languages: UPC, Titanium, Co-Array FORTRAN