CCO - 129 - 7 - Introduction to High Performance Computing

Total of Credits: 8
Hours for Theoretical Classes: 60
Hours for Exercises or Seminars: 60

Objective

This course presents an introduction to High Performance Computing (HPC), with a focus on practical aspects of parallel and distributed programming on the main HPC platforms available nowadays. This course is an introduction to High Performance Computing (HPC) with focus on scalable parallel algorithms and their efficient implementation on modern machines. Scale refers to being efficient as both the problem size grows, and efficient as the system size (i.e., the number of processing cores) grows. The course covers from basic algorithmic design to the implementation on the major HPC programming styles: data parallelism, and task parallelism on both shared- and distributed-memory programming.

Catalog Description

  • HPC architectures, parallel programming, performance and scalability
  • Principles of parallel algorithms design. Problem decomposition
  • Programming with shared memory. Processes, communication and synchronization (IPC). Threads, communication and synchronization in shared memory. OpenMP
  • Programming with distributed memory. Message Passing (MPI). Other models
  • Programming GPUs
  • Understanding computing in Clusters, Supercomputers, Clouds

Main Bibliography

  1. Grama, A.; Gupta, A.; Karypis, G.; Kumar, V. Introduction to Parallel Computing. Adisson-Wesley, 2003.
  2. Foster, I. Designing and Building Parallel Programs. MIT Press 1999. Disponível em: www-unix.mcs.anl.gov/dbpp.

Complementary Bibliography

  1. William Gropp, Ewing Lusk, Anthony Skjellum. Using MPI, third edition: Portable Parallel Programming with the Message-Passing Interface, 3 rd edition. The MIT Press, 2014. ISBN-10: 0262527391.
  2. Timothy G. Mattson, Yun He, Alice E. Koniges. The OpenMP Common Core: Making OpenMP Simple Again, 2019. The MIT Press. ISBN-10: 0262538865.
  3. Ruud Van Der Pas, Eric Stotzer, Christian Terboven. Using OpenMP-The Next Step: Affinity, Accelerators, Tasking, and SIMD. The MIT Press, 2017. ISBN-10 : 0262534789.
  4. Andrews, G. Foundations of Multithreaded, Parallel, and Distributed Programming, Addison Wesley, 2000. MPDbook.
  5. Chapman, B.; Jost, G. and van der Pas, R. Using OpenMP: Portable Shared Memory Parallel Programming. MIT Press, 2007.
  6. David B. Kirk, Wen-mei W. Hwu. Programming Massively Parallel Processors: A Hands-on Approach, 3rd Edition. Morgan Kaufmann, 2016. ISBN-10: 0128119861.