High Performance Computing
Modern Systems and Practices
- 2nd Edition - September 19, 2024
- Imprint: Morgan Kaufmann
- Authors: Thomas Sterling, Maciej Brodowicz, Matthew Anderson
- Language: English
- Paperback ISBN:9 7 8 - 0 - 1 2 - 8 2 3 0 3 5 - 0
- eBook ISBN:9 7 8 - 0 - 3 2 3 - 9 0 2 1 2 - 0
Performance Computing: Modern Systems and Practices is a fully comprehensive and easily accessible treatment of high performance computing, covering fundamental concepts and essen… Read more
Purchase options
Institutional subscription on ScienceDirect
Request a sales quotePerformance Computing: Modern Systems and Practices is a fully comprehensive and easily accessible treatment of high performance computing, covering fundamental concepts and essential knowledge while also providing key skills training. With this book, students will begin their careers with an understanding of possible directions for future research and development in HPC, domain scientists will learn how to use supercomputers as a key tool in their quest for new knowledge, and practicing engineers will discover how supercomputers can employ HPC systems and methods to the design and simulation of innovative products.
This new edition has been fully updated, and has been reorganized and restructured to improve accessibility for undergraduate students while also adding trending content such as machine learning and a new chapter on CUDA.
- Covers enabling technologies, system architectures and operating systems, parallel programming languages and algorithms, scientific visualization, correctness and performance debugging tools and methods, GPU accelerators, and big data problems
- Provides numerous examples that explore the basics of supercomputing while also providing practical training in the real use of high-end computers
- Helps users with informative and practical examples that build knowledge and skills through incremental steps
- Features sidebars of background and context to present a live history and culture of this unique field
Students enrolled in upper level parallel and high-performance computing courses in the US
- Cover image
- Title page
- Table of Contents
- Copyright
- Dedication to Paul Messina
- Foreword
- Preface
- Acknowledgments
- Chapter 1. An Introduction to High Performance Computing and This Textbook
- 1.1. Preamble
- 1.2. An Introduction to High Performance Computing
- 1.3. Impact of Supercomputing on Science, Society, and Security
- 1.4. Supercomputer Performance
- 1.5. Sources of High Performance Computing Performance
- 1.6. A Brief Description of a High Performance Computer
- 1.7. An Overview of This Textbook
- 1.8. Summary and Outcomes
- Chapter 2. Commodity Clusters
- 2.1. Introduction
- 2.2. Hardware Architecture
- 2.3. Programming Interfaces
- 2.4. Software Environment
- 2.5. Basic Methods of Use
- 2.6. Summary and Outcomes
- Chapter 3. Foundational Concepts of High Performance Computing Through a Brief History
- 3.1. Introduction: The Hammer of the Mind
- 3.2. The Foundations of Calculation
- 3.3. Mechanical Calculator: Automation of Arithmetic
- 3.4. Fundamental Principles of Digital Computation
- 3.5. Performance
- 3.6. Summary and Outcomes
- Chapter 4. Models and Metrics for Machines
- 4.1. Introduction
- 4.2. Throughput Versus Time
- 4.3. SLOWER: Performance Factors
- 4.4. Amdahl’s Law
- 4.5. Memory Hierarchy
- 4.6. Summary and Outcomes
- Chapter 5. Benchmarking
- 5.1. Introduction
- 5.2. Key Properties of an HPC Benchmark
- 5.3. Standard HPC Community Benchmarks
- 5.4. Highly Parallel Computing Linpack
- 5.5. HPC Challenge Benchmark Suite
- 5.6. High Performance Conjugate Gradients
- 5.7. NAS Parallel Benchmarks
- 5.7. MLPerf
- 5.8. Graph500
- 5.9. Microbenchmarks and Miniapplications
- 5.10. Summary and Outcomes
- Chapter 6. Libraries
- 6.1. Introduction
- 6.2. The Basic Linear Algebra Subprograms
- 6.3. Linear Algebra PACKage
- 6.4. General Sparse Linear Algebra
- 6.5. Summary and Outcomes
- Chapter 7. The Essential SLURM: Resource Management
- 7.1. Managing Resources
- 7.2. The Essential Slurm
- 7.3. Summary and Outcomes
- Chapter 8. Visualization
- 8.1. Introduction
- 8.2. Foundational Visualization Concepts
- 8.3. Gnuplot
- 8.4. Matplotlib
- 8.5. The Visualization Toolkit
- 8.6. ParaView
- 8.7. VisIt
- 8.8. Visual Intelligence
- 8.9. Summary and Outcomes
- Chapter 9. Performance Monitoring and Measurement
- 9.1. Introduction
- 9.2. Time Measurement
- 9.3. Performance Profiling
- 9.4. Monitoring Hardware Events
- 9.5. Summary and Outcomes
- Chapter 10. Symmetric Multiprocessor Architecture
- 10.1. Introduction
- 10.2. Architecture Overview
- 10.3. Processor Core Architecture
- 10.4. Memory Hierarchy
- 10.5. External I/O Interfaces
- 10.6. Summary and Outcomes
- Chapter 11. The Essential OpenMP
- 11.1. Introduction
- 11.2. Overview of OpenMP Programming Model
- 11.3. Parallel Threads and Loops
- 11.4. Synchronization
- 11.5. Reduction
- 11.6. Summary and Outcomes
- 11.7. Exercises
- Chapter 12. Examples for Multiple-Thread Shared-Memory Programs
- 12.1. Solving the Black–Scholes Equation
- 12.2. Solving the Wave Equation
- 12.3. Summary and Outcomes
- Chapter 13. Distributed Memory Architecture
- 13.1. Introduction
- 13.2. Massively Parallel Processor
- 13.3. Node Microprocessor
- 13.4. Node Main Memory
- 13.5. Multicore Nodes
- 13.6. System Area Network and Message Passing
- 13.7. Clusters and Other Massively Parallel Processing Variants
- 13.8. Summary and Outcomes
- Chapter 14. The Essential MPI
- 14.1. Introduction
- 14.2. Message-Passing Interface Standards
- 14.3. Message-Passing Interface Basics
- 14.4. Communicators
- 14.5. Point-to-Point Messages
- 14.6. Synchronization Collectives
- 14.7. Communication Collectives
- 14.8. Summary and Outcomes
- Chapter 15. Examples for Message-Passing Programs
- 15.1. Solving a Stochastic Differential Equation With Runge–Kutta
- 15.2. Differentiating Field Variables by Finite Difference
- Chapter 16. Heterogeneous and Graphical Processor Unit Computer Architecture
- 16.1. Heterogeneous System Organization
- 16.2. Graphical Processing Unit Structure Overview
- 16.3. Graphical Processing Unit Memory Components
- 16.4. Parallel Flow Control
- 16.5. Central Processing Unit–Graphical Processing Unit Interoperability
- 16.6. Summary and Outcomes
- Chapter 17. The Essential OpenAC
- 17.1. Overview of GPU Programming
- 17.2. Programming Concepts
- 17.3. Compilation and Installation
- 17.4. Environment Variables
- 17.5. Library Calls
- 17.6. Directives
- 17.7. Data Movement Commands
- 17.8. Variable Scope
- 17.9. Synchronization Commands
- 17.10. Parallel Loops
- 17.11. Summary and Outcomes
- 17.12. Questions and Problems
- Chapter 18. Examples of GPU Programs
- 18.1. cuBLAS Example
- 18.2. Differentiating Field Variables by Finite Difference
- 18.3. Questions
- Chapter 19. Machine Learning
- 19.1. Introduction
- 19.2. Handwritten Digits
- 19.3. Distributed Machine Learning Training
- 19.4. Summary and Outcomes
- Chapter 20. Mass Storage and File Systems
- 20.1. Introduction
- 20.2. Brief History of Storage
- 20.3. Storage Device Technology
- 20.4. Role and Function of File Systems
- 20.5. Summary and Outcomes
- 20.6. Questions and Problems
- Chapter 21. Intermediate MPI
- 21.1. User-Defined Data Types
- 21.2. Enhanced Point-to-Point Communication
- 21.3. Custom Communicators and Topologies
- 21.4. Collective Operation Redux
- 21.5. Parallel Input/Output
- 21.6. Summary and Outcomes
- Chapter 22. Checkpointing
- 22.1. Introduction
- 22.2. System-Level Checkpointing
- 22.3. Application-Level Checkpointing
- 22.4. Machine Learning Checkpointing
- 22.5. Summary and Outcomes
- Chapter 23. Computing in the Future: A Personal Perspective
- 23.1. Introduction
- 23.2. Limitations of Modern High Performance Computing Systems and Methods
- 23.3. Cloud Computing
- 23.4. Memory-Centric Computing
- 23.5. Neuromorphic Computing
- 23.6. Analog Computing
- 23.7. Superconducting Supercomputing
- 23.8. Quantum Computing
- 23.9. Summary and Outcomes
- Index
- Edition: 2
- Published: September 19, 2024
- Imprint: Morgan Kaufmann
- No. of pages: 480
- Language: English
- Paperback ISBN: 9780128230350
- eBook ISBN: 9780323902120
TS
Thomas Sterling
Thomas Sterling is Professor of Informatics and Computing at Indiana University. He serves as the Executive Associate Director of the Center for Research in Extreme Scale Technologies (CREST) and as its Chief Scientist. He is most widely known for his pioneering work in commodity cluster computing as leader of the Beowulf Project for which he and colleagues were awarded the Gordon Bell Prize. Professor Sterling currently leads a team of researchers at IU to enable a new generation of extreme scale computing systems and applications. He is the co-author of six books and holds six patents. He has taught a graduate level course upon which this textbook will be heavily informed, five times.
Affiliations and expertise
Indiana University Bloomington, USAMB
Maciej Brodowicz
Maciej Brodowicz is Advanced Parallel Computing Engineer at the Center for Research in Extreme Scale Technologies (CREST) at Indiana University.
Affiliations and expertise
Indiana University Bloomington, USAMA
Matthew Anderson
Matthew Anderson works in the Department of English at the University of New England, Maine, USA.
Affiliations and expertise
Department of English, University of New England, Maine, USARead High Performance Computing on ScienceDirect