CUDA Programming
A Developer's Guide to Parallel Computing with GPUs
- 1st Edition - November 13, 2012
- Latest edition
- Author: Shane Cook
- Language: English
If you need to learn CUDA but don't have experience with parallel computing, CUDA Programming: A Developer's Introduction offers a detailed guide to CUDA with a grounding in parall… Read more
If you need to learn CUDA but don't have experience with parallel computing, CUDA Programming: A Developer's Introduction offers a detailed guide to CUDA with a grounding in parallel fundamentals. It starts by introducing CUDA and bringing you up to speed on GPU parallelism and hardware, then delving into CUDA installation. Chapters on core concepts including threads, blocks, grids, and memory focus on both parallel and CUDA-specific issues. Later, the book demonstrates CUDA in practice for optimizing applications, adjusting to new hardware, and solving common problems.
- Comprehensive introduction to parallel programming with CUDA, for readers new to both
- Detailed instructions help readers optimize the CUDA software development kit
- Practical techniques illustrate working with memory, threads, algorithms, resources, and more
- Covers CUDA on multiple hardware platforms: Mac, Linux and Windows with several NVIDIA chipsets
- Each chapter includes exercises to test reader knowledge
Software engineers, programmers, hardware engineers, students / advanced students
Preface
Chapter 1. A Short History of Supercomputing
Introduction
Von Neumann Architecture
Cray
Connection Machine
Cell Processor
Multinode Computing
The Early Days of Gpgpu Coding
The Death of the Single-Core Solution
Nvidia and Cuda
Gpu Hardware
Alternatives to Cuda
Conclusion
Chapter 2. Understanding Parallelism with GPUs
Introduction
Traditional Serial Code
Serial/Parallel Problems
Concurrency
Types of Parallelism
Flynn’s Taxonomy
Some Common Parallel Patterns
Conclusion
Chapter 3. CUDA Hardware Overview
PC Architecture
GPU Hardware
CPUs and GPUs
Compute Levels
Chapter 4. Setting Up CUDA
Introduction
Installing the Sdk Under Windows
Visual Studio
Linux
Mac
Installing a Debugger
Compilation Model
Error Handling
Conclusion
Chapter 5. Grids, Blocks, and Threads
What it all Means
Threads
Blocks
Grids
Warps
Block Scheduling
A Practical Example—Histograms
Conclusion
Chapter 6. Memory Handling with CUDA
Introduction
Caches
Register Usage
Shared Memory
Constant Memory
Global Memory
Texture Memory
Conclusion
Chapter 7. Using CUDA in Practice
Introduction
Serial and Parallel Code
Processing Datasets
Profiling
An Example Using AES
Conclusion
References
Chapter 8. Multi-CPU and Multi-GPU Solutions
Introduction
Locality
Multi-CPU Systems
Multi-GPU Systems
Algorithms on Multiple GPUS
Which GPU?
Single-Node Systems
Streams
Multiple-Node Systems
Conclusion
Chapter 9. Optimizing Your Application
Strategy 1: Parallel/Serial GPU/CPU Problem Breakdown
Strategy 2: Memory Considerations
Strategy 3: Transfers
Strategy 4: Thread Usage, Calculations, and Divergence
Strategy 5: Algorithms
Strategy 6: Resource Contentions
Strategy 7: Self-Tuning Applications
Conclusion
Chapter 10. Libraries and SDK
Introduction
Libraries
CUDA Computing SDK
Directive-Based Programming
Writing Your Own Kernels
Conclusion
Chapter 11. Designing GPU-Based Systems
Introduction
CPU Processor
GPU Device
PCI-E Bus
GeForce cards
CPU Memory
Air Cooling
Liquid Cooling
Desktop Cases and Motherboards
Mass Storage
Power Considerations
Operating Systems
Conclusion
Chapter 12. Common Problems, Causes, and Solutions
Introduction
Errors With CUDA Directives
Parallel Programming Issues
Algorithmic Issues
Finding and Avoiding Errors
Developing for Future GPUs
Further Resources
Conclusion
References
Index
"I must mention chapters 7, which deals with the practicalities of using the SDK, and 9, which offers advice and a detailed breakdown of areas that can limit the performance of a CUDA application. Together, these chapters transform this good book into the kind of excellent text that all CUDA developers can find useful, regardless of their relative experience."—ComputingReviews.com, July 12, 2013
"This book is one of the most comprehensive on the subject published to date…it will guide those acquainted with GPU/CUDA from other books or from NVIDIA product documentation through the optimization maze to efficient CUDA/GPU coding."—ComputingReviews.com, April 25, 2013
- Edition: 1
- Latest edition
- Published: November 13, 2012
- Language: English
SC
Shane Cook
Shane Cook is Technical Director at CUDA Developer, a consultancy company that helps companies exploit the power of GPUs by re-engineering code to make the optimal use of the hardware available. He formed CUDA Developer upon realizing the potential of heterogeneous systems and CUDA to disrupt existing serial and parallel programming technologies. He has a degree in Applied Software Engineering, specializing in the embedded software field. He has worked in senior roles with blue chip companies over the past twenty years, always seeking to help to develop the engineers in his team. He has worked on C programming standards including the MISRA Safer C used by widely in the automotive software community, and previously developed code for companies in the Germany automotive and defense contracting industries as well as Nortel and Ford Motor Company.
Affiliations and expertise
Technical Director, CUDA Developer, Munich, GermanyRead CUDA Programming on ScienceDirect