Video Lectures on CUDA: https://lnkd.in/eaxYNCJV
The supplementary material for the lecture series, titled "Programming Massively Parallel Processors: A Hands-on Approach," provides a thorough guide for both beginners and advanced learners interested in CUDA programming and PyTorch integration. Spanning various topics, the lectures cover profiling and integrating CUDA kernels in PyTorch, an introduction to compute and memory architecture, advanced quantization, CUDA performance checklists, and building production-ready CUDA libraries. Each lecture includes videos, notebooks, slides, and code available in designated folders, ensuring comprehensive learning resources. Notable lectures feature speakers such as Mark Saroufim, Jeremy Howard, Thomas Viehmann, and Andreas Koepf, with dates ranging from January to May 2024. Key sessions include optimizing PyTorch optimizers, GPU collective communication with NCCL, and working with fused kernels. These materials are designed to equip learners with practical skills and deep understanding, enabling effective use of CUDA and PyTorch for enhanced programming capabilities and performance optimization.
The supplementary material for the lecture series, based on the "Programming Massively Parallel Processors: A Hands-on Approach" book, offers a comprehensive guide for both beginners and advanced learners interested in CUDA programming and PyTorch integration. Below is a detailed breakdown of the lecture content:
Lecture 1: Profiling and Integrating CUDA Kernels in PyTorch
Lecture 2: Recap Ch. 1-3 from the PMPP book
Lecture 3: Getting Started With CUDA
Lecture 4: Intro to Compute and Memory Architecture
Lecture 5: Going Further with CUDA for Python Programmers
Lecture 6: Optimizing PyTorch Optimizers
Lecture 7: Advanced Quantization
Lecture 8: CUDA Performance Checklist
Lecture 9: Reductions ...