Learn more about the Memory Wall here → https://ibm.biz/~NjOWdJHJy
More GPU compute doesn't always mean faster AI models.
Kaoutar El Maghraoui explains the memory wall, arithmetic intensity, and why LLM inference can become memory-bound or compute-bound.
Learn how memory bandwidth, batching, and the roofline model impact AI performance.
00:00 Why Faster Chips Don’t Always Make AI Workloads Faster
00:32 The Memory Wall: The Hidden Bottleneck in Modern Computing
01:00 Compute Growth vs Memory Bandwidth Explained
02:39 Arithmetic Intensity: The Metric That Predicts Performance
03:48 Understanding the Roofline Model for AI Optimization
05:27 Compute-Bound vs Memory-Bound Workloads Explained
08:25 LLM Decode vs Prefill: Why Performance Varies
09:48 How Batching Improves AI Inference Efficiency
10:35 Why Leading LLM Serving Systems Rely on Continuous Batching
11:22 Practical Steps to Diagnose and Optimize AI Performance
AI news moves fast. Sign up for a monthly newsletter for AI updates from IBM → https://ibm.biz/~nnWVjebwV
AI was used in the creation of the transcript and metadata for this video.
#aimodels #aiaccelerator #ai #machinelearning
---------------------------------------------------------------------------------------------------------
Find us on YouTube:
? IBM Technology: https://www.youtube.com/@IBMTechnology
? IBM: https://www.youtube.com/@IBM
? IBM Developer: https://youtube.com/@IBMDeveloperAdvocates