International Symposium on Field-Programmable Gate Arrays (ISFPGA)’s Post

Follow along with this tutorial on how to run Llama 2 inference on Intel Arc A-Series GPUs via Intel Extension for PyTorch. #oneapi #llama2 #pytorch #iamintel

🔍 Understanding the Difference: CPU vs. GPU The CPU is designed to handle a wide range of tasks. It excels at serial processing and is optimised for single-thread performance. Key characteristics include: 👉 Versatility: Executes general-purpose computing tasks. 👉 Core Count: Typically features fewer cores, but each core is powerful. 👉 Task Management: Ideal for tasks requiring complex computations. The GPU is highly efficient at parallel processing, making it indispensable for tasks involving massive datasets. Key attributes include: 👉 Specialisation: Optimised for graphics rendering and parallel tasks. 👉 Core Count: Contains thousands of smaller, efficient cores. 👉 Performance: Excels in handling repetitive calculations, making it ideal for applications in AI, machine learning, and scientific research. #GPU #CPU #Computing

I'm happy to share a new coding tutorial for doing fast matrix multiplication on #NVIDIA Hopper GPUs! This covers the warpgroup matrix-multiply-accumulate (WGMMA) instruction that specifically targets the Tensor Cores on Hopper GPUs. Using tools from the CUTLASS library, we go into detail on all aspects of correctly invoking WGMMA as a primitive for matmul when writing a CUDA kernel -- how tensor data should be laid out in memory for WGMMA, how to use CUTLASS to define these layouts of your data, and how to synchronize WGMMA as an async instruction to guard against race conditions and ensure correct behavior of your kernel. If you've read the blog post on FlashAttention-3, you'll know how heavily FA-3 exploits WGMMA -- both in terms of its higher throughput and asynchronous capabilities -- to achieve its impressive performance gains. Our hope is that this tutorial can help similarly unlock the potential of the Hopper architecture when coding up your own projects and research ideas! We're also planning at least two followups to this tutorial - one covering the overall structure of an efficient GEMM kernel with a focus on copy-compute overlapping techniques such as warp specialization, and another on persistent kernels and the Stream-K algorithm for GEMM. Work done in collaboration with my colleagues at Colfax and Hieu Pham. https://lnkd.in/g-tsnnha

Method identified to double computer processing speeds: Simultaneous and Heterogeneous Multithreading. The researchers describe their development of a proposed SHMT framework on an embedded system platform that simultaneously uses a multi-core ARM processor, an NVIDIA GPU, and a Tensor Processing Unit hardware accelerator. The system achieved a 1.96 times speedup and a 51% reduction in energy consumption. - https://lnkd.in/ddXSxSYu

Exciting Breakthrough in LLM Inference! 🚀 EmbeddedLLM has successfully ported vLLM to ROCm 5.6! We're thrilled to announce that LLM inference has achieved parity with Nvidia A100 using AMD MI210, a significant milestone in our journey towards high throughput LLM inference. 🌐💥 Why AMD? We're all-in with AMD because they have the ingredients to build the future of ubiquitous LLM machines. From edge devices to laptops, AMD's advanced CDNA3 GPU blocks and Zen 4 CPU blocks paired with high-bandwidth memory (HBM) are set to revolutionize LLM inference everywhere! 🔮✨ We're betting on Lisa Su, and so should you! 😉 Our journey with AMD is just beginning! Stay tuned for our upcoming developments: 🔬 Benchmark on MI250X & MI300A: We can't wait to explore the performance enhancements of these advanced hardware. 💻 Port the latest vLLM v0.2.1.post1: Our preliminary results show that vllm-rocm v0.2.1 is 2x faster on MI210, a promising step towards faster LLM inference. 🎯 Support more models: We're broadening the scope of our work to cater to a wider range of use cases. 🔧 Performance Optimization: We recognize the importance of performance in a high throughput LLM serving system and are committed to optimizing it. Join us on this exciting journey as we pave the way for the future of computing! Stay tuned for more updates next week on our LinkedIn and Github ➡ https://lnkd.in/gbk3Fm29 . 🌟💻 #AMD #LLMInference #vLLM #LLM #GenAI #GenerativeAI #FutureTech #DigitalTransformation 

I am pleased to have helped contribute to this paper which demonstrates the accuracy of DeePMD to predict the thermal conductivities of Au and Ag at varying temperatures. In addition, we show the near-linear scalability of the workload on up to 128 Intel(R) Xeon(R) Platinum 8480+ CPUs. DeePMD is an open source package that can be used to train an AI model based on data from AIMD simulations. When paired with LAMMPS, it can be used to accelerate classical MD simulations while retaining AIMD levels of accuracy. A very warm thank you to my co-author, Nalini Kumar, for help bringing this to fruition. Please find the paper here: - https://lnkd.in/gMpnf8zy If you are at the SC23 conference, please try to attend the AI4S workshop on Monday to see the presentation. #iamintel #sc23

Transformers just got a major upgrade with FlashAttention-3, pushing the boundaries of speed and efficiency on modern GPUs. Key Highlights: 🌟 Speedup: Achieves 1.5-2x faster performance than its predecessor. 🌟 Throughput: Reaches up to 740 TFLOPS on FP16 and nearly 1.2 PFLOPS on FP8. 🌟 GPU Utilization: Boosts GPU utilization to 75% on H100 GPUs, up from 35%. 🌟 Quantization Error Reduction: Cuts quantization errors by up to 2.6 times. Core Innovations: 🔧 Overlapping Computation and Data Movement: Keeps the GPU busy by performing multiple operations simultaneously. 🔧 Interleaving Matmul and Softmax Operations: Enhances efficiency and throughput. 🔧 Leveraging Low-Precision FP8: Utilizes modern GPU features like WGMMA (Warpgroup Matrix Multiply-Accumulate) and TMA (Tensor Memory Accelerator) to maximize performance. FlashAttention-3 is not just an upgrade; it’s a game-changer. Harness the power of the latest GPU technology and take your Transformer models to new heights. 🚀 #AI #MachineLearning #DeepLearning #GPUs #Innovation #FlashAttention3

Great article on our 2nd gen Versal devices combining updated AI Engines, FPGA fabric and Arm CPUs -- enabling single-chip intelligence for AI-driven embedded systems

Intel Software has released its 2024 Intel oneAPI developer tools with new features and enhancements that accelerate AI, HPC and rendering applications on various platforms, including Intel CPUs, GPUs and AI accelerators. Find out what's new >> https://lnkd.in/ezxUAKPK #hpc #ai #gamedevelopment #datascience #highperformancecomputing #inteloneapi #developertools

Ultralytics YOLOv8 Integration with OpenVino 😎 ✅ Intel Corporation OpenVino delivers high-performance inference on Intel CPUs, GPUs, and FPGAs. ✅ Supports heterogeneous execution across various Intel hardware. ✅ Model Optimizer imports and optimizes models from popular frameworks. ✅ Easy to use with 80+ tutorial notebooks, including YOLOv8 optimization. 🔗 Explore more: https://lnkd.in/d7RpUGnN #computervision #objectdetection #yolov8 #researchanddevelopment