Younes Belkada’s Post

Are you curious about transfer learning with PyTorch?🔥 This article walks us through fine-tuning a VGG model for image classification on the Caltech-101 dataset and includes a code walkthrough. Get started → https://lnkd.in/g-pnukp4

I updated my LLMs repo with a 5th way to implement multihead attention using PyTorch's torch.nn.MultiheadAttention class. • It's 3.5x slower than scaled_dot_product_attention (which uses flash attention v2 in PyTorch 2.2, and flash attention v1 in PyTorch 2.1). • And it's even 1.2x slower than my own from-scratch implementation. So, maybe don't use that one. Link to the code: https://lnkd.in/gcKY-yMd

just reinstall Pytorch using pip... to make Cuda available.

Implementing Soft Nearest Neighbor Loss in PyTorch - Abien Fred Agarap - https://lnkd.in/extRbjxh

This is a win for #sustainability. In effect, ML applications may better leverage GPU capabilities, saving more time and energy 🚀

New post on DebuggerCafe - RT-DETR for Object Detection - paper explanation and inference using PyTorch https://lnkd.in/ghAUjnqB #RTDETR #DetectionTransformer #DeepLearning

PyTorch 2.3 is here 😎🔥 PyTorch 2.3 offers support for user-defined Triton kernels in torch.compile, allowing for users to migrate their own Triton kernels from eager without experiencing performance regressions or graph breaks. Details: https://hubs.la/Q02tY8PB0

PyTorch gets sophisticated with every release. It is the biggest enabler in democratizing AI.

New post on DebuggerCafe - Fine-Tuning FCOS using PyTorch https://lnkd.in/g6Cd9JHK #ObjectDetection #PyTorch #FCOS #ComputerVision #DeepLearning #FineTuning #DeepLearningSmokeDetection

PyTorch has a new attention function for LLMs called FlexAttention that supports various attention variants. I just took it for a spin and compared it to other implementations of multi-head attention, and it seems to work great in combination with torch.compile. However, if you are not compiling your models (e.g., due to compatibility issues), it may be better not to use it for multi-head attention. The notebook with the benchmark code is available here: https://lnkd.in/gcKY-yMd