Groq’s Post

I updated my LLMs repo with a 5th way to implement multihead attention using PyTorch's torch.nn.MultiheadAttention class. • It's 3.5x slower than scaled_dot_product_attention (which uses flash attention v2 in PyTorch 2.2, and flash attention v1 in PyTorch 2.1). • And it's even 1.2x slower than my own from-scratch implementation. So, maybe don't use that one. Link to the code: https://lnkd.in/gcKY-yMd

Did you know that you can use Hugging Face PEFT library to inject LoRA/AdaLoRA/IA3 into any PyTorch module? Simply use `inject_adapter_in_model` by passing the corresponding peft config and model. As simple as that! Read out more about the "Low Level API" of PEFT in the dedicated section of the documentation: https://lnkd.in/eGQPKGhV

just reinstall Pytorch using pip... to make Cuda available.

PyTorch Internals! A comprehensive guide to how you can understand the PyTorch codebase & contribute to it. Here's the link: https://lnkd.in/dnDsJTQG

We have added CUDA kernels for HQQ offering a significant speed-up. Should work for both inference and training via the new ATEN_BACKPROP backend

PyTorch 2.0 succeeds its predecessor, PyTorch 1.x, with improved performance and efficiency. Here’s a rundown of these new features and capabilities: Backward Compatibility: - Code, model and API compatibility with PyTorch 1.x - Maintained compatibility with libraries and dependencies Fusing Operations: - Fuses multiple operations into a single GPU kernel - Reduces kernel launching overhead Reducing Type Conversion Overhead: - Identifies and minimizes unnecessary type conversion code - Avoids redundancies Reusing Buffers: - Reduced memory allocation - Lower memory consumption during model training Read on to learn about all the improvements PyTorch 2.0 brings and how they work: https://bit.ly/47QsWR2

PyTorch's top priorities have always been flexibility and hackability while continuing to deliver good performance. PyTorch 2.0 introduces new compiler technologies that improve PyTorch's performance. Shashank Prasanna will introduce the new PyTorch 2.0 and provide an overview of the technology stack that powers the new torch.compile() API: TorchDynamo, AITAutograd, PrimTorch, and TorchInductor. Come, join us on March 9th, 6 pm ET to learn more about what's new in PyTorch2.0. #pytorch #linuxfoundation #nyc #meetup

New post on DebuggerCafe - Fine-Tuning FCOS using PyTorch https://lnkd.in/g6Cd9JHK #ObjectDetection #PyTorch #FCOS #ComputerVision #DeepLearning #FineTuning #DeepLearningSmokeDetection