Mobius Labs

Proud to be part of one of open source proponents from Germany. It truly is the path forward for GenAI.

💭 How can we use the full power of AI and make AI based business models financially viable? 💭 Recently had the chance to speak to Appu Shaji, CEO and Chief Scientist at APEX Ventures’ portfolio company Mobius Labs. Appu shared his perspective on the potential of large language models (LLMs) in #AI and discussed how Mobius Labs can help make AI based business models financially viable using their unique #quantization approach to help reduce compute cost. Here are some questions covered by our discussion: ⚛️ What is #quantization and Half-Quadratic Quantization? 🚀 What advancements in the development of LLMs can we expect over the next few years? 📖 Read the full article below for a deep-dive in LLMs: https://lnkd.in/ePUsH_eG #ArtificialIntelligence #MachineLearning #FutureofTech #TechTalk #LLMs #HQQ

Our CEO and Chief Scientist, Appu Shaji shares some of our thoughts on why reducing compute costs is likely the #1 challenge we need to address for Generative AI to become truly viable. The ability to run both large and small models on consumer-grade hardware will be key, empowering enterprises and individuals around the world with low-cost, open-source, and customizable software.

Releasing the 4bit version of @NousResearch 's Hermes-3-70b model using HQQ. https://lnkd.in/efXGnHUS . Did not have time yet to do a detailed benchmark, but we wanted to share it since many people have been asking for it.

Really proud to feature in this list.

Introducing Gemlite ( https://lnkd.in/gDi2MPaK ) : A collection of simple CUDA kernels to help developers easily create their own “fused” General Matrix-Vector Multiplication (GEMV) CUDA code for low-bit quantized models. Get it at https://lnkd.in/guX3nPuY Gemlite’s focus isn’t on being the fastest but on providing flexible, easy-to-understand, and customizable code. It’s designed to be accessible, especially for beginners in CUDA programming.  We believe that releasing Gemlite to the community now can fill a critical gap—addressing the current lack of available low-bit kernels. With great GenAI model power comes great computational demand. Let’s tame this beast together! #AI #MachineLearning #Quantization #CUDA #DeepLearning

🚀 Excited to announce the release of the HQQ Llama-3.1-70b 4-bit quantized model! You can check it out here: https://lnkd.in/eHJeKwWG. This model achieves 99% of the base model's performance across all benchmarks, with slightly better results in some cases. The best part? It requires 3.3x less GPU VRAM compared to the FP16 version, leading to significant compute savings while still maintaining high quality. For instance, the quantized model can run on an A6000 for basic use cases like chat, whereas the FP16 version would need 4x A6000 or 2x A100-80GB GPUs. Compute is the major driver of costs and complexity in deploying large models. This release represents one humble but hopefully useful drop towards our mission of alleviating those challenges. #AI #MachineLearning #Quantization #Llama

Our own Hicham Badri will be a hack lead at Cuda mode. He is a master of quantising/compressing models. Really looking forward to hack sessions and cool things we can do by making models work in a budget friendly way.

🚀 Exciting News! 🚀 We’re thrilled to announce the release of our high-quality Llama-3.1 8B 4-bit HQQ/calibrated quantized model! This model achieves an impressive 99.3% relative performance to FP16 while delivering the fastest inference speed for transformers. Check it out here: Llama-3.1 8B Model #AI #MachineLearning #Llama3.1 #Quantization #Inference

Really excited about this. Throw all questions on quantization to Hicham Badri way.