Tune in remotely at 8:00 PT on July 10 for a TPC seminar by Valentin Reis. Learn how Groq co-designed a compilation-based software stack and a class of accelerators called LPUs, with high utilization and low end-to-end system latency. He’ll review the challenges of breaking models apart over networks of LPUs, and outline how this hardware/software system architecture keeps enabling breakthrough LLM inference latency at all model sizes. https://lnkd.in/gdQBuS7U
Groq’s Post
More Relevant Posts
-
Don't miss Arash Taheri, Technical Lead, Groq Compiler, at the 8th Annual Toronto Machine Learning Summit (TMLS). He'll be speaking about modifying PyTorch models to enable custom data-types and persist precision information through Groq Compiler. https://hubs.la/Q02FjrDt0
-
-
See you there!
7 Days to AI Revolution: IMAGINE AI LIVE - IMPACT NYC Countdown Begins! 🗽 ⚡ 🚀 In just one week, Cornell Tech transforms into the epicenter of AI innovation. How? Through the force of our AI-powered Human Village! • From concept to reality in 11 weeks • Announced only 6 weeks ago! • Now, 7 days from igniting the AI world 🔥! This isn't just an event. It's a testament to human ingenuity amplified by AI. Our global village of visionaries has moved mountains: • Cutting-edge sponsors • Innovative partners • World-class speakers • AI pioneers and enthusiasts • Incredible founders Together, we've harnessed AI agents and bleeding-edge tech to create an experience that will redefine the future of AI. 📽Watch our reel! Spot yourself or your company? Reshare and show the world you're part of this AI revolution! Got your ticket? Tell us what you're excited for: • Mind-bending AI workshops? • Networking with the AI elite? • Glimpsing the future of tech? 🎟Haven't secured your spot? There's still time! Visit imagineai.live now! This isn't just an event. It's the launchpad for the next era of AI innovation. Be there as we make history in the heart of NYC! 🗽 ⚡ 🚀 #ImagineAILive #AIRevolution #NYCTech #FutureIsNow #AIVillage #AIforImpact
-
Join Jonathan Ross at VB Transform on July 9. He'll delve into the transformative impact of AI inference on enterprise technology, show a live demo highlighting Groq capabilities & share why by next year, over half of the world's inference computing will run on Groqchips. https://hubs.la/Q02Dr67T0
-
-
We have a mission to drive the cost of compute to zero. It's an infinite goal that pushes us to always search for efficiencies in our technology stack. We just posted a new paper relating to our LPU, AI Inference Technology and power usage. https://lnkd.in/gGBMq7db
-
🚀🚀🚀 🙏 Artificial Analysis
Fast to launch & very fast output speed! Groq has launched their Gemma 2 9B offering and is serving it at ~600 output tokens/s Gemma 2 9B is worthy alternative to Llama 3 8B and other smaller models. It is particularly attractive for generalist and communication-focused use-cases as shown by its Chatbot Arena (1185) & MMLU (71%) score exceeding Llama 3 8B (1153, 68%). For more specific use-cases it is worth conducting more narrow tests, e.g. for coding Gemma 2 9B well underperforms Llama 3 8B (40% vs. 62% on HumanEval). Groq is offering the model at $0.2 per 1M Input & Output tokens, in-line with Fireworks. Congratulations Groq on the fast-launch and impressive performance. We look forward to benchmarking other providers as they begin to host the Gemma 2 models, including potentially Google itself Analysis of Gemma 2 Instruct (9B): https://lnkd.in/gC6Xnj3a Analysis of providers: https://lnkd.in/gb9S5khK
-
