Zachary Mueller’s Post

With model weights being squeezed down to (almost) single bits and matrix multiplications giving way to signed addition, can machine learning be liberated from backprop and GPUs?

Have you heard of GPU's? 🤔👀 💡 They power up scientific experiments. 💻They decode data & manage data processing. ✅They help make Al training seamless and easy. In simple terms, GPUs are tech's heavy lifters. 💪 Learn more: https://lnkd.in/evYEWEH8 #gpustocks #gpu #themotleyfool

In our newly published article, we show the feasibility and effect of implementing parallel data reduction for NP-hard optimization problems on GPUs. Actually, the article is not that new ... just after two years under review, it is now freely downloadable until Nov 14th via https://lnkd.in/dwb_SxDq #algorithm #kernelization #datareduction #NP

AI Workbench is a transformative approach that makes it easy to work with GPUs and containers on your choice of system. Some of you may remember a previous product that I worked on. This one is so much better. Give it a try and let me know what you think!!!

 With the Akhet Server VarioFlex 5U with dual GPU and the high performance computing platform Akhet VarioScaler xI with multiple dual GPUs, Pyramid Computer GmbH is presenting two new systems at Embedded World that are optimized for machine learning applications. In addition to both systems, a camera bar is shown that includes two 48MP cameras and an ARM Cortex A73 with integrated NPU. The machine vision solution recognizes objects based on a trained model. https://lnkd.in/eBnaCeVc #machinevision #imageprocessing #edgecomputing #embeddedsystems #machinelearning #ai #deeplearning

Pushing the Limits of the Two-Tower Model 🏢 Pushing the limits of the Two Towers: A deep dive into the world of ML models with multiple GPUs and efficient training techniques. 🚀 Key points: 1️⃣ Two Tower architecture: A popular approach for building large scale machine learning models that can handle vast amounts of data. 🏋️♂️ 2️⃣ Scaling up with GPUs: Using multiple GPUs to speed up training and inference processes. 🖥️💨 3️⃣ Efficient training techniques: Exploring ways to optimize resource allocation and reduce training time. ⏱️⚡ 4️⃣ Challenges: Overcoming bottlenecks and synchronizing GPU usage for seamless performance. 🔄🎛️ 5️⃣ Improved performance: The benefits of leveraging two towers and optimizing GPU utilization for enhanced model accuracy. 👍⚙️ #MachineLearning #TwoTowerArchitecture #GPUComputing #EfficientTraining #ModelPerformance https://buff.ly/3GE0QgO

What’s the first problem you’re likely to encounter when fine-tuning an LLM? Out of memory error. No, not a CUDA OOM, just a regular host out of memory error. Why? Because before you even get to multi-GPU training with model parallel frameworks like Deepspeed, you need to load the pretrained checkpoint into host memory. To make matters worse for machines with multiple GPUs, because Deepspeed is also a data-parallel framework, you actually need to load the checkpoint into host memory once for each GPU in your job! Now training your 7B parameter Llama-2 model in float32 with 8 GPUs requires 7 * 4 * 8 = 224 GiB of host memory just to load it onto the GPUs. The good news is: if you use an optimized LLM training framework like Ludwig.ai, you can get the host memory overhead back down to a more reasonable 7 * 4 = 28 GiB of host memory even when training on multiple GPUs. How did we do it? By loading the model weights into memory once before training begins and inserting them as numpy arrays into the #Ray object store, we can then zero-copy read the weights directly from shared memory into each GPU worker process. You can read more about this approach here: https://lnkd.in/gX4xkHgb Doing this yourself would normally be a fair bit of cumbersome code, but in Ludwig you get it for free just by running with Ray as the backend runtime. Try it out for yourself: https://lnkd.in/gaiMg9aZ

This is a quick demonstration of the capabilities of local GPUs with the currently available machine learning tools for consistent video.#stablediffusion #machinelearning #aiart