Chetan Arvind Patil’s Post

Introducing ASRock Industrial's groundbreaking AI Guru Software, developed in tandem with the NUC Ultra 100 BOX / NUCS Ultra 100 BOX Series. Powered by the latest Intel Core Ultra processors, this integration unlocks unparalleled AI capabilities for personal computing. Discover the potential of AI Guru's pre-built models, seamlessly integrated with the Intel Distribution of OpenVINO Toolkit. Downloadable from the ASRock website, this software empowers users to unleash rapid AI inference on their NUC Ultra 100 BOX / NUCS Ultra 100 BOX Series with ease. Embark on your AI journey effortlessly—install, select models, connect your input source—and explore endless AI possibilities. Learn more: https://buff.ly/3vPP4xQ #ASRockIndustrial #AIInnovation #NUCUltra100 #IntelCoreUltra #AIForPersonalComputing

⚡ Amazing work from the HPE AI CoE, who were significantly able to reduce energy consumption using Intel-Optimized Software and Hardware⚡ "While fine-tuning we implemented the optimizations that come with Intel IPEX and AMP reducing the duration of the experiment by 3 and consuming 4x less energy with an insignificant loss in accuracy of 0.4%. In TensorFlow’s case, we reduced the duration by 2.5 and consumed 3x less energy, without experiencing a drop in accuracy." "In inference, we implemented the optimizations that come with Intel IPEX/ITEX and AMP and converted the baseline model to Intel INC, OpenVINO IR format, and NNCF. Here, we experienced latency reduction, throughput speed up, energy savings, and thus dropped carbon emissions in all cases. All, with a minimal accuracy drop lower than 1%. There were remarkable results where optimizations from the PyTorch experiments consumed at least 68% less energy than the baseline model and in the case of the TensorFlow experiments, they all classified at least 7x more images per second." #Intel4thGen #AI #inference #finetuning #HPE #Intel #sustainability

Exciting Times in AI! 🌐 The AI landscape is evolving at an incredible pace, transforming the way companies operate. From language processing to code creation, the possibilities seem limitless. Embarking on the AI journey comes with considerations—hardware, costs, integration, and scalability. Enter the 5th Gen Intel® Xeon® Scalable processors, designed to elevate your AI experience. 🔮 These processors effortlessly handle demanding AI workloads, managing up to 20 billion parameters without immediate hardware upgrades. Our co-founder Chelsea explains how for all these you do not need to start from scratch—Intel offers pre-trained, optimized models that are easy to customize. For complex tasks, an open-standards framework allows seamless integration of accelerators and GPUs. Plus, the 5th gen is compatible with 4th gen for a hassle-free upgrade. Curious to dive into the magic of AI with Intel? Click the link to learn more: https://bit.ly/4aoea6D https://lnkd.in/eifFCinG Intel Corporation #IntelAmbassador #ad #IntelXeon #IntelAmbassador #Technology #Innovation #Intel #AI

NVIDIA’s Bill Dally highlighted at Hot Chips Symposium last year that, of the 1000x increase to single chip inference performance over the last 10 years, the largest single contributor was more efficient number representation (~16x improvement) vs only a ~2.5x improvement from GPU process node shrinkage (28nm to 5nm). Last week saw a new paper from Microsoft Research on quantizing LLMs down to ternary values {-1,0,1}: The Era of 1-bit LLMs: All Large Language Models are in 1.58Bits (https://lnkd.in/gaUquHEk) Some of the excitement surrounds the use of ternary values not requiring matrix multiplication, just significantly less energy-intensive addition operations. Will be interesting to see if this potentially game-changing approach bears fruit as more results become available. #aihardware #llms #ai

Now that Gaudi 3 AI accelerator is out by few months and highlighted during Intel Vision 2024 (1.7X the training performance, 50% better inference, and 40% better efficiency than Nvidia’s market-leading H100 processors), here are some deep dive and quick observations and I am happy to see that we have choices to make for different AI workloads. A) 128 GB HBM (High Bandwidth Memory) of Gaudi stands out in showing high performance B) Higher Throughput (TPS) – just due to the ability to use higher batch size Pre training throughput – just due to better management during back-propagation process C) Power efficiency is much better D) The performance is very obvious against H100 or H200 for higher workloads (70B or 180B) compared to lower workloads (<30B). And its across different model and execution parameters observed. Broadly Gaudi 3 comes with 4 DCORE (Deep Learning Core) each having: A) 2 MMES (matrix multiplication engines) – each can take 64k parallel operations, very powerful for deep learning workloads handling complex matrix operations B) 16 TPCS (Tensor processor cores) C) 24 MB L2 Cache Hope to see more details on Software suite (more on AI ecosystem than low level components) and ease for developer community without getting into details of architecture, compute, networking or host interface details of this. #Intel #Gaudi3 #AI

Pure-play foundries United Microelectronics Corporation (UMC) and #Powerchip Semiconductor Manufacturing (#PSMC) have received orders for Si interposers, allowing them to enter the supply chain for in-demand AI GPUs, according to industry sources. #TSMC's #CoWoS packaging faces supply constraints due to high demand, leading to spillover effects on orders. Sources noted that UMC has been identified as winning some interposer orders for #Nvidia's #AI chips with Siliconware Precision Industries (SPIL) handling back-end WoS orders. Additionally, PSMC chairman Frank Huang confirmed interposer orders that recently commenced, signifying the pure-play #foundry's foray into the #supplychain of the AI industry. The orders generate higher-than-expected gross profits and serve as an additional catalyst for PSMC's revenue growth.

"Itching" for AI news that you may not have heard yet ? Meet Etched. Founded by Chris Zhu and Gavin Uberti, this start-up, focussing on hardwiring Transformer architecture on the chip itself, is backed by guys likes Peter Theil and raised over $120 m in 2 years. Though the sales pitch is how it is better than GPUs and NVDIA's of the world, I think, if what they are building is true, this will be a game-changer, only if (I believe) works / integrates with others including competition tech (for more than one model). Have a look at their website if you have not seen it yet. Ps: Both founders are super young, just what they can dream of building and then getting the right support, can make anyone else feel, old, very old :) Psst: Wondering what is happening with Intel, AMD and the other semiconductor giants? https://lnkd.in/eyHKpFY7

As LLMs grow more advanced, they become more complex resulting in obstacles like size and latency that affect performance. Discover how to use weight-only quantization (WOQ) to improve LLM inference and overcome these limitations using Intel Extension for PyTorch running on the latest AI PCs equipped with Intel Core Ultra processors: https://intel.ly/4ajiiDH #oneAPI

How AI is changing how chips are built ⚡ #theCUBEpod https://lnkd.in/gEj43Quh Conventional wisdom was to make computing chips smaller, faster and pack as many as possible onto a motherboard. There’s a new model emerging, and NVIDIA is leading the charge, according to our analysts. Recapping highlights from #GTC, John Furrier notes Nvidia’s position with their new Blackwell  chip for real-time #GenAI processing. “[Nvidia president] Jensen Huang pointed out with Blackwell they want to make the largest chip possible, because the benefits of having a chip gives you power and all that high-speed interconnect built into the chip works better,” Furrier says. “Chips talk to each other, it’s going to start chips talking to each other on these transformer engines,” he furthers. “They’re essentially building the data center of the future.” Watch the full episode: https://lnkd.in/gEj43Quh #CIOnews #AIinfra #EnterpriseComputing #ThisWeekInTech #MarketInsights

Intel’s Aurora achieves exascale to become the fastest AI system: Intel, in collaboration with Argonne National Laboratory and Hewlett Packard Enterprise (HPE), has revealed that its Aurora supercomputer has exceeded the exascale computing threshold reaching speeds of 1.012 exaflops to become the fastest AI-focused system. “The Aurora supercomputer surpassing exascale will allow it to pave the road to tomorrow’s discoveries,” said Ogi Brkic, Intel’s VP... Read more » The post Intel’s Aurora achieves exascale to become the fastest AI system appeared first on AI News. http://dlvr.it/T6pp19