🥷🏻 Have you read the Chip War? There's a secret weapon nobody could talk about. Now we finally can! Big crazy privilege to announce that Cocoa 🍫 has backed Walter Goodwin and Fractile on their mission to build AI hardware for near-instant inference of models.
🤖 What does Fractile do?
Fractile develops AI hardware (chips) with in-compute memory, for faster inference.
🤔 What does this mean and why does it matter?
AI models are repeated basic arithmetic operations. But when a trained language model is run for a user (inference), over 99% of the total compute time is spent not on arithmetic but on moving model weights from memory to the processor. This is because GPUs, like most computers, are ‘von Neumann machines’, in which the processor is separate from the memory it operates on but memory-to-processor connection has limited bandwidth.
Fractile turns the chips architecture upside down and builds circuits for the arithmetic operations directly into the memory that stores network weights, eliminating any need to move model weights from memory to the processor and achieving faster inference. By eliminating any need to move model weights from memory to the processor, Fractile chips deliver 100x higher throughput, 1/100th of the cost.
🚀 What’s the opportunity?
Believe it or not, Nvidia (2.76tn$ market cap) has a structural weakness. It has captured the lion’s share of AI compute since 2011, accounting for more than 95% of all hardware used. Their GPUs work for training but are not efficient for inference as processor and memory are separated, meaning deployment is hugely hardware-intensive and hardware has become the bottleneck for AI workloads to continue scaling.
❤️🔥 Who’s behind Fractile?
The best. Of the best. ‘Semis’ (as insiders refer to the world of semiconductors, so paradoxically cute that it always makes me smile) is a very competitive and merciless market. Producing chips is extremely expensive and designing the architecture requires to make opinionated bets on where the (AI) world is going. Hence it requires a team with a profound understanding of i) semiconductors, ii) AI, iii) tech in general and iv) business. Walter is a student of tech and a Killer with a Heart and has managed to assemble’s Europe’s best ‘semis’ talent around him.
Are you obsessed with building the next-generation AI hardware? Join Fractile on their mission, they are hiring: https://lnkd.in/db48D_Ww
#letsdothis
Kindred (forever grateful to 👑 Leila Rastegar Zegna for connecting me to Walter), NATO Innovation Fund (NIF), Oxford Science Enterprises, Inovia Capital, Stan Boland and Hermann Hauser
https://lnkd.in/d3FP2uqX
