NVIDIA Senior Research Manager & Lead of Embodied AI (GEAR Group). Stanford Ph.D. Building Humanoid robot and gaming foundation models. OpenAI's first intern. Sharing insights on the bleeding edge of AI.
This may be Apple's biggest move on open-source AI so far: MLX, a PyTorch-style NN framework optimized for Apple Silicon, e.g. laptops with M-series chips. The release did an excellent job on designing an API familiar to the deep learning audience, and showing minimalistic examples on OSS models that most people care about: Llama, LoRA, Stable Diffusion, and Whisper. I expect no less from my former colleague Awni Hannun, spearheading this effort at Apple. Thanks for the early Christmas gift! 🎄🎁 MLX source: https://lnkd.in/g5pNrZYF Well-documented, self-contained examples: https://lnkd.in/gqp4pCbY
Wade Rosko
8mo
This is neat, but it is also niche. I know a lot of folks are developing and running local models, but this isn't helpful if we want to deploy these models... unless Apple is considering shipping their chips for servers
Elshan Magsudov
8mo
Hi Jim Fan just curious, how feasible is distribted computational network?
Michael Taylor
8mo
How large a LLama 2 model can MLX + apple silicon run?
Daniel Lau
8mo
Any idea what the difference in power consumption is compared to training and running these models versus nvidia silicon?
Dixon D.
8mo
Finally!! good intervention because trying to run convolutional neural nets on MPS is fraught with errors. My kernel dies so frequently when running models. Thanks for sharing this.
It's so simple and elegant. I was astonished that my M2 30cpu/36gpu took what seem like FOREVER with a simple 5b llm to return any kind of answer locally. When I dug into it, the concurrency hadn't yet been mastered, or so it seemed for my pleeb understanding. Glad to hear things are moving along.
Nicola Zara
8mo
How about starting to make their software accessible from non-apple devices? Literally all of their work assumes of enforces that the end user will have an apple device, and this is no different. The fact that you needbto purchase a mac to release an app on IOS simply because of their policies is absurd, and developing NN applications for M chips only seems more of the same...
Shreyas Mocherla
8mo
This is big BIG news! Apple's extraordinary GPU capabilities combined with their in-house Deep Learning framework is just killer. When Apple advertised their new Mac line-ups, they emphasised significantly on AI/ML development. It is all the more reason to purchase an M-series Mac now. Also, there is a new player in the ring and I can't help but wonder how MLX will compare against much bigger players such as TensorFlow, PyTorch and JAX. Will Keras adopt MLX and release Keras v3.1? Maybe I'm asking too much ;)
So, "`if torch.backends.mps.is_available(): import mlx.nn as nn else import torch.nn as nn`"? ;-) Glad to hear this, however many of us would've preferred if Apple had just contributed to PyTorch's MPS backend. https://twitter.com/drscotthawley/status/1674807755272486914