Microsoft CEO Satya Nadella
(Photo by JASON REDMOND/AFP via Getty Images)
At this year's Microsoft Build, the company's annual developers conference, the obvious theme was AI and Copilot popping up everywhere. We saw this in new PCs announced on Monday, and now with much more detail on the Copilot stack that will be built into Windows, including a Windows Copilot Runtime to run AI applications locally.
The company also announced enhancements to its various Copilot products, including support for OpenAI's GPT-4o and a Team Copilot for groups. But the biggest focus was on developer tools, including improvements to Copilot Studio (such as the ability to build extensions to other applications and agents tailored to specific functions), and its broader Azure AI Studio, which includes many models and is now generally available, as well as major changes to GitHub.
Microsoft CEO Satya Nadella began his Build keynote by echoing themes from the Copilot+ PC announcement. For the past 70 years, there were two dreams that defined where we wanted computing to go—"to have computers that understand us, instead of us having to understand computers" and, in a world where we've digitized so much information, to have "computers help us, reason, plan, and act more effectively on all that information."
AI is letting us do this. "The rate of diffusion is unlike anything I've seen in my professional life, and it's just increasing," Nadella said.
Microsoft offers three platforms to encourage this transition: Microsoft Copilot, Copilot Stack, and Copilot+ PCs. The biggest change to me is the Copilot Stack, which includes the Windows Copilot Runtime. This enables applications to call the various AI components and will eventually enable AI features in more applications—many running locally on Copilot+ PCs.
"What Win32 was to the graphical user interface, we believe the Windows Copilot Runtime will be for AI," Nadella said. This includes a new Copilot Library with more than 40 models such as Phi-Silica and many other small language models (SLMs), along with the APIs to integrate these models into applications, and some no-code integrations for the most basic applications.
Developers seemed thrilled that PyTorch will now run natively on Windows powered by Direct ML and that the Web Neural Network (WNN) API will also run directly on Windows. Some of this is still in progress, including RAG, vector embeddings, and integrated text summarization, which should be out later this year. But the concept is intriguing.
Azure AI Studio, which was announced previously but is now generally available, now offers a bunch of new models including GPT-4o, Microsoft's Phi-3-vision small language model, and other Phi-3 models. Plus, models like Llama and those from Cohere, Mistral, and a bunch of startups are now extended through a partnership with Hugging Face.
GitHub's new features are also AI-oriented, with Microsoft saying there are now 1.8 million GitHub Copilot users. New features include the ability to look up infrastructure features from services like Docker and a GitHub Copilot for Azure that lets users deploy their applications to Azure using natural language. A preview of a GitHub Copilot Workspace showed it helping developers break down a project into multiple steps, covering everything from the issue to a development plan to the code itself and the code repository.
On the top of the stack, Microsoft Copilot is what most people will see. The new Team Copilot can create agendas, take notes, and list tasks in a meeting, or surface the most important messages in a chat. The company introduced new features to extend Copilot more easily from within Copilot Studio. This includes a connector to applications from Power Platform to several third-party tools from Adobe, Service Now, Snowflake, and others. It also will gain the ability to create "Agents" that work on your behalf asynchronously and eventually automate long-running business tasks.
One concern I've heard from many people is the price and complexity of running AI. Microsoft talked about deploying Azure AI services in more countries and offering the latest accelerators from Nvidia, AMD, and its own Microsoft Maia. Overall, Nadella said the cost of making a call to GPT-4 has gotten 12 times cheaper and six times faster since it launched, a good sign.
Later, Microsoft CTO Kevin Scott reiterated that point. "You can count on things getting more robust and cheaper at a pretty aggressive clip over time."
Speaking of Scott, in his portion of the keynote, he brought up Khan Academy founder Salman Khan, who talked about using ChatGPT and Microsoft tools to create Khanmigo for Teachers. The two firms also announced that Microsoft would donate Azure AI-optimized infrastructure so Khan Academy can offer K-12 educators in the US free access to the pilot of Khanmigo for Teachers, now powered by Azure OpenAI Service.
"The model isn't a product, and the systems aren't a silver bullet," Scott noted. "You still actually have to understand who your customer is, what problem you're trying to solve, and how to deal with a whole bunch of gnarly things on top of this incredibly interesting powerful tool."
Scott urged developers to "focus on things that have made the transition from impossible to merely difficult. That's where all the interesting stuff is." He noted that the platforms are exponentially progressing so that things that are too expensive to do right now or are too fragile, "will become cheap and robust before you can even blink your eye."
He then brought up OpenAI CEO Sam Altman, who discussed how developers use GPT-4 APIs and the company's partnership with Microsoft.
When asked about the future of models, Altman said, "The most important thing is that the models are just going to get smarter." He noted that in each generation, the models can do more things. Of course, he said "speed and costs really matter to us," noting that with GPT-4o, OpenAI brought the price down by half.
He also discussed security and safety, saying that when ChatGPT first came out, there was a lot of concern about the problems, but now GPT-4 is generally considered robust enough and safe enough for a wide variety of cases. This took an enormous amount of work across both product teams and fundamental research.
Altman said this is probably the most exciting time to be building a product or doing a startup since the mobile boom or the start of the internet. Big opportunities come during a platform shift, and we haven't had a platform shift in a while. "This is a special time and take advantage of it," he said. AI is enabling a product shift, but it doesn't break the rules of business, and it doesn't get you out of the hard work of building a great product, a great company, or a great service.
The Build announcements weren't all about AI. Microsoft announced real-time intelligence within its Fabric data platform, and a partnership with Meta to bring "volumetric" (3D) Windows apps to the Meta Quest. But there's no question that AI took center stage.
Scott concluded his section by saying that Microsoft and its partners are building the platform, but it would be meaningless without the applications built on top of it. "I'm incredibly grateful for all of the things that you all have done on the platform over the past year, and I am incredibly excited to see what you all are going to go do in the year ahead."
Hands On: Microsoft's 2024 Surfaces Level Up With Copilot AI, Arm Silicon
Get Our Best Stories!
Sign up for What's New Now to get our top stories delivered to your inbox every morning.
This newsletter may contain advertising, deals, or affiliate links. Subscribing to a newsletter indicates your consent to our Terms of Use and Privacy Policy. You may unsubscribe from the newsletters at any time.
Thanks for signing up!
Your subscription has been confirmed. Keep an eye on your inbox!
Sign up for other newsletters
