Microsoft is reportedly planning to spend $100 billion on a supercomputer — codenamed Stargate — to power OpenAI's next-generation models. If true, that's not a data center budget. That's a moonshot. This week in ML news, we break down what that actually means, why there's a hidden AGI clause that could blow the whole deal up, and everything else that happened across the industry: Stability AI's CEO departure, Grok 1.5's launch, OpenAI's synthetic voice tease, and a landmark talk on how to build LLMs in 2024.
What Is Microsoft's $100B Supercomputer for OpenAI?
According to people familiar with the matter (the eternal hedge of tech journalism), Microsoft is in talks to build a supercomputer so large it would dwarf anything previously constructed for AI training. The project, reportedly called Stargate, carries a price tag of around $100 billion — a number so large it prompts an obvious question: what exactly are they expecting this thing to produce?
Critics quoted in Fortune made a sharp observation: the only way a single OpenAI model could justify that kind of investment in a single data center is if the model were, in fact, AGI. Anything short of that and the math just doesn't work. But here's where the story gets genuinely interesting — Microsoft probably doesn't want it to be AGI.
OpenAI's partnership agreement with Microsoft includes a clause that limits Microsoft's commercial rights to OpenAI technology that falls short of AGI. Once OpenAI declares they've hit AGI, Microsoft's licensing advantages essentially evaporate. OpenAI, not Microsoft, decides when that threshold has been crossed. So Microsoft is in the strange position of bankrolling a machine that, if it works too well, cuts them out of the reward.
Will the $100 billion supercomputer materialize? It's all still in the "people familiar with the matter" phase. But here's a prediction worth making: when OpenAI eventually announces they've reached AGI, it will be one of the most underwhelming moments in tech history. They'll point to some arbitrary internal metric, issue a blog post about safety and responsibility, and the world will collectively go, "...that's it?" We're not saying AGI won't eventually be real and significant — just that OpenAI's announcement of it will almost certainly be a political and financial maneuver dressed up as a milestone.
Why Did Stability AI's CEO Emad Mostaque Resign?
In a significant leadership shakeup, Emad Mostaque has resigned as CEO of Stability AI, the company he founded and built into one of the most recognizable names in open-source AI. He stated his departure is motivated by a desire to ensure AI remains open and decentralized — which, reading between the lines, suggests some tension with the direction Stability AI has been heading.
Stability AI's journey has been a fascinating arc. They burst onto the scene with Stable Diffusion, a fully open-source image generation model that genuinely shook the industry. Then, like many AI companies facing the cold reality of needing revenue, they shifted toward an open-weights model: free for personal use, but commercial use requires payment. That's a reasonable business decision — companies need money — but it represents a real philosophical shift from the early days.
The company still produces serious work. Stable Diffusion XL is an incredibly capable model and remains at the forefront of open image generation research. But the future direction of Stability AI without its founder is genuinely uncertain. As for Emad himself — whatever he builds next will be worth watching.
What Is Grok 1.5 and How Does It Stack Up?
Shortly after xAI made headlines by releasing the full weights of Grok 1 in a genuinely open-source manner, Twitter (now X) announced Grok 1.5. The new model brings two headline features: improved reasoning capabilities and a 128,000-token context window.
On benchmarks, Grok 1.5 holds its own against current comparable models — it's not the undisputed best model available, but it's a clear improvement over Grok 1 and competitive with what's in its class. The 128K context length is particularly notable. Token windows across the industry have been expanding at a pace that would have seemed absurd two years ago, and figuring out how to actually use that much context effectively is becoming its own research challenge.
Grok 1.5 will be available on X, and for those with access to the Grok model on the platform, it's worth testing when it rolls out.
What Is OpenAI's Voice Engine and Should You Care?
OpenAI published a blog post titled Navigating the Challenges and Opportunities of Synthetic Voices, detailing early experiments with something called Voice Engine. The concept: give it a text prompt and a 15-second audio sample of any speaker, and it generates natural-sounding speech in that person's voice reading whatever text you provide.
The technology itself is impressive — custom voice cloning has been improving rapidly and the quality bar is now extraordinarily high. But let's be direct about what this blog post actually is: it's an ad. It's a teaser trailer. Every sentence is couched in safety language, every capability is hedged with responsible AI framing, and nothing of real technical substance is disclosed. It's the AI industry equivalent of a luxury perfume commercial — they're not selling you a product yet, they're selling you a feeling about the brand. When the actual Voice Engine launches, we'll have something real to evaluate. Until then, file this under "OpenAI marketing."
Does GPT-4 Really Have 1.8 Trillion Parameters?
At GTC, Jensen Huang casually dropped some numbers about OpenAI's newest model: 1.8 trillion parameters, trained using 30 billion quadrillion FLOPs. Yes, that's the actual unit being cited. Thirty billion quadrillion. At some point these numbers transcend engineering and enter the realm of Bond villain ransom demands.
To be fair, these figures aren't impossible — large mixture-of-experts architectures can reach parameter counts in that range, and frontier model training runs do consume staggering compute. But the cadence at which these numbers are being tossed around at keynotes, unverified and uncontextualized, is worth noting. Whether or not 1.8 trillion is the real count, the trend is clear: the biggest models are getting incomprehensibly large, and the compute required to train them is scaling faster than most people expected.
Can You Actually Earn Money Building OpenAI GPTs?
OpenAI announced they're partnering with a small group of US builders to test usage-based GPT earnings — essentially an App Store model where creators of popular GPTs earn money when other users interact with their custom agents. The vision is a "vibrant ecosystem" where builders are rewarded for creativity and impact.
It's an interesting idea, but there's a fundamental tension here: language models are a universal interface. A GPT is largely a prompt flow — and prompt flows are trivially portable. Nothing stops a user from taking the same task to Claude, Gemini, or any other frontier model. The vendor lock-in that makes app stores work (proprietary APIs, exclusive hardware, platform-specific features) doesn't naturally exist in the LLM space. Whether OpenAI can manufacture that stickiness through ecosystem incentives alone is one of the more interesting strategic questions in the industry right now.
How to Build Large Language Models in 2024
On a more educational note, Thomas Wolf from Hugging Face released a talk titled A Little Guide to Building LLMs in 2024, and it's excellent. The recording is on YouTube and the slides are publicly available.
The talk covers the full modern LLM training pipeline, including:
- How to evaluate and prepare training data at scale
- Parallelism strategies for efficient distributed training
- Modern architectural and fine-tuning considerations for 2024
If you're serious about understanding how state-of-the-art models are actually built — not just used — this is one of the best freely available resources right now. Go watch it.
The Week in Review: It's All OpenAI, All the Time
Look at this week's headlines honestly and a pattern emerges: most of the biggest stories involve OpenAI in some way — the Microsoft supercomputer, the Voice Engine teaser, the GPT earnings program, the Sora first-impressions post. The company is generating an enormous volume of news, most of it carefully managed and brand-forward. One can't help but notice that since Ilya Sutskever stepped back from the spotlight, the company seems to be moving faster and more commercially than ever. Make of that what you will.
