The one-screen version: at the Open Compute Project summit on October 12, Microsoft unveiled Maia 300, its own inference chip, and AMD introduced Helion, an open rack blueprint for agentic AI. Both are aimed squarely at loosening Nvidia's grip. Neither is something a team of one can buy or rent in 2026 — Maia ships in 2027, Helion configs land in November for enterprises. So the near-term action is nothing to purchase. The thing that actually changed is the number both companies led with: not teraflops, but tokens per watt. Here's why that reframes how you'll buy inference, and the one move that still protects you.
1. Microsoft Maia 300: the hyperscaler builds its own inference engine#
Microsoft debuted Maia 300, the third generation of its in-house accelerator, as an inference-first chip — the part of the workload that runs a model after it's trained, which is where most production spend actually lives. The specs: four compute chiplets on TSMC's 2nm node, eight stacks of HBM4 for 288GB of memory (the same capacity as Nvidia's flagship Rubin, at roughly 20% less bandwidth), inside a 950-watt budget. A 96-chip rack draws about 167kW and, Microsoft says, delivers up to seven times the token throughput of a Maia 200 rack.
The headline claim is 40% more tokens per megawatt than leading GPU-based systems — but note the unit, and note the caveat: that's Microsoft's own projection, measured against unnamed systems and untested by any third party at launch. Production is slated for 2027, with multi-gigawatt deployments and a stated goal of making Maia the primary inference system across Microsoft's products.
What it changes for you: nothing you can act on this quarter — you can't rent a Maia. What matters is the direction. Microsoft joins Amazon (Trainium) and Google (TPU) in running its own inference silicon, which means the biggest buyers of Nvidia are also its biggest emerging competitors. More competition at the chip layer is why the GPU rental floor keeps falling — and a falling floor is the single best thing that happens to a bootstrapper's inference bill. Microsoft framing its chip around tokens per megawatt, not FLOPS, is the tell for section 3.
2. AMD Helion: the rack gets an open blueprint, and the agent loop gets its own box#
AMD's Helion is a different kind of move: not a chip, but an open rack-scale blueprint built on OCP's Open Rack standard, which AMD says it will contribute back to OCP. Two configurations tell the story. One is CPU-only — 6th-gen EPYC processors reaching 32,768 cores in a rack, built to run the agent orchestration loop (the tool calls, retries, and control flow) that doesn't need a GPU. The other pairs EPYC CPUs with Instinct MI350P GPUs for the model-inference half. The design separates agent execution from model inference — a recognition that an agentic workload is mostly glue code around occasional model calls, and you shouldn't burn GPU budget on the glue.
Partners already lined up to build systems include Cisco, Dell, HPE, Lenovo, QCT, Supermicro, Hon Hai and Wiwynn; configs are planned for November, with broader availability in the first half of 2027.
What it changes for you: this is yesterday's portability theme at the rack level. Just like Decart's chip-portability layer being absorbed, the fight here is over who controls the rack — Nvidia's proprietary NVLink-stitched rack versus an open OCP spec anyone can build to. An open rack standard, if it takes, is downward price pressure on everything you rent. The second idea is one to steal regardless of hardware: in your own architecture, treat the agent loop and the model call as separate, independently-scalable things. You're paying GPU prices for CPU work if your orchestrator and your inference live in the same box.
3. The coda: the metric moved, and it isn't about your electricity bill#
Here's the part worth internalizing. Both announcements led with efficiency per watt because power — not chips — is now the binding constraint on AI buildout. Morgan Stanley reckons US AI data-centre startups raised about $33.5B in 2026 with the money flowing toward power; you can't energize racks you can't power. So the vendor that squeezes more tokens out of a megawatt can deploy more total inference under the same grid connection — which is the real lever on how low the rental floor can go.
The trap is to read "tokens per watt" as a line item on your electricity bill. It mostly isn't — raw energy per token is tiny next to amortized hardware. We work the actual math in a companion piece: tokens per watt is a capacity metric, not an energy-cost metric, and that distinction changes which vendor claims deserve your attention.
The one line to take into the week: you can't buy a Maia 300 or an Helion rack, and you shouldn't try to. What you can do is update the question you ask a hosted-inference vendor in 2027 — swap "how many FLOPS?" for "tokens per dollar?", which is just tokens-per-watt wearing a price tag — and keep your inference swappable at the API boundary so that whichever of these chips wins, you move to it in an afternoon, not a quarter. Yesterday's edition on the giants buying the portability layers is here; for the longer arc on custom inference silicon, we covered Maia 200 and the multi-silicon bet earlier.
