Multi-LoRA serving turns "one GPU per model" into "one GPU per base model, amortized across hundreds of tenants." Here are the tools that do it, and the kernel trick that makes it work.
Ollama just ripped out llama.cpp and bolted in Apple's MLX on the Mac. The switch is a tell about where your bottleneck actually lives — and when the older engine still wins.
The Model Context Protocol defines three server primitives split by who's in control — the model, the app, the user. The ecosystem implemented one of them.
Most MCP servers only answer requests. Sampling and elicitation are the two features that let a server reach back through the client — one to the model, one to the human — and almost no one implements either.
Loading every tool definition into context and round-tripping every result is how MCP agents stall. Code execution flips the model into a programmer — and moves the hard part to your sandbox.
Your chunks lose the document around them before they're ever embedded. Jina and Anthropic solve it in opposite places — one in vector space for free, one in the text for a price.
The weights are the easy part — the math you can do on a napkin. What silently OOMs your server in production is the KV cache, and almost nobody budgets for it.
When retrieval underperforms, everyone reaches to fine-tune the LLM. The cheaper, higher-leverage move is to fine-tune the embedding model — and almost all the gain comes from one ingredient.
MCP standardized how an agent calls a tool. It said almost nothing about how the agent logs in as you — and that gap is the whole product these three are selling.
The real divide in open-source RAG isn't which library to import — it's whether to build with one at all, or deploy a finished engine. Three engines, three very different bets.
GRPO is now a commodity all three ship. The thing that actually sorts them is who owns the distributed orchestration — and how you keep one starving inference engine fed.
Three ways to serve embeddings at scale that look like rivals but answer a different question: should embeddings be a dedicated specialist, or ride on the GPU already running your LLM?
The new realtime models hear and speak in one step, no text in the middle. That deletes the seam where you used to read, log, and control everything. Here's the real trade.
The frameworks that get the most attention disagree on something basic — what an agent's action even is. One writes code, one wires a graph, one casts a team.
The listicle treats these as three flavors of the same choice. They aren't — two are ends of one axis, and the third sits on a different axis entirely. Pick by your environment, not your vibe.
The benchmark you compare on today expires in three weeks. The license you build on doesn't. Pick an open-weight family the way it will still matter next quarter — by what you're allowed to do with it, and what it costs to serve.
Forcing a model to emit valid JSON is a solved problem. Doing it without slowing generation to a crawl is the one that produced three new engines — and your serving stack probably already picked one for you.
Three self-hosted chat UIs that look interchangeable on a feature checklist — but each one is really built for a different person, and picking the wrong one means fighting the grain forever.
An MoE model computes like a small model and remembers like a giant one. That split is great for a token factory and a trap for a single self-hosted agent.
One agent, twenty MCP servers, and a context window drowning in tool definitions. The gateway is the layer that puts a single governed door in front of all of them.
Three open tools for making synthetic fine-tuning data. The model that generates it stopped being the hard part — the part that decides whether your dataset helps or quietly poisons your model is what happens after.
The four tools map to four architectural postures — and in a year when the companies keep getting acquired out from under their users, the posture is what you're actually choosing.
Two ways to build an agent that drives software: send it screenshots and let it move the cursor, or hand it the page's structure and let it act on elements. The split isn't old vs new — it's general vs reliable.
Three repos for retrieving over PDFs as images instead of parsed text — and why the real choice between them is who owns the multi-vector storage problem, not who has the best model.
They get pitched as competitors. They're not even the same kind of thing — and the difference that actually decides your architecture is what each one costs you in tokens.
They aren't ranked by capability. They differ on where the agent runs and who holds the steering wheel — and that decides your blast radius, not your benchmark score.
Three engines, one job: turn a model into a high-throughput endpoint. The feature gaps are closing — what's left is portability, vendor lock-in, and which project is still being built.
Speculative decoding makes a single LLM response 2–6x faster without changing a token of the output. The reason it works — and why the newest method wins — is a fact about your GPU, not your model.