If you want a frontier-adjacent open model running on hardware you can rent by the hour, Qwen3.6-35B-A3B is the one to reach for in July 2026. It's a mixture-of-experts model — 35B total parameters, but only about 3B fire per token — and it's Apache 2.0, the license with no commercial strings. The headline for a solo builder: because so few parameters activate per token, a quantized copy fits on a single 24GB GPU, runs at small-model speed, and never phones home. Below is the exact serving command, the VRAM arithmetic, and the volume where a hosted API is still the cheaper call.
The one thing to get right: MoE means memory, not compute#
"A3B" is the part that changes the economics. In a mixture-of-experts model, a router selects a small set of "experts" for each token, so Qwen3.6-35B-A3B does roughly the per-token math of a 3B model while holding the knowledge of a 35B one. The catch is that you still have to keep all 35B parameters resident in GPU memory — the experts you didn't use this token, you'll use next token.
So the constraint on one GPU is memory, not throughput. That's what quantization solves:
| Precision | Approx. weight size | Fits on |
|---|---|---|
| BF16 (full) | ~70GB | Multiple GPUs |
| FP8 | ~35GB | One 40–48GB card |
| 4-bit (GGUF/NVFP4) | ~18–21GB | One 24GB card (RTX 4090 / L4-class) |
(Sizes are computed from the parameter count, not quoted from a spec sheet — confirm the exact GGUF file size on the model card before you provision. Leave headroom above the weight size for the KV cache, which grows with context length.)
The practical read: quantize to 4-bit and you're on a single 24GB card. Want full BF16 quality? That's a multi-GPU box, and at that point re-run the self-host-vs-API cost breakdown before you commit.
Serve it with vLLM 0.26.0 (shipped July 25)#
Both major engines put out fresh releases this week — vLLM 0.26.0 and SGLang 0.5.16 both landed on July 25, 2026 — and both expose an OpenAI-compatible endpoint, so your client code is unchanged either way.
Install and serve with vLLM:
pip install "vllm==0.26.0"
# Qwen's published example — note this is a FOUR-GPU, full-precision config:
vllm serve Qwen/Qwen3.6-35B-A3B --port 8000 \
--tensor-parallel-size 4 --max-model-len 262144 --reasoning-parser qwen3That command is straight from Qwen's model card, and it is not the one-GPU recipe — --tensor-parallel-size 4 shards across four cards and --max-model-len 262144 reserves KV cache for the full 262K context. On a single 24GB card you want a quantized weight, TP of 1, and a context length you'll actually use:
# One-GPU shape: quantized weight, single card, realistic context window
vllm serve Qwen/Qwen3.6-35B-A3B-FP8 --port 8000 \
--tensor-parallel-size 1 --max-model-len 32768 --reasoning-parser qwen3Substitute the specific quantized repo you pulled (an FP8 or 4-bit variant). Dropping --max-model-len from 262K to something like 32K is the difference between the KV cache fitting or OOM-ing on a 24GB card — set it to your real prompt budget, not the maximum.
Or SGLang 0.5.16, same shape#
pip install "sglang==0.5.16"
# Qwen's four-GPU example:
python -m sglang.launch_server --model-path Qwen/Qwen3.6-35B-A3B \
--port 8000 --tp-size 4 --context-length 262144 --reasoning-parser qwen3For one card, mirror the vLLM changes: a quantized --model-path, --tp-size 1, and a smaller --context-length. SGLang tends to win on prefix-cache reuse and MoE load balancing; vLLM is the more common default. Both shipped fresh releases the same day — our vLLM 0.26 vs SGLang 0.5.16 breakdown covers what each one changed and which to pick for which workload, and the vLLM vs SGLang vs Ollama piece starts one level up.
The number that decides it: break-even#
Here's the honest part. Self-hosting is not automatically cheaper — it's cheaper at volume.
- A dedicated 24GB cloud GPU runs roughly $400–900/month, and it costs that whether you push a billion tokens through it or zero.
- The hosted Qwen API is roughly $0.14 per million input tokens and $1.00 per million output tokens (directional — confirm current pricing with your provider).
Multiply your real monthly volume against those API rates. Below a few hundred million tokens a month, the API almost always wins on pure cost, and it wins on zero ops. Self-hosting earns its keep when you have data residency requirements, steady high volume, hard rate-limit ceilings, or a plan to fine-tune — the last of which Apache 2.0 makes fully yours. This is the same math we ran for Kimi K3's 1.4TB weights; the difference is that a 3B-active MoE actually fits your budget, so the break-even lands somewhere a solo founder can reach.
The move#
If your workload is bursty or small, use the API — the setup is a key and the idle cost is zero. If it's steady, private, or you want your own fine-tune, pull a 4-bit Qwen3.6-35B-A3B, serve it with --tensor-parallel-size 1 on a 24GB card, and cap --max-model-len to your real context budget. The MoE architecture is what makes the second option newly sane on one GPU: 35B-class answers, ~3B-active compute, Apache 2.0, no vendor. Benchmark it on your own prompts before you wire it into production — the cost-per-completed-task number, not the per-token headline, is the one that's telling the truth.
