Short version: Starting the server is one command. The two numbers that decide whether you should run it yourself come before and after that command — the VRAM math (which tells you what fits on the GPUs you can afford) and the cost-per-million (which tells you whether self-hosting actually beats paying a hosted API). This walkthrough gives you both, with the commands in between, using vLLM — the OpenAI-compatible inference server that's become the default way to serve open weights.

1. The one command#

vLLM (the stable 0.25.x line as of August 2026) turns a Hugging Face model id into an OpenAI-compatible API with a single command:

pip install vllm

vllm serve Qwen/Qwen3-32B \
  --host 0.0.0.0 --port 8000 \
  --served-model-name qwen3-32b \
  --api-key "$VLLM_API_KEY"

That exposes /v1/chat/completions, /v1/completions, and /v1/models on port 8000. Prefer containers? The official image's entrypoint is vllm serve, so everything after the image name is a serve flag:

docker run --runtime nvidia --gpus all --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --env "HF_TOKEN=$HF_TOKEN" \
  vllm/vllm-openai:latest \
  --model Qwen/Qwen3-32B --tensor-parallel-size 2

The five flags you'll actually reach for: --tensor-parallel-size N (shard the model across N GPUs), --max-model-len (cap context to shrink the KV cache), --gpu-memory-utilization (default 0.9), --quantization fp8, and --api-key. Everything else is a default you can leave alone until you have a reason not to.

2. The VRAM math (do this before you rent anything)#

The whole "will it fit" question is two terms. Weights first:

weights ≈ params × bytes-per-param — fp16/bf16 = 2 bytes, fp8/int8 = 1, int4 = 0.5

Then the KV cache and overhead: multiply the weight size by roughly 1.3–1.5 for moderate context and concurrency, more like 1.5–2.0 for long-context or high-concurrency production. That second term is what people forget, and it's why a model that "fits" on paper OOMs on boot.

Worked, on Qwen3-32B (32.8B params, dense, Apache-2.0):

The capacities you're fitting under: H100 = 80GB, H200 = 141GB, B200 = 192GB. Memorize those three numbers and the arithmetic above and you can size any serve in your head.

3. The models you can't self-host on a founder budget#

This is the part the hype skips. The two open models everyone is talking about this week are not single-GPU serves:

So when a launch post says "open weights," run the VRAM math before you get excited. Open means you may run it; it doesn't mean you can afford to. For a founder, a dense ~30B model is the sweet spot where self-hosting is a single-GPU decision instead of a capex project.

4. Cost-per-million: when self-hosting actually wins#

Here's the formula that ends most self-hosting debates:

cost per 1M tokens = (GPU hourly ÷ 3600) ÷ throughput-tokens-per-sec × 1,000,000

Plug in real numbers. Qwen3-32B on a single H100 benchmarks around ~2,350 tokens/sec aggregate (batched, GPUStack); an H100 rents for roughly $2.50/hr on the cheaper clouds. That's:

(2.50 / 3600) / 2350 × 1e6 ≈ $0.30 per 1M output tokens

Now compare: DeepSeek V4 Flash's hosted API is $0.28 per 1M output ($0.14 input). Your self-hosted 30B model lands at break-even with a near-frontier hosted model — and only if the GPU runs near 100% utilization. Drop to 40% utilization and your effective cost nearly triples while the API price stays flat. Put differently: that H100 at $2.50/hr is ~$1,800/month, which buys ~6.4B output tokens of DeepSeek Flash — and a fully-pinned H100 barely produces that many.

Self-hosting a mid-size open model rarely beats a well-priced hosted API on raw price. It wins when you're buying something the API can't sell you: data residency, a custom fine-tune, predictable latency, or a model that won't be deprecated out from under you.

That's the honest read. If your only goal is the lowest $/token and a good hosted model exists, re-price your routing and pay the API. Self-host when control is the product — and watch your KV-cache hit rate, because at scale that's the number that actually moves the bill.

5. Hit the endpoint#

Because it's OpenAI-compatible, the client is whatever you already use — just repoint base_url:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1",
                api_key="YOUR_VLLM_API_KEY")

resp = client.chat.completions.create(
    model="qwen3-32b",   # matches --served-model-name
    messages=[{"role": "user", "content": "Say hello in one word."}],
)
print(resp.choices[0].message.content)

That's the whole loop: vllm serve, size it with the VRAM math, price it with the cost-per-million, point your client at it. The command was never the hard part — the two numbers around it are. Once you've decided to run your own, the next question is where: where to actually rent a GPU to serve an open model walks the clouds — CoreWeave, Lambda, Nebius, RunPod, Together — and the utilization break-even that decides between renting by the hour and paying by the token.