Status: Accepted · Date: 2026-07 · Deciders: TF engineering leadership
The Enterprise MVP scope doc proposed Ollama (or single-node vLLM) as a
deliberate stand-in engine — "v1 is about the management experience, not
serving performance" — with the argument that the OpenAI-compatible surface
makes the engine swappable later.
Use vLLM from day one. NVIDIA Dynamo is not part of the MVP — it's the
scale-out track's evaluation (status board).
- The swap argument cuts the other way. If the engine is truly swappable,
the cost of choosing the production engine now is near zero — while choosing
Ollama buys a guaranteed migration later. vLLM's OpenAI-compatible server is
one container with well-known Helm patterns; on a k8s+GPU cluster it is not
meaningfully harder to deploy than Ollama.
- Ollama is awkward specifically on Kubernetes. Model pull lifecycle
inside pods, GGUF-only quantization, no tensor parallelism, and throughput
far below vLLM under concurrent load. The demo audience (Accenture/Dell
technical evaluators) may well load-test the thing.
- Credibility. "The demo runs the same engine as production" is a stronger
channel-partner story than "we'll swap the engine later." It also makes every
benchmark number transferable.
- The team we're hiring for the runtime pillar works on vLLM. Every hour
spent operating it — even in a demo — compounds; hours spent on Ollama don't.
- MVP week 1 carries slightly more setup than
ollama pull (model download,
GPU memory sizing, a values file) — accepted, and mitigated by shipping a
tested values-single-node.yaml with a known-good 7–8B model.
- Air-gapped/CPU-only laptop demos lose Ollama's convenience; if that becomes a
real ask, a CPU llama.cpp profile can be added to the chart later without
touching anything above the gateway.
- Everything above the gateway remains engine-agnostic — the principle from the
MVP doc survives; only the default changes.