Token Factory turns raw GPU capacity into a tokens-as-a-service business. It
sits between a GPU fleet and the developers/applications consuming AI, and owns
the surface the customer actually pays for: the OpenAI-compatible API, the model
catalog, per-token metering, multi-tenant isolation, SLAs, and the dashboards.
We are building it for two markets at once, with one shared engineering
effort:
The core idea that keeps the two motions coherent:
We build capabilities once. Editions are packaging.
A budget-enforcing gateway is the same capability whether it ships inside a
single-server enterprise install or a multi-region NeoCloud platform. See
Editions & Packaging.
| Pillar | What it owns | Primary tools | State today |
|---|---|---|---|
| Inference Runtime | Tokens/sec/$ — the serving engine | vLLM, llm-d, NVIDIA Dynamo (eval) | To build (hiring focus) |
| Model Registry | Catalog, weights distribution, BYOW | HF Hub, Harbor/ORAS (eval), Dragonfly (eval) | To build |
| Gateway | Keys, routing, regions, failover | LiteLLM + Token Factory Control Plane | Shipped & in production (amazee.ai) |
| Observability | Metering you can bill on + telemetry | LiteLLM spend logs, Prometheus, Grafana, DCGM | Partially shipped |
The gateway pillar — and much of metering, identity, and the portal — is not
greenfield. It ships today as the amazee.ai Private AI Gateway and becomes
three Token Factory components: the
Control Plane (formerly the
amazee.ai backend), the Portal (formerly
MOAD), and Identity (formerly
moad-auth) — with paying customers in production. The engineers who built them
are joining this team.
Status data lives in
src/data/capabilities.ts
and changes via PR — treat a status change like a code change. Work is tracked
in Jira (TKF);
ticket states are refreshed automatically on a schedule.