Future state
The Model Registry is deliberately future state: it has no dedicated
repository and its features are not issue-tracked yet — we only care about
LLMs and tokens right now, and the MVP's weight-caching need lives under the
Serving Engine capability.
This page records the target design so the staged approach is agreed before
the pillar gets staffed.
The system of record for models: what exists, what versions, what licenses,
what they cost, what hardware they run on, and how weights get to a replica.
The catalog metadata this pillar owns is what the commercial surface above
(pricing pages, portal model pickers) renders.
This is not just an OCI registry problem: base models fan out into
quantization variants (FP16/FP8/INT4) and LoRA deltas composed at serve time;
weights are too large to pull cross-region on cold-start; customer uploads need
a training-to-inference workflow.
We deliberately stage this pillar — the MVP needs almost none of it, the
NeoCloud edition needs all of it:
| Stage | Mechanism | Tools | Edition |
|---|---|---|---|
| 1. Pull & cache | vLLM pulls from Hugging Face; snapshots cached on a PVC so restarts don't re-download — tracked under Serving Engine | HF Hub, k8s PVC | MVP 🚧 |
| 2. Curated catalog | Catalog service with licenses, hardware profiles, pricing metadata; the existing /public/models API ✅ grows into this |
Control Plane | Enterprise 📋 |
| 3. Artifact registry | Content-addressed model artifacts, signed (cosign) & scanned, curated + customer-private under one access model | Harbor + ORAS 🔍, Sigstore | Enterprise/NeoCloud 📋 |
| 4. Multi-site distribution | Chunked/P2P distribution so a 200 GB checkpoint lands once per site, not once per replica | Dragonfly 🔍 | NeoCloud 🔍 |
| 5. BYOW & fine-tunes | Customer weight upload, LoRA artifacts, promote-from-training-to-serving in one motion | registry + runtime integration | NeoCloud 📋 |
| Feature | Status | Editions | Ticket | Notes |
|---|---|---|---|---|
| Public model catalog API | 🟢 Shipped | Enterprise, NeoCloud | — | Unauthenticated catalog of served models with metadata, consumed by portals and docs. |
| Curated open-weight catalog | ⚪ Planned | Enterprise, NeoCloud | — | Ready-to-serve, licensed catalog (Mistral, Qwen, DeepSeek, MiniMax, Kimi…) with hardware profiles per model. |
| OCI-based model artifact registry | 🟡 Evaluating | Enterprise, NeoCloud | — | Content-addressed model storage (Harbor/ORAS) treating curated and customer artifacts under one access model. |
| Artifact signing & scanning | ⚪ Planned | Enterprise, NeoCloud | — | Every artifact versioned, signed (cosign) and scanned before serve. |
| Bring Your Own Weights (BYOW) | ⚪ Planned | NeoCloud | — | Customers upload proprietary or fine-tuned weights and deploy them through the same registry primitive. |
| LoRA / fine-tune artifact management | ⚪ Planned | NeoCloud | — | Fine-tune deltas as first-class artifacts, composable against base models at serve time. |
| Multi-site P2P weight distribution | 🟡 Evaluating | NeoCloud | — | Chunked/P2P distribution (Dragonfly) so a 200 GB checkpoint lands once per site, not once per replica. |