DisaggregatedSet

DisaggregatedSet is a Kubernetes controller and CRD (Custom Resource Definition) that extends LeaderWorkerSet (LWS) to support disaggregated inference workloads — use cases where different roles (e.g., prefill, decode, encode) need to run on separate, independently-scaled groups of pods.

This is especially useful for large language model (LLM) inference services where:

• The prefill role (generating the initial KV cache from the input prompt) is compute-bound and benefits from larger pod groups.
• The decode role (token-by-token autoregressive generation) is memory-bandwidth-bound and can run on smaller groups.
• The encode role (optional context encoding) may have different resource requirements from either.

DisaggregatedSet was introduced in KEP-766 to address these multi-role, multi-resource serving patterns with a single, declarative Kubernetes resource.

Relationship to LeaderWorkerSet

DisaggregatedSet does not replace LeaderWorkerSet — it orchestrates multiple LeaderWorkerSets.

Each role defined in a DisaggregatedSet spec maps to an independent LeaderWorkerSet, deployed in the same namespace. This means:

DisaggregatedSet
├── roles[0]: prefill  →  creates LeaderWorkerSet "disaggdeployment-xxx-prefill"
├── roles[1]: decode   →  creates LeaderWorkerSet "disaggdeployment-xxx-decode"
└── roles[2]: encode   →  creates LeaderWorkerSet "disaggdeployment-xxx-encode"

Each child LWS inherits all standard LWS capabilities: rolling updates, subgroup policies, exclusive placement, volume claim templates, and health monitoring.

Roles in DisaggregatedSet

A DisaggregatedSet spec contains a roles list. Each role defines:

DisaggregatedSet coordinates lifecycle and rollouts across roles. Each role’s replica count, rollout strategy, and pod template can be configured independently, while the controller manages them as a single cohesive unit.

When to Use DisaggregatedSet vs Plain LWS

Use plain LWS when:

• All inference pods are homogeneous (same model, same resources).
• You do not need to separate prefill from decode.
• You are running training jobs or batch workloads without disaggregation.

Use DisaggregatedSet when:

• You are running disaggregated LLM inference (e.g., vLLM with P/D disaggregation, SGLang).
• Different inference phases require different GPU types or different pod group sizes.
• You want to scale prefill and decode replicas independently based on different traffic patterns.
• You are evaluating disaggregated serving architectures and need first-class Kubernetes support.

Key Design Principles

1. LWS-native — DisaggregatedSet is built on top of LWS, not alongside it. This means LWS features (failure handling, subgroup topology, exclusive placement) are available per role. Note: rollout strategy is owned by the DisaggregatedSet controller, which replaces the per-LWS rollout to coordinate updates across roles.

2. Coordinated rollouts — Rollouts across roles are coordinated by DisaggregatedSet to preserve capacity ratios (e.g., prefill-to-decode ratio) throughout the update process. Partition-based rollout is not supported.

3. Declarative — The entire multi-role inference topology is expressed in a single YAML manifest, making it easy to version-control and apply via GitOps.

Further Reading

• KEP-766: DisaggregatedSet design document
• Installation guide
• API Reference
• Examples