DisaggregatedSet

DisaggregatedSet is a Kubernetes controller and CRD (Custom Resource Definition) that extends LeaderWorkerSet (LWS) to support disaggregated inference workloads — use cases where different roles (e.g., prefill, decode, encode) need to run on separate, independently-scaled groups of pods.

This is especially useful for large language model (LLM) inference services where:

• The prefill role (generating the initial KV cache from the input prompt) is compute-bound and benefits from larger pod groups.
• The decode role (token-by-token autoregressive generation) is memory-bandwidth-bound and can run on smaller groups.
• The encode role (optional context encoding) may have different resource requirements from either.

DisaggregatedSet was introduced in KEP-766 to address these multi-role, multi-resource serving patterns with a single, declarative Kubernetes resource.

Relationship to LeaderWorkerSet

DisaggregatedSet does not replace LeaderWorkerSet — it orchestrates multiple LeaderWorkerSets.

Each role defined in a DisaggregatedSet spec maps to an independent LeaderWorkerSet, deployed in the same namespace. Child LeaderWorkerSets use a slice index and a revision hash in their names:

DisaggregatedSet "my-inference"
├── roles[0]: prefill  →  LeaderWorkerSet "my-inference-0-<rev>-prefill"
├── roles[1]: decode   →  LeaderWorkerSet "my-inference-0-<rev>-decode"
└── roles[2]: encode   →  LeaderWorkerSet "my-inference-0-<rev>-encode"

Naming format: <DisaggregatedSet-name>-<slice>-<revision-hash>-<role-name>. The revision hash is dynamic — always select child resources with labels (disaggregatedset.x-k8s.io/name, disaggregatedset.x-k8s.io/role, disaggregatedset.x-k8s.io/slice) rather than hardcoding names.

Each child LWS inherits standard LWS capabilities such as subgroup policies, exclusive placement, volume claim templates, and health monitoring. Rollout strategy for the set is owned by the DisaggregatedSet controller (see below).

Key Design Principles

1. LWS-native — DisaggregatedSet is built on top of LWS, not alongside it. This means LWS features (failure handling, subgroup topology, exclusive placement) are available per role. Note: rollout strategy is owned by the DisaggregatedSet controller, which replaces the per-LWS rollout to coordinate updates across roles.

2. Coordinated rollouts — Rollouts across roles are coordinated by DisaggregatedSet to preserve capacity ratios (e.g., prefill-to-decode ratio) throughout the update process. Each role’s rolloutStrategy.rollingUpdateConfiguration sets its maxSurge and maxUnavailable, defaulting to maxSurge: 0, maxUnavailable: 1 like a LeaderWorkerSet, so a rollout needs no capacity beyond the role’s replicas. Set maxSurge: 1, maxUnavailable: 0 on a role to bring the new revision up before the old one drains. Partition-based rollout is not supported.

3. Declarative — The entire multi-role inference topology is expressed in a single YAML manifest, making it easy to version-control and apply via GitOps.

Rollout availability

For a visual introduction to coordinated updates, see Rollout Strategy. It explains the fractional coordination window and how it works alongside each role’s maxSurge and maxUnavailable.

For routing requests while old and new revisions coexist, see Revision Gating (with llm-d).

Each role’s maxUnavailable limits how many of its groups can be unavailable. For fixed replica targets, its availability floor is max(0, replicas - maxUnavailable). If a required role still present in the target has a floor of zero, the controller may remove its last old group without waiting for every role in that old revision to retire together. Other roles retain their own Ready-group floors, even while the old revision is incomplete.

This can interrupt service: a revision needs a Ready group in every required role to serve traffic. For example, explicitly setting maxUnavailable: 1 permits a singleton Prefill role to be entirely unavailable, just like 100%. A fixed 1 Prefill / 2 Decode update with explicit maxSurge: 0 and maxUnavailable: 1 on both roles can drain old 1P/2D to 0P/1D, start new 1P/1D, then replace the remaining old Decode once the new Decode is Ready. Three equal-size group slots suffice. The old Decode continues to count against its own budget after Prefill is removed.

When every required current role has a positive floor, the controller also preserves complete-revision serving capacity. A merely unhealthy peer does not waive that protection. Keeping a singleton role continuously available requires maxUnavailable: 0 and a positive maxSurge, and can require spare capacity. Bootstrap recovery can also require an extra group; scheduler rejection never automatically lowers a positive availability floor to zero.

Scaling during a rollout

For an animated comparison of Deferred and Immediate scale-down, see Scaling During a Rollout.

This applies to static roles[].spec.replicas and External RoleScaler targets. Changing replicas or the scaling policy does not create a new revision.

To allow scale-down before old revisions finish draining:

spec:
  scalingPolicy:
    scaleDownDuringRollout: Immediate

Omitting the policy preserves Deferred behavior. Immediate means the controller considers reductions now, not that Pods disappear instantly: availability and startup safeguards can delay them.

For roles still required by the target, both old and target reductions protect effective desired - maxUnavailable Ready groups, with a minimum of zero. The effective count normally follows the latest request; under Deferred, an External role with overlapping old replicas keeps at least the already-created target count. Requesting more capacity raises this floor. If fewer groups are currently Ready, the controller preserves those groups while growing the target; a smaller old revision is not permission to reduce availability further.

If ordinary scale-up becomes blocked by coordination, either policy can use one temporary extra group per eligible role. That role’s existing target groups must all be Ready and its total Spec must be at its surge ceiling. The extra cannot stack while the role remains over its ceiling or waits for readiness. This does not enable Deferred to shrink the target in response to a lower replica request.

When an Immediate target revision has more replicas than its latest desired count, both Ordinal and Hash follow the same rule for each role:

• If any target Pod is assigned to a node, even if it is still loading, keep at least the final desired count. The same protection applies when scheduling information is incomplete.
• If every target group is confirmed unscheduled, the count may go below desired toward desired + maxSurge - old replicas. Availability checks and the first group of each required role still limit that reduction. This does not shrink a target already at or below desired.

For example, per role: old Spec/Ready 4, target Spec 5 with none Ready, desired 4, maxSurge: 1, and maxUnavailable: 0. A scheduled target can reduce to 4; an entirely unscheduled target can reduce to 1. The first case keeps 8 total groups above the latest ceiling of 5 until new readiness permits old groups to drain, potentially for a long time if startup stalls.

This protects the replica count, not particular Pods. It can keep some unscheduled groups alongside scheduled ones; removing excess above desired can still delete scheduled work, and placement may change before deletion.

Renamed-role rollouts can start a bounded next batch using Ready replacements without waiting for another old group to leave; if a target role later loses all Ready groups, recovery under either policy may reclaim extra groups, even Ready ones, while preserving the last Ready group per role and protected old capacity. Immediate finishes when old replicas are gone and target Spec and Ready equal the latest desired count, including zero.

See KEP-1022 for the exact budgets, renamed-role availability, and recovery rules.