Rollout Strategy

A serving application may need Encode, Prefill, and Decode to work together. Updating all the Encode groups while barely starting Prefill and Decode can leave new groups without the matching roles they need.

DisaggregatedSet coordinates their replacement by proportion, not by equal numbers of groups. The fractional coordination window limits how far one role’s rollout can lead another. Separately, each role’s maxSurge and maxUnavailable control extra replicas and availability.

The example below replaces 4 Encode, 13 Prefill, and 7 Decode groups from revision A with revision B.

Set the rollout budgets

The unit is a replica group, which can contain a leader Pod and worker Pods. A group counts as Ready only when all its expected Pods are Ready. A running leader alone is not enough.

For example, set both budgets to 10% in each role’s spec.rolloutStrategy.rollingUpdateConfiguration:

rolloutStrategy:
  rollingUpdateConfiguration:
    maxSurge: 10%
    maxUnavailable: 10%

• maxSurge allows extra groups above the desired count, including groups still starting. Percentages round up.
• maxUnavailable allows Ready capacity to fall below the desired count. Percentages round down.

The minimum Ready count is desired - maxUnavailable; the maximum requested count is desired + maxSurge. Both count across old and new revisions of the role. Terminating Pods can take time to disappear.

What is the fractional window?

The window limits how far one role can get ahead of the others. Its width is 1 / smallest non-zero role count, calculated automatically.

For the old revision, progress is the fraction of its original groups removed. The window moves with the least-advanced role; the others can advance within its width. It is not a batch size.

The new revision’s growth is coordinated separately, using the fraction of each role’s target already requested.

The window is not an unavailability allowance

Fitting inside the window is not enough: a removal must also respect each role’s maxUnavailable budget.

The new revision counts as replacement serving capacity only once it has a Ready group in every required role. Groups still starting do not count as Ready capacity.

These checks govern ordinary rollout steps; they cannot prevent unrelated Pod or node failures or guarantee enough GPU capacity. For changing replica counts, see Scaling during a rollout. The design proposal also covers interrupted rollouts and bounded recovery.