From ea40257911b988a6a6bbb066e84ddc5b52016409 Mon Sep 17 00:00:00 2001 From: Ben Perry Date: Thu, 22 May 2025 11:53:00 -0500 Subject: [PATCH 1/2] Proposal for placement sort order and score visibility Signed-off-by: Ben Perry --- .../225-placement-score-visibility/README.md | 173 ++++++++++++++++++ .../metadata.yaml | 13 ++ 2 files changed, 186 insertions(+) create mode 100644 enhancements/sig-architecture/225-placement-score-visibility/README.md create mode 100644 enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml diff --git a/enhancements/sig-architecture/225-placement-score-visibility/README.md b/enhancements/sig-architecture/225-placement-score-visibility/README.md new file mode 100644 index 0000000..c366de4 --- /dev/null +++ b/enhancements/sig-architecture/225-placement-score-visibility/README.md @@ -0,0 +1,173 @@ +# Placement Score Visibility + +## Release Signoff Checklist + +- [] Enhancement is `provisional` +- [] Design details are appropriately documented from clear requirements +- [] Test plan is defined +- [] Graduation criteria for dev preview, tech preview, GA +- [] User-facing documentation is created in [website](https://github.com/open-cluster-management-io/open-cluster-management-io.github.io/) + +## Summary + +This proposal extends existing Placement mechanisms to surface the scores of each cluster +in PlacementDecision so that users (or future OCM features) can make advanced scheduling decisions +based on the scored state of clusters. + +In addition, it adds support for different sort strategies on the decisions generated by a Placement. +By default, decisions will continue to be sorted by "Name". An additional sort strategy "Score" will +enable sorting decisions numerically in descending order by prioritizer score. + +## Motivation + +Placement scores are a powerful feature of OCM for scheduling workloads, but there is currently +no reliable way for users to see what scores were assigned to clusters for a placement. Scores are +reported in events, but these are ephemeral. Since PlacementDecisions are orderd alphanumerically +there is no good way to see the relative scores of clusters in a decision other than recomputing +them for each cluster in the decision group(s). + +A user may wish to use Placement to find all clusters that match some conditions, but then pick +only the top N clusters for their workload. While this could be accomplished by setting +`numberOfClusters: N` on the placement, this means a new placement must be created for every value +of N. Another use-case would be to distribute workloads out to selected clusters randomly weighted +by their scores. There is currently no way to do this in OCM without recomputing scores for each cluster +listed in the PlacementDecisions. Displaying scores in an easy to consume manner gives more flexibility +to the user, and enables a single Placement to be reused for many different scheduling decisions +(reducing API pressure on the hub). + +## Proposal + +Add `SortBy` to `PlacementSpec` to configure the sort order for PlacementDecision. This field +is an enum with values "Name" and "Score". Add `Score` to `ClusterDecision` to reflect the +latest score of each cluster in the decision. + +### Design Details + +#### API change + +Field for sorting decisions + +```go +type PlacementSpec struct { + ... + + // SortBy sets the sort order for decisions. + // It can be "Name", "Score", or "", where "" is "Name" by default. + // If sortBy is "Name", decisions will be orderd alphanumerically by cluster name + // If sortBy is "Score", decisions will be ordered numerically in descending order by score, + // then by cluster name in the event of a tie + // +optional + SortBy PlacementSortByType `json:"sortBy,omitempty"` +} + +// +kubebuilder:validation:Enum=Name;Score +type PlacementSortByType string + +const ( + // PlacementSortByName sorts decisions alphanumerically by cluster name + PlacementSortByName PlacementSortByType = "Name" + // PlacementSortByScore sorts decisions numerically in descending order by score. + // If one or more clusters are tied on score they will be sorted by name + PlacementSortByScore PlacementSortByType = "Score" +) +``` + +Display score to consumer of decisions + +```go +type ClusterDecision struct { + ... + + // Score is the computed score for the cluster based on configured prioritizers + Score int64 `json:"score"` +} + +``` + +#### hub implementation + +In `scheduling_controller.go`, update `divideDecisionGroups` to check the value of `SortBy` before +sorting and splitting clusters into groups. Any pre-determined decision groups set in +`placement.Spec.DecisionStrategy.GroupStrategy.DecisionGroups` will be individually ordered within their +named groups by the `SortBy` strategy. Any remaining clusters will be sorted across all remaining +groups before being split. + +Prioritizer scores will be passed along with the clusters slice so that each cluster's score can +be looked up when generating `clusterapiv1beta1.ClusterDecision`. + +#### examples + +Sort placement decisions by score + +```yaml +apiVersion: cluster.open-cluster-management.io/v1beta1 +kind: Placement +metadata: + name: h100 + namespace: example +spec: + clusterSets: + - global + predicates: + - requiredClusterSelector: + labelSelector: + matchLabels: + gpu.example.com/h100: enabled + prioritizerPolicy: + configurations: + - scoreCoordinate: + addOn: + resourceName: hardware-availability + scoreName: h100 + type: AddOn + weight: 1 + sortBy: Score +--- +apiVersion: cluster.open-cluster-management.io/v1beta1 +kind: PlacementDecision +metadata: + labels: + cluster.open-cluster-management.io/decision-group-index: "0" + cluster.open-cluster-management.io/decision-group-name: "" + cluster.open-cluster-management.io/placement: h100 + name: h100-decision-1 + namespace: example + ownerReferences: + - apiVersion: cluster.open-cluster-management.io/v1beta1 + controller: true + kind: Placement + name: h100 +status: + decisions: + - clusterName: cluster-2 + score: 80 + - clusterName: cluster-1 + score: 65 + - clusterName: cluster-5 + score: 10 + - clusterName: cluster-6 + score: 10 +``` + + +### Test Plan +- test that the default alphanumeric sort order remains the same +- test that `sortBy: Score` properly orders clusters across decision groups +- test that named decision groups are sorted individually based on `sortBy` +- test that decision groups are updated when scores change + +### Graduation Criteria +N/A + +### Upgrade Strategy +It will need upgrade on CRD of Placement and PlacementDecision on hub cluster. No changes to agents. + +When a user needs to use this feature with an existing `Placement`, the user needs to add +`sortBy: Score` to their placements. No changes are required to update existing decisions +with score (regardless of sortBy). Once the placement controller is updated, all +`PlacementDecisions` will be updated with prioritizer scores on their next reconciliation. + +### Version Skew Strategy +- The new fields are optional, and if not set, the placement decisions will be ordered the same +as in previous versions. +- Older versions of the placement controller will ignore the newly added fields diff --git a/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml b/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml new file mode 100644 index 0000000..aebfe88 --- /dev/null +++ b/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml @@ -0,0 +1,13 @@ +title: workoad-completion +authors: + - "@bhperry" +reviewers: + - "@deads2k" + - "@elgnay" + - "@zhujian7" +approvers: + - "@elgnay" +creation-date: 2025-05-22 +last-updated: 2025-05-22 +status: provisional +see-also: [] From 3c59f2cc79a62461cae4c1f74889af156dca4518 Mon Sep 17 00:00:00 2001 From: Ben Perry Date: Tue, 27 May 2025 09:39:17 -0500 Subject: [PATCH 2/2] Update metadata Signed-off-by: Ben Perry --- .../225-placement-score-visibility/metadata.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml b/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml index aebfe88..5404b8c 100644 --- a/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml +++ b/enhancements/sig-architecture/225-placement-score-visibility/metadata.yaml @@ -1,4 +1,4 @@ -title: workoad-completion +title: placement-score-visibility authors: - "@bhperry" reviewers: