Support topologySpreadConstraints about agones HOT 8 OPEN

Sorry, preferedDuringSchedulingIgnoredDuringExecution written in the comment is incorrect. preferredDuringSchedulingIgnoredDuringExecution is correct.
Actually, I set the settings in the comments below to agones-system's allocator.
#3533 (comment)

from agones.

The kubernetes scheduler will generally try to best-effort spread the deployment for the allocator / controller / extensions across nodes, but it will not force spreading. If you have something like 3 nodes in the agones system namespace, it would make sense for us to be able to specify constraints to ensure that the pods don't all land on the same node.

I haven't read too much about the topologySpreadConstraints; I'm wondering if that is something that we can add to the installation yaml in a way that will "just work" (e.g. spread the pods if there are multiple nodes but allow them to be packed if there is only one node) or if this is a field we will need to expose as a new helm parameter.

from agones.

I believe that if you set whenUnsatisfiable: ScheduleAnyway, it allows to be packed if there is only one node.
https://kubernetes.io/docs/concepts/scheduling-eviction/topology-spread-constraints/#spread-constraint-definition

from agones.

We do support setting arbitrary affinity. Could the same thing not be done with kubernetes.io/hostname podAntiAffinity - or does that not work for this use case?

See:
https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity:~:text=In%20the%20following%20example%20Deployment%20for%20the%20Redis%20cache%2C%20the%20replicas%20get%20the%20label%20app%3Dstore.%20The%20podAntiAffinity%20rule%20tells%20the%20scheduler%20to%20avoid%20placing%20multiple%20replicas%20with%20the%20app%3Dstore%20label%20on%20a%20single%20node.%20This%20creates%20each%20cache%20in%20a%20separate%20node.

from agones.

Confirmed that with the follwoing podAntiAffinity rules, we can make the spread across zone per component.
Note: you can take topology.kubernetes.io/zone as well.

values.yaml

  allocator:
    affinity:
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                  - key: multicluster.agones.dev/role
                    operator: In
                    values:
                      - allocator
              topologyKey: topology.gke.io/zone

As a result of Agones instllation into GKE with 3 nodes/zones.

(⎈ |gke-regular-01:agones-system) ❯ k get pods -o custom-columns="Pod name":.metadata.name,"Node":.spec.nodeName
Pod name                             Node
agones-allocator-7fbb58b9b-6ppkn     gke-gke-regular-01-pool-01-14f53d6d-s44k
agones-allocator-7fbb58b9b-gsrtf     gke-gke-regular-01-pool-01-f15e50b8-4d0q
agones-allocator-7fbb58b9b-vgmpz     gke-gke-regular-01-pool-01-d0308131-mrl6
agones-controller-658488b8c-8hqzj    gke-gke-regular-01-pool-01-14f53d6d-s44k
agones-controller-658488b8c-nkblv    gke-gke-regular-01-pool-01-d0308131-mrl6
agones-extensions-56cd877688-6glhq   gke-gke-regular-01-pool-01-d0308131-mrl6
agones-extensions-56cd877688-mq5kr   gke-gke-regular-01-pool-01-14f53d6d-s44k
agones-ping-6dd86d54c8-95gjt         gke-gke-regular-01-pool-01-f15e50b8-4d0q
agones-ping-6dd86d54c8-dwrbl         gke-gke-regular-01-pool-01-14f53d6d-s44k

from agones.

Thanks for the suggestion!

In the case of podAntiAffinity, if the number of pods is larger than the number of nodes, I think the zones will not be distributed evenly.
On the other hand, if we use maxSkew in topologySpreadConstraints, we can distribute zones evenly, so I thought topologySpreadConstraints is more appropriate.

However, in my environment, the number of each component of agones-system is not large, so I will use podAntiAffinity.

from agones.

There was a problem, so I reopened this issue.
I would like to adopt topologySpreadConstraints instead of preferedDuringSchedulingIgnoredDuringExecution for the following reasons

• preferredDuringSchedulingIgnoredDuringExecution is not a function to ensure zone distribution.
  • In my verification, it was sometimes deployed on one server, and if there are more pods than zones, they are not evenly distributed.
• When using requiredDuringSchedulingIgnoredDuringExecution, if there are the same number of pods as zones, rolling updates will not be possible. New pods cannot be started because there are already pods deployed in each zone.
• we use maxSkew in topologySpreadConstraints, we can distribute zones evenly

from agones.

Out of curiosity - what preferedDuringSchedulingIgnoredDuringExecution configuration did you use?

from agones.