Targeted Deployment of Workloads in a Kubernetes Environment

In many enterprise Kubernetes deployments there is a requirement to have some control on the deployment of workloads to specific hosts.  This is primarily due to the following reasons:

  • Specific hardware for certain workloads
  • Specific nodes meant for certain users
  • Performance requirements for specific workloads

In this article let’s look at some of the basic primitives that Kubernetes provides to achieve such a requirement.

Basic Primitives

  • Node Selector​

Allows PODs (containers) to select nodes based on node labels

  • Node Affinity​

Conceptually similar to Node Selector but with more controls – soft, hard requirements, relative weights

  • Taints ​

Allows nodes to repel PODs (containers)

​Using the Primitives

Let’s go through some examples to understand the usage of the primitives.

Using Node Selector​

nodeselector-1

Steps

  • Cluster Admin sets the labels on the nodes​
  • User specifies the nodeSelector in the POD deployment specification​

Using Node Affinity​

There are currently two types of node affinity

requiredDuringSchedulingIgnoredDuringExecution Specified rules must be met for the POD to be scheduled onto a node. Think of it as a ‘hard’ requirement

preferredDuringSchedulingIgnoredDuringExecutionSpecifies rules that are met on best effort basis and not guaranteed. Think of it as a ‘soft’ requirement.

The “IgnoredDuringExecution” means that if labels on a node change at runtime such that the affinity rules on a pod are no longer met, the pod will still continue to run on the node.

A typical example.

spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: arch
            operator: In
            values:
            - ppc64le
      preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 1
        preference:
          matchExpressions:
          - key: arch
            operator: In
            values:
            - ppc64le

Using Taints and Tolerations​

taint-1

Steps

  • Admin sets the taints on the nodes​
  • User specifies the tolerations in the deployment specification​

Using Node Selector, Taints and Tolerations​

taint-2

Steps

  • Admin sets the labels and taints on the nodes​
  • User specifies the nodeSelector (or affinity) and tolerations in the deployment specification​

Read more about the basic primitives here – https://kubernetes.io/docs/concepts/configuration/assign-pod-node/

Tying all these together

You can leverage these primitives in your helm charts to make the deployment smooth for users.

Take a look at the following examples

Hope you find this useful.

Pradipta Kumar Banerjee

I'm a Cloud and Linux/ OpenSource enthusiast, with 16 years of industry experience at IBM. You can find more details about me here - Linkedin

You may also like...