Kubernetes Object Configuration with Kustomize


January 17, 2024

This guide is based on the latest realease of Kustomize at the time of writing (v5.3.0). Please check the oficial guide if you are using any newer versions.

Introduction

Kustomize is a standalone tool to customize Kubernetes objects through a kustomization file (source).

Bases and Overlays and Composing Resources

  • base: A base is a directory with a kustomization.yaml, which contains a set of resources and associated customization.
  • overlay: An overlay is a directory with a kustomization.yaml that refers to other kustomization directories as its bases. A base has no knowledge of an overlay and can be used in multiple overlays.

The resources field, in the kustomization.yaml file, defines the list of resources to include in a configuration.

For example, kustomization.yaml:

resources:
- deployment.yaml
- service.yaml

Let’s see an example with the following files:

  • base/deployment.yaml
  • base/service.yaml
  • base/kustomization.yaml
  • dev/kustomization.yaml
  • prod/kustomization.yaml

base/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-nginx
spec:
  selector:
    matchLabels:
      run: my-nginx
  replicas: 2
  template:
    metadata:
      labels:
        run: my-nginx
    spec:
      containers:
      - name: my-nginx
        image: nginx

base/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: my-nginx
  labels:
    run: my-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
  selector:
    run: my-nginx


base/kustomization.yaml:

resources:
- deployment.yaml
- service.yaml

dev/kustomization.yaml:

resources:
- ../base
namePrefix: dev-

prod/kustomization.yaml

resources:
- ../base
namePrefix: prod-

For each kustomization.yaml file run the following command to view resources in a path and apply resources. Additionally, you can compare the Deployment object dev-my-nginx against the state that the cluster would be in if the manifest was applied:

# To view resources in a path
kubectl kustomize <kustomization_directory>

# To apply resources
kubectl apply -k <kustomization_directory>

# compare the Deployment object
kubectl diff -k <kustomization_directory>

Kustomize Built-Ins

In the kustomization.yaml file:

namespace: my-namespace
namePrefix: dev-
nameSuffix: "-001"
commonLabels:
  app: bingo
commonAnnotations:
  oncallPager: 800-555-1212
buildMetadata: [managedByLabel]
# originAnnotations, transformerAnnotations are
# two other options in addition to managedByLabel

Labels

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

labels:
  - pairs:
      someName: someValue
      owner: alice
      app: bingo
    # add labels to spec.selector.matchLabels
    includeSelectors: true 
    # add labels to spec.template.metadata.labels
    includeTemplates: true 

ConfigMapGenerator

Assume we have application.properties file content as:

FOO=Bar

Then your kustomization.yaml looks like:

From a file

configMapGenerator:
- name: example-configmap-1
  files:
  - application.properties

From literals

# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: my-java-server-env-vars
  literals:
  - JAVA_HOME=/opt/java/jdk
  - JAVA_TOOL_OPTIONS=-agentlib:hprof

From ENV variables

# kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
configMapGenerator:
- name: tracing-options
  envs:
  - tracing.env

Patches

Patches can be used to apply different customizations to Resources.

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

patches:

# JSON inline patch
# op can be [replace, delete, ]
  - patch: |-
      - op: replace
        path: /some/existing/path
        value: new value    
    target:
      kind: MyKind
      labelSelector: "env=dev"
    options:
      # If these options are set to false
      # the kind and name of the resource will be untouched
      allowNameChange: true
      allowKindChange: true

# Inline Strategic Merge
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: dummy-app
        labels:
          app.kubernetes.io/version: 1.21.0      
          
  - patch: |-
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: not-used
      spec:
        template:
          spec:
            containers:
              - name: nginx
                image: nginx:1.21.0      
    target:
      labelSelector: "app.kubernetes.io/name=nginx"

# Same thing as above but with JSON6902
  - patch: |-
      - op: add
        path: /metadata/labels/app.kubernetes.io~1version
        value: 1.21.0      
    target:
      group: apps
      version: v1
      kind: Deployment

  - patch: |-
      - op: replace
        path: /spec/template/spec/containers/0/image
        value: nginx:1.21.0      
    target:
      labelSelector: "app.kubernetes.io/name=nginx"

# Using Path Strategic Merge
  - path: add-label.patch.yaml
  - path: fix-version.patch.yaml
    target:
      labelSelector: "app.kubernetes.io/name=nginx"

Where we have add-label.patch.yaml and and fix-version.patch.yaml as:

add-label.patch.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dummy-app
  labels:
    app.kubernetes.io/version: 1.21.0

fix-version.patch.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: not-used
spec:
  template:
    spec:
      containers:
        - name: nginx
          image: nginx:1.21.0

images

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

images:
- name: postgres
  newName: my-registry/my-postgres
  newTag: v1
- name: nginx
  newTag: 1.8.0
- name: my-demo-app
  newName: my-app
- name: alpine
  digest: sha256:24a0c4b4a4c0eb97a1aabb8e29f18e917d05abfe1b7a7c07857230879ce7d3d3

replacements

Replacements are used to copy fields from one source into any number of specified targets.

kustomization.yaml:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

replacements:
  # a path to a replacement
  - path: replacement.yaml

  # inline replacements
  - source:
      kind: Deployment
      fieldPath: metadata.name
    targets:
    - select:
        name: my-resource

replacement.yaml:

source:
  kind: Deployment
  fieldPath: metadata.name
targets:
  - select:
      name: my-resource

Check the full schema of replacements from the official source.


Linux Kubernetes