Kubernetes Horizontal Pod Autoscaler - Practical Hands-on Tutorial

Step-by-step practical tutorial on setting up and using Kubernetes Horizontal Pod Autoscaler with Metrics Server, deployment configuration, and load testing

Horizontal Pod Autoscaler Example

Download the YAML configuration files for the HPA example

Download components.yaml Download deploy.yaml Download pod.yaml

Check Metrics Server

kubectl get po -n kube-system

Check if the Metrics Server is installed in your cluster. Look for a pod called metrics-server in the kube-system namespace.

Install Metrics Server

kubectl apply -f components.yaml

If the Metrics Server is not installed, apply the components.yaml file to install it.

Create the Deployment

kubectl apply -f deploy.yaml
kubectl get pods

Create the deployment and verify that the pods are running.

Set Autoscaling Limits

kubectl autoscale deployment hpa-deployment --cpu-percent=50 --min=1 --max=4
kubectl get hpa

Set up Horizontal Pod Autoscaler to scale the deployment when CPU usage exceeds 50%, with a minimum of 1 pod and maximum of 4 pods.

Deploy BusyBox

kubectl apply -f pod.yaml

Deploy a BusyBox pod that we'll use to generate load on our application.

Connect to BusyBox

kubectl exec mybox -it -- /bin/sh

Connect to the BusyBox container to run commands that will generate load.

Increase Load

while true; do wget -q -O- http://php-apache; done
kubectl get hpa

Run this endless loop in the BusyBox container to generate load on the application. In a separate terminal, check the HPA status to see the scaling in action.

Launch K9s

k9s

In a separate terminal, launch K9s to visually monitor what's happening in your cluster.

Stop the Load

Press Ctrl-C
exit

Press Ctrl-C to terminate the endless loop and type exit to leave the BusyBox container.

Cleanup

kubectl delete hpa hpa-deployment
kubectl delete -f pod.yaml --grace-period=0 --force
kubectl delete -f deploy.yaml
kubectl delete -f components.yaml

Delete all resources created during this tutorial. The last command is optional if you want to keep the Metrics Server for other uses.

YAML Configuration Files

components.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - nodes/stats
  - namespaces
  - configmaps
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --secure-port=443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        - --kubelet-insecure-tls
        - --metric-resolution=15s
        image: k8s.gcr.io/metrics-server/metrics-server:v0.5.0
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          initialDelaySeconds: 20
          periodSeconds: 10
        resources:
          requests:
            cpu: 100m
            memory: 200Mi
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

Metrics Server Configuration Explanation:

Service Account:

Creates a service account named metrics-server in the kube-system namespace for the Metrics Server to use.

RBAC Configuration:

  • ClusterRole → Defines permissions for reading metrics
  • RoleBinding → Binds the service account to the role
  • ClusterRoleBinding → Binds the service account to cluster-wide roles

Service:

Exposes the Metrics Server on port 443 within the cluster.

Deployment:

  • image → Uses the official Metrics Server image
  • args → Includes --kubelet-insecure-tls for development environments
  • resources → Defines CPU and memory requests
  • securityContext → Runs with restricted permissions

APIService:

Registers the Metrics Server API with Kubernetes API server.

deploy.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hpa-deployment
spec:
  selector:
    matchLabels:
      run: php-apache
  replicas: 1
  template:
    metadata:
      labels:
        run: php-apache
    spec:
      containers:
      - name: php-apache
        image: guybarrette/hpa-example
        ports:
        - containerPort: 80
        resources:
          limits:
            cpu: 500m
          requests:
            cpu: 200m
---
apiVersion: v1
kind: Service
metadata:
  name: php-apache
  labels:
    run: php-apache
spec:
  ports:
  - port: 80
  selector:
    run: php-apache

Deployment Configuration Explanation:

Deployment:

  • name: hpa-deployment → Names the deployment
  • replicas: 1 → Starts with one pod
  • selector.matchLabels → Selects pods with label run: php-apache
  • image: guybarrette/hpa-example → Uses a sample PHP Apache image
  • resources → Sets CPU limits (500m) and requests (200m)

Service:

  • name: php-apache → Names the service
  • port: 80 → Exposes the service on port 80
  • selector → Routes traffic to pods with label run: php-apache

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: mybox
spec:
  restartPolicy: Always
  containers:
  - name: mybox
    image: busybox
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi    
    command:
      - sleep
      - "3600"

BusyBox Pod Configuration Explanation:

Pod:

  • name: mybox → Names the pod
  • restartPolicy: Always → Always restart the pod if it fails

Container:

  • image: busybox → Uses the lightweight BusyBox image
  • resources → Sets CPU and memory requests and limits
  • command → Runs the sleep command for 3600 seconds (1 hour)

Purpose:

This pod is used to generate load on the PHP Apache deployment by running an endless loop that continuously makes HTTP requests.