Skip to content

Release notes for kOps 1.24 series

Significant changes

Hetzner Cloud support

Adds experimental Hetzner Cloud support. To get started check the docs

Karpenter support

By enabling the Karpenter feature flag, users can now create InstanceGroups managed by Karpenter:

spec:
  manager: Karpenter

You can also start a Karpenter-only cluster with kops create cluster --instance-manager=karpenter ...

kOps will directly manage the Karpenter Provisioner resources. Read more about how Karpenter works on kOps in the Karpenter docs.

Other significant changes

  • Adds support for Ubuntu 22.04

  • Adds support for Rocky Linux 8

  • Adds support for overriding the Kubernetes version when upgrading a cluster by using the --kubernetes-version flag.

  • The minimum version for the Terraform AWS Provider has been bumped to 4.0.0 to address the deprecation of the aws_s3_bucket_object resource and its replacement with the aws_s3_object resource. Such resources will be destroyed and recreated without downtime when applying the changes.

  • ARM64 support for nvidia device driver. Nvidia nodes on ARM64 requires Ubuntu 22.04 AMIs.

Breaking changes

  • The nfs-common/nfs-utils package is no longer installed by default. Use the packages option at instance group level to add it back.

Control plane taints and labels

As of Kubernetes version 1.24, the control plane (formerly master) nodes no longer have the deprecated node-role.kubernetes.io/master label. The deprecated node-role.kubernetes.io/master taint has been replaced by node-role.kubernetes.io/control-plane. If you run your own workload on the control plane, you have to adjust your Pod spec to accommodate for this change.

The following shows a node affinity/node label selector and tolerations that works with both new and old control plane nodes:

spec:
  affinity:
    nodeAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        nodeSelectorTerms:
        - matchExpressions:
          - key: node-role.kubernetes.io/control-plane
            operator: Exists
        - matchExpressions:
          - key: node-role.kubernetes.io/master
            operator: Exists
  tolerations:
  - key: node-role.kubernetes.io/control-plane
    operator: Exists
  - key: node-role.kubernetes.io/master
    operator: Exists

Removing the kubernetes.io/role label

The deprecated kubernetes.io/role label has been removed for all roles as of Kubernetes version 1.24. Use node-role.kubernetes.io/<role> label instead.

Cert Manager removes old API versions

Cert Manager upgraded from 1.6 to 1.8. This has backwards-breaking changes. See upgrading from 1.6 to 1.7 and [1.1.7 to 1.8.

In particular, if you are using the snapshot-controller addon, upgrade your cluster to kOps 1.23 before upgrading to kOps 1.24 to ensure the certificate has the correct API version.

Other breaking changes

  • Support for Kubernetes version 1.18 has been removed.

  • Support for Aliyun/Alibaba Cloud has been removed.

  • Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

  • Cluster Autoscaler can no longer run on spot instances. This is to avoid cluster autoscaler not being scheduled if the spot instances terminate.

Required actions

Deprecations

  • Support for Kubernetes version 1.19 is deprecated and will be removed in kOps 1.25.

  • Support for Kubernetes version 1.20 is deprecated and will be removed in kOps 1.26.

  • All legacy addons are deprecated in favor of managed addons, including the metrics server addon and the autoscaler addon.

  • Due to lack of maintainers, the CloudFormation support has been deprecated. The current implementation will be left as-is until the implementation needs updates or otherwise becomes incompatible. At that point, it will be removed. We very much welcome anyone willing to contribute to this target.

  • Support for Docker has been removed for Kubernetes 1.24+. See https://kubernetes.io/blog/2020/12/02/dockershim-faq

  • The minimum supported version of the AWS EBS CSI Driver is now v1.3.1.

Other changes of note

Full change list since 1.23.0 release

1.24.0-alpha.1 to 1.24.0-alpha.2

1.24.0-alpha.2 to 1.24.0-alpha.3

1.24.0-alpha.3 to 1.24.0-alpha.4

1.24.0-alpha.4 to 1.24.0-alpha.5

1.24.0-alpha.5 to 1.24.0-beta.1

1.24.0-beta.1 to 1.24.0-beta.2

1.24.0-beta.2 to 1.24.0-beta.3

1.24.0-beta.3 to 1.24.0

1.24.0 to 1.24.1

1.24.1 to 1.24.2

1.24.2 to 1.24.3

1.24.3 to 1.24.4

1.24.4 to 1.24.5