Kublr Release 1.24.0 (2022-12-23)

Kublr Quick Start

sudo docker run --name kublr -d --restart=unless-stopped -p 9080:9080 kublr/kublr:1.24.0

The Kublr Demo/Installer docker container can be run on ARM-based PC, such as MacBook M1.

Follow the full instructions in Quick start for Kublr Demo/Installer.

The Kublr Demo/Installer is a lightweight, dockerized, limited-functionality with base trial license Kublr Platform which can be used to:

  • Test setup and management of a standalone Kubernetes cluster
  • Setup a full-featured Kublr Platform
  • Trial license use

The Kublr Demo/Installer stores all of the data about the created clusters inside the Docker container. If you delete the Docker container you will lose all data about the created clusters and the Kublr platforms. However, you will not lose the clusters and the platforms themselves.

Overview

The Kublr 1.24.0 release introduces support for Kubernetes 1.24, AWS Outposts, vSphere CSI drivers, and upgraded Keycloak v20.0.1. It also includes Ubuntu 22.04 support and improvements to the Kublr license management UI.

All Kublr components are checked for vulnerabilities using Aquasecurity trivy scaner. In addition to these major features, the release also includes various other improvements and fixes.

Important Changes

  • Ubuntu 22.04 support

  • New versions of Kubernetes

  • Deprecations

    • Kuberntes v1.19 (v1.19.16/agent 1.19.16-9) is End Off Support
    • Kuberntes v1.20 (v1.20.14 by default) deprecated and will be removed in Kublr v1.25.0
    • Ubunut 16.04 removed from UI
  • Keyclaok upgraded to v20.0.1

  • ContainerD CRI

    • Insecure registry support
  • Kuberntes Pod Security Policy feature is deprecated and will be removed from Kubernetes v1.25

    • (PSA) Enabled by default starting with Kublr agent v1.23
    • (PSP) Enabled in Kublr agent v1.23, deprecated and disabled in agent v1.24
    • Kublr enforce privileged PodSecurityConfiguration:
      apiVersion: pod-security.admission.config.k8s.io/v1beta1
      kind: PodSecurityConfiguration
      defaults:
        enforce: "privileged"
        enforce-version: "latest"
        audit: "privileged"
        audit-version: "latest"
        warn: "privileged"
        warn-version: "latest"
      exemptions:
        usernames: []
        runtimeClasses: []
        namespaces: [kube-system,kublr]
      
  • Kublr backup controller is deprecated in Kublr v1.23 and will be changed to new implemantation in Kublr v1.25.0

Important Changes

  • Support for Ubuntu 22.04 has been added.

  • Support for Kubernetes v1.24 and v1.23 has been added, and v1.20 has been deprecated and will be removed in the next release.

  • Support for Kubernetes v1.19 and Ubuntu 16.04 have been deprecated and will no longer be supported.

  • Keycloak has been upgraded to v20.0.1 and the Kublr Control Plane now uses the Bitnami Keycloak Helm chart.

  • ContainerD CRI now supports insecure registry.

  • The Kubernetes Pod Security Policy feature has been deprecated and will be removed in Kubernetes v1.25. Pod Security Policies are enabled by default in Kublr agent v1.23 and deprecated and disabled in agent v1.24. Kublr enforces the following baseline PodSecurityConfiguration:

    apiVersion: pod-security.admission.config.k8s.io/v1beta1
    kind: PodSecurityConfiguration
    defaults:
      enforce: "baseline"
      enforce-version: "latest"
      audit: "baseline"
      audit-version: "latest"
      warn: "baseline"
      warn-version: "latest"
    exemptions:
      usernames: []
      runtimeClasses: []
      namespaces: [kube-system,kublr]
    
    
  • The Kublr backup controller is deprecated in Kublr v1.23 and will be replaced with a new implementation in Kublr v1.25.0.

Improvements

  • Kublr agents

    • Upgraded patch versions of supported Kubernetes versions
    • Helm 2 and Tiller removed
    • vSphere CSI/CPI fully implemented
  • Kublr Operator:

    • Helm v3.8.0 is used by default
    • Helm client version customization support in kublr.features
    • Feature deployment dependency wait time can be configured via the following values:
      failureRequeueIn: 20s
      failureRequeueInMax: 4m
      
  • Kublr Monitoring:

    • Grafana 8.5.13
    • KubeStateMetrics 4.20.2
    • Victoria Metrics cluster 0.9.32
    • Updated Grafana dashboard
  • Kublr Logging:

  • Kublr feature ingress

  • vSphere:

    • CSI/CPI with enabled CSIMigrationvSphere feature gates fully implemented. Use the following cluster specification:
      spec:
        kublrAgentConfig:
          cloud_controller_manager:
            enabled: true
        kublr:
          features:
            csi_drivers:
              vsphere: csi
      
  • AWS:

  • Azure:

    • Additional resources should be deleted after removal from cluster specification
    • API versions in Azure generator templates updated to the latest
  • Stability, Reliability and security

    • kubectl version in all Kublr components determined by k8s version
    • All KCP ingresses TLS configuration moved to Keycloak defined
  • Various UI Improvements

    • NodeJS updated to 16.18.0 LTS

Fixes

  • Increased memory limits for generator and API to address performance issues.
  • Reduced the number of k8s proxy requests made by the Kublr API to improve efficiency.
  • Fixed an issue where vSphere clusters could not be cloned.
  • Improved the reload time for Prometheus configuration in the Monitoring controller.
  • Added the vpcCIDRblock to the exclude list for proxy use in the Generator.
  • Ensured that the number of CSI controller replicas is equal to the number of master nodes.

AirGap Artifacts list

To use Kublr in an airgap environment, you will need to download the following BASH scripts from the repository at https://repo.kublr.com:

You will also need to download the following Helm package archives and Docker images:

Supported Kubernetes versions

v1.24

v1.23

v1.22

v1.21 (Deprecated in 1.25.0)

v1.20 (Deprecated in 1.24.0, End of support in 1.25.0)

Components versions

Kubernetes

ComponentVersionKublr AgentNotes
Kubernetes1.241.24.8-4default v1.24.8
1.231.23.14-4
1.221.22.16-4
1.211.21.14-10Deprecated in 1.25.0
1.201.20.14-14End of support in 1.25.0

Kublr Control Plane

ComponentVersion
Kublr Operator1.24.0
Kublr Control Plane1.24.0

Kublr Platform Features

ComponentVersion
Kubernetes
Dashboardv2.4.0
Kublr System1.24.0
LocalPath Provisioner (helm chart version)0.0.22-11
Ingress1.24.0
nginx ingress controller (helm chart version)4.2.3
cert-manager (helm chart version)1.5.3
Centralized Logging**1.24.0 *
ElasticSearch7.10.2
Kibana7.10.2
SearchGuard52.3.0
SearchGuard Kibana plugin51.0.0
SearchGuard Admin7.10.2-52.3.0
RabbitMQ3.9.5
Curator5.8.1
Logstash7.10.2
Fluentd3.3.0
Fluentbit1.8.10
Centralized Monitoring1.23.0
Prometheus2.37.1 LTS
Kube State Metrics (helm chart version)4.20.2
AlertManager0.22.0
Grafana8.5.13
Victoria Metrics
Cluster0.9.32
Agent0.8.12
Alert0.4.35

Known issues and limitations

  1. For Kublr Kubernetes clusters deployed on vSphere, you must unmount CD/DVD drive 1 from each virtual machine manually using the vCenter console.

  2. vSphere clusters cannot be deleted automatically. You must remove the VMs manually and unregister the cluster after that.

  3. Kublr Control Plane can only be updated from v1.21.0 or above.

  4. Kubernetes v1.24.0 and above is not supported on SUSE Linux.

  5. The Kublr Control Plane ingress SSL certificate is now managed by the Keycloak ingress rule with the name host-name.domain-name-tls.

  6. AWS CSI volume resizer have wrong images, please use latest Kublr agent versions