Kublr Release 1.25.0 (2023-04-07)

This release has a known critical issue, use Kublr 1.25.3 or a later version instead

Kublr Quick Start

To quickly get started with Kublr, run the following command in your terminal:

sudo docker run --name kublr -d --restart=unless-stopped -p 9080:9080 kublr/kublr:1.25.0

The Kublr Demo/Installer docker container can be run on ARM-based PC, such as MacBook M1.

Follow the full instructions in Quick start for Kublr Demo/Installer.

The Kublr Demo/Installer is a lightweight, dockerized, limited-functionality Kublr Platform with a base trial license. It can be used for the following purposes:

  • Test setup and management of a standalone Kubernetes cluster
  • Setting up a full-featured Kublr Platform
  • Trial license use

The Kublr Demo/Installer stores all data about created clusters inside the Docker container. If you delete the container, you will lose all data about the created clusters and the Kublr platforms. However, you will not lose the clusters and platforms themselves.

Overview

The Kublr 1.25.0 release introduces several new features and improvements, including:

  • Support for Kubernetes 1.25
  • Full ARM-based cluster support
  • CRI-O container runtime
  • OpenSearch-based centralized log collection
  • Updates for PostgreSQL and MongoDB in Kublr Control Plane

All Kublr components are checked for vulnerabilities using Aquasecurity trivy scaner. In addition to these major features, the release also includes various other improvements and fixes.

Important Changes

  • ARM nodes architecture support for:

    • Kublr agent / Kubernetes

      Please note that the annotation is deprecated. Please use the following format instead:

        spec:
          kublrAgentBinary:
            kublrAgentRef: 1.22.17-4
          kublrSeederBinary:
            kublrAgentRef: 1.25.6-4
      

      Instead of the previous format:

        spec:
          kublrAgentTgzUrl: https://repo.kublr.com/repository/gobinaries/kublr/1.22.17-4/kublr-1.22.17-4-linux.tar.gz
          kublrSeederTgzUrl: https://repo.kublr.com/repository/gobinaries/kublr/1.25.6-4/kublr-1.25.6-4-linux.tar.gz
      
    • Kublr Control Plane

    • NGNIX Ingress / CertManager components

    • Monitoring components

    • Logging components:

      • Kublr components and controllers
      • OpenSearch components
      • ELK not implemented for use ARM nodes, you will need to use AMD nodes for this components
  • New versions of Kubernetes:

  • Deprecations:

    • Kubernetes v1.20 (v1.20.14/agent 1.20.14-17) is End of Support
    • Kubernetes v1.21 (v1.21.14 by default) deprecated and will be removed in Kublr v1.26.0
    • Kublr cert-updater is deprecated and should not be used in Kublr agents v1.22 and above
    • Ubuntu 18.04 deprecated and will be removed in Kublr v1.26.0
  • CR-IO CRI Technical preview:

    • Upgrade from Docker/ContainerD is not supported. To switch to CR-IO, use the following cluster specification:
    spec:
      kublrAgentConfig:
        kublr:
          setup:
            runtime_fallback_order: crio
    
  • PostgreSQL upgraded to v11.19.0 with Replication Manager extensions v5.3 If your Kublr Control Plane cluster runs in High Availability (HA) mode:

    spec:
      features:
        controlPlane:
          chart:
            version: 1.24.2
          highAvailability: true
    

    You will need to scale down PostgreSQL replica count to 1, before upgrade begins:

    spec:
      features:
        controlPlane:
          chart:
            version: 1.24.2
          highAvailability: true
          values:
            postgresql:
              postgresql:
                replicaCount: 1
    

    KCP 1.25.0 uses Bitnami PostgreSQL HA helm chart v10.0.9, and the Postgres Replication Manager extensions to v5.3. should be upgraded to v5.3.

  • OpenSearch v2.6.2 is now included in Kublr Centralized Log Collection as a technical preview. To enable this feature, use the following cluster specification:

    spec:
      features:
        logging:
          values:
            opensearch:
              enabled: true
    
  • The Kubernetes Pod Security Policy feature is now deprecated and will be removed in Kubernetes v1.25.

    • PSA is enabled by default starting with Kublr agent v1.23.
    • PSP was enabled in Kublr agent v1.23, deprecated in agent v1.24, and disabled in agent v1.25.
    • Kublr now enforces privileged PodSecurityConfiguration by default. Please use the following YAML configuration:
      apiVersion: pod-security.admission.config.k8s.io/v1beta1
      kind: PodSecurityConfiguration
      defaults:
        enforce: "privileged"
        enforce-version: "latest"
        audit: "privileged"
        audit-version: "latest"
        warn: "privileged"
        warn-version: "latest"
      exemptions:
        usernames: []
        runtimeClasses: []
        namespaces: [kube-system,kublr]
      

Improvements

  • Kublr agents:

    • Upgraded patch versions of supported Kubernetes versions
    • ARM architecture support
    • CR-IO CRI support
    • GCE CSI fully implemented
    • Secomp annoataions added to CoreDNS
  • Kublr Operator:

    • mapkubeapi plugin integrated
    • Kubernetesv1.26 Go client use
  • Kublr Monitoring:

    • Kube-State-Metric server upgraded to 0.6.2
  • Kublr Logging:

    • FluentD upgraded to v4.3.3
  • Kublr feature ingress:

  • vSphere:

    • CPI enabled by default in Kubernetes v1.24 and above
    • Denied using SDRS as default storage class
    • Fixed issue with cluster removal for some nodes
  • AWS:

    • Fully supported Amazon Linux 2 AMIs
  • Azure:

    • Deployment errors now appear on Kublr event tab
  • GCP:

    • Fixed issue with master node SSH
  • Stability, Reliability, and Security:

    • Default terraform version upgraded to v1.1.7
      • Deprecated versions 0.13/0.14 removed from default terraform controller (still available for download by terraform controller for use)
    • oauth2-proxy migrated to use distorless image
    • node-shell plugin added into Kublr Web console
    • Fixed Alertmanager additional config error
  • Various UI Improvements:

    • PSA configuration can be selectable instead of PSP
    • License warning now affects the footer and duplicates null
    • For AWS and GCP installations, use masters without workers for persistent features
    • For vSphere installations, SDRS is now denied as the default storage class
    • Current version of Kublr Agent is no longer displayed while editing/cloning the cluster.

Fixes

  • Increased memory limits for cluster controller, feature controller, and API
  • Fixed GCP master node SSH issue
  • Fixed clusters deployment with PSA in restricted mode, which failed at the Cluster Dashboard step

AirGap Artifacts list

To use Kublr in an airgap environment, you will need to download the following BASH scripts from the repository at https://repo.kublr.com:

You will also need to download the following Helm package archives and Docker images list:

Supported Kubernetes versions

v1.25

v1.24

v1.23

v1.22 (Deprecated in 1.26.0)

v1.21 (Deprecated in 1.25.0, End of support in 1.26.0)

Components versions

Kubernetes

ComponentVersionKublr AgentNotes
Kubernetes1.251.25.6-4 (important to use 1.25.6-6)default v1.25.6
1.241.24.9-4 (important to use 1.24.9-6)
1.231.23.15-4 (important to use 1.23.15-6)
1.221.22.17-4 (important to use 1.22.17-8)Deprecated in 1.26.0
1.211.21.14-22 (important to use 1.21.14-24)End of support in 1.26.0

Kublr Control Plane

ComponentVersion
Kublr Operator1.25.0
Kublr Control Plane1.25.0

Kublr Platform Features

ComponentVersion
Kubernetes
Dashboardv2.7.0
Kublr System1.25.0
LocalPath Provisioner (helm chart version)0.0.23-12
Ingress1.25.0
nginx ingress controller (helm chart version)4.2.3
cert-manager (helm chart version)1.10.0
Centralized Logging1.25.0
ElasticSearch7.10.2
SearchGuard53.6.0
Kibana7.10.2
SearchGuard Kibana plugin53.0.0
SearchGuard Admin7.10.2-53.6.0
OpenSearch (helm chart version)2.6.2
OpenSearch Dashboards(helm chart version)
RabbitMQ3.9.5
Curator5.8.1
Logstash7.10.2
Fluentd4.3.3
Fluentbit1.8.12
Centralized Monitoring1.25.0
Prometheus2.37.1 LTS
Kube State Metrics (helm chart version)4.24.0
AlertManager0.22.0
Grafana8.5.13
Victoria Metrics
Cluster0.9.59
Agent0.8.35
Alert0.5.21

Known issues and limitations

  1. For Kublr Kubernetes clusters deployed on vSphere, perhaps you will need unmount CD/DVD drive 1 from each virtual machine manually using the vCenter console on update error.

  2. Elasticsearch supports only in AMD64 architecture, you can use tag 7.17.9 for ARM64 cluster nodes.

  3. vSphere CSI limitation Please refer to VMWare documentation for vSphere Functionality Supported by vSphere Container Storage Plug-in