kubermatic · SheryarButt · Mar 17, 2026
diff --git a/content/conformance-ee/_index.en.md b/content/conformance-ee/_index.en.md
@@ -0,0 +1,6 @@
++++
+title = "Conformance EE Docs"
+sitemapexclude = true
++++
+
+Conformance EE is a Ginkgo v2-based end-to-end conformance test framework for Kubermatic Enterprise Edition, automatically generating, running, and reporting thousands of test scenarios across cloud providers.
diff --git a/content/conformance-ee/main/_index.en.md b/content/conformance-ee/main/_index.en.md
@@ -0,0 +1,49 @@
++++
+title = "Kubermatic Conformance EE"
+date = 2026-03-17T10:07:15+02:00
+weight = 7
+description = "Learn how to use Kubermatic Conformance EE to automatically generate, run, and report end-to-end conformance test scenarios for Kubermatic Kubernetes Platform clusters."
++++
+
+## What is Conformance EE?
+
+[Conformance EE](https://github.com/kubermatic/conformance-ee) is a project by Kubermatic that provides an automated end-to-end conformance test framework for **Kubermatic Kubernetes Platform (KKP) Enterprise Edition**. It automatically generates, runs, and reports thousands of test scenarios by combinatorially applying configuration modifiers to cloud provider clusters and machine deployments.
+
+## Motivation and Background
+
+Validating Kubernetes cluster configurations across multiple cloud providers, OS distributions, Kubernetes versions, and feature combinations is a complex and error-prone task when done manually. The number of possible configuration permutations grows exponentially, making exhaustive manual testing impractical.
+
+Conformance EE solves this by:
+
+- **Combinatorially generating test scenarios** from cluster modifiers (CNI, proxy mode, expose strategy, etc.) and machine modifiers (CPU, memory, disk, OS distribution, etc.)
+- **Deduplicating clusters** using SHA-256 hashing to avoid creating redundant clusters when different modifier combinations produce the same effective configuration
+- **Parallelizing test execution** across Ginkgo nodes with concurrent worker pools
+- **Providing live visibility** into test progress via Kubernetes ConfigMap reporting and JUnit XML output
+- **Offering an interactive TUI** for configuring and launching test runs directly from the terminal
+
+## Key Features
+
+- **Scenario Matrix Generation**: Combinatorial generation of thousands of test scenarios from provider-discovered settings and user configuration
+- **SHA-256 Deduplication**: Prevents duplicate cluster creation by hashing sanitized cluster specs
+- **Parallel Execution**: 100 concurrent workers across Ginkgo nodes with max 4 clusters created in parallel
+- **Multi-Provider Support**: Currently supports KubeVirt with extensible provider architecture
+- **Interactive Terminal UI**: Built with Bubble Tea for configuration and execution management
+- **In-Cluster Deployment**: Runs as Kubernetes Jobs with ConfigMap-based live reporting
+- **JUnit XML Reports**: Compatible with all major CI systems (Jenkins, GitLab CI, GitHub Actions)
+- **YAML Configuration**: Flexible configuration system for providers, releases, distributions, resources, and timeouts
+
+## Table of Content
+
+{{% children depth=5 %}}
+{{% /children %}}
+
+## Further Information
+
+- [Conformance EE GitHub Repository](https://github.com/kubermatic/conformance-ee)
+- [Kubermatic Kubernetes Platform](https://www.kubermatic.com/products/kubermatic-kubernetes-platform/)
+
+Visit [kubermatic.com](https://www.kubermatic.com/) for further information.
+
+{{% notice tip %}}
+For latest updates follow us on Twitter [@Kubermatic](https://twitter.com/Kubermatic)
+{{% /notice %}}
diff --git a/content/conformance-ee/main/architecture/_index.en.md b/content/conformance-ee/main/architecture/_index.en.md
@@ -0,0 +1,161 @@
++++
+title = "Architecture"
+date = 2026-03-17T10:07:15+02:00
+weight = 5
++++
+
+Conformance EE is a Ginkgo v2-based test framework that follows a three-phase pattern: **Build**, **Execute**, and **Report**. It uses combinatorial scenario generation, SHA-256 deduplication, and parallel execution to efficiently validate KKP cluster configurations.
+
+## High-Level Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    Ginkgo Test Suite                            │
+│   provider/kubevirt/provider_suite_test.go                     │
+│   provider/kubevirt/provider_kubevirt_test.go                  │
+└──────────────────────┬──────────────────────────────────────────┘
+                       │ calls
+                       ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                  build.GetTableEntries()                        │
+│   Combinatorially generates all test scenarios                  │
+│   Deduplicates using SHA-256 of sanitized cluster specs         │
+└──────────────────────┬──────────────────────────────────────────┘
+                       │ uses
+          ┌────────────┼────────────┐
+          ▼            ▼            ▼
+    ┌──────────┐ ┌──────────┐ ┌──────────────┐
+    │settings/ │ │options/  │ │build/        │
+    │cluster.go│ │options.go│ │provider/     │
+    │kubevirt  │ │          │ │kubevirt.go   │
+    └──────────┘ └──────────┘ └──────────────┘
+
+                  Parallel Ginkgo nodes
+          ┌─────────────────────────────────┐
+          │  Node 1     Node 2  ...  Node N │
+          │  cluster/   utils/              │
+          │  ensure.go  provider_suite_     │
+          │  update.go  utils.go            │
+          └─────────────────────────────────┘
+
+                  Reporting
+          ┌─────────────────────────────────┐
+          │  utils/junit_reporter.go        │
+          │  utils/reports_configmap.go     │
+          └─────────────────────────────────┘
+```
+
+## Three-Phase Pattern
+
+### 1. Build Phase
+
+The `build.GetTableEntries()` function generates the complete scenario matrix before any test spec runs:
+
+1. **Discover Infrastructure** — Calls the provider's `DiscoverDefaultDatacenterSettings()` to enumerate VPCs, subnets, storage classes, instance types, etc. from the live infrastructure cluster.
+2. **Generate Cluster Specs** — Combinatorially applies cluster modifiers (CNI, proxy mode, expose strategy, etc.) and datacenter modifiers (network, storage class).
+3. **Deduplicate Cluster Specs** — Serializes each sanitized `ClusterSpec` to JSON, hashes with SHA-256, and uses a `(dcHash[:6], specHash[:6], k8sVersion)` key to identify truly distinct clusters.
+4. **Generate Machine Specs** — For each unique cluster, combinatorially applies machine modifiers (CPU, memory, disk, OS distribution, image source, DNS policy, eviction strategy, etc.).
+5. **Deduplicate Machine Specs** — Same SHA-256 approach applied to machine specs.
+
+The result is a `map[string]*Scenario` where each key is a cluster dedup key and each `Scenario` contains the cluster spec plus all machines to test against it.
+
+### 2. Execute Phase
+
+Test scenarios are executed in parallel across Ginkgo nodes:
+
+- **Suite Setup** (`SynchronizedBeforeSuite`): Expensive one-time operations (project creation, cluster setup) run only on the first parallel Ginkgo node. Results are shared via `[]byte` to all nodes.
+- **Spec Execution**: Each Ginkgo node picks up specs from the scenario table and runs the full machine deployment lifecycle.
+- **Suite Teardown** (`SynchronizedAfterSuite`): Cluster deletion and report publishing run only on the first node after all others finish.
+
+### 3. Report Phase
+
+- **JUnit XML**: Per-spec JUnit XML files written to the reports directory, consumable by any CI system.
+- **ConfigMap Live Reporting**: After each spec, results are patched into a shared Kubernetes ConfigMap using JSON merge patches for live visibility into test progress.
+
+## Components
+
+### Scenario Generator (`build/`)
+
+The scenario generator is the core engine that produces the test matrix. It uses 100 concurrent workers to parallelize cluster and machine spec generation:
+
+```
+Discover Infrastructure
+    ↓ (VPCs, subnets, storage classes, instance types)
+Generate Cluster Specs
+    ↓ (apply 28 cluster modifiers across 17 groups)
+Deduplicate Cluster Specs
+    ↓ (SHA-256 hash: dcNameHash[:6] + specHash[:6] + k8sVersion)
+For Each Unique Cluster:
+    Generate Machine Specs
+        ↓ (apply machine modifiers: CPU, memory, disk, OS, DNS, etc.)
+    Deduplicate Machine Specs
+        ↓ (SHA-256 hash of sanitized spec + machineName)
+Return Map[clusterDedupKey]*Scenario
+```
+
+### Cluster Lifecycle Manager (`cluster/`)
+
+Handles the full lifecycle of KKP clusters:
+
+- **Creation**: Create cluster resource, wait for reconciliation (10 min), validate control plane health, add cleanup finalizers, run smoke tests
+- **Upgrade**: Patch Kubernetes version, wait for reconciliation, re-run health checks
+- **Deletion**: Delete with 25 min timeout, wait for cleanup finalizer-based PV/LB deletion
+
+### Machine Deployment Manager (`utils/`)
+
+Manages machine deployments within user clusters:
+
+- **Setup**: Create `MachineDeployment`, attach OSP annotations, wait for node references, label nodes, wait for Ready state and pod readiness
+- **Update**: Patch kubelet version, wait for rollout, verify new nodes
+
+### Interactive TUI (`ui/`)
+
+A Bubble Tea-based terminal interface that guides users through:
+
+1. Environment selection (local or existing cluster)
+2. Provider selection (currently KubeVirt)
+3. Kubeconfig and credential configuration
+4. Kubernetes version, distribution, datacenter, and modifier selection
+5. Test execution and live monitoring
+
+### Deployment Engine (`deploy/`)
+
+Creates Kubernetes resources for in-cluster test execution:
+
+- Kubernetes client setup from kubeconfig
+- Namespace and RBAC (cluster-admin) creation
+- ConfigMap for provider configuration
+- Secret for kubeconfig credentials
+- Job definition with mounted config and credentials
+
+## Directory Structure
+
+```
+conformance-ee/
+├── cmd/                    # CLI entry point
+│   └── main.go
+├── deploy/                 # Kubernetes deployment utilities
+│   └── kubernetes.go
+├── tests/                  # Core test framework
+│   ├── build/              # Scenario generation engine
+│   │   ├── build.go        # GetTableEntries(), worker pools
+│   │   ├── scenario.go     # Entry point for scenario generation
+│   │   ├── types.go        # Core types (Scenario, clusterJob)
+│   │   ├── labels.go       # Ginkgo label helpers
+│   │   ├── provider.go     # Provider-agnostic building
+│   │   └── provider/       # Provider implementations
+│   ├── cluster/            # Cluster lifecycle
+│   │   ├── ensure.go       # Creation and health validation
+│   │   └── update.go       # Version upgrade logic
+│   ├── options/            # Configuration loading
+│   │   └── options.go      # YAML config and runtime options
+│   ├── settings/           # Modifier definitions
+│   │   ├── types.go        # Core modifier types
+│   │   ├── cluster.go      # 28 cluster modifiers
+│   │   └── kubevirt.go     # KubeVirt machine/DC modifiers
+│   ├── utils/              # Test utilities and reporters
+│   └── provider/kubevirt/  # Ginkgo test suite
+├── ui/                     # Interactive TUI
+├── Dockerfile.ginkgo       # Multi-stage Docker build
+└── go.mod                  # Go module dependencies
+```
diff --git a/content/conformance-ee/main/compatibility-matrix/_index.en.md b/content/conformance-ee/main/compatibility-matrix/_index.en.md
@@ -0,0 +1,38 @@
++++
+title = "Compatibility Matrix"
+date = 2026-03-17T10:07:15+02:00
+weight = 35
++++
+
+This matrix reflects the tested combinations of Conformance EE with Kubermatic Kubernetes Platform and Kubernetes versions.
+
+## Component Compatibility
+
+| Conformance EE | KKP Version | Kubernetes | Go | Ginkgo |
+|----------------|-------------|------------|----|--------|
+| main | v2.30 | v1.31, v1.32 | 1.25 | v2.28.1 |
+
+## Supported Cloud Providers
+
+| Provider | Status | Notes |
+|----------|--------|-------|
+| KubeVirt | Supported | Full infrastructure discovery and scenario generation |
+
+## Supported OS Distributions
+
+| Distribution | Versions | KubeVirt |
+|-------------|----------|----------|
+| Ubuntu | 20.04, 22.04 | ✔️ |
+| RHEL | 8, 9 | ✔️ |
+| Flatcar | 3374.2.2 | ✔️ |
+| Rocky Linux | 8, 9 | ✔️ |
+
+## Key Dependencies
+
+| Dependency | Version |
+|------------|---------|
+| Kubermatic SDK | v2.30.0 |
+| k8s.io/client-go | v0.35.2 |
+| KubeVirt API | v1.3.1 |
+| Bubble Tea | v1.3.10 |
+| Gomega | v1.39.0 |
diff --git a/content/conformance-ee/main/configuration/_index.en.md b/content/conformance-ee/main/configuration/_index.en.md
@@ -0,0 +1,12 @@
++++
+title = "Configuration"
+date = 2026-03-17T10:07:15+02:00
+weight = 20
++++
+
+Conformance EE is configured via a YAML file pointed to by the `CONFORMANCE_TESTER_CONFIG_FILE` environment variable. If not set, built-in defaults are used.
+
+## Table of Content
+
+{{% children depth=5 %}}
+{{% /children %}}