Configuration Templates¶
Ready-to-use configuration examples.
Platform Configuration¶
Two values.yaml shapes for the operator chart, carrying the KMetal spec under kmetal.spec.
See Platform Configuration Reference for every field, and Configuration Best Practices for what to decide before the first install.
Lab / development — one central node, no external egress, node-local storage for etcd:
kmetal:
spec:
networking:
kubeOVN:
podCIDR: 10.16.0.0/16
tunnelInterface: eth1
centralNodes:
- lab-01
serviceCIDR: 10.96.0.0/16
providerNetworks:
- name: provider
interface: eth2
subnets:
- name: external
cidrBlock: 192.168.100.0/24
gateway: 192.168.100.1
excludeIPs:
- 192.168.100.1
defaultProviderSubnet: external
loadBalancer:
tenantControlPlanes:
addresses:
- 192.168.100.10-192.168.100.20
storage:
# Golden images and machine root disks. Has to snapshot, so this cannot
# be local-path even in a lab — nothing boots without it.
underclusterClassName: platform-storage
# Node-local is fine for etcd, and this is where a lab can use it.
etcdClassName: local-path
# Its own name, so machine root disks are not billed to the tenant's
# storage quota.
tenantClassName: tenant-storage-class
clusterClass:
values:
dataVolume:
# Required even here: the chart refuses to render without it, and
# it must serve the same driver as underclusterClassName.
volumeSnapshotClass: platform-storage-snapshot
multiTenancy:
# Without this Capsule enforces nothing: a Tenant owner it was never
# told about is accepted and then ignored.
users:
- kind: Group
name: system:serviceaccounts:kmetal-tenants
Production — three central nodes for OVSDB quorum, a dedicated overlay VLAN, network-attached tenant storage, and the console on the management segment:
imagePullSecrets:
- name: clastix-ghcr
kmetal:
spec:
networking:
kubeOVN:
podCIDR: 10.16.0.0/16
podGateway: 10.16.0.1
tunnelInterface: bond1.516
tunnelType: geneve
# Odd count, so the Raft cluster survives losing one.
centralNodes:
- worker-01
- worker-02
- worker-03
serviceCIDR: 10.96.0.0/16
providerNetworks:
- name: provider
# Taken wholesale into OVN's bridge — never the node's own interface.
interface: bond1.517
excludeNodes:
- controller-01
subnets:
- name: external
# Zero: bond1.517 already tags at the kernel.
vlanID: 0
cidrBlock: 10.10.20.0/24
gateway: 10.10.20.1
excludeIPs:
- 10.10.20.1
- 10.10.20.2..10.10.20.9 # reserved for platform use
defaultProviderSubnet: external
loadBalancer:
tenantControlPlanes:
addresses:
- 10.10.20.100-10.10.20.200
management:
addresses:
- 10.10.30.200-10.10.30.250
storage:
# Golden images and every machine's root disk: must support snapshots.
underclusterClassName: platform-storage
# Tenant control-plane etcd. Omit to follow the class above; name it
# separately only where that class is the wrong one for etcd.
etcdClassName: platform-etcd
# Handed to tenant workloads, and the key their storage quota counts.
tenantClassName: tenant-storage-class
tenantClaimPropertySets:
- accessModes:
- ReadWriteOnce
volumeMode: Filesystem
clusterClass:
values:
dataVolume:
# No chart default from v1.10.0: rendering fails without this. Must
# serve the same driver as storage.underclusterClassName, the class
# the golden image is snapshotted on.
volumeSnapshotClass: platform-storage-snapshot
kubeadm:
containerDisk:
# Must match the version tenant Clusters request.
tag: v1.34.1
multiTenancy:
# Opt-in throughout — kMetal names nobody. A Tenant owner missing here
# is invisible to admission: its Environments are created outside every
# tenant, with no quota applied, and nothing complains.
users:
- kind: Group
name: system:serviceaccounts:kmetal-tenants
- kind: Group
name: kmetal:tenant-owners
# Owners of every Tenant, present and future. Never someone who also
# owns a tenant of their own.
administrators:
- kind: Group
name: kmetal:platform-team
# The other half of the same problem: when one provider group covers
# every account it has to go in users for tenant owners to match, which
# also drags the platform team into tenant admission. Naming their group
# here takes them back out.
# ignoreUserWithGroups:
# - kmetal:cluster-admins
console:
# Must fall inside loadBalancer.management above.
address: 10.10.30.200
tls:
issuerRef:
name: kmetal-ca
kind: ClusterIssuer
registry:
imagePullSecretName: clastix-ghcr
chartPullSecretName: clastix-ghcr-oci
Neither template sets component versions, replica counts, or resource limits: those travel with the kMetal release.
To move the platform's controllers off the control plane, add spec.placement — see Component Configuration.
Tenant Clusters¶
Tenants are created as CAPI Cluster resources referencing kMetal's kubevirt-kubeadm ClusterClass. The control plane (Kamaji KamajiControlPlane/TenantControlPlane) and infrastructure (CAPK KubevirtCluster) are auto-created from the topology.
Basic (single-replica control plane):
apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
name: basic-tenant
namespace: <tenant-namespace>
spec:
clusterNetwork:
pods:
cidrBlocks: ["10.200.0.0/16"]
services:
cidrBlocks: ["10.201.0.0/16"]
topology:
classRef:
name: kubevirt-kubeadm
namespace: kmetal-capi-providers
version: v1.34.1
variables:
- name: controlPlane
value:
dataStoreName: default
network:
serviceAddress: <metallb-vip>
serviceType: LoadBalancer
certSANs: [<public-fqdn-or-ip>]
- name: machineSize
value: small
- name: bootstrapConfig
value:
serverAddress: <bootstrap-server>
- name: network
value:
subnet: <ovn-subnet-name>
workers:
machineDeployments:
- class: default-worker
name: md-0
replicas: 1
Production (HA control plane, multiple workers):
apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
name: ha-tenant
namespace: <tenant-namespace>
labels:
environment: production
spec:
clusterNetwork:
pods:
cidrBlocks: ["10.200.0.0/16"]
services:
cidrBlocks: ["10.201.0.0/16"]
topology:
classRef:
name: kubevirt-kubeadm
namespace: kmetal-capi-providers
version: v1.34.1
variables:
- name: controlPlane
value:
dataStoreName: default
network:
serviceAddress: <metallb-vip>
serviceType: LoadBalancer
certSANs: [<public-fqdn-or-ip>]
- name: machineSize
value: medium
- name: bootstrapConfig
value:
serverAddress: <bootstrap-server>
- name: network
value:
subnet: <ovn-subnet-name>
workers:
machineDeployments:
- class: default-worker
name: md-0
replicas: 3
Applications¶
Stateless:
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
spec:
replicas: 3
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: web
image: nginx:1.24
ports:
- containerPort: 80
resources:
requests: { cpu: 100m, memory: 128Mi }
limits: { cpu: 500m, memory: 512Mi }
---
apiVersion: v1
kind: Service
metadata:
name: web-app
spec:
selector:
app: web-app
ports:
- port: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: web-app
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
ingressClassName: haproxy
tls:
- hosts:
- app.company.com
secretName: web-app-tls
rules:
- host: app.company.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web-app
port:
number: 80
Stateful:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: database
spec:
serviceName: database
replicas: 3
selector:
matchLabels:
app: database
template:
metadata:
labels:
app: database
spec:
containers:
- name: postgres
image: postgres:15
env:
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgres-creds
key: password
ports:
- containerPort: 5432
volumeMounts:
- name: data
mountPath: /var/lib/postgresql/data
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: ["ReadWriteOnce"]
storageClassName: "" # Replace with your platform's StorageClass
resources:
requests:
storage: 20Gi
Security¶
Network Policy:
# Default deny
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny
spec:
podSelector: {}
policyTypes:
- Ingress
- Egress
---
# Allow ingress
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-ingress
spec:
podSelector:
matchLabels:
app: web-app
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector: {} # Restrict to your tenant cluster's ingress controller namespace
ports:
- protocol: TCP
port: 80