Skip to content

Create Your First Cluster

This guide walks you through creating your first tenant cluster on kMetal. The tenant cluster runs as a Cluster API Cluster referencing the platform-provided kubevirt-kubeadm ClusterClass — the ClusterClass handles the rest (Kamaji control plane, KubeVirt worker VMs, networking).

Prerequisites

Before creating your first cluster, ensure you have:

  • A working kMetal installation (see Install kMetal).
  • kubectl access to the under cluster.
  • A Tenant and an Environment (see Create First Environment). The Cluster is created in the Environment's namespace.
  • The name of the Environment's SubnetClaim, which the Cluster joins its worker VMs to.
  • A CNI for tenant clusters, already declared by the platform. kMetal installs none, and nodes stay NotReady until one is delivered — see Give tenant clusters a CNI.

Step 1: Create the Cluster

Create a Cluster resource referencing the kubevirt-kubeadm ClusterClass:

apiVersion: cluster.x-k8s.io/v1beta2
kind: Cluster
metadata:
  name: solar
  namespace: dynamo-prod
spec:
  clusterNetwork:
    pods:
      cidrBlocks:
        - 10.93.0.0/16
    services:
      cidrBlocks:
        - 10.96.0.0/16
  topology:
    classRef:
      name: kubevirt-kubeadm
      namespace: kmetal-capi-providers
    variables:
      - name: controlPlane
        value:
          dataStoreName: default
          network:
            serviceType: LoadBalancer
      - name: machineSize
        value: small
      - name: network
        value:
          subnet: primary
    version: v1.34.1
    workers:
      machineDeployments:
        - class: default-worker
          name: md-0
          replicas: 2

Three fields tie this Cluster to the Environment it belongs to:

Field Value here What it binds
metadata.namespace dynamo-prod The Environment. Capsule owns that namespace, so the Cluster belongs to the dynamo tenant, and its hosted control plane runs there.
network.subnet primary The Environment's SubnetClaim, by name. The ClusterClass resolves it against the Cluster's namespace to the NetworkAttachmentDefinition the worker VMs attach through, and stamps the control plane with the matching Kube-OVN subnet.
controlPlane.network.serviceType LoadBalancer How the control plane is reached. The Service takes an address from the tenant-cp-pool MetalLB pool.

Nothing here names a VPC or a CIDR: the Cluster inherits both from the Environment's claims. That is the whole point of the split — the tenant declares their network once, and every Cluster in the Environment joins it by naming a subnet.

network.subnet must match the SubnetClaim name

The value is the name of the SubnetClaim in the Cluster's namespace — primary here, from Create First Environment. A name with no claim behind it leaves the worker VMs with no network to attach to.

The remaining variables describe the cluster itself rather than its place in the Environment: version is the Kubernetes version, machineSize the worker VM shape, dataStoreName the Kamaji datastore holding the control plane's etcd data, and workers.machineDeployments how many workers to boot. clusterNetwork is internal to the tenant cluster — its pods and Services — and is unrelated to the Environment's subnet.

Apply it:

kubectl apply -f solar.yaml

Step 2: Monitor Cluster Creation

Cluster creation typically takes a few minutes (control plane comes up first; workers boot from KubeVirt DataVolumes).

# CAPI Cluster + child resources
kubectl get cluster,kamajicontrolplane,tenantcontrolplane,kubevirtcluster -n dynamo-prod

# Workers
kubectl get machinedeployment,machines -n dynamo-prod

# Kamaji control plane pods (live in the tenant namespace)
kubectl get pods -n dynamo-prod -l kamaji.clastix.io/name=solar

Step 3: Access Your Cluster

Kamaji creates the admin kubeconfig as a Secret in the same namespace as the Cluster:

kubectl get secret solar-admin-kubeconfig -n dynamo-prod \
  -o jsonpath='{.data.admin\.conf}' | base64 -d > solar.kubeconfig

The Secret's kubeconfig points at a private address

The server address in that Secret is the control plane's internal address on the under cluster. A tenant cannot reach it. The address to use is the one the control plane's LoadBalancer Service publishes, and it has to be put into the kubeconfig before it will work from outside.

There are three ways to get a usable kubeconfig.

Use the kmetal-kubeconfig utility. It takes the Environment's namespace and the cluster name, and returns a kubeconfig already pointing at the LoadBalancer address:

kmetal-kubeconfig dynamo-prod solar > solar.kubeconfig

Download it from the console. The web console serves the same kubeconfig, with the address already substituted.

Do it by hand. Read the address off the control plane's Service and rewrite the server field:

$ kubectl get svc -n dynamo-prod

$ kubectl --kubeconfig solar.kubeconfig config set-cluster solar \
  --server=https://<loadbalancer-address>:<port>

Then use it:

$ export KUBECONFIG=solar.kubeconfig
$ kubectl get nodes
NAME                           STATUS   ROLES    AGE   VERSION
solar-md-0-zdd6v-xjvlw-dk48m   Ready    <none>   27h   v1.34.1
solar-md-0-zdd6v-xjvlw-jmhqt   Ready    <none>   27h   v1.34.1

Nodes that join but stay NotReady

That is a missing CNI, not a broken cluster: the nodes registered, and nothing has given them pod networking.

kMetal ships no CNI for tenant clusters — the platform delivers the one it chose through a Sveltos ClusterProfile, and until that profile matches this cluster its nodes never go Ready. Check with kubectl get clustersummaries -n dynamo-prod on the under cluster, and see Give tenant clusters a CNI.

Step 4: Validate Your Cluster

Deploy a test workload:

kubectl create deployment nginx --image=nginx:alpine
kubectl get pods
kubectl run test-pod --image=busybox --rm -it -- wget -qO- nginx

Next Steps

Your first cluster is ready. Continue with:

For platform administrators, see the Admin Guide for configuration and operations.