Architecture

How the controller, control planes, isolation, and sync engine fit together.

tenantplane separates the user-facing tenant API from the host-cluster implementation details. Everything is driven by three custom resources reconciled by a single controller running on the host cluster.

tenantplane architecture

Components

The controller

A controller-runtime manager watches TenantCluster resources (and re-reconciles when a referenced IsolationProfile or SyncPolicy changes). For each TenantCluster it drives the whole lifecycle: isolation, control plane, kubeconfig extraction, and sync.

Tenant control plane

Each tenant gets a small k3s control plane running as a single-replica StatefulSet in a host namespace, fronted by a headless Service. The controller runs k3s with the agent and bundled add-ons disabled so the pod is purely an API server + datastore. The datastore is SQLite for the current milestone. The control-plane volume honors spec.controlPlane.storage (StorageClass + size), so the same spec works against the EBS, Azure Disk, and Persistent Disk CSI drivers; spec.controlPlane.expose.loadBalancer optionally publishes the tenant API through a cloud load balancer, with the provisioned address reported as status.externalEndpoint.

Anatomy of a tenant control plane: StatefulSet, k3s pod, headless Service, and kubeconfig Secret

Isolation

The referenced IsolationProfile is compiled into concrete host objects — a default-deny NetworkPolicy, a ResourceQuota, a LimitRange, and Pod Security Admission labels on the namespace. tenantplane’s own control-plane pods carry an exemption label so the default-deny policy never cuts them off.

Sync engine

Once the control plane is Ready, the controller connects to it with the extracted kubeconfig and runs the sync engine. For each resource kind the SyncPolicy marks toHost, the engine lists tenant objects, maps each to a deterministic host object, applies it, and garbage-collects host objects whose tenant source is gone.

The sync convergence pass

Once the control plane is Ready, each toHost resource kind runs the same four deterministic steps, recording a decision at every step:

tenantplane sync convergence pass

Determinism and reverse mapping

Every host object tenantplane creates is named from <resource>-x-<virtual namespace>-x-<tenant> (hashed when that would exceed a DNS label) and carries reverse-mapping labels and annotations. That means:

  • the same tenant object always maps to the same host object, and
  • any host object can be traced back to the tenant object that caused it.

This is what makes explain-sync able to predict placement before anything is applied.

Isolation modes

tenantplane isolation modes: shared, dedicated, and private, with migration paths between them
ModeWorkloads run onStatus
sharedHost nodes with software isolation controlsImplemented
dedicatedA selected node pool, shared infra servicesPlanned
privateSeparate worker nodes, CNI, and CSIPlanned

The design goal is migration from shared → dedicated → private without recreating the tenant API state.


Found a gap? Open an issue or PR.