This is unreleased documentation for SUSE® Virtual Clusters v1.1.0 (Dev).

Resource Sync

In shared mode, K3k virtual clusters run their workloads directly on the host cluster while maintaining logical isolation. Since your pods execute on the host, they need access to essential resources like Services, ConfigMaps, Secrets, and PersistentVolumeClaims. K3k automatically syncs these resources from your virtual cluster to the host, enabling your workloads to run seamlessly.

This guide explains what resources are synced, why, and how to control sync behavior for your use case.

Quick Start

By default, K3k syncs the essential resources needed for your workloads to run:

  • Enabled: Services, ConfigMaps, Secrets, PersistentVolumeClaims

  • Disabled: Ingresses, PriorityClasses, StorageClasses

The default enabled syncs (Services, ConfigMaps, Secrets, PVCs) are required for most workloads to function correctly. Disabling these should only be done if you know what you’re doing and understand the impact on your applications.

For most use cases, the defaults work out of the box. You only need to configure sync when you want external access (Ingresses), custom scheduling priorities (PriorityClasses), or shared storage infrastructure (StorageClasses).

Understanding Resource Sync

K3k syncs resources in two directions depending on their purpose:

┌────────────────┐    Workload Resources     ┌─────────────────┐
│                │ <──────────────────────── │                 │
│                │   Services, ConfigMaps,   │                 │
│  Host Cluster  │   Secrets, PVCs, etc.     │ Virtual Cluster │
│                │                           │                 │
│                │ ────────────────────────> │                 │
│                │  Infrastructure Resources │                 │
└────────────────┘      StorageClasses       └─────────────────┘

Workload Resources (Virtual → Host):

Your applications need services, configuration, and secrets to run. These flow from your virtual cluster to the host where your pods actually execute.

Infrastructure Resources (Host → Virtual):

Storage classes represent available storage infrastructure. These flow from the host into your virtual cluster so your applications can request the right type of storage.

Resource Types

Services

Default: Enabled | Direction: Virtual → Host

Services provide network endpoints to access your applications. When you create a service in your virtual cluster, K3k automatically makes it available on the host cluster so traffic can reach your pods.

When to use:

  • Always leave enabled for basic networking

  • Required for ClusterIP, LoadBalancer, and NodePort services

  • Disabling this will break networking for your workloads

  • only disable if you know what you’re doing

ConfigMaps

Default: Enabled | Direction: Virtual → Host

ConfigMaps store configuration data for your applications. Since your pods run on the host cluster, their ConfigMaps must be available there.

When to use:

  • Leave enabled if your applications use ConfigMaps

  • Disable only if your applications don’t use any ConfigMaps

  • Use label selectors to sync only specific configuration

Example - Sync only app-specific ConfigMaps:

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: my-cluster
spec:
  sync:
    configMaps:
      enabled: true
      selector:
        app: my-application

Secrets

Default: Enabled | Direction: Virtual → Host

Secrets store sensitive data like passwords and certificates. Your pods need access to these secrets on the host cluster.

Security:

Synced secrets remain isolated within your cluster’s namespace boundary and aren’t shared with other virtual clusters.

When to use:

  • Leave enabled if your applications use Secrets

  • Disable only if your applications don’t use any Secrets

  • Use label selectors for granular control over which secrets are synced

Example - Sync only explicitly labeled secrets:

spec:
  sync:
    secrets:
      enabled: true
      selector:
        sync-enabled: "true"

PersistentVolumeClaims

Default: Enabled | Direction: Virtual → Host

PersistentVolumeClaims (PVCs) request storage for your applications. K3k syncs these to the host where the actual storage provisioning happens.

When to use:

  • Leave enabled if your applications need persistent storage

  • Disable only if your applications don’t use any PersistentVolumeClaims

Example - Disable if not needed:

spec:
  sync:
    persistentVolumeClaims:
      enabled: false

Ingresses

Default: Disabled | Direction: Virtual → Host

Ingresses define HTTP/HTTPS routing rules for external access to your applications.

Why disabled by default:

Not all applications need external access. Enable Ingresses when you need to expose services outside the cluster.

TLS Secret Translation:

When you create an Ingress in your virtual cluster, any TLS secrets it references are synced from the virtual cluster to the host with translated names (just like other resources). This is the default behavior.

If you want to use a shared TLS secret that already exists on the host cluster (for example, a wildcard certificate used across multiple virtual clusters), you can disable TLS secret translation with disableTLSSecretTranslation: true. With this option, the Ingress will reference the secret by its original name on the host, allowing you to use centrally-managed certificates.

When to use:

  • Production applications that need external HTTP/HTTPS access

  • Custom domain routing

  • SSL/TLS termination

Example - Default behavior (secrets synced from virtual cluster):

spec:
  sync:
    ingresses:
      enabled: true
      disableTLSSecretTranslation: false  # Default: sync TLS secrets from virtual cluster

Example - Use shared host TLS secret:

# Host cluster has a wildcard certificate
apiVersion: v1
kind: Secret
metadata:
  name: wildcard-tls-cert
  namespace: default
type: kubernetes.io/tls
data:
  tls.crt: <cert-data>
  tls.key: <key-data>

---
# Virtual cluster Ingress references the shared secret by name
apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: my-cluster
  namespace: default
spec:
  sync:
    ingresses:
      enabled: true
      disableTLSSecretTranslation: true  # Use host secret name directly

PriorityClasses

Default: Disabled | Direction: Virtual → Host

PriorityClasses define pod scheduling priorities. Higher priority pods can preempt lower priority ones when resources are constrained.

Why disabled by default:

Priority settings affect cluster-wide scheduling and could impact other workloads. Enable only when you need guaranteed scheduling for critical applications.

Security consideration: In multi-tenant environments, carefully evaluate whether virtual clusters should define their own priorities.

When to use:

  • Critical production workloads that must not be evicted

  • Multi-tier applications with priority ordering

Example:

spec:
  sync:
    priorityClasses:
      enabled: true

StorageClasses

Default: Disabled | Direction: Host → Virtual

StorageClasses define available storage types (SSD, HDD, cloud storage, etc.). Unlike other resources, StorageClasses sync FROM the host TO your virtual cluster.

Why reverse direction:

StorageClasses represent storage infrastructure provided by the host. Your virtual cluster consumes these storage options rather than creating them.

Important:

When StorageClass sync is enabled, ALL StorageClasses on the host are synced to your virtual cluster by default. Use the k3k.io/sync-enabled label on the host to prevent specific StorageClasses from being synced.

Two-Level Filtering:

  1. Host-Level Label (Optional):

Add k3k.io/sync-enabled label to StorageClasses on the host to control sync * No label: StorageClass WILL be synced * Label = "true": StorageClass WILL be synced
* Label = "false": StorageClass will NOT be synced (use this to block unwanted StorageClasses).

Cluster-Level Selector (Optional):

Further filter which StorageClasses appear in your virtual cluster based on labels

When to use:

  • Enable when your applications need to choose specific storage types

  • Multi-tier storage (fast SSD for databases, cheap HDD for backups)

  • Cloud provider-specific storage classes

Example - Sync all StorageClasses:

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: my-cluster
spec:
  sync:
    storageClasses:
      enabled: true  # All host StorageClasses will be synced

Example - Block a StorageClass from syncing:

# On the Host Cluster - prevent this from syncing to virtual clusters
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: admin-only-storage
  labels:
    k3k.io/sync-enabled: "false"  # This StorageClass will NOT be synced
provisioner: custom.csi.driver

Example - Sync only specific StorageClasses using selector:

# On the Host Cluster
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
  labels:
    tier: premium
provisioner: ebs.csi.aws.com

---
# In Your Virtual Cluster Configuration - only sync premium tier
apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: my-cluster
spec:
  sync:
    storageClasses:
      enabled: true
      selector:
        tier: premium  # Only sync StorageClasses with tier=premium label

Configuration Options

Cluster-Level Configuration

Configure sync for individual virtual clusters:

apiVersion: k3k.io/v1beta1
kind: Cluster
metadata:
  name: my-cluster
  namespace: default
spec:
  sync:
    services:
      enabled: true
    configMaps:
      enabled: true
    secrets:
      enabled: true
    persistentVolumeClaims:
      enabled: true
    ingresses:
      enabled: false  # Disabled by default
    priorityClasses:
      enabled: false  # Disabled by default
    storageClasses:
      enabled: false  # Disabled by default

Policy-Level Configuration

Use VirtualClusterPolicy to apply the same sync settings to all clusters in a namespace. This is ideal for multi-tenant environments where you want consistent behavior.

Policy overrides cluster configuration:

When a policy is applied, its sync settings take precedence over individual cluster settings.

apiVersion: k3k.io/v1beta1
kind: VirtualClusterPolicy
metadata:
  name: production-policy
  namespace: prod-clusters
spec:
  sync:
    services:
      enabled: true
    ingresses:
      enabled: true  # Enable external access for production
    storageClasses:
      enabled: true
      selector:
        tier: premium  # Only premium storage in production

All clusters in the prod-clusters namespace inherit these settings.

Using Label Selectors

Label selectors let you sync only specific resources based on labels. An empty selector syncs all resources of that type.

Example - Environment-specific ConfigMaps:

spec:
  sync:
    configMaps:
      enabled: true
      selector:
        environment: production

Example - CSI-specific StorageClasses:

spec:
  sync:
    storageClasses:
      enabled: true
      selector:
        csi-driver: aws-ebs