This is unreleased documentation for SUSE® Virtualization v1.6 (Dev).

Kube-OVN Operator

kubeovn-operator is used to manage the lifecycle of Kube-OVN as a secondary CNI on underlying SUSE Virtualization clusters.

Enabling `kubeovn-operator`

You must enable kubeovn-operator to deploy Kube-OVN to a SUSE Virtualization cluster for advanced SDN capabilities such as virtual private cloud (VPC) and subnets for virtual machine workloads.

On the SUSE Virtualization UI, go to Advanced → Add-ons.
Select kubeovn-operator (Experimental), and then select ⋮ → Enable.

The add-on deploys kubeovn-operator and creates the default Configuration object named configuration.kubeovn.io, which uses sane SUSE Virtualization-specific defaults for configuring the Kube-OVN CNI.

The following is an example of a Configuration object:

apiVersion: kubeovn.io/v1
kind: Configuration
metadata:
  name: kubeovn
  namespace: kube-system
spec:
  cniConf:
    cniBinDir: /opt/cni/bin
    cniConfFile: /kube-ovn/01-kube-ovn.conflist
    cniConfigDir: /etc/cni/net.d
    cniConfigPriority: "90"
    localBinDir: /usr/local/bin
  components:
    OVSDBConTimeout: 10
    OVSDBInactivityTimeout: 10
    checkGateway: true
    enableANP: false
    enableBindLocalIP: true
    enableExternalVPC: true
    enableIC: false
    enableKeepVMIP: true
    enableLB: true
    enableLBSVC: false
    enableLiveMigrationOptimize: true
    enableNATGateway:  true
    enableNP: true
    enableOVNIPSec: false
    enableTProxy: false
    hardwareOffload: false
    logicalGateway: false
    lsCtSkipOstLportIPS: true
    lsDnatModDlDst: true
    secureServing: false
    setVLANTxOff:  false
    u2oInterconnection:  false
  debug:
    mirrorInterface: mirror0
  dpdkCPU:  "0"
  dpdkMEMORY: "0"
  dpdkVersion: "19.11"
  dualStack:
    joinCIDR: fd00:100:64::/112
    pingerExternalAddress: 2606:4700:4700::1111
    pingerExternalDomain: google.com.
    podCIDR: fd00:10:16::/112
    podGateway: fd00:10:16::1
    serviceCIDR: fd00:10:96::/112
  global:
    images:
      kubeovn:
        dpdkRepository: kube-ovn-dpdk
        repository: kube-ovn
        supportArm: true
        thirdParty:  true
        vpcRepository: vpc-nat-gateway
    registry:
      address: docker.io/kubeovn
  hugePages: "0"
  hugepageSizeType:  hugepages-2Mi
  imagePullPolicy: IfNotPresent
  ipv4:
    joinCIDR: 100.64.0.0/16
    pingerExternalAddress: 1.1.1.1
    pingerExternalDomain: google.com.
    podCIDR: 10.54.0.0/16
    podGateway: 10.54.0.1
    serviceCIDR: 10.55.0.1
  ipv6:
    joinCIDR: fd00:100:64::/112
    pingerExternalAddress: 2606:4700:4700::1111
    pingerExternalDomain: google.com.
    podCIDR: fd00:10:16::/112
    podGateway: fd00:10:16::1
    serviceCIDR: fd00:10:96::/112
  kubeOvnCNI:
    requests:
      cpu: "100m"
      memory: "100Mi"
    limits:
      cpu: "1"
      memory:  "1Gi"
  kubeOvnController:
    requests:
      cpu: "200m"
      memory: "200Mi"
    limits:
      cpu: "1"
      memory: "1Gi"
  kubeOvnMonitor:
    requests:
      cpu: "200m"
      memory: "200Mi"
    limits:
      cpu: "200m"
      memory: "200Mi"
  kubeOvnPinger:
    requests:
      cpu: "100m"
      memory: "100Mi"
    limits:
      cpu: "200m"
      memory: "400Mi"
  kubeletConfig:
    kubeletDir: /var/lib/kubelet
  logConfig:
    logDir: /var/log
  masterNodesLabel: node-role.kubernetes.io/control-plane=true
  networking:
    defaultSubnet: ovn-default
    defaultVPC: ovn-cluster
    enableECMP: false
    enableEIPSNAT: true
    enableMetrics:  true
    enableSSL: false
    netStack: ipv4
    networkType: geneve
    nodeSubnet: join
    ovnLeaderProbeInterval: 5
    ovnNorthdNThreads: 1
    ovnNorthdProbeInterval: 5000
    ovnRemoteOpenflowInterval: 10
    ovnRemoteProbeInterval: 10000
    podNicType: veth-pair
    probeInterval:  180000
    tunnelType: vxlan
    nodeLocalDNSIPS: ""
    vlan:
      providerName: provider
      vlanId: 1
      vlanName: ovn-vlan
  openVSwitchDir: /var/lib/rancher/origin/openvswitch
  ovnCentral:
    requests:
      cpu: 300m
      memory: 200Mi
    limits:
      cpu: 3
      memory: 4Gi
  ovnDir: /etc/origin/ovn
  ovsOVN:
    limits:
      cpu:  2
      memory: 1000Mi
    requests:
      cpu: 200m
      memory: 200Mi
  performance:
    gcInterval: 360
    inspectInterval: 20
    ovsVSCtlConcurrency: 100

This Configuration object has been validated against SUSE Virtualization installations. You can fine-tune the object by editing the kubeovn-operator add-on YAML, but exercise caution when making changes to the configurationSpec field.

Ensure that the Kube-OVN IPv4 pod and service CIDR blocks do not overlap with the Harvester pod and service CIDR blocks.

Disabling `kubeovn-operator`

Ensure that no virtual machines are using VM networks backed by Kube-OVN SDN components.

Disabling the kubeovn-operator add-on is a disruptive process.

You can disable kubeovn-operator using the following commands:

kubectl delete configuration kubeovn -n kube-system --wait=false

kubectl delete validatingwebhookconfiguration kube-ovn-webhook --ignore-not-found

kubectl delete ips --all

kubectl delete subnets join ovn-default --ignore-not-found

kubectl delete vpc ovn-cluster --ignore-not-found

# Remove annotations/labels in namespaces and nodes
kubectl annotate node --all ovn.kubernetes.io/cidr-
kubectl annotate node --all ovn.kubernetes.io/gateway-
kubectl annotate node --all ovn.kubernetes.io/ip_address-
kubectl annotate node --all ovn.kubernetes.io/logical_switch-
kubectl annotate node --all ovn.kubernetes.io/mac_address-
kubectl annotate node --all ovn.kubernetes.io/port_name-
kubectl annotate node --all ovn.kubernetes.io/allocated-
kubectl annotate node --all ovn.kubernetes.io/chassis-
kubectl label node --all kube-ovn/role-
kubectl annotate ns --all ovn.kubernetes.io/cidr-
kubectl annotate ns --all ovn.kubernetes.io/exclude_ips-
kubectl annotate ns --all ovn.kubernetes.io/gateway-
kubectl annotate ns --all ovn.kubernetes.io/logical_switch-
kubectl annotate ns --all ovn.kubernetes.io/private-
kubectl annotate ns --all ovn.kubernetes.io/allow-
kubectl annotate ns --all ovn.kubernetes.io/allocated-

# Remove annotations in all pods of all namespaces
for ns in $(kubectl get ns -o name | awk -F/ '{print $2}'); do
  echo "annotating pods in namespace $ns"
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/cidr-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/gateway-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/ip_address-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/logical_switch-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/mac_address-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/port_name-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/allocated-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/routed-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/vlan_id-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/network_type-
  kubectl annotate pod --all -n $ns ovn.kubernetes.io/provider_network-
done

You must reboot each node to complete the uninstallation process. Once the nodes are rebooted, you can disable the kubeovn-operator add-on from the Harvester UI.

Kube-OVN Operator

Enabling kubeovn-operator

Disabling kubeovn-operator

Enabling `kubeovn-operator`

Disabling `kubeovn-operator`