Nodegroups on a shared network

A nodegroup can be attached to a network in another of your OpenStack projects. Your virtual machines in that project and your cluster can then talk privately, without a floating IP per machine. This is available on clusters running Kubernetes 1.35 or later.

In this guide, the VM project is the OpenStack project that owns the network and your virtual machines, and the cluster is your Kubernetes cluster, which runs in its own project.

How it works

• Each node in the nodegroup gets a second address, on a subnet of the shared network, next to its address on the cluster network.
• Only that nodegroup is on the shared network. Other nodegroups are not affected and cannot reach it.
• A nodegroup can use one shared network, and reaches only one subnet on it. If you need several subnets, use one nodegroup per subnet.
• The network stays in the VM project. You keep full control of it, and of the security groups of your virtual machines.

Prepare the network

In the VM project, create the network and subnet, or pick an existing one. The subnet must:

• Not overlap the cluster’s node, pod or service ranges. If it does, we cannot attach it. A new cluster can be created with a Service range of your choice to avoid an overlap.
• Have room for the nodegroup: one address per node at the nodegroup’s maximum size, plus one address per availability zone the nodegroup uses. Nodes that cannot get an address never join the cluster.
• Have the same MTU as the cluster network. With a smaller MTU, pods on those nodes get a smaller MTU too, and large UDP packets to them are dropped.

Ask us to attach it

Contact support and tell us:

• the VM project, and the network (and subnet, if the network has more than one);
• the cluster and the nodegroup that should use it.

We share the network with the cluster’s project and attach it to the nodegroup. You do not need to share it yourself. The nodegroup’s nodes are replaced once, and no other nodes are replaced.

Let your virtual machines accept traffic from the cluster

Traffic from pods to your virtual machines comes from the nodes’ addresses on the shared subnet, not from the pods’ own addresses. Allow the shared subnet’s range in the security groups of the virtual machines that the cluster should reach.

Reach your Services from the virtual machines

Your virtual machines cannot reach pods directly. Expose a Service through an internal load balancer with its address on the shared subnet:

apiVersion: v1
kind: Service
metadata:
  name: my-service
  annotations:
    service.beta.kubernetes.io/openstack-internal-load-balancer: "true"
    loadbalancer.openstack.org/subnet-id: <shared subnet ID>
    loadbalancer.openstack.org/member-subnet-id: <the cluster's own subnet ID>
spec:
  type: LoadBalancer
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080

All three annotations are needed, and they must be on the Service when you create it. They are read only when the load balancer is created, so adding them later means deleting and recreating the Service.

• Without openstack-internal-load-balancer, the load balancer gets a public IP and is reachable from the internet.
• Without member-subnet-id, the load balancer accepts connections but cannot reach any node, so every request fails.

Ask support for the cluster’s own subnet ID if you do not have it.

If you use our Envoy Gateway addon, put the same three annotations on your Gateway under spec.infrastructure.annotations. A Gateway with them gets an internal load balancer on the shared subnet, and your other Gateways stay public. Your ClientTrafficPolicy stays the same.

Remove it

1. Delete the Services, and Gateways, that have a load balancer on the shared subnet. While one exists, the network cannot be unshared or deleted.
2. Contact support to detach the network from the nodegroup. Its nodes are replaced once more, without the second address.
3. We stop sharing the network. You can then delete it in the VM project.

Deleting the cluster also deletes its load balancers on the shared subnet.