Access Private Kubernetes Clusters with NetBird Kubernetes Operator

Accessing private Kubernetes clusters can be challenging, especially when connecting from remote locations or having multiple clusters to manage. NetBird Kubernetes operator simplifies this process by enabling secure access to your Kubernetes clusters using custom resource configurations and annotations to expose your cluster and services in your NetBird network.

The NetBird Kubernetes operator automatically creates Networks and Resources in your NetBird account, allowing you to seamlessly access your Kubernetes services and control plane from your NetBird network.

Deployment

Prerequisites

• (Recommended) helm version 3+
• kubectl version v1.11.3+.
• Access to a Kubernetes v1.11.3+ cluster.
• (Recommended) Cert Manager.

Installation

You have two methods of installing the NetBird Kubernetes operator: using Helm or the install.yaml file.

Using Helm

1. Add helm repository.

helm repo add netbirdio https://netbirdio.github.io/kubernetes-operator

2. (Recommended) Install cert-manager for k8s API to communicate with the NetBird operator.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.17.0/cert-manager.yaml

3. Add NetBird API token. You can create a PAT by following the steps here.

kubectl create namespace netbird
kubectl -n netbird create secret generic netbird-mgmt-api-key --from-literal=NB_API_KEY=$(cat ~/nb-pat.secret)

4. (Recommended) Create a values.yaml file, check helm show values netbirdio/kubernetes-operator for more info.

ingress:
  enabled: true
  router:
    enabled: true

netbirdAPI:
  keyFromSecret: "netbird-mgmt-api-key"

# by default the managementURL points to the NetBird cloud service: https://api.netbird.io:443
# managementURL: "https://netbird.example.io:443"

5. Install using helm install:

helm install --create-namespace -f values.yaml -n netbird netbird-operator netbirdio/kubernetes-operator

6. Check installation

kubectl -n netbird get pods

Output should be similar to:

NAME                                                    READY   STATUS    RESTARTS   AGE
netbird-operator-kubernetes-operator-67769f77db-tmnfn   1/1     Running   0          42m

kubectl -n netbird get services

Output should be similar to:

NAME                                                   TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)    AGE
netbird-operator-kubernetes-operator-metrics           ClusterIP   192.168.194.165   <none>        8080/TCP   47m
netbird-operator-kubernetes-operator-webhook-service   ClusterIP   192.168.194.222   <none>        443/TCP    47m

Updating or Modifying the Operator Configuration

The configuration or version update of the operator can be done with helm upgrade:

helm upgrade --create-namespace -f values.yaml -n netbird netbird-operator ./helm/kubernetes-operator

Using install.yaml

kubectl create namespace netbird
kubectl apply -n netbird -f https://raw.githubusercontent.com/netbirdio/kubernetes-operator/refs/heads/main/manifests/install.yaml

Expose Kubernetes Control Plane to your NetBird Network

To access your Kubernetes control plane from a NetBird network, you can expose your Kubernetes control plane as a NetBird resource by enabling the following option in the operator values:

ingres:
  enabled: true
  kubernetesAPI:
    enabled: true

The operator will create a NetBird network resource similar to the example below:

Expose Kubernetes Services to NetBird Network

Kubernetes services is a common way to route traffic to your application pods. With the NetBird operator ingress you can expose services to your NetBird network as resources by using annotations in your services. The operator will create networks, resources, and add routing peers to your NetBird configuration.

By default, the ingress configuration is disabled. You can enable it with the following values using the helm chart:

ingress:
  enabled: true
  router:
    enabled: true

You can expose services using the annotations netbird.io/expose: "true" and netbird.io/groups: "resource-group"; see the example below:

apiVersion: v1
kind: Service
metadata:
  name: app
  annotations:
    netbird.io/expose: "true"
    netbird.io/groups: "app-access"
spec:
  selector:
    app: app
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
  type: ClusterIP

This will create a Network and a resource similar to the example below:

Other annotations can be used to further configure the resources created by the operator:

Control Access to Your Kubernetes Resources with Access Control Policies

By default, resources created by the operator will not have any access control policies assigned to them. To allow the operator to manage your access control policies, configure policy bases in your values.yaml file. In this file, you can define source groups, name suffixes, and other settings related to access control policies. Afterward, you can tag the policies in your service annotations using the annotation netbird.io/policy: "policy-base". See the examples values.yaml below:

ingress:
  enabled: true
  router:
    enabled: true
  policies:
    app-users:
      name: App users # Required, name of policy in NetBird console
      description: Policy for app users access # Optional
      sourceGroups: # Required, name of groups to assign as source in Policy.
        - app-users
      protocols: # Optional, restricts protocols allowed to resources, defaults to ['tcp', 'udp'].
        - tcp
      bidirectional: false
    k8s-admins:
      name: App admins
      sourceGroups:
        - app-admins

After adding the policy base and applying the configuration, you can use the app-users and k8s-admins bases for your services and Kubernetes API configurations.

Linking Policy Bases to the Kubernetes API Service

To link a policy base to the Kubernetes API, we need to update the operator configuration by adding the policy and groups to the kubernetesAPI key as follows:

  kubernetesAPI:
    enabled: false
    groups:
      - k8s-access
    policies:
      - k8s-admins

After updating and applying the configuration, you should see a policy similar to the one below:

Linking Policy Bases to Kubernetes Services

You can link policy bases with the annotation netbird.io/policy: where you can simply add one or more bases to the service, see the example below where we link the base "app-users" to our app service:

apiVersion: v1
kind: Service
metadata:
  name: app
  annotations:
    netbird.io/expose: "true"
    netbird.io/groups: "app-access"
    netbird.io/policy: "app-users"
spec:
  selector:
    app: app
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
  type: ClusterIP

The operator will create a policy in your management account similar to the one below:

You can reference multiple policy bases using a comma separated list of policy bases: netbird.io/policy: "app-users,app-admins"

Accessing Remote Services Using Sidecars

To access services running in different locations from your Kubernetes clusters, you can deploy NetBird sidecars—additional containers that run alongside your Kubernetes service containers within the same pod.

A NetBird sidecar joins your network as a regular peer and becomes a subject to access control, routing, and DNS configurations as any other peer in your NetBird network. This allows your Kubernetes application traffic to be securely routed through the NetBird network, enabling egress-like access to remote services from your Kubernetes services across various locations or cloud providers.

To enable sidecar functionality in your deployments, you first need to generate a setup key, either via the UI (see image below) or by following this guide for more details.

Next, you'll create a secret in Kubernetes and add a new resource called NBSetupKey. The NBSetupKey name can then be referenced in your deployments or daemon sets to specify which setup key should be used when injecting a sidecar into your application pods. Below is an example of a secret and an NBSetupKey resource:

apiVersion: v1
stringData:
  setupkey: EEEEEEEE-EEEE-EEEE-EEEE-EEEEEEEEEEEE
kind: Secret
metadata:
  name: app-setup-key

NBSetupKey:

apiVersion: netbird.io/v1
kind: NBSetupKey
metadata:
  name: app-setup-key
spec:
  # Optional, overrides management URL for this setupkey only
  # defaults to https://api.netbird.io
  # managementURL: https://netbird.example.com
  secretKeyRef:
    name: app-setup-key # Required
    key: setupkey # Required

After adding the resource, you can reference the NBSetupKey in your deployments or daemon-sets as shown below:

kind: Deployment
...
spec:
...
  template:
    metadata:
      annotations:
        netbird.io/setup-key: app-setup-key # Must match the name of an NBSetupKey object in the same namespace
...
    spec:
      containers:
...

Using Extra Labels to Access Multiple Pods Using the Same Name

Starting with v0.27.0, NetBird supports extra DNS labels, allowing you to define extended DNS names for peers. This enables grouping peers under a shared DNS name and distributing traffic using DNS round-robin load balancing.

To use this feature, create a setup key with the “Allow Extra DNS Labels” option enabled. See the example below for reference:

And add the annotation netbird.io/extra-dns-labels to your pod; see the example below:

kind: Deployment
...
spec:
...
  template:
    metadata:
      annotations:
        netbird.io/setup-key: app-setup-key # Must match the name of an NBSetupKey object in the same namespace
        netbird.io/extra-dns-labels: "app"
...
    spec:
      containers:
...

With this setup, other peers in your NetBird network can reach these pods using the domain app.<NETBIRD_DOMAIN> (e.g., for NetBird cloud, app.netbird.cloud). The access will be made using a DNS round-robin fashion for multiple pods.

Get started

Use NetBird

• Make sure to star us on GitHub
• Follow us on Twitter
• Join our Slack Channel
• NetBird latest release on GitHub