VMware Event Broker Appliance – Part V – Working with Multiple Kubernetes Clusters (Knative)

In Part II, I demonstrated how to copy the .kube/config file from the VEBA appliance to your local workstation for remote access. In this post, we will demonstrate one way to work with multiple Kubernetes clusters. Maybe you have multiple VEBA installations and want to be able to switch between them. Or maybe you’re working with other Kubernetes clusters besides VEBA. In my case, it’s both – I have multiple VEBA appliances, and I am also doing other modern apps learning. I need to easily switch between clusters.

The kubectl command supports an environment variable called KUBECONFIG – point that variable to a Kubernetes config file and kubectl will interact with the specified cluster.

I create a folder in my OneDrive to store my K8s configuration files. I used WinSCP to copy the .kube/config file from my VEBA host, VEBA03, to OneDrive. Look at Part II if you need to see a walkthrough on this process. I rename the config file to veba03-config

Now I want to create a function that will set my KUBECONFIG environment variable, and a command alias to easily call the function

Function SetKC-VEBA03 {
    $env:KUBECONFIG="$HOME\OneDrive - VMware, Inc\VMware\.k8s.configs\veba03-config"
    Write-Host $env:KUBECONFIG
}
Set-Alias -Name kc-veba03 -Value SetKC-VEBA03

Now I can call the function by running kc-veba03

C:\git> kc-veba03   
 C:\Users\pkremer\OneDrive - VMware, Inc\VMware.k8s.configs\veba03-config

Let’s see if we can list the VEBA pods – success!

C:\git> kubectl get pods -A
 NAMESPACE            NAME                                          READY   STATUS      RESTARTS   AGE
 contour-external     contour-5869594b-jtb7p                        1/1     Running     0          17h
 contour-external     contour-5869594b-ls9vr                        1/1     Running     0          17h
 contour-external     contour-certgen-v1.10.0-k5hkm                 0/1     Completed   0          17h
(output truncated)

I also add a reset command – this clears my environment variable. I like to do this as an additional layer of security – it’s easy to accidentally perform an operation on the wrong cluster. I’ve gotten myself into the habit of running a reset command every time I’m done working with a cluster.

Function ResetKubeconfig {Remove-Item env:KUBECONFIG}
Set-Alias -Name "kr" -Value ResetKubeConfig

Now if I try to run a kubectl command, I get an error because in the absence of any configuration, kubectl will try to connect to localhost – I do not have Kubernetes running on my laptop, hence the error.

C:\git> kr
 C:\git> kubectl get pods -A
 Unable to connect to the server: dial tcp 127.0.0.1:8080: connectex: No connection could be made because the target machine actively refused it.

Now I need these aliases to run every time I open PowerShell. Do do this, I need to locate and edit my PowerShell profile.

C:\git> echo $profile
 C:\Users\pkremer\Documents\PowerShell\Microsoft.PowerShell_profile.ps1

I edit my profile and add all of the functions to switch between contexts. Now any time I launch a PowerShell instance, my aliases will be there.

Function ResetKubeconfig {Remove-Item env:KUBECONFIG}
Set-Alias -Name "kr" -Value ResetKubeConfig

# MAPBU Demo Lab
Function SetKC-mapd-Node1 {
    $env:KUBECONFIG="$HOME\OneDrive - VMware, Inc\VMware\.k8s.configs\mapd-primary.yml"
    Write-Host $env:KUBECONFIG
}
Set-Alias -Name kc1 -Value SetKC-mapd-Node1

Function SetKC-mapd-Node2 {
    $env:KUBECONFIG="$HOME\OneDrive - VMware, Inc\VMware\.k8s.configs\mapd-secondary.yml"
    Write-Host $env:KUBECONFIG
}
Set-Alias -Name kc2 -Value SetKC-mapd-Node2

# VEBA03 - Home Lab
Function SetKC-VEBA03 {
    $env:KUBECONFIG="$HOME\OneDrive - VMware, Inc\VMware\.k8s.configs\veba03-config"
    Write-Host $env:KUBECONFIG
}
Set-Alias -Name kc-veba03 -Value SetKC-VEBA03

# VEBA04 - Home Lab
Function SetKC-VEBA04 {
    $env:KUBECONFIG="$HOME\OneDrive - VMware, Inc\VMware\.k8s.configs\veba04-config"
    Write-Host $env:KUBECONFIG
}
Set-Alias -Name kc-veba04 -Value SetKC-VEBA04

# List all Kubernetes Control Aliases
Function ShowKC {get-alias | Where { $_.Name -Like "kc*"} } 
Set-Alias -name kc -Value ShowKC

Set-Alias -Name k -Value kubectl

I can now easily switch between clusters

C:\git> kc-veba03
 C:\Users\pkremer\OneDrive - VMware, Inc\VMware.k8s.configs\veba03-config
C:\git> k get nodes
NAME     STATUS   ROLES                  AGE   VERSION
veba03   Ready    control-plane,master   17h   v1.20.2
C:\git> kc-veba04
 C:\Users\pkremer\OneDrive - VMware, Inc\VMware.k8s.configs\veba04-config
C:\git> k get nodes
NAME     STATUS   ROLES                  AGE   VERSION
veba04   Ready    control-plane,master   30m   v1.20.2
C:\git> kr
C:\git> k get nodes
Unable to connect to the server: dial tcp 127.0.0.1:8080: connectex: No connection could be made because the target machine actively refused it.