VMware Event Broker Appliance – Part XIII – Deploying Go Functions (OpenFaaS)

VEBA - VMware by pkremer

In Part XII, we talked about advanced function troubleshooting. In this post, we go through deploying a function written in a language I’ve never worked with before – Go!

The programming language Go is a critical building block for VEBA. The event router reads events from vCenter, converts them into a consistent format, and pushes them into the event processor. The event router is written in Go.

The sample functions in VEBA have mostly been PowerCLI and Python, we haven’t had many of them written in Go. However, Frankie Gold has started changing that with some Go commits. I just finished Frankie’s joint VMworld session with Dan Linsley – Golang for vCenter Admins – 10 Quick Steps [HCP1263] and I was able to deploy a Go function despite having never worked with Go.

The sample function is slick use case – VMs are tagged with a desired state hardware configuration, and that configuration gets applied to the VM when the VM gets powered off. In this screenshot, a VM gets tagged with 3 tags – “CPU:4, Memory:4096, and CPU Hot Add:true”. When the VM is powered off, the Go function automatically applies those changes.

Let’s follow the steps as shown in the VMworld session

Step 1

Install Go

This is a simple next-next install from https://golang.org/.

Step 2

Install git. I covered this topic in Part II.

Step 3 and 4… and 4.5

Install VEBA and the OpenFaas CLI. I covered VEBA in Part I and OpenFaaS CLI in Part II.

You also need Docker, which I cover in Part VII.

Step 5

Here is where I will diverge a bit from the VMworld video. It was recorded before the example function had been pushed into the main VEBA repo. But it’s in there now, so you don’t need to follow the examples of manually creating folders and dropping files in there. All you have to do is clone the repo. I change directories to the git folder on my computer and type:

git clone https://github.com/vmware-samples/vcenter-event-broker-appliance.git

Now you need to pull down the OpenFaas template. I change directories into the function’s folder /examples/go/vm-reconfig-via tag and type:

$ faas-cli template store pull golang-http
Fetch templates from repository: https://github.com/openfaas-incubator/golang-http-template at master
2020/10/01 11:10:10 Attempting to expand templates from https://github.com/openfaas-incubator/golang-http-template
2020/10/01 11:10:11 Fetched 4 template(s) : [golang-http golang-http-armhf golang-middleware golang-middleware-armhf] from https://github.com/openfaas-incubator/golang-http-template

Step 6

Modify stack.yml and vcconfig.toml to match your environment

vcconfig.toml – Change server to match my vCenter, enter username and password, and since my vCenter has a self-signed cert, I keep the insecure flag as true.

[vcenter]
server = "vc01.ad.patrickkremer.com"
user = "administrator@vsphere.local"
password = "*************"
insecure = true

stack.yml

I change the gateway to match my VEBA appliance and change my docker image from vmware to kremerpatrick, which is my Docker ID.

version: 1.0
provider:
  name: openfaas
  gateway: https://veba01
functions:
  vm-reconfig-via-tag-fn:
    lang: golang-http
    handler: ./handler
    image: kremerpatrick/veba-go-vm-reconfig-via-tag:latest
    environment:
      write_debug: true
    secrets:
      - vcconfig
    annotations:
      topic: VmPoweredOffEvent

Step 7

Set your environment variables. Per the VMworld session, your code must be in your GOPATH. And I have to add the path to the Go binaries to my PATH.

pkremer@pkremer-w02 MINGW64 ~/Documents/git/vcenter-event-broker-appliance/examples/go/vm-reconfig-via-tag (development)
$ export GOPATH=/c/Users/pkremer/Documents/git/vcenter-event-broker-appliance
pkremer@pkremer-w02 MINGW64 ~/Documents/git/vcenter-event-broker-appliance/examples/go/vm-reconfig-via-tag (development)
$ export PATH=$PATH:/c/go/bin

Deploy the function. This is very similar to how we deploy other functions elsewhere in this series, but the extra argument –build-arg=GO111MODULE=on is required.

pkremer@pkremer-w02 MINGW64 ~/Documents/git/vcenter-event-broker-appliance/examples/go/vm-reconfig-via-tag (development)
$ faas-cli up --build-arg=GO111MODULE=on --tls-no-verify

Unfortunately the code errors out

The command '/bin/sh -c test -z "$(gofmt -l $(find . -type f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run \"gofmt -s -w\" on your Golang code"; exit 1; }' returned a non-zero code: 1
[0] < Building vm-reconfig-via-tag-fn done in 1.38s.
[0] Worker done.

Total build time: 1.38s
Errors received during build:
- [vm-reconfig-via-tag-fn] received non-zero exit code from build, error: The command '/bin/sh -c test -z "$(gofmt -l $(find . -type 
f -name '*.go' -not -path "./vendor/*" -not -path "./function/vendor/*"))" || { echo "Run \"gofmt -s -w\" on your Golang code"; exit 
1; }' returned a non-zero code: 1

We need to format the Go code with go fmt. We cd into the /handler directory and run the go fmt command

pkremer@pkremer-w02 MINGW64 ~/Documents/git/vcenter-event-broker-appliance/examples/go/vm-reconfig-via-tag/handler (development)     
$ go fmt
go: downloading github.com/vmware/govmomi v0.23.1
go: downloading github.com/openfaas/templates-sdk v0.0.0-20200723110415-a699ec277c12
go: downloading github.com/pelletier/go-toml v1.8.0
handler.go
helpers.go
handler_test.go

cd up one level and try to build the function again.

pkremer@pkremer-w02 MINGW64 ~/Documents/git/vcenter-event-broker-appliance/examples/go/vm-reconfig-via-tag (development)
$ faas-cli up --build-arg=GO111MODULE=on --tls-no-verify

Note, if you get Docker errors it’s possible you’re not logged in, you can get this even if Docker Desktop is running. You can fix this with a simple ‘docker login’

pkremer@pkremer-w02 MINGW64 ~/Documents/git/vcenter-event-broker-appliance/examples/go/vm-reconfig-via-tag (development)
$ docker login
Authenticating with existing credentials…
Login Succeeded

This time the build succeeds and our function is deployed!

[0] < Pushing vm-reconfig-via-tag-fn [kremerpatrick/veba-go-vm-reconfig-via-tag:latest] done.
[0] Worker done.

Deploying: vm-reconfig-via-tag-fn.

Deployed. 202 Accepted.
URL: https://veba01/function/vm-reconfig-via-tag-fn

Step 8

Now I need to create tags and apply them to a VM. Remember, the idea behind this function is to apply a VM configuration at power-off based on how a VM is tagged. One option to create tags quickly is govmomi. I cover a bit of govmomi in Part III. Alternatively, you can create tags manually, which is what we will do in this post. Which tags are supported? The README file shows the following: 

- config.hardware.numCPU
- config.hardware.memoryMB
- config.hardware.numCoresPerSocket
- config.memoryHotAddEnabled
- config.cpuHotRemoveEnabled
- config.cpuHotAddEnabled

I head over to vCenter and go to Tags by clicking Menu>Tags & Custom Attributes. Then I click on Categories.

I’m going to change the number of CPUs and the memory configuration in this post, so I need to create a category for config.hardware.numCPU and config.hardware.memoryMB. I click New and create the config.hardware.numCPU category. I repeat the process for config.hardware.memoryMB. For this function you need to specify one tag per object, otherwise it will not understand which tag to apply.

Then I switch to Tags and click New to create a tag

I create a tag for 2 vCPUs. This can be repeated as many times as you want – I created a tag for 2, 3, and 4 vCPUs.

Then I create tags for 2GB, 3GB, and 4GB of memory.

Now I go to a VM – in my lab this is just a shell of a VM, there is no OS installed. You can see that the VM has 1 vCPU and 1 GB of RAM.

I right click the VM, go to Tags & Custom Attributes, then Assign Tag

I want 2GB of memory and 2vCPU.

You can see my tags are added.

Monitoring the function’s logfile is not necessary to run the function, but I’m going to do it for this post. I SSH into the VEBA appliance and run this to list the pods:

kubectl get pods -A
NAMESPACE        NAME                                      READY   STATUS      RESTARTS   AGE
kube-system      antrea-agent-ljrkw                        2/2     Running     6          2d20h
kube-system      antrea-controller-78d8dbd4fb-5cvkx        1/1     Running     4          2d20h
kube-system      coredns-6dcc67dcbc-wcdp6                  1/1     Running     3          2d20h
kube-system      coredns-6dcc67dcbc-x7sn8                  1/1     Running     3          2d20h
kube-system      etcd-veba01                               1/1     Running     2          2d20h
kube-system      kube-apiserver-veba01                     1/1     Running     2          2d20h
kube-system      kube-controller-manager-veba01            1/1     Running     2          2d20h
kube-system      kube-proxy-7m4ct                          1/1     Running     2          2d20h
kube-system      kube-scheduler-veba01                     1/1     Running     2          2d20h
openfaas-fn      vm-reconfig-via-tag-fn-5446576d4f-skvmt   1/1     Running     0          90m
openfaas         alertmanager-58f8d787d9-zccdc             1/1     Running     3          2d20h
openfaas         basic-auth-plugin-dd49cd66b-vhfsc         1/1     Running     6          2d20h
openfaas         faas-idler-59ff9778fd-nmzgb               1/1     Running     8          2d20h
openfaas         gateway-74f6f9489b-nhmqg                  2/2     Running     12         2d20h
openfaas         nats-6dfbf45d77-gfd2m                     1/1     Running     2          2d20h
openfaas         prometheus-5f5494b54f-vzdzp               1/1     Running     3          2d20h
openfaas         queue-worker-59b67bf4-jtcdr               1/1     Running     6          2d20h
projectcontour   contour-5cddfc8f6-7ptzw                   1/1     Running     6          2d20h
projectcontour   contour-5cddfc8f6-vs9hv                   1/1     Running     6          2d20h
projectcontour   contour-certgen-2lwch                     0/1     Completed   0          2d20h
projectcontour   envoy-ck8cj                               1/1     Running     2          2d20h
vmware           tinywww-7fcfc6fb94-tqgsz                  1/1     Running     2          2d20h
vmware           vmware-event-router-587977459d-mlx6b      1/1     Running     9          2d20h

I find the reconfig via tag pod and tail it.

root@veba01 [ ~ ]# kubectl logs -n openfaas-fn vm-reconfig-via-tag-fn-5446576d4f-skvmt --follow
Forking - ./handler []
2020/10/01 17:08:57 Started logging stderr from function.
2020/10/01 17:08:57 Started logging stdout from function.
2020/10/01 17:08:57 OperationalMode: http
2020/10/01 17:08:57 Timeouts: read: 10s, write: 10s hard: 10s.
2020/10/01 17:08:57 Listening on port: 8080
2020/10/01 17:08:57 Writing lock-file to: /tmp/.lock
2020/10/01 17:08:57 Metrics listening on port: 8081

Now I power on and power off my VM. Here’s a portion of the logfile output showing successful reconfiguration.


2020/10/01 18:59:20 stdout: Success (202): {"TaskID":"task-16855","Action":"Set numCPU to 2, memoryMB to 2048."}.

I check my VM and it now has 2 vCPU and 2GB of RAM.

Note that you cannot assign different tags for CPU and memory until you remove the existing tags. This is what I see when I try to assign tags to testvm1

This behavior is by design since we only allowed one tag per object when we created the category.

If I remove both the memory and CPU tag, I can assign a new set of tags, power the VM on and off, and my VM will be updated to 4GB of RAM and 4 vCPU.

2 comments

Leave a Reply

Your email address will not be published. Required fields are marked *