Posted on

Troubleshooting vSphere 7 Installation – Home Lab Edition

My homelab boots off of cheap USB sticks and has done so since 2016.

I easily upgraded my vCenter (VCSA for the win), but could not get Update Manager to upgrade my hosts, it kept throwing an ‘unable to run upgrade script’ error. I tried command-line upgrade without success, then I tried dropping the ISO image on the USB stick with Rufus, booting off the stick and having the installer overwrite the boot image with the installation. I’ve done this same process many times over the past 4 years without issue. However, it kept failing this time. The installation would progress to various percentages, but never beyond 70%

I’ve been working with VMware products since the 3.0 days and I never knew this, thank you to William Lam for his troubleshooting suggestion in VMware’s slack channels. When the progress bar of the installation screen is up, you can press Alt+F1 and get a command prompt. Then you can log in with root and no password. From there you can navigate the filesystem and tail logs, including the install log and vmkernel log. I saw lots of storage errors in the vmkernel log, timeouts and retries.

I then yanked one of the SSD drives in my Synology out and popped it into the host. vSphere 7 installed without issue. Looks like I either need to buy some high-end USB drives, or maybe use this problem to justify swapping out some of the smaller drives in the Synology with larger ones. Then I can rotate the smaller ones into service as boot drives in the hosts.

Posted on

Unable to connect virtual NIC in vCloud Air DRaaS

I had a customer open a service request, they were in the middle of a DR test using vCloud Air DRaaS and were unable to connect 1 virtual machine to the network. It kept erroring out with a generic unable to connect error.

It turns out that their VM had a VMDK sized with a decimal point, like 50.21GB instead of just 50GB. I don’t see it often, but this sometimes happens when P2V a machine. The vCloud Director backend can’t handle the decimal point in the disk size, so it errors out.

I’m not entirely sure why the error happens, but the fix is to resize your source disk to a non-decimal number and run replication again.

Posted on

Android 4.1.2 – Unable to use GPS

I just got the Android 4.1.2 push to my Verizon Droid Bionic and suddenly my map application stopped working. When I tapped the locate icon, Android threw an error “Please enable Google Apps location access”. My first thought was in Location Access under Settings, but that didn’t work. A Google search turned up this dotTech post directing me to Settings > Accounts > Google > Location Settings, but I couldn’t find any way to grant access to my Google account. In the end, I figured out I had to upgrade my account to Google+. Hopefully this post saves you a few hours of tearing your hair out.

Posted on

VMware View Composer starts, but does no work.

I worked on a client outage over the weekend, Virutal Center and View Composer were down. It started with a disk full situation on the SQL server hosting the vCenter, Composer, and Events databases. The client was shut down for winter break, so the Composer outage was not noticed for several days. After fixing the SQL Server disk space problem, everything came back up. I was able to restart all services and they appeared to be running. Composer started without issue, but it didn’t respond to any commands – any operations I requested in View Manager were ignored. I didn’t find any obvious errors in the logs.

I ran through the troubleshooting options in KB1030698 without finding any issues. I validated the SDK was responding by going to https://vcenteripaddress/sdk/vimService.wsdl . I couldn’t find any cause for the outage, so I opened up a Sev-1 ticket with VMware Support.

The support tech concluded that a problem with the ADAM database was preventing Composer from doing the job. He had me shut down all but one connection broker, then restart the View services on the remaining broker. At this point, commands issued on the broker were obeyed by Composer. We deleted or refreshed all of the desktops listed under Problem Desktops. Once we were sure that the ADAM database reflected the true state of the environment as reflected in vCenter, we restarted the other brokers. They synced databases and the problem was resolved.

Posted on

Unable to delete vSphere datastore

I had a nearly empty datastore in one of my vSphere clusters that I wanted to destroy so I could steal some of the LUNs to grow a different datastore. I migrated all machines and files off the datastore, it was completely empty, but I kept getting an error from vCenter saying it was unable to delete the datastore because the resource was in use. I eventually discovered that a VM in the cluster had its CD-ROM connected to one of the ISOs that I had moved off the datastore. Disconnected the CD-ROM and I was good to go.