Tag Archives: NIC

Moving PVS VMs from e1000 to VMXNET3 network adapter

A client needed to remove the e1000 NIC from all VMs in a PVS pool and replace it with the VMXNET3 adapter. PVS VMs are registered by MAC address – replacing the NIC means a new MAC, and PVS has to be updated to allow the VM to boot.

I needed a script to remove the old e1000 NIC, add a new VMXNET3 NIC, and register the new NIC’s MAC with PVS. I knew I would easily accomplish the VM changes with PowerCLI, but I didn’t know what options there were with Citrix. I found what I needed in MCLIPSSNapin, a PowerShell snap-in installed on all PVS servers. The snap-in gives you Powershell control over just about anything you need to do on a PVS server.

I didn’t want to install PowerCLI on the production PVS servers, and I didn’t want to install PVS somewhere else or try manually copying files over. I decided I needed one script to swap out the NICs and dump a list of VMs and MAC address to a text file. Then a second script to read the text file and make the PVS changes.

First, the PowerCLI script. We put the desktop pool into maintenance mode with all desktops shut down. It takes about 10 seconds per VM to execute this script.

Param(
	[switch] $WhatIf
,
	[switch] $IgnoreErrors
,
	[ValidateSet("e1000","vmxnet3")]
	[string] 
 	$NICToReplace = "e1000"
)

# vCenter folder containing the VMs to update
$FOLDER_NAME = "YourFolder"

# vCenter Name
$VCENTER_NAME = "YourvCenter"

#The portgroup that the replacement NIC will be connected to
$VLAN_NAME = "VLAN10"

#If you want all VMs in $FOLDER_NAME, leave $VMFilter empty. Otherwise, set it to a pipe-delimited list of VM names
$VMFilter = ""
#$VMFilter = "DESKTOP001|DESKTOP002"

$LOG_FILE_NAME = "debug.log"

Connect-VIServer $VCENTER_NAME

$NICToSet = "e1000"

if ( $NICToReplace -eq "e1000" )
{
	$NICToSet = "vmxnet3"
}
elseif ( $NICToReplace -eq "vmxnet3" )
{
	$NICTOSet = "e1000"
}


function LogThis
{
	Param([string] $LogText,
      	[string] $color = "Gray")
 Process
 {
    write-host -ForegroundColor $color $LogText 
    Add-Content -Path $LOG_FILE_NAME $LogText
 }
}

if ( Test-Path $LOG_FILE_NAME )
{
    Remove-Item $LOG_FILE_NAME
}

$errStatus = $false
$warnStatus = $false
$msg = ""

if ( $VMFilter.Length -eq 0 )
{
	$vms = Get-Folder $FOLDER_NAME | Get-VM
}
else
{
	$vms = Get-Folder $FOLDER_NAME | Get-VM | Where{ $_.Name -match $VMFilter }
}

foreach ($vm in $vms)
{
	$vm.Name
	$msg = ""


	if ( $vm.NetworkAdapters[0] -eq $null )
	{
		$errStatus = $true
		$msg = "No NIC found on " + $vm.Name
		LogThis $msg "Red"

	}
	else
	{
		if ( ($vm.NetworkAdapters | Measure-Object).Count  -gt 1)		{
			$errStatus = $true
			msg = "Multiple NICs found on " + $vm.Name
			LogThis $msg "Red"

		}
		else
		{
			if ( $vm.NetworkAdapters[0].type -ne $NICToReplace )
			{
				$warnStatus = $true
				$msg = "NIC is not " + $NICToReplace + ", found" + $vm.NetworkAdapters[0].type + " on " + $vm.Name
				LogThis $msg "Yellow"				
			}

				LogThis $vm.Name,$vm.NetworkAdapters[0].MacAddress

		}

	}



}

if ( $errStatus = $true -and $IgnoreErrors -ne $true)
{
	LogThis "Errors found, please correct and rerun the script." "Red"
 
}
else
{
	if ( $warnStatus = $true )
	{
		LogThis "Warnings were found, continuing." "Yellow"
	}
	foreach ( $vm in $vms )
	{
		if ( $WhatIf -eq $true )
		{
			$msg = "Whatif switch enabled, would have added " + $NICToSet + " NIC to " + $vm.Name
			LogThis $msg
		}
		else
		{
			$vm.NetworkAdapters[0] | Remove-NetworkAdapter -confirm:$false
			$vm | New-NetworkAdapter -NetworkName $VLAN_NAME -StartConnected -Type $NICToSet -confirm:$false
		}
	}

	if ( $VMFilter.Length -eq 0 )
	{
		$vms = Get-Folder $FOLDER_NAME | Get-VM
	}
	else
	{
		$vms = Get-Folder $FOLDER_NAME | Get-VM | Where{ $_.Name -match $VMFilter }
	}

	LogThis("Replaced MAC addresses:")
	foreach ( $vm in $vms )
	{
		LogThis $vm.Name,$vm.NetworkAdapters[0].MacAddress
	}
	
	
}

The script offers a -Whatif switch so you can run it in test mode without actually replacing the NIC. It writes all its output to $LOG_FILE_NAME. First it logs the VMs with their old MAC, then the replaced MAC. The output looks something like this:
VD0001 00:50:56:90:00:0a
VD0002 00:50:56:90:00:0b
VD0003 00:50:56:90:00:0c
VD0004 00:50:56:b8:00:0d
VD0005 00:50:56:b8:00:0e
Replaced MAC addresses:
VD0001 00:50:56:90:57:1b
VD0002 00:50:56:90:57:1c
VD0003 00:50:56:90:57:1d
VD0004 00:50:56:90:57:1e
VD0005 00:50:56:90:57:1f

Scan the logfile for any problems in the top section. The data after “Replaced MAC addresses:” is what the PVS server needs. Copy this over to the PVS host. Now we need to use MCLIPSSnapin, but first we have to register the DLL. I followed this Citrix blog for syntax:
“C:\Windows\Microsoft.NET\Framework64\v2.0.50727\installutil.exe” “C:\Program Files\Citrix\Provisioning Services Console\McliPSSnapIn.dll”

I copied the VM names and new MAC addresses to a text file vmlist.txt and put it on my PVS server, in the same folder as the following PowerShell script. It runs very quickly, it takes only a few seconds even if you are updating hundreds of VMs.

Add-PSSnapIn mclipssnapin
$vmlist = get-content "vmlist.txt"
foreach ($row in $vmlist)
{
	$vmname=$row.Split(" ")[0]
	$macaddress=$row.Split(" ")[1]
	$vmname
	$macaddress
	Mcli-Set Device –p devicename=$vmname –r devicemac=$macaddress
}

Now, replace the PVS pool’s image with one that is prepared for a VMXNET3 adapter and boot the pool. Migration complete!

Guest NICs disconnected after upgrade

We are upgrading our infrastructure to ESXi 4.1 and had an unexpected result in Development cluster where multiple VMs were suddenly disconnected after vMotion. It sounded a lot like a problem that I had seen before where a misconfiguration in the number of ports on a vSwitch prevents vMotioned VMs from being able to connect to the switch. If a vSwitch has too few available ports, the VMs that vMotion over are unable to connect to the switch. You generally avoid this with host profiles, but it’s possible a host in the Dev cluster fell out of sync. In any event, the server that was being upgraded this evening had been rebuilt and it wasn’t worth trying to figure out what the configuration might have been. I needed to go through, find all VMs that should have been connected but weren’t, and reconnect them. I decided that I needed:

  • VMs that were currently Powered On – obviously as Powered Off VMs are all disconnected
  • VMs with NICs currently set to “Connect at Power On” so I could avoid connecting something that an admin had intentionally left disconnected
  • VMs with NICs currently not connected

Note that this script will change network settings and REBOOT VMs if you execute it. I was watching the script while it executed, I pinged the guest DNS name first to ensure the IP wasn’t already on the network, then connected the NIC, then pinged again to make sure it was back on the network. I figured I could Control-C to stop if something looked wrong. I rebooted all of the guests to avoid any failed service / failed task problems that might have occurred while the guests were disconnected.

$vms=get-cluster "Development" | get-vm | Where { $_.PowerState -eq "PoweredOn" }| Sort-Object Name
foreach ($vm in $vms)
{
   $nics = $vm | get-networkadapter | Where {$_.ConnectionState.Connected -eq $false -and $_.ConnectionState.StartConnected -eq $true}
   if ($nics -ne $null)
   {
  	 foreach ( $nic in $nics )
  	 {
	     	write-host $vm.Name
	     	write-host $nic
	      	ping $vm.Guest.HostName -n 5
		$nic | Set-NetworkAdapter -Connected $true -confirm:$false
	 }
 
        ping $vm.Guest.HostName -n 5
	$vm | Restart-VMGuest
 
   }

Kernel Handles and Network Disconnects

We’ve had recurring problems recently with a file server mysteriously losing connection to the network for a few seconds. After going through all of the usual hardware checks, NIC, switch, drivers, etc, we came across this Microsoft NT Debugging Blog entry on kernel file handles. It essentially says to check running processes via Task Manager, Select Columns and add the “Handle Count” column, then sort by it. Anything over 5,000 handles is something to question. We found a process with over 350,000 handles causing our timeout issues. Filed a bug report with the application vendor and the problem disappeared.