Working Through an Unsupported Cluster Configuration Scenario in Virtual Machine Manager

~ Chuck Timon | Senior Support Escalation Engineer

The insidious and not very clear virtual machine status Unsupported Cluster Configuration has been the bane of many a System Center Virtual Machine Manager (VMM) administrator’s existence. This status can surface at the most inopportune times, and it is usually when one is feeling confident things are running smoothly.

The screenshot above is from a very basic 2-Node Windows Server 2012 Hyper-V Cluster and it shows two virtual machines reporting an Unsupported Cluster Configuration status. Imagine, if you will, that this is a 64-Node Windows Server 2012 Hyper-V Failover Cluster hosting thousands of virtual machines and a large number of them display an Unsupported Cluster Configuration status. What do you do? How does one go about isolating the problem(s) and fixing the cluster? Make no mistake, this needs to be fixed because critical management tasks like migrating virtual machines\storage cannot be executed using SCVMM while virtual machines are in this state. To illustrate, when I right-click on one of the virtual machines to examine the options available, the ‘migration’ options are not available.

As a ‘Plan B’ you could always execute the migrations from the Hyper-V Failover Cluster itself, but that defeats the purpose of centralized management, right?

There is a fair amount of publicly available information that discusses this scenario. Information is available on the Microsoft TechNet site, but there are other sources as well such as personal blogs and forum posts. An Unsupported Cluster Configuration can be the result of a number of things mostly involving misconfigurations either in the virtual machines themselves or in the Hosts in the cluster. Resolving this type of problem also depends on the version of System Center Virtual Machine Manager that is running in the environment.

The purpose of this blog is to make a PowerShell script written by the SCVMM product team publicly available. The script identifies network configuration issues across hosts in a Hyper-V cluster managed by VMM 2012 (and SP1). I personally have used this script to assist a customer that was experiencing this issue across a large number of virtual machines hosted in a 16-Node Hyper-V cluster. This script was able to pinpoint no less than 10 misconfigurations involving multiple virtual switches configured in the cluster. Once identified, we were able to resolve all issues in just a few minutes.

The main types of error messages we are concerned with here are ones that result in an Unsupported Cluster Configuration status primarily due to a networking misconfiguration(s) somewhere in the host cluster. An example of such an error (Error (13921)) can be seen when checking the Status of a virtual machine as shown here:

There are other clues available in the SCVMM console that can help focus the search for the problem(s). Since this is a cluster, the first clue can be found by viewing the Virtual Switch information for the cluster itself.

In my cluster, I have three virtual switches configured on each Host (Public, Cluster and ISCSI). As you can see above, there is no reference to the Public switch, which is a problem. Going back to the customer I previously mentioned, there were no virtual switches listed in his display (and his was a 16-Node cluster). Next, I examined the network configuration for one of the problematic virtual machines and saw it was disconnected from the Public Virtual Machine network. Knowing a Virtual Machine network is associated with a Logical Network, I switched to the Hosts view for the Logical Networks and located the Hosts in the cluster. I was able to determine that one of the adapters in one of the Hosts was Non-compliant because it was not associated with any Logical Network (Error (13706)).

Another clue is available in the network configuration for the virtual machine itself. Opening the properties of the virtual machine in the SCVMM console and inspecting the Network Adapter configuration, you can see the adapter is Not Connected even though the virtual machine is mapped to an existing VM Network.

The SCVMM Jobs display is informative as well. Locate either the latest Refresh virtual machine or Refresh host cluster job, and you will see indicators that there are problems at both the Host and Virtual Machine level. The Refresh host cluster job generates an informational message (Information (26845)):

While the Refresh virtual machine job generates an error (Error (13921)):

While playing Sherlock Holmes can be fun, one could just as easily run the script and have it point out the problem(s). I pulled the script out of the SCVMM library, ran it, and it quickly identified the problem for me –

The script requires two inputs: The name of the cluster and the name of a virtual switch configured in the cluster. This means that if more than one virtual switch is missing in the properties of the cluster, as shown above, the script must be run to identify problems for each missing switch configuration. It is recommended that the script be run until no problems are reported just to be sure all network configuration issues have been addressed.

Here is the script:

#-------------------------------------------------
#Finds Why HA network is not reported as HA by VMM
#This script is provided by the SCVMM Product Team 'as-is' and you use it at your own risk
#-------------------------------------------------
function CompareList($list1, $list2)
{
if( $list1.count -ne $list2.count )
{
return $false
}
for($ndx = 0; $ndx -lt $list1.count ; $ndx++)
{
if( $list1[$ndx].ToString() -ne $list2[$ndx].ToString() )
{
return $false
}
}
return $true
}
function ValidateHANetwork($clusterName, $switchName)
{
$validationResult = $true
$cluster = get-scvmhostcluster -name $clusterName
$vNic = Get-SCVirtualNetwork -VMHost $cluster.nodes[0] | where {$_.name -eq $switchName}
if( $vNic -eq $null )
{
Write-Host "Virtual Switch not found in host " $cluster.nodes[0].Name
$validationResult = $false
}
else
{
if( $vNic.VMHostNetworkAdapters.Count -eq 0 )
{
Write-Host "Virtual Switch not attached to external network card on host " + $cluster.nodes[0].Name
$validationResult = $false
}
else
{
foreach($node in $cluster.nodes)
{
Write-Host "==========================================================================="
Write-Host "Comparing " $node.Name
Write-Host "==========================================================================="
$vNicToCompare = Get-SCVirtualNetwork -VMHost $node | where {$_.name -eq $switchName}
if( $vNicToCompare -eq $null )
{
$validationResult = $false
Write-Host "Virtual Switch not found in host " $node.Name
}
if( $vNicToCompare.VMHostNetworkAdapters.Count -eq 0)
{
$validationResult = $false
Write-Host "Virtual Switch not attached to external network card on host " $node.Name
}
if( $vNic.VMHostNetworkAdapters[0].LogicalNetworks -eq $null )
{
if( $vNicToCompare.VMHostNetworkAdapters[0].LogicalNetworks -ne $null )
{
$validationResult = $false
Write-Host "Net Adapter " $vNic.VMHostNetworkAdapters.Name " for " $cluster.nodes[0].name " is not connected to logical network but Net adapter " $vNicToCompare.VMHostNetworkAdapters.Name " for " $node.Name
}
}
else
{
$ln1 = @($vNic.VMHostNetworkAdapters[0].LogicalNetworks | Sort-Object Name)
$ln2 = @($vNicToCompare.VMHostNetworkAdapters[0].LogicalNetworks | Sort-Object Name)
$ln1Id = @($vNic.VMHostNetworkAdapters[0].LogicalNetworks | Sort-Object ID | select ID)
$ln2Id = @($vNicToCompare.VMHostNetworkAdapters[0].LogicalNetworks | Sort-Object ID | select ID)
$result = CompareList $ln1Id $ln2Id
if( $result -eq $false )
{
Write-Host "Logical Networks on Adapters don't match"
Write-Host "------------------------------------------------"
Write-Host "Host " $cluster.nodes[0].name " Logical networks for adapter " $vNic.VMHostNetworkAdapters[0].Name
@($ln1) | ft *
Write-Host "------------------------------------------------"
Write-Host "Host " $node.name " Logical networks for adapter " $vNicToCompare.VMHostNetworkAdapters[0].Name
@($ln2) | ft *
Write-Host "------------------------------------------------"
$validationResult = $false
}
else
{
$sub1 = @($vNic.VMHostNetworkAdapters[0].SubnetVLans | Sort-Object Name)
$sub2 = @($vNicToCompare.VMHostNetworkAdapters[0].SubnetVLans | Sort-Object Name)
$sub1Id = @($vNic.VMHostNetworkAdapters[0].SubnetVLans | Sort-Object ID | select ID)
$sub2Id = @($vNicToCompare.VMHostNetworkAdapters[0].SubnetVLans | Sort-Object ID | select ID)
$result = CompareList $sub1Id $sub2Id
if( $result -eq $false )
{
Write-Host "Subnets on Adapters don't match"
Write-Host "------------------------------------------------"
Write-Host "Host " $cluster.nodes[0].name " Subnets"
@($sub1) | ft *
Write-Host "------------------------------------------------"
Write-Host "Host " $node.name " Subnets"
@($sub2) | ft *
Write-Host "------------------------------------------------"
$validationResult = $false
}
else
{
if( $vNic.VMHostNetworkAdapters[0].VLanMode -ne $vNicToCompare.VMHostNetworkAdapters[0].VLanMode )
{
Write-Host "VlanModes on Adapters don't match"
Write-Host "------------------------------------------------"
Write-Host "Host " $cluster.nodes[0].name " Subnets"
$vNic.VMHostNetworkAdapters[0].VLanMode
Write-Host "------------------------------------------------"
Write-Host "Host " $node.name " Subnets"
$vNicToCompare.VMHostNetworkAdapters[0].VLanMode
Write-Host "------------------------------------------------"
$validationResult = $false
}
}
}

}
}
}
return $validationResult
}
if ($args.Length -ne 2 )
{
Write-Host "Usage: CheckClusterNetwork <clusterName> <switchName>"
}
else
{
$result = ValidateHANetwork $args[0] $args[1]
if( $result -eq $true)
{
Write-Host "The cluster virutal network is HA and is configured correctly"
}
}

While we are unable to provide any support for the script itself, I hope you will find this useful. 

Chuck Timon | Senior Support Escalation Engineer | Microsoft GBS Management and Security Division

Get the latest System Center news onFacebookandTwitter:

System Center All Up: http://blogs.technet.com/b/systemcenter/
System Center – Configuration Manager Support Team blog: http://blogs.technet.com/configurationmgr/
System Center – Data Protection Manager Team blog: http://blogs.technet.com/dpm/
System Center – Orchestrator Support Team blog: http://blogs.technet.com/b/orchestrator/
System Center – Operations Manager Team blog: http://blogs.technet.com/momteam/
System Center – Service Manager Team blog: http://blogs.technet.com/b/servicemanager
System Center – Virtual Machine Manager Team blog: http://blogs.technet.com/scvmm

Windows Intune: http://blogs.technet.com/b/windowsintune/
WSUS Support Team blog: http://blogs.technet.com/sus/
The AD RMS blog: http://blogs.technet.com/b/rmssupp/

App-V Team blog: http://blogs.technet.com/appv/
MED-V Team blog: http://blogs.technet.com/medv/
Server App-V Team blog: http://blogs.technet.com/b/serverappv

The Forefront Endpoint Protection blog : http://blogs.technet.com/b/clientsecurity/
The Forefront Identity Manager blog : http://blogs.msdn.com/b/ms- identity-support/
The Forefront TMG blog: http://blogs.technet.com/b/isablog/
The Forefront UAG blog: http://blogs.technet.com/b/edgeaccessblog/

vmm 2008 scvmm 2008 vmm 2012 scvmm 2012 vmm2008 scvmm2008 vmm2012 scvmm2012 SP1 R2