Yet Another CX-4 / VAAI With vSphere 5.x Post


Long have I known about the problems of upgrading from vSphere 4 to vSphere 5 when it came to VAAI.  Long had I heard the discussion and seen the blog posts about who would support what.  I even made sure to make my voice heard on the subject.  It really hadn’t been of terrible concern – things were running just fine.  Until they weren’t.

When Good Enough Goes Bad

My recent adventures with VMware Converter to create test VMs had left me with a rather sour taste in my mouth.  There were periods of the process when my newly created LUNs, consisting of a pool of over 50 disks – about half high-performance FC and half SATA – that the performance just wasn’t right.  Something…something was off.  I couldn’t put my finger on it, but the drives were ripping along, then all of the sudden things would go silent.  No CPU usage on the VM.  Little to no disk activity in the pool or from the host…but the VM was just stuck.

I connected to one of the hosts directly and noticed a warning about the volume being locked.  Some searching lead me down the path of troubleshooting, and I found only more problems.

As it turns out, we had some configuration issues with about half of the hosts in one cluster.  Some hosts were using Failover mode 1, others were using Failover mode 4 (ALUA).  VAAI on the CX4 requires failover mode 4.  With vSphere 5, EMC finally decided to support VAAI in October 2012.  Through a series of “maintenance mode” operations, we were able to set all hosts to ALUA, but still, Hardware Acceleration appeared “Unsupported” on most hosts.

PowerPath/VE

Years ago we had purchased PowerPath/VE, but due to the cost we decided not to continue adding it to our standard build.  For those original hosts, however, we never uninstalled it (since, hey, we still had licenses, so why not?).  When vCenter necessitated a server rebuild, the PowerPath licensing server wasn’t moved to the new server and it was operating unlicensed.  Which…shouldn’t be a problem…right?  Perhaps.  Being sticklers for configuration consistency, we wanted to make sure that all hosts had the same multipathing.  We removed PowerPath from our AutoDeploy image profile and from the physical hosts and removed the claim rules.  Still, hardware acceleration was “Unknown” on some hosts.  Now what?

Storage Claim Rules in Host Profile

Host Profile, eat your heart out.  As it turns out, we needed to add a claim rule (65431) for VMW_VAAIP_CX and VAAI_FILTER for DGC, otherwise the modules weren’t loading.  Why this is the case isn’t quite clear for me, but if any readers care to enlighten me, I’d appreciate it.

Results

So far, I haven’t experienced the same “storage locking” issue, but time will tell.  Hopefully this gremlin has been put to bed and we’ll revise our processes in the future to ensure all hosts are configured in a consistent manner.

Leave a Reply