It Can’t Just Be Me – A Rant On Hyper-V and SCVMM 2012’s Failings

After having a rather rough start to a school year (read: ass whooped) it’s time to unleash some hatred that has been building for several years.  I’ll try to give a fair shake, but be warned, there are years of bottled up emotions Hyper-V Failures Cause Double Facepalmhere.

Early on, Hyper-V was being touted as a serious competitor to ESX.  I was skeptical at the time given how well ESX worked and how long it had to mature, but I wanted to give Hyper-V a fair shake.  There were a number of comparisons at the time that gave it an “ok” grade but none of the ones I found really ripped it a new one.  They all agreed on two points:  It loads virtual machines, and is free.

However, these two points do not an enterprise virtualization platform make.  “Don’t worry, Hyper-V 2.0 in 2008 R2 will fix these woes!” they said.  So, evaluate 2008 R2 I did.  We had a number of VMs that we needed to run.  At the time, ESX had no free option, so we decided to run Hyper-V.  My experience was basically the same as all the other reviewers:  It loads virtual machines and is free.  The difference was, I hated it.  I hated the stupid interface.  I hated the buggy GUI.  I hated the stupid limited OS requirements for networking (teaming?! standby nics? wru?!).  I really hated just about everything about it.  I was told by fantatical Hyper-V evangelists that I was just lacking SCVMM, and that everything is taken care of with it.  So, I held of judgement and said, “meh” then switched the host to ESXi when it was free.

Years passed and our VDI pilot initiative came up.  I did a POC with ESXi and Hyper-V.  In the Hyper-V POC I demonstrated that VMs do in fact load, but for the most part the user buy-in was done using VMware.  When it came to implement our pilot, Hyper-V won out due to cost savings.  (It’s free and don’t need no fancy features, remember?!).  I immediately hated it.  It became clear quite quickly that the consultant designing our VDI solution had extremely limited experience with Hyper-V, let alone Citrix on top of Hyper-V, and led to some really awkward conversations like, “Lets go to technet and read what it says.”  Ok, thanks.  I’d like my $300/hr back please.  In fact, you pay me.

After we got the ball rolling with SCVMM 2008 R2 and Hyper-V 2008 R2, we had a functioning Citrix XenDesktop environment.  The promised land!  Everything would be awesome!  Except…there were these little nagging things with SCVMM that just didn’t work the way a sane person would think they should.  So again I consulted ‘teh interwebs’ and was told (again) that it was “fixed in (the next version) 2012”.  Luckily, System Center 2012 was due out in a week!  I immediately jumped to SCVMM 2012 and was relieved…things were better!  However, they still sucked.  There are all sorts of stupid little behaviors with SCVMM and Hyper-V.  Granted,  a lot of them may be mitigated through deep tuning, I’d like to think that some things should “just work” in basic scenarios out of the box.  Let me list a few of my gripes here.

  • The UI.  Apart from being terrible, its slow to load and takes up GOBS of memory.  Thanks for making me swap to hard disk with 8GB of RAM to load a grid view.  Loading a minimal VM list the VMM console takes my machine ~500MB of memory just to open.
  • Job execution speed.  All too often I discovered that “Just do it in Hyper-V” or “Failover cluster manager” was the answer to the speed at which jobs would execute and return results to VMM.  Basic tasks like editing a machine’s memory can take a minute just to open the GUI, make the change and close it.
  • Online editing.  Are you joking?  IDE drives? Network adapters can’t change their VLANs while powered on?
  • Hot adding memory/processors.  Wait, what?  You’re telling me you can do this with datacenter on VMware but not on Microsoft’s flagship virtualization platform that is bringing them to the cloud era?  Yup.
  • Memory Limits.  64 GB max in a VM.  Ok, maybe this seemed like a lot in 2008/2009, but honestly, it’s tiny.  Tying platform releases to other major OS releases really doesn’t allow the product to evolve in order to keep up with changing business needs.
  • Maximum number of commands.  Highlight 20 VMs and execute a command like power off?  Nope!  Watch as your server becomes unresponsive to WINRM commands.  Yes, you can tweak WinRM and make this “feel” a bit better, but honestly wth?
  • Stupid little GUI bugs.  Left click VM 1.  Shift left click VM5.  Right click, VM2 unselects, can’t open properties.  Ok, wth.  Clear selection by left clicking VM6?  VM1 and VM6 are selected!  What?!
  • More stupid GUI bugs.  Click VM 1 on host 1.  Click host 2.  Click VM2 on Host 2.  Power off action.  Read GUI “Are you sure you want to power off VM2?”.  Accept, watch in horror as VM1 and VM2 power off.
  • The job log.  Use it for 10 min with a large number of jobs for the window.  If you haven’t suicided I’ll buy you a beer.
  • Powershell cmdlets.  Do you like to hit the pipe?  I do.  I pipe a lot.  In fact, it’s almost as if Microsoft designed powershell around piping objects to other objects for efficiency.  Try it.  I dare you.  get-vm YourFilter | Get-VirtualHardDisk | Remove-VirtualHardDisk.  (This is most fun if you have SCSI disks.  Warning:  Get-VirtualHardDisk won’t be acting on your get-vm filtered output.  This command will screw your production servers if they have scsi disks by removing them)
  • -Whatif.  Whatif SCVMM cmdlet’s didn’t suck?  Oh wait, many don’t support -whatif.  Oops.
  • Memory priority.  Set it in Hyper-V.  Can’t set in SCVMM.  What?
  • VM’s getting “locked” by stupid jobs like refresh VM, failing your commands.  Isn’t this why we invented queues?
  • The get-job cmdlet.  Honestly, die.
  • Checkpoints disks aren’t removed unless you power the VM off
  • Memory swap files (*.bin) cannot be relocated away from configuration file
  • Memory swap files cannot be disabled by any means
  • Configuration files are XML, but you can’t just copy them from machine to machine.  You must Export (to…XML…) then import the machine.
  • The concept of the VMM library.  F-A-I-L.  UNC? Nope.  Let’s require an agent.
  • Mounting ISOs copies ISO  to VM directory regardless of preference.
  • ^-  Doesn’t always get cleaned up after unmounting
  • In Hyper-V its a snapshot, in SCVMM it is checkpoint.
  • Some things require Hyper-V manager when you have SCVMM
  • Some things require Failover cluster manager when you have SCVMM
  • Hyper-V Clusters are a sin unto themselves.
  • Cluster shared volumes (And yes, 2012, I know there are improvements but take a moment to consider all the caveats) are a sin
  • The fact that apparently no one else on earth actually uses it in production.  Just try getting decent support from Microsoft partners or using modern versions of software
  • The fact that everything is apparently fixed in the next version.  How many versions have I heard this?!
  • PXE Boot only on legacy NIC
  • Legacy NICs – 100mbitwth
  • Synthetic NICs
  • Oops they’re just called Network Adapters in SCVMM, not Synthetic
  • No VLAN setting in templates

I’ll probably add to this list, and make clean up this list a bit more if I can be bothered to think about Hyper-V and SCVMM anymore.  To be honest, I’m glad we’re moving our 16 hosts to ESXi (even standard – hell, free would be better if we had vCenter support for it!).

Leave a comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

0 thoughts on “It Can’t Just Be Me – A Rant On Hyper-V and SCVMM 2012’s Failings

  • Peter Mueller

    I implemented esx 3.5 when it came out at my school division, at the time hyperV was in beta and vmotion was a ‘feature’ that MS would maybe add in the future. (MS Marketing at the time said moving a live VM to a different host was a dangerous practice and something customers shouldn’t want to do anyways).

    Every time a new revision of HyperV comes out it’s always reviewed as not quite as good as the competition, but hey, it’s cheaper.

    IMHO it comes with a cost too great to bear. You will have to pry my vsphere licenses out of my cold dead hands, they are pure gold to me.

    • Karl

      Nobody ever pays attention to TCO, Microsoft is forced to fight in the virtualized arena because of most virtual appliances being written in Linux and the cloud. Hyper V is key for them to keep Microsoft licensing and as a spring board to serenade people into Azure.

  • James S

    I used to work for a VMware reseller and now run a hosting company built on ESX. ESX is certainly a stable product and I love everything about it except the cost and the fact the VMware owns it. Working for the reseller, I consistently saw them attempting to kill their channel by stealing accounts and pulling a bunch of underhanded BS. Over time, I finally said F*#@ VMware and decided to move over to Hyper-V, which we’re in the middle of doing now. It has been a long journey and I’ve almost jumped out of the window several times now. The culprit…SCVMM 2012, what a huge piece of sh!t. Everything fails, tons of bugs, and nothing makes sense, it’s like Paris Hilton wrote the software in Dreamweaver.

    I have finally realized that I needed to use PowerShell to do everything. Once I started using PowerShell to perform the configuration tasks, etc… I had much better luck. Microsoft needs to step up their game significantly if they want anyone to take SCVMM seriously, and this is all coming from a die hard Microsoft guy. I did the updates, I read the documents, there’s just no sugar coating that can make SCVMM taste like Virtual Center.

    Thank you for writing this article, it’s good to know that I’m not crazy and that other people are running into the same laughable issues as me!

    • Atum Post author

      Even with the cmdlets things get insane (see the points about terrible cmdlet design). Let’s not forget that -erroraction is ignored on some cmdlets, and there’s limited -whatif support (and piping… almost no [sane] piping!). Thanks for your comment – like you said, it IS good to know you’re not the only one out there.

  • Will Moore

    This whole article made me laugh because sadly I can relate to every single word. I am currently going through a nightmarish implementation of a 16 node hyper-v 2012 r2 cluster using SCVMM 2012 R2 for “Management”. Windows 2012 R2 (Hyper-V 3.0) was supposed to fix all the shortcomings of Hyper-V. SCVMM R2 introduced some nice new cloud and networking features. New features don’t matter if the underlying foundation completely blows. in additon, 95% of the annoyances you listed above still exist in 2012 R2 along with the new features that introduce new annoyances because they don’t work. You still have to use FCM and Hyper-V manager to do certain tasks. 3 tools to manage my cluster nodes and guests still? No thanks. To make matters worse we are experiencing these issues: which basically make our brand new 16 node HP c7000 blade chassis completely unusable for production. Opened tickets with MS Premier support and HP but still no fix (1 month and counting). This is a hyper-v sucking problem not an HP hardware problem BTW. I don’t care how much marketing fluff MS spews about Hyper-V and its superiority over VMWare. There is no comparing the two. The only thing keeping Hyper-V afloat is the licensing model. Like you said, its loads VMs and it’s free, beyond that, it is a complete failure.

    • Atum Post author

      Oh man that is not promising. I have installed 2012 R2 but haven’t yet been able to install SCVMM 2012 R2 in the lab to try things out! Keep fighting the good fight — sorry to hear you’re still having trouble but I can only hope Microsoft sees this and similar post(s). Unfortunately I haven’t found a huge amount of “What the heck is wrong with SCVMM/HyperV?” posts out there, which is partly why I was inspired to write this rant. It felt like I was the only one!

      • RobbieCrash

        The culprit here seems to be SCVMM, or SC in general. From what I can tell, clustering, failover, HA, all the rest of it, works great when you’re managing things manually through the Hyper-V msc. As soon as you start trying to get SCVMM working, everything goes to shit.

        Have you tried using a clustered Hyper-V setup without SCVMM?

        We’re looking at implementing a 3node cluster to serve about 20VMs, so nothing that can’t be handled manually. From what I can tell, as long as we stay away from SCVMM we’ll be ok.

        • Atum Post author

          Using Hyper-V manager seems to be the most stable, and granted, a number of the issues DO relate to being WinRM. However as I stated, I’ve tried Hyper-V 1 and Hyper-V 2 (as well as now a small Hyper-V 3 test lab) and “all problems” still seem to be resolved in the next version. However, for 20 VMs in a small cluster I doubt you’d run into problems. Just use the latest server version.

        • Will Moore

          Hyper-V isn’t an enterprise product plain and simple. I define enterprise in this case as being able to scale out to greater than 2 or 3 nodes in a cluster (while remaining stable), having a solid centralized management platform and integration points with 3rd party vendors. Did you know that MS recommends running a “Full” validation test when adding a node to a cluster? Did you know that a “Full” validation test takes the cluster roles offline? How’s that for scale out features?? What a joke. I can’t tell you how many people have said to me “it works great on my two node cluster”, or, “I’m running it at home and it works great!”. That really doesn’t help. The real problem comes when you have the need to create a “private cloud” for your organization and you need it to scale and still remain reliable while having ONE tool to manage the whole thing. Hyper-V won’t do it in its current form. If you have never used a real hypervisor, do yourself a favor and download vmware’s 60 day trial (or XEN server for that matter) and setup a cluster with vCenter. I know VMware costs more but it works and if you do a full TCO analysis it becomes even more attractive. We spent about 4 months working with MS trying to get our 2012 R2 hyper-v cluster working. R2 was never stable. We then went back to 2012 in an 8 node cluster and it works ok We have now setup a 3 node Vmware cluster on the same exact hardware and it has worked flawlessly for over 3 weeks. Bottom line….you get what you pay for and hyper-v is “FREE”!

  • kim

    I could not describe it better. SCVMM is on top of my list of terrible products used in enterprises. We uses it as part of TFS lab and not a day goes without errors and lost connections between SCVMM and hyper-v. I’ve created 4-5 MS support cases and they haven’t been able to fix it. according to them a lot of the winRM bugs are solved in server 2012, but i havent tested it.
    We originally brought hardware to run VMware, but management was blinded by the word free and they were ex MS people as well. so now we have to big hosts for hyper-v.
    For the time wasted we could have created a lab and SCVMM that would perform better.

  • jdemii

    I love your comments, ROFL, I’m gonna take them to my manager because he keeps telling me that I just need to get used to it. No I don’t. I had something working perfectly before, whenever there was a problem with vCenter, it was always fixable in a logical manner. I loved it. Of course it’s gonna cost more. Maybe vmware has gotten too greedy, if they hadn’t raised their prices that much, our managers would never have considered Hyper-V.

  • PaulFrehely

    How about the epic fail of no clipboard in the Console App?!! FOC has it, HV Manager has it…SCVMM doesn’t have it.
    Installing VM additions leaves the ISO mounted after the install,which then screws things up if you want to Live Migrate
    The only thing I use if for now is deployments and mass startups/shutdowns. I’ve never had the issue you described in respects to this. I can shutdown or power up 50 VM’s without any problem. VM’s are all on CSV SANs with FOC enabled.

    The biggest fail is having to use SCVMM, FOC, and HV to do things instead of SCVMM being a catch-all tool. CSV stability has been really dodgy in 2012 and that was supposed to be the big up sell.

    This tool is a pretty good catch-all, but it does have a few bugs still –

  • Brit Davis

    I will just add my own experience: I’ve been running Hyper-V since it was born. I currently have serveral 2008 R2 standalone hosts and a three-node cluster in a c7000 running bl495c blades. Using SCVMM 2012 SP1. This has been my live production envoronment for several years. Once I got the WinRM stuff sorted out, I’ve had zero problems. I do LM from time to time intra-cluster, I move VMs to and from the standalone hosts, even in different sites, and it, for me, has just worked. Soon I’m moving all of this to 2012 R2 and SOFS CiB for SMB 3.0 storage. See you on the other side.

    I guess I’m just lucky.

  • Luc Barendrecht

    Great article, thanks. I’m in Sales and knew nothing about VMware & Hyper-V until recently. My tech-colleagues couldn’t explain to me why our prices are higher for our customers. Partly because we use the VMware Enterprise Plus license we have to up our prices and I want to be able to explain to customers why — thus my research.

    I almost started to believe that i had no good explanation why we use VMware over Hyper-V but after this comment and the one ping-backed to at the end of the comment section — i am convinced that i have some solid arguments on the phone and during conversations that our products are not just more expensive for no reason.

    Thanks again!
    Luc Barendrecht
    Sales & Marketing for IT creation

  • Chris

    Just want to comment that I have been on Hyper-V since 2008 (pre R2) and SCVMM 2008 also because of the economy of the licensing model.

    At any rate, last year I moved the environment to SCVMM and Server 2012R2 and leveraged SCVMM to fully deploy converged networking on 2x10GbE to a SoFS SMB 3 storage target. You can easily build out the fabric for bare metal cluster deployments and NVGRE.

    I had some struggles with understanding how to build the fabric out in SCVMM. But…It works fine once understood how the building blocks fit.

  • C W

    I’ve heard too many horror stories of vCenter guys trying to set up SCVMM and totally screwing it up and/or hating it (or consultants doing it who aren’t familiar with SCVMM and causing the customer to hate it). First thought: SCVMM is NOT vCenter, and you cant expect or treat or configure them the same. I see it ALL the time where someone certified in VMware “tries” to set up HyperV and SCVMM and totally screws it up. I’ll be the first to admit that SCVMM has some shortcomings (and granted, a lot of these gripes ARE fixed in 2012 R2+)… especially the memory usage. However, a properly configured 2012 R2 cluster and SCVMM 2012 R2 solution works quite well. It is A LOT of knowledge to absorb though; every SC product is practically a career. There are challenges, but honestly I bet I could find a nearly identical writeup by a MS SCVMM guy who tried to set up vCenter 🙂

    Bottom line, stick with what you know. As someone pointed out about TCO, the TRUE TCO is going to be defined by A) what you know B) what you can support the best. Even if vCenter went free next week, you wouldn’t see me switching because in the long run my TCO calculations work out better with Hyper-V and SCVMM because that’s what I know, that’s what I can support and that’s what I’ve spent years building my knowledge in.

    • Larry

      “There are challenges, but honestly I bet I could find a nearly identical writeup by a MS SCVMM guy who tried to set up vCenter”

      Sorry but you are wrong we have both. Production is vcenter/vmware 6.x. DEV and QA environments are SCVMM 2012 R2/Hyper V 2012 R2 with all of the updates.

      We have had Hyper V since 2008 R2 (V2). We tested the first version but without live migrations it was a no go.

      VMware is 100x times easier to support in every single way. Stability is 100x times better with VMware.

      The Microsoft solutions are way over complex. You state “every SC product is practically a career” and it could not be more true. This is a COST, that IT departments need to take into account. With limited IT staff’s VMware is easier to manage and those limited IT people can do more.

      You can do more and more with SCVMM, but you still need to use FO and even Hyper V manager at times. MS clustering is a huge weak point. It is general purpose clustering built upon a general purpose OS that can be used for various MS solutions, including Hyper V. I cant tell you have many times a cluster has just had a bad day at the MS FO clustering level, bringing our Hyper V environments to the ground. Much better in the 2012 R2 world but still issues. Cluster validations that still take hours on big clusters and keep your fingers crossed that including the storage tests does not bring down volumes and crash VM’s. Manually moving control of volumes to other nodes is a MUST IMHO before rebooting a host. Automatically letting the move is asking for trouble.

      I have had crashes that orphaned VM’s and caused other problems (looooooong GUID issues) and I sit on the phone with Premiere support (hidden cost of Hyper V) while MS people from FO clusters, Hyper V and the SCVMM groups argue on how to fix an issue. No I am not making this stuff up.

      Upgrading a cluster is about to get better in Hyper V with the next version. Until now is was a huge task because you either needed extra hardware or you had to bust a node out and create a new cluster. That whole process on say a 8 node cluster took days (long ones) or weeks. With VMware half a day at most, from upgrading vcenter to upgrading the 8 hosts. I upgrade vmware in the middle of the day without any hesitation and that is in our production environment.

      If my employer forced me to move our production servers to Hyper V I would do it. However I would be looking for a new job at the same time. There is no way I would run production work loads in Hyper V and be responsible for it.

  • Dave Graham

    I presently have 136 Hosts managed by SCVMM 20212 R2, this includes our production and lab environment and offsite servers and it’s just a terrible piece of garbage that constantly fails.. The one shinning light that looked really promising was the virtual machine servicing tool, but that doesn’t work with R2 for crying out loud so we have no way to keep our templates up to date without using Azure and just installing everything that was approved on the WSUS server. I’m also having an issue with not being able to remove some SMB file shares we added during our mass conversion from ESX to Hyper-V, we converted about 700 virtual machines so anytime I try and refresh the host cluster it takes forever to time out because the shares don’t exist anymore, but does SCVMM let me remove them? No, why would I want to do that..