It appears that one of the newest features in vSphere 4 acts as a doubled edge sword.  The ability to have “hot add” and “hot plug” support has become a must in virtual environments. In fact, features that enable an administrator to add virtual CPU’s and memory on the fly without having to power off a virtual machine is partly responsible for the success of vSphere 4 on both ESX and ESXi hosts. Using this technology adds so much to the virtual world and the applications that run on them that thrive on processing power and resources. All of this combined makes vSphere insanely flexible and enormously robust in a way that leads to virtually no downtime. Unfortunately, the newest “hot plug” support for virtual devices does come at a cost.

<!--more-->With version 7 of the virtual hardware you have the ability to “hot plug”, or in our scenario, unplug devices at any time simply by selecting “Safely Remove Hardware and Eject Media” on the taskbar of any virtual machine with VMware Tools installed. This is a great feature for production and staging environments to quickly setup machines in custom scenarios with little effort on the administrative side and more importantly without an outage. However, if a user were to accidently eject a critical piece of hardware it could render the virtual machine unusable. In the event that a user was to remove or eject a virtual network adapter by utilizing the above method the machine would instantly drop network connectivity and the only way to access the machine would be through the vCenter console. To add to this distress, by removing the virtual hardware from the guest OS, you have also inadvertently removed the virtual hardware from the vCenter console as well as leaving virtually no trace that it was ever there in the beginning. This could cause huge headaches when troubleshooting if you were unaware of this feature.

This is not to say that the technology still isn’t amazing it is just to be used with caution, the old saying “With great power comes great responsibility.” definitely holds true here. There are special cases where the “hot add” feature can be disabled thus preventing these kinds of mistakes. However, in our scenario there is still no official way to disable the feature if you are using the vmxnet3 driver which is very widely used.

You can view the KB article this issue here.
(<a href="http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1020718">http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1020718</a>)

And a fix here
(<a href="http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1012225">http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&amp;cmd=displayKC&amp;externalId=1012225</a>)