I had this EXACT same issue and couldn't figure it out until I found your post and checked the port group for management and low and behold it didnt take the vswitch setting for the load balancing!! Couple clicks to make it IP HASH and all was good. Thanks!!