vmkTree and ESXi

If you are in the virtulization arena, you have to be following Duncan. And you should know what he says is the word of God when it comes to virtualization. Going through his blog this morning, I came across ‘vmktree’ (tool to monitor performance counters on your hosts and VMs). As he has mentioned, it’s the tool from back in the days written by, Lars. He spoke highly of it and I had to give it a go. I was amazed at what this tool has to offer, you can run it off your vMA appliance and it has a nice web interface as opposed to the numbers you see in ESXTOP. As Duncan mentioned, those numbers don’t really help you see a trend. I won’t go into much further details. Lars has very simple installation instructions that works. Below is the link:

http://vmktree.org/?p=14

The God himself has endorsed this tool below:
http://www.yellow-bricks.com/2010/12/23/cool-tool-vmktree/

 

High Availability and Primary Hosts

One of the things that is often overlooked is the complexities involved in making HA work. Because of its not so complex interface, we forget what happens in the background. When we add the first host to a cluster that has HA enabled, it becomes a primary host. Likewise the first 5 host that enter a cluster are all primary hosts. Any host beyond the 5th host is tagged as a secondary host.

The primary hosts are responsible for making sure there is peace and harmony. However, when all these 5 hosts go missing, HA stops working. This is why it’s crucial to understand what could cause these hosts to fail and what happens when we lose a primary.

When we put a primary host in maintenance mode, a secondary host is promoted to the primary status. What does this tell you? The roles would be moving around depending on how big your cluster is and how often you place your hosts in maintenance.

However when a primary host fails or power’s off, a secondary is not promoted. The cluster now only has 4 primaries. And if another primary fails, we have 3 primaries. I guess you get the picture. Basically you cannot afford to lose more than 4 primary hosts in a cluster. So having more than four hosts in a HA cluster racked in the same cabinet may not be the best idea. What if all 5 primary hosts reside in the same cabinet and the whole cabinet loses power? That’s right your monstrous HA cluster will not work.

 

Semi-Dead Host in vSphere

So another frustration that I came across recently after upgrading to ESXi 4.1 was that one of the hosts came back with the “Remove” option grayed out. Everything seemed to be working but the grayed out option on one host certainly points to some issue that hasn’t caused obvious problems, YET. I tried restarting the vCenter service, server and manually removed the vpxa and everything under the sun. No LUCK! What worked was very interesting and a little odd. Though this may not be a documented feature, I suggest you do the following to save yourself the headache:

 

Create a temp cluster

Place the host in question to maintenance mode and and move it to the temp cluster just created

Remove any distributed switches

Right click on the on cluster and remove the cluster.

Now the host has been removed. You may now go ahead and add the host back to the cluster, using “Add Host” wizard. This time around the “Remove” option will not be grayed out.

Why cant you delete the original cluster and why create a new temp cluster? Well if you delete the original cluster, you will loose all your Resource Pools, something I was unwilling to do and frankly nobody should have to.