Right-size up

In the first Industry Insight in this four-part series on how to make the most of a company's virtual environment, I looked at the problem of virtual machine (VM) sprawl: how proliferating zombie VMs can tie up resources and cancel the benefits of virtualising in the first place.

In this Industry Insight, I consider another problem that undermines the business case for virtualising: misallocation and over-allocation of system resources.

Virtualisation works because it allows users to over-commit physical resources, knowing that most processes don't need as much as they are allocated - or won't need them all at the same time. It's like the way airlines overbook flights, knowing there's a fairly predictable proportion of people who won't turn up.

But when does virtual over-commitment, which allows companies to squeeze the most out of their resources, become real over-commitment that leads to sluggish performance and reduced productivity?

And what about the opposite problem - when companies get too cautious and give VMs resources they don't actually need? Assigning too many resources is wasteful and inefficient - it needlessly ties up processing power, memory or precious storage that could be better used somewhere else.

Both these problems can occur as a result of the common practice of creating new VMs from a template. This is by far the easiest way to do it: every new machine gets, say, two CPUs, 8Gb of RAM and 100Gb of storage space. That's a good, average spec - but unfortunately, not every VM has average needs. Some will need more resources, and some will need much less.

To make matters worse, it's very seldom possible to know in advance which machines are going to need more resources than the template calls for, and which are going to need less - there's no foolproof way to ensure every VM starts out with optimal resources. The right way to manage resource allocation is dynamically, in response to actual usage and needs. With a virtual machine, unlike a physical server, it can't just be set running and left in a cupboard for months on end - a close eye needs to be kept on it.

There's no foolproof way to ensure every VM starts out with optimal resources.

But, thanks to the multiple layers of abstraction created in the process of virtualising, it is almost impossible to understand the true resource usage of any VM using the same tools and approaches that are used in physical environments. To find the right balance between efficiency and performance, monitoring and reporting tools specifically designed for managing virtual environments are needed.

Because the rules of virtual environments are so different, it's also important to choose monitoring tools that give recommendations, not just reports. What is required is not just one more report, but an expert system that contains a lot of the specialist knowledge any organisation running a virtual environment needs.

Companies also need a system that can identify problems affecting specific servers, departments or applications. Throwing more RAM at a slow machine, for example, isn't always the right solution. What if the real problem is disk related, or an application with a memory leak? Monitoring tools need to supply that information - in the specific context of a virtualised environment with over-committed resources.

Then, of course, in addition to monitoring and reporting, a management tool is needed that will allow the company to manage its VMs dynamically, allocating and taking away resources according to their needs.

These needs may change from month-to-month, or even day-to-day, which is one of the reasons the whole job can't be outsourced to a software system. Human decision-making is still paramount, because humans know things about context and the future that machines can't.

For example, if a monitoring tool notices that a particular server has been under-utilised for six months, it may recommend switching resources away from that machine. The human who knows that the company's financial year-end is coming up in one months' time will also know not to implement that particular recommendation.

Between an IT manager's knowledge of the business context, and a good monitoring tool's knowledge of the virtual environment, it is possible to run a virtualised IT shop that is truly dynamic and responsive to the changing needs of the organisation. This will help to deliver both better performance and lower cost, in turn, helping the IT department to meet ROI targets set by the business.

Once companies are able to monitor and manage VM resource allocation seamlessly, they'll be well on the way to having a right-sized virtual environment. The next step is long-term capacity planning - look out for the third Industry Insight to find out more.