Virtualization In Software Development

The Limits of Virtualization

There are, developers acknowledge, part of the process where virtualization can’t help.

“One area where virtualization doesn’t help is in stress testing our product, which is a network-based backup appliance,” notes 3X’s Floehr. “While we could test 10 simultaneous clients virtually, it isn’t a true stress test because the 10 clients are only going through, say, four physical CPUs and two NICs, which are constraints that would not exist if they were 10 physical clients in the real world.”

“Virtualization does add some complexity,” adds MarkMail’s Mitchell. “It always adds some latency. But, even worse, you can’t rely on VMs to give a clear view of the performance characteristics as you change the underlying execution environment, such as going from development and testing into production.”

Finally, Mitchell notes that “The current generation of desktop virtualizers still have too many annoying quirks, bugs and missing features,” at least with regard to their use by developers for software development.

“For load testing or scaling, you have to run on the raw hardware, the physical machine,” says Microsoft’s Mark Friedman. “Performance characteristics, unless you expect to deploy on a VM, are very different. And the same applies to the state of the art for performance measurements; you can’t trust the [Physical Machines] you get on a VM currently.” Although, Friedman notes, over time he expects this to be fixed.

Also, says Friedman, the para-virtualization approach used by the virtualization industry can change some does change some underlying aspects of the OS, notably the OS drivers. “So if you have a dependency on specific drivers or hardware environment, that will be impacted. For example, there’s a virtual NIC, and there’s a specific driver. The driver is ‘virtualization-aware.’ You have to understand that if you need testing against a native set of drivers, you won’t see that within a VM. So this impacts people doing driver development, but not web developers, etc.”

And, Friedman points out, “if you have a multi-tiered application, which requires several machines, consolidating these to shared hardware means “you get a functional but not a performance test.”

Despite those quibbles, virtualization already gives developers real benefits. “Virtualization makes our developers more productive once they learn to work with the technology,” says Microsoft’s Friedman. “It allows them to spend more of their time in coding, testing and debugging. It eliminates a lot of time and delay spent in set-up and preparation, which isn’t where we want them to be spending the bulk of their time.”