Cache and Memory subsystems
Current systems can be memory-performance limited in a number of situations, so it is crucial to show the cache and memory subsystems performance both in term of bandwidth and latency.
To collect these data, I used Sandra Cache and Memory tests.
First, let see the L1/L2/L3 cache bandwidth results:
While VMware, VirtualBox and KVM give us quite similar in results, Xen is behind the competitors. This can be a results of a more-heavy hypervisor or, more probably, of the “double encapsulation” of the guest system.
Now, it's time for memory bandwidth results:
The story is reversed now: Xen is slight faster, followed by KVM, VMware and, finally, by VirtualBox (which lag considerably behind the other). It seems that Xen do a very good use of the nested page table feature.
Other than bandwidth, a very important parameter is latency. Let see the latency data for caches first:
The results are quite similar, but VMware seems a bit more slow while fetching data from L2 cache.
Now the memory latency results:
The results are very close, with VMware at the slowest end and Xen at the fastest end.
Please consider that repeating the test many times, I often obtained quite different results. While it is interesting to analyze these data, many factor are at play here. For example, the slower show of VMware can be simply due to some host-side work that the system was doing in that precise moment. At the other end, for the same reason, it can be that Xen has the potential to be faster than it showed here.
However, all in all, Xen seems to be the fastest hypervisor in these memory related tests.
UPDATE: a recent article comparing KVM vs VirtualBox can be found here: http://www.ilsistemista.net/index.php/virtualization/12-kvm-vs-virtualbox-40-on-rhel-6.html