Bonnie++: sequential I/O bandwidth
The bonnie++ suite enable us to do some very interesting measurements. The first graph show the sequential I/O bandwidth and CPU usage using a single character at a time:
As you can see, the absolute values are quite low (~1600 KB/sec max) but this is expected: hard disks are block devices and so they are very inefficient in this kind of test. Moreover, issuing only one character at a time the CPU is very busy. This means that this test actually measure the filesystem efficiency rather than raw I/O bandwidth. You can see that BTRFS is by large the worse here, as it can transfer only ~200 KB/sec with a CPU load of around 100%. In contrast, EXT3 and EXT4 are much faster in the input test but only slight better in the output test. XFS seems the best balanced here: its input speed is only a little behind EXT3/4 rate, while scoring 2X better in the output test.
However, hard disk are block devices: this means that the block I/O test is way more meaningful:
Here we can see that the absolute values are quite well matched, depending on hard disk performance other than filesystem structure. However, the CPU load data indicate that not all filesystems are equally efficient: XFS seems the best here, followed by EXT4, then EXT3 and finally by BTRFS. While the differences are not so big (we are speaking about some % points), they are interesting indeed.
Now, let's have a look at direct data transfer (eg: without any OS/FS caching scheme):
This graph seems to indicate that for application requiring direct access to filesystem, bypassing the OS/FS cache (eg: video recording, some virtual machine software, ecc.), EXT3/4 are the best choice, followed by BTRFS and, at last place, XFS.
This first round of tests prove that we don't have a silver bullet in this arena: we don't have an always-better filesystem. It all boil down to your usage pattern. However, it seems that BTRFS can be a little more problematic to profile.
But let's examine some other bonnie++ tests...
UPDATE: preparing the system for another benchmark, I noticed that, in contrast to what written in Fedora 14 documentation, write barriers were non enabled on EXT3 filesystem. Please read the updated "Conclusions" page.