Conclusions
If you had the patience to read this long article, I'm sure that you agree with me if I say that filesystems benchmarking is not an easy task. There are many, many variables at work here; to remember you only the greatest:
-
different kernel release can produce dramatically different results;
-
different usage patterns favor different filesystems;
-
different filesystem options can have a large (positive or negative) impact on its performance in specific test.
So, with these caveats in mind, it is possible to draw some meaningful conclusions and give you some educated hints? Fortunately, yes.
Let's start from the most obvious: the best balanced filesystem seems to be the mature, almost aging EXT3. UPDATE: EXT3 was incorrecty benchmarked with barrier off. Please read the note at the end of this page. This is natural, as it received most cumulative improvements over a long period of time. It has very good sequential and random write speeds and reasonable read speed, factors that are of utmost importance on several different tasks. For example, if you plan to run a database server you are almost forced to use EXT3, as all other filesystems seems to have big problems with synchronized random write speed. Also, you can't go wrong with EXT3 if you use it on your workstation as its performances are quite good in a great amount of different jobs. Finally, EXT3 is more stable than the others FS as most of its bug are by now already worked out.
However, this not means that EXT3 is the perfect FS: first, it that lacks some important features as delayed allocation and online compression. It lacks native snapshots capability also but you can use LVM to overcome this. It is more fragmentation-prone that EXT4 and XFS and it is very slow in creating/deleting large amount of files, denoting a not-so-good metadata handling. Moreover, it use more CPU cycles than EXT4 and XFS, but with todays CPU I don't think that this is a great problem. If you can live with these minor faults, EXT3 is the right filesystem for you.
How about its successor, the EXT4 filesystem? It is an important evolutionary step, with welcomed featured as delayed allocation, extents and much more efficient metadata handling.
However, its performances are not so stellar: especially in the synchronized write and database tests it fall behind EXT3 by a great margin. UPDATE: EXT3 showed faster results because it was benchmarked with write barrier off. Please read the note at the end of this page. Moreover, as it is a younger filesystem, it is often touched by patchs that, while correcting a problem, can cause a speed regression. While you can try it on your workstation, I suggest you to let it slowly mature and not use it on servers right now. Especially if you plan to use application that bypass OS/FS cache and write directly on disk (video recording, some virtual machine, ecc.) do not use it unless you have at lest the 2.6.36 kernel version, as the async I/O layer has a critical bug on previous kernels. You can read more at https://bugzilla.kernel.org/show_bug.cgi?id=16165
Speaking about XFS, I can say that it is a very good FS, with great read speed, good metadata handling and very good scalability.
However, while it seems better than EXT4 overall, it share with the latter some serious problem: the slow random write speed and the same async I/O layer bug (which should be corrected on 2.6.36 kernel). This means that I discourage its use on database servers (for performance reasons) and on servers hosting virtual machines (for the slow random write and the async I/O bug). However, XFS should be great for the average workstation.
BTRFS is a new, next-generation filesystem and, while its look very good from a features standpoint (it has online compression and native snapshot capabilities!) it is too slow and with too many “spiked edge” to be used on anything other than a test machine. I am sure that in the future this filesystem will grow very well but we are not at a point where it is worth considering for regular use.
UPDATE: preparing the system for another benchmark, I noticed that, in contrast to what written in Fedora 14 documentation (http://docs.fedoraproject.org/en-US/Fedora/14/html-single/Storage_Administration_Guide/index.html#writebarrieronoff) , write barriers were not enabled by default on EXT3 filesystem, while EXT4, XFS and BTRFS have barriers correctly enabled by default. Write barriers are a mechanism to ensure that, after and fsync(), all your data are really stored on a disk platter. Without barriers you risk that some fsynched data are not really stored to a disk platter, but that they are temporarily stored on a volatile disk cache (DRAM). So, while EXT3 is very fast in some benchmarks, it can be more prone to data loss under certain circumstances (eg: sudden power loss) for fsynced data too. Please keep this in mind while examining my results. I'm sorry to notice this thing only now, but in this case Fedora documentation was really misleading...
If you want to talk with me about this article and/or the testing methods, please contact me using the email address
Have a nice day!
References:
[1] http://en.wikipedia.org/wiki/Comparison_of_file_systems
[2] http://www.kernel.org/pub/Linux/kernel/v2.6/Linux-2.6.36.tar.bz2