EXT3 vs EXT4 vs XFS vs BTRFS linux filesystems benchmark

Written by Gionatan Danti on . Posted in Linux & Unix

User Rating:  / 128
PoorBest 

Conclusions

If you had the patience to read this long article, I'm sure that you agree with me if I say that filesystems benchmarking is not an easy task. There are many, many variables at work here; to remember you only the greatest:

  • different kernel release can produce dramatically different results;

  • different usage patterns favor different filesystems;

  • different filesystem options can have a large (positive or negative) impact on its performance in specific test.

So, with these caveats in mind, it is possible to draw some meaningful conclusions and give you some educated hints? Fortunately, yes.

Let's start from the most obvious: the best balanced filesystem seems to be the mature, almost aging EXT3. UPDATE: EXT3 was incorrecty benchmarked with barrier off. Please read the note at the end of this page. This is natural, as it received most cumulative improvements over a long period of time. It has very good sequential and random write speeds and reasonable read speed, factors that are of utmost importance on several different tasks. For example, if you plan to run a database server you are almost forced to use EXT3, as all other filesystems seems to have big problems with synchronized random write speed. Also, you can't go wrong with EXT3 if you use it on your workstation as its performances are quite good in a great amount of different jobs. Finally, EXT3 is more stable than the others FS as most of its bug are by now already worked out.

However, this not means that EXT3 is the perfect FS: first, it that lacks some important features as delayed allocation and online compression. It lacks native snapshots capability also but you can use LVM to overcome this. It is more fragmentation-prone that EXT4 and XFS and it is very slow in creating/deleting large amount of files, denoting a not-so-good metadata handling. Moreover, it use more CPU cycles than EXT4 and XFS, but with todays CPU I don't think that this is a great problem. If you can live with these minor faults, EXT3 is the right filesystem for you.

How about its successor, the EXT4 filesystem? It is an important evolutionary step, with welcomed featured as delayed allocation, extents and much more efficient metadata handling.

However, its performances are not so stellar: especially in the synchronized write and database tests it fall behind EXT3 by a great margin. UPDATE: EXT3 showed faster results because it was benchmarked with write barrier off. Please read the note at the end of this page. Moreover, as it is a younger filesystem, it is often touched by patchs that, while correcting a problem, can cause a speed regression. While you can try it on your workstation, I suggest you to let it slowly mature and not use it on servers right now. Especially if you plan to use application that bypass OS/FS cache and write directly on disk (video recording, some virtual machine, ecc.) do not use it unless you have at lest the 2.6.36 kernel version, as the async I/O layer has a critical bug on previous kernels. You can read more at https://bugzilla.kernel.org/show_bug.cgi?id=16165

Speaking about XFS, I can say that it is a very good FS, with great read speed, good metadata handling and very good scalability.

However, while it seems better than EXT4 overall, it share with the latter some serious problem: the slow random write speed and the same async I/O layer bug (which should be corrected on 2.6.36 kernel). This means that I discourage its use on database servers (for performance reasons) and on servers hosting virtual machines (for the slow random write and the async I/O bug). However, XFS should be great for the average workstation.

BTRFS is a new, next-generation filesystem and, while its look very good from a features standpoint (it has online compression and native snapshot capabilities!) it is too slow and with too many “spiked edge” to be used on anything other than a test machine. I am sure that in the future this filesystem will grow very well but we are not at a point where it is worth considering for regular use.

UPDATE: preparing the system for another benchmark, I noticed that, in contrast to what written in Fedora 14 documentation (http://docs.fedoraproject.org/en-US/Fedora/14/html-single/Storage_Administration_Guide/index.html#writebarrieronoff) , write barriers were not enabled by default on EXT3 filesystem, while EXT4, XFS and BTRFS have barriers correctly enabled by default. Write barriers are a mechanism to ensure that, after and fsync(), all your data are really stored on a disk platter. Without barriers you risk that some fsynched data are not really stored to a disk platter, but that they are temporarily stored on a volatile disk cache (DRAM). So, while EXT3 is very fast in some benchmarks, it can be more prone to data loss under certain circumstances (eg: sudden power loss) for fsynced data too. Please keep this in mind while examining my results. I'm sorry to notice this thing only now, but in this case Fedora documentation was really misleading...

If you want to talk with me about this article and/or the testing methods, please contact me using the email address This email address is being protected from spambots. You need JavaScript enabled to view it.

Have a nice day!

 

References:

[1] http://en.wikipedia.org/wiki/Comparison_of_file_systems

[2] http://www.kernel.org/pub/Linux/kernel/v2.6/Linux-2.6.36.tar.bz2

Comments   

 
#1 Max 2012-08-26 15:49
Thank you for this very interesting article. I'll go with ext3, mostly because of your verdict for it being a good and balanced filesystem.
 
 
#2 Gionatan Danti 2012-08-26 16:11
Hi Max,
I'm happy that you found my article interesting.

However please note that, as you can read in the updates, EXT3 was incorrectly benchmarked with write barrier off, while all other filesystems used barrier on.

This means that EXT3 data were somewhat flawed. For up-to-date data, please read my latest article here: http://www.ilsistemista.net/index.php/linux-a-unix/33-btrfs-vs-ext3-vs-ext4-vs-xfs-performance-on-fedora-17.html

Thanks.
 
 
#3 Kuba 2012-10-09 15:01
Hi Gionatan,

Could you please let me know what sysbench parameter you have used to perform these tests?


Thanks,
Jakub
 
 
#4 http:// 2014-03-08 09:57
I believe what you published was very reasonable. But,
what about this? suppose you typed a catchier post
title? I mean, I don't want to tell you how to run your website, however suppose you added something to possibly get people's attention?
I mean EXT3 vs EXT4 vs XFS vs BTRFS linux filesystems benchmark is a little vanilla.

You ought to glance at Yahoo's home page and see how they write
article headlines to get people to open the links.

You might add a video or a pic or two to get readers excited about everything've got to say.
Just my opinion, it could bring your website a little bit more interesting.
 

You have no rights to post comments