EXT4 vs XFS: large volumes with high-end RAID controller

Written by Gionatan Danti on . Posted in Linux & Unix

User Rating:  / 7
PoorBest 

Testbed and methods

The Dell R510 have the following hardware and software configuration:

  • 2x Intel Xeon E5620 with HT OFF (4 cores, 4 threads, 12 MB L3 cache) @ 2.4 GHz

  • 16 GB DDR3 RAM (32 GB total RAM)

  • PERC H700 RAID Controller with 1024 MB on-board cache

  • 12x 2 TB 7.2K RPM SATA 3Gps disks

  • Red Hat Enterprise Linux 6.0 64 bit

 

The 12 disks were assigned to 2 RAID array:

  • a first, 2 disks RAID 1 array for OS installation

  • a second, 10 disks RAID 6 array for the benchmark runs

 

The benchmarked filesystems were created by issuing these commands:

  • EXT4: mkfs.ext4 /dev/sdb1 -E stride=16,stride-width=128

  • XFS: mkfs.xfs /dev/sdb1 -d su=64k,sw=8

 

To run the benchmarks, I used the following softwares:

  • bonnie++-1.96-1.el6.rf.x86_64.rpm

  • sysbench-0.4.12-1.el6.x86_64.rpm

  • mysql-server-5.1.52-1.el6_0.1.x86_64.rpm

  • mysql-bench-5.1.52-1.el6_0.1.x86_64.rpm

  • postgresql-server-8.4.7-1.el6_0.1.x86_64.rpm

  • postgresql-test-8.4.7-1.el6_0.1.x86_64.rpm

 

Please note that the benchmarked filesystems were optimized for the physical array layout (in this case, 8active data disks and 64 KB stripe size). Remember that, as stated before, the PERC H700 controller has a battery backupped cache, so write barriers were disabled (for a more in-depth discussion about write barriers, please read here: http://www.ilsistemista.net/index.php/linux-a-unix/13-ext4-vs-xfs-large-volumes-with-low-end-raid-controller.html?start=1 )

For the records, I run the entire benchmark suite with write barriers also, but I didn't register any significant performance drop, with two important exceptions: PgBench scores dip as much as ~60%. Why?

First, remember that write barriers are used to guarantee that data were actually written to the disk platters. This kind of guarantee is mainly (but not exclusively) needed when the OS and applications issue fsynched I/O requests, a thing that PostgreSQL does massively.

Second, as the controller has a write-back battery backupped cache, it can theoretically masks write barrier commands and prevents them to propagate to the physical hard-disks. In this manner, write barriers are generally “cached” at the controller level and do not generate any performance drop. However, if the cache fills up, write barriers commands eventually begin to hit the disks, lowering performance. This is what probably happens with PgBench results.

I run each benchmark at least 3 times and then reported the mean value.

A note on the CPU load number: as this Dell R510 has 8 physical processors that can manage 8 hardware threads (HyperThreading was set to OFF), the maximum CPU load percentage, as reported by the Linux kernel, is 800%. So, if when you read something similar to “100% CPU load”, this mean that, on average, only one core (from the 8 available) was fully utilized.

Comments   

 
#1 Evgeny 2012-12-10 07:21
it's looks that you fsck time ext4 isn't true.
I think that you ran fsck.ext4 /dev/sd__something__
without "-f" key. It's means that fsck FS state
tune2fs -l
...
Filesystem state: clean
...
and if it clean do nothing
 

You have no rights to post comments