Cyberoam CR25ia UTM firewall review - throughput and latecy examined

Written by Gionatan Danti on . Posted in Hardware analysis

User Rating:  / 2
PoorBest 

Throughput inspection

To measure large packets and streaming speeds, I use two very useful tools: netperf and wget. Lets see CR25ia results with various features enabled:

Cyberoam CR25ia performance

TCP_SENDFILE, TCP_STREAM, and UDP_STREAM are upload-bond tests, as it is the client machine that send data to the server using various methods and underlying protocols (eg: TCP vs UDP).

We can see that, while using only stateful inspection, we get about 250 Mbit/sec TCP performance and over 950 Mb/sec UDP performance, exceeding Cyberoam own expectations (especially on UDP traffic). Enabling UTM comports a pronounced speed drop, with TCP throughput in the range of 24-34 Mb/sec. These values, while good for a firewall of this category, are considerably lower then Cyberoam-advertised 50 Mb/sec. 

A note on UDP tests: realizing that, due to packet buffering, UDPSTREAM scores can be skewed on some UTM-enabled firewalls, I decided to report UDP scores for “firewall only” scenarios.

When working inside a VPN, maximum (non-UTM) TCP AES throughput is at ~43 Mb/s, while 3DES one is at ~19 Mb/sec. Again, these values are considerably lower then stated, while UDP scores are a little better. Adding UTM to the picture obviously led to even lower score, with ~14/10 Mb/s throughput for AES and 3DES traffic respectively.

What about the wget (http and ftp) results? Lets see:

Cyberoam CR25ia performance

Wget shown us a similar picture: note the excellent firewall speed (over 30 MB/sec = >240 Mbit/sec) and the much lower but respectable AES128 speed (~6 MB/s = about 45 Mbit/s). On the other hand, 3DES performance is much slower, at about 2.5 MB/sec (~20 Mbit/sec).

Until now, wget and netperf results seems to go hand in hand, but note how UTM performances differ between FTP and HTTP traffic: the former is at about 3 MB/sec (24 Mb/sec) and perfectly mimics netperf TCP score, while the latter is considerably higher at about 7.5 MB/sec (~60 Mb/sec) exceeding Cyberoam-stated 50 Mb/s UTM speed. It seems that for UTM on HTTP traffic we have a fast device here.

UTM + VPN scores are obviously lower, at 1-1.5 MB/sec (12-16 Mb/sec) for 3DES/AES traffic respectively.

Please note that, on throughput tests, it seems that the “firewall only” vs “UTM started but not enabled” scenarios are mostly the same. However, as we soon find, the latency benchmarks will show some differences.

Comments   

 
#1 cmoegele 2012-09-10 07:10
I´am so glad to find your site after googling since month for good research on such Hardware performance.
Thank you for your work!
 
 
#2 blog 2016-07-05 17:47
I absolutely love your blog and find most of your post's to be just what I'm looking for.
can you offer guest writers to write content in your case?
I wouldn't mind creating a post or elaborating on many of the subjects you write regarding here.
Again, awesome web log!
 

You have no rights to post comments