Mikrotik RouterBOARD 750 GL review - throughput and latency examined

Testing methodologies

Characterize a firewall's performances, as usual with benchmarking, is not an easy task: you must account for different workloads and different possible configurations that, in turn, cause very different loads on the device. The various corners of network performance can be described by measuring quite different things, nominally:

• small & large packets throughput with simple stateful inspection

• small & large packets throughput with VPN encryption (3DES and AES, with SHA1 and DH/PFS Group2)

• small & large packets throughput with advanced filter enabled (eg: layer7 pattern matching)

• connections creation and management speed

• added latency for packets traversing the device

To correctly measure such things, I developed a simple script-driven benchmark that use the following programs:

• Netperf and wget, to test large packets and streaming throughput

• Ping, to test large / big / fragmented packets handling and latency

• Apache Benchmark, to test connections creation speed

To clearly remark how different conditions can lead to very different results, I run the tests with the following different configurations:

1. a no-firewall, switch-only case (I run the bench without the RB750GL)

2. a very simple, one accept rule firewall config

3. a little more complex setup, with others 10 firewall rules added before the previous simple accept rule (in this manner we can quantify the added penalty for rules traversing by the firewall engine)

4. a simple configuration similar to #2, but with one added string matching rule to deny a specific keyword (“facebook”), similar in concept to url-filtering

5. a simple configuration similar to #2, but with one added regular expression matching rule to deny a specific regexp (“facebook|twitter”), similar in concept to advanced url-filtering

6. a simple configuration similarto #2, but with both client and server machine behind a IPSec VPN between two RB750GL (for measuring VPN throughput and latency penalty)

I distinguished between string and regular expression matching because they can give very different results: while string matching is fundamentally simpler, it isdone against any packet traversing the string matching rule. On the other hand, a regular expression match is considerably heavier but it is applied to only the first 2 KB of each connection traversing the affected rule.

All tests and benchmarks where run between two laptop machine: a Dell E6410 client machine (Core i5-520M, 4 GB RAM and Gigabit Ethernet), running Debian 6 amd64, and a Dell D620 (Core2 T7200, 4 GB RAM and Gigabit Ethernet) server machine, running Red Hat Linux 6 amd64.

The entire small network was working at Gigabit speed.