APU vs TLSense CPU performance comparison

This article has been updated on August 6, 2020.

If you plan to run a VPN server/client on your router or IPS/IDS software, you may be interested in exact CPU performance before making a purchase. For that reason, we have executed a CPU benchmark on all routers offered in the store. 

Methodology

All tests have been executed using sysbench 1.0.18 and openssl 1.1.1d on the Debian 10 4.19.0-9-amd64 system.  

The following commands have been executed to measure raw CPU performance. Note, we measure 1 thread, 2 threads, and 4 threads.

sysbench --test=cpu --cpu-max-prime=50000 --time=30 run --threads=1
sysbench --test=cpu --cpu-max-prime=50000 --time=30 run --threads=2
sysbench --test=cpu --cpu-max-prime=50000 --time=30 run --threads=4

And these commands were executed to measure the encryption/decryption performance.

openssl speed -elapsed -evp aes-128-cbc 
openssl speed -elapsed -evp aes-256-cbc 
openssl speed -elapsed -evp aes-128-gcm 
openssl speed -elapsed -evp aes-256-gcm 

Data

The raw results are available in the text file, but that's a little dry to read and analyze so here's a read-only google spreadsheet with the data and charts https://docs.google.com/spreadsheets/d/1kbNbFivVorgUQsrLl_fVUkrCqesYnC0Hm8hY-Ft7n8s/edit?usp=sharing

Single-core CPU performance

The single-thread performance is essential for applications that can't take advantage of multiple CPU cores. For example Snort is single-threaded. 

BSD operating systems (pfSense and OPNsense) use one CPU core per TCP connection. Linux-based operating systems such as OpenWRT or IPFire don't have that limitation.

i3-4010U is just a little faster than APU. i5-4510U has roughly double single-thread performance of APU in single-core.

It's interesting that the most powerful router we sell is only about 4x more performant than APU in a single-thread CPU test.

Multi-core CPU performance

Multiple-core performance is important for applications that are multi-threaded such as Suricata or Wireguard VPN. As mentioned earlier, it also helps with routing multiple TCP connections concurrently. 
 

CPU performance summary

The results speak for themselves. i5-4210U is roughly 80-100% more performant than APU. i7-6500U is approximately 130-180% more performant. 

I was a little surprised about how good APU routers compare to the more expensive and more power-consuming CPUs. More about this in summary.

Encryption/Decryption performance 

All CPUs have AES-NI support, so the encryption/decryption should be relatively fast. Let's compare two popular algorithms AES-CBC and AES-GCM.

AES-CBC performance comparison

The AES-CBC performance follows roughly the CPU performance from the previous chart. No big surprises here.

TLSense i5-4200U is roughly twice as performant as APU with AES-CBC. i5-6200U is approximately 2 times more performant than i5-4210U. 

The performance scales roughly with the price.

AES-CBC is an older, less performant algorithm. It's also less secure than AES-GCM, so we should not pay too much attention to these numbers, as everyone is encouraged to use AES-GCM these days.

AES-GCM performance comparison

I expected the AES-GCM test to be proportional to the CPU test, but that's not the case. Intel CPUs have much better AES-GCM acceleration than APU. 
 

The data clearly shows that TLSense routers are much better at VPN and other cryptographic operations than APU.

APU2 can achieve roughly 140Mbit/s throughput on OpenWRT with AES-128-GCM.

Based on this chart, we can estimate that i5-4210U can achieve up to ~650Mbit/s, and i7-6500U can run at full Gigabit. 

Power consumption comparison

Power consumption is an important factor for devices that are always-on 24/7/365. 

All routers offered by us have efficient, low-power CPUs.

APU consumes 6W while other routers consume 15W of electricity. APU is 2.5 more energy efficient. 

i5-7500F has a desktop-class CPU which makes it more energy hungry. 

Assuming that the router is operating around the clock, Passively cooled TLSense devices will consume 131.4kWh/year while APU consumes 52.56kWj/year. 
 

Summary

APU routers have less horsepower than TLSense routers, but that's not surprising. 

With the power consumption and price in mind, APU routers compare very well for every-day routing needs. Most users don't run a VPN client or IPS/IDS software on the router. For those users, APU will perform just as well as TLSense.

IPS/IDS and VPN are where the TLSense routers shine. Here the performance is 400-800% better than APU.