A Comprehensive Raspberry Pi 3 Benchmark. Part 1

Christopher Stanton, element14

Our good friend biglesp put the Raspberry Pi 2 to the test the last time around, though I feel that he missed a good number of possible benchmarks to run on the hardware. With that in mind, I'm going to test the main boards of the Raspberry Pi that I can get my hands on and not only run the main benchmarks Les ran, but also a good few more thanks to Roy Longbottom's benchmark collection. I also have some approximate information on idle power usage, which everyone appears to be interested in.

One technical capability that I would have loved to put through its paces with Roy's benchmarks is the OpenGL driver that the Pi Foundation have made available. Unfortunately the Roy benchmarks are currently not compatible with it, so I guess it has just been testing against the binary blob for now. Regardless, the VideoCore is one part of the ARM processor that practically has not changed from the early Model B to the most recent version of the Raspberry Pi.

Figure 1.	Raspberry Pi 3 Model B with quad core A53 processor.

It does not hurt to go over the versions of Raspberry Pi again, and here are the models that I tested (Table 1).

Table 1. Raspberry Pi boards for tests.

Board Raspberry Pi 3 Model B Raspberry Pi 2 Model B Raspberry Pi Model B+ Processor Chipset BCM 2837 64bit ARMv8 Cortex A53 Quad Core BCM 2836 32bit ARMv7 Cortex A7 Quad Core BCM2835 32bit ARMv11 Single Core Processor Speed 1.2 Ghz per core 900 Mhz per core 700 Mhz per core Max Power Draw 2.5 A 1.8 A 1.8 A

If you want more information on the specifications you can check the comparison chart linked in the Raspberry Pi space. What we're mainly concerned with is the processor, power usage and somewhat also RAM, however the RAM runs at 400Mhz for each board so that may make little difference.

What does make somewhat of a difference is that the Raspberry Pi 3 (Figure 1) is an ARMv8 Processor with 64bit support, however, Raspbian at the time of writing does not have 64bit support. In part this is so that it is backwards compatible with the previous Raspberry Pi. There is yet to be a 64bit Repo. Debian has been building up its 64bit support over a period of years, and potentially someone could compile together a kernel and binaries for a 64bit Jessie Debian for the Raspberry Pi 3. Go on, accept the challenge!

Taste Testing the Pi

Before I get into the software used for the benchmarking, I decided to run a few commands to get the processor information and supported instructions (Table 2).

Table 2. cpuinfo command - processor information and supported instructions.

cpuinfo Board: Raspberry Pi 3 Model B Raspberry Pi 2 Model B Raspberry Pi Model B+ Processor: 0 0 0 model name: ARMv7 processor rev 4 (v7l)   ARMv7 Processor rev 5 (v7l) ARMv6- compatible processor rev 7 (v6l) BogoMIPS: 76.80 57.60 2.00 Features: half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32     half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm half thumb fastmult vfp edsp java tls CPU implementer: 0x41 0x41 0x41 CPU architecture: 7 7 7 CPU variant: 0x0 0x0 0x0 CPU part: 0xd03 0xc07 0xb76   CPU revision: 4 5 7   <... and so on for each core...> <... and so on for each core...>   Hardware: BCM2709 BCM2709 BCM2708 Revision: a02082 a01041 0010 Serial: 0000000056163283 000000008e511ef0 000000007a4bc337

What is pretty clear with cpuinfo is that it identifies the Raspberry Pi 3 processor as an ARMv7, this appears to be a failure in the software for now but this may change with a kernel/firmware update to ARMv8. What is interesting is that we have a new feature: crc32 (Table 3).

Table 3. lscpu command - processor information.

lscpu Board: Raspberry Pi 3 Model B Raspberry Pi 2 Model B Raspberry Pi Model B+ Architecture: armv7l armv7l armv6l Byte Order: Little Endian Little Endian Little Endian CPU(s): 4 4 1 On-line CPU(s) list:   0-3 0-3 0 Thread(s) per core: 1 1 1 Core(s) per socket: 4 4 1 Socket(s): 1 1 1 Model name: ARMv7 Processor rev 4 (v7l) ARMv7 Processor rev 5 (v7l) ARMv6-compatible processor rev 7 (v6l) CPU max MHz: 1200.0000 900.000 700.0000   CPU min MHz: 600.0000 600.000 700.0000

With the lscpu command we can see that the Pi 3 processor is perhaps still not being identified properly, but at least that it, like the Pi 2 before it, can perform cpufreq scaling (which is also shown when issuing the lshw command). We can also clearly see that we have a confirmed, rather gladly, four core processor with the Pi 2 and Pi 3. Phew!

If you want to run these commands yourself on your Raspberry Pi with Raspbian then you can make sure you have them installed thusly:

sudo apt-get update
sudo apt-get install lscpu cpuinfo lshw

From a terminal window either within your desktop environment or from pressing CTRL-ALT-F1 (to F7, typically).

Power Usage

You can find some strange little devices (Figure 2) that will plug in line with your USB hardware and it will tell you the most useful of things, such as how much power is being drawn! So I thought "sweet! Let's see how much power these draw while idle!" (Table 4).

Figure 2.	USB-voltmeter for measuring Raspberry Pi Power Usage.

Table 4. Raspberry Pi Power Usage.

Raspberry Pi 3 Model B Raspberry Pi 2 Model B Raspberry Pi Model B+ 5.19 V 1.141 W 5.19 V 1.141 W 5.19 V 1.141 W 0.22 А 00006 mAh 0.20 А 00102 mAh 0.19 А 00003 mAh

Let me give these values some context, each Raspberry Pi was set to boot to the terminal, so that the X windows environment was not running. The only devices connected were a HDMI to DVI adapter to a 19" Widescreen monitor, a Dell USB keyboard, a 16gByte Class 10 microSD card and the power supply, which was providing 5 Volts, 2 Amps. There was no ethernet cable plugged in (though I can note that when it was, the power usage went up in all cases). We can see from the charts that the Pi 3 is pulling more power at idle than its predecessors, it is likely worth noting that the on board WiFi adapter of the Pi 3 was active, though it was not associated with an access point. The Bluetooth 4.1 adapter status was not intentionally active, as there was no driver loaded or software to use it. So it is likely the increase in power usage is due to the WiFi chip.

SysBench

Now this software has been around since 2004, it was originally intended for input/output (io) file operations and database benchmarking. Thanks to being open source it gradually developed into an almost all-round system benchmark which also includes aspects of processor testing as well as IO and databases.

SysBench's processor tests verify prime numbers by going through all possible divisions and only being satisfied when the result is zero. This does mean that it does not test all features of the processor, except for raw number crunching. SysBench was ran with the following parameters:

sysbench --num-threads=1 --test=cpu --cpu-max-prime=20000 --validate run (Table 5)
sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 --validate run (Table 6)

Here is a breakdown of the command line parameters:

• sysbench - The name of the software to run
• --num-threads - This is the number of processes to run, in the tests we run 1 thread and then 4 threads, this means that it will create 1 or 4 processes and run one process per core. Since the Raspberry Pi 1 Model B+ has one core it made sense to run a one core test on each model of Raspberry Pi for fairness alongside running 4 threads.
• --test=cpu - This parameter ensures we are only testing the processor, as mentioned previously SysBench can perform other tests, too
• --cpu-max-prime - This is the maximum prime number value we want to calculate up to.
• --validate - This ensures that the results we have returned are valid
• run - The software can emulate or test rather than actually perform the requested benchmark, so we want to tell it to actually run it.

Table 5. sysbench with 1 thread.

Board: Raspberry Pi 3 Model B Raspberry Pi 2 Model B Raspberry Pi Model B+ Total time:  477.0617 s 768.6476 s 1318.933 s per request statistics:   min. 47.69 ms 76.42 ms 131.59 ms avg. 47.7 ms 76.86 ms  131.89 ms max. 49.91 ms 82.15 ms  300.23 ms diff between min and max: 2.22 ms 5.73 ms 168.64 ms

With the Pi 3 versus the Pi 1, the results show about a 94% total speed difference, compared to the Pi 2 there is a 47% speed difference on the total time taken. What I find most interesting about these results, though, is the difference between the minimum and maximum time taken. The Pi 3 shows a 194% speed improvement over the B+ and an 88% speed improvement over the Pi 2. So while the overall time may not feel that big of a deal, per request it is performing them a lot faster. There's also evidence that it is overall significantly faster than the B+.

Table 6. sysbench with 4 threads.

Board: Raspberry Pi 3 Model B Raspberry Pi 2 Model B Raspberry Pi Model B+ Total time:  119.4716 s 191.8972 s 1321.493 s per request statistics:   min. 47.69 ms 76.38 ms 412.94 ms avg. 47.78 ms 76.74 ms 528.54 ms max. 59.04 ms 101.45 ms 573.00 ms diff between min and max: 11.35 ms 25.07 ms 168.64 ms

This test is an unfair comparison with the Pi 1 B+ since it only has one processor core, so it is not surprising that it takes significantly longer than the Pi 2 and 3. The Pi 2 closes the gap a bit closer when working with all four cores here than with the single core test though the Pi 3 still outperforms by 75% with the difference between the min and max time it takes for the requests, overall there is still only a 47% speed difference which is almost half.

Overall the Pi 3 comes out on top as faster, though it would be interesting to see how the Pi 2 compares if it was over clocked. Still, something to remember is that computing prime numbers is just one function of what this processor is capable of.

To be continued