Poor man's benchmark

Hi,

Can anybody with a Mele A2000 or any other A10 can run this command line :

time echo "scale=2000; 4*a(1)" | bc -l

It's just an easy (and not very precise) way to benchmark the CPU.

For the record, here is the results for my Dockstar and my Raspberry Pi :

Dockstar :
real 0m13.814s
user 0m13.770s
sys 0m0.010s

Raspberry Pi
real 0m20.451s
user 0m20.400s
sys 0m0.010s

I expect A10 result to be at least 3 to 4 times faster than Dockstar.

I already did a benchmark comparison of Kirkwood (GoFlex/Dockstar) vs. A10. It basically confirms your assumptions on the speed of A10 vs. Kirkwood.

Thanks for the Raspi results, though. It would also be nice if somebody also ran lmbench on the 'pi, just to reconfirm that it is really a slow dog. . .

I should try running lmbench on my N800. Just to confirm that a 5 year old ARM device is likely faster than a 'pi ;)

Just for the hell of it

real 0m51.598s
user 0m51.160s
sys 0m0.050s





(That's my old NSLU2 :-) )

Just for the hell of it. . .

real 0m0.086s
user 0m0.010s
sys 0m0.000s

That's on my Mele in Android, in a Debian chroot, with 1080p movie playing in the background in Android, while compiling a kernel in another debian chroot on the same device. . . ;) (okay, I'm only joking about the movie and the kernel . . .)

But still, obviously the other devices are just slightly outclassed . . .

Somehow I don't think it makes for a good benchmark, though. How can you compare against 0.000?

Thanks a lot Gnexus,

I guess that answer my question : I'll need an A10 (maybe a cubieboard) to host my website in a pure headless server (so no need of open video driver for now)

I'm not sure computing Pi decimals will help your mele a lot in serving files faster (depends on so many parameters - like the mele wemac network interface...)

Like everyone else, i give it a try.
Here we go:

# time echo "scale=2000; 4*a(1)" | bc -l 
-bash: bc: command not found

real    0m0.007s
user    0m0.000s
sys     0m0.000s

I'm faster than gnexus ;-)

... now the real one:
---
Linux ecX 3.0.42+ #6 PREEMPT Thu Sep 13 22:21:51 CEST 2012 armv7l GNU/Linux

Cross-compiled with gcc-4.7-arm-linux-gnueabihf on amd64.
-mcpu=cortex-a8 -mtune=cortex-a8 -mfpu=vfpv3 -ftree-vectorize -mfloat-abi=hard -ffast-math -fsingle-precision-constant --param l2-cache-size=512 -funswitch-loops -march=armv7-a

rootfs: debian armhf testing

# cat /proc/cpuinfo 
Processor       : ARMv7 Processor rev 2 (v7l)
BogoMIPS        : 1006.38
Features        : swp half thumb fastmult vfp edsp neon vfpv3 
...
# time echo "scale=2000; 4*a(1)" | bc -l > /dev/null

real    0m13.968s
user    0m13.960s
sys     0m0.000s

@Guillaume

I know this is too soon to use any A10 with Linux for now. I'm patient and I'll wait for another board with better support or for a better support for A10.

The only advantage the RPi has is the support and the community : the hardware is just below average. And yet I absolutely don't regret buying one (and I'll maybe buy another one in a month or so).

Seeing your results I'm doubting that Gnexus had bc installed ;)

Your results are almost the same as my Dockstar, I'm a little disappointed. But I guess the vfp should give a slight boost
anyway.

Quote

I know this is too soon to use any A10 with Linux for now.

Then why is everybody doing it on a daily basis? The new Mele is just one device out of countless others that work with no issues. The older Mele's work fine, and now the new ones also work along with a huge list of other devices that is still growing.

Quote

Seeing your results I'm doubting that Gnexus had bc installed ;)

Oh! Then did you not read the GoFlex vs A10 benchmark which basically confirm my bc results.

Since you did not appreciate the chroot results, then here they are from native A10 Mele Debian Sid:
real 0m0.080s
user 0m0.010s
sys 0m0.060s

That is with lightdm, X and a full LXDE running.


But bc is not a benchmark. The results vary too much. So if you have a 'Pi and wish to help people with a decent comparison then post lmbench for the Raspi. . . That would be a real comparison. Using bc is just blather. . .

gnexus Wrote:
-------------------------------------------------------
>

Quote

Seeing your results I'm doubting that
> Gnexus had bc installed ;)

>
> Oh! Then did you not read the
> GoFle
> x vs A10 benchmark which basically confirm
> my bc results.
>
> Since you did not appreciate the chroot results,
> then here they are from native A10 Mele Debian
> Sid:
> real 0m0.080s
> user 0m0.010s
> sys 0m0.060s
>
> That is with lightdm, X and a full LXDE running.
>
>
> But bc is not a benchmark. The results vary too
> much. So if you have a 'Pi and wish to help
> people with a decent comparison then post lmbench
> for the Raspi. . . That would be a real
> comparison. Using bc is just blather. . .

I really tried to read your comparison thread with lmbench. But there's way too much information for me and it's very hard to read.

I know bc is not a benchmark, it allow to get a general idea quickly (but nothing precise).
I'll compile lmbench on my Pi as soon as I receive my new SD card.

Quote

I really tried to read your comparison thread with lmbench. But there's way too much information

Yeah. I agree. It's just the raw data. I was going to eventually create graphs from the data, but got sidetracked.

So here's a summary of the important parts:

3.2.0-2-kirkwood

integer add: 0.97 nanoseconds
integer mul: 2.61 nanoseconds
integer div: 149.21 nanoseconds
float add: 36.52 nanoseconds
float mul: 30.52 nanoseconds
float div: 163.00 nanoseconds
double add: 51.88 nanoseconds
double mul: 46.69 nanoseconds
double div: 541.32 nanoseconds
float bogomflops: 332.25 nanoseconds
double bogomflops: 805.17 nanoseconds

3.0.36+ allwinner

integer add: 1.00 nanoseconds
integer mul: 5.96 nanoseconds
integer div: 74.49 nanoseconds
float add: 8.87 nanoseconds
float mul: 9.96 nanoseconds
float div: 32.86 nanoseconds
double add: 8.87 nanoseconds
double mul: 10.96 nanoseconds
double div: 56.75 nanoseconds
float bogomflops: 104.26 nanoseconds
double bogomflops: 175.48 nanoseconds

Quote
Since you did not appreciate the chroot results, then here they are from native A10 Mele Debian Sid: real 0m0.080s user 0m0.010s sys 0m0.060s

Please double check that you have bc installed, this result is unbelieving! Or can you try the same with your Intel laptop? My laptop give me below result, it is still far behind your A10 :-)

wy@wy-desktop:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Pentium(R) 4 CPU 2.40GHz
stepping : 9
cpu MHz : 2393.657
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe pebs bts cid xtpr
bogomips : 4787.31
clflush size : 64
cache_alignment : 128
address sizes : 36 bits physical, 32 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 15
model : 2
model name : Intel(R) Pentium(R) 4 CPU 2.40GHz
stepping : 9
cpu MHz : 2393.657
cache size : 512 KB
physical id : 0
siblings : 2
core id : 0
cpu cores : 1
apicid : 1
initial apicid : 1
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe pebs bts cid xtpr
bogomips : 4788.38
clflush size : 64
cache_alignment : 128
address sizes : 36 bits physical, 32 bits virtual
power management:

wy@wy-desktop:~$ time echo "scale=2000; 4*a(1)" | bc -l > /dev/null

real 0m5.900s
user 0m5.872s
sys 0m0.004s
wy@wy-desktop:~$

Quote
javatmn
Please double check that you have bc installed, this result is unbelieving!

Right! My desktop needs about 4 sec and my A10 tablet about 17 sec.

You were of course correct. Bc was not installed:

dpkg -s bc
dpkg-query: package 'bc' is not installed and no information is available

Sorry. I assumed the command would not even run if bc was not installed.

Here are the correct results:

real    0m16.649s
user    0m15.860s
sys     0m0.120s

Obviously bc is not a good benchmark. Look how much my results vary from guillaume and how close the are to Vlad's dockstar. The results are basically useless, and that is why I did not really bother checking bc before.

Anyway, here also are some meaningless results for a Core2 Duo 2GHz:

real    0m3.378s
user    0m3.336s
sys     0m0.008s

Just to verify bc is there on the C2D:

rpm -q bc
bc-1.06.95-6.fc17.x86_64

Lmbench on the C2D locks up for some reason. But I may try it later on my Atom netbook. For another comparison a kernel compile only takes about 15 minutes less on the Atom as on an A10 (but still a couple of hours). It takes a full day on Kirkwood. Next kernel compile I will try to time them. Unlike bc lmbench and a kernel compile are very good benchmarks.

Hi,

It's interesting to see a 3-4s difference with mine in this single simple mono limited benchmark.
Gnexus, What is your context to get 16s ?
Is it the vanilla kernel compile flags and is it a classic arm rootfs (not armhf) ?

I find the A10 performing pretty damned good for its price range.
Oh guys, i'm a big fan ;-)

I'm still waiting to get a RaspPi, now nearly 2-3 months, i don't count anymore (rs-components). They should be ashamed.
I'm curious to test this marketing hoax and see the poor Pi performing poorly with this poor man's test ;-)

I don't have a Marvell Feroceon armv5l dockstar to test, but it's clear the A10 is more recent and should crush the Dockstar with any smart benchmark.

Quote
Vlad59
I guess that answer my question : I'll need an A10 (maybe a cubieboard) to host my website in a pure headless server (so no need of open video driver for now)

I know this is too soon to use any A10 with Linux for now. I'm patient and I'll wait for another board with better support or for a better support for A10.

I think you are wrong. The A10 (a2000 or cubieboard) are perfectly capable to operate as pure headless servers and i'm going to prove it to you soon ;-) i will put them in a real production environment:
- 2 datacenters with bgp failover
- 2 production grids of 16 nodes each
- a high level rr-dns
- a lower level software linux ipvs load-balancing/fail-over in direct routing managed by the A10
- lighttpd

As far as i have test it, the A10 is perfectly stable and i'm sure it will be able to easily reach one or two years uptime without reboot (i will have to wait for this).
I'm sure too this cluster will be able to manage an average ~10K access / second with a 1Gbps real throughput.
My first benchmark on this are nearly 1300 req/sec with a 60Mbps throughput for one mele a2000 with small files... so for 32 nodes minus the ipvs load, i'm quite confident...

Quote

It's interesting to see a 3-4s difference with mine in this single simple mono limited benchmark.
Gnexus, What is your context to get 16s ?
Is it the vanilla kernel compile flags and is it a classic arm rootfs (not armhf) ?

It is a Debian armhf with 3.0.39+ a10linux kernel. Armhf probably is the reason for the difference if you are not using it. Armel runs faster in most scenarios. That is why kirkwood is still so competitive using bc (but look at the lmbench results for a better picture).

Quote

I find the A10 performing pretty damned good for its price range. Oh guys, i'm a big fan ;-)

Considering that, in kernel compilation at least, the cpu itself performs close to what my Atom 1.6 GHz does, I would have to agree! Too bad it only supports 1GB max memory. If the A10 could support 2GB memory and a working GPU in lInux it would be quite competitive with Atom in many tasks.

Quote

I don't have a Marvell Feroceon armv5l dockstar to test, but it's clear the A10 is more recent and should crush the Dockstar with any smart benchmark.

Look at the lmbench results above. . .

Quote

- lighttpd

Noooohhh! Nginx is much better!

Quote
Gnexus
Noooohhh! Nginx is much better!

Ah! ;-)

In french, we say: "la querelle d'église" (something like "a church quarrel").
I don't know any expression describing this in english.

It's the kind of discussion which often validate in the end Godwin's law ;-)

I tried both two years ago.

For pure static files, Lighttpd matches more my way of thinking:
- straightforward to configure
- just single process (why having workers when you use 'epoll'? - still only talking about static files needs)
- ipv6
- not a commercial software

I will perhaps retry nginx for the fun.

And perhaps you have nginx shares ? ;-)

Not sure where I found this, but another "poor man's benchmark" that I have been using is this:

echo '2^2^20' | time -p bc > /dev/null

On my Pogoplug:

real 69.91
user 16.18
sys 0.14


On my NSLU2:

real 197.77
user 146.21
sys 33.15

On another NSLU2:

real 373.10
user 281.35
sys 57.61

Anyone mind running this on any of the A10-type machines?

There is nothing more dangerous than a bored cat.

Vlad,

hi .. in respect of the pi i think you have to separate the GPU from CPU .. the pi SOC is basically a streaming package (in fact it is a GPU with a CPU tacked on ;-) .. the Pi has excellent support (from broadcom/pi foundation and the community) so for its intended purpose to teach kids to program and interface (dont forget the Gertboard) its excellent .. likewise as an xmbc media player its pretty good as well (video drivers work well)

.. on the other hand the A10 is the complete opposite .. pretty much zero SOC manufacturer support and no chance of the same kind of community developing .. so the A10 boxes (and boards) are relegated to transient use by a small group of "enthusiasts" until something better comes along and/or allwinner immediately kills it off when it pushes its dual core solution

the something better will no doubt be a quad core cpu/gpu board for <US$100 (you can already see a korean board a touch more expensive than $100) ...

... as i only want a headless server to replace an old hacked debian plug the mele (once sata fix is in place) should do fine .. but for the general public the pi is by far the best thing around by a country mile

@Hyena

I like the Pi, the drivers / kernel / firmware are still evolving fast. It allow running Xbmc almost flawlessly, So I'm really happy buying it.

But I used my Dockstar as Nginx / Php server and it was just usable with the slow CPU. But I'm sure the Pi is not powerfull enough for the same use.

So what I'm looking for is pure CPU power to execute PHP scripts. The A10 could be it, I still don't know.

@Guillaume, I ordered my Pi from Farnell in august and was delivered in France 2 weeks later. Hope RS will finally ship your Pi soon.

Quote
Vlad59
So what I'm looking for is pure CPU power to execute PHP scripts. The A10 could be it, I still don't know.

The A10 is great but it is still a 1Ghz single core with ~slow ram ;)
Its purpose is not to execute interpreted language like php hungry for ram and context switches.

A classic intel/amd arch seems to me the only option here.
Perhaps the future dual/quad core arm like the cortex A15 MPCore with more/fast ram will be able to do this job properly, time will tell.

I'm no talking about running google.com on an A10.

But I ran my blog on a dockstar for 1.5 years with almost 650 pages rendered a day and it was fine.

Of course I used Nginx + Php + APC (to cache PHP opcode) and heavily used nginx output caching.

I'm currently paying for a VPS and would like to get back to hosting my blog myself at home.

Quote

And perhaps you have nginx shares ? ;-)

No. I like nginx because:
- straightforward to configure
- just single process (why having workers when you use 'epoll'? - still only talking about static files needs)
- ipv6
- not a commercial software

;-)

I also like nginx because you can use it as a reverse proxy, and it uses much less cpu than lighttpd.
Besides, the numbers speak for themselves - according to netcraft nginx is now 9.63% of Web servers on the 'net.
lighttpd hosted about 0.85% only.

I used to use lighttpd, but that was before nginx had english docs. . .

Quote
darethehair
echo '2^2^20' | time -p bc > /dev/null

This said "bash: time: command not found" so:

$ time echo '2^2^20' | bc > /dev/null
real 23.27
user 22.81
sys 01.58

------------------------------------------------------------
My A10 tablet (Point Of View [NL] ProTab2XXL 10" ICS)

äxl Wrote:
-------------------------------------------------------
>

Quote
darethehair
> echo '2^2^20' | time -p bc > /dev/null

>
> This said "bash: time: command not found" so:
>

$ time echo '2^2^20' | bc > /dev/null
> real 23.27
> user 22.81
> sys 01.58

Thanks for running that! With that variation of the benchmark:

Pogoplug

real 0m16.085s
user 0m16.000s
sys 0m0.000s

This suggests to me that these A10 devices are not quite as fast as my Pogoplug, so if I was looking for an improvement to the speed of my mini webserver, I should keep waiting a bit longer. That being said, it would sure be nice to have the luxury of a monitor plugged to see what is going on during the boot process (when problems occur)! :)

There is nothing more dangerous than a bored cat.

Eh?( On my desktop:

$ echo '2^2^20' | time -p bc > /dev/null
real 6.01
user 5.76
sys 0.01

$ time -p echo '2^2^20' | bc > /dev/null
real 6.03
user 5.75
sys 0.01

------------------------------------------------------------
My A10 tablet (Point Of View [NL] ProTab2XXL 10" ICS)

äxl Wrote:
-------------------------------------------------------
> Eh?( On my desktop:
>

$ echo '2^2^20' | time -p bc > /dev/null
> real 6.01
> user 5.76
> sys 0.01

>

$ time -p echo '2^2^20' | bc > /dev/null
> real 6.03
> user 5.75
> sys 0.01

Yea, on some of my machines the two syntax styles seem to be treated the same way, but not (for example) on my Pogoplug (for whatever reason).

There is nothing more dangerous than a bored cat.

Goflex Home: Debian Wheezy, 3.2.35 PREEMPT RT kernel:
----
root@debian:~# time echo "scale=2000; 4*a(1)" | bc -l > /dev/null

real 0m11.832s
user 0m11.790s
sys 0m0.000s
-----
root@debian:~# time echo '2^2^20'| bc > /dev/null

real 0m15.964s
user 0m15.900s
sys 0m0.020s
-----
Intel Atom N270 1600:
dmitry@Dmitry-AOA150:~$ time echo "scale=2000; 4*a(1)" | bc -l > /dev/null

real 0m11.368s
user 0m11.073s
sys 0m0.016s
----
dmitry@Dmitry-AOA150:~$ time echo '2^2^20'| bc > /dev/null

real 0m15.438s
user 0m15.237s
sys 0m0.016s

Here's another new and quite relevant device, since it is getting down into the ARM price range of under $200:

time echo "scale=2000; 4*a(1)" | bc -l > /dev/null 

real    0m5.783s
user    0m5.694s
sys     0m0.013s

uname -a
Linux cb01 3.7.1 #4 SMP Mon Jan 7 16:14:30 GMT 2013 x86_64 x86_64 x86_64 GNU/Linux

cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Celeron(R) CPU 847 @ 1.10GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 800.000
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave lahf_lm arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 2195.02
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Celeron(R) CPU 847 @ 1.10GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 800.000
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave lahf_lm arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 2195.02
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

Device is $199 Acer C7 Chromebook running Fedora 18.
Performance is within 2 s of my old Core2Duo 2.0GHz notebook.
And yes, KVM is working with this kernel I just compiled, so it can run Win7-64 under KVM with very good performance!