Welcome! Log In Create A New Profile

Advanced

Kirkwood and Oxnas Samba Performance

Posted by daviddyer 
Forum List Message List New Topic
daviddyer
Kirkwood and Oxnas Samba Performance
March 24, 2017 09:55PM
System 1 : Dockstar with usb 2.0 external drive, Debian wheezy with kernel 3.9.5

System 2: Pogoplug Pro with USB 3.0 external drive (since Pogopro has usb 2.0 only, it will work at USB 2.0 speed), Debian Jessie with kernel 4.4.54

RAW usb hard drive speed for both system is about the same, about 29MB/s read ( dd if=/dev/sda of=/dev/null bs=4M count=256 )

Same samba conf file. (sftp one smb.conf to the other system, changed folder name etc)

Copy one 3GB iso file from Kirkwood/ Dockstar to a windows 7 machine, speed is about 25-26MB/s

copy the same iso file from Oxnas / PogoPro to the same windows 7 machine, speed is about 15MB/s

Oxnas is not so good on samba with usb drive?
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 24, 2017 10:56PM
daviddyer,

> System 1 : Dockstar with usb 2.0 external drive, D
> ebian wheezy with kernel 3.9.5

1. This is a very old kernel. Why?

2. Is the Samba conf file the same used in both systems?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
daviddyer
Re: Kirkwood and Oxnas Samba Performance
March 25, 2017 07:51AM
1 Yes. Very old kernel. If it works, I won't mess with it.

2 Basically the same smb.conf. I sftp transferred smb.conf from one system to the other, then just changed the shared folder name, etc.

3 I will change the network cable and try again.
Reply Quote
daviddyer
Re: Kirkwood and Oxnas Samba Performance
March 25, 2017 08:28AM
Update: Changed network cable. Still the same result. Pogopro's samba speed is much slower.

Dockstar & PogoPro are connected to the same 8-port switch.

Windows Task Manager attached.



Edited 1 time(s). Last edit at 03/25/2017 08:29AM by daviddyer.
Reply Quote
Attachments:
open | download - dockstar_pogopro.jpg (193.1 KB)
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 25, 2017 03:32PM
daviddyer,

We don't have a network benchmark comparison between Kirkwood and Oxnas yet. I've run this a long time ago and can't remember what the numbers look like.

Would you like to run iperf test? I assume you have a Gbits switch in the midle? It would be good to have this added to the Wiki thread! The test is meaningful only if If you have another Linux box that you can connect to this switch, and the Pogo V3 and Dockstar connected to the same switch. And the cables all have the same rating (5e or 6) for Gbits.

The Oxnas network driver might have degraded Samba performance. So if we can eliminate this variable first, then we can look in other parts.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 25, 2017 03:45PM
iperf tests should be posted in this thread:

http://forum.doozan.com/read.php?2,28829,28829#msg-28829

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
daviddyer
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 01:35PM
Well, the "server" side used a crappy Realtek 1000M card... (Linux Server01 3.16.0-4-686-pae #1 SMP Debian 3.16.39-1+deb8u2 (2017-03-07) i686 GNU/Linux on Intel Q6600 with 3GB memory)

For Dockstar / Kirkwood system (Linux Dockstar01 3.9.5-dockstar-goflex-shyd #3 Sun Jun 9 11:36:48 CEST 2013 armv5tel GNU/Linux) 

# iperf -c 192.168.7.201 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.7.201, TCP port 5001
TCP window size:  117 KByte (default)
------------------------------------------------------------
[  3] local 192.168.7.17 port 33542 connected with 192.168.7.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   857 MBytes   719 Mbits/sec
[  5] local 192.168.7.17 port 5001 connected with 192.168.7.201 port 47774
[  5]  0.0-10.0 sec   734 MBytes   615 Mbits/sec



=======================================================
For Pogoplug Pro / Oxnas system (Debian Wheezy, Linux PogoPro 3.18.5-oxnas-tld-1 #2 SMP PREEMPT Thu Feb 12 11:17:40 PST 2015 armv6l GNU/Linux )
(I tried another kernel version... not the same as I mentioned before) 

#  iperf -c  192.168.7.201 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.7.201, TCP port 5001
TCP window size:  175 KByte (default)
------------------------------------------------------------
[  5] local 192.168.7.18 port 46676 connected with 192.168.7.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   478 MBytes   401 Mbits/sec
[  4] local 192.168.7.18 port 5001 connected with 192.168.7.201 port 51278
[  4]  0.0-10.0 sec   412 MBytes   345 Mbits/sec

Pogoplug Pro with latest kernel (Jessie) 

root@PogoPro:~# uname -a
Linux PogoPro 4.4.54-oxnas-tld-1 #2 SMP PREEMPT Sat Mar 18 23:09:58 PDT 2017 armv6l GNU/Linux
root@PogoPro:~# iperf -c 192.168.7.201 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.7.201, TCP port 5001
TCP window size:  118 KByte (default)
------------------------------------------------------------
[  5] local 192.168.7.18 port 45262 connected with 192.168.7.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   366 MBytes   307 Mbits/sec
[  4] local 192.168.7.18 port 5001 connected with 192.168.7.201 port 51280
[  4]  0.0-10.0 sec   353 MBytes   296 Mbits/sec

Looks like even worse than old wheezy with old kernel.


Maybe this can explain why the Mfr. changed back to Kirkwood?


If you need more tests, tell me.



Edited 2 time(s). Last edit at 03/27/2017 01:49PM by daviddyer.
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 01:53PM
Thanks daviddyer,

That's a big drop in performance! I suspected this because the same smb config should behave the same way, considering we are using mainline Debian packages. The only significant difference is the ethernet driver which is an OXNAS version, versus Marvell version.

I'll take a peek at this code to see if I can find anything.

I think we also should run a Kirkwood-->OXNAS and Kirkwood-->Kirkwood iperf tests. That'll narrow down the problem further (Realtek is not that good a card comparing to Marvell).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 03/27/2017 02:01PM by bodhi.
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 02:05PM
By the way, it was a really good test :)

I think the OpenWRT/LEDE version in 4.4.54-tld-1 might have made it even worse.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 03/27/2017 02:07PM by bodhi.
Reply Quote
daviddyer
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 03:45PM
Kirkwood/Dockstar to Kirkwood/Dockstar (Both has 3.9.5 kernel) 

# iperf -c 192.168.7.27 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.7.27, TCP port 5001
TCP window size:  164 KByte (default)
------------------------------------------------------------
[  3] local 192.168.7.17 port 56289 connected with 192.168.7.27 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   892 MBytes   748 Mbits/sec
[  5] local 192.168.7.17 port 5001 connected with 192.168.7.27 port 49458
[  5]  0.0-10.0 sec   894 MBytes   750 Mbits/sec

Oxnas / Pogoplug Pro to Kirkwood / Dockstar 

root@PogoPro:~# iperf -c 192.168.7.27 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.7.27, TCP port 5001
TCP window size: 74.4 KByte (default)
------------------------------------------------------------
[  5] local 192.168.7.18 port 49320 connected with 192.168.7.27 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   456 MBytes   382 Mbits/sec
[  4] local 192.168.7.18 port 5001 connected with 192.168.7.27 port 33083
[  4]  0.0-10.0 sec   955 MBytes   801 Mbits/sec

Oxnas / PogoPro to Oxnas / PogoPro 

root@Pro02:~# iperf -c 192.168.7.18 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.7.18, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------
[  4] local 192.168.7.58 port 47564 connected with 192.168.7.18 port 5001
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   506 MBytes   424 Mbits/sec
[  5] local 192.168.7.58 port 5001 connected with 192.168.7.18 port 41192
[  5]  0.0-10.0 sec   600 MBytes   503 Mbits/sec
Reply Quote
daviddyer
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 03:47PM
If we can't improve Oxnas's network performance, I guess we don't need to continue the project of adding usb 3.0 to pogoplug pro...
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 05:55PM
daviddyer Wrote:
-------------------------------------------------------
> If we can't improve Oxnas's network performance, I
> guess we don't need to continue the project of add
> ing usb 3.0 to pogoplug pro...

That's a different issue altogether. Just increasing the speed of the disk drive would make the transfer rate substantially faster. Curently the Pogo V3 network is slow, but not that slow to the point where USB3.0 is brought down to USB 2.0 speed!

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
daviddyer
Re: Kirkwood and Oxnas Samba Performance
March 27, 2017 07:29PM
Well, I tried sata boot Pogoplug Pro then samba, the hard drive I used is quite old, (still a lot faster than USB 2.0 hard drive, About 70MB/s read), samba read speed was about 18MB/s for big files.



Edited 1 time(s). Last edit at 03/27/2017 07:31PM by daviddyer.
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 28, 2017 04:00AM
I think I know where the network problem is. I'll run a few tests to confirm.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 03/28/2017 04:00AM by bodhi.
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 28, 2017 05:29AM
daviddyer Wrote:
-------------------------------------------------------
> Well, I tried sata boot Pogoplug Pro then samba, t
> he hard drive I used is quite old, (still a lot fa
> ster than USB 2.0 hard drive, About 70MB/s read),
> samba read speed was about 18MB/s for big files.

That's what I meant. Only when you use HDD or USB 3.0 you could get the xfer rate up close to network speed. But the OXNAS network speed is abnormal.

I'll post a temporary fix afer I run some more tests.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
March 28, 2017 11:21PM
The reason network performance is degraded on the Pogo V3 OXNAS is: flow control. The OXNAS ethernet driver set this by default. We need to turn it off in the driver, but that will be incorprated in the next kernel release. At the moment, the workaround is to do turn it off with ethtool.

Turn off flow control 

ethtool -A eth0 rx off tx off

Test runs data

Kirkwood to OXNAS 

root@Dockstar:~# iperf -c 192.168.0.222 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.222, TCP port 5001
TCP window size:  109 KByte (default)
------------------------------------------------------------
[  3] local 192.168.0.220 port 58460 connected with 192.168.0.222 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   819 MBytes   687 Mbits/sec
[  5] local 192.168.0.220 port 5001 connected with 192.168.0.222 port 58634
[  5]  0.0-10.0 sec   710 MBytes   594 Mbits/sec


OXNAS to Kirkwood 

root@PogoPro:~# iperf -c 192.168.0.220 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.220, TCP port 5001
TCP window size:  219 KByte (default)
------------------------------------------------------------
[  5] local 192.168.0.222 port 58646 connected with 192.168.0.220 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   693 MBytes   581 Mbits/sec
[  4] local 192.168.0.222 port 5001 connected with 192.168.0.220 port 58472
[  4]  0.0-10.0 sec   862 MBytes   723 Mbits/sec

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
a7kr
Re: Kirkwood and Oxnas Samba Performance
April 05, 2017 05:25PM
I'll post here since a web search led me to this thread.

I have two pogoplug pros, they're fantastic little devices and the work that's shared here in making them useful is exemplary.

each of them have a single hard drive connected through sata, and a flash drive with the rootfs. ethernet performance appears to be suboptimal and the above suggested solution doesn't appear to have any affect (248 is a laptop, while 201 and 202 are pogoplugs): 

#  uname -ar 

Linux Black 4.4.38-oxnas-tld-5 #1 SMP PREEMPT Sun Dec 11 17:32:48 PST 2016 armv6l GNU/Linux
# ethtool -A eth0 rx off tx off

rx unmodified, ignoring
tx unmodified, ignoring
no pause parameters changed, aborting
# iperf -s

------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[  4] local 192.168.100.201 port 5001 connected with 192.168.100.248 port 63587
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec   408 MBytes   342 Mbits/sec
[  5] local 192.168.100.201 port 5001 connected with 192.168.100.202 port 49440
[  5]  0.0-10.0 sec   506 MBytes   423 Mbits/sec
# iperf -c 192.168.100.248

------------------------------------------------------------
Client connecting to 192.168.100.248, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------
[  3] local 192.168.100.201 port 35782 connected with 192.168.100.248 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   434 MBytes   364 Mbits/sec
# iperf -c 192.168.100.202

------------------------------------------------------------
Client connecting to 192.168.100.202, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------
[  3] local 192.168.100.201 port 39032 connected with 192.168.100.202 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec   517 MBytes   434 Mbits/sec
# lsblk

NAME      MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda         8:0    0   1.8T  0 disk 
`-sda1      8:1    0   1.8T  0 part /mnt/srv
sdb         8:16   1  14.9G  0 disk 
`-sdb1      8:17   1  14.9G  0 part /
mtdblock0  31:0    0    14M  0 disk 
mtdblock1  31:1    0   114M  0 disk
# dd if=/dev/sda of=/dev/null bs=4M count=256

256+0 records in
256+0 records out
1073741824 bytes (1.1 GB) copied, 9.97461 s, 108 MB/s

I haven't done extensive samba testing but read speeds are around 20MB/s, when copying files from the pogoplug to the laptop.

would there be anything else that I could try to increase ethernet performance, at least in iperf?
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
April 05, 2017 05:53PM
a7kr,

> each of them have a single hard drive connected th
> rough sata, and a flash drive with the rootfs. eth
> ernet performance appears to be suboptimal and the
> above suggested solution doesn't appear to have an
> y affect (248 is a laptop, while 201 and 202 are p
> ogoplugs):

What is your output on both 201 and 202 for: 

ethtool -a eth0
Make sure "ethtool -a eth0" show both TX and RX are off: 
Pause parameters for eth0:
Autonegotiate:	on
RX:		off
TX:		off
RX negotiated:	off
TX negotiated:	off
Try disconnect the ethernet cable that connects the Pogo Pro to the switch, and then plug in the ethernet cable.

Note that if you turn flow control off, and then reboot the Pogo Pro before testing, then you dont have to unplug the ethernet cable.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
a7kr
Re: Kirkwood and Oxnas Samba Performance
April 05, 2017 06:14PM
yes, they're off for both and the pogoplugs were rebooted.

I retested iperf after the reboot: 

# iperf -c 192.168.100.202 -r 
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.100.202, TCP port 5001
TCP window size:  149 KByte (default)
------------------------------------------------------------
[  5] local 192.168.100.201 port 36610 connected with 192.168.100.202 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   521 MBytes   437 Mbits/sec
[  4] local 192.168.100.201 port 5001 connected with 192.168.100.202 port 47870
[  4]  0.0-10.0 sec   487 MBytes   408 Mbits/sec
# iperf -c 192.168.100.201 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.100.201, TCP port 5001
TCP window size:  105 KByte (default)
------------------------------------------------------------
[  5] local 192.168.100.202 port 47872 connected with 192.168.100.201 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   512 MBytes   429 Mbits/sec
[  4] local 192.168.100.202 port 5001 connected with 192.168.100.201 port 36612
[  4]  0.0-10.0 sec   519 MBytes   434 Mbits/sec
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
April 06, 2017 01:45PM
Thanks a7kr,

All the tests I've done is on a different box (Kirkwood) with the Pogo V3. I'll try to run this 2 Pogo V3 test when I have a chance (ATM, I don't have them on the same switch).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
bodhi
Re: Kirkwood and Oxnas Samba Performance
April 08, 2017 09:59PM
Here is my network benchmarks for OXNAS to OXNAS with flow control turned off.

The performance (~600Mbs) is not as good as when one end is a Kirkwood box (~700mbs). But it is not that bad. 

root@PogoPro:~# uname -a
Linux PogoPro 4.4.54-oxnas-tld-1 #2 SMP PREEMPT Sat Mar 18 23:09:58 PDT 2017 armv6l GNU/Linux

root@PogoPro:~# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:	on
RX:		off
TX:		off
RX negotiated:	off
TX negotiated:	off

root@PogoClassic:/boot# uname -a
Linux PogoClassic 4.4.54-oxnas-tld-1 #2 SMP PREEMPT Sat Mar 18 23:09:58 PDT 2017 armv6l GNU/Linux

root@PogoClassic:/boot# ethtool -a eth0
Pause parameters for eth0:
Autonegotiate:	on
RX:		off
TX:		off
RX negotiated:	off
TX negotiated:	off

root@PogoPro:~# iperf -c 192.168.0.240 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.240, TCP port 5001
TCP window size:  101 KByte (default)
------------------------------------------------------------
[  5] local 192.168.0.222 port 48058 connected with 192.168.0.240 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   706 MBytes   592 Mbits/sec
[  4] local 192.168.0.222 port 5001 connected with 192.168.0.240 port 41364
[  4]  0.0-10.0 sec   740 MBytes   620 Mbits/sec

root@PogoClassic:/boot# iperf -c 192.168.0.222 -r
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to 192.168.0.222, TCP port 5001
TCP window size:  101 KByte (default)
------------------------------------------------------------
[  5] local 192.168.0.240 port 41358 connected with 192.168.0.222 port 5001
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec   738 MBytes   619 Mbits/sec
[  4] local 192.168.0.240 port 5001 connected with 192.168.0.222 port 48052
[  4]  0.0-10.0 sec   706 MBytes   591 Mbits/sec

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Reply Quote
Newer Topic Older Topic
Author:

Your Email:


Subject:
Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: