Performance Issues with Debian@Zyxel NSA 325v2

Hello everybody,

I've installed Debian on my Zyxel NSAv2 NAS in order to get rid of the awful legacy Zyxel UI.

The installation worked perfectly, debian is now running on my nas. I've installed it on a SD card, located in a SD Card reader in one of the read USB ports (none of my "normal" pen drives worked).

root@debian:~# uname -a
Linux debian 3.14.0-kirkwood-tld-1 #1 PREEMPT Tue Apr 1 22:54:58 PDT 2014 armv5tel GNU/Linux

Unfortunately , I've got some speed issues. When rsyncing a file from the first harddisk to /dev/null I get an average speed of 24.59MB/s. While running a backup with rsync from one disk to the other, i get transferspeeds around 15MB/s

Sometimes, the drives are realy slow:

root@debian:~# dd if=/media/Daten/Daten/Videos/Neu/movie.mkv of=/media/Backup/Daten/Videos/Neu/movie.mkv oflag=direct
220932+0 records in
220932+0 records out
113117184 bytes (113 MB) copied, 370.886 s, 305 kB/s

Running hdparm benchmark I get the following results, harddisk was unmounted during the tests:

root@debian:~# hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   538 MB in  2.00 seconds = 268.61 MB/sec
 Timing buffered disk reads: 318 MB in  3.02 seconds = 105.27 MB/sec
root@debian:~# hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   638 MB in  2.00 seconds = 319.04 MB/sec
 Timing buffered disk reads: 394 MB in  3.02 seconds = 130.68 MB/sec
root@debian:~# hdparm -Tt /dev/sda

/dev/sda:
 Timing cached reads:   634 MB in  2.00 seconds = 316.26 MB/sec
 Timing buffered disk reads: 392 MB in  3.01 seconds = 130.02 MB/sec

This is my first harddrive:

root@debian:~# hdparm -I /dev/sda

/dev/sda:

ATA device, with non-removable media
	Model Number:       WDC WD30EURX-63T0FY0                    
	Serial Number:      WD-WCC4N0069999
	Firmware Revision:  80.00A80
	Transport:          Serial, SATA 1.0a, SATA II Extensions, SATA Rev 2.5, SATA Rev 2.6, SATA Rev 3.0

This is the second harddrive:

/dev/sdc:

ATA device, with non-removable media
	Model Number:       ST3000VX000-9YW166                      
	Serial Number:      Z1F1G645
	Firmware Revision:  CV13    
	Transport:          Serial, SATA Rev 3.0

I've formated both drives with btrfs.

SMART tells me, the drives are fine:

root@debian:~# smartctl -H /dev/sda
smartctl 6.4 2014-10-07 r4002 [armv5tel-linux-3.14.0-kirkwood-tld-1] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

root@debian:~# smartctl -H /dev/sdc
smartctl 6.4 2014-10-07 r4002 [armv5tel-linux-3.14.0-kirkwood-tld-1] (local build)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

Does anybody have an idea, why the performance isn't as expected? When moving files on the harddrives attached to an Windows PC and installed in an external USB case I get transferspeeds around 100MB/s.



Edited 1 time(s). Last edit at 11/14/2016 03:55AM by haarwurm.

While I can't offer any specific advice I suspect the problem might be the weak CPU. btrfs might hit your CPU a bit. If "top" indeed shows high CPU utilization during the copy you might want to try another file system like ext4 or even vfat (just to verify where the bottleneck is).

I agree with Muchatet. Try a different file system to verify.

Also use rsync command to do this to see if the performance is normal. btrfs might have problem getting contiguous blocks when you use dd.

rsync -a /media/Daten/Daten/Videos/Neu/movie.mkv /media/Backup/Daten/Videos/Neu/movie.mkv

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Thank you very much for the suggestions. Since I can't check an other file system at the moment, I've decided to upgrade first to the newest U-boot / rootfs.

But, no luck with this topic either: http://forum.doozan.com/read.php?3,31187,31187#msg-31187

I finally managed to upgrade debian as well u-boot on my NAS.

root@debian:~# uname -a
Linux debian 4.8.3-kirkwood-tld-1 #1 PREEMPT Thu Oct 20 23:33:18 PDT 2016 armv5tel GNU/Linux

U-Boot has the version 2016.05.

Unfortunately the performance still isn't as expected. I'm copying a file from local disk /dev/sda to local disk /dev/sdb:

root@debian:~# hdparm -I /dev/sda | grep speed
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)

root@debian:~# hdparm -I /dev/sdb | grep speed
           *    Gen1 signaling speed (1.5Gb/s)
           *    Gen2 signaling speed (3.0Gb/s)

root@debian:~# for i in 1 2 3; do hdparm -tT --direct /dev/sda; done

/dev/sda:
 Timing O_DIRECT cached reads:   354 MB in  2.01 seconds = 176.07 MB/sec
 Timing O_DIRECT disk reads: 270 MB in  3.01 seconds =  89.76 MB/sec

/dev/sda:
 Timing O_DIRECT cached reads:   352 MB in  2.01 seconds = 175.45 MB/sec
 Timing O_DIRECT disk reads: 272 MB in  3.00 seconds =  90.54 MB/sec

/dev/sda:
 Timing O_DIRECT cached reads:   350 MB in  2.00 seconds = 174.62 MB/sec
 Timing O_DIRECT disk reads: 272 MB in  3.01 seconds =  90.49 MB/sec

root@debian:~# for i in 1 2 3; do hdparm -tT --direct /dev/sdb; done

/dev/sdb:
 Timing O_DIRECT cached reads:   334 MB in  2.01 seconds = 166.54 MB/sec
 Timing O_DIRECT disk reads: 268 MB in  3.01 seconds =  89.10 MB/sec

/dev/sdb:
 Timing O_DIRECT cached reads:   334 MB in  2.01 seconds = 166.55 MB/sec
 Timing O_DIRECT disk reads: 322 MB in  3.01 seconds = 107.02 MB/sec

/dev/sdb:
 Timing O_DIRECT cached reads:   348 MB in  2.01 seconds = 173.32 MB/sec
 Timing O_DIRECT disk reads: 328 MB in  3.01 seconds = 108.98 MB/sec

root@debian:~# mount | grep 'sd[a|b]'
/dev/sda1 on /media/Daten type ext3 (rw,relatime,data=ordered)
/dev/sdb1 on /media/backup type ext3 (rw,relatime,data=ordered)

root@debian:~# ls -la /media/Daten/gparted-live-0.23.0-2-amd64.iso -h
-rwxrw---- 1 datenzugriff datenzugriff 252M Aug 25 10:10 /media/Daten/gparted-live-0.23.0-2-amd64.iso

root@debian:~# for i in 1 2 3; do time cp /media/Daten/gparted-live-0.23.0-2-amd64.iso /media/backup/$i; done

real    0m4.825s
user    0m0.000s
sys     0m4.370s

real    0m4.904s
user    0m0.030s
sys     0m4.170s

real    0m4.928s
user    0m0.000s
sys     0m4.220s

root@debian:~# for i in 1 2 3; do time rsync /media/Daten/gparted-live-0.23.0-2-amd64.iso /media/backup/ -Wa; rm /media/backup/gparted-live-0.23.0-2-amd64.iso; sync; done

real    0m14.519s
user    0m6.980s
sys     0m6.800s

real    0m14.504s
user    0m7.290s
sys     0m6.630s

real    0m14.285s
user    0m7.050s
sys     0m6.640s

A bit of googling lead to the conclusion, that the CPU is the bottleneck, during rsyncing the file, CPU showed allways 100% utilization.

Do you have any suggestions, how to possibly solve this problem and keep the ability of rsync?

haarwurm,

> A bit of googling lead to the conclusion, that the
> CPU is the bottleneck, during rsyncing the file,
> CPU showed allways 100% utilization.

That's interesting. I run my NSA325v2 as a Linux development box with 1 HDD, so never paid attention to this. Let me add another disk to this box (when I have time, can't promise when) and double check your results.

To clarify the test, that file size is 113 MB? and what file system is on it? I saw that it is not Ext4, have you tried that?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 12/04/2016 03:17PM by bodhi.

Hi,

the filesize is ~252 MB. The used file system is on both harddrives ext3, I thought, this should need even less performance then ext4 with journaling.

I'm curious, what your tests will show.

root@debian:~# ls -la /media/Daten/gparted-live-0.23.0-2-amd64.iso -h
-rwxrw---- 1 datenzugriff datenzugriff 252M Aug 25 10:10 /media/Daten/gparted-live-0.23.0-2-amd64.iso

haarwurm,

Quote

root@debian:~# for i in 1 2 3; do time cp /media/Daten/gparted-live-0.23.0-2-amd64.iso /media/backup/$i; done

real 0m4.825s
user 0m0.000s
sys 0m4.370s

root@debian:~# for i in 1 2 3; do time rsync /media/Daten/gparted-live-0.23.0-2-amd64.iso /media/backup/ -Wa; rm /media/backup/gparted-live-0.23.0-2-amd64.iso; sync; done

real 0m14.519s
user 0m6.980s
sys 0m6.800s

On closer look. This seems normal. rsync is always going to be slower than cp. However, there are options in rsync that you should try that would make the number better.

Regarding CPU load of 100%. Usually it is good to see that. If one app is the only thing that is running then you would want to to maximize the CPU usage. OTOH, try running a different task that would consume a lot of CPU, and see if rsync and that task will share the CPU appropriately. Only when rsync is running and you can't run other apps effectively, then worry.

With that said, usually my rsync across the Gbits network that do the daily back up for the entire development folder takes about 5% to 30% CPU. So you should try the options available in rsync to see if the default settings has made it much worse.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 12/13/2016 02:44PM by bodhi.