File Transfer speed

Am using Debian in Pogoplug series 4. The file transfer rate is very slow, i.e. 3MBPS via wifi and 6MBPS wired. Is it due to the OS (Debian) resides in SD card?.

assuming that you are using SAMBA with no performance tuning.....

The symptoms you describe are more likely due to the read and write speeds of your SD card.

The fact that you are getting lower speeds on wifi than on wired is the nature of the beast with wifi - given that it is basically a half duplex mechanism with lots of overheard so i dont that what you are experiencing is anything to be concerned over.

My use case is the following :

Wired : Pogoplug V4 with SATA disk ------> 18Mb over samba
Wired : Pogoplug V4 mobile with usb hdd ---> 15Mb over samba

These seem to be regarded as acceptable and in the high end of the expected speeds, there comes a wall eventually you cant climb over in respects of the memory size and cpu cycles available

vaidyasr,

It's the SD card that give you those speeds. Try USB or SATA for data drive to see much improvement. You can keep the system on SD card.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi
Am not writing from SD card. Am trying to move data from one USB drive to another. I was trying dd command to test the write speeds on the USB drive and got 6mbps. I am using class 4 sdcard. I have even changed to a class 10 sdcard. Still same results.

Is it possible to install embdebian on the nand? Am using pogo plug v4



Edited 1 time(s). Last edit at 10/04/2014 07:48PM by vaidyasr.

vaidyasr,

The rootfs has little to do with it, unless your'e swaping to a swap file or partition on the drive that the rootfs is on while copying.

If you're copying from USB-to-USB, then it has a lot to do with those drives speeds, and how they are formatted. So even if the rootfs is on the SD card, it should not be a factor.

You can verify it by running iotop on a different terminal, and watch if the SD card is somehow being accessed. You can also see the real and practical transfer speeds while copy files with this.

iotop -o -b -qqq

Another similar tool is iostat.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Bodhi,
Thanks for the answer. Am using NTFS formatted USB 3.0 drives. I will check with the iotop command and post the results.

the NTFS formatted cards are the issue. NTFS read/write on linux is REALLY intensive on cpu usage.

You would do better with fat32 if you want compatability with M$ systems, or switch to ext3 or ext4 for Linux and other variants. Ideally you would want to be using UFS filesystem



Edited 1 time(s). Last edit at 10/06/2014 07:40AM by Gravelrash.

bodhi,
When I am moving a file from a networked device to the HDD's connected to the Pogoplug v4 (formatted as NTFS), I got the following results
441 be/4 root 0.00 B/s 4.12 M/s 0.00 % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o rw
441 be/4 root 0.00 B/s 4.25 M/s 0.00 % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o rw
441 be/4 root 0.00 B/s 4.14 M/s 0.00 % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o rw
441 be/4 root 0.00 B/s 4.22 M/s 0.00 % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o rw
441 be/4 root 0.00 B/s 4.25 M/s 0.00 % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o rw

So, the read write speed is 4..1MB/s average right?

vaidyasr Wrote:
-------------------------------------------------------
> bodhi,
> When I am moving a file from a networked device to
> the HDD's connected to the Pogoplug v4 (formatted
> as NTFS), I got the following results
> 441 be/4 root 0.00 B/s 4.12 M/s 0.00
> % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o
> rw
> 441 be/4 root 0.00 B/s 4.25 M/s 0.00
> % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o
> rw
> 441 be/4 root 0.00 B/s 4.14 M/s 0.00
> % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o
> rw
> 441 be/4 root 0.00 B/s 4.22 M/s 0.00
> % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o
> rw
> 441 be/4 root 0.00 B/s 4.25 M/s 0.00
> % 0.00 % mount.ntfs /dev/sdb1 /share/My_Book -o
> rw
>
> So, the read write speed is 4..1MB/s average
> right?

If it is connected wirelessly, then yes, it's normal. If it is wired then it's a little slow, even for NTFS.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

am transferring files via wired connection only. So, what could be the reason?.

NTFS is slow. If you must use it, use big_writes. For example:

mount -L $disk_label -o noatime,big_writes /media/$disk_label

Google it and you will find may different tweaks to improve the speed.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Thanks bodhi,
The big_writes option will improved the read speed too, or only write speed?

vaidyasr Wrote:
-------------------------------------------------------
> Thanks bodhi,
> The big_writes option will improved the read speed
> too, or only write speed?

Should improve both.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi,
I tried to mount with big_writes option, but the output of mount option is below after mounted.

/dev/sda1 on /share/My_Book_3 type fuseblk (rw,nosuid,nodev,noatime,user_id=0,group_id=0,allow_other,blksize=4096)

Is that right?. am using ntfs-3g.

speed test for NTFS drive.
dd if=/dev/zero of=testfile bs=1024 count=102400
102400+0 records in
102400+0 records out
104857600 bytes (105 MB) copied, 63.3855 s, 1.7 MB/s

vaidyasr,

It won't show up in the mount command. You can confirm this with ps:

ps -deaf | grep ntfs

And to test it, copy a few large files from one plug to another or from one drive to another with time command:

time cp

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 10/16/2014 04:42PM by bodhi.

For file copy from one USB to another connected on the same hub, 9MB/s.
Network transfer wired from another machine to USB drive in Pogoplug 11.7MB/s

Is the above values are normal?.

vaidyasr Wrote:
-------------------------------------------------------
> For file copy from one USB to another connected on
> the same hub, 9MB/s.
> Network transfer wired from another machine to USB
> drive in Pogoplug 11.7MB/s
>
> Is the above values are normal?.

You are using NTFS so it is normal.

Pogo V4 USB 3.0 should help the speed (not as much as SATA), but if you try Ext4 or Ext3, the speed should significantly improved. If you don't plan to unplug it and mount on Windows/Mac, just use Samba only, then it should be format Ext3/Ext4.

And also if you plan to use it on Windows/Mac but as read-only drive (possible with free driver), then Ext3/Ext4 still a better choice.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Thanks bodhi

@vaidyasr, are you using a gigabit switch between the Pogoplug and the other machine? Does the other machine have a gigabit ethernet adapter?

11.7MB is around the limit of a 100MB network.

Sorry if I missed that detail in the thread.


vaidyasr Wrote:
-------------------------------------------------------
> For file copy from one USB to another connected on
> the same hub, 9MB/s.
> Network transfer wired from another machine to USB
> drive in Pogoplug 11.7MB/s
>
> Is the above values are normal?.

You must mount the drive or put in your /etc/fstab
ntfs-3g defaults,noatime,big_writes 0 0

Now edit your file in /etc/samba/smb.conf

[global]
        log level = 0
        guest account = root
        load printers = no
        socket options = IPTOS_LOWDELAY TCP_NODELAY SO_KEEPALIVE
        wins server = 192.168.1.1
        map to guest = Bad User
        netbios name = POGOPLUG
        write cache size = 2097152
        server string = POGOPLUG
        printing = bsd
        workgroup = WORKGROUP
        os level = 20
        use sendfile = yes
        printcap name = /dev/null
        security = user
        disable spoolss = yes

 #unprotected/public share without login

[usb]
  path = /mnt
  read only = no
  public = yes
  writable = yes
  #force user = root
  guest ok = yes

for path just put your shared directory.
Next run the command

service samba restart

You should see increase in transfer speed assuming your drives support much higher speed. USB and SD cards are slow. I'm using USB stick and Physical hard drive. USB stick is like a turtle compare to physical drive :)



Edited 2 time(s). Last edit at 10/23/2014 12:13AM by Almaz.