Build your own Pro NAS – Seagate Goflex Net with Debian Linux, RAID1 and OpenMediaVault

d2a,

This is a very common problem when you are migrating form USB to HDD. Some u-boot envs need adjustment so that the rootfs on the HDD can be mounted. Especially when you have mutiple HDDs and USBs in the slot.

The solution was described here: http://forum.doozan.com/read.php?3,8044,8152#msg-8152

So if you're not sure about the above solution, then boot back to USB, and get the envs listing, post it here and I'll point you to the correct envs to modify.

fw_printenv

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

thanks bodhi - i thought i'd set the uboot envs correctly but it's a murky world for me...

ethact=egiga0
bootdelay=3
baudrate=115200
mainlineLinux=yes
console=ttyS0,115200
led_init=green blinking
led_exit=green off
led_error=orange blinking
mtdparts=mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
mtdids=nand0=orion_nand
partition=nand0,2
stdin=serial
stdout=serial
stderr=serial
rescue_set_bootargs=setenv bootargs console=$console ubi.mtd=2 root=ubi0:rootfs ro rootfstype=ubifs $mtdparts $rescue_custom_params
rescue_bootcmd=if test $rescue_installed -eq 1; then run rescue_set_bootargs; nand read.e 0x800000 0x100000 0x400000; bootm 0x800000; else run pogo_bootcmd; fi
pogo_bootcmd=if fsload uboot-original-mtd0.kwb; then go 0x800200; fi
force_rescue=0
force_rescue_bootcmd=if test $force_rescue -eq 1 || ext2load usb 0:1 0x1700000 /rescueme 1 || fatload usb 0:1 0x1700000 /rescueme.txt 1; then run rescue_bootcmd; fi
ubifs_mtd=3
ubifs_set_bootargs=setenv bootargs console=$console ubi.mtd=$ubifs_mtd root=ubi0:rootfs rootfstype=ubifs $mtdparts $ubifs_custom_params
ubifs_bootcmd=run ubifs_set_bootargs; if ubi part data && ubifsmount rootfs && ubifsload 0x800000 /boot/uImage && ubifsload 0x1100000 /boot/uInitrd; then bootm 0x800000 0x1100000; fi
usb_scan=usb_scan_done=0;for scan in $usb_scan_list; do run usb_scan_$scan; if test $usb_scan_done -eq 0 && ext2load usb $usb 0x800000 /boot/uImage 1; then usb_scan_done=1; echo "Found bootable drive on usb $usb"; setenv usb_device $usb; setenv usb_root /dev/$dev; fi; done
usb_scan_list=1 2 3 4
usb_scan_1=usb=0:1 dev=sda1
usb_scan_2=usb=1:1 dev=sdb1
usb_scan_3=usb=2:1 dev=sdc1
usb_scan_4=usb=3:1 dev=sdd1
usb_init=run usb_scan
usb_device=0:1
usb_root=/dev/sda1
usb_rootfstype=ext2
usb_rootdelay=10
usb_set_bootargs=setenv bootargs console=$console root=$usb_root rootdelay=$usb_rootdelay rootfstype=$usb_rootfstype $mtdparts $usb_custom_params
usb_boot=mw 0x800000 0 1; ext2load usb $usb_device 0x800000 /boot/uImage; if ext2load usb $usb_device 0x1100000 /boot/uInitrd; then bootm 0x800000 0x1100000; else bootm 0x800000; fi
ethaddr=xx:xx:xx:xx:xx:xx
rescue_installed=1
serverip=xxx
ipaddr=xxx
if_netconsole=ping $serverip
start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;
preboot=run if_netconsole start_netconsole
usb_bootcmd=run usb_init; setenv usb_root LABEL=rootfs; run usb_set_bootargs; run usb_boot
arcNumber=3089
sata_boot=ide reset; mw 0x800000 0 1; ext2load ide 0:1 0x800000 /boot/uImage; if ext2load ide 0:1 0x1100000 /boot/uInitrd; then bootm 0x800000 0x1100000; else bootm 0x800000; fi
sata_bootcmd=run usb_set_bootargs; run sata_boot
bootcmd=usb start; run force_rescue_bootcmd; run ubifs_bootcmd; run usb_bootcmd; usb stop; run sata_bootcmd; run rescue_bootcmd; run pogo_bootcmd; reset

right now my usb drive is at /dev/sdc with partition 1 labelled rootfs, my hd is at /dev/sda with partition 1 labelled rootfs_hd

cheers for any guidance

d2a,

Quote

However, i'd like to move the bootfs onto one of the sata disks

So if the HDD is the boot drive, then this is wrong:

Quote

usb drive is at /dev/sdc with partition 1 labelled rootfs, my hd is at /dev/sda with partition 1 labelled rootfs_hd

The HDD /dev/sda1 should be labeled rootfs. And note that the other USB drives should not have any old rootfs on it (because rootfs contains /boot/uImage and /boot/uInitrd).

The simple rule: the boot partition should be the only one with the kernel files /boot/uImage, /boot/uInitrd, and the rootfs partition should be labeled rootfs(normally, there is a single partition that satisfies these conditions, but sometime people have 2 partitions).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Ah! that's what I get for following the guide too slavishly. of course - thanks. the rootfs_hd partition is a direct copy of the USB stick contents, thus the only other disk containing kernel files, so once i've changed the partition label, shutdown, removed the USB stick, and rebooted all should be well. i'll go try...

...perfect! worked like a charm. thanks again for the help.

incidentally, am i running the most up-to-date kernel for this device? i'd like to move to a more recent openmediavault version but would hate to lose a working setup for the sake of an update

D2a,

Not the latest. But I recall some users reported pproblem with OMV on 3.16, search for their posts, I think 3.14 or 3.15 is ok, but not sure.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi guys,

Sorry its a been a while (working sucks !).
Anyway... got myself a few different USB sticks, including a brand new Sandisk Cruzer 4GB Blade that arrived this morning. Tried them all, but
I still get exactly same results: "Starting kernel...", flashing green light, and then nothing

Any last thoughts before I rip this thing apart and use it as a spare hard drive ?

Thanks,
Ian

IanD

Quote

Any last thoughts before I rip this thing apart and use it as a spare hard drive ?

Don't :) Your problem is simple to fix. It is a rootfs mounting problem.

Since you already have the new rootfs on the Cruzer Blade, do these:

- label the rootfs partition rootfs, using tune2fs -L on another Linux box (assuming the Cruzer Blade was mounted as sdb1):

tune2fs -L rootfs /dev/sdb1

- Plug in the the USB , power up.

- interrupt netconsole and change the envs:

setenv usb_bootcmd 'run usb_init; run usb_set_bootargs; run usb_boot'
setenv usb_init 'run usb_scan; setenv usb_root LABEL=rootfs'
setenv usb_rootfstype

- and continue booting

boot

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 10/07/2014 04:00AM by bodhi.

Hi Bodhi,

I cant thank you enough for your continued assistance...

I ran the tune2fs as advised (2nd cmd below):

root@BlackBox:/media# fdisk -l /dev/sde

Disk /dev/sde: 4004 MB, 4004511744 bytes
124 heads, 62 sectors/track, 1017 cylinders, total 7821312 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0005413b

   Device Boot      Start         End      Blocks   Id  System
/dev/sde1   *        2048     6797311     3397632   83  Linux
/dev/sde2         6797312     7821311      512000   82  Linux swap / Solaris

root@BlackBox:/media# tune2fs -L rootfs /dev/sde1
tune2fs 1.42.9 (4-Feb-2014)
root@BlackBox:/media#

The booted up the Goflex and ran the netconsole commands (note, only usb drive plugged in, no hard drive):

GoFlexNet> setenv usb_bootcmd 'run usb_init; run usb_set_bootargs; run usb_boot'
setenv usb_bootcmd 'run usb_init; run usb_set_bootargs; run usb_boot'
GoFlexNet> setenv usb_init 'run usb_scan; setenv usb_root LABEL=rootfs'
setenv usb_init 'run usb_scan; setenv usb_root LABEL=rootfs'
GoFlexNet> setenv usb_rootfstype
setenv usb_rootfstype
GoFlexNet> boot
boot
(Re)start USB...
USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 2 USB Device(s) found
       scanning bus for storage devices... 1 Storage Device(s) found
Loading file "/rescueme" from usb device 0:1 (usbda1)
** File not found /rescueme
reading /rescueme.txt

** Unable to read "/rescueme.txt" from usb 0:1 **
Creating 1 MTD partitions on "nand0":
0x000002500000-0x000010000000 : "mtd=3"
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: smallest flash I/O unit:    2048
UBI: sub-page size:              512
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI error: validate_ec_hdr: bad VID header offset 2048, expected 512
UBI error: validate_ec_hdr: bad EC header
UBI error: ubi_io_read_ec_hdr: validation failed for PEB 0
UBI error: ubi_init: cannot attach mtd1
UBI error: ubi_init: UBI error: cannot initialize UBI, error -22
UBI init error 22
Loading file "/boot/uImage" from usb device 0:1 (usbda1)
1 bytes read
Found bootable drive on usb 0:1
Loading file "/boot/uImage" from usb device 0:1 (usbda1)
2756040 bytes read
Loading file "/boot/uInitrd" from usb device 0:1 (usbda1)
6032465 bytes read
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   Linux-3.16.0-kirkwood-tld-2
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    2755976 Bytes = 2.6 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   initramfs-3.16.0-kirkwood-tld-2
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    6032401 Bytes = 5.8 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

And again, the light goes green on the front. I waited >10 mins but nothing else happens (nmap & router have no new devices showing)

-------------------------------------------------------------------

Some background info on how I created the USB (in case I'm just being stoopid):
I created the root (ext3) & swap partitions on the Cruzer Blade (/dev/sde) using GParted, and set the boot flag
I then ran these commands from the command line:

mount -t auto /dev/sde1 /media/debian
cd /media/debian
tar -xjf /home/ian/Downloads/Debian-3.16.0-kirkwood-tld-2-rootfs-bodhi.tar.bz2
cd /
sync
umount /media/debian

Lastly, here's a complete Uboot listing again, including PRINTENV output, in case it sheds any light:

root@pi:/home/ian# ncat -u 192.168.0.10 6666

U-Boot 2011.12 (Apr 18 2012 - 23:08:20)
Seagate GoFlexNet
arm-none-linux-gnueabi-gcc (Sourcery G++ Lite 2009q3-67) 4.4.1
GNU ld (Sourcery G++ Lite 2009q3-67) 2.19.51.20090709
Hit any key to stop autoboot:  0 
(Re)start USB...
USB:   Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 1 USB Device(s) found
       scanning bus for storage devices... 0 Storage Device(s) found
** Block device usb 0 not supported

** Invalid boot device **
Creating 1 MTD partitions on "nand0":
0x000002500000-0x000010000000 : "mtd=3"
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size:   131072 bytes (128 KiB)
UBI: logical eraseblock size:    129024 bytes
UBI: smallest flash I/O unit:    2048
UBI: sub-page size:              512
UBI: VID header offset:          512 (aligned 512)
UBI: data offset:                2048
UBI error: validate_ec_hdr: bad VID header offset 2048, expected 512
UBI error: validate_ec_hdr: bad EC header
UBI error: ubi_io_read_ec_hdr: validation failed for PEB 0
UBI error: ubi_init: cannot attach mtd1
UBI error: ubi_init: UBI error: cannot initialize UBI, error -22
UBI init error 22
** Block device usb 0 not supported
** Block device usb 1 not supported
** Block device usb 2 not supported
** Block device usb 3 not supported
** Block device usb 0 not supported
** Block device usb 0 not supported
Wrong Image Format for bootm command
ERROR: can't get kernel image!
stopping USB..
### JFFS2 loading 'uboot-original-mtd0.kwb' to 0x800000
Scanning JFFS2 FS:   read_nand_cached: error reading nand off 0xfb8200 size 8192 bytes
read_nand_cached: error reading nand off 0x13b0200 size 8192 bytes
read_nand_cached: error reading nand off 0x2279600 size 8192 bytes
read_nand_cached: error reading nand off 0x227b000 size 8192 bytes
 done.
find_inode failed for name=uboot-original-mtd0.kwb
load: Failed to find inode
### JFFS2 LOAD ERROR<0> for uboot-original-mtd0.kwb!
### JFFS2 loading 'uboot-original-mtd0.kwb' to 0x800000
Scanning JFFS2 FS:   read_nand_cached: error reading nand off 0xfb8200 size 8192 bytes
read_nand_cached: error reading nand off 0x13b0200 size 8192 bytes
read_nand_cached: error reading nand off 0x2279600 size 8192 bytes
read_nand_cached: error reading nand off 0x227b000 size 8192 bytes
 done.
find_inode failed for name=uboot-original-mtd0.kwb
load: Failed to find inode
### JFFS2 LOAD ERROR<0> for uboot-original-mtd0.kwb!
resetting ...

U-Boot 2011.12 (Apr 18 2012 - 23:08:20)
Seagate GoFlexNet
arm-none-linux-gnueabi-gcc (Sourcery G++ Lite 2009q3-67) 4.4.1
GNU ld (Sourcery G++ Lite 2009q3-67) 2.19.51.20090709
Hit any key to stop autoboot:  9 
 8 

 0 
GoFlexNet> 

GoFlexNet> printenv
printenv
arcNumber=3338
baudrate=115200
bootcmd=usb start; run force_rescue_bootcmd; run ubifs_bootcmd; run usb_bootcmd; usb stop; run rescue_bootcmd; run pogo_bootcmd; reset
bootdelay=10
console=ttyS0,115200
ethact=egiga0
ethaddr=02:50:43:1e:91:fa
force_rescue=0
force_rescue_bootcmd=if test $force_rescue -eq 1 || ext2load usb 0:1 0x1700000 /rescueme 1 || fatload usb 0:1 0x1700000 /rescueme.txt 1; then run rescue_bootcmd; fi
if_netconsole=ping $serverip
ipaddr=192.168.0.10
led_error=orange blinking
led_exit=green off
led_init=green blinking
mainlineLinux=yes
mtdids=nand0=orion_nand
mtdparts=mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
ncip=192.168.0.3
partition=nand0,2
pogo_bootcmd=if fsload uboot-original-mtd0.kwb; then go 0x800200; fi
preboot=run if_netconsole start_netconsole
rescue_bootcmd=if test $rescue_installed -eq 1; then run rescue_set_bootargs; nand read.e 0x800000 0x100000 0x400000; bootm 0x800000; else run pogo_bootcmd; fi
rescue_installed=0
rescue_set_bootargs=setenv bootargs console=$console ubi.mtd=2 root=ubi0:rootfs ro rootfstype=ubifs $mtdparts $rescue_custom_params
serverip=192.168.0.3
start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;
stderr=nc
stdin=nc
stdout=nc
ubifs_bootcmd=run ubifs_set_bootargs; if ubi part data && ubifsmount rootfs && ubifsload 0x800000 /boot/uImage && ubifsload 0x1100000 /boot/uInitrd; then bootm 0x800000 0x1100000; fi
ubifs_mtd=3
ubifs_set_bootargs=setenv bootargs console=$console ubi.mtd=$ubifs_mtd root=ubi0:rootfs rootfstype=ubifs $mtdparts $ubifs_custom_params
usb_boot=mw 0x800000 0 1; ext2load usb $usb_device 0x800000 /boot/uImage; if ext2load usb $usb_device 0x1100000 /boot/uInitrd; then bootm 0x800000 0x1100000; else bootm 0x800000; fi
usb_bootcmd=run usb_init; setenv usb_root LABEL=rootfs; run usb_set_bootargs; run usb_boot
usb_device=0:1
usb_init=run usb_scan
usb_root=/dev/sda1
usb_rootdelay=10
usb_rootfstype=ext3
usb_scan=usb_scan_done=0;for scan in $usb_scan_list; do run usb_scan_$scan; if test $usb_scan_done -eq 0 && ext2load usb $usb 0x800000 /boot/uImage 1; then usb_scan_done=1; echo "Found bootable drive on usb $usb"; setenv usb_device $usb; setenv usb_root /dev/$dev; fi; done
usb_scan_1=usb=0:1 dev=sda1
usb_scan_2=usb=1:1 dev=sdb1
usb_scan_3=usb=2:1 dev=sdc1
usb_scan_4=usb=3:1 dev=sdd1
usb_scan_list=1 2 3 4
usb_set_bootargs=setenv bootargs console=$console root=$usb_root rootdelay=$usb_rootdelay rootfstype=$usb_rootfstype $mtdparts $usb_custom_params

Environment size: 2626/131068 bytes
GoFlexNet>

(interestingly, I found I can only get to the GoflexNet (? I have a GoFlex Home) command prompt if I let uBoot do a full cycle first)

Cheers,
Ian

Ian,

Did you run everything, especiall the tar command, as root user? Must be root.

This is actually GFNet u-boot , but should work fine for GFHome.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi Bodhi,

Yes, everything was done as root.

Ian,

If you've done what I suggested above, try looking for the plug in the network. Netconsole does not display more output than "Starting Kernel ….".

If no sight of the plug, your last resort is serial console:
http://forum.doozan.com/read.php?8,13263

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hello all, my name is Tolis and after a damaged backup disk I finally decided it's time to build a cheap NAS and I found the goflex net "solution" and Don's awesome article. I've already ordered one and I have a new disk, which has already replaced my damaged one. I'd like to ask a few questions though:

1. Is it possible to do this procedure in a windows environment? I have some experience with linux, but I'm currently on a laptop without direct access to a linux pc.
2. Is it possible to install the kernel etc in the internal memory? So that it boots from there without needing a usb drive or being installed in the disk.
3. if 2 is possible, could I use my whole disk with NTFS partition without a reformat/repartition, and then buy a second disk to use a RAID1 array? if it's not, then can I boot from the hdd, have the data partition in NTFS and a RAID1 system when I later install a 2nd disk?

My goal is to make a small case with the goflex, 2x 3.5" 1TB disks (with an internal larger power supply), a slow fan and maybe a status lcd display.

TOLiS_GR,

> 1. Is it possible to do this procedure in a
> windows environment? I have some experience with
> linux, but I'm currently on a laptop without
> direct access to a linux pc.

Not possible. u-boot installation must be done on the GFNet itself. However, linux rootfs can be preprared on a Windows PC running Linux Live CD.

> 2. Is it possible to install the kernel etc in the
> internal memory? So that it boots from there
> without needing a usb drive or being installed in
> the disk.

Yes. However, you should install on USB first. Get it running and then move on to NAND install.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Thank you bodhi! what worries me is that the GFNet has 256MB flash but everything I read seems to use much more. e.g. 4GB usb stick, 4GB partition on the HDD etc.

Anyway, thanks again! I'm waiting for the GFNet to arrive and start playing :)

@bodhi, thanks for answering all the questions :D

@IanD, bodhi's had a good try at solving the issue, I hope you can get it sorted. Mainly my experience has been persistence, try everything you can think of, then try what you didn't think of first ... eventually it'll resolve ... but many hours may be expended between A and B

I had many issues with flash drives. This is what my drive looks like - http://skatter.com/2009/09/sandisk-cruzer-flash-drive-review/ ... keep on trying different configs until one works :D

I suggested the davygravy rootfs image in my post, as this is the first one that worked reliably for me. So, not wishing to step on bodhi's toes, more trying to help with alternatives, but have you tried this image ?

I also experienced that sometimes I'd power-up and it wouldn't boot. However simply power cycling would boot the device. So don't give up on the first attempt of non-booting ! ... Booting can also be delayed by fsck I think, so good idea to give the device a while before giving up on that attempt.

@TOLiS_GR - I'm a bit confused by the last message. The GFN has 256Mb of flash, and in order to "simply" install debian a 2GB stick or more is probably required. These days flash sitcks are cheap, so confused about what you're asking ... Overall I'd recommend installing debian to Hard Drive as early as possible, as I have little confidence in the longevity of the flash sticks for OS install. If you must use a flash stick, buy a second one and duplicate the working one keep it ready as a spare for when the main one dies. Depending on your level of experience, it's possible I believe to get debian to run from the inbuilt NAND, which is a neat solution. But personally, it's a little more involved than I currently want to do.

@ ALL - after posting the post, I have to report that my GoFlex NET NAS is still working as per the config I described. A couple of notes :
1. Please make sure you set the correct arcNumber, GFN is 3089. If memory serves you'll need to do this in step 3 of my guide, after installing the new uboot and prior to rebooting. Davygravy's NSA320 post says how to do it, fw_setenv I think :D
2. If you've setup raid, don't forget to add the SWAP partitions to your fstab file. If you don't and you have a power outage (or pull the plug) you won't be able to boot the device. Reason is that when you restore the power, the raid is marked has been marked as dirty. The boot process will try to do a fsck on boot, and the 128Mb memory in the device isn't enough to run the fsck. If you've added your swap partitions in fstab, then it will have enough memory :D ... many persistent hours expended fixing that one !
3. I added netconsole kernel support as described on these forums, which can help with debugging. From memory davygravy's kernel has it enabled, so just a matter of configuring it. Also a good idea to turn on linux boot logging, so at least you can examine the boot messages on another linux system (or via bootable media on same device), in case of non-booting. This is how I eventually found the problem and solution for (2.)
4. I'd recommend, as said already - getting rid of the flash drive at your earliest opportunity. I still have the flash drive in case the device won't boot from HDD. It's how I was able to fix any issues, as the rescue system I used doesn't support HDD ...
5. These forums have 99% of the solutions for anything, so search persistently and you'll generally find the solution here. Google is a powerful tool, sometimes I've found help in archlinux forums, debian help pages and elsewhere. eg Google - "Doozan netconsole kernel support"

My current project is to make a similar build on my second GFN with MySQL installed and use the device as a replicant hot-backup over the network for a database running on another machine. Debian is very powerful and adaptable :D

Good luck in your Debian'ising ...

Cheers

DC

@Don,

Sqlite might be a better choice with less memory footprint.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

PS @TOLiS_GR - if you install Debian on GFN internal NAND, then yes you could use the entire space on the disk. Linux will support reading/writing NTFS disk, although as I recall not recommended for performance. However if you're planning to keep it NTFS so you can easily connect to a windows machine, then might be a good reason for keeping it NTFS.

Linux won't do RAID on NTFS, NTFS is a Microsoft filesystem, Linux often uses ext2, ext3, ext4. For RAID on linux using mdadm it's a special RAID partition type id="fd", onto which you put ext3 or 4 (usually) ... So, YOU HAVE TO repartition if you want to use linux RAID. The alternative would be to use something like rsync to mirror the data, but that's not the same as RAID, and has it's advantages and disadvantages.

And lastly don't make the mistake I've often made being cheap on disks. If you're setting up for instance a 2xHDD RAID 1 setup, don't try to set it up with the data on one of the drives. Set it up with two disks that don't have any data, copy your data in, then use the original disk as a backup. Being cheap on drives seems good up until the point you lose your data, then you'll wonder why you didn't buy another disk ... obvious, been there done that, got the "t" shirt :D

Cheers

DC

Heh.. the reason I asked if I could run everything from the internal NAND is mainly to avoid having to make partitions to the disks and get rid of the USB , it's mostly a "convenience" and a way to keep things neat than anything else. But the low amount of internal NAND makes me sceptical, I'll try it and report here :) (I'm waiting for my GFN to arrive, should be here after the weekend). I guess I could also use a 2.5" hdd in the usb port to run the os etc from and keep the sata disks only for data - again to avoid using a USB flash and keeping the data disks "simple".

As fas as the disks go, I currently have a WD Blue 1TB 3.5" in an external case, and a Seagate 1TB 3.5" (can't remember which) in another external case, mirrored. They are both backups as my data is in my internal drive. I got a WD disk as all three of my previous failed disks where seagate, 2.5". My plan is to use 2 new 1TB in a RAID array in the NAS, and also keep my other 1TB as a backup.

"If you're setting up for instance a 2xHDD RAID 1 setup, don't try to set it up with the data on one of the drives. Set it up with two disks that don't have any data, copy your data in, then use the original disk as a backup." <- That's exactly what I'm gonna do! :)

Now as fas as the file system goes.. I'd like to use NTFS just to keep things simpler (for me) as I could install a disk in my pc, but if linux doesn't do RAID in NTFS I'll have to go for ext3/4.

At the moment data protection is my first concern, speed/performance comes second. I've been using a USB2 connection and my router has 100Mbit LAN so I guess I won't see any significant speed gains before changing my router :)

TOLiS_GR,

When I said "move on the NAND install", it sounds simple but it is not :) Most likely you will have to install Emdebian to even have a chance of fiiting the kernel and rootfs on it.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

It's never easy :) but I have some experience with embedded systems (it's kindah my job, but on a different field, I guess it's time to learn linux too), I just finished reading Jeff's How-To.

What I can't find is space requirements for the OpenMediaVault, would it fit? If my target is a local network NAS without any special features - just a raid1 "disk" on my local network do I really need it or could I just "share" the disks on the net using Emdebian.

"Worst" case scenario, I could use a small "real" hdd in the usb port and just don't care about neither the space nor the longevity of a usb flash disk.

Really simple if your boot drive is a USB thumb drive:
http://forum.doozan.com/read.php?2,13702,13704#msg-13704

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

That's pretty interesting too :) !! I guess I'll just have to wait and start "working" as soon as my GFNet arrives!

Till then I'll keep reading the forum!

As there's not a "thanks" button, thanks to all of you :)

@bohdi, thanks dude ... but ... I'm using MySQL for the main database and I thought replication same version to same version would be easiest. The GoFlex Net is a slave, so all it has to do is repeat the updates ... will look more into SQLLite, thanks for suggestion.

Currently compiling (and fixing bugs) MySQL 5.6.12 on GoFlex Net ... maybe I'll end up with a working executable, fingers crossed !

Also thanks for posting the post about kernel netconsole ... it works great, as I have no serial cable facility :D I can also report that the windows version of netconsole, does allow interruption of boot ... I thought it didn't because of what Jeff said, then yesterday I pressed ESC (I think) and it paused and allowed me to type commands ... so bonus !

And thanks for all your work on the kernels, I will at some point when I'm brave enough have to have a go at a 3.16 or 3.17 ... wanted to ask if anyone's had any success with anything beyond wheezy on GFN ? ... or if indeed it's worth the effort, considering Wheezy is current stable release.

@TOLiS_GR you're welcome dude ... I like your optimism with regards debian + OMV on NAND, however as bodhi's pointed out subtly, I think that's a task and a half, and then some ... looking forward to your post on how to do it actually, definitely like to have a go at it :D ... my initial idea was something like QNAP have in their NAS'es without the price tag. OMV + debian is quite close ... wish you the best of luck anyway ...

Cheers

DC

Well, according to Jeff, emdebian needs 72MB on UBIFS, if OMV can fit in the remaining it could be done I guess, if not..

we could install a bigger nand :) from the pictures of the board I can only assume that the BGA chip near the marvell soc is the RAM, so on the other side is the NAND, being smd is something I can work with. Just had a quick look in the marvell's datasheet and it seems that it can boot from multiple sources. If it is possible to boot from the usb we can then repartition the new nand, install uboot there etc. I worked with an ARM7 board once upon a time that could boot from two different memory chips and that was the way to update uboot there, so I guess this may be doable.

Of course, it's a long shot but the board is cheap, nand chips are cheap.. I guess that's a whole new subject :)

I'll keep you posted as soon as I have anything to share :)

@TOLiS_GR I'm guessing the 72MB is for the absolute core stuff Debian needs to exist ... my Debian installation, with all the packages etc is now running into the GBs ... so you'll probably need some OS storage outside of the NAND if you want OMV and debian that actually does stuff apart from exist ... with the caveat that I tend to keep stuff almost indefinitely, so there's probably plenty of logs etc that could be purged !

OMV installs quite a lot of packages, some are necessary, others aren't ... I'm considering dumping "collectd" as it just sits there using a little CPU and memory, in order that one has fancy graphs in OMV ... there's probably other components that aren't necessary ...

Generally I use OMV to save on admin headache, but the more I'm using linux/debian the more capable I'm getting to config everything via the terminal. Also there are other admin tools for linux that might be lighter weight, perhaps run on lighthttpd with a simpler web ui. That might make life easier cramming everything into the NAND ...

As for upgrading the NAND, fun job for someone with the tools, however I have a $6/£4 soldering iron and basic electronics knowledge, so I probably won't go down that route. If I really wanted a reliable solution on NAND, then I'd consider industrial USB NAND or SSD into one of the goflex ports. Domestic flash drives, SD cards etc simply aren't designed for day-in day-out usage for an OS ... In my opinion obviously, bodhi's already shared a link which tells how to reduce IO to a minimum to hopefully extend the flash drive's lifetime.

I've no way of confirming this, but the SATA ports on the GoFlexNet may have port mulitplier capability. If so a multiplexer unit can be bought which would bring 5 drives to each SATA port. UK has Span who sell these multipliers, worldwide elsewhere I don't know where you'd buy. I've used similar technology in the past, wasn't overly impressed with it, I have to say - it seems a cheap imitation of the multiplexing capabilities of SAS ports OR dedicated RAID cards like Adaptec sell. As for booting of one of the port multiplied drives, I have no idea ...

What has worked for me is to split up the hard drive (as already described). Currently my new build has 8GB for Debian, 4GB for swap and then the rest is MDADM RAID ... same on both drives, so I get 8GB of swap which Debian can RAID itself, I just need to tell it where the swap is. One disk has rootfs in the first partition and the other is a "spare". I'm not confident or comfortable enough yet to RAID the rootfs, I think it's probably an extra headache and complication not needed. My point is that where I have RAID partitions you could use NTFS if you'd like to be able to direct connect the drive to a windows machine, the 12GB used by debian and it's swap don't really eat into a 1TB drive that much ... just don't let windows try to format those partitions, when you do hook it up to win7 ...

And before you say - "why do you need so much swap" ... For swap sizing I read many conflicting articles, in the end decided "make it big" and let debian decide what it wants to do with it ... downsides for large swap are greedy processes that run at a snails pace, upside is thinks like my fsck that I can't see going on will actually complete :D ... also I can monitor the box with top, so greedy processes need either uninstalling or configuring to use less memory ...

Looking forward to see how you get on, hopefully I'll learn something :D

Cheers

DC

@ALL ... I just upgraded to wheezy on my GoFlexNet/Debian/OMV box ... main difficulty was the php5-pam ... in brief I couldn't tell you exactly the quickest route to solve, but what I suspect is :

1. apt-get update
2. apt-get build-essential subversion wget dpkg-dev dpkg
2. apt-get upgrade
3. Repeat 2. until all squeeze packages are up to date.
4. Follow the steps on https://wiki.debian.org/InstallOpenMediaVaultOnDebian under the section for wheezy. What you're trying to do is make a php5-pam deb file to install. My svn command got stuck so I had to download a zip from sourceforge and sftp that over to the box, extract and compile the .deb file from that. I included wget above because it's useful if you need to download any other .deb files.
5. Modify /etc/apt/source.list for wheezy
6. apt-get update
7. apt-get upgrade
8. Repeat 7 until all wheezy packages are up to date.
9. apt-get dist-upgrade
10. cat /etc/*-release
11. In your /boot create a backup folder then cp * to it
12. build a new uInitrd file, please make *SURE YOU'VE GOT THIS RIGHT*, for example :
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initrd.img-3.3.2-kirkwood-dg -d /boot/initrd.img-3.3.2-kirkwood-dg /boot/uInitrd
13. sync
14. shutdown -r 0
15. Fingers crossed and you'll be able to ssh back into your new OMV/wheezy box :D

I had to hack my upgrade, because I installed php5-pam after I'd updated my /etc/apt/sources.list and had problems upgrading my packages to wheezy ... so this is why I think it's best to install the new php5-pam from OMV prior to upgrading to wheezy.

Good luck

Cheers

DC

Don Charisma ... because anything is possible with Charisma

My blog - http://DonCharisma.org
Our commercial site - http://DonCharisma.com

Heh.. replacing a NAND ic seems soooooooooo easy right now :)

I don't think you'll learn anything at all from me!!! :) (or anyone else in here.. each topic/post I read is overwhelming with information!!)

Now, as far as the port multipliers go etc, I believe that for more drivers and a bigger NAS money wouldn't be much of a concern and someone could build a better machine e.g. if you go for 6 HDDs you're already talking about a few hundreds euros (I'm in Greece btw, still euro here.. ), in that territory it would be "dumb" to NOT spend another hundred and get something better as far as the controlling solution goes. I'd also use a dedicated SSD for the OS ... but where's the fun :D

So, if OMV is mainly used for the admin, could I make something useful without it? e.g. install the minimum emdebian, then install the needed packaged to provide raid support, lan connectivity etc. (I still wanna try OMV though)

LOL, you seem to share my optimistic dreams, seems easy to dream, but harder to implement ... probably I could upgrade the NAND, but .... what I have is working OK, so if it ain't broken, don't fix it ...

Throw more money at it and yes, almost anything's possible ... I sold a 16 port adaptec pcie x8 raid card, I simply don't need 64TB of storage ... and the cost of 16 disks, well it's too much, like 2300 euros which I don't have ! ... RAID6 hardware RAID is beautifully fast and very solid reliability wise on Adaptec/Seagate setup, but I simply don't need that kind of power, it's overkill ... plus not very portable, which GoFlex Net and a couple of Backup Plus drives is ...

Also had a QNAP pro NAS, which I sold on ... I like that OMV is working towards making an open source version similar to QNAP's interface ... so hence the OMV ...

And LOL, there's plenty I learn all the time from others here and off of Google :D

What I'd suggest is getting up and running on a USB flash drive with OMV, see what it does, how you like it, and get more of a feel for embedded OSes, Debian, linux etc ... I have the same install now running on wheezy as in the above, the main issue is php5-pam on wheezy, as it exists in squeeze but not wheezy, so you have to compile your own ... I'd recommend going wheezy rather than squeeze for the end distro.

It may be possible to scale down the installation to the point it'll fit on GoFlex Net's NAND as it is ... things like busybox (a tiny implementation of a lot of the linux commands) would save a lot of space ... HOWEVER ... what you're attempting means you'd need a good understanding of Debian/Linux/Embedded systems, which you may not have to start ... might be attempting to walk before you can run ... I had similar ideas when I first started, but practically, it's about getting the job done ... and this comes from LONG hours with many challenges along the way behind me ...

Also suggest thinking about what you want the box to do at the end of the day, so you have a goal to work towards and you'll know when you've achieved it ... I generally try to make mine dual purpose, maybe more, depends on what the purposes are ... I just setup a MySQL slave server, it maxes out all the resources whilst data copying in ... once datas been copied in I can use the disk also for storing backups, and perhaps run my downloads with wget on it ... and so on ... a couple of main purposes that don't conflict too much, and then some other little jobs it can do ...

I don't know the best web admin system, OMV is pretty nice for that ... webmin is another option, that might be lighter weight ... happy to be enlightened on what's better or serves varying purposes better ...

OMV is designed to be similar to say QNAP's user interface, which is mainly at the end of the day about admin ... setup samba for instance with menus ... add plugins automagically like dlna or torrents ... and so on ... all stuff that can be done on command line, but often helps to not have to reinvent the wheel every time you want to do something, so a helpful admin UI is often much appreciated !

Anyway, try and keep it simple at first and build slowly on what you learn along the way ...

Cheers

DC

Don Charisma ... because anything is possible with Charisma

My blog - http://DonCharisma.org
Our commercial site - http://DonCharisma.com

I just agree with everything!! :) Gonna start "playing" with a USB flash mounted system to see how small I can make it and then I'll start "worrying" about getting everything in the NAND.

That "busybox" rung a bell.. once I compiled a kernel for a GT2440 board I had around (a mini2440 clone) which I wanted to use as a server.. then I moved out of the country and abandoned everything.. Optimism is the key word.. I WILL make it :) (at least we do try! :D )

But apart from the kernel and the one time I set an ubuntu server pc (both just by following directions on the net) I don't have any serious experience with linux.

I'll keep you posted on the process! for the time being I'm just researching and waiting for the GFNet to arrive.

Busybox is used on openwrt and things like Ubiquiti's airos ... it's very very compact and has a large number of the linux commands ...

And yes, my advice is to have intermediate goals that you can easily reach. For instance if you want to have an easy to administer and maintain NAS with built in UI, boots off internal NAND, then get there first with the USB drive ... then plan what's next and move closer to the ultimate goal :D ... anyway I think you understand that ...

The reason I say this is that it can be extremely time consuming working on linux and embedded systems, especially whilst one is still learning ...

Love to hear of your experiences, and be great if you get a .bin image for it together to share so others could run same system direct from NAND ... And sharing the process of building it, is even more helpful as people can mod it to their own requirements ...

BFN

D

Don Charisma ... because anything is possible with Charisma

My blog - http://DonCharisma.org
Our commercial site - http://DonCharisma.com