Not Booting with 2nd USB attached.

Well I have the latest Debian installed on a flash drive, and it works perfectly. But if I try to boot with a secound USB device attached, it then fails to boot.

This is the error I get

DRAM:  128 MiB
NAND:  128 MiB
In:    serial
Out:   serial
Err:   serial
Net:
Led:    GREEN
mii0
Main Loop
Hit any key to stop autoboot:  0
(Re)start USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 4 USB Device(s) found
       scanning usb for storage devices... 2 Storage Device(s) found
** File not found /boot/uImage **
** File not found /boot/uInitrd **
** File not found /boot/dts/ox820-pogoplug-classic.dtb **
Wrong Image Format for bootm command

Led:    ORANGE (Failed)
ERROR: can't get kernel image!
stopping USB..
resetting ...

My env's look like this

autoload=no
baudrate=115200
bootargs=console=ttyS0,115200n8
bootcmd=run dt_bootcmd_usb
bootdelay=5
console=console=ttyS0,115200n8
dt_bootcmd_usb=usb start; run dt_usb_bootcmd; usb stop; reset
dt_bootm=bootm $uimage_addr $uinitrd_addr $dtb_addr
dt_load_dtb=ext2load usb 0:1 0x62c00000 /boot/dts/ox820-pogoplug-classic.dtb
dt_load_initrd=ext2load usb 0:1 $uinitrd_addr /boot/uInitrd
dt_load_uimage=ext2load usb 0:1 $uimage_addr /boot/uImage
dt_usb_boot=run dt_load_uimage; run dt_load_initrd; run dt_load_dtb; run dt_bootm
dt_usb_bootcmd=run usb_set_bootargs; run dt_usb_boot
dtb_addr=0x62c00000
ethact=mii0
ethaddr=00:25:31:02:4d:03
mtdids=nand0=41000000.nand
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
stderr=serial
stdin=serial
stdout=serial
uimage_addr=0x60500000
uinitrd_addr=0x60e00000
usb_device=0:1
usb_set_bootargs=setenv bootargs console=ttyS0,115200 root=/dev/sda1 rootdelay=10

shatterstar78,

The problem is the rootfs drive was probably not device 0:1

dt_load_initrd=ext2load usb 0:1 $uinitrd_addr /boot/uInitrd
dt_load_uimage=ext2load usb 0:1 $uimage_addr /boot/uImage

When you have 2 USB drives attached, which ever was detected first is assigned 0:1, the other is 1:1. So you will need some logic in the envs to select the correct drive. The default envs are only the basic steps to get it working with one drive.

As an experiment, you can stop the console and do

>usb start
then
>usb info
or 
>usb part

Most likely you will find out that the rootfs drive is 1:1.

I will see if I can post the changes necessary to select the drive.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 08/21/2014 05:27PM by bodhi.

shatterstar78 ,

You should do this:

1. Set the envs to scan USB drives (try this in serial console first, once it booted, set them in Debian using fw_setenv):

usb_device=0:1
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)

usb_scan=usb_scan_done=0;for scan in $usb_scan_list; do run usb_scan_$scan; if test $usb_scan_done -eq 0 && ext2load usb $usb 0x800000 /boot/uImage 1; then usb_scan_done=1; echo "Found bootable drive on usb $usb"; setenv usb_device $usb; setenv usb_root /dev/$dev; fi; done
usb_scan_1=usb=0:1 dev=sda1
usb_scan_2=usb=1:1 dev=sdb1
usb_scan_3=usb=2:1 dev=sdc1
usb_scan_4=usb=3:1 dev=sdd1
usb_scan_list=1 2 3 4

usb_init=run usb_scan; setenv usb_root LABEL=rootfs
usb_set_bootargs=setenv bootargs console=ttyS0,115200 root=$usb_root rootdelay=10 mtdparts=$mtdparts

dt_usb_bootcmd=run usb_init; run usb_set_bootargs; run dt_usb_boot
dt_load_initrd=ext2load usb $usb_device $uinitrd_addr /boot/uInitrd
dt_load_uimage=ext2load usb $usb_device $uimage_addr /boot/uImage

2. Label your USB drive boot partition as rootfs (sdxx is the mounted letter of the drive rootfs partition):

tune2fs -L rootfs /dev/sdxx

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 3 time(s). Last edit at 08/21/2014 05:35PM by bodhi.

I had a similar problem, but sometimes it would boot and sometimes it wouldn't. It's great to get the above information, but I just kept swapping USB cables until my boot drive always came up as 0:1. Seems like an easy thing to try.

-JT

bodhi Wrote:
-------------------------------------------------------
> shatterstar78 ,
>
> You should do this:
>
> 1. Set the envs to scan USB drives (try this in
> serial console first, once it booted, set them in
> Debian using fw_setenv):
>

> usb_device=0:1
> mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
> 
> usb_scan=usb_scan_done=0;for scan in
> $usb_scan_list; do run usb_scan_$scan; if test
> $usb_scan_done -eq 0 && ext2load usb $usb 0x800000
> /boot/uImage 1; then usb_scan_done=1; echo "Found
> bootable drive on usb $usb"; setenv usb_device
> $usb; setenv usb_root /dev/$dev; fi; done
> usb_scan_1=usb=0:1 dev=sda1
> usb_scan_2=usb=1:1 dev=sdb1
> usb_scan_3=usb=2:1 dev=sdc1
> usb_scan_4=usb=3:1 dev=sdd1
> usb_scan_list=1 2 3 4
> 
> usb_init=run usb_scan; setenv usb_root
> LABEL=rootfs
> usb_set_bootargs=setenv bootargs
> console=ttyS0,115200 root=$usb_root rootdelay=10
> mtdparts=$mtdparts
> 
> dt_usb_bootcmd=run usb_init; run usb_set_bootargs;
> run dt_usb_boot
> dt_load_initrd=ext2load usb $usb_device
> $uinitrd_addr /boot/uInitrd
> dt_load_uimage=ext2load usb $usb_device
> $uimage_addr /boot/uImage
> 
>

>
> 2. Label your USB drive boot partition as rootfs
> (sdxx is the mounted letter of the drive rootfs
> partition):
>

> tune2fs -L rootfs /dev/sdxx
>

Thanks for the info, but something is not right. When I try using the above code it hangs at boot with the following..

[    3.675013]  sda: sda1
[    3.683909] sd 1:0:0:0: [sda] Attached SCSI removable disk
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
modprobe: module unknown not found in modules.dep
mount: can't read '/etc/fstab': No such file or directory
Begin: Running /scripts/local-bottom ... done.
done.
Begin: Running /scripts/init-bottom ... mount: mounting /dev on /root/dev failed: No such file or directory
done.
Target filesystem doesn't have requested /sbin/init.
No init found. Try passing init= bootarg.
modprobe: module i8042 not found in modules.dep
modprobe: module atkbd not found in modules.dep
modprobe: module uhci-hcd not found in modules.dep
modprobe: module ohci-hcd not found in modules.dep


BusyBox v1.20.2 (Debian 1:1.20.0-7) built-in shell (ash)
Enter 'help' for a list of built-in commands.

/bin/sh: can't access tty; job control turned off
(initramfs)

I did notice that when setting the following

usb_set_bootargs=setenv bootargs console=ttyS0,115200 root=$usb_root rootdelay=10 mtdparts=$mtdparts

The output looks like this

usb_set_bootargs=setenv bootargs console=ttyS0,115200 root= rootdelay=10 mtdparts=mtdparts=41000000.nand:14m(boot),-(data)

Notice its not setting anything for "root=". Any ideas?
Also what is the proper way to set the env's to look like this "root=$usb_root rootdelay=10 mtdparts=$mtdparts" and not like this "root= rootdelay=10 mtdparts=mtdparts=41000000.nand:14m(boot),-(data)"
Why is it replacing "$mtdparts" with the actual mtdparts?



Edited 2 time(s). Last edit at 08/21/2014 11:09PM by shatterstar78.

Sounds like you've misplaced the single quote somewhere. The correct syntax is:

setenv usb_init 'run usb_scan; setenv usb_root LABEL=rootfs'

The actual parameter to the kernel is mtdparts=41000000.nand:14m(boot),-(data). So the first mtdparts word is a text, the second is the variable (which has $ in front of the name).

setenv mtdparts 'mtdparts=41000000.nand:14m(boot),-(data)'
setenv usb_set_bootargs 'setenv bootargs console=ttyS0,115200 root=$usb_root rootdelay=10 rootfstype=ext3 mtdparts=$mtdparts'

If set correctly, the kernel command line looks like this in Linux

cat /proc/cmdline 
console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 rootfstype=ext3 mtdparts=41000000.nand:14m(boot),-(data)

Note: you can verify it in serial console by executing each command like like this

> setenv usb_init 'run usb_scan; setenv usb_root LABEL=rootfs'
> run usb_init
> printenv usb_root

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 08/22/2014 12:17AM by bodhi.

bodhi Wrote:
-------------------------------------------------------
> Sounds like you've misplaced the single quote
> somewhere. The correct syntax is:
>
>

> setenv usb_init 'run usb_scan; setenv usb_root
> LABEL=rootfs'
>

>
> The actual parameter to the kernel is
> mtdparts=41000000.nand:14m(boot),-(data).
> So the first mtdparts word is a text, the
> second is the variable (which has $ in front of
> the name).
>
>

> setenv mtdparts
> 'mtdparts=41000000.nand:14m(boot),-(data)'
> setenv usb_set_bootargs 'setenv bootargs
> console=ttyS0,115200 root=$usb_root rootdelay=10
> rootfstype=ext3 mtdparts=$mtdparts'
>

>
> If set correctly, the kernel command line looks
> like this in Linux
>

> cat /proc/cmdline 
> console=ttyS0,115200 root=LABEL=rootfs
> rootdelay=10 rootfstype=ext3
> mtdparts=41000000.nand:14m(boot),-(data)
>

>
> Note: you can verify it in serial console by
> executing each command like like this
>
>

> > setenv usb_init 'run usb_scan; setenv usb_root
> LABEL=rootfs'
> > run usb_init
> > printenv usb_root
>

Yeah you were right I had a typo, used (") instead of ('). So now it works, but its odd. If the storage drive is plugged in the front usb, then the OS flash drive can be plugged into any of the back 3 usb slots and it works. But if the storage drive is plugged into any of the back usb slots then it will not work. The one exception is the usb slot right obove the network plug. So basically if the OS flash drive is plugged into the USB slot above the network port, then you can have a storage drive in any other USB port and still boot. Is this how it's supposed to work? I mean it's fine if it is, I can deal with it, but I was just wondering.

shatterstar,

Quote

If the storage drive is plugged in the front usb, then the OS flash drive can be plugged into any of the back 3 usb slots and it works. But if the storage drive is plugged into any of the back usb slots then it will not work. The one exception is the usb slot right obove the network plug. So basically if the OS flash drive is plugged into the USB slot above the network port, then you can have a storage drive in any other USB port and still boot

Is the OS drive the only one with rootfs partition label? To get the rootfs mounted successfully, there must be only one partition with this label.

And the behavior you saw is normal. Usually, the USB port right next to the ethernet port is the one that will be detected firts and assigned as sda, if all the drives are of the same brand and speed. But if they are different brands and aslo one flash type, other HDD type… then the enumeration order is unpredictable.

So if you watch the boot log in serial console, there migh have been some timeout errors when the drives were scanned. If there is no timeout error, then I think you still have problem with the rootfs partition label not set up correctly.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> shatterstar,
>
>

Quote

If the storage drive is plugged in the
> front usb, then the OS flash drive can be plugged
> into any of the back 3 usb slots and it works. But
> if the storage drive is plugged into any of the
> back usb slots then it will not work. The one
> exception is the usb slot right obove the network
> plug. So basically if the OS flash drive is
> plugged into the USB slot above the network port,
> then you can have a storage drive in any other USB
> port and still boot

>
> Is the OS drive the only one with rootfs partition
> label? To get the rootfs mounted successfully,
> there must be only one partition with this label.
>
>
> And the behavior you saw is normal. Usually, the
> USB port right next to the ethernet port is the
> one that will be detected firts and assigned as
> sda, if all the drives are of the same brand and
> speed. But if they are different brands and aslo
> one flash type, other HDD type… then the
> enumeration order is unpredictable.
>
> So if you watch the boot log in serial console,
> there migh have been some timeout errors when the
> drives were scanned. If there is no timeout error,
> then I think you still have problem with the
> rootfs partition label not set up correctly.

Yes the OS drive is the only one with rootfs partition label. Watching the boot log gives no timeout errors. I have tried several drives all different brands and sizes. Besides the OS drive none of the other drives have lables, but they are all formatted to ext3, if that makes any difference. This is my settings now..

baudrate=115200
bootargs=console=ttyS0,115200n8
bootcmd=run dt_bootcmd_usb
bootdelay=5
console=console=ttyS0,115200n8
dt_bootcmd_usb=usb start; run dt_usb_bootcmd; usb stop; reset
dt_bootm=bootm $uimage_addr $uinitrd_addr $dtb_addr
dt_load_dtb=ext2load usb 0:1 0x62c00000 /boot/dts/ox820-pogoplug-classic.dtb
dt_load_initrd=ext2load usb $usb_device $uinitrd_addr /boot/uInitrd
dt_load_uimage=ext2load usb $usb_device $uimage_addr /boot/uImage
dt_usb_boot=run dt_load_uimage; run dt_load_initrd; run dt_load_dtb; run dt_bootm
dt_usb_bootcmd=run usb_init; run usb_set_bootargs; run dt_usb_boot
dtb_addr=0x62c00000
ethact=mii0
ethaddr=00:25:31:02:4d:03
mtdids=nand0=41000000.nand
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
stderr=serial
stdin=serial
stdout=serial
uimage_addr=0x60500000
uinitrd_addr=0x60e00000
usb_device=0:1
usb_init=run usb_scan; setenv usb_root LABEL=ROOTFS
usb_scan=usb_scan_done=0;for scan in $usb_scan_list; do run usb_scan_$scan; if test $usb_scan_done -eq 0 && ext2load usb $usb 0x800000 /boot/uImage 1; then usb_scan_done=1; echo "Found bootable drive on usb $usb"; setenv usb_device $usb; setenv usb_root /dev/$dev; fi; done
usb_scan_1=usb=0:1 dev=sda1
usb_scan_2=usb=1:1 dev=sdb1
usb_scan_3=usb=2:1 dev=sdc1
usb_scan_4=usb=3:1 dev=sdd1
usb_scan_list=1 2 3 4
usb_set_bootargs=setenv bootargs console=ttyS0,115200 root=$usb_root rootdelay=10 rootfstype=ext3 mtdparts=$mtdparts

Is the disk parttion label ROOTFS or rootfs?

e2label /dev/sda1

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 08/24/2014 04:04PM by bodhi.

bodhi Wrote:
-------------------------------------------------------
> Is the disk parttion label ROOTFS or
> rootfs?
>

> e2label /dev/sda1
>

It's ROOTFS on both the settings and the OS drive.

I think there is something that we're not seeing. You should post the output of:

When it booted:
- fw_printenv
- cat /etc/fstab (of the rootfs drive)
- mount

When it failed to boot:
- The entire serial boot log, from the banner until it fails to mount rootfs.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

OK, this is when it's booted..

root@POGOPLUG:~# fw_printenv
baudrate=115200
bootargs=console=ttyS0,115200n8
bootcmd=run dt_bootcmd_usb
bootdelay=5
console=console=ttyS0,115200n8
dt_bootcmd_usb=usb start; run dt_usb_bootcmd; usb stop; reset
dt_bootm=bootm $uimage_addr $uinitrd_addr $dtb_addr
dt_load_dtb=ext2load usb 0:1 0x62c00000 /boot/dts/ox820-pogoplug-classic.dtb
dt_load_initrd=ext2load usb $usb_device $uinitrd_addr /boot/uInitrd
dt_load_uimage=ext2load usb $usb_device $uimage_addr /boot/uImage
dt_usb_boot=run dt_load_uimage; run dt_load_initrd; run dt_load_dtb; run dt_bootm
dt_usb_bootcmd=run usb_init; run usb_set_bootargs; run dt_usb_boot
dtb_addr=0x62c00000
ethact=mii0
ethaddr=00:25:31:02:4d:03
mtdids=nand0=41000000.nand
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
stderr=serial
stdin=serial
stdout=serial
uimage_addr=0x60500000
uinitrd_addr=0x60e00000
usb_device=0:1
usb_init=run usb_scan; setenv usb_root LABEL=ROOTFS
usb_scan=usb_scan_done=0;for scan in $usb_scan_list; do run usb_scan_$scan; if test $usb_scan_done -eq 0 && ext2load usb $usb 0x800000 /boot/uImage 1; then usb_scan_done=1; echo "Found bootable drive on usb $usb"; setenv usb_device $usb; setenv usb_root /dev/$dev; fi; done
usb_scan_1=usb=0:1 dev=sda1
usb_scan_2=usb=1:1 dev=sdb1
usb_scan_3=usb=2:1 dev=sdc1
usb_scan_4=usb=3:1 dev=sdd1
usb_scan_list=1 2 3 4
usb_set_bootargs=setenv bootargs console=ttyS0,115200 root=$usb_root rootdelay=10 rootfstype=ext3 mtdparts=$mtdparts

fstab on ROOTFS drive..

root@POGOPLUG:~# cat /etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/dev/root      /               ext3    noatime,errors=remount-ro 0 1
tmpfs          /tmp            tmpfs   defaults          0       0

mount...

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=15023,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=12224k,mode=755)
/dev/disk/by-label/ROOTFS on / type ext3 (rw,noatime,errors=remount-ro,user_xattr,acl,barrier=1,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=24440k)
tmpfs on /tmp type tmpfs (rw,relatime)
rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)

and when it fails...

U-Boot SPL 2013.10 (Apr 27 2014 - 19:08:27)
  Boot device: NAND
Attempting to set PLLA to 800 MHz ...
  plla_ctrl0 : 0000030a
  plla_ctrl1 : 00400000
  plla_ctrl2 : 007f0068
  plla_ctrl3 : 00000193

PLLA Set


U-Boot 2013.10-tld-3 (May 15 2014 - 14:54:14) for OXNAS

DRAM:  128 MiB
NAND:  128 MiB
In:    serial
Out:   serial
Err:   serial
Net:
Led:    GREEN
mii0
Main Loop
Hit any key to stop autoboot:  0
(Re)start USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 4 USB Device(s) found
       scanning usb for storage devices... Device NOT ready
   Request Sense returned 02 3A 00
3 Storage Device(s) found
** File not found /boot/uImage **
** Bad device size - usb 1 **
1 bytes read in 156 ms (0 Bytes/s)
Found bootable drive on usb 2:1
7708972 bytes read in 375 ms (19.6 MiB/s)
1913968 bytes read in 206 ms (8.9 MiB/s)
** File not found /boot/dts/ox820-pogoplug-classic.dtb **
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   Linux-3.15.3-oxnas-tld-1
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    7708908 Bytes = 7.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 60e00000 ...
   Image Name:   initramfs-3.15.3-oxnas-tld-1
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    1913904 Bytes = 1.8 MiB
   Load Address: 60000000
   Entry Point:  60000000
   Verifying Checksum ... OK
ERROR: Did not find a cmdline Flattened Device Tree
Could not find a valid device tree
stopping USB..
resetting ...

shatterstar,

I suspect your boot drive is this one:

Quote

scanning usb for storage devices... Device NOT ready

and then

Quote

Found bootable drive on usb 2:1
7708972 bytes read in 375 ms (19.6 MiB/s)
1913968 bytes read in 206 ms (8.9 MiB/s)
** File not found /boot/dts/ox820-pogoplug-classic.dtb **
## Booting kernel from Legacy Image at 60500000 ...
Image Name: Linux-3.15.3-oxnas-tld-1
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 7708908 Bytes = 7.4 MiB
Load Address: 60008000
Entry Point: 60008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 60e00000 ...
Image Name: initramfs-3.15.3-oxnas-tld-1
Image Type: ARM Linux RAMDisk Image (gzip compressed)
Data Size: 1913904 Bytes = 1.8 MiB
Load Address: 60000000
Entry Point: 60000000
Verifying Checksum ... OK
ERROR: Did not find a cmdline Flattened Device Tree
Could not find a valid device tree

It seems even though you have only one drive that has ROOTFS partition label, you also have a rootfs on another drive? by that I meant a rootfs file system that has /boot/uImage? Note the first line:

Quote

Found bootable drive on usb 2:1

and that USB partition does not have the appropriate DTB

Quote

** File not found /boot/dts/ox820-pogoplug-classic.dtb **

Let me clarify a bit about the boot process:

- The problem with booting with multiple drives attached is that it is not possible for U-Boot to know which drive you want to boot with. So the scanning was implemented as above to find the first partition that has a /boot/uImage file and use that as a confirmation.

- The ROOTFS label is for helping the Linux kernel to know which partition contains the actual rootfs. This happens later on in the boot process.

So if you also have a /boot/uImage file on any of the drives other than the boot drive, then the scan detection, as implemented, won't work.

BTW, as note to self, I should implement a better detection scheme in the future:) I've noticed the scanning scheme (which was originally written by Jeff) even though very robust for the Dockstar USB ports, has some limitation, that could be improved in the newer version of u-boot.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 08/25/2014 05:14PM by bodhi.

The thing is all the other drives that I tried using are absolutely blank, freshly formatted to ext3. Just had an idea, im going to make a new OS drive. Its the only thing that has not changed in all these tests. Thinking it may be some compatibility problem with this specific flash drive. I'll let you know if it works.

Edit:
Looks like my theory was incorrect. Tried a new flash drive with a fresh install and latest kernel, but no luck still the same exact behavior. I'm out of ideas.



Edited 1 time(s). Last edit at 08/26/2014 08:19AM by shatterstar78.