Pogo E02 hangs at boot ... again

First off let me apologize for dredging this stinkin' issue up again. But, it seems that every time I touch one of my E02s, something goes wrong and over the last five years or so, it's always been the same thing - it hangs loading the kernel. And if you say it's my fault because I'm using old non-FDT systems, that's probably true. There's never enough time to update anything that works. Si non confectus, non reficiat.

At present all four E02s are down due to hopeful updates of the uboot that didn't fix the original issue and made things much worse than they were when the original problem occurred.

Herewith the information that I do have.

Console log when attempting to boot the web server disk, a SATA drive connected through a USB adapter. This drive was working when disconnected from the host this morning due to an apparent network interface failure.

U-Boot 2017.07-tld-1 (Sep 05 2017 - 00:13:18 -0700)
Pogo E02

SoC:   Kirkwood 88F6281_A0
DRAM:  256 MiB
WARNING: Caches not enabled
NAND:  128 MiB
In:    serial
Out:   serial
Err:   serial
Net:   egiga0
Hit any key to stop autoboot:  0
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found
## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0
loading envs from usb 0 ...
** Bad device usb 0 **
running scan_disk ...
Scan device usb
device usb 0:1
** Bad device usb 0 **
device usb 1:1
** Bad device usb 1 **
device usb 2:1
** Bad device usb 2 **
device usb 3:1
** Bad device usb 3 **
loading uImage ...
** Bad device usb 0 **
loading uInitrd ...
** Bad device usb 0 **
loading DTB /boot/dts/kirkwood-pogo_e02.dtb ...
** Bad device usb 0 **
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   kernel 3.2.0-4-kirkwood
   Created:      1970-01-01   1:32:00 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1608000 Bytes = 1.5 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
Using machid 0xffffffff from environment

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

What's in /boot on that (ext4) drive:

root:/tmp> mount /dev/sda1 /tmp/sda1
[  600.080285] EXT4-fs (sda1): recovery complete
[  600.085591] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
root:/tmp> cd /tmp/sda1/boot
root:/tmp/sda1/boot> ls -l
total 33532
-rw------- 1 root root 2240040 Jan 25  2016 System.map-4.4.0-kirkwood-tld-1
-rw-r--r-- 1 root root  140949 Jan 25  2016 config-4.4.0-kirkwood-tld-1
drwxr-xr-x 2 root root    4096 Jan 26  2016 dts
-rw-r--r-- 1 root root 7778398 Sep  3  2017 initrd.img-4.4.0-kirkwood-tld-1
-rw-r--r-- 1 root root 7511582 Jan 25  2016 linux-headers-4.4.0-kirkwood-tld-1_1.0_armel.deb
-rw-r--r-- 1 root root 3154896 Feb 18  2016 uImage
-rw-r--r-- 1 root root 7179935 Feb 18  2016 uInitrd
-rw------- 1 root root 3154832 Jan 25  2016 vmlinuz-4.4.0-kirkwood-tld-1
-rwxr-xr-ox 1 root root 3154832 Jan 25  2016 zImage-4.4.0-kirkwood-tld-1

Console log when booting from the Patriot 64GB USB flash drive. This drive booted correctly when last used in its pwn host system.

U-Boot 2017.07-tld-1 (Sep 05 2017 - 00:13:18 -0700)
Pogo E02

SoC:   Kirkwood 88F6281_A0
DRAM:  256 MiB
WARNING: Caches not enabled
NAND:  128 MiB
In:    serial
Out:   serial
Err:   serial
Net:   egiga0
Hit any key to stop autoboot:  0
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 3 USB Device(s) found
       scanning usb for storage devices...
Use USB retry period from the environment: 15 second(s)
1 Storage Device(s) found

Partition Map for USB device 0  --   Partition Type: DOS

Part    Start Sector    Num Sectors     UUID            Type
  1     63              12482442        6c623631-01     83 Boot
  2     12482505        803250          6c623631-02     82
  3     13285755        16997253        6c623631-03     83

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0

## Unknown partition table type 0
loading envs from usb 0 ...
** File not found /boot/uEnv.txt **
running scan_disk ...
Scan device usb
device usb 0:1
1 bytes read in 446 ms (0 Bytes/s)
Found bootable drive on usb 0
loading uImage ...
1608064 bytes read in 530 ms (2.9 MiB/s)
loading uInitrd ...
6890741 bytes read in 670 ms (9.8 MiB/s)
loading DTB /boot/dts/kirkwood-pogo_e02.dtb ...
** File not found /boot/dts/kirkwood-pogo_e02.dtb **
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   kernel 3.2.0-4-kirkwood
   Created:      1970-01-01   1:32:00 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    1608000 Bytes = 1.5 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   ramdisk 3.2.0-4-kirkwood
   Created:      1970-01-01   1:32:01 UTC
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    6890677 Bytes = 6.6 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
   Loading Ramdisk to 0f47d000, end 0fb0f4b5 ... OK
Using machid 0xffffffff from environment

Starting kernel ...

Uncompressing Linux... done, booting the kernel.

What's in /boot on that (ext3) drive:

root:/tmp> mount /dev/sda1 /tmp/sda1
[  404.623759] EXT4-fs (sda1): mounting ext3 file system using the ext4 subsystem
[  404.667004] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null
root:/tmp/sda1/boot> ls -l
total 26260
-rw-r--r-- 1 root root 1209040 Jun 11  2013 System.map-3.2.0-4-kirkwood
-rw-r--r-- 1 root root  107530 Jun 11  2013 config-3.2.0-4-kirkwood
-rw-r--r-- 1 root root 6890677 Dec 31  1969 initrd.img-3.2.0-4-kirkwood
-rw-r--r-- 1 root root 1608064 Dec 31  1969 uImage
-rw-r--r-- 1 root root 1608064 Dec 31  1969 uImage.bak
-rw-r--r-- 1 root root 6890741 Dec 31  1969 uInitrd
-rw-r--r-- 1 root root 6890741 Dec 31  1969 uInitrd.bak
-rw-r--r-- 1 root root 1608000 Jun 11  2013 vmlinuz-3.2.0-4-kirkwood

The environment contents on the current E02:

PogoE02> printenv
arcNumber=3542
baudrate=115200
bootargs=console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec
bootcmd_exec=run load_uimage; if run load_initrd; then if run load_dtb; then bootm $load_uimage_addr $load_initrd_addr $load_dtb_addr; else bootm $load_uimage_addr $load_initrd_addr; fi; else if run load_dtb; then bootm $load_uimage_addr - $load_dtb_addr; else bootm $load_uimage_addr; fi; fi
bootcmd_uenv=run uenv_load; if test $uenv_loaded -eq 1; then run uenv_import; fi
bootdelay=10
bootdev=usb
device=0:1
devices=usb
disks=0 1 2 3
dtb_file=/boot/dts/kirkwood-pogo_e02.dtb
ethact=egiga0
ethaddr=52:3b:20:9c:11:51
if_netconsole=ping $serverip
ipaddr=192.168.0.209
led_error=orange blinking
led_exit=green off
led_init=green blinking
load_dtb=echo loading DTB $dtb_file ...; load $bootdev $device $load_dtb_addr $dtb_file
load_dtb_addr=0x1c00000
load_initrd=echo loading uInitrd ...; load $bootdev $device $load_initrd_addr /boot/uInitrd
load_initrd_addr=0x1100000
load_uimage=echo loading uImage ...; load $bootdev $device $load_uimage_addr /boot/uImage
load_uimage_addr=0x800000
machid=ffffffff
mainlineLinux=yes
mtdids=nand0=orion_nand
mtdparts=mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
partition=nand0,2
preboot_nc=run if_netconsole start_netconsole
scan_disk=echo running scan_disk ...; scan_done=0; setenv scan_usb "usb start";  for dev in $devices; do if test $scan_done -eq 0; then echo Scan device $dev; run scan_$dev; for disknum in $disks; do if test $scan_done -eq 0; then echo device $dev $disknum:1; if load $dev $disknum:1 $load_uimage_addr /boot/uImage 1; then scan_done=1; echo Found bootable drive on $dev $disknum; setenv device $disknum:1; setenv bootdev $dev; fi; fi; done; fi; done
scan_usb=usb start
serverip=192.168.0.220
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts $custom_params
start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;
stderr=serial
stdin=serial
stdout=serial
uenv_addr=0x810000
uenv_import=echo importing envs ...; env import -t $uenv_addr $filesize
uenv_init_devices=setenv init_usb "usb start"; for devtype in $devices; do run init_$devtype; done;
uenv_load=run uenv_init_devices; setenv uenv_loaded 0; for devtype in $devices;  do for disknum in 0; do run uenv_read_disk; done; done;
uenv_read=echo loading envs from $devtype $disknum ...; if load $devtype $disknum:1 $uenv_addr /boot/uEnv.txt; then setenv uenv_loaded 1; fi
uenv_read_disk=if test $devtype -eq mmc; then if $devtype part; then run uenv_read;  fi; else if $devtype part $disknum; then run uenv_read; fi;  fi
usb_ready_retry=15

Environment size: 2898/131068 bytes
PogoE02>

BuckNaked,

Let me read your reports and explain what to do if you want to stay with the old non-FDT kernel.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Thanks, bodhi. I owe you another beer.

Do you suppose this is a topic that should go into the wiki, or have non-FDT kernels been abandoned by everybody else?

BuckNaked Wrote:
-------------------------------------------------------
> Thanks, bodhi. I owe you another beer.
>
> Do you suppose this is a topic that should go into
> the wiki, or have non-FDT kernels been abandoned
> by everybody else?

Most have moved on to FDT kernels. Non FDT kernels are still seen in embedded boxes that, use certain legacy apps that cannot be upgraded, or are very low costs.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Buck,


Basically, we are going to tell U-Boot 2017.07-tld-1 to look for a non-existent DTB file so it will take a different path and boot without it. And the non-FDT kernel expects to see arcNumber and machid for the Pogo E02.

The approach below is the easiest way to do this, and also will allow, in the future, the box to be booted with a FDT kernel without touching u-boot envs.

-----

Mount your rootfs on another Linux box. Assuming it is mounted at /media/sdb1. Create the uEnv.txt at /media/sdb1/boot with this content:

cat /media/sdb1/boot/uEnv.txt

dtb_file=/boot/dts/no.dtb
arcNumber=3542
machid=dd6

Save the uEnv.txt and sync

sync

Plug the rootfs back into the Pogo E02 and power up.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 4 time(s). Last edit at 01/19/2018 06:02PM by bodhi.

Update:

At present I am trying to bring up the most recent FDT version on that system's SSD since its firmware has already been flashed to the 2017.07 version. That would hopefully solve the boot problems without doctoring up the non-FDT system drive.

If that fails I'll go back to the non-FDT system and fiddle with it, but as bodhi has noted, non-FDT systems are becoming less than supportable in production mode.

Buck,

> At present I am trying to bring up the most recent
> FDT version on that system's SSD since its
> firmware has already been flashed to the 2017.07
> version. That would hopefully solve the boot
> problems without doctoring up the non-FDT system
> drive.

Good call. It will be easier to maintain the system from now on.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)