Pogoplug V4 fails to boot 4.12.8

I'll start by saying that whoever at Cloud Engines eliminated the "Enable SSH" option should be staked out in the warm, friendly sun to concentrate their sugars for the benefit of our hard-working friends, the ants. Connecting to the serial header on the V4 is a colossal annoyance. Then I went through three USB-to-logic-level serial adapters before finding one that the Pogoplug would condescend to speak to, all the time wondering if I'd fried the board while wiring on the serial port vias.

But I got the new u-boot loaded in, and then as usual everything went straight into the outhouse. I cannot get this board to boot any of my known working images for the E02, and I can't get it to boot the 4.12.8 distributed either. For the E02 images it hangs right after "OK, booting the kernel" and for the 4.12.8 it simply whines that it can't get the kernel image.

I have a working E02 which I'm using to generate the images and boot drives for the V4.

That said, here's the background.

========================= Environment ========================

This is pretty much a standard environment as distributed, with the exceptions
of dtb_file, load_dtb and machid.

Pogov4> printenv
arcNumber=2097
baudrate=115200
bootargs=console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec
bootcmd_exec=run load_uimage; if run load_initrd; then if run load_dtb; then bootm ; else bootm ; fi; else if run load_dtb; then bootm - ; else bootm ; fi; fi
bootcmd_uenv=run uenv_load; if test -eq 1; then run uenv_import; fi
bootdelay=10
bootdev=usb
device=0:1
devices=usb ide mmc
disks=0 1 2 3
dtb_file=/boot/dts/kirkwood-pogoplug_v4.dtb
ethact=egiga0
ethaddr=52:3b:20:9c:11:51
if_netconsole=ping
init_ide=ide reset
init_mmc=mmc rescan
init_usb=usb start
ipaddr=192.168.0.212
led_error=orange blinking
led_exit=green off
led_init=green blinking
load_dtb=echo loading DTB /boot/dts/kirkwood-pogoplug_v4.dtb ...; load usb 0:1 0x1c00000 /boot/dts/kirkwood-pogoplug_v4.dtb
load_dtb_addr=0x1c00000
load_initrd=echo loading uInitrd ...; load usb 0:1 0x1100000 /boot/uInitrd
load_initrd_addr=0x1100000
load_uimage=echo loading uImage ...; load usb 0:1 0x800000 /boot/uImage
load_uimage_addr=0x800000
machid=ffffffff
mainlineLinux=yes
mtdids=nand0=orion_nand
mtdparts=mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
partition=nand0,2
preboot_nc=run if_netconsole start_netconsole
scan_disk=echo running scan_disk ...; scan_done=0; setenv scan_usb "usb start"; setenv scan_ide "ide reset"; setenv scan_mmc "mmc rescan"; for dev in $devices; do if test $scan_done -eq 0; then echo Scan device $dev; run scan_$dev; for disknum in $disks; do if test $scan_done -eq 0; then echo device $dev $disknum:1; if load $dev $disknum:1 $load_uimage_addr /boot/uImage 1; then scan_done=1; echo Found bootable drive on $dev $disknum; setenv device $disknum:1; setenv bootdev $dev; fi; fi; done; fi; done
scan_ide=ide reset
scan_mmc=mmc rescan
scan_usb=usb start
serverip=192.168.0.220
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts $custom_params
start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;
stderr=serial
stdin=serial
stdout=serial
uenv_addr=0x810000
uenv_import=echo importing envs ...; env import -t $uenv_addr $filesize
uenv_init_devices=setenv init_usb "usb start"; setenv init_ide "ide reset"; setenv init_mmc "mmc rescan"; for devtype in $devices; do run init_$devtype; done;
uenv_load=run uenv_init_devices; setenv uenv_loaded 0; for devtype in $devices; do for disknum in 0; do run uenv_read_disk; done; done;
uenv_loaded=0
uenv_read=echo loading envs from $devtype $disknum ...; if load $devtype $disknum:1 $uenv_addr /boot/uEnv.txt; then setenv uenv_loaded 1; fi
uenv_read_disk=if test $devtype -eq mmc; then if $devtype part; then run uenv_read; fi; else if $devtype part $disknum; then run uenv_read; fi; fi
usb_ready_retry=15

Environment size: 3003/131068 bytes
Pogov4>

===================== BOOT DEVICE GENERATION ==================

root:/tmp/sdd1> cd etc
root:/tmp/sdd1/etc> ed fstab
(editing session removed)
root:/tmp/sdd1/etc>cat fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point> <type> <options> <dump> <pass>

LABEL=rootfs / ext4 noatime,user_xattr,commit=300,errors=continue 0 1
root:/tmp/sdd1/etc> cd /tmp/sdd1/boot
root:/tmp/sdd1/boot> ls
System.map-4.12.1-kirkwood-tld-1 uImage
config-4.12.1-kirkwood-tld-1 uInitrd
dts vmlinuz-4.12.1-kirkwood-tld-1
initrd.img-4.12.1-kirkwood-tld-1 zImage-4.12.1-kirkwood-tld-1
linux-headers-4.12.1-kirkwood-tld-1_1.0_armel.deb
root:/tmp/sdd1/boot> cp -a zImage-4.12.1-kirkwood-tld-1 zImage.fdt
root:/tmp/sdd1/boot> cat dts/kirkwood-pogoplug_v4.dtb >> zImage.fdt
root:/tmp/sdd1/boot> mv uImage uImage.orig
root:/tmp/sdd1/boot> mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-4.12.1-kirkwood-tld-1 -d zImage.fdt uImage
Image Name: Linux-4.12.1-kirkwood-tld-1
Created: Sat Sep 30 23:06:45 2017
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 3831812 Bytes = 3742.00 kB = 3.65 MB
Load Address: 00008000
Entry Point: 00008000
root:/tmp/sdd1/boot> sync
root:/tmp/sdd1/boot> sync

================================= AT BOOT TIME =========================

U-Boot 2016.05-tld-1 (Jun 12 2016 - 13:41:47 -0700)
Pogoplug V4

SoC: Kirkwood 88F6192_A1
DRAM: 128 MiB
WARNING: Caches not enabled
NAND: 128 MiB
MMC: kwsdio: 0
In: serial
Out: serial
Err: serial
Net: egiga0
Hit any key to stop autoboot: 0
starting USB...
USB0: USB EHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
scanning usb for storage devices...
Use USB retry period from the environment: 15 second(s)
1 Storage Device(s) found

Reset IDE: ide_preinit failed

Partition Map for USB device 0 -- Partition Type: DOS

Part Start Sector Num Sectors UUID Type
1 63 6249222 0002c900-01 83 Boot
2 6249285 385560 0002c900-02 82
loading envs from usb 0 ...
** File not found /boot/uEnv.txt **

no IDE devices available
Card did not respond to voltage select!
importing envs ...
## Info: input data size = 129 = 0x81
running scan_disk ...
Scan device usb
device usb 0:1
** File not found /boot/uImage **
device usb 1:1
** Bad device usb 1 **
device usb 2:1
** Bad device usb 2 **
device usb 3:1
** Bad device usb 3 **
Scan device ide

Reset IDE: ide_preinit failed
device ide 0:1
** Bad device size - ide 0 **
device ide 1:1
** Bad device size - ide 1 **
device ide 2:1
** Bad device ide 2 **
device ide 3:1
** Bad device ide 3 **
Scan device mmc
device mmc 0:1
Card did not respond to voltage select!
** Bad device mmc 0 **
device mmc 1:1
MMC Device 1 not found
MMC Device 1 not found
** Bad device mmc 1 **
device mmc 2:1
MMC Device 2 not found
MMC Device 2 not found
** Bad device mmc 2 **
device mmc 3:1
MMC Device 3 not found
MMC Device 3 not found
** Bad device mmc 3 **
loading uImage ...
** File not found /boot/uImage **
loading uInitrd ...
** File not found /boot/uInitrd **
loading DTB /boot/dts/kirkwood-pogoplug_v4.dtb ...
** File not found /boot/dts/kirkwood-pogoplug_v4.dtb **
Wrong Image Format for bootm command
ERROR: can't get kernel image!
Pogov4>

==================================================================

/boot/dts/kirkwood-pogoplug_v4.dtb must be there; I generated the new uimage from it.

Suggestions will be gladly received.

Hi Buck,

A little bit busy, but I will be back here several hours later to check your progress. 2 observations for now.

Right here, it means that your rootfs was not done properly.

Scan device usb 
device usb 0:1 
** File not found /boot/uImage **

And this should not be done running the new u-boot. The DTB is loaded separately by the new u-boot. So the original uImage must be used.

root:/tmp/sdd1/boot> cp -a zImage-4.12.1-kirkwood-tld-1 zImage.fdt 
root:/tmp/sdd1/boot> cat dts/kirkwood-pogoplug_v4.dtb >> zImage.fdt 
root:/tmp/sdd1/boot> mv uImage uImage.orig 
root:/tmp/sdd1/boot> mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-4.12.1-kirkwood-tld-1 -d zImage.fdt uImage 
Image Name: Linux-4.12.1-kirkwood-tld-1 
Created: Sat Sep 30 23:06:45 2017 
Image Type: ARM Linux Kernel Image (uncompressed) 
Data Size: 3831812 Bytes = 3742.00 kB = 3.65 MB 
Load Address: 00008000 
Entry Point: 00008000

Please mount the this rootfs on the Pogo E02 and get the file listing of these 2 directories, with other info, and post here (assuming it was mounted at /media/sdb1):

ls -latr /media/sdb1/
ls -latr /media/sdb1/boot
e2label /dev/sdb1
mount

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Bodhi, always good to hear from you. You should certainly be paid more for this job. Here's the information.

I do not know why e2label fails claiming disk corruption. fsck reports that the disk is clean.

The 4 GB USB pen drive being used for the boot is the same one I use for doing rescue operations on the E02. I cleaned off the Debian system (tried both systems first, one FDT and one non-FDT, neither completed the boot) and unTARed the 4.12.8 distribution into the boot partition.

The following from the E02 system with the proposed V4 drive attached as /dev/sdc.

root:/tmp> fsck /dev/sdc1
fsck from util-linux 2.29.2
e2fsck 1.43.4 (31-Jan-2017)
rootfs: clean, 14454/195456 files, 152198/781152 blocks


root:/tmp> cfdisk /dev/sdc
                                       Disk: /dev/sdc
                      Size: 3.8 GiB, 4083351552 bytes, 7975296 sectors
                             Label: dos, identifier: 0x0002c900

    Device        Boot        Start       End   Sectors     Size   Id Type
>>  /dev/sdc1     *              63   6249284   6249222       3G   83 Linux
    /dev/sdc2               6249285   6634844    385560   188.3M   82 Linux swap / Solaris
    Free space              6635520   7975295   1339776   654.2M

 ┌────────────────────────────────────────────────────────────────────────────────────────┐
 │  Partition type: Linux (83)                                                            │
 │      Attributes: 80                                                                    │
 │ Filesystem UUID: 3842cf07-0991-4a80-90a8-910e09d7a0c5                                  │
 │Filesystem LABEL: rootfs                                                                │
 │      Filesystem: ext4                                                                  │
 │      Mountpoint: /tmp/sdc1 (mounted)                                                   │
 └────────────────────────────────────────────────────────────────────────────────────────┘


root:/tmp> mount /dev/sdc1 /tmp/sdc1


root:/tmp> ls -latr /tmp/sdc1/
total 92
drwxr-xr-x 11 root root  4096 Jan  1  1970 var
drwxrwxrwt  2 root root  4096 Jan  1  1970 tmp
drwxr-xr-x 57 root root  4096 Jan  1  1970 etc
drwxr-xr-x  2 root root  4096 Jan  1  2011 sys
drwxr-xr-x  2 root root  4096 Jan 22  2012 proc
drwxr-xr-x  2 root root  4096 Jan 22  2012 home
drwx------  2 root root 16384 Feb 24  2012 lost+found
drwxr-xr-x 10 root root  4096 Feb 24  2012 usr
drwxr-xr-x  2 root root  4096 Feb 24  2012 srv
drwxr-xr-x  2 root root  4096 Feb 24  2012 opt
drwxr-xr-x  2 root root  4096 Feb 24  2012 media
drwxr-xr-x  5 root root  4096 Feb 24  2012 dev
drwxr-xr-x  2 root root  4096 Feb  8  2013 run
drwxr-xr-x  2 root root  4096 Feb 10  2015 mnt
drwxr-xr-x 21 root root  4096 Feb 17  2015 .
drwxr-xr-x 14 root root  4096 Jul 20 08:55 lib
drwxr-xr-x  2 root root  4096 Jul 24 00:12 sbin
drwxr-xr-x  2 root root  4096 Jul 24 00:12 bin
drwx------  5 root root  4096 Jul 24 01:49 root
drwxr-xr-x  3 root root  4096 Sep 30 23:06 boot
drwxrwxrwt  3 root root    80 Oct  1 13:06 ..


root:/tmp> ls -latr /tmp/sdc1/boot
total 43692
drwxr-xr-x 21 root root    4096 Feb 17  2015 ..
drwxr-xr-x  2 root root    4096 Jul 15 22:14 dts
-rw-r--r--  1 root root  153501 Jul 16 03:21 config-4.12.1-kirkwood-tld-1
-rwxr-xr-x  1 root root 3821528 Jul 16 04:42 zImage-4.12.1-kirkwood-tld-1
-rw-------  1 root root 3821528 Jul 16 07:43 vmlinuz-4.12.1-kirkwood-tld-1
-rw-------  1 root root 2504031 Jul 16 07:43 System.map-4.12.1-kirkwood-tld-1
-rw-r--r--  1 root root 8435004 Jul 16 08:06 linux-headers-4.12.1-kirkwood-tld-1_1.0_armel.deb
-rw-r--r--  1 root root 3821592 Jul 20 08:11 uImage.orig
-rw-r--r--  1 root root 7245632 Jul 24 00:15 initrd.img-4.12.1-kirkwood-tld-1
-rw-r--r--  1 root root 7245696 Jul 24 00:18 uInitrd
-rwxr-xr-x  1 root root 3831812 Sep 30 23:06 zImage.fdt
drwxr-xr-x  3 root root    4096 Sep 30 23:06 .
-rw-r--r--  1 root root 3831876 Sep 30 23:06 uImage


root:/tmp> e2label /dev/sdc1
e2label: Superblock checksum does not match superblock while trying to open /dev/sdc1
Couldn't find valid filesystem superblock.


root:/tmp> mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=30447,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=25092k,mode=755)
/dev/sda1 on / type ext4 (rw,noatime,commit=300,data=ordered)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=244640k)
tmpfs on /tmp type tmpfs (rw,relatime)
/dev/sdb1 on /sdb1 type ext4 (rw,noatime,commit=600,data=ordered)
/dev/sdc1 on /tmp/sdc1 type ext4 (rw,relatime,data=ordered)


root:/tmp> cat /tmp/sdc1/etc/fstab
# /etc/fstab: static file system information.
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
LABEL=rootfs    /               ext4    noatime,user_xattr,commit=300,errors=continue 0 1
tmpfs          /tmp            tmpfs   defaults          0       0

I will continue fiddling with it today and see if I can make any progress.

As a side note, the NAND rescue system is not available on this V4 as I rather foolishly failed to install it after installing the U-boot and before restarting the unit. Lesson learned the hard way.

After several hours of exploring possibilities, I have a solution. I don't know if it will be of value to anyone else but here it is.

This Pogoplug V4 will not boot from a USB drive. It will only boot from an SD chip, and it is particular about which of those it will accept.

Despite the USB drive being readable and bootable on every other system, the Pogo V4 will not read from it. At least not reliably. It's convinced that while the drive might be partitioned, it's not readable.

I loaded the distribution onto an 8 GB SDHC chip and tried to boot that. It booted ... once. Attempting to reboot resulted in multiple errors due to drive corruption.

So I dug out an older 2 GB SD chip and loaded the distribution on that. It seems to boot reliably and I'm doing the software updates at this time.

I would have to advise anyone trying to bring up Debian on a Pogo V4 to use an older SD chip. Presumably a hard drive on a IDE/SATA-to-USB adapter would also work.

As an aside, it is probably fortunate that whoever was selling the lot of 100 Pogo V4s on fleabay called off the auction. I was in the lead with two days left. I had it in mind to sell preconfigured V4s as instant web sites, but I don't think I could deal with this another hundred times.



Edited 1 time(s). Last edit at 10/01/2017 01:43PM by BuckNaked.

BuckNaked,

> This Pogoplug V4 will not boot from a USB drive.
> It will only boot from an SD chip, and it is
> particular about which of those it will accept.

That's not the typical experience that I have with these Marvell plugs. USB is always the best way to boot Debian. But it is picky about what drive brand you use. Sandisk is always the best.

> So I dug out an older 2 GB SD chip and loaded the
> distribution on that. It seems to boot reliably
> and I'm doing the software updates at this time.

That's a good approach. USB driver in u-boot is not always kept up to date with Linux mainline. Except for the last year or so, u-boot source tree is pretty up-to-date.

> I would have to advise anyone trying to bring up
> Debian on a Pogo V4 to use an older SD chip.

My advice is also consider a Sandisk USB thumb drive (exception: USB 3.0 32GB Ultra Fit, this particular drive is bad for booting).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Quote
Bodhi
My advice is also consider a Sandisk USB thumb drive.

Makes good sense. I was using a PNY and assumed that since it booted everywhere else, it would also boot in the Pogo V4. So much for assumptions.

Hey Buck, I'll also add that I was able to enable SSH via serial without soldering on the connections. I just held them in place long enough to edit the startup script and was good to go after that. Cramped my hand a little, but saved the pain and worry of mini-soldering.

cdlenfert,

> Hey Buck, I'll also add that I was able to enable
> SSH via serial without soldering on the
> connections. I just held them in place long enough
> to edit the startup script and was good to go
> after that. Cramped my hand a little, but saved
> the pain and worry of mini-soldering.

That's another technique :) I used 3 shirt-pins, 3 test clips (with wires attached). The solder buttons are soft enough to push the pins into them and stay connected for hours.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Well, I had to plug into it for several days while I was trying to figure out what was going on, so I figured a temporary solution wasn't a good option.

After clearing the TX and RX vias, I soldered Arduino Dupont-style jumpers (pin ends) into the vias and grounded to the solder pad on the side of the SD socket. This gives me decent quality pins to plug into the USB serial adapter's pin receptacles next time I need to work on it. (I figure that's a "when", not "if".) And then placed heat-shrink over the exposed pins so nothing would short out while stuffed inside the case.