Use netconsole to troubleshoot uBoot without a serial cable

Because I'm lazy and forgetful, I use a very simple script to run netconsole. It even cleans up after itself.

#!/bin/bash

trap 'kill $ncpid' SIGINT SIGTERM EXIT
if [ -z "$1" ]; then
    echo Missing destination host
    exit 1
else
    nc -l -u -p 6666 &
    ncpid=$!
    nc -u "$1" 6666
fi

I got netconsole to work for a while (though read-only - I did not get the background process to work for read-write access), but now after upgrading to the new (2017) uboot I can't see any console output.

My "desktop" is a Pogo V4, the device I want to watch is a Pogo E02.

Here's the netconsole output from two reboots when it was working:

root@PogoV4Debian:/boot# nc -l -u -p 6666

U-Boot 2016.05-tld-1 (Jun 12 2016 - 13:23:43 -0700)
Pogo E02
gcc (Debian 4.9.2-10) 4.9.2
GNU ld (GNU Binutils for Debian) 2.25
Hit any key to stop autoboot:  0
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 3 USB Device(s) found
       scanning usb for storage devices...
Use USB retry period from the environment: 15 second(s)
1 Storage Device(s) found

Partition Map for USB device 0  --   Partition Type: DOS

Part    Start Sector    Num Sectors     UUID            Type
  1     2048            15872000        c56503a8-01     83 Boot
loading envs from usb 0 ...
** File not found /boot/uEnv.txt **
running scan_disk ...
Scan device usb
device usb 0:1
1 bytes read in 2629 ms (0 Bytes/s)
Found bootable drive on usb 0
2875692 bytes read in 416 ms (6.6 MiB/s)
6535284 bytes read in 546 ms (11.4 MiB/s)
** File not found /boot/dts/dts/kirkwood-pogo_e02.dtb **
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   Linux-3.18.5-kirkwood-tld-1
   Created:      2017-11-02  23:05:54 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    2875628 Bytes = 2.7 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   initramfs-3.18.5-kirkwood-tld-1
   Created:      2015-02-19   1:49:36 UTC
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    6535220 Bytes = 6.2 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK


Starting kernel ...

U-Boot 2016.05-tld-1 (Jun 12 2016 - 13:23:43 -0700)
Pogo E02
gcc (Debian 4.9.2-10) 4.9.2
GNU ld (GNU Binutils for Debian) 2.25
Hit any key to stop autoboot:  0
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 3 USB Device(s) found
       scanning usb for storage devices...
Use USB retry period from the environment: 15 second(s)
1 Storage Device(s) found

Partition Map for USB device 0  --   Partition Type: DOS

Part    Start Sector    Num Sectors     UUID            Type
  1     2048            15872000        c56503a8-01     83 Boot
loading envs from usb 0 ...
** File not found /boot/uEnv.txt **
running scan_disk ...
Scan device usb
device usb 0:1
1 bytes read in 242 ms (0 Bytes/s)
Found bootable drive on usb 0
3848064 bytes read in 427 ms (8.6 MiB/s)
6349577 bytes read in 557 ms (10.9 MiB/s)
** File not found /boot/dts/dts/kirkwood-pogo_e02.dtb **
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   Linux-4.13.5-kirkwood-tld-1
   Created:      2017-11-03   1:42:30 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3848000 Bytes = 3.7 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   initramfs-4.13.5-kirkwood-tld-1
   Created:      2017-11-03   1:42:54 UTC
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    6349513 Bytes = 6.1 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK


Starting kernel ...

But, like I said, after updating to U-Boot.2017... I get no output.

Here are my envs (note that I tried changing a couple of the nc-related envs to see if that would make a difference, so if one or two look funny that's probably why, but on the first try they were entered as shown in the kernel install instructions):

root@debian:~# fw_printenv
bootcmd_exec=run load_uimage; if run load_initrd; then if run load_dtb; then bootm $load_uimage_addr $load_initrd_addr $load_dtb_addr; else bootm $load_uimage_addr $load_initrd_addr; fi; else if run load_dtb; then bootm $load_uimage_addr - $load_dtb_addr; else bootm $load_uimage_addr; fi; fi
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec
bootcmd_uenv=run uenv_load; if test $uenv_loaded -eq 1; then run uenv_import; fi
bootdelay=10
bootdev=usb
device=0:1
ethact=egiga0
if_netconsole=ping $serverip
led_error=orange blinking
led_exit=green off
led_init=green blinking
dtb_file=/boot/dts/kirkwood-pogo_e02.dtb
load_dtb_addr=0x1c00000
load_initrd_addr=0x1100000
load_uimage_addr=0x800000
mainlineLinux=yes
mtdids=nand0=orion_nand
mtdparts=mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data)
partition=nand0,2
scan_disk=echo running scan_disk ...; scan_done=0; setenv scan_usb "usb start";  setenv scan_ide "ide reset";  setenv scan_mmc "mmc rescan"; for dev in $devices; do if test $scan_done -eq 0; then echo Scan device $dev; run scan_$dev; for disknum in $disks; do if test $scan_done -eq 0; then echo device $dev $disknum:1; if load $dev $disknum:1 $load_uimage_addr /boot/uImage 1; then scan_done=1; echo Found bootable drive on $dev $disknum; setenv device $disknum:1; setenv bootdev $dev; fi; fi; done; fi; done
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts $custom_params
start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;
stderr=serial
stdin=serial
stdout=serial
uenv_addr=0x810000
uenv_import=echo importing envs ...; env import -t $uenv_addr $filesize
uenv_init_devices=setenv init_usb "usb start";  setenv init_ide "ide reset";  setenv init_mmc "mmc rescan"; for devtype in $devices; do run init_$devtype; done;
uenv_load=run uenv_init_devices; setenv uenv_loaded 0; for devtype in $devices;  do for disknum in 0; do run uenv_read_disk; done; done;
uenv_read_disk=if test $devtype -eq mmc; then if $devtype part; then run uenv_read;  fi; else if $devtype part $disknum; then run uenv_read; fi;  fi
uenv_read=echo loading envs from $devtype $disknum ...; if load $devtype $disknum:1 $uenv_addr /boot/uEnv.txt; then setenv uenv_loaded 1; fi
usb_ready_retry=15
arcNumber=3542
machid=dd6
ethaddr=00:25:31:04:80:8C
disks=0 1 2 3 4 5 6 7
preboot_nc=setenv nc_ready 0; for pingstat in 1 2 3 4 5; do; sleep 1; if run if_netconsole; then setenv nc_ready 1; fi; done; if test $nc_ready -eq 1; then run start_netconsole; fi
ipaddr=192.168.222.168
devices=usb
load_initrd=ext2load usb 0:1 0x1100000 /boot/uInitrd
load_uimage=ext2load usb 0:1 0x800000 /boot/uImage
usb_boot=run load_dtb; run load_uimage; if run load_initrd; then bootm 0x800000 0x1100000 0x1c00000; else bootm 0x800000 - 0x1c00000; fi
serverip=192.168.222.164
load_dtb=ext2load usb 0:1 0x1c00000 /boot/dts/kirkwood-pogo_e02.dtb
preboot=run if_netconsole start_netconsole
root@debian:~#

your serverip for the Pogo V4 "serverip=192.168.222.164" is correct and you have started netconsole on it before booting the E02?

huge,

There should be no problem with Pogo E02 netconsole. You probably have some wrong envs.

-bodhi
===========================
Forum Wiki
bodhi's corner

feas Wrote:
-------------------------------------------------------
> your serverip for the Pogo V4
> "serverip=192.168.222.164" is correct and you have
> started netconsole on it before booting the E02?

Yes ... I think I have that part right:

V4 IP address (== serverip on E02): .222.164
E02 IP address (== ipaddr on E02): .222.168

And on the V4 I type

root@PogoV4Debian:/boot# nc -l -u -p 6666

before I boot or reboot the E02.

But I've also tried:

root@PogoV4Debian:/boot# nc -l -u -p 6666 &
[2] 8363
root@PogoV4Debian:/boot# nc -u 192.168.1.100 6666

...and...

root@PogoV4Debian:/boot# nc -klu 6666

Before the upgraded U-Boot, typing "nc -l -u -p 6666" worked to see console output during boot, and if I used the background process it actually allowed me to interrupt during the 10-second countdown but then I never saw the Pogo>> prompt (or whatever I used to see when I had a serial cable attached).

After the upgrade, anything I've tried just gives me a blank screen until I ^C out of it.


Thanks once again for both of your replies & help

On Pogo V4 (must have IP 192.168.222.164), try

nc -lup 6666 192.168.222.168  6666

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 1 time(s). Last edit at 11/03/2017 03:02AM by bodhi.

bodhi Wrote:
-------------------------------------------------------
> huge,
>
> There should be no problem with Pogo E02
> netconsole. You probably have some wrong envs.

I'm noticing a couple of differences between current envs and what I posted back in the vast E02 install thread last week. The old ones look possibly more robust using variables instead of static addresses/values. But I don't know if there's any way the new ones are problematic:


OLD

load_dtb=echo loading DTB $dtb_file ...; load $bootdev $device $load_dtb_addr $dtb_file
load_initrd=echo loading uInitrd ...; load $bootdev $device $load_initrd_addr /boot/uInitrd
load_uimage=echo loading uImage ...; load $bootdev $device $load_uimage_addr /boot/uImage

NEW

load_dtb=ext2load usb 0:1 0x1c00000 /boot/dts/kirkwood-pogo_e02.dtb
load_initrd=ext2load usb 0:1 0x1100000 /boot/uInitrd
load_uimage=ext2load usb 0:1 0x800000 /boot/uImage

bodhi Wrote:
-------------------------------------------------------
> On Pogo V4 (must have IP 192.168.222.164), try
>
>

> nc -lup 6666 192.168.222.168  6666
>

-bodhi
===========================
Forum Wiki
bodhi's corner

bodhi Wrote:
-------------------------------------------------------
> bodhi Wrote:
> -------------------------------------------------------
> > On Pogo V4 (must have IP 192.168.222.164), try
> >
> >

> > nc -lup 6666 192.168.222.168  6666
> >

Success, thank you! Actually that exact command got me an error message, but that led me to a post (from you, of course) telling another noob that there were actually (at least) two netconsole/netcat programs, which led me to the magic combo that seems to work for me:

nc.traditional -l -u -p 6666 &

nc.traditional -u -p 6666 192.168.222.168 6666


I thought I was just doing this to have a rescue plan in place in case something goes wrong, but in fact the console output from both my E02 and my Mobile have a few lines in them that look like something wrong. But I'll dig around and see what I can find out about them before I bug you in another thread.

Thanks once again.

> Success, thank you! Actually that exact command
> got me an error message, but that led me to a post
> (from you, of course) telling another noob that
> there were actually (at least) two
> netconsole/netcat programs, which led me to the
> magic combo that seems to work for me:
>
> nc.traditional -l -u -p 6666 &
> nc.traditional -u -p 6666 192.168.222.168 6666

Cool! yes, on Ubuntu the netcat version is important.

-bodhi
===========================
Forum Wiki
bodhi's corner

I am always just using:

nc -lu 192.168.1.35 6666

Only this single line. And it reliably enables monitoring and interactivity with the connected GoFlexNet for me.

Tip for who use nc in Windows 10(or 7, 8, 8.1).

As you all know, Windows blocked ping for security reasons. And netconsole would not start before server responds. So you must allow it before use nc.

Open Windows Power Shell (as an administrator) or cmd (as an administrator), and run this command.

netsh advfirewall firewall add rule name="ICMP Allow" protocol=icmpv4:8,any dir=in action=allow

(I searched for this tip, and i don't find. I hope it was helpful.)

Having just reinstalled Stock on my GoFlex Home unit, I wondered if it might be possible to install netconsole on it, .... or might it already be installed somewhere? If not can I simply wget it?

balanga Wrote:
-------------------------------------------------------
> Having just reinstalled Stock on my GoFlex Home
> unit, I wondered if it might be possible to
> install netconsole on it, .... or might it already
> be installed somewhere? If not can I simply wget
> it?

Stock uboot is very old, and there is no netconsole support. Netconsole is integrated in uboot. The only way to have netconsole is to use new uboot:

https://forum.doozan.com/read.php?3,12381

-bodhi
===========================
Forum Wiki
bodhi's corner

The good thing about Stock is that it's all embedded, or is it possible to just update uboot and leave the OS in situ?

Apologies if this is a dumb question...

balanga Wrote:
-------------------------------------------------------
> The good thing about Stock is that it's all
> embedded, or is it possible to just update uboot
> and leave the OS in situ?
>
> Apologies if this is a dumb question...

It is possible with some extra steps to set up.

But it would be a waste of effort to install new uboot and then run it that way.

-bodhi
===========================
Forum Wiki
bodhi's corner

bodhi Wrote:
-------------------------------------------------------
> balanga Wrote:
> -------------------------------------------------------
> > The good thing about Stock is that it's all
> > embedded, or is it possible to just update
> uboot
> > and leave the OS in situ?
> >
> > Apologies if this is a dumb question...
>
> It is possible with some extra steps to set up.
>
> But it would be a waste of effort to install new
> uboot and then run it that way.

Can you explain? I'm reasonably happy with the stock system since it is embedded and not installed on external media. If I could enhance uboot to run netconsole and be able to read uEnv.txt on boot to overide the default variables that would be sufficient for me.

This forum mission is to replace stock OS with a GNU/Linux OS that is much more powerful, and tracking mainlined Debian Linux. Basically we are making low cost ARM boxes performing like expensive NAS boxes, and keeping them up-to-date with latest Linux software development.

-bodhi
===========================
Forum Wiki
bodhi's corner

My net console seems to be no properly working.

I've sent the following:

fw_setenv serverip 192.168.1.19 (my pi fixe IP)
fw_setenv ipaddr 192.168.1.200 (free IP in the dhcp)

I sent the command:

nc -l -u -p 6666

I made a reboot and I only get the following:

root@Raspberrypy-4:/home/pi# nc -l -u -p 6666

U-Boot 2017.07-tld-1 (Sep 05 2017 - 00:42:03 -0700)
ZyXEL NSA325 2-Bay Power Media Server
gcc (Debian 6.3.0-18) 6.3.0 20170516
GNU ld (GNU Binutils for Debian) 2.28

Then the boot continues as normal but I cannot see any realtime log on nc.

If I use the command

nc -klu 6666

things are going better and I arrive up to the Starting Kernel....

pi@Raspberrypy-4:~ $ nc -klu 6666

U-Boot 2017.07-tld-1 (Sep 05 2017 - 00:42:03 -0700)
ZyXEL NSA325 2-Bay Power Media Server
gcc (Debian 6.3.0-18) 6.3.0 20170516
GNU ld (GNU Binutils for Debian) 2.28
Hit any key to stop autoboot:  0
Initializing devices...
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found

Reset IDE: Bus 0: OK Bus 1: OK
  Device 0: Model: WDC WD20EZRX-00D8PB0 Firm: 80.00A80 Ser#:  WD-WMC4M3352905
            Type: Hard Disk
            Supports 48-bit addressing
            Capacity: 1907729.0 MB = 1863.0 GB (3907029168 x 512)
  Device 1: Model: Hitachi HTS541680J9SA00  Firm: SB2OC7BP Ser#: SB2281KGDP5BJE
            Type: Hard Disk
            Supports 48-bit addressing
            Capacity: 76319.0 MB = 74.5 GB (156301488 x 512)
Unknown command 'mmc' - try 'help'
Loading envs from usb 0...
** Bad device usb 0 **
Loading envs from usb 1...
** Bad device usb 1 **
Loading envs from usb 2...
** Bad device usb 2 **
Loading envs from usb 3...
** Bad device usb 3 **
Loading envs from ide 0...
** File not found /boot/uEnv.txt **
Loading envs from ide 1...
121 bytes read in 1077 ms (0 Bytes/s)
... envs loaded
importing envs ...
running scan_disk ...
Scan device ide

Reset IDE: Bus 0: OK Bus 1: OK
  Device 0: Model: WDC WD20EZRX-00D8PB0 Firm: 80.00A80 Ser#:  WD-WMC4M3352905
            Type: Hard Disk
            Supports 48-bit addressing
            Capacity: 1907729.0 MB = 1863.0 GB (3907029168 x 512)
  Device 1: Model: Hitachi HTS541680J9SA00  Firm: SB2OC7BP Ser#: SB2281KGDP5BJE
            Type: Hard Disk
            Supports 48-bit addressing
            Capacity: 76319.0 MB = 74.5 GB (156301488 x 512)
device ide 0:1
** File not found /boot/uImage **
device ide 1:1
1 bytes read in 935 ms (0 Bytes/s)
Found bootable drive on ide 1
loading uImage ...
5435888 bytes read in 2039 ms (2.5 MiB/s)
loading uInitrd ...
9671091 bytes read in 2852 ms (3.2 MiB/s)
loading DTB /boot/dts/kirkwood-nsa325.dtb ...
14014 bytes read in 3255 ms (3.9 KiB/s)
## Booting kernel from Legacy Image at 00800000 ...
   Image Name:   Linux-5.13.6-kirkwood-tld-1
   Created:      2021-09-24   0:22:58 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    5435824 Bytes = 5.2 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
   Image Name:   initramfs-5.13.6-kirkwood-tld-1
   Created:      2021-09-24   1:06:32 UTC
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    9671027 Bytes = 9.2 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02c00000
   Booting using the fdt blob at 0x2c00000


Starting kernel ...

Edited 1 time(s). Last edit at 04/20/2022 04:30PM by miazza.

> Starting kernel ...
>

During u-boot booting, netconsole output stops here.

See the Wiki thread

Quote

Netconsole

Use netconsole to troubleshoot uBoot without a serial cable
Use netconsole to troubleshoot Debian kernel booting

BTW, it is also time or you to browse the Wiki thread page for topics that might be of interest to you!

-bodhi
===========================
Forum Wiki
bodhi's corner