Use netconsole to troubleshoot uBoot without a serial cable
Posted by Jeff
|
Re: Use netconsole to troubleshoot uBoot without a serial cable October 27, 2013 09:51AM |
Registered: 11 years ago Posts: 93 |
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 10, 2014 03:22PM |
Registered: 10 years ago Posts: 13 |
I found this thread after following Alice down the rabbit hole...
I'm having problems getting netconsole to help troubleshoot my GoFlex Home problems.
Back in September of 2013, I installed Arch Linux ARM on it, using the instructions over at the archlinuxarm.org GoFlex Home page. So I'm using v0.6 of the UBIT environment from http://jeff.doozan.com/debian/goflex/v0.6/uInitrd and http://jeff.doozan.com/debian/goflex/v0.6/ubit_start. The GoFlex was successfully serving up a software RAID 5 array on a Sans Digital RAID storage array connected via a SATA-eSATA adapter cable to the GoFlex (yes - the Kirkwood does have a built-in SATA port multiplier!). The storage array was in JBOD mode, and md was properly running a RAID 5 config with four 2T drives - sweet.
Everything was working fine, until I pulled a bone-headed move. I accidentally crashed my Linux laptop, and the only USB flash drive I had on hand was the one in the GoFlex, so I "borrowed" it. I copied the archlinuxarm image using the dd utility on my wife's MacBook. After I rebuilt my laptop, I used dd to re-image the USB flash drive and stuck it back in the GoFlex. Nada. Nothing works... Seems OS X does weird things to ISO images when using dd...
I'm confused as hell about using nc. Does Jeff's uboot image run nc by default, or does it have to be manually setup? I don't have a USB-serial cable converter, so I can't get to the GoFlex console port to configure it.
I did connect the GoFlex directly to my laptop via Ethernet and ran Wireshark. The only packets I can see coming from the GoFlex are a couple of arp requests from 10.10.10.6 asking who has 10.10.10.5. I configured the laptop's eth0.1/.2./3 with 10.10.10.5/.4/.2, respectively, and ran "nc -lup 6666", but get no response at all. I tried "nc -lup 6666 &; nc -l 10.10.10.6 6666", and still nothing.
Any ideas?
I'm having problems getting netconsole to help troubleshoot my GoFlex Home problems.
Back in September of 2013, I installed Arch Linux ARM on it, using the instructions over at the archlinuxarm.org GoFlex Home page. So I'm using v0.6 of the UBIT environment from http://jeff.doozan.com/debian/goflex/v0.6/uInitrd and http://jeff.doozan.com/debian/goflex/v0.6/ubit_start. The GoFlex was successfully serving up a software RAID 5 array on a Sans Digital RAID storage array connected via a SATA-eSATA adapter cable to the GoFlex (yes - the Kirkwood does have a built-in SATA port multiplier!). The storage array was in JBOD mode, and md was properly running a RAID 5 config with four 2T drives - sweet.
Everything was working fine, until I pulled a bone-headed move. I accidentally crashed my Linux laptop, and the only USB flash drive I had on hand was the one in the GoFlex, so I "borrowed" it. I copied the archlinuxarm image using the dd utility on my wife's MacBook. After I rebuilt my laptop, I used dd to re-image the USB flash drive and stuck it back in the GoFlex. Nada. Nothing works... Seems OS X does weird things to ISO images when using dd...
I'm confused as hell about using nc. Does Jeff's uboot image run nc by default, or does it have to be manually setup? I don't have a USB-serial cable converter, so I can't get to the GoFlex console port to configure it.
I did connect the GoFlex directly to my laptop via Ethernet and ran Wireshark. The only packets I can see coming from the GoFlex are a couple of arp requests from 10.10.10.6 asking who has 10.10.10.5. I configured the laptop's eth0.1/.2./3 with 10.10.10.5/.4/.2, respectively, and ran "nc -lup 6666", but get no response at all. I tried "nc -lup 6666 &; nc -l 10.10.10.6 6666", and still nothing.
Any ideas?
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 10, 2014 11:00PM |
Admin Registered: 13 years ago Posts: 18,545 |
beausoleil,
> I'm confused as hell about using nc. Does Jeff's
> uboot image run nc by default, or does it have to
> be manually setup? I don't have a USB-serial cable
> converter, so I can't get to the GoFlex console
> port to configure it.
You do need to configure it, as Jeff described in the instruction. This set up must be done on the GoFlex Home before you can connect to it with netconsole server on another computer (in this example, 192.168.1.2 is the IP addr of this computer).
But it is not too late! you can download my latest Debian rootfs and put that on new USB thumb. i.e. use second USB thumb drive for this (see the download link in my signature).
After you've booted into Debian, setup netconsole right away for future recovery purpose. And then mount your USB stick, and see if it can be fixed.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
> I'm confused as hell about using nc. Does Jeff's
> uboot image run nc by default, or does it have to
> be manually setup? I don't have a USB-serial cable
> converter, so I can't get to the GoFlex console
> port to configure it.
You do need to configure it, as Jeff described in the instruction. This set up must be done on the GoFlex Home before you can connect to it with netconsole server on another computer (in this example, 192.168.1.2 is the IP addr of this computer).
fw_setenv serverip 192.168.1.2 fw_setenv ipaddr 192.168.1.100 fw_setenv if_netconsole 'ping $serverip' fw_setenv start_netconsole 'setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;' fw_setenv preboot 'run if_netconsole start_netconsole'
But it is not too late! you can download my latest Debian rootfs and put that on new USB thumb. i.e. use second USB thumb drive for this (see the download link in my signature).
After you've booted into Debian, setup netconsole right away for future recovery purpose. And then mount your USB stick, and see if it can be fixed.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 11, 2014 02:31PM |
Registered: 10 years ago Posts: 13 |
Thanks, bohdi.
Here's what I did:
Followed your link and setup a second USB flash drive with Debian. Then I connected the GoFlex Home via ethernet directly to my laptop's ethernet port. I used wireshark on my laptop to monitor the ethernet traffic while the GoFlex Home booted with the Debian USB flash drive in its single USB port. The packet capture showed that the address 10.10.10.6 was arping for 10.10.10.5's mac address.
So I edited my laptop's /etc/network config file to add secondary addresses in addition to the existing eth0 address of 192.168.1.1: 10.10.10.5 on eth0:0, 10.10.10.4 on eth0:1, and 10.10.10.2 on eth0:2.
I then opened three Gnome terminals. In the first, I ran the command as a normal user the command "nc -lup 6666 &" then "nc -u 10.10.10.6 6666" in the foreground. In the second, "nc -l 10.10.10.5 6666", the third, "nc -l 10.10.10.4 6666", and the fourth, "nc -l 10.10.10.2 6666"
I get the following from "nc -u 10.10.10.6 6666":
I get nothing from the others. But at least it's a start. And I can confirm that the kernel is booting, as Wireshark finds that the GoFlex on 10.10.10.6 is sending the following to 10.10.10.4 (although the nc instance listening to the socket at 10.10.10.4 doesn't):
The boot process basically hangs there.
So how (and what) do I change the Magic number to? How do I configure nc so I can have an interactive session so I can change the appropriate uboot firmware variable to boot the Debian image?
Edited 1 time(s). Last edit at 02/11/2014 02:44PM by beausoleil.
Here's what I did:
Followed your link and setup a second USB flash drive with Debian. Then I connected the GoFlex Home via ethernet directly to my laptop's ethernet port. I used wireshark on my laptop to monitor the ethernet traffic while the GoFlex Home booted with the Debian USB flash drive in its single USB port. The packet capture showed that the address 10.10.10.6 was arping for 10.10.10.5's mac address.
So I edited my laptop's /etc/network config file to add secondary addresses in addition to the existing eth0 address of 192.168.1.1: 10.10.10.5 on eth0:0, 10.10.10.4 on eth0:1, and 10.10.10.2 on eth0:2.
I then opened three Gnome terminals. In the first, I ran the command as a normal user the command "nc -lup 6666 &" then "nc -u 10.10.10.6 6666" in the foreground. In the second, "nc -l 10.10.10.5 6666", the third, "nc -l 10.10.10.4 6666", and the fourth, "nc -l 10.10.10.2 6666"
I get the following from "nc -u 10.10.10.6 6666":
U-Boot 2010.09 (Feb 16 2011 - 18:44:23)
UBIT v0.6 by Jeff Doozan and Peter Carmichael
(Re)start USB...
USB: Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices... 1 Storage Device(s) found
Hit any key to stop autoboot: 0
UBIFS error (pid 0): ubifs_get_sb: cannot open "ubi:fast", error -19
Error reading superblock on volume 'ubi:fast'!
UBIFS not mounted, use ubifs mount to mount volume first!
Loading file "/boot/uImage" from usb device 0:1 (usbda1)
2399792 bytes read
Loading file "/uInitrd" from usb device 0:1 (usbda1)
** File not found /uInitrd
Loading file "/boot/uInitrd" from usb device 0:1 (usbda1)
6568066 bytes read
## Booting kernel from Legacy Image at 00680000 ...
Image Name: Linux-3.13.1-kirkwood-tld-2
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 2399728 Bytes = 2.3 MiB
Load Address: 00008000
Entry Point: 00008000
Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 01100000 ...
Image Name: initramfs-3.13.1-kirkwood-tld-2
Image Type: ARM Linux RAMDisk Image (gzip compressed)
Data Size: 6568002 Bytes = 6.3 MiB
Load Address: 00000000
Entry Point: 00000000
Verifying Checksum ... OK
Loading Kernel Image ... OK
OK
Starting kernel ...
I get nothing from the others. But at least it's a start. And I can confirm that the kernel is booting, as Wireshark finds that the GoFlex on 10.10.10.6 is sending the following to 10.10.10.4 (although the nc instance listening to the socket at 10.10.10.4 doesn't):
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 3.13.1-kirkwood-tld-2 (root@tldDebian) (gcc version 4.6.3 (Debian 4.6.3-14) ) #1 PREEMPT Tue Feb 4 03:19:07 PST 2014 [ 0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977 [ 0.000000] CPU: VIVT data cache, VIVT instruction cache [ 0.000000] Machine: Seagate GoFlex Home [ 0.000000] Memory policy: Data cache writeback [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 32512 [ 0.000000] Kernel command line: console=ttyS0,115200 mtdparts=orion_nand:1M(u-boot),6M(uImage),-(root) root=/dev/sda1 rootdelay=10 rootfstype=auto netconsole=@10.10.10.6/eth0,@10.10.10.4/ [ 0.000000] PID hash table entries: 512 (order: -1, 2048 bytes) [ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) [ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) [ 0.000000] Memory: 116220K/131072K available (4659K kernel code, 346K rwdata, 1588K rodata, 211K init, 425K bss, 14852K reserved) [ 0.000000] Virtual kernel memory layout: [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB) [ 0.000000] fixmap : 0xfff00000 - 0xfffe0000 ( 896 kB) [ 0.000000] vmalloc : 0xc8800000 - 0xff000000 ( 872 MB) [ 0.000000] lowmem : 0xc0000000 - 0xc8000000 ( 128 MB) [ 0.000000] modules : 0xbf000000 - 0xc0000000 ( 16 MB) [ 0.000000] .text : 0xc0008000 - 0xc06220ac (6249 kB) [ 0.000000] .init : 0xc0623000 - 0xc0657e4c ( 212 kB) [ 0.000000] .data : 0xc0658000 - 0xc06ae824 ( 347 kB) [ 0.000000] .bss : 0xc06ae824 - 0xc0718d6c ( 426 kB) [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] NR_IRQS:114 [ 0.000000] sched_clock: 32 bits at 200MHz, resolution 5ns, wraps every 21474836475ns [ 0.000000] Console: colour dummy device 80x30 [ 20.256020] Calibrating delay loop... 1191.11 BogoMIPS (lpj=5955584) [ 20.345821] pid_max: default: 32768 minimum: 301 [ 20.345902] Security Framework initialized [ 20.345955] Mount-cache hash table entries: 512 [ 20.346480] Initializing cgroup subsys devices [ 20.346497] Initializing cgroup subsys freezer [ 20.346507] Initializing cgroup subsys net_cls [ 20.346516] Initializing cgroup subsys blkio [ 20.346572] CPU: Testing write buffer coherency: ok [ 20.346869] Setting up static identity map for 0x469248 - 0x469284 [ 20.348210] devtmpfs: initialized [ 20.350236] prandom: seed boundary self test passed [ 20.354248] prandom: 100 self tests passed [ 20.354256] pinctrl core: initialized pinctrl subsystem [ 20.354491] regulator-dummy: no parameters [ 20.354726] NET: Registered protocol family 16 [ 20.355137] DMA: preallocated 256 KiB pool for atomic coherent allocations [ 20.355548] cpuidle: using governor ladder [ 20.355558] cpuidle: using governor menu [ 20.355666] Kirkwood: MV88F6281-A1, TCLK=200000000. [ 20.355685] Feroceon L2: Cache support initialised. [ 20.359735] bio: create slab <bio-0> at 0 [ 20.360186] vgaarb: loaded [ 20.360411] usbcore: registered new interface driver usbfs [ 20.360464] usbcore: registered new interface driver hub [ 20.360569] usbcore: registered new device driver usb [ 20.361201] Switched to clocksource orion_clocksource [ 20.381660] NET: Registered protocol family 2 [ 20.382318] TCP established hash table entries: 1024 (order: 0, 4096 bytes) [ 20.382344] TCP bind hash table entries: 1024 (order: 0, 4096 bytes) [ 20.382364] TCP: Hash tables configured (established 1024 bind 1024) [ 20.382432] TCP: reno registered [ 20.382446] UDP hash table entries: 256 (order: 0, 4096 bytes) [ 20.382468] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes) [ 20.382671] NET: Registered protocol family 1 [ 20.382999] RPC: Registered named UNIX socket transport module. [ 20.383010] RPC: Registered udp transport module. [ 20.383017] RPC: Registered tcp transport module. [ 20.383024] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 20.383269] Unpacking initramfs... [ 20.927169] Freeing initrd memory: 6408K (c1101000 - c1743000) [ 20.927298] NetWinder Floating Point Emulator V0.97 (double precision) [ 20.927894] audit: initializing netlink socket (disabled) [ 20.927947] type=2000 audit(0.660:1): initialized [ 20.928728] zbud: loaded [ 20.928898] VFS: Disk quotas dquot_6.5.2 [ 20.928952] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [ 20.929334] NFS: Registering the id_resolver key type [ 20.929394] Key type id_resolver registered [ 20.929403] Key type id_legacy registered [ 20.929422] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 20.929431] Installing knfsd (copyright (C) 1996 okir@monad.swb.de). [ 20.929604] jffs2: version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc. [ 20.929876] msgmni has been set to 239 [ 20.929997] Key type big_key registered [ 20.931914] alg: No test for stdrng (krng) [ 20.932006] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) [ 20.932018] io scheduler noop registered [ 20.932026] io scheduler deadline registered [ 20.932050] io scheduler cfq registered (default) [ 20.932263] mv_xor mv_xor.0: Marvell shared XOR driver [ 20.971264] mv_xor mv_xor.0: Marvell XOR: ( xor cpy ) [ 21.011262] mv_xor mv_xor.0: Marvell XOR: ( xor cpy ) [ 21.011364] mv_xor mv_xor.1: Marvell shared XOR driver [ 21.051263] mv_xor mv_xor.1: Marvell XOR: ( xor cpy ) [ 21.091262] mv_xor mv_xor.1: Marvell XOR: ( xor cpy ) [ 21.091527] Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled [ 21.112099] serial8250.0: ttyS0 at MMIO 0xf1012000 (irq = 33, base_baud = 12500000) is a 16550A [ 21.638716] console [ttyS0] enabled [ 21.643061] NAND device: Manufacturer ID: 0x98, Chip ID: 0xda (Toshiba NAND 256MiB 3,3V 8-bit) [ 21.651738] NAND device: 256MiB, SLC, page size: 2048, OOB size: 64 [ 21.658038] Scanning device for bad blocks [ 21.877644] 3 cmdlinepart partitions found on MTD device orion_nand [ 21.883957] Creating 3 MTD partitions on "orion_nand": [ 21.889126] 0x000000000000-0x000000100000 : "u-boot" [ 21.894355] 0x000000100000-0x000000700000 : "uImage" [ 21.899552] 0x000000700000-0x000010000000 : "root" [ 21.905346] libphy: Fixed MDIO Bus: probed [ 21.921337] libphy: orion_mdio_bus: probed [ 21.925535] mv643xx_eth: MV-643xx 10/100/1000 ethernet driver version 1.4 [ 22.933258] mv643xx_eth_port mv643xx_eth_port.0 eth0: port 0 with MAC address 02:50:43:67:ed:34 [ 22.942181] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 22.948740] ehci-pci: EHCI PCI platform driver [ 22.953271] ehci-orion: EHCI orion driver [ 22.957352] orion-ehci orion-ehci.0: EHCI Host Controller [ 22.962822] orion-ehci orion-ehci.0: new USB bus registered, assigned bus number 1 [ 22.970651] orion-ehci orion-ehci.0: irq 19, io mem 0xf1050000 [ 22.991233] orion-ehci orion-ehci.0: USB 2.0 started, EHCI 1.00 [ 22.997359] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 [ 23.004198] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 23.011473] usb usb1: Product: EHCI Host Controller [ 23.016375] usb usb1: Manufacturer: Linux 3.13.1-kirkwood-tld-2 ehci_hcd [ 23.023121] usb usb1: SerialNumber: orion-ehci.0 [ 23.028220] hub 1-0:1.0: USB hub found [ 23.032050] hub 1-0:1.0: 1 port detected [ 23.036458] mousedev: PS/2 mouse device common for all mice [ 23.042404] rtc-mv rtc-mv: rtc core: registered rtc-mv as rtc0 [ 23.048335] i2c /dev entries driver [ 23.052425] drop_monitor: Initializing network drop monitor service [ 23.058905] TCP: cubic registered [ 23.062272] NET: Registered protocol family 17 [ 23.066833] Key type dns_resolver registered [ 23.071616] registered taskstats version 1 [ 23.076577] rtc-mv rtc-mv: setting system clock to 2014-02-11 19:57:08 UTC (1392148628) [ 23.085527] Freeing unused kernel memory: 208K (c0623000 - c0657000) [ 23.161862] udevd[54]: starting version 175 [ 23.309703] SCSI subsystem initialized [ 23.351359] usb 1-1: new high-speed USB device number 2 using orion-ehci [ 23.383373] sata_mv sata_mv.0: cannot get optional clkdev [ 23.388853] sata_mv sata_mv.0: slots 32 ports 1 [ 23.438269] scsi0 : sata_mv [ 23.451376] ata1: SATA max UDMA/133 irq 21 [ 23.514785] usb 1-1: New USB device found, idVendor=0718, idProduct=0708 [ 23.521551] usb 1-1: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [ 23.528735] usb 1-1: Product: Micro [ 23.532257] usb 1-1: Manufacturer: Memorex [ 23.536375] usb 1-1: SerialNumber: 0707330DB5DBC920 [ 23.557907] usb-storage 1-1:1.0: USB Mass Storage device detected [ 23.564347] scsi1 : usb-storage 1-1:1.0 [ 23.569730] usbcore: registered new interface driver usb-storage [ 23.801245] ata1: SATA link down (SStatus 0 SControl F300) [ 24.563633] scsi 1:0:0:0: Direct-Access Memorex Micro PMAP PQ: 0 ANSI: 4 [ 24.600216] sd 1:0:0:0: [sda] 7570752 512-byte logical blocks: (3.87 GB/3.60 GiB) [ 24.608692] sd 1:0:0:0: [sda] Write Protect is off [ 24.614182] sd 1:0:0:0: [sda] No Caching mode page found [ 24.619532] sd 1:0:0:0: [sda] Assuming drive cache: write through [ 24.629805] sd 1:0:0:0: [sda] No Caching mode page found [ 24.635211] sd 1:0:0:0: [sda] Assuming drive cache: write through [ 24.642679] sda: sda1 [ 24.647930] sd 1:0:0:0: [sda] No Caching mode page found [ 24.653312] sd 1:0:0:0: [sda] Assuming drive cache: write through [ 24.659435] sd 1:0:0:0: [sda] Attached SCSI removable disk [ 24.675466] sd 1:0:0:0: Attached scsi generic sg0 type 0 [ 33.869693] netpoll: netconsole: local port 6665 [ 33.874414] netpoll: netconsole: local IPv4 address 10.10.10.6 [ 33.880271] netpoll: netconsole: interface 'eth0' [ 33.885027] netpoll: netconsole: remote port 6666 [ 33.889751] netpoll: netconsole: remote IPv4 address 10.10.10.4 [ 33.895764] netpoll: netconsole: remote ethernet address ff:ff:ff:ff:ff:ff [ 33.902705] netpoll: netconsole: device eth0 not up yet, forcing it [ 33.910713] mv643xx_eth_port mv643xx_eth_port.0 eth0: link down [ 37.030038] mv643xx_eth_port mv643xx_eth_port.0 eth0: link up, 1000 Mb/s, full duplex, flow control disabled [ 37.042367] console [netcon0] enabled [ 37.046054] netconsole: network logging started [ 37.094352] F2FS-fs (sda1): Magic Mismatch, valid(0xf2f52010) - read(0x39c60) [ 37.101657] F2FS-fs (sda1): Can't find a valid F2FS filesystem in 1th superblock [ 37.109832] F2FS-fs (sda1): Magic Mismatch, valid(0xf2f52010) - read(0x0) [ 37.116744] F2FS-fs (sda1): Can't find a valid F2FS filesystem in 2th superblock [ 37.248186] uhci_hcd: USB Universal Host Controller Interface driver [ 37.261458] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver [ 37.277435] hidraw: raw HID events driver (C) Jiri Kosina [ 37.285173] usbcore: registered new interface driver usbhid [ 37.290816] usbhid: USB HID core driver
The boot process basically hangs there.
So how (and what) do I change the Magic number to? How do I configure nc so I can have an interactive session so I can change the appropriate uboot firmware variable to boot the Debian image?
Edited 1 time(s). Last edit at 02/11/2014 02:44PM by beausoleil.
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 11, 2014 03:09PM |
Admin Registered: 13 years ago Posts: 18,545 |
Try this for the second netconsole command:
nc -up 6666 <ipaddr of GFhome> 6666
When you see the count down, press ctrl-j repeatedly.
Once you are at the prompt, get the output of
printenv
And post here! I recalled some user who was in the same situation had to adjust the bootargs so the Debian kernel will recognizes the file system properly(it identified the rootfs type incorectly).
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
nc -up 6666 <ipaddr of GFhome> 6666
When you see the count down, press ctrl-j repeatedly.
Once you are at the prompt, get the output of
printenv
And post here! I recalled some user who was in the same situation had to adjust the bootargs so the Debian kernel will recognizes the file system properly(it identified the rootfs type incorectly).
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 11, 2014 03:13PM |
Registered: 10 years ago Posts: 13 |
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 11, 2014 03:18PM |
Admin Registered: 13 years ago Posts: 18,545 |
killall nc
to kill all previous nc instances before running new session.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
to kill all previous nc instances before running new session.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 11, 2014 03:33PM |
Registered: 10 years ago Posts: 13 |
So when you said "Try this for the second netconsole command", you meant not to run an instance in the background first... ;-)
So what's preferable? Change all the references of "ext2" to "ext3", or just convert the filesystem on the flash drive to ext2?
Marvell>> printenv printenv bootcmd=run fast_boot; run usb_boot; run hd_boot; run ubi_boot bootdelay=3 baudrate=115200 ide_start=ide reset ubi_start=ubi part root usb_start=usb start nc_test=ping $ncip nc_start=setenv stdin nc; setenv stdout nc; setenv stderr nc; version silent_boot=run silent_rd ubi_args ubi_fallback fast_boot=run fast_rd ubi_args ubi_fallback usb_boot=for scan in 0 1 2 3; do run usb_args_$scan ext2_kern ext2_boot; run ext2_rd ubi_fallback; run fat_kern fat_boot; run fat_rd ubi_fallback;done usb_args_0=boot_dev='usb 0:1'; dev_args='root=/dev/sda1 rootdelay=10' usb_args_1=boot_dev='usb 1:1'; dev_args='root=/dev/sdb1 rootdelay=10' usb_args_2=boot_dev='usb 2:1'; dev_args='root=/dev/sdc1 rootdelay=10' usb_args_3=boot_dev='usb 3:1'; dev_args='root=/dev/sdd1 rootdelay=10' hd_boot=run ide_start; for scan in 0 1; do run hd_args_$scan ext2_kern ext2_boot; run ext2_rd ubi_fallback; run fat_kern fat_boot; run fat_rd ubi_fallback; done
So what's preferable? Change all the references of "ext2" to "ext3", or just convert the filesystem on the flash drive to ext2?
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 11, 2014 09:00PM |
Admin Registered: 13 years ago Posts: 18,545 |
Your envs listing was cut off! Should be more.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 07:39AM |
Registered: 10 years ago Posts: 13 |
So I tried again, and got to the Marvell>> prompt.
In an attempt to make further troubleshooting, I decided to change the netconsole destination, because I wanted to plug the GoFlex back into my wireless router, and access it remotely rather than a direct connection to my laptop. I made the following changes:
At that point, I lost the connection to the GoFlex. I waited a couple minutes before powering it down.
On my laptop, I moved the 10.10.10.x address info from eth0 to eth1 (my wireless interface). I connected the GoFlex to a LAN port on the wireless router, setup an nc session on my laptop ("nc -up 6666 192.168.2.101 6666"), and powered up the GoFlex, but got nothing. A Wireshark trace revealed that the GoFlex is still using the 10.10.10.6 source address - but now trying to reach 10.10.10.3 instead of .5 as before. So I tried "nc -up 6666 10.10.10.6", but got nothing on nc.
I monitored the boot process from Wireshark. It appears it's sending the ubit log info to 10.10.10.3 instead of .5 as before. I have 10.10.10.5, .4, .3 and .2 as secondary addresses on my laptop's wireless interface. Wireshark shows that my laptop's 10.10.10.3 interface is replying to .5 that the destination port (6666) is unreachable. What's the actual mechanism for the IP stack to source UDP data coming into various pseudo-interfaces? I'm a network guy, not an OS guy, so I don't know exactly how the IP stack handles data destined for different ip addresses but the same port #.
Anyway, here's the packet capture:
For now, since the u-bit boot handler is getting nowhere after the environment changes I made, I'll try defining only a single 10. address - 10.10.10.3 on my laptop's wireless interface and see if I can get back into the GoFlex and report back here what I find...
In an attempt to make further troubleshooting, I decided to change the netconsole destination, because I wanted to plug the GoFlex back into my wireless router, and access it remotely rather than a direct connection to my laptop. I made the following changes:
Marvell>> setenv serverip 192.168.2.100 Marvell>> setenv if_netconsole 'ping $serverip' Marvell>> setenv start_netconsole 'setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;' Marvell>> setenv preboot 'run if_netconsole start_netconsole' Marvell>> setenv ipaddr 192.168.2.101 Marvell>> saveenv
At that point, I lost the connection to the GoFlex. I waited a couple minutes before powering it down.
On my laptop, I moved the 10.10.10.x address info from eth0 to eth1 (my wireless interface). I connected the GoFlex to a LAN port on the wireless router, setup an nc session on my laptop ("nc -up 6666 192.168.2.101 6666"), and powered up the GoFlex, but got nothing. A Wireshark trace revealed that the GoFlex is still using the 10.10.10.6 source address - but now trying to reach 10.10.10.3 instead of .5 as before. So I tried "nc -up 6666 10.10.10.6", but got nothing on nc.
I monitored the boot process from Wireshark. It appears it's sending the ubit log info to 10.10.10.3 instead of .5 as before. I have 10.10.10.5, .4, .3 and .2 as secondary addresses on my laptop's wireless interface. Wireshark shows that my laptop's 10.10.10.3 interface is replying to .5 that the destination port (6666) is unreachable. What's the actual mechanism for the IP stack to source UDP data coming into various pseudo-interfaces? I'm a network guy, not an OS guy, so I don't know exactly how the IP stack handles data destined for different ip addresses but the same port #.
Anyway, here's the packet capture:
U-Boot 2010.09 (Feb 16 2011 - 18:44:23) UBIT v0.6 by Jeff Doozan and Peter Carmichael Hit any key to stop autoboot: 10 ... 9 ... 8 ... 7 ... 6 ... 5 ... 4 ... 3 ... 2 ... 1 ... 0 UBIFS error (pid 0): ubifs_get_sb: cannot open "ubi:fast", error -19 Error reading superblock on volume 'ubi:fast'! UBIFS not mounted, use ubifs mount to mount volume first! ** Block device usb 0 not supported ** Invalid boot device ** ** Block device usb 1 not supported ** Invalid boot device ** ** Block device usb 2 not supported ** Invalid boot device ** ** Block device usb 3 not supported ** Invalid boot device ** Reset IDE: Bus 0: not available Bus 1: not available ** Bad partition 1 ** ** Bad partition 1 ** No Powersaving mode 7F Error (no IRQ) dev 0 blk 0: status 0x7f ** Can't read from device 0 ** ** Unable to use ide 0:1 for fatload ** No Powersaving mode 7F Error (no IRQ) dev 0 blk 0: status 0x7f ** Can't read from device 0 ** ** Unable to use ide 0:1 for fatload ** ** Bad partition 1 ** ** Bad partition 1 ** No Powersaving mode 7F Error (no IRQ) dev 1 blk 0: status 0x7f ** Can't read from device 1 ** ** Unable to use ide 1:1 for fatload ** No Powersaving mode 7F Error (no IRQ) dev 1 blk 0: status 0x7f ** Can't read from device 1 ** ** Unable to use ide 1:1 for fatload ** UBIFS error (pid 0): ubifs_get_sb: cannot open "ubi:ramdisk", error -19 Error reading superblock on volume 'ubi:ramdisk'! UBIFS not mounted, use ubifs mount to mount volume first! Marvell>>
For now, since the u-bit boot handler is getting nowhere after the environment changes I made, I'll try defining only a single 10. address - 10.10.10.3 on my laptop's wireless interface and see if I can get back into the GoFlex and report back here what I find...
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 12:10PM |
Registered: 10 years ago Posts: 13 |
I can't get any further than doing a printenv at the Marvell>> prompt. According to Wireshark, the GoFlex sends a malformed UDP packet and it looks like nc on my laptop barfs on it and exits. But the GoFlex continues sending data, so I'll parse it out from Wireshark and post it here... Might have to go back to a wired connection - could be xmit collisions on wifi...
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 12:58PM |
Registered: 10 years ago Posts: 13 |
Here's the beginning of full text of the netconsole output which I extracted from Wireshark:
nc on my laptop barfed on the first mangled packet. The trace included ICMP packets echoed back to the source saying the destination port was invalid (since nc crashed). I filtered those out to figure what that mangled line should read...
EDIT - I should also mention that I've also changed the fs type on the usb flash drive from ext3 to ext2, and modified its /etc/fstab to reflect that. Are there any init scripts or config files that I might also need to change?
Edited 1 time(s). Last edit at 02/12/2014 02:07PM by beausoleil.
Marvell>> printenv printenv bootcmd=run fast_boot; run usb_boot; run hd_boot; run ubi_boot baudrate=115200 ide_start=ide reset ubi_start=ubi part root usb_start=usb start nc_test=ping $ncip nc_start=setenv stdin nc; setenv stdout nc; setenv stderr nc; version silent_boot=run silent_rd ubi_args ubi_fallback fast_boot=run fast_rd ubi_args ubi_fallback usb_boot=for scan in 0 1 2 3; do run usb_args_$scan ext2_kern ext2_boot; run ext2_rd ubi_fallback; run fat_kern fat_boot; run fat_rd ubi_fallback;done usb_args_0=boot_dev='usb 0:1'; dev_args='root=/dev/sda1 rootdelay=10' usb_args_1=boot_dev='usb 1:1'; dev_args='root=/dev/sdb1 rootdelay=10' usb_args_2=boot_dev='usb 2:1'; dev_args='root=/dev/sdc1 rootdelay=10' usb_args_3=boot_dev='usb 3:1'; dev_args='root=/dev/sdd1 rootdelay=10' hd_boot=run ide_start; for scan in 0 1; do run hd_args_$scan ext2_kern ext2_boot; run ext2_rd ubi_fallback; run fat_kern fat_boot; run fat_rd ubi_fallback; doneThis next line was mangled:
hd_args_0=boot_dhd_args_0=boot_dev='ide 0:1'; deev='ide 0:1'; dev_args='root=/dev_args='root=/dev/sda1'v/sda1'I think it should read:
hd_args_0=boot_dev='ide 0:1'; dev_args='root=/dev/sda1'Here's the rest of the printenv output:
hd_args_1=boot_dev='ide 1:1'; dev_args='root=/dev/sdb1' ubi_boot=run ubi_rd ubi_args ubi_fallback ubi_args=dev_args='ubi.mtd=root' rescue=dev_args='ubi.mtd=root rootfstype=preboot'; boot_dev='usb 0:1'; run ext2_kern ext2_auto boot_rd; boot_dev='ide 0:1'; run ide_start ext2_kern ext2_auto boot_rd; run silent_rd ubi_fallback; run fast_rd ubi_fallback; run ubi_rd ubi_fallback chain=nand read.e 0x800000 0x480000 0x80000; go 0x800200 ethact=egiga0 console=ttyS0,115200 mtdids=nand0=orion_nand set_bootargs=setenv bootargs console=$console $mtdparts $dev_args $rd_args netconsole=@$ipaddr/eth0,@$ncipk/ ext2_kern=ext2load $boot_dev $addr_kern /boot/uImage ext2_auto=ext2load $boot_dev $addr_rd /boot/uInitrd ext2_rd=ext2load $boot_dev $addr_rd /uInitrd ext2_boot=run ext2_rd boot_rd; run ext2_auto boot_auto; run boot_kern fat_kern=fatload $boot_dev $addr_kern /boot/uImage fat_rd=fatload $boot_dev $addr_rd /uInitrd fat_boot=run fat_rd boot_rd ubi_kern=ubifsmount boot; ubifsload $addr_kern /boot/uImage ubi_rd=ubifsmount ramdisk; ubifsload $addr_rd /uInitrd ubi_fallback=run ubi_kern boot_rd silent_rd=ubifsmount silent; ubifsload $addr_rd /uInitrd fast_rd=ubifsmount fast; ubifsload $addr_rd /uInitrd boot_kern=run set_bootargs; bootm $addr_kern boot_rd=run set_bootargs; bootm $addr_kern $addr_rd boot_auto=rd_args='rootfstype=auto'; run boot_rd addr_kern=0x680000 addr_rd=0x1100000 ipaddr=10.10.10.6 ncipk=10.10.10.4 serverip=10.10.10.3 led_init=green blinking led_exit=green off led_error=orange blinking ethaddr=02:50:43:67:ed:34 mtdparts=mtdparts=orion_nand:1M(u-boot),6M(uImage),-(root) partition=nand0,0 mtddevnum=0 mtddevname=u-boot if_netconsole=ping $serverip start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version; preboot=run if_netconsole start_netconsole ncip=10.10.10.3 bootdelay=10 stdin=nc stdout=nc stderr=nc Environment size: 2867/131068 bytes Marvell>>
nc on my laptop barfed on the first mangled packet. The trace included ICMP packets echoed back to the source saying the destination port was invalid (since nc crashed). I filtered those out to figure what that mangled line should read...
EDIT - I should also mention that I've also changed the fs type on the usb flash drive from ext3 to ext2, and modified its /etc/fstab to reflect that. Are there any init scripts or config files that I might also need to change?
Edited 1 time(s). Last edit at 02/12/2014 02:07PM by beausoleil.
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 03:35PM |
Admin Registered: 13 years ago Posts: 18,545 |
Even without seeing the rest of the boot log, I think your problem is with the rootfs type. See this solution:
http://forum.doozan.com/read.php?3,14982,14994#msg-14994
If that still does not get it to boot into Debian, then I need to examine the bootlog again.
And no, there is no need to change any thing in the rootfs. Even fstab change is only a precaution (not really needed if the rootfs file system is ext3 and the kernel expects Ext2, athough not the other way around).
Update: yes, rootfs type is the problem.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Edited 2 time(s). Last edit at 02/12/2014 04:03PM by bodhi.
http://forum.doozan.com/read.php?3,14982,14994#msg-14994
If that still does not get it to boot into Debian, then I need to examine the bootlog again.
And no, there is no need to change any thing in the rootfs. Even fstab change is only a precaution (not really needed if the rootfs file system is ext3 and the kernel expects Ext2, athough not the other way around).
Update: yes, rootfs type is the problem.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Edited 2 time(s). Last edit at 02/12/2014 04:03PM by bodhi.
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 04:07PM |
Registered: 10 years ago Posts: 13 |
No go.
First, I did a printenv to see what boot_auto was set to:
Must've got another malformed packet. Re-ran nc and got back in. Then I did:
Just for sanity: my usb flash drive is partitioned MBR, not GPT. Boot flag is set, partition id is 0x83,
Edited 1 time(s). Last edit at 02/12/2014 04:39PM by beausoleil.
First, I did a printenv to see what boot_auto was set to:
Marvell>> printenv boot_auto printenv boot_auto boot_auto=rd_args='rootfstype=auto'; run boot_rdjon@jon-Latitude-E6500 ~ $
Must've got another malformed packet. Re-ran nc and got back in. Then I did:
Marvell>> setenv boot_auto 'setenv rd_args rootfstype=ext2; run boot_rd' setenv boot_auto 'setenv rd_args rootfstype=ext2; run boot_rd' Marvell>> boot boot UBIFS error (pid 0): ubifs_get_sb: cannot open "ubi:fast", error -19 Error reading superblock on volume 'ubi:fast'! UBIFS not mounted, use ubifs mount to mount volume first! ** Block device usb 0 not supported ** Block device usb 0 not supported ** Invalid boot device ** ** Invalid boot device ** ** Block device usb 1 not supported ** Block device usb 1 not supported ** Invalid boot device ** ** Invalid boot device ** ** Block device usb 2 not supported ** Block device usb 2 not supported ** Invalid boot device ** ** Invalid boot device ** ** Block device usb 3 not supported ** Block device usb 3 not supported ** Invalid boot device ** ** Invalid boot device ** Reset IDE: Bus 0: not available Bus 1: not available ** Bad partition 1 ** ** Bad partition 1 ** No Powersaving mode 7F Error (no IRQ) dev 0 blk 0: status 0x7f ** Can't read from device 0 ** ** Unable to use ide 0:1 for fatload ** No Powersaving mode 7F Error (no IRQ) dev 0 blk 0: status 0x7f ** Can't read from device 0 ** ** Unable to use ide 0:1 for fatload ** ** Bad partition 1 ** ** Bad partition 1 ** No Powersaving mode 7F Error (no IRQ) dev 1 blk 0: status 0x7f ** Can't read from device 1 ** ** Unable to use ide 1:1 for fatload ** No Powersaving mode 7F Error (no IRQ) dev 1 blk 0: status 0x7f ** Can't read from device 1 ** ** Unable to use ide 1:1 for fatload ** UBIFS error (pid 0): ubifs_get_sb: cannot open "ubi:ramdisk", error -19 Error reading superblock on volume 'ubi:ramdisk'! UBIFS not mounted, use ubifs mount to mount volume first! Marvell>>
Just for sanity: my usb flash drive is partitioned MBR, not GPT. Boot flag is set, partition id is 0x83,
Edited 1 time(s). Last edit at 02/12/2014 04:39PM by beausoleil.
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 05:05PM |
Registered: 10 years ago Posts: 13 |
Finally got it to boot.
With the GoFlex still powered on, I removed the flash drive and placed it in my laptop just to verify the partition info.Then I plugged it back into the GoFlex.
From the Marvell>> prompt, I issued a "usb tree" command, and found that usb service wasn't started. So I started it by "usb start", and it found my drive. Then I did a "usb tree" again just to verify, then issued "usb storage" and "usb dev" just for kicks. Then issued "boot".
I then saw it found the /boot/uImage and /boot/uInitrd files, and it said it was booting the kernel image. I then used Wireshark to see what it did on the network - it got a dhcp lease from the wireless router (I'm in temporary corporate housing during a move and don't have admin access to the wifi router...)
So now that I'm in via ssh and have a root shell, what info do you need to see so I can save the firmware environment to nand?
Found this problem:
Edited 2 time(s). Last edit at 02/12/2014 05:09PM by beausoleil.
With the GoFlex still powered on, I removed the flash drive and placed it in my laptop just to verify the partition info.Then I plugged it back into the GoFlex.
From the Marvell>> prompt, I issued a "usb tree" command, and found that usb service wasn't started. So I started it by "usb start", and it found my drive. Then I did a "usb tree" again just to verify, then issued "usb storage" and "usb dev" just for kicks. Then issued "boot".
I then saw it found the /boot/uImage and /boot/uInitrd files, and it said it was booting the kernel image. I then used Wireshark to see what it did on the network - it got a dhcp lease from the wireless router (I'm in temporary corporate housing during a move and don't have admin access to the wifi router...)
So now that I'm in via ssh and have a root shell, what info do you need to see so I can save the firmware environment to nand?
Found this problem:
root@debian:~# fw_printenv
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200
root@debian:~#
Edited 2 time(s). Last edit at 02/12/2014 05:09PM by beausoleil.
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 12, 2014 11:06PM |
Admin Registered: 13 years ago Posts: 18,545 |
beausoleil,
This is because you're running an older version of U-Boot. This version probably defines the envs at a different location than 0xC0000. At this point, you have 2 different options:
1. Update U-Boot to the current version by running Jeff's script, see http://projects.doozan.com/uboot. This will allow you to take advantage of newer features.
2. Find out where your U-Boot defines the envs and then modify /etc/fw_env.config. This is a very safe thing to do, since you already running Debian.
With no serial console, updating U-Boot has some small risk. However, Jeff's script is the most reliable way to update your u-Boot, so I would do that. Just remember don't reboot if you see any error during the script execution, post the log here if that happens. And also, your u-Boot envs will be very different, your netconsole will need to be set up again.
After updating u-Boot, setup netconsole right away before rebooting. Again, if anything goes wrong during updating u-Boot, you will need serial console.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
> root@debian:~# fw_printenv
> Warning: Bad CRC, using default environment
> bootcmd=bootp; setenv bootargs root=/dev/nfs
> nfsroot=${serverip}:${rootpath}
> ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:$
> {hostname}::off; bootm
> bootdelay=5
> baudrate=115200
> root@debian:~#
>
This is because you're running an older version of U-Boot. This version probably defines the envs at a different location than 0xC0000. At this point, you have 2 different options:
1. Update U-Boot to the current version by running Jeff's script, see http://projects.doozan.com/uboot. This will allow you to take advantage of newer features.
2. Find out where your U-Boot defines the envs and then modify /etc/fw_env.config. This is a very safe thing to do, since you already running Debian.
With no serial console, updating U-Boot has some small risk. However, Jeff's script is the most reliable way to update your u-Boot, so I would do that. Just remember don't reboot if you see any error during the script execution, post the log here if that happens. And also, your u-Boot envs will be very different, your netconsole will need to be set up again.
After updating u-Boot, setup netconsole right away before rebooting. Again, if anything goes wrong during updating u-Boot, you will need serial console.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 13, 2014 07:03AM |
Registered: 10 years ago Posts: 13 |
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 14, 2014 09:31AM |
Registered: 10 years ago Posts: 13 |
This is frustrating as hell. Still having trouble with nc...
Can anyone explain the correct setup (secondary ip addresses on my laptop) to reliably access the console ports while the GFH is booting? Specifically, how do I monitor stdout/stdin/stderr?
It seems that there are differences in nc depending on whether it's the bsd version (default on my Linux Mint 16 laptop) or the Debian traditional (v.1.10-40)
On my laptop, I have eth1 defined as 192.168.2.100/24, eth1:0 10.10.10.5/8, eth1:1 10.10.10.4/8, eth1:2 10.10.10.3/8, and eth1:3 10.10.10.2/8
Up until this morning, I was using the bsd version. I uninstalled it and am trying the traditional (v.1.10-40) version.
With the bsd version of nc, using "What's happening is that when I invoke the old version using "nc -up 6666 10.10.10.6 6666", I can get a session to the GFH (which is 10.10.10.6). U-Boot starts, but errors out because the USB service doesn't start for some reason. I can issue a "usb start" command, then "boot", and it begins loading the kernel, but as far as I can tell, it halts at "Starting kernel ...".
This same version of nc suffers from the "printenv" problem someone alluded to earlier in this thread. As I said in a previous post, Wireshark says the nc barfs when it receives a malformed packet from the GFH. Since it seems it's reproducible, it looks like a bug on the GFH side, since every time I issue a "printenv" command, the output results in a malformed packet every time it gets to the "hd_args_0" variable.
Here's the problem. If I give nc the "-k" option to set the keepalive option on the socket, then I can't specify the GFH's host address - I have to invoke it like this: "nc -klu 6666" - and I get nothing. Wireshark shows that my laptop issues an IMCP "destination unreachable (port unreachable) message back to the GFH. nc on the GFH continues sending the console output, but nc on my laptop doesn't see it.
I tried removing the bsd and installing the traditional version, but all I can get using "nc -lkup 6666 10.10.10.6 6666" is the first couple of lines of the U-Boot log output ("U-Boot 2010.09..."), then nothing. Wireshark shows the same thing - my laptop responds with ICMP destination unreachable messages, which the GFH ignores while it merrily sends each UDP console log packet to my laptop's 10.10.10.3 address...
Any ideas?
Can anyone explain the correct setup (secondary ip addresses on my laptop) to reliably access the console ports while the GFH is booting? Specifically, how do I monitor stdout/stdin/stderr?
It seems that there are differences in nc depending on whether it's the bsd version (default on my Linux Mint 16 laptop) or the Debian traditional (v.1.10-40)
On my laptop, I have eth1 defined as 192.168.2.100/24, eth1:0 10.10.10.5/8, eth1:1 10.10.10.4/8, eth1:2 10.10.10.3/8, and eth1:3 10.10.10.2/8
Up until this morning, I was using the bsd version. I uninstalled it and am trying the traditional (v.1.10-40) version.
With the bsd version of nc, using "What's happening is that when I invoke the old version using "nc -up 6666 10.10.10.6 6666", I can get a session to the GFH (which is 10.10.10.6). U-Boot starts, but errors out because the USB service doesn't start for some reason. I can issue a "usb start" command, then "boot", and it begins loading the kernel, but as far as I can tell, it halts at "Starting kernel ...".
This same version of nc suffers from the "printenv" problem someone alluded to earlier in this thread. As I said in a previous post, Wireshark says the nc barfs when it receives a malformed packet from the GFH. Since it seems it's reproducible, it looks like a bug on the GFH side, since every time I issue a "printenv" command, the output results in a malformed packet every time it gets to the "hd_args_0" variable.
Here's the problem. If I give nc the "-k" option to set the keepalive option on the socket, then I can't specify the GFH's host address - I have to invoke it like this: "nc -klu 6666" - and I get nothing. Wireshark shows that my laptop issues an IMCP "destination unreachable (port unreachable) message back to the GFH. nc on the GFH continues sending the console output, but nc on my laptop doesn't see it.
I tried removing the bsd and installing the traditional version, but all I can get using "nc -lkup 6666 10.10.10.6 6666" is the first couple of lines of the U-Boot log output ("U-Boot 2010.09..."), then nothing. Wireshark shows the same thing - my laptop responds with ICMP destination unreachable messages, which the GFH ignores while it merrily sends each UDP console log packet to my laptop's 10.10.10.3 address...
Any ideas?
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 14, 2014 04:36PM |
Registered: 10 years ago Posts: 5 |
Although this is only a minor point, the reason that your (U-Boot) netconsole appears to halt at "Starting kernel" is that at that point U-Boot is done, and with it the netconsole functionality.
The next stage is the ksyslog output from the kernel, which will go to a memory buffer. However, the Debian standard initramfs allows the Linux "netconsole" functionality, which will send the ksyslog output somewhere over the network (in full-on rsyslog/syslog-ng compatible formats).
But this has to be enabled, and it is a one-way channel.
To enable it, you need to add an argument to bootargs (so /proc/cmdline from the kernel) of the form:
.... where the first 6666 is used as the origin port (so irrelevant to all intents and purposes), $ipaddr is the address of the sending/booting device, $serverport is the port on the destination server to send the traffic, and obviously $serverip is the address you want to send to.
I use rsyslog, and so have a file in /etc/rsyslog.d called 10-network-udp.conf that contains:
This gives me the output from the kernel like this:
Which is great for the warm-n-fuzzies, but frankly the most useful bit is usually the command line!
The critical problem is that this is only the kernel logging, and so the next stage of the boot, the initramfs init script doesn't have it's output relayed. So you don't get to see the messages of the form "Mounting root file system / mount failed due to some incredibly simple error that you could fix in a heartbeat if only you knew what it was". And you don't get to type anything at the "console" where the machine is sitting smugly waiting for you to type "fsck -y /dev/root" or something equally simple.
The next stage is the ksyslog output from the kernel, which will go to a memory buffer. However, the Debian standard initramfs allows the Linux "netconsole" functionality, which will send the ksyslog output somewhere over the network (in full-on rsyslog/syslog-ng compatible formats).
But this has to be enabled, and it is a one-way channel.
To enable it, you need to add an argument to bootargs (so /proc/cmdline from the kernel) of the form:
netconsole=6666@$ipaddr/eth0,$serverport@$serverip/
.... where the first 6666 is used as the origin port (so irrelevant to all intents and purposes), $ipaddr is the address of the sending/booting device, $serverport is the port on the destination server to send the traffic, and obviously $serverip is the address you want to send to.
I use rsyslog, and so have a file in /etc/rsyslog.d called 10-network-udp.conf that contains:
# enable UDP syslog reception $ModLoad imudp $UDPServerRun 514 if $fromhost-ip startswith '10.10.10.' then /var/log/net-udp.log & ~
This gives me the output from the kernel like this:
Jan 29 16:22:55 10.10.10.6 [ 0.000000] Booting Linux on physical CPU 0x0 Jan 29 16:22:55 10.10.10.6 [ 0.000000] Initializing cgroup subsys cpuset Jan 29 16:22:55 10.10.10.6 [ 0.000000] Initializing cgroup subsys cpu Jan 29 16:22:55 10.10.10.6 [ 0.000000] Initializing cgroup subsys cpuacct Jan 29 16:22:55 10.10.10.6 [ 0.000000] Linux version 3.12.0-kirkwood-tld-3 (root@tldDebian) (gcc version 4.6.3 (Debian 4.6.3-1 4) ) #3 PREEMPT Wed Nov 27 16:10:53 PST 2013 Jan 29 16:22:55 10.10.10.6 [ 0.000000] CPU: Feroceon 88FR131 [56251311] revision 1 (ARMv5TE), cr=00053977 Jan 29 16:22:55 10.10.10.6 [ 0.000000] CPU: VIVT data cache, VIVT instruction cache Jan 29 16:22:55 10.10.10.6 [ 0.000000] Machine: Pogoplug V4 Jan 29 16:22:55 10.10.10.6 [ 0.000000] Memory policy: ECC disabled, Data cache writeback Jan 29 16:22:55 10.10.10.6 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 32512 Jan 29 16:22:55 10.10.10.6 [ 0.000000] Kernel command line: console=ttyS0,115200 root=/dev/sda1 rootdelay=10 rootfstype=ext3 m tdparts=orion_nand:3M(u-boot),3M(uImage),3M(uImage2),8M(failsafe),112M(root) netconsole=6666@10.10.10.6/eth0,514@10.10.10.10/ Jan 29 16:22:55 10.11.10.6 [ 0.000000] PID hash table entries: 512 (order: -1, 2048 bytes) Jan 29 16:22:55 10.10.10.6 [ 0.000000] Dentry cache hash table entries: 16384 (order: 4, 65536 bytes) Jan 29 16:22:55 10.10.10.6 [ 0.000000] Inode-cache hash table entries: 8192 (order: 3, 32768 bytes) Jan 29 16:22:55 10.10.10.6 [ 0.000000] Memory: 116572K/131072K available (4398K kernel code, 333K rwdata, 1540K rodata, 206K i nit, 420K bss, 14500K reserved)
Which is great for the warm-n-fuzzies, but frankly the most useful bit is usually the command line!
The critical problem is that this is only the kernel logging, and so the next stage of the boot, the initramfs init script doesn't have it's output relayed. So you don't get to see the messages of the form "Mounting root file system / mount failed due to some incredibly simple error that you could fix in a heartbeat if only you knew what it was". And you don't get to type anything at the "console" where the machine is sitting smugly waiting for you to type "fsck -y /dev/root" or something equally simple.
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 14, 2014 05:36PM |
Admin Registered: 13 years ago Posts: 18,545 |
Setup netconsole to see kernel messages without changing uBoot envs:
http://forum.doozan.com/read.php?2,9522
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
http://forum.doozan.com/read.php?2,9522
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 14, 2014 05:48PM |
Admin Registered: 13 years ago Posts: 18,545 |
beausoleil,
> This is frustrating as hell. Still having trouble
> with nc…
Remember that you are running an old version of u-Boot. u-Boot netconsole has been buggy up until the latest u-Boot :) so I would say if you install the new u-Boot by running Jeff's script, then netconsole will be much more reliable. Still, there were reports about intermittent problems with it.
Do you have a second plug? that allows you to use one to monitor the other? this setup never failed on me.
Other than that, I guess the only thing you can do is trying different nc versions on your laptop.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
> This is frustrating as hell. Still having trouble
> with nc…
Remember that you are running an old version of u-Boot. u-Boot netconsole has been buggy up until the latest u-Boot :) so I would say if you install the new u-Boot by running Jeff's script, then netconsole will be much more reliable. Still, there were reports about intermittent problems with it.
Do you have a second plug? that allows you to use one to monitor the other? this setup never failed on me.
Other than that, I guess the only thing you can do is trying different nc versions on your laptop.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 14, 2014 05:49PM |
Registered: 10 years ago Posts: 13 |
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 17, 2014 10:15AM |
Registered: 10 years ago Posts: 13 |
bodhi,
Not sure what you mean by second plug. Second PogoPlug device? No.
And I can't run Jeff's upgrade script because I can't get to the console of the GFH. The best I'm getting now is "kernel booting". But I have no idea if it completes its boot. What ip address does that image of Debian default to if it there's no dhcp server on the network segment it's on?
Not sure what you mean by second plug. Second PogoPlug device? No.
And I can't run Jeff's upgrade script because I can't get to the console of the GFH. The best I'm getting now is "kernel booting". But I have no idea if it completes its boot. What ip address does that image of Debian default to if it there's no dhcp server on the network segment it's on?
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 17, 2014 11:34PM |
Admin Registered: 13 years ago Posts: 18,545 |
beausolei,
Setup a static IP in your rootfs. Example (if your router is 192.168.0.1):
#cat /etc/network/interfaces
After that if you can ping it, then likely it's booted. See if you can ssh in.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
Setup a static IP in your rootfs. Example (if your router is 192.168.0.1):
#cat /etc/network/interfaces
auto lo eth0
iface lo inet loopback
iface eth0 inet static
address 192.168.0.100
netmask 255.255.255.0
gateway 192.168.0.1
After that if you can ping it, then likely it's booted. See if you can ssh in.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: Use netconsole to troubleshoot uBoot without a serial cable February 22, 2014 11:05AM |
Registered: 11 years ago Posts: 17 |
|
uBoot problem March 14, 2014 05:07PM |
Registered: 10 years ago Posts: 7 |
Hi, i follow the tutorial to revert Pogoplug E02 to factory and then...
Edit /etc/init.d/rcS script, and remove the comment(#) in front of the line reading /etc/init.d/hbmgr.sh start
The device response the PING, but putty say :Network error: conection refused, maybe i do something wrong with te vi command
Thanks for your help
EDITED
Using this post: http://forum.doozan.com/read.php?2,12096
i have DEBIAN runing, reinstaled uBOOT with this commands:
cd /tmp
wget http://jeff.doozan.com/debian/uboot/ins ... ot_mtd0.sh
chmod +x install_uboot_mtd0.sh
./install_uboot_mtd0.sh
Same error whit putty: NETWORK ERROR: CONECTION REFUSED
and using NETCONSOLE, here is the LOG :
THANK YOU
Edited 1 time(s). Last edit at 03/14/2014 05:08PM by Ornella.
Edit /etc/init.d/rcS script, and remove the comment(#) in front of the line reading /etc/init.d/hbmgr.sh start
The device response the PING, but putty say :Network error: conection refused, maybe i do something wrong with te vi command
Thanks for your help
EDITED
Using this post: http://forum.doozan.com/read.php?2,12096
i have DEBIAN runing, reinstaled uBOOT with this commands:
cd /tmp
wget http://jeff.doozan.com/debian/uboot/ins ... ot_mtd0.sh
chmod +x install_uboot_mtd0.sh
./install_uboot_mtd0.sh
Same error whit putty: NETWORK ERROR: CONECTION REFUSED
and using NETCONSOLE, here is the LOG :
U-Boot 2011.12 (Feb 20 2012 - 21:21:59)
Pogoplug E02
arm-none-linux-gnueabi-gcc (Sourcery G++ Lite 2009q3-67) 4.4.1
GNU ld (Sourcery G++ Lite 2009q3-67) 2.19.51.20090709
Hit any key to stop autoboot: 0
(Re)start USB...
USB: Register 10011 NbrPorts 1
USB EHCI 1.00
scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices... 0 Storage Device(s) found
** Block device usb 0 not supported
** Invalid boot device **
Creating 1 MTD partitions on "nand0":
0x000002500000-0x000008000000 : "mtd=3"
UBI: attaching mtd1 to ubi0
UBI: physical eraseblock size: 131072 bytes (128 KiB)
UBI: logical eraseblock size: 129024 bytes
UBI: smallest flash I/O unit: 2048
UBI: sub-page size: 512
UBI: VID header offset: 512 (aligned 512)
UBI: data offset: 2048
UBI: attached mtd1 to ubi0
UBI: MTD device name: "mtd=3"
UBI: MTD device size: 91 MiB
UBI: number of good PEBs: 726
UBI: number of bad PEBs: 2
UBI: max. allowed volumes: 128
UBI: wear-leveling threshold: 4096
UBI: number of internal volumes: 1
UBI: number of user volumes: 0
UBI: available PEBs: 715
UBI: total number of reserved PEBs: 11
UBI: number of PEBs reserved for bad PEB handling: 7
UBI: max/mean erase counter: 1/1
UBIFS error (pid 0): ubifs_get_sb: cannot open "ubi:rootfs", error -19
Error reading superblock on volume 'ubi:rootfs'!
** Block device usb 0 not supported
** Block device usb 1 not supported
** Block device usb 2 not supported
** Block device usb 3 not supported
** Block device usb 0 not supported
** Block device usb 0 not supported
Wrong Image Format for bootm command
ERROR: can't get kernel image!
stopping USB..
### JFFS2 loading 'uboot-original-mtd0.kwb' to 0x800000
Scanning JFFS2 FS: ......... done.
### JFFS2 load complete: 524288 bytes loaded to 0x800000
## Starting application at 0x00800200 ...
THANK YOU
Edited 1 time(s). Last edit at 03/14/2014 05:08PM by Ornella.
|
Re: uBoot problem March 14, 2014 05:32PM |
Admin Registered: 13 years ago Posts: 18,545 |
You have not completed booting yet:
Make sure the Debian rootfs USB is plugged in!
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices... 0 Storage Device(s) found
** Block device usb 0 not supported
Make sure the Debian rootfs USB is plugged in!
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: uBoot problem March 14, 2014 05:49PM |
Registered: 10 years ago Posts: 7 |
http://forum.doozan.com/read.php?4,3896 i found this POST, and now have a RESCUE SYSTEM INSTALED..
and now?
-bodhi THANKS FOR YOUR HELP, sorry for MY english
and now?
-bodhi THANKS FOR YOUR HELP, sorry for MY english
|
Re: uBoot problem March 14, 2014 08:38PM |
Admin Registered: 13 years ago Posts: 18,545 |
Ornella,
No problem :)
Just plug in the USB rootfs that you've created above
and boot with it.
Rescue system is installed in NAND, for rescue purpose. But it's not a full Debian system in the sense that you want to use it day-to-day. The rootfs that you have created in the USB drive is a full system Debian that allows you to install new packages and use it for any thing you want.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
No problem :)
Just plug in the USB rootfs that you've created above
Using this post: http://forum.doozan.com/read.php?2,12096 i have DEBIAN runing, reinstaled uBOOT with this commands:
and boot with it.
Rescue system is installed in NAND, for rescue purpose. But it's not a full Debian system in the sense that you want to use it day-to-day. The rootfs that you have created in the USB drive is a full system Debian that allows you to install new packages and use it for any thing you want.
-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)
|
Re: uBoot problem March 14, 2014 11:00PM |
Registered: 10 years ago Posts: 7 |
