unusually long boot process

Hello Bodhi,

I'm using a few Samsung GoFlexNet for years using the jessie based rootfs.

Devices are booting by loading uImage, uInitrd and the DTB-file from an usb flashdisk
[ U-Boot 2017.07-tld-1 (Sep 05 2017 - 00:17:19 -0700) ] and switches to HDD (LVM
on LUKS).

At the beginning of this year I decided to switch the GoFlexNet to the buster based rootfs.

The previous (jessie-based) constellation worked like a charm

Now there are problems now after uBoot has smoothly loaded linux image, ramdisk
and dtb-file and control goes to the linux kernel.

dmesg shows endless sequences of these:

...
[ 1349.955698]  sdb: sdb1
[ 1350.683343] usb 1-1: reset high-speed USB device number 2 using orion-ehci
[ 1350.886601] sd 2:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK cmd_age=0s
[ 1350.896637] sd 2:0:0:0: [sdb] tag#0 CDB: Read(10) 28 00 00 e8 cc 50 00 00 20 00
[ 1350.904742] blk_update_request: I/O error, dev sdb, sector 15256656 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0
[ 1350.918145] sd 2:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=0s
[ 1350.928178] sd 2:0:0:0: [sdb] tag#0 Sense Key : Unit Attention [current] 
[ 1350.935775] sd 2:0:0:0: [sdb] tag#0 Add. Sense: Not ready to ready change, medium may have changed
[ 1350.945540] sd 2:0:0:0: [sdb] tag#0 CDB: Read(10) 28 00 00 e8 cc 78 00 00 88 00
[ 1350.953640] blk_update_request: I/O error, dev sdb, sector 15256696 op 0x0:(READ) flags 0x80700 phys_seg 9 prio class 0
[ 1350.965644] sd 2:0:0:0: [sdb] tag#0 device offline or changed
[ 1350.972160] blk_update_request: I/O error, dev sdb, sector 15256656 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[ 1350.983840] Buffer I/O error on dev sdb, logical block 1907082, async page read
...

but finally the system comes up and work as expected. This takes about 12 to
15 minutes.

The old jessie based constellation was up in about 1 to 2 minutes.

There hasn't been changed anything regarding the hardware. Especially the
USB flashdisk used is the same.

Do you have have any hint solving this ?

Thanks in advance,

Joe

PS.:

root@XX-1:/tmp# uname -a
Linux xx-1 5.7.1-kirkwood-tld-1 #1.0 PREEMPT Sun Jun 7 20:57:38 PDT 2020 armv5tel GNU/Linux

root@xx-1:/tmp# cat /proc/cmdline 
console=ttyS0,115200 rw rootwait loglevel=8 ipv6.disable=1 zswap.enabled=1 zswap.compressor=lz4 rootfstype=ext4 root=/dev/mapper/vg0-root

root@xx-1:/tmp# lsblk
NAME           MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda              8:0    0 111.8G  0 disk  
`-sda1           8:1    0 111.8G  0 part  
  `-cr_sda1    253:0    0 111.8G  0 crypt 
    |-vg0-root 253:1    0     6G  0 lvm   /
    |-vg0-var  253:2    0     6G  0 lvm   /var
    |-vg0-home 253:3    0     4G  0 lvm   /home
    |-vg0-tmp  253:4    0     2G  0 lvm   /tmp
    `-vg0-swap 253:5    0     2G  0 lvm   [SWAP]
sdb              8:16   1   7.3G  0 disk  
`-sdb1           8:17   1   7.3G  0 part  
mtdblock0       31:0    0     1M  0 disk  
mtdblock1       31:1    0     4M  0 disk  
mtdblock2       31:2    0    32M  0 disk  
mtdblock3       31:3    0   216M  0 disk  

jfp,


This is the second time I see this type of error reported lately (the other report was for a different box). So I suspect it must be something wrong in Ext4 driver or zswap that you guys hit.

Data point: my Goflex Net also runs on USB rootfs, and I don't see the problem since I upgraded to kernel 5.7.1.

Quote

192.168.0.234
Seagate GoFlex Net
Linux version 5.7.1-kirkwood-tld-1 (root@tldDebian) (gcc version 8.3.0 (Debian 8.3.0-6), GNU ld (GNU Binutils for Debian) 2.31.1) #1.0 PREEMPT Sun Jun 7 20:57:38 PDT 2020
Debian 10.3
console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=orion_nand:1M(u-boot),4M(uImage),32M(rootfs),-(data) ipv6.disable=1
uboot_version=U-Boot 2017.07-tld-1 (Sep 05 2017 - 00:17:19 -0700)
Netconsole is enable
--- System Stats:
/dev/sda: ST9500325AS: 36°C SMART check: PASSED
/dev/sdb: ST1000LM024 HN-M101MBB: 34°C SMART check: PASSED
/dev/sdc: SanDisk: drive supported, but it doesn't have a temperature sensor.
SMART check:
Sun 21 Jun 2020 02:07:31 PM PDT up 1 week, 4 days, 10 hours, 42 minutes

Try to run without zswap:

Quote

zswap.enabled=1

Remove that bootarg and reboot.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 06/21/2020 04:32PM by bodhi.

Thank you for responding quickly.

As recommended I tried it with "zswap.enabled" as well as with additionallly "zswap.compressor" removed.

Same behaviour, same dmesg messages :-(

Further maybe helpful hints are welcome.

With best regards,
Joe

jfp,

Use a different USB drive, create the new rootfs using the buster rootfs Debian-5.2.9-kirkwood-tld-1-rootfs-bodhi.tar.bz2. And boot with it, see if the error is gone. And make sure you format it as EXT3.

If the error is no longer shown, then backup your current rootfs (if you already installed lots of packages). Reformat it as EXT3 and create the new rootfs on it (or restore the backup).

Note that the bootarg "rootfstype=ext4" is no longer relevant. The system will detect the file system type automatically. So remove it.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Thank you,
for your hints.

TL;DR : vendor has obviously changed one or some component(s) of the USB flashdisk brand/model I used for years, which caused the initially reported obscure behaviour

Using the udevadm command I could recognize that the devices I bought at different occasions have amongst others different model IDs. This is explains the different behaviour despite of using seemigly identical devices.

When the "right" flashdisk is used the box starts smoothly despite of the use of ext2/3/4. Also the zswap functionality is working as expected.

In conclusion, uBoot seems to have the more robust concept for accessing these devices, because the boot files of all USB sticks I tried were loaded quickly and without error messages.

Thank you for your support in this case, but also for your contributions to this community and the ongoing intensive efforts to maintain this forum.

Best Regards,
Joe

Joe,

You're welcome!

> Using the udevadm command I could recognize
> that the devices I bought at different occasions
> have amongst others different model IDs. This is
> explains the different behaviour despite of using
> seemigly identical devices.
>
> When the "right" flashdisk is used the box starts
> smoothly despite of the use of ext2/3/4. Also the
> zswap functionality is working as expected.

Cool!

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)