Changing boot sequence for making stubborn SSD (bad firmware?) boot.

Hi Bodhi!
It's me again, with that annoying Transcend SSD.
The problem is, that this SSD doesn't work reliably after the IDE reset. Sometimes it's working and some times it's timing out. This is not solved by increasing the time span between the ide resets, it seems just completely random.
So what I need is a boot sequence that repeats the ide reset until the SSD is in working condition.

As far as I understood, that's what

bootcmd_uenv=run uenv_load; if test $uenv_loaded -eq 1; then run uenv_import; fi; sleep 5

is doing already.

But of course there are consecutive ide resets in the general boot sequence, like for example in:

scan_disk=echo running scan_disk ...; scan_done=0; setenv scan_usb "usb start"; setenv scan_ide "ide reset";  setenv scan_mmc "mmc 5.00rescan"; for dev in $devices; do if test $scan_done -eq 0; then echo Scan device $dev; run scan_$dev; for disknum in $disks; do if test $scan_done -eq 0; then echo device $dev $disknum:1; if load $dev $disknum:1 $load_uimage_addr /boot/uImage 1; then scan_done=1; echo Found bootable drive on $dev $disknum; setenv device $disknum:1; setenv bootdev $dev; fi; fi; done; fi; done

5.00


And as the result of those additional ide resets is random for this SSD again, it will not work for booting from this SSD.

So I wonder, what the additional ide resets are actually needed for, while scanning for the boot drive. Shouldn't all connected devices be already available from the initial

 bootcmd_uenv 

on?


I now have the time for fiddling a bit again and I wanted to get this to work.

Thank you! :)



Edited 6 time(s). Last edit at 04/16/2018 01:35PM by ElMariachi.

ElMariachi,

Take out bootcmd_uenv. It is only used when you need uEnv.txt. scandisk is what require to find the partition that contains the kernel files.

Normally it does not cause problem to initialize the HDD again. However, in some rare case, such as this SSD, it might help not to do that twice (because of the spinning up time).

When you have serial console, or have set up netconsole, uEnv.txt is no longer needed.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hello bodhi!
I think there's a missunderstanding, or I didn't describe it precisely enough. This SSD has some very special behaviour.
The problem is, that the SSD being correctly recognized by UBoot is just completely random, even at the first scan. It might be the first scan that is working, or it might any subsequent scan. But also after it worked once, then the next try is random again. So what this thing actually needs, is a loop that would repeat scannig so often, until the SSD has been successfully recognized and then there must be no further scanning done.

That's why I mentioned the UEnv loop, because I believe it could be done by searching for a file (like uenv.txt) and repeat the ide reset until the file has been found - and then it must boot directly without scanning again, or otherwise it wouldn't work again.



Edited 4 time(s). Last edit at 05/01/2018 04:23PM by ElMariachi.

ElMariachi,

I see. Seems you need a slightly different scan logic to do that. Let me make another env to try.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I'm not too lazy I hope :) I already ha some ideas:
I made a modified uEnv.txt and simply set bootdev and dev manually in this, in an own variable:

scan_fake=setenv device 0:1; setenv bootdev ide

(I'm not yet sure about the correct parameters for device)


then I changed the normal bootcmd from:

bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; if run bootcmd_exec; then; else run bootcmd_pogo;  fi; reset

to

bootcmd=run bootcmd_uenv; run scan_fake; run set_bootargs; if run bootcmd_exec; then; else run bootcmd_pogo;  fi; reset

So I think we just need to change bootcmd_uenv so, that it repeats until it succesfully loaded an uEnv.tx. for making sure that the SSD is currently working.
Only problem with this method is, that of course the device wouldn't boot at all anymore, unless the SSD with uEnv.txt on it is inserted and it wouldn't work with USB drives at all anymore. So it could be a good idea to limit the maximum times the loop will run to about 10 tries.
But probably you got an overall better solution..



Edited 6 time(s). Last edit at 05/01/2018 05:59PM by ElMariachi.

ElMariachi,

You have the right idea!

However, no need to change bootcmd or add new scan_fake.

I would just change the scan_disk and the bootcmd_uenv in u-boot.

1. Looping in bootcmd_uenv
2. Scan disk without restarting disk drive (we know it'salready done in bootcmd_uenv).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

ElMariachi,


Try these envs.

Set the max_scan_uenv to some large number such as 100 and see if it will be enough (we don't want an infinite loop during booting).

setenv max_scan_uenv 100
setenv bootcmd_uenv 'setenv index 0; while itest $index < $max_scan_uenv; do run uenv_load; if test $uenv_loaded -eq 1; then run uenv_import; exit; fi; setexpr index $index + 1; done'
setenv scan_disk 'echo running scan_disk ...; scan_done=0; setenv scan_usb "usb start";  setenv scan_ide "ide reset";  setenv scan_mmc "mmc rescan"; for dev in $devices; do if test $scan_done -eq 0; then echo Scan device $dev; for disknum in $disks; do if test $scan_done -eq 0; then echo device $dev $disknum:1; if load $dev $disknum:1 $load_uimage_addr /boot/uImage 1; then scan_done=1; echo Found bootable drive on $dev $disknum; setenv device $disknum:1; setenv bootdev $dev; fi; fi; done; fi; done'

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi!

I have been working on this for some days now. The loop is working fine, but it would never boot. I tracked the problem down to the point, that the load command from UBoot (I also tried ext4load) would never work. Issuing the load command would always (100%) lead to "IDE read: device 0 not ready", regardless the partition was found before.

I think there's nothing we could do about this anymore, as it seems to be a compatibility issue between UBoot and this drive, and I will have to buy another drive.
What do you think?

ElMariachi,

> always (100%) lead to "IDE read: device 0 not
> ready", regardless the partition was found
> before.
>
> I think there's nothing we could do about this
> anymore, as it seems to be a compatibility issue
> between UBoot and this drive, and I will have to
> buy another drive.
> What do you think?

If it is that unreliable, then I would say yes. Use another drive to ensure booting consistently.

You can still use this drive for rootfs, but if you want to have only 1 drive plugged in then I'd say why spending too much time to get it working with u-boot. There are many other SSD drives that do not work with the current u-boot, not just this one.

I woud not spend too much time to get to the root cause of where u-boot chokes on a certain model of SDD drives, because I know it will work when I switch SSD.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Of course. It's just, that I am generally very short on money because of my illness, and if I was going to buy any cheap SSD now, I don't have any guarantee, that it will be U-Boot compatible in advance. So it's a gamble with limited resources. That's why I was willing to spend more time on that matter.



Edited 1 time(s). Last edit at 05/08/2018 07:48AM by ElMariachi.

ElMariachi,

Basically, It is possible to make the logic simple enough so that it works with this SSD, and use it as boot drive.

When you want to be able to boot with USB drive when you remove the SSD and plug in the USB, then you can to redefine some envs inside uEnv.txt on that USB /boot.

So let me see I understood. Will this load the uEnv.txt from the SSD successfully?

setenv devices 'ide usb'
setenv max_scan_uenv 100
setenv bootcmd_uenv 'setenv index 0; while itest $index < $max_scan_uenv; do run uenv_load; if test $uenv_loaded -eq 1; then run uenv_import; exit; fi; setexpr index $index + 1; done'

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

No, it won't. That's what I meant before. The load command from UBoot will always make the SSD irresponsive. Sorry, I didn't want to stress you further with this, I just tried to explain, why I was willing to put more time into this, then most people might think would be reasonable. I think there's no way for making this bad SSD boot directly. Maybe it could work using Grub on the internal flash or by loading Grub from an old USB pen drive or such, but I don't want to take more of your time for this either.
Thank you! :)



Edited 1 time(s). Last edit at 05/08/2018 02:58PM by ElMariachi.

ElMariachi,

> No, it won't. That's what I meant before. The load
> command from UBoot will always make the SSD
> irresponsive.

I see.

> Sorry, I didn't want to stress you
> further with this, I just tried to explain, why I
> was willing to put more time into this, then most
> people might think would be reasonable. I think
> there's no way for making this bad SSD boot
> directly.

How about booting from a USB, and use this SSD as rootfs? we will put the kernel boot files on USB, and keep the SSD as rootfs (once the system starts, the USB is no longer relevant). This set up has 100% chance to work properly, and there no need to replace the SSD :)


Format the USB thumb drive as EXT3. Single partition.

su -
cp -aR /media/SSD/boot /media/USB/
sync

Plug both SSD and USB in. Power up, interrupt serial console and set the env. All you need to change in u-boot env is the devices.

setenv devices usb
boot

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 05/08/2018 04:17PM by bodhi.

Hi bodhi!
I have been workign at this for some time, and have gone through more and more problems since..
I tried another old SSD I had laying there, but it seems that this SSD is dead, as it cannot write anymore at all, even after secure erasing..
I tried an old spare USB flash drive and it seems like UBoot doesn't like that drive either :(

Than one week later something strange happened: I wanted to continue the experiments, but I cannot SSH into the rescue system anymore. For some incomprehensible reason it's claiming the password was wrong. Of course the internal flash is always mounted as read only and I haven't changed the password for ages.. maybe it's a problem with the internal flash.
Whatever the reason is, of course I need the rescue system for going on with this, and so I used my every-day-USB-flash-drive for booting a rootfs from this. Then I wanted to mount and chroot the rescue system for setting a new root password on the internal NAND, but I cannot mount this.

root@debian:~# mount -t ext4 /dev/mtdblock3 /tmp/nand/
mount: wrong fs type, bad option, bad superblock on /dev/mtdblock3,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

I also tried ext2 and ext3, because I wasn't sure about the filesystem on the rescue NAND, anymore, but neither worked.
What do I need to do different?



Edited 4 time(s). Last edit at 05/25/2018 08:29AM by ElMariachi.

ElMariachi,

Please remind me which rescue system you have installed :) If it was davygravy V2 version?

In any case, most recue system in NAND use UBIFS. So look to see if the Debian rootfs has brought up the mtd partition that houses the rescue system.

dmesg | grep -i ubi

And if it has done that, mount it:

mount -t ubifs /dev/mtdblock3 /tmp/nand/

If it has not, then you need to run ubiattach to bring it up before mounting:

ubiattach /dev/ubi_ctrl -m 3

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I have Jeff's old rescue system V2 on there. I could not find a working link for davygrav's and also instructions for 2.8 where overall confusing. I think the newer ones aren't ready for use, yet, are they?

OK, I now tried, but:
1. dmesg | grep -i ubi doesn't find anything.
2. ubiattach wasn't found, so I installed mtd-utils
3. After installing mtd-utils I issued ubiattach /dev/ubi_ctrl -m 3

root@debian:~# ubiattach /dev/ubi_ctrl -m 3
UBI device number 0, total 1728 LEBs (222953472 bytes, 212.6 MiB), available 1684 LEBs (217276416 bytes, 207.2 MiB), LEB size 129024 bytes (126.0 KiB)
root@debian:~# dmesg|grep -i mtd
[    7.677669] 4 ofpart partitions found on MTD device orion_nand
[    7.683557] Creating 4 MTD partitions on "orion_nand":
[  433.140918] UBIFS error (pid: 1588): cannot open "/dev/mtdblock3", error -22
[  896.387856] ubi0: attaching mtd3
[  896.738409] ubi0: attached mtd3 (name "root", size 216 MiB)

4. Then I tried mounting again:

root@debian:~# mount -t ubifs /dev/mtdblock3 /tmp/nand 
mount: wrong fs type, bad option, bad superblock on /dev/mtdblock3,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

But the partition has to be there, because the rescue system is still booting up and the SSH server is responding (and ubiattach finds the name)...
Maybe the NAND of this device is faulty? And maybe that's the reason for all those problems? I did check the NAND before flashing UBoot and rescue system, though.. there where no errors reported.



Edited 7 time(s). Last edit at 05/25/2018 08:15PM by ElMariachi.

ElMariachi,

> But the partition has to be there, because the
> rescue system is still booting up and the SSH
> server is responding (and ubiattach finds the
> name)...

My bad (a senior moment) :) you need to mount the volume name. So the command needs to be:

mount -t ubifs ubi0:rescue  /tmp/nand

Note: rescue is the volume name in partition 3. So list them to see the exact name as it appear in this system:

cat /proc/mtd

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi Bodhi!
This problem is solved for now, because in between I have tried reflashing recovery v2, which forced me to flash UBoot, which again bricked my device. I now have recovered the device using serial console, enabled net console and booted from an USB drive. Next task will be installing the rescue system again.

Do you have a working link for davygravy's 2.8.2?

:)

Thanks,
ElMariachi



Edited 1 time(s). Last edit at 05/25/2018 10:22PM by ElMariachi.

ElMariachi,

> Do you have a working link for davygravy's 2.8.2?
>

I don't use rescue system.

I think you should stay with Jeff's V2. You can run that with my latest u-boot.

As I recalled , davygravy rescue 2.8.x had some problem (being out of date), and he has been inactive, so no supports if you run into the same problem others have. I have one plug with Jeff's V2 rescue system (for experiment) and it still boot OK when I checked.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

OK, then I'll use the classic one.
I was interested in newer version because of some tools that lack useful features for recovery and repair operations, like the too old wget and hdparm.

One problem is, that installing V2 from a current debian stops with

##
## uBoot is not up-to-date
## Please install the newest uBoot and then re-run this installer.
##

I had to change the if statement for making it run through. As that is why I started messing around with UBoot before, maybe it would be a good idea to provide a slightly modified script?



Edited 2 time(s). Last edit at 05/25/2018 10:41PM by ElMariachi.

ElMariachi,

> I had to change the if statement for making it run
> through. As that is why I started messing around
> with UBoot before, maybe it would be a good idea
> to provide a slightly modified script?

Good to just make the script assumes that u-boot is good to go when it sees my released u-boot already installed.

As I've mentioned, rescue systems v2 and v2.x are not actively supported so you're pretty much on your own :) After you've changed the script and it works fine for you, please post that in the rescue system V2 thread to help latecomers.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

So it seems like I misunderstood:
I though Jeff's V2 was supported. If not - which one is?
V3 seems dead and V4 is beta.



Edited 3 time(s). Last edit at 05/26/2018 05:38AM by ElMariachi.

ElMariachi,

Jeff and davygravy has not been active in the forum for a long time. Their rescue systems work well, so you can use either one. However, these rescue system are basically frozen. If users need help booting them, I will help. But that's all the supports I can provide. Ie. They will not be updated.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Yes, ok. I'm ok with that :)