Debian on Dell Kace M300

daviddyer Wrote:
-------------------------------------------------------
> I guess we can skip this ....I don't think anyone
> will use that box with the original 16GB "sd". So
> he / she will need to remove the old drive and he
> / she needs to open the case anyway.. When he /
> she opens the case, then booting with uart or
> changing shadowed password can be easy.

That's really not the point here. The simplest installation is where you can connect serial console from Windows/Linux/Mac and do everything at stock OS shell (like with other Kirkwood boxes).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 06/01/2022 03:29AM by bodhi.

> not a single soul knows, it was hard coded onto a
> USB key that is lost to the ages.

When you can log innto stock OS, you are on the rootfs in the internal SSD.

> as defined in the old guide. but yet the pogo E02
> config I have lists /dev/mtd0 as the defined MTD
> device.

That's really apple and orange. Pogo E02 is a completely different beast (each Kirkwood plug is different from each other, unless it is from the same manufacturer). The Pogo E02 has NAND flash and the MTD definition is very different from this M300 MTDs.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

yeah I get that, the problem is that he has the new uboot, but old envs.

his envs from serial are as follows:

M300> printenv
baudrate=115200
bootargs=console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $(mtdparts) earlyprintk=serial
bootargs_console=console=ttyS0,115200 mtdparts=spi_flash:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env) root=LABEL=root rw
bootcmd=run kernel_config; run set_bootargs; run bootcmd_exec; run set_bootargs_stock; run bootcmd_stock
bootcmd_exec=echo Booting Debian …; usb start; run load_uimage; run load_initrd; bootm $(load_uimage_addr) $(load_initrd_addr)
bootcmd_stock=echo Booting stock ...; run bootcmd_ide
bootdelay=5
ethaddr=18:03:73:41:dc:4c
fdtcontroladdr=7fb9d5c0
kernel_config=setenv mainlineLinux yes; setenv mtdparts mtdparts=spi0.0:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env); setenv load_uimage_addr 0x800000; setenv load_initrd_addr 0x02100000
load_initrd=ext2load usb 0:1 $(load_initrd_addr) /boot/uInitrd
load_initrd_addr=0x02100000
load_uimage=ext2load usb 0:1 $(load_uimage_addr) /boot/uImage
load_uimage_addr=0x800000
mainlineLinux=no
mtdparts=mtdparts=spi0.0:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env)
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $(mtdparts) earlyprintk=serial
set_bootargs_stock=setenv mainlineLinux no; setenv bootargs_console console=ttyS0,115200 mtdparts=spi_flash:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env) root=LABEL=root rw
stderr=serial
stdin=serial
stdout=serial
ver=U-Boot 2022.04-tld-1 (Mar 20 2022 - 18:15:15 -0700)\
Dell KACE M300

Environment size: 1426/4092 bytes
M300>

he can change the envs to get it to boot debian properly, but, saveenv from the prompt won't stick. at all.
gonna look through but if anyone has an old version of the stock uboot or a dd image of the stock SSD we can shove on another drive, that'd help immensely.

> yeah I get that, the problem is that he has the
> new uboot, but old envs.

To recover from this problem, use uEnv.txt capability.

The file uboot.2022.04-tld-1.m300.environment is the envs listing that I used in creating the uboot.2022.04-tld-1.m300.environment.4K.img. This can be used as uEnv.txt.

Moun your USB rootfs on another Linux box. Copy the uboot.2022.04-tld-1.m300.environment to the rootfs. Assuming it is mounted at /media/sdb1.

cd /media/sdb1/boot
cp -a uboot.2022.04-tld-1.m300.environment uEnv.txt

Add these 3 lines to the end of uEnv.txt (replace the ethaddr, ipaddr, and serverip values to the appropriate ones for your box). This is to ensure you can ssh into the box after it boots. But if you have serial console then adjust them later

ethaddr=18:03:73:xx:xx:xx
ipaddr=192.168.0.249
serverip=192.168.0.220

That's it. Save the file and sync

sync

And unmount.

Take it to the M300 and boot.

After log in, flash the default envs image following the instruction in new u-boot installation.

flash_unlock /dev/mtd1
flashcp -v uboot.2022.04-tld-1.m300.environment.4K.img  /dev/mtd1

Expected output 

Erasing blocks: 1/1 (100%)
Writing data: 4k/4k (100%)
Verifying data: 4k/4k (100%)

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> daviddyer Wrote:
> -------------------------------------------------------
> > I guess we can skip this ....I don't think
> anyone
> > will use that box with the original 16GB "sd".
> So
> > he / she will need to remove the old drive and
> he
> > / she needs to open the case anyway.. When he /
> > she opens the case, then booting with uart or
> > changing shadowed password can be easy.
>
> That's really not the point here. The simplest
> installation is where you can connect serial
> console from Windows/Linux/Mac and do everything
> at stock OS shell (like with other Kirkwood
> boxes).

OK. So I guess nobody knows the stock OS login credential :) Then the installation is still needed to be a 2-step process. Or with kwboot (but this introduces another hurdle for newbees).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> After log in, flash the default envs image
> following the instruction in new u-boot
> installation.
>
>

> flash_unlock /dev/mtd1
> flashcp -v
> uboot.2022.04-tld-1.m300.environment.4K.img 
> /dev/mtd1
> 
> Expected output 
> 
> Erasing blocks: 1/1 (100%)
> Writing data: 4k/4k (100%)
> Verifying data: 4k/4k (100%)
>

and we get an unexpected output with fw_env.config set properly.
md5sum of the downloaded tarball is correct.

root@debian:~/uboot# flash_unlock /dev/mtd1
flash_unlock: error!: could not unlock device: /dev/mtd1

              error 22 (Invalid argument)
root@debian:~/uboot# flashcp -v uboot.2022.04-tld-1.m300.environment.4K.img  /dev/mtd1
Erasing blocks: 1/1 (100%)
Writing data: 4k/4k (100%)
Verifying data: 4k/4k (100%)File does not seem to match flash data. First mismatch at 0x00000000-0x00001000
root@debian:~/uboot#

and we still cant manipulate it in Linux with the current env addressing.

root@debian:~/uboot# fw_printenv
baudrate=115200
bootcmd=run kernel_config; run set_bootargs; run bootcmd_exec; run set_bootargs_stock; run bootcmd_stock
bootcmd_exec=echo Booting Debian …; usb start; run load_uimage; run load_initrd; bootm $(load_uimage_addr) $(load_initrd_addr)
bootcmd_stock=echo Booting stock ...; run bootcmd_ide
bootdelay=5
ethaddr=18:03:73:41:dc:4c
kernel_config=setenv mainlineLinux yes; setenv mtdparts mtdparts=spi0.0:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env); setenv load_uimage_addr 0x800000; setenv load_initrd_addr 0x02100000
load_initrd=ext2load usb 0:1 $(load_initrd_addr) /boot/uInitrd
load_uimage=ext2load usb 0:1 $(load_uimage_addr) /boot/uImage
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $(mtdparts) earlyprintk=serial
set_bootargs_stock=setenv mainlineLinux no; setenv bootargs_console console=ttyS0,115200 mtdparts=spi_flash:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env) root=LABEL=root rw
root@debian:~/uboot# fw_setenv bootdelay 15
root@debian:~/uboot# fw_printenv
baudrate=115200
bootcmd=run kernel_config; run set_bootargs; run bootcmd_exec; run set_bootargs_stock; run bootcmd_stock
bootcmd_exec=echo Booting Debian …; usb start; run load_uimage; run load_initrd; bootm $(load_uimage_addr) $(load_initrd_addr)
bootcmd_stock=echo Booting stock ...; run bootcmd_ide
bootdelay=5
ethaddr=18:03:73:41:dc:4c
kernel_config=setenv mainlineLinux yes; setenv mtdparts mtdparts=spi0.0:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env); setenv load_uimage_addr 0x800000; setenv load_initrd_addr 0x02100000
load_initrd=ext2load usb 0:1 $(load_initrd_addr) /boot/uInitrd
load_uimage=ext2load usb 0:1 $(load_uimage_addr) /boot/uImage
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $(mtdparts) earlyprintk=serial
set_bootargs_stock=setenv mainlineLinux no; setenv bootargs_console console=ttyS0,115200 mtdparts=spi_flash:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env) root=LABEL=root rw
root@debian:~/uboot# fw_printenv bootdelay
bootdelay=5
root@debian:~/uboot#

root@debian:~/uboot# cat /etc/fw_env.config
/dev/mtd1 0x0 00001000 0x00001000
root@debian:~/uboot#

it's a very weird pickle we're in.
kernel used here is 5.13.6, happens on 5.17.4 as well, and the same behavior is experienced on both of my M300s which are booting and running just fine. one cannot change the uboot envs within debian. at all.
it also doesn't seem to be letting us flash envs at all. even tried doing it on one of my boxes for a sanity check and it didn't let me.

we are indeed using /boot/uEnv.txt aside and uboot doesn't touch it.
I've even tried re-flashing the uboot image or even erasing the serial flash and it persists.
I can erase the flash and then read it back with data still in it like nothing happened.
the flash chip is definitely not bad, there's no bad sectors if I do the entire dmesg | grep -i bad thing.

I'm just so many levels of confused.



Edited 1 time(s). Last edit at 06/01/2022 06:03PM by sudos.

Yes. I know what's going on there. It's all because the SPI flash was protected initially (Dell engineers are too clever by half).

Will get back to this thread and explain in more details.

In a nutshell, to start fresh we can always flash stock u-boot back and from there everything should be like we expect it to be.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> OK. So I guess nobody knows the stock OS login
> credential :) Then the installation is still
> needed to be a 2-step process. Or with kwboot (but
> this introduces another hurdle for newbees).

As I recall nobody ever found it. I know I looked and came up empty, so I quickly gave up and mounted the SSD on another machine and used chroot.

bodhi Wrote:
-------------------------------------------------------
> Yes. I know what's going on there. It's all
> because the SPI flash was protected initially
> (Dell engineers are too clever by half).
>
> Will get back to this thread and explain in more
> details.
>
> In a nutshell, to start fresh we can always flash
> stock u-boot back and from there everything should
> be like we expect it to be.

that's what we're thinking, but that doesn't explain why this also happens on two other working m300s with the new 2022 uboot, and proper envs, with what should be a uboot that supports the flash chip and shouldn't lock it by default, and a kernel that keeps it unlocked as well. what you're pretty much telling me is, the uboot env parameters for all M300s currently flashed are not able to have their envs edited. Can you confirm on your end that you can edit your uboot envs from within the bullseye rootfs?

instead what I'm thinking is happening here is there's a difference with how kernel 5.x and Debian Buster/Bullseye are trying to talk to this specific serial mtd device... there were indeed changes to the mtd stuff with buster that broke a lot of things if memory serves.
but, it could be as you say, that the flash is possibly locked at boot time before even the 2022 uboot is able to be executed?

for now, I'll tell my friend to shut this down and put it to the side, a copy of the stock uboot would definitely help heaps towards normalcy. I just sftp'd over an image of my second M300's stock SSD (thankfully didn't delete it like I thought I did) and I dd'd it to his and we're gonna work from there once we have a uboot image.


renojim Wrote:
-------------------------------------------------------
> bodhi Wrote:
> -------------------------------------------------------
> > OK. So I guess nobody knows the stock OS login
> > credential :) Then the installation is still
> > needed to be a 2-step process. Or with kwboot
> (but
> > this introduces another hurdle for newbees).
>
> As I recall nobody ever found it. I know I looked
> and came up empty, so I quickly gave up and
> mounted the SSD on another machine and used
> chroot.

it'd be easier to just get it overwith and have the end user pop the SSD onto another machine to change the shadow file, they have to in order to use the serial header and use another Linux machine anyway to make the rootfs reliably. whether this is a PC running bare metal, another kirkwood device or a VM, it's all the same within reason. blame Dell in the end for making this device not all sunshine and flowers to get going, but given it's a business/enterprise-ish grade device there's reasoning behind it aside from Dell being Dell in the end.

at least we can give them credit where credit is due, the M300 looks and feels like it can take a bullet and still keep running.

also no need to bother with kwboot, so long as you follow along and do the necessary steps to the letter, kwboot'ing should never be necessary now that we have a working uboot that can be flashed. the only two things that need to be specified in the new guide somehow is that the older version files on the stock OS's mtdutils will say 384K/0K and 4K/0K instead of 384K/384K and 4K/4K as expected, and a warning that the serial header speaks 3.3v TTL and as such you need a MAX232 adapter or Raspberry Pi (or beaglebone etc.) to interact with it at 115200/8n1, same with most other kirkwood devices, and to not hook up the 3.3v line as it's not necessary.

Attached is a serial pinout diagram I drew up in MSPaint to help with consolidating information down (to attach to the install guide), it's detailed enough to help identify which pins should go where when someone hooks serial to one of these.



Edited 1 time(s). Last edit at 06/02/2022 02:55AM by sudos.

sudos, I just checked on my M300 with U-Boot 2022.04-tld-1 (Mar 20 2022 - 18:15:15 -0700), Kernel 5.16.5, and Bullseye (11.2) and I can set environment variables and they stick after a power cycle.

> sudos, I just checked on my M300 with U-Boot
> 2022.04-tld-1 (Mar 20 2022 - 18:15:15 -0700),
> Kernel 5.16.5, and Bullseye (11.2) and I can set
> environment variables and they stick after a power
> cycle.

Thanks! that answered sudos question.

Here is what I think happened for the problem with changing envs in Debian, even with new u-boot installed.

This box SPI flash was protected in stock condition. And that bit is sticky so the first thing we need to do is booting with serial console. As instructed in my Installation procedure

https://forum.doozan.com/read.php?2,61344,90019#msg-90019

Quote

3.a. Execute the following command to unprotect the SPI flash:

protect off all

After you have done that, it remains unprotected forever. However, if you do a saveenv at stock u-boot prompt, the SPI flash is set to protected mode again.

Two possibilities why this problem occured for sudos's friends.

1. The new u-boot was installed, but the entire SPI flash has not been unprotected at stock u-boot prompt (yes, it is possible to do this). So now with the new u-boot, we can not unprotect it while at the prompt. Therefore it remains protected in Debian.

2. It was unprotected as Step 3a was executed properly. However, at some point, the saveenv command was "accidentally" executed. It is not a wrong thing to do, but there is side-effect with this stock u-boot. And then the new u-boot was installed. Now the flash remains protected in Debian, and you cannot anything to changes it.

What renojim did was following the correct steps: install new Debian rootfs, and set the envs in stock u-boot. Now the SPI flash is unprocted in Deban. Much later on, the new u-boot is installed and at the same time, the new envs image was flashed. And the end result is you can no longer protect the SPI flash. It is a good thing, what we want.

And l will explain it further in a different post why I have not added the ability to unlock/lock the SPI flash.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

...except on both of my kace boxes, I never sent the saveenv command, and yet I can't edit either of them. I know for a true absolute fact I didn't run saveenv at all as both of my envs on both local boxes were set from Linux only as per the instructions. I never did anything at the uboot prompt aside from "protect off all" and "boot" at the stock prompt. every other thing was done in Linux and I don't have means to do a saveenv command from inside Linux.

I just downgraded to kernel 5.15.5 as well to test if it's a kernel regression but that doesn't seem to be the case either.
it just boggles the mind as to what's happening here.

doing some more reading I did come across a stackoverflow post of someone saying they could manipulate the serial NOR flash (what we have here with the macronix part in the M300) but on a different dev board through doing sf probe 0x0; sf unlock; in the bootarg. is this functionality removed with the above? if not, that may be the saving grace here to try. I can see from the help printout on the 2022 uboot that the sf command is there, so logic says yes, but the functionality you speak of may be that which is removed.



Edited 4 time(s). Last edit at 06/02/2022 06:35PM by sudos.

> except on both of my kace boxes

Quite strange. renojim and I can set the envs with fw_setenv without problem. Let me gather and list all the necessary settings so you can compare with.

> doing some more reading I did come across a
> stackoverflow post of someone saying they could
> manipulate the serial NOR flash (what we have here
> with the macronix part in the M300) but on a
> different dev board through doing sf probe 0x0;
> sf unlock; in the bootarg. is this
> functionality removed with the above? if not, that
> may be the saving grace here to try.

That could work. sf command is still in the new u-boot. I'll give it a shot to see if it's true.

===

I have looked into a fail-safe way to unlock the SPI flash for good in Debian. But for some reason the Linux SPI driver did not behave correctly as I expected. So I ran out of time and shelved that. And have not done anything to investigate the u-boot side.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Here is the info on my 2nd M300 box.

[    0.000000][    T0] Linux version 5.17.1-kirkwood-tld-1 (root@tldDebianVM) (arm-linux-gnueabi-gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1.0 PREEMPT Fri Apr 1 16:17:12 PDT 2022

root@m300:~# cat /proc/cmdline

console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=spi0.0:0x7f000@0(uboot),0x1000@0x7f000(u-boot-env)

dmesg shows

[    4.018091][    T1] spi-nor spi0.0: mx25l4005a (512 Kbytes)
[    4.023766][    T1] 2 cmdlinepart partitions found on MTD device spi0.0
[    4.030415][    T1] Creating 2 MTD partitions on "spi0.0":
[    4.035944][    T1] 0x000000000000-0x00000007f000 : "uboot"
[    4.041941][    T1] 0x00000007f000-0x000000080000 : "u-boot-env"

root@m300:~# cat /proc/mtd

dev:    size   erasesize  name
mtd0: 0007f000 00001000 "uboot"
mtd1: 00001000 00001000 "u-boot-env"

root@m300:~# cat /etc/fw_env.config

# Configuration file for fw_(printenv/saveenv) utility.
# Up to two entries are valid, in this case the redundand
# environment sector is assumed present.
# Notice, that the "Number of sectors" is ignored on NOR.

# Kirkwood
# Dell KACE M300
# MTD device name       Device offset   Env. size       Flash sector size       Number of sectors
/dev/mtd1 0x0 00001000 0x00001000

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

We ended up fixing the problem... but it's a lot more in-depth than I was expecting to go.
the SPI flash chip was so write-protected even the section that holds the normally read/write area with the write protect bits was write protected. I'm posting the write-up here so to be able to reference in case someone else has this problem down the line. Even with the installation guide rewrite underway, someone somewhere is going to mess up in the exact same way and hopefully this helps towards a solution.

So here's what we had to do to fix it:
MX25L4006E is the serial flash chip we're working with, same basic functionality as an MX25L4005A as identified in uboot and elsewhere in Debian.

in order to tell if the chip is write protected, you need to attach serial and interrupt uboot.

at the prompt, type:

sspi 0:0.0@0 16 05FF
sspi 0:0.0@0 24 05FFFF

Expected output if your flash is write-protected:

M300> sspi 0:0.0@0 16 05FF
FF9C
M300> sspi 0:0.0@0 24 05FFFF
FF9C9C

If this is what you get back, your serial flash chip's Status Register Write Disable (SWRD) bit is set off and your chip is in hardware protection mode.

The good news: this is fixable!
the bad news: it requires you to completely shuck the motherboard from the tray, with serial attached, and run the board without heatsinking for a little while during the fix. this should be non-hazardous to the SoC while it is idle.

NOTE: the following may damage your board. I am not liable if you are oblivious and start prodding voltages where they do not belong. This fix requires a bit of knowledge and steady hand due to component sizes. keep safe, do this in a well-lit area and DOUBLECHECK YOUR CONNECTIONS. Use any and all tools at your disposal to make this as pain-free as possible.

Attached is page 5 of the Macronix MX25L4006E datasheet. keep this open in a PDF reader of your choosing. you will NEED it for pinouts for sanity. the pinout you want is for the 8-pin SOP package but the pinout is the exact same across both package types so it really does not matter. You can also grab a copy of the full datasheet from Octopart at this link.

I'm naming this "The M300 Paperclip Fix."
What we're going to do is briefly attach Pin 3 (#WP, Write Protection) to Pin 8 (VCC +3.3v) while running some commands through u-boot while the system is powered on and at the u-boot prompt. BE MINDFUL OF THE PIN 1 DOT ON YOUR SERIAL FLASH CHIP. if this is not enough of a tell, Pin 3 is immediately next to resistor R154 which measures 1K and latches to ground.

Your weapon of choice for this should be either a thin paperclip, a stripped-back twist-tie, or a piece of thin solid-core wire.

I suggest you solder one end of this to the VCC leg (pin 8) so you don't need to hold both ends to the chip and can attach and detatch quickly as needed. Be sure your solder joint is solid so it doesn't come loose. we don't want to keep VCC connected to pin 3 for too long because you're both putting +3.3v through the pin and also the 1K resistor to ground. the VCC connection on pin 8 provides enough power to defeat the pulldown, so more than 3.26mA. Mine measures 1.013K (with an abused 80s chonker of a desktop battery-powered multimeter) so 3.3 / 1013 = 3.26mA required to keep it at 3.3V, but the power dissipation through the resistor would be (3.3 V)^2 / (1.013 K) = 0.011W. the smallest SMT resistor we could find does 0.05W so this should be safe, but still be careful.

once more: DO NOT CONNECT VCC TO PIN 3 UNTIL YOU ARE READY TO PERFORM THE COMMANDS.

once you are at the u-boot prompt again, copy these two lines and be ready to paste them into your serial terminal of choice. in my example, I'm using PuTTY, so if I copy both lines it allows me to paste one and then does not paste a carriage return after the second, leaving it at the prompt. you need to press Enter once to send the second line.

sspi 0:0.0@0 8 06
sspi 0:0.0@0 16 010C

now, CAREFULLY connect pin 3 to pin 8, and paste the two lines in. Remember to press Enter on the second line so it sends.

Expected output:

M300>sspi 0:0.0@0 8 06
FF
M300> sspi 0:0.0@0 16 010C
300> onversion error on
FFFF

At this point, you can now disconnect the bodge connection to Pin 3 from pin 8 and move it out of harm's way.
Run the following command to check if the commands took hold.

M300>sspi 0:0.0@0 16 05FF

Expected output:

M300>sspi 0:0.0@0 16 05FF
FF0C

if for some reason you got:

M300>sspi 0:0.0@0 16 05FF
FF0E

then this is benign, don't worry about it.

If you did not get any of these two results, reset without power cycling, interrupt u-boot and try again from the top.

-----
If you were successful, power-cycle the board and once again interrupt u-boot to get a prompt.
from here, run these commands:

sspi 0:0.0@0 8 06
sspi 0:0.0@0 16 0100
sspi 0:0.0@0 8 04
sspi 0:0.0@0 16 05FF

Expected output:

M300> sspi 0:0.0@0 8 06
FF
M300> sspi 0:0.0@0 16 0100
FFFF
M300> sspi 0:0.0@0 8 04
FF
M300> sspi 0:0.0@0 16 05FF
FF00

once again, verify you get this when running the final command:

M300> sspi 0:0.0@0 16 05FF
FF00

If this is what you have, Congratulations, you have SUCCESSFULLY disabled the hardware AND software protection bits on your MX25L4006E.

To check, for sanity, that you can now write to u-boot, you can boot to Debian and try to manipulate your envs through the libubootenv tools.

root@yamada:~# fw_setenv bootdelay 15
root@yamada:~# fw_printenv bootdelay
bootdelay=15

GREAT SUCCESS! You can now gracefully shut down and undo any soldering changes you made and close your box back up. Your kirkwood SoC has been running without a heatsink this whole time and it is probably very hot to the touch so be careful. however, it shouldn't get too hot that you'll burn up the chip doing this as previously mentioned.

I thank my friend for helping me with this and his other friend who is an absolute rockstar wizard. He supposedly does firmware for a living and there's probably enough NDAs involved that I did not bother asking for whom. but, because the modern u-boot had the appropriate commands enabled, we were able to get this far. While it took some poking to get electron pixies flowing where they needed to, it resolved the problem without having to resort to more drastic measures (i.e. desolder the entire chip and re-flash.)

once again, I reiterate, I'm not liable for any damages you incur doing this. while this is somewhat a "how-to" it's more of a "how we fixed it doing hacky workarounds" and honestly the nuclear option of removing the whole serial chip from the board for a re-flash is a lot safer, so I recommend that instead if you're equipped to do such work.

Attached this post is the Macronix datasheet as previously mentioned and a picture of the flash chip showing exactly which two pins you need to worry about if this happens to you and you wish to follow along. but again, I recommend against doing this as it was very hacky and better solutions exist.

the TL;DR of the post: the serial flash was somehow put into full-on hardware protection mode and would not accept any inputs for editing. by jumpering VCC to the write protect pin temporarily and running some commands, we were able to take the long way around forcing the chip out of hardware protection mode and fully unlocked the chip forever with the 2022-04 u-boot going forward.



Edited 1 time(s). Last edit at 06/03/2022 02:08AM by sudos.

sudos,

I'm glad you guys solved your problem!

After seeing all that impressive works and technical know-hows, I feel bad to post this observation :) you could have just flashed stock u-boot back, start fresh, and solve this problem in an easier way. I think I've mentioned this before in my previous post.

Most of the time, the KISS principle is the best method.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> you could have just flashed stock
> u-boot back, start fresh, and solve this problem
> in an easier way. I think I've mentioned this
> before in my previous post.
>
> Most of the time, the KISS principle is the best
> method.

Hardware protection mode means the flash as a whole, all 100% of it was locked down and will not accept writes no matter what and would not take unlocking commands from the uboot command line no matter how hard you tried... Flashing stock uboot would have been impossible as it was software write protected and then hardware write protected over it. Without probing the pin with voltage and running those commands, it would have acted like a one-time programmable flash chip for the rest of its life.

Just glad both my boxes are good now and my friend's box will happen sometime tomorrow when he's back from work.

sudos, that's amazing! I'm not clear on what the "mess up" was that caused this in the first place. Do you believe that when it was received it did not have hardware protection enabled, but something during the process enabled it?

> Hardware protection mode means the flash as a
> whole, all 100% of it was locked down and will not
> accept writes no matter what and would not take
> unlocking commands from the uboot command line no
> matter how hard you tried... Flashing stock uboot
> would have been impossible as it was software
> write protected and then hardware write protected
> over it. Without probing the pin with voltage and
> running those commands, it would have acted like a
> one-time programmable flash chip for the rest of
> its life.
>
> Just glad both my boxes are good now and my
> friend's box will happen sometime tomorrow when
> he's back from work.

There are a couple diferent ways. If flashing is not possible in Debian, you can kwboot the stock u-boot image, and that stock u-boot will be able to unlock the SPI flash. I'm pretty certain that is possible. Note that once you unprotect the flash using stock u-boot, it stays unprotected forever. The only time when it got protected again was what I described before.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> There are a couple diferent ways. If flashing is
> not possible in Debian, you can kwboot the stock
> u-boot image, and that stock u-boot will be able
> to unlock the SPI flash. I'm pretty certain that
> is possible. Note that once you unprotect the
> flash using stock u-boot, it stays unprotected
> forever. The only time when it got protected again
> was what I described before.

only works if the chip is only software write protected. under normal circumstances, with a stock uboot in an unmodified virgin system, this would have definitely unlocked it. if the chip is hardware write protected however, where in this case it was in a hardware protection mode, it will only ever accept read commands and will take any write commands given as if it were unlocked, but blindly ignores them so to make you think it unlocked or flashed data to it when it actually didn't. this is what I was told by the person that works with these macronix parts for a living as it's the brand they use in-house... and putting them into said hardware protection mode is the final step out the door to prevent changes.

but, I am not saying you're wrong, because you are definitely correct, but only if the chip was software locked. this is a totally different ball game, hence the need to document it. I'm not the first to do this sort of screw-up given my friend had the same issue, and I definitely won't be the last. better I have the week-long headache about this now and save someone else from throwing their M300 across the room later :D

we got the other box's flash unlocked an hour ago and flashed/edited the correct envs to it, so it's smooth sailing from here.

> I'm not the
> first to do this sort of screw-up given my friend
> had the same issue, and I definitely won't be the
> last. better I have the week-long headache about
> this now and save someone else from throwing their
> M300 across the room later :D

Not that drastic :) since this box can be kwboot. I would use that to boot the system, and use uEnv.txt to modify the envs on the fly. That will get you to boot Debian and change envs in uEnv.txt.

And then later try to see if flashing the new u-boot at serial console will work (while running stock u-boot).

There are just so many ways to skin the same cat. But I'm not going to argue with hardware experts (surely they know more than I do).

> we got the other box's flash unlocked an hour ago
> and flashed/edited the correct envs to it, so it's
> smooth sailing from here.

Glad to hear!

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Tie up some loosends.

Kirkwood thermal works on this box.

root@tldDebian6:~# lsmod | grep -i kirkwood
kirkwood_thermal       16384  0

root@tldDebian6:~# myinfo 
tldDebian6
192.168.0.232  
Dell KACE M300
Linux version 5.18.6-kirkwood-tld-1 (root@tldDebian) (gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2) #1.0 PREEMPT Fri Jun 24 15:26:02 PDT 2022
Debian 11.3

root@tldDebian6:~# alias | grep cpu
alias get_cpu_temperature='echo "CPU Temperature "$[`cat /sys/class/hwmon/hwmon0/temp1_input | cut -c1-2`]" C"'

root@tldDebian6:~# get_cpu_temperature 
CPU Temperature 45 C

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Cool. Works on my 5.13.6 file system running Logitech Media Server:

root@Dell-M300:~# get_cpu_temperature
CPU Temperature 43 C

> Cool. Works on my 5.13.6 file system running
> Logitech Media Server:
>
>

> root@Dell-M300:~# get_cpu_temperature
> CPU Temperature 43 C
>

Strange thing is kirkwood-thermal does not work with the other Kirkwood boxes with 6281 SoC (such as GoFlex Net). I have not tried researching this topic yet, though.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> > Cool. Works on my 5.13.6 file system running
> > Logitech Media Server:
> >
> >

> > root@Dell-M300:~# get_cpu_temperature
> > CPU Temperature 43 C
> >

>
> Strange thing is kirkwood-thermal does not work
> with the other Kirkwood boxes with 6281 SoC (such
> as GoFlex Net). I have not tried researching this
> topic yet, though.

I see why

Quote
./devicetree/bindings/thermal/kirkwood-thermal.txt

This version is for Kirkwood 88F8262 & 88F6283 SoCs. Other kirkwoods
don't contain a thermal sensor.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Something to report... Even with adjusted envs to take care of the larger-than-11MB initramfs with 5.18.6, whenever I'm doing an update that requires the rewrite of uInitrd as outlined in the kernel thread, the M300 always wants to just not-boot afterwards. This also happened previously on older kernels as well, but only past 5.15.

I'm not sure what the deal is, I don't have either of mine hooked to serial console anymore, but the only solution I've found is to use a rescue flash drive I made up to boot them with and then chroot into the SSD's install. from there, run an update-initramfs -u and do the mkimage command for uInitrd as usual, sync a few times, and shutdown. it usually comes right back up without issues after removing the flash drive and cycling power.

I haven't tried to see if this is able to be replicated on either my Pogo v4 or E02... haven't bothered to turn either of them on since the M300s went into full-time use. I can recall this being an issue with the Pogo v4 for a while and then magically the problems went away for a long time after, but with the M300s it's been every single time.

Aside from this, it might be an idea to include a script for /boot in the next kernel release and subsequent further rootfs images that takes care of the command part of this by looking for the latest kernel version in /boot and injecting that into the mkimage command... probably with a yes/no choice to have the user verify the command is to be run with the kernel version gathered.

now that this has been said, I'm off to the basement to rescue-disk both the M300s and get them booted again...

sudos,

> Something to report... Even with adjusted envs to
> take care of the larger-than-11MB initramfs with
> 5.18.6, whenever I'm doing an update that requires
> the rewrite of uInitrd as outlined in the kernel
> thread, the M300 always wants to just not-boot
> afterwards. This also happened previously on older
> kernels as well, but only past 5.15.
>
> I'm not sure what the deal is, I don't have either
> of mine hooked to serial console anymore,

It's hard to say what's going on without serial console (or netconsole if it stuck in u-boot).

Is this SSD formatted Ext3 or Ext4?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------

> Is this SSD formatted Ext3 or Ext4?

Both are formatted Ext4. Next time it comes up and fails, i'll try to get a console running to capture the issue for clarity's sake.

> > Is this SSD formatted Ext3 or Ext4?
>
> Both are formatted Ext4. Next time it comes up and
> fails, i'll try to get a console running to
> capture the issue for clarity's sake.

I hope you did a "finalize" ( (no lazy_init) when you created this Ext4 rootfs.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> > > Is this SSD formatted Ext3 or Ext4?
> >
> > Both are formatted Ext4. Next time it comes up
> and
> > fails, i'll try to get a console running to
> > capture the issue for clarity's sake.
>
> I hope you did a "finalize" ( (no lazy_init) when
> you created this Ext4 rootfs.


wanted to give an update on this after so long, finally found one of my max3232 adapters and got one of them hooked up-- I made a booboo when I did the kernel upgrade around 5.18 or so it seems, and set the load_dtb_addr env to 0x2c00000 as "required" with the newer larger kernels... turns out, I'm a dummy, the M300's U-Boot is much newer and should be kept at the stock 0x1c00000 instead.

This should probably get added to the kernel page so others with an M300 don't give themselves this issue by accident, given the U-Boot is much newer as stated, and the difference between a 2017 version and a 2022 version is like comparing a Model A and a Tesla.

and yeah I didn't do a lazy_init when I created the ext4 rootfs, it's good, so I knew that wasn't the issue, I thought it was just some crazy stuff with the uboot but it turns out it was a true PEBKAC error with the uboot all along.... just took me 5 months to get around to it.

One of my two M300s has become an Owntone server in that time, and I've been using it to listen to music via iTunes on my PCs and on my Roku Soundbridge M1001. Turns out the 2GHz kirkwood has JUST enough horsepower to transcode FLAC files to something usable by iTunes in the end (probably WAV.)

The other one has been sitting on and dormant as the test box for anything I want to try running on these. I've since given away my Pogoplug E02 to a friend in CA and the v4 is still kicking around... somewhere, buried under other projects. but given how much power just the M300 has alone, I'm unlikely to use it ever again. It was a good starter box, but now we're playing with real fire. :D



Edited 1 time(s). Last edit at 02/01/2023 08:47AM by sudos.