Welcome! Log In Create A New Profile

Advanced

HowTo: openWrt on OXNAS boxes

Posted by chessplayer 
HowTo: openWrt on OXNAS boxes
August 01, 2022 07:54AM
How to install OpenWrt on OXNAS boxes

This thread is based on my tutorial HowTo: OpenWrt on Kirkwood boxes (Attention: if it is one of the Kirkwood Pogoplugs you have, it is that tutorial you want to follow!). The basic approach is
the same for the OXNAS', but some modifications are required (and, as we will see, not everything is perfect as of the original writeup of this tutorial on August 1st, 2022)

The procedure described in this thread is confirmed to work for both the Pogoplug Pro as well as the classic v3.

Credits

As always, credits are due to bodhi as well as to shv for his post on the subject and similarly to echowarrior108 for his efforts and also to David Golle over at OpenWrt.

Assumptions

I assume that the box in question has the latest uBoot and is able to boot a USB device with the latest kernel and rootfs according to the first two links in section Oxnas plugs of the wiki.

However, it will be absolutely necessary to be able to reach the box via netconsole or serial console.

The plan

We now want to install OpenWrt on our box. The steps we will need to take are
  1. Boot into debian and do some modifications to the uBoot environment variables
  2. Download, rename and copy to a FAT32 USB drive an OpenWrt Image
  3. Reboot the box without a USB device but with netconsole to interact with it <- this is crucial
  4. Use netconsole to boot OpenWrt image from the USB drive created before
  5. SSH into OpenWrt and make sure the uBoot environment is found <- this is crucial (and not quite working for the OXNAS yet)
  6. Install to NAND via sysupgrade OpenWrt
  7. Boot from NAND and SSH into OpenWrt and make sure the uBoot environment is always found (which is possible, but does not yet have quite the desired effect)
  8. Do some additional configuration
  9. Enjoy!

Implementation
So, let us look at these steps in more detail.

1. Modify uBoot variables

First, we need to boot the box to debian, log in and then do the following:
fw_setenv bootcmd_owrt 'nboot 60500000 0 440000; bootm'

All the other variables can stay the way they were before, but we have to add the OpenWrt boot option to the bootcmd. If we still have bodhi's environment variables as they came, this would be:
fw_setenv bootcmd 'run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt'

The important part is to add 'run bootcmd_owrt' to the bootcmd you are currently using. So, if you changed something, e.g. to be able to boot with a separate boot partition or from SATA or anything like that, just append the OpenWrt boot option to the one you are using.

2. Download an OpenWrt image

Next, we need the appropriate image. Start on the release page of OpenWrt and choose the one you like best and then navigate to
/targets/oxnas/ox820/
Then, download the uImage appropriate for your box. Rename it to uImage and place it in the root of a FAT32 formatted USD device.

Remark: When this tutorial was written, it meant that I used version 21.02.3, so concretely
  1. this image for the classic Pogoplug v3
  2. this image for the Pogoplug Pro

Hint: It makes sense to keep the browser tab open for the final step 6.

3. Reboot to NAND with (net)console

Then, we need to reboot the box without the debian USB device, but with a way of interacting with it, preferably via netconsole. The latter needs to be started on the server beforehand with
nc -l -u -p 6666
Wait until after a few error messages you see the prompt, e.g.
OX820>
in the console window (alternatively, you can force this via a keystroke during the bootdelay, if you configured one).

4. Reboot with OpenWrt image

Insert the drive you created in step 2 into the box and boot it by issuing the following command at the (net)console prompt:

usb reset; fatload usb 0 0x60500000 uImage; bootm 0x60500000
This should end with
Starting kernel ...
in the console output, meaning that OpenWrt is now running in RAM.

5. Log in to OpenWrt

Find the IP address of your box and ssh into it (user root without password). The following step is absolutely crucial but does not yet quite work the way it should.

Verify your mtd layout. Ideally, (see bodhi's u-Boot post) it should be:
root@OpenWrt:# cat /proc/mtd

dev:    size   erasesize  name
mtd0: 00e00000 00020000 "boot"
mtd1: 07200000 00020000 "data"

Instead, what it will probably be (at least it was for me) is:

root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"

This means that for OpenWrt, the boot-section is further subdivided, while the ubi-/data-section is the same - and this also seems to be the source of the remaining issues with this setup (see below).


Temporarily generate or modify /etc/fw_env.config
Check whether the file /etc/fw_env.config has the following contents (which is easily calculated from the above MTD layout in comparison to bodhi's):
/dev/mtd1 0xc0000 0x20000 0x20000
If not or if the file does not exist at all, the easiest way to modify/generate it is
echo '/dev/mtd1 0xc0000 0x20000 0x20000' > /etc/fw_env.config

Afterwards, check that the uBoot environment is found by printing the variables to screen using
fw_printenv

Remarks:
  1. Since OpenWrt only runs in RAM at the moment, this is a temporary change. If you want to be able to access the uBoot environment from the OpenWrt rescue system later on, this step will most likely have to be repeated (see step 7 below).
  2. If you would rather modify an existing file, you can do so using vim (which is pre-installed in OpenWrt). If, like me, you do not want to learn vim and prefer nano, you can instead do
      opkg update && opkg install nano  

6. Install to NAND

Finally, we can perform a sysupgrade of OpenWrt/LEDE either using the WIF (Web Interface) or the command line. Since we are already logged in via ssh, I only describe the latter option, while the former may be found, e.g., in step 4.b of Jörg's original post for the Kirkwoods. We must do:
opkg update
opkg install ca-certificates
cd /tmp
wget <URL of our desired sysupgrade file>
sysupgrade <name of our desired sysupgrade file>
The desired upgrade file may be found the same way as we found the image in step 2 above starting from the release page (but, ideally, you kept the browser tab open in step 2). There you can just copy the link address to the sysupgrade file (tar or bin depending on the release) and paste it into the ssh window.

Remark: Coresponding to the above uImages, the concrete files for the sysupgrade were
  1. this upgrade file for the classic Pogoplug v3
  2. this upgrade file for the Pogoplug Pro

The upgrade process will result in an automatic reboot (during which I removed the USB stick, but that may not be necessary).

7. Boot from NAND and log in to OpenWrt

There is, unfortunately, a very good chance that OpenWrt in NAND also cannot find the uBoot environment (as before in the case of the image loaded into RAM). Therefore, you have to repeat step 5. above in order to be able to modify the uBoot environment. This time, it will be a permanent modification of /etc/fw_env.config allowing you to make changes to the uBoot environment using your new rescue system. At least, this is the whole idea. But, unfortunatly, while we can READ the environment now, trying to write something to it will yield the following result:
root@OpenWrt:~# fw_setenv testvar 'something'
Can't open /dev/mtd1: Permission denied
Error: can't write fw_env to flash


8. Further configuration

Install nano (and more, if desired)
This step is completely optional, especially if you are happy to use vim. If not, do
opkg update
opkg install nano
Then you can use nano for the upcoming modifications (and opkg to install a bunch of additional packages, if you so desire).

MAC-Address
Other than the Kirkwood boxes, the OXNAS' seem not to pick up the ethaddr from the uBoot-environment. This results in a randomly assigned MAC-address at each boot and therefore (usually) ia a different IP on the local LAN for the box after each boot - definitely undesirable. Therefore, we must make use of OpenWrt's config options (as explained fully in the documentation). There are two ways of achieving this:

Option 1: within the ssh-console
nano /etc/config/network
Modify this by specifying the MAC-address (option macaddr) of your box at the appropriate locations. This should look like:
config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'xxxx:yyyy:zzzz::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0'
        # add option here
        option macaddr '00:25:31:xx:yy:zz'

config interface 'lan'
        option device 'br-lan'
        option proto 'dhcp'

config interface 'lan6'
        option device 'br-lan'
        option proto 'dhcpv6'

config device
        option name 'eth0'
        # add option here
        option macaddr '00:25:31:xx:yy:zz'
Option2: using Luci (the Web-IF)

You just open https://<IP-of-your-box> in your webbrowser and press "Login". Then, navigate to "Network -> Interfaces" and choose the "Devices"-tab. Then press the "Configure ..."-button for both "br-lan" and "eth0" and input the MAC-address there.

Edit ubootenv
This is just for sanity, since OpenWrt also specifies the location of the uBoot-environment in the file /etc/config/ubootenv. This should for reasons of sanity reflect the values we also specified in /etc/fw_env.config and should therefore be modified to look like this:
config ubootenv
        option dev '/dev/mtd1'
        option offset '0xc0000'
        option envsize '0x2000'
        option secsize '0x2000'
        option numsec '1'
Unfortunately, this also does not yield a writeable uBoot-environment ...

9. Enjoy

---
Standart ist der Standardfehler



Edited 1 time(s). Last edit at 08/01/2022 09:08AM by chessplayer.
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 10:15AM
Actually, here is some additional info: even if I boot the latest kernel / rootfs combi of bodhi's, I end up with a randomly assigned MAC-address. During the boot process, I can observe that nc connects and uses the ethaddr specified in the uBoot-env (shows up briefly in my FritzBox), but after booting Debian, it has a random MAC-address, so it is not picked up from the uBoot-env.

So, here also I have to make it use the appropriate one by specifying it in /etc/network/interfaces:

auto lo eth0
iface lo inet loopback
iface eth0 inet dhcp
    hwaddress ether 00:25:31:XX:YY:ZZ

Consequently, if I actually want to identify the individual plugs, I will have to modify this before I use a particular USB drive with a specific plug ... - not good. So, I believe that really there is something fishy with uBoot (and possibly its interaction with newer kernels (?) - see below)

Remarks:
  1. I am not 100% sure, but I think that before flashing OpenWrt, this was NOT an issue ...
  2. When booting an OLD kernel / rootfs (Debian Stretch with Linux-4.4.176-oxnas-tld-1), the correct MAC-address is used without having to specify it anywhere but in the uBoot-env

-> I am confused!

Cheers,

chessplayer

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 10:17AM
Well, openwrt put the dtb file in the kernel (like booting debian with old u-boot), and the mtd definition is in the dtb.

You can modify the dts file and compile your own kernel, then combine it with the openwrt root file system, build a volume to write to the ubi file system, then adjust the u-boot parameters.

The dts file is located here:

~/openwrt/target/linux/oxnas/files/arch/arm/boot/dts

The file name is ox820-cloudengines-pogoplugpro.dts

Make a copy of ox820-cloudengines-pogoplugpro.dts for backup.

Open the file and change it a little, mainly in the mtd area.

change it to something like this

        nand@0 {
                reg = <0>;
                #address-cells = <1>;
                #size-cells = <1>;
                nand-ecc-mode = "soft";
                nand-ecc-algo = "hamming";

                partition@0 {
                        label = "boot";
                        reg = <0x00000000 0xe00000>;
                };

                partition@e00000 {
                        label = "data";
                        reg = <0x00e00000 0x07200000>;
                };

Then re-compile openwrt.

Final files can be found in ~/openwrt/bin/targets/oxnas/ox820, we just need the kernel.

(Luci will NOT be in the system with default settings. :P) so we just use our kernel + downloaded openwrt root file system.

Get the rootfs from openwrt upgrade file, write them to ubi with the kernel, then adjust u-boot parameters.

Now we will have writable u-boot enviroment.



Edited 6 time(s). Last edit at 08/03/2022 10:58PM by daviddyer.
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 11:21AM
In the original dts file (from openwrt source)

...
                partition@0 {
                        label = "stage1";
                        reg = <0x00000000 0x00040000>;
                        read-only;
                };

                partition@40000 {
                        label = "u-boot";
                        reg = <0x00040000 0x00380000>;
                        read-only;
                };

                partition@3c0000 {
                        label = "u-boot-env";
                        reg = <0x003c0000 0x00080000>;
                };

...

Those areas are read-only



Edited 2 time(s). Last edit at 08/01/2022 02:03PM by daviddyer.
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 03:15PM
David is correct about the DTS has read-only mtd0. That's the reason why the envs are not writeable.


However, the goal here is to set the envs location to what u-boot expects to find. If this is possible then it's the optimum solution. Becaus having envs at the correct location, one can always set envs in serial/net console.

Check whether the file /etc/fw_env.config has the following contents (which is easily calculated from the above MTD layout in comparison to bodhi's): 
/dev/mtd1 0xc0000 0x20000 0x20000

The above is really a work around, which is not the best. Because whatever you set in u-boot, it won't apply to what OpenWrt sees.

I'll post my thought about this subject in the next post.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 04:37PM
1st test.

In net/serial console,

setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'

And then boot owrt in RAM (or boot into owrt on flash).
cat /proc/cmdline
Expected output would include the mtdparts above.

And then
cat /proc/mtd
Expected output would be the new mtd layout (but match most of original owrt definition for the last 3 mtds)

Adjust the env configs (the original in owrt rootfs?)
config ubootenv
        option dev '/dev/mtd2'
        option offset '0x0'
        option envsize '0x2000'
        option secsize '0x2000'
        option numsec '1'

echo '/dev/mtd2 0x0 0x20000 0x20000' > /etc/fw_env.config

And then,
fw_printenv
or
blparam

I expected to see the envs as stock OpenWrt u-boot would see.

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 2 time(s). Last edit at 08/01/2022 04:43PM by bodhi.
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 04:43PM
2nd test.


In net/serial console,

setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'

And then boot owrt in RAM (or boot into owrt on flash).
cat /proc/cmdline
Expected output would include the mtdparts above.

And then
cat /proc/mtd
Expected output would be the new mtd layout (but match most of original owrt definition for the last 3 mtds)

Adjust the env configs
config ubootenv
        option dev '/dev/mtd1'
        option offset '0x0'
        option envsize '0x2000'
        option secsize '0x2000'
        option numsec '1'

echo '/dev/mtd1 0x0 0x20000 0x20000' > /etc/fw_env.config

And then,
fw_printenv

I expected to see the envs as my u-boot would see.

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 1 time(s). Last edit at 08/01/2022 04:44PM by bodhi.
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 04:45PM
Please post the logs of both tests.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 05:13PM
1st test

OX820> setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
OX820> run bootcmd_owrt
run bootcmd_owrt

Loading from nand0, offset 0x440000
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574642 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
Skipping bad block 0x004e0000
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574642 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK


Starting kernel ...

In OpenWrt:

BusyBox v1.33.2 (2022-04-16 12:59:34 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.3, r16554-1d4dea6d4f
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"
root@OpenWrt:~#

So, unfortunately, not what you expected... Now, the other manipulations cannot really lead to anything, I guess (and they don't):

root@OpenWrt:~# cd /etc/config/
root@OpenWrt:/etc/config# cp ubootenv ubootenv_bodhi
root@OpenWrt:/etc/config# nano ubootenv
root@OpenWrt:/etc/config# cp ubootenv ubootenv_owrt
root@OpenWrt:/etc/config# cd ..
root@OpenWrt:/etc# cp fw_env.config fw_env.config_bodhi
root@OpenWrt:/etc# nano fw_env.config
root@OpenWrt:/etc# cp fw_env.config fw_env.config_owrt
root@OpenWrt:/etc# fw_printenv
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200
root@OpenWrt:/etc# blparam
-ash: blparam: not found
root@OpenWrt:/etc#

2nd test

OX820> setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
OX820> run bootcmd_lede
run bootcmd_lede

Loading from nand0, offset 0x440000
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574642 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
Skipping bad block 0x004e0000
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574642 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK


Starting kernel ...

and in OpenWrt again ...

BusyBox v1.33.2 (2022-04-16 12:59:34 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.3, r16554-1d4dea6d4f
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"
root@OpenWrt:~#

same as before ... So, also the rest gives the same result:

root@OpenWrt:~# cd /etc
root@OpenWrt:/etc# nano fw_env.config
root@OpenWrt:/etc# cd config/
root@OpenWrt:/etc/config# nano ubootenv
root@OpenWrt:/etc/config# fw_printenv
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200
root@OpenWrt:/etc/config#

Thanks for the suggestions and sorry for the bad news! And, of course:

root@OpenWrt:/etc# cp fw_env.config_bodhi fw_env.config
root@OpenWrt:/etc# fw_printenv bootcmd_exec
bootcmd_exec=run load_uimage; if run load_initrd; then if run load_dtb; then bootm 0x60500000 0x60e00000 0x62c00000; else bootm 0x60500000 0x60e00000; fi; else if run load_dtb; then bootm 0x60500000 - 0x62c00000; else bootm 0x60500000; fi; fi

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 05:31PM
chessplayer,

root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200

That's the problem. The mtdparts is not in the boot args. And this cmdline is a default one, not what we need, which should be many more parameters.

In serial/net console,
setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
printenv bootcmd_lede
or just print the whole envs set so I can see why it was missing:
printenv

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 1 time(s). Last edit at 08/01/2022 05:35PM by bodhi.
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 07:02PM
Well, I think change the dts in openwrt then recompile is the best solution for this box. In this way, Bodhi's debian and the openwrt will have the same mtd definition and it will be easy to use.

This is in openwrt (with the modified dts file)


BusyBox v1.33.1 (2021-10-24 09:01:35 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
root@OpenWrt:~# cat /etc/fw_env.config
# MTD device name       Device offset   Env. size       Flash sector size      Number of sectors
# pogoplug pro
  /dev/mtd0               0x00100000      0x20000         0x20000
root@OpenWrt:~# fw_setenv joe doe
root@OpenWrt:~# fw_printenv joe
joe=doe
root@OpenWrt:~# fw_setenv joe
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00e00000 00020000 "boot"
mtd1: 07200000 00020000 "data"
root@OpenWrt:~#

Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 07:21PM
daviddyer,

does this survive a sysupgrade from OpenWrt? (Just out of curiosity - the use case for that is probably not too obvious ...). My guess is it wouldn't, would it?

Cheers,

chessplayer

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 07:26PM
This is in Debian (NFS root)

Linux pogopronfs 5.4.101-oxnas-tld-1 #1.0 SMP PREEMPT Sat Feb 27 20:19:53 PST 2021 armv6l

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Thu Feb  3 18:03:10 2022 from 192.168.7.87
root@pogopronfs:~# cat /etc/fw_env.config
# MTD device name       Device offset   Env. size       Flash sector size      Number of sectors
# pogoplug pro
  /dev/mtd0               0x00100000      0x20000         0x20000
root@pogopronfs:~# fw_setenv joe doe
root@pogopronfs:~# fw_printenv joe
joe=doe
root@pogopronfs:~# fw_setenv joe
root@pogopronfs:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00e00000 00020000 "boot"
mtd1: 07200000 00020000 "data"
root@pogopronfs:~#

Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 07:30PM
chessplayer Wrote:
-------------------------------------------------------
> daviddyer,
>
> does this survive a sysupgrade from OpenWrt? (Just
> out of curiosity - the use case for that is
> probably not too obvious ...). My guess is it
> wouldn't, would it?
>
> Cheers,
>
> chessplayer

I'm sure it won't survive a sysupgrade from Openwrt. :P

Perhaps it won't go since the mtd definitions are different.

If upgrade is needed, we have to do it manually. (change mtd definitions in dts, re-compile kernel, re-flash the volume in the ubi)
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 07:45PM
bodhi Wrote:
-------------------------------------------------------
> chessplayer,
>
>
> root@OpenWrt:~# cat /proc/cmdline
> earlyprintk console=ttyS0,115200
>
>
> That's the problem. The mtdparts is not in the
> boot args. And this cmdline is a default one, not
> what we need, which should be many more
> parameters.
>
> In serial/net console,
>
> setenv mtdparts
> 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
> printenv bootcmd_lede
>
> or just print the whole envs set so I can see why
> it was missing:
>
> printenv
>

Ah, I understand. Yes, because of the "nboot" stuff, I did not set this (other than in the case of the Kirkwoods). I will change that tomorrow and see what comes off it (almost 3 am over here - time for bed now ...)

Cheers,

chessplayer

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 07:57PM
David,

> I'm sure it won't survive a sysupgrade from
> Openwrt. :P
>
> Perhaps it won't go since the mtd definitions are
> different.

Of course it will not survive sysupgrade :)

>
> If upgrade is needed, we have to do it manually.
> (change mtd definitions in dts, re-compile kernel,
> re-flash the volume in the ubi)

That's what I try to find out (after chessplayer done the 2 tests). The sysupgrade will show whether the envs are touched during the upgrade, and in what way. We know the kernel and ubi partitions are updated. Perhaps if you have the box running, you can verify it, too.

Perhaps I can find a way to remedy that without people have to recompile the kernel each time.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 01, 2022 10:47PM
I tried sysupgrade -v with these two files from https://downloads.openwrt.org/releases/21.02.3/targets/oxnas/ox820/

cloudengines_pogoplugpro-ubifs-sysupgrade.tar

cloudengines_pogoplugpro-ubifs-ubinized.bin

Well, nothing changed after the "upgrade", I still have my good old version and it can still boot. :P

I think I should call it a "stable" version.


BusyBox v1.33.1 (2021-10-24 09:01:35 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
root@OpenWrt:~# free
              total        used        free      shared  buff/cache   available
Mem:         120516       12324      102700          48        5492       80848
Swap:             0           0           0
root@OpenWrt:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
ubi0:rootfs             103220     10516     92704  10% /
tmpfs                    60256        48     60208   0% /tmp
tmpfs                      512         0       512   0% /dev
root@OpenWrt:~# uname -a
Linux OpenWrt 5.10.82 #0 SMP Sat Dec 4 00:39:11 2021 armv6l GNU/Linux
root@OpenWrt:~# ls
root@OpenWrt:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
ubi0:rootfs             103220     10516     92704  10% /
tmpfs                    60256        48     60208   0% /tmp
tmpfs                      512         0       512   0% /dev
root@OpenWrt:~# ls
root@OpenWrt:~# cd /tmp
root@OpenWrt:/tmp# s
-ash: s: not found
root@OpenWrt:/tmp# ls
TZ             extroot        resolv.conf    shm            tmp
board.json     lock           resolv.conf.d  state
etc            log            run            sysinfo
root@OpenWrt:/tmp# cd /boot
root@OpenWrt:/boot# ls -l
-rw-r--r--    1 root     root       3868064 Feb  3 01:11 kernel
root@OpenWrt:/boot# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00e00000 00020000 "boot"
mtd1: 07200000 00020000 "data"
root@OpenWrt:/boot# df
Filesystem           1K-blocks      Used Available Use% Mounted on
ubi0:rootfs             103220     10516     92704  10% /
tmpfs                    60256        48     60208   0% /tmp
tmpfs                      512         0       512   0% /dev
root@OpenWrt:/boot# cd /tmp
root@OpenWrt:/tmp# wget -c https://downloads.openwrt.org/releases/21.02.3/target
s/oxnas/ox820/openwrt-21.02.3-oxnas-ox820-cloudengines_pogoplugpro-ubifs-sysupgr
ade.tar
Downloading 'https://downloads.openwrt.org/releases/21.02.3/targets/oxnas/ox820/openwrt-21.02.3-oxnas-ox820-cloudengines_pogoplugpro-ubifs-sysupgrade.tar';
Connecting to 168.119.138.211:443
Writing to 'openwrt-21.02.3-oxnas-ox820-cloudengines_pogoplugpro-ubifs-sysupgrade.tar'
openwrt-21.02.3-oxna 100% |*******************************| 11310k  0:00:00 ETA
Download completed (11581772 bytes)
root@OpenWrt:/tmp# sysupgrade -v openwrt-21.02.3-oxnas-ox820-cloudengines_pogopl
ugpro-ubifs-sysupgrade.tar
Tue Aug  2 03:29:35 UTC 2022 upgrade: Saving config files...
etc/config/dropbear
etc/config/firewall
etc/config/fstab
etc/config/luci
etc/config/mdadm
etc/config/network
etc/config/rpcd
etc/config/system
etc/config/ubootenv
etc/config/ucitrack
etc/config/uhttpd
etc/dropbear/dropbear_ed25519_host_key
etc/dropbear/dropbear_rsa_host_key
etc/fw_env.config
etc/group
etc/hosts
etc/inittab
etc/luci-uploads/.placeholder
etc/opkg/keys/2f8b0b98e08306bf
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/shinit
etc/sysctl.conf
etc/uhttpd.crt
etc/uhttpd.key
etc/uhttpd.key
etc/uhttpd.crt
Tue Aug  2 03:29:35 UTC 2022 upgrade: Commencing upgrade. Closing all shell sessions.
Connection to 192.168.7.148 closed by remote host.
Connection to 192.168.7.148 closed.
pi@DreamNFS:~$ ssh root@192.168.7.216
The authenticity of host '192.168.7.216 (192.168.7.216)' can't be established.
ED25519 key fingerprint is SHA256:R8zNsCUzQwWhOzKPikBdekZKJXBLfHiTIjmJfrBPfZE.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.7.216' (ED25519) to the list of known hosts.
root@192.168.7.216's password:


BusyBox v1.33.1 (2021-10-24 09:01:35 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
root@OpenWrt:~# uname -a
Linux OpenWrt 5.10.82 #0 SMP Sat Dec 4 00:39:11 2021 armv6l GNU/Linux
root@OpenWrt:~# cd /tmp
root@OpenWrt:/tmp# ls
TZ             extroot        resolv.conf    shm            tmp
board.json     lock           resolv.conf.d  state
etc            log            run            sysinfo
root@OpenWrt:/tmp# wget -c https://downloads.openwrt.org/releases/21.02.3/target
s/oxnas/ox820/openwrt-21.02.3-oxnas-ox820-cloudengines_pogoplugpro-ubifs-ubinize
d.bin
Downloading 'https://downloads.openwrt.org/releases/21.02.3/targets/oxnas/ox820/openwrt-21.02.3-oxnas-ox820-cloudengines_pogoplugpro-ubifs-ubinized.bin';
Connecting to 168.119.138.211:443
Writing to 'openwrt-21.02.3-oxnas-ox820-cloudengines_pogoplugpro-ubifs-ubinized.bin'
openwrt-21.02.3-oxna 100% |*******************************|  8192k  0:00:00 ETA
Download completed (8388608 bytes)
root@OpenWrt:/tmp# sysupgrade -v openwrt-21.02.3-oxnas-ox820-cloudengines_pogopl
ugpro-ubifs-ubinized.bin
Tue Aug  2 03:34:23 UTC 2022 upgrade: Image metadata not present
Tue Aug  2 03:34:23 UTC 2022 upgrade: Use sysupgrade -F to override this check when downgrading or flashing to vendor firmware
Image check failed.
root@OpenWrt:/tmp# sysupgrade -v -F openwrt-21.02.3-oxnas-ox820-cloudengines_pog
oplugpro-ubifs-ubinized.bin
Tue Aug  2 03:35:09 UTC 2022 upgrade: Image metadata not present
Image check failed but --force given - will update anyway!
Tue Aug  2 03:35:09 UTC 2022 upgrade: Saving config files...
etc/config/dropbear
etc/config/firewall
etc/config/fstab
etc/config/luci
etc/config/mdadm
etc/config/network
etc/config/rpcd
etc/config/system
etc/config/ubootenv
etc/config/ucitrack
etc/config/uhttpd
etc/dropbear/dropbear_ed25519_host_key
etc/dropbear/dropbear_rsa_host_key
etc/fw_env.config
etc/group
etc/hosts
etc/inittab
etc/luci-uploads/.placeholder
etc/opkg/keys/2f8b0b98e08306bf
etc/passwd
etc/profile
etc/rc.local
etc/shadow
etc/shells
etc/shinit
etc/sysctl.conf
etc/uhttpd.crt
etc/uhttpd.key
etc/uhttpd.key
etc/uhttpd.crt
Tue Aug  2 03:35:09 UTC 2022 upgrade: Commencing upgrade. Closing all shell sessions.
Connection to 192.168.7.216 closed by remote host.
Connection to 192.168.7.216 closed.
pi@DreamNFS:~$ ssh root@192.168.7.172
The authenticity of host '192.168.7.172 (192.168.7.172)' can't be established.
ED25519 key fingerprint is SHA256:R8zNsCUzQwWhOzKPikBdekZKJXBLfHiTIjmJfrBPfZE.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.7.172' (ED25519) to the list of known hosts.
root@192.168.7.172's password:


BusyBox v1.33.1 (2021-10-24 09:01:35 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
root@OpenWrt:~#


Netconsole logs

root@m300:~# ncat -l -u -p 6666

U-Boot 2015.10-tld-2 (Oct 21 2017 - 22:00:02 -0700)
OXNAS OX820
gcc (Debian 6.3.0-18) 6.3.0 20170516
GNU ld (GNU Binutils for Debian) 2.28
Hit any key to stop autoboot:  0
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found

Reset IDE: SATA PHY not ready for device 0
ide_preinit failed

no USB devices available

no IDE devices available
running scan_disk ...
Scan device usb
device usb 0:1
** Bad device usb 0 **
device usb 1:1
** Bad device usb 1 **
device usb 2:1
** Bad device usb 2 **
device usb 3:1
** Bad device usb 3 **
Scan device ide

Reset IDE: SATA PHY not ready for device 0
ide_preinit failed
device ide 0:1
** Bad device size - ide 0 **
device ide 1:1
** Bad device ide 1 **
device ide 2:1
** Bad device ide 2 **
device ide 3:1
** Bad device ide 3 **
loading uImage ...
** Bad device usb 0 **
loading uInitrd ...
** Bad device usb 0 **
loading DTB /boot/dts/ox820-pogoplug-pro.dtb ...
** Bad device usb 0 **
Wrong Image Format for bootm command
ERROR: can't get kernel image!
UBI: attaching mtd1 to ubi0
UBI: scanning is finished
UBI: attached mtd1 (name "mtd=1", size 114 MiB) to ubi0
UBI: PEB size: 131072 bytes (128 KiB), LEB size: 129024 bytes
UBI: min./max. I/O unit sizes: 2048/2048, sub-page size 512
UBI: VID header offset: 512 (aligned 512), data offset: 2048
UBI: good PEBs: 911, bad PEBs: 1, corrupted PEBs: 0
UBI: user volume: 1, internal volumes: 1, max. volumes count: 128
UBI: max/mean erase counter: 6/3, WL threshold: 4096, image sequence number: 1508376787
UBI: available PEBs: 0, total reserved PEBs: 911, PEBs reserved for bad PEB handling: 19
Loading file '/boot/kernel' to addr 0x60500000 with size 3868064 (0x003b05a0)...
Done
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   ARM OpenWrt Linux-5.10.82
   Created:      2021-12-04   0:39:11 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3868000 Bytes = 3.7 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK


Starting kernel ...


U-Boot 2015.10-tld-2 (Oct 21 2017 - 22:00:02 -0700)
OXNAS OX820
gcc (Debian 6.3.0-18) 6.3.0 20170516
GNU ld (GNU Binutils for Debian) 2.28
Hit any key to stop autoboot:  0
starting USB...
USB0:   USB EHCI 1.00
scanning bus 0 for devices... 2 USB Device(s) found
       scanning usb for storage devices... 0 Storage Device(s) found

Reset IDE: SATA PHY not ready for device 0
ide_preinit failed

no USB devices available

no IDE devices available
running scan_disk ...
Scan device usb
device usb 0:1
** Bad device usb 0 **
device usb 1:1
** Bad device usb 1 **
device usb 2:1
** Bad device usb 2 **
device usb 3:1
** Bad device usb 3 **
Scan device ide

Reset IDE: SATA PHY not ready for device 0
ide_preinit failed
device ide 0:1
** Bad device size - ide 0 **
device ide 1:1
** Bad device ide 1 **
device ide 2:1
** Bad device ide 2 **
device ide 3:1
** Bad device ide 3 **
loading uImage ...
** Bad device usb 0 **
loading uInitrd ...
** Bad device usb 0 **
loading DTB /boot/dts/ox820-pogoplug-pro.dtb ...
** Bad device usb 0 **
Wrong Image Format for bootm command
ERROR: can't get kernel image!
UBI: attaching mtd1 to ubi0
UBI: scanning is finished
UBI: attached mtd1 (name "mtd=1", size 114 MiB) to ubi0
UBI: PEB size: 131072 bytes (128 KiB), LEB size: 129024 bytes
UBI: min./max. I/O unit sizes: 2048/2048, sub-page size 512
UBI: VID header offset: 512 (aligned 512), data offset: 2048
UBI: good PEBs: 911, bad PEBs: 1, corrupted PEBs: 0
UBI: user volume: 1, internal volumes: 1, max. volumes count: 128
UBI: max/mean erase counter: 6/3, WL threshold: 4096, image sequence number: 1508376787
UBI: available PEBs: 0, total reserved PEBs: 911, PEBs reserved for bad PEB handling: 19
Loading file '/boot/kernel' to addr 0x60500000 with size 3868064 (0x003b05a0)...
Done
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   ARM OpenWrt Linux-5.10.82
   Created:      2021-12-04   0:39:11 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3868000 Bytes = 3.7 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK


Starting kernel ...



Edited 2 time(s). Last edit at 08/01/2022 11:01PM by daviddyer.
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 06:42AM
bodhi Wrote:
-------------------------------------------------------
[...snip]
>
> That's what I try to find out (after chessplayer
> done the 2 tests). The sysupgrade will show
> whether the envs are touched during the upgrade,
> and in what way. We know the kernel and ubi
> partitions are updated. Perhaps if you have the
> box running, you can verify it, too.
>
> Perhaps I can find a way to remedy that without
> people have to recompile the kernel each time.


So, as promised, I tested again.

Remarks:
  1. I noticed that the MTD-layout is the same for tests 1 and 2 - just hope this is intentional. In that case, the only difference in the tests is the location of the uBoot-env (/mtd2 in the first test and /mtd1 in the second)
  2. Then, I set the following uBoot environment variables from Debian:

fw_setenv mtdparts 'mtdparts=41000000.nand:14m(boot),-(data)'
fw_setenv partition 'nand0,0'
fw_setenv set_bootargs_owrt 'setenv bootargs console=ttyS0,115200 $mtdparts'
fw_setenv bootcmd_owrt 'run set_bootargs_owrt; nboot 60500000 0 440000; bootm'
fw_setenv bootcmd 'run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt'

0th Test

Just using the settings from Debian gives the following

root@OpenWrt:~# fw_printenv | tail -n 5
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
partition=nand0,0
set_bootargs_owrt=setenv bootargs console=ttyS0,115200 $mtdparts
bootcmd_owrt=run set_bootargs_owrt; nboot 60500000 0 440000; bootm
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt
root@OpenWrt:~# cat /etc/fw_env.config
/dev/mtd1 0xc0000 0x20000 0x20000
root@OpenWrt:~# cat /etc/config/ubootenv

config ubootenv
	option dev '/dev/mtd1'
	option offset '0xc0000'
	option envsize '0x2000'
	option secsize '0x2000'
	option numsec '1'

root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"
root@OpenWrt:~#

Now for bodhi's tests ...

In both cases, I will (temporarily) change the MTD-layout during boot, so I get:

OX820> setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
OX820> run bootcmd_owrt
run bootcmd_owrt

Loading from nand0, offset 0x440000
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574642 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
Skipping bad block 0x004e0000
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574642 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK


Starting kernel ...

1st Test

Here, we try to read / write from /dev/mtd2:

root@OpenWrt:~# fw_printenv | tail -n 5
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
partition=nand0,0
set_bootargs_owrt=setenv bootargs console=ttyS0,115200 $mtdparts
bootcmd_owrt=run set_bootargs_owrt; nboot 60500000 0 440000; bootm
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt
root@OpenWrt:~# cp /etc/fw_env.config_test1 /etc/fw_env.config && cp /etc/config/ubootenv_test1 /etc/config/ubootenv
root@OpenWrt:~# cat /etc/fw_env.config
/dev/mtd2 0x0 0x20000 0x20000
root@OpenWrt:~# cat /etc/config/ubootenv

config ubootenv
	option dev '/dev/mtd2'
	option offset '0x0'
	option envsize '0x2000'
	option secsize '0x2000'
	option numsec '1'

root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"
root@OpenWrt:~# fw_printenv | tail -n 5
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200
root@OpenWrt:~#

2nd Test

Here, we try to read / write from /dev/mtd1:

root@OpenWrt:~# fw_printenv | tail -n 5
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
partition=nand0,0
set_bootargs_owrt=setenv bootargs console=ttyS0,115200 $mtdparts
bootcmd_owrt=run set_bootargs_owrt; nboot 60500000 0 440000; bootm
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt
root@OpenWrt:~# cp /etc/fw_env.config_test2 /etc/fw_env.config && cp /etc/config/ubootenv_test2 /etc/config/ubootenv
root@OpenWrt:~# cat /etc/fw_env.config
/dev/mtd1 0x0 0x20000 0x20000
root@OpenWrt:~# cat /etc/config/ubootenv

config ubootenv
	option dev '/dev/mtd1'
	option offset '0x0'
	option envsize '0x2000'
	option secsize '0x2000'
	option numsec '1'

root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"
root@OpenWrt:~# fw_printenv | tail -n 5
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200
root@OpenWrt:~#

Conclusion: It looks to me like NAND boot of OpenWrt never really sees the uBoot-env (does not read from bodhi's location during boot)

Cheers,

chessplayer

---
Standart ist der Standardfehler



Edited 1 time(s). Last edit at 08/02/2022 04:50PM by chessplayer.
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 03:41PM
chessplayer,


> Conclusion: It looks to me like NAND
> boot of OpenWrt never really sees the uBoot-env
> (does not read from bodhi's location during boot)

That nand boot has nothing to do with the mtdparts.

So I suspect that the tests were not done the way I expected to see. mtdparts were not passed into the OpenWrt kernel properly. In both tests in your log above, the mtds were default to OpenWrt original layout. Which does not make sense if the bootargs was set correctly.

root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"

========

Please post this for both tests

In serial serial console, at the last statement, before booting
printenv
And then in OpenWrt command line shell,
uname -a
cat /proc/cmdline
cat /proc/mtd
dmesg | grep -2 0x0   # or just entire dmesg log

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 1 time(s). Last edit at 08/02/2022 03:42PM by bodhi.
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 05:23PM
Hi bodhi,

I believe doing the first test is enough already, since the problem seems to be in the bootloader command line being ignored (according to dmesg):

Booting
U-Boot 2015.10-tld-2 (Oct 21 2017 - 22:00:02 -0700)
OXNAS OX820
gcc (Debian 6.3.0-18) 6.3.0 20170516
GNU ld (GNU Binutils for Debian) 2.28
Hit any key to stop autoboot: 10 
 0 
OX820> setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
setenv mtdparts 'mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)'
OX820> printenv
printenv
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt
bootcmd_exec=run load_uimage; if run load_initrd; then if run load_dtb; then bootm 0x60500000 0x60e00000 0x62c00000; else bootm 0x60500000 0x60e00000; fi; else if run load_dtb; then bootm 0x60500000 - 0x62c00000; else bootm 0x60500000; fi; fi
bootcmd_lede=nboot 60500000 0 440000; bootm
bootcmd_owrt=run set_bootargs_owrt; nboot 60500000 0 440000; bootm
bootcmd_uenv=run uenv_load; if test $uenv_loaded -eq 1; then run uenv_import; fi
bootdelay=10
bootdev=usb
device=0:1
devices=usb ide
disks=0 1 2 3
dtb_file=/boot/dts/ox820-pogoplug-classic.dtb
ethact=dwmac.40400000
ethaddr=00:25:31:XX:YY:ZZ
if_netconsole=ping $serverip
ipaddr=192.168.YYY.ZZZ
load_dtb=echo loading DTB $dtb_file ...; load $bootdev $device 0x62c00000 $dtb_file
load_initrd=echo loading uInitrd ...; load $bootdev $device 0x60e00000 /boot/uInitrd
load_uimage=echo loading uImage ...; load $bootdev $device 0x60500000 /boot/uImage
mainlineLinux=yes
mtdids=nand0=41000000.nand
mtdparts=mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)
nc_ready=1
ncip=192.168.YYY.ZZZ
partition=nand0,0
preboot=run preboot_nc
preboot_nc=setenv nc_ready 0; for pingstat in 1 2 3 4 5; do; sleep 1; if run if_netconsole; then setenv nc_ready 1; fi; done; if test $nc_ready -eq 1; then run start_netconsole; fi
scan_disk=echo running scan_disk ...; scan_done=0; setenv scan_usb "usb start";  setenv scan_ide "ide reset";  setenv scan_mmc "mmc rescan"; for dev in $devices; do if test $scan_done -eq 0; then echo Scan device $dev; run scan_$dev; for disknum in $disks; do if test $scan_done -eq 0; then echo device $dev $disknum:1; if load $dev $disknum:1 0x60500000 /boot/uImage 1; then scan_done=1; echo Found bootable drive on $dev $disknum; setenv device $disknum:1; setenv bootdev $dev; fi; fi; done; fi; done
serverip=192.168.YYY.ZZZ
set_bootargs=setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts $custom_params
set_bootargs_lede=setenv bootargs console=ttyS0,115200 $mtdparts
set_bootargs_owrt=setenv bootargs console=ttyS0,115200 $mtdparts
start_netconsole=setenv ncip $serverip; setenv bootdelay 10; setenv stdin nc; setenv stdout nc; setenv stderr nc; version;
stderr=nc
stdin=nc
stdout=nc
uenv_import=echo importing envs ...; env import -t 0x60510000  $filesize
uenv_init_devices=setenv init_usb "usb start";  setenv init_ide "ide reset";  setenv init_mmc "mmc rescan"; for devtype in $devices; do run init_$devtype; done;
uenv_load=run uenv_init_devices; setenv uenv_loaded 0; for devtype in $devices;  do for disknum in 0; do run uenv_read_disk; done; done;
uenv_read=echo loading envs from $devtype $disknum ...; if load $devtype $disknum:1 0x60500000 /boot/uEnv.txt; then setenv uenv_loaded 1; fi
uenv_read_disk=if test $devtype -eq mmc; then if $devtype part; then run uenv_read;  fi; else if $devtype part $disknum; then run uenv_read; fi;  fi
usb_custom_params=zswap.enabled=1
usb_ready_retry=15

Environment size: 3150/131068 bytes
OX820> run bootcmd_owrt
run bootcmd_owrt

Loading from nand0, offset 0x440000
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574682 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
## Booting kernel from Legacy Image at 60500000 ...
   Image Name:   ARM OpenWrt Linux-5.4.188
   Created:      2022-04-16  12:59:34 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3574682 Bytes = 3.4 MiB
   Load Address: 60008000
   Entry Point:  60008000
   Verifying Checksum ... OK


Starting kernel ...

in OpenWrt
BusyBox v1.33.2 (2022-04-16 12:59:34 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 21.02.3, r16554-1d4dea6d4f
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:~# fw_printenv | tail -n 5
mtdparts=mtdparts=41000000.nand:14m(boot),-(data)
partition=nand0,0
set_bootargs_owrt=setenv bootargs console=ttyS0,115200 $mtdparts
bootcmd_owrt=run set_bootargs_owrt; nboot 60500000 0 440000; bootm
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt
root@OpenWrt:~# cp /etc/fw_env.config_test1 /etc/fw_env.config && cp /etc/config/ubootenv_test1 /etc/config/ubootenv
root@OpenWrt:~# cat /etc/fw_env.config
/dev/mtd2 0x0 0x20000 0x20000
root@OpenWrt:~# cat /etc/config/ubootenv

config ubootenv
	option dev '/dev/mtd2'
	option offset '0x0'
	option envsize '0x2000'
	option secsize '0x2000'
	option numsec '1'

root@OpenWrt:~# uname -a
Linux OpenWrt 5.4.188 #0 SMP Sat Apr 16 12:59:34 2022 armv6l GNU/Linux
root@OpenWrt:~# cat /proc/cmdline
earlyprintk console=ttyS0,115200
root@OpenWrt:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00040000 00020000 "stage1"
mtd1: 00380000 00020000 "u-boot"
mtd2: 00080000 00020000 "u-boot-env"
mtd3: 009c0000 00020000 "kernel"
mtd4: 07200000 00020000 "ubi"
root@OpenWrt:~# dmesg | grep -i 0x0
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Bootloader command line (ignored): console=ttyS0,115200 mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)
[    0.858752] 0x000000000000-0x000000040000 : "stage1"
[    0.865351] 0x000000040000-0x0000003c0000 : "u-boot"
[    0.872015] 0x0000003c0000-0x000000440000 : "u-boot-env"
[    0.878963] 0x000000440000-0x000000e00000 : "kernel"
[    0.885488] 0x000000e00000-0x000008000000 : "ubi"
[    7.585502] blk_update_request: I/O error, dev mtdblock1, sector 2048 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[    7.607247] blk_update_request: I/O error, dev mtdblock1, sector 2048 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[   12.999470] blk_update_request: I/O error, dev mtdblock1, sector 2048 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[   13.021309] blk_update_request: I/O error, dev mtdblock1, sector 2048 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
root@OpenWrt:~# fw_printenv | tail -n 5
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200
root@OpenWrt:~#

So, my guess is that it does not really make sense to perform the second test at all under the circumstances, or does it?

Cheers,

chessplayer

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 05:42PM
Ok, one more thing: I now made this mtdparts-definition permanent and had a look at what we see in debian then:

root@debian ~ $ uname -a
Linux debian 5.4.179-oxnas-tld-1 #1.0 SMP PREEMPT Mon Feb 14 21:50:21 PST 2022 armv6l GNU/Linux
root@debian ~ $ cat /proc/cmdline
console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)
root@debian ~ $ cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00e00000 00020000 "boot"
mtd1: 07200000 00020000 "data"
root@debian ~ $ dmesg | grep -i 0x0
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Kernel command line: console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)
[    8.477432] 0x000000000000-0x000000e00000 : "boot"
[    8.485373] 0x000000e00000-0x000008000000 : "data"
root@debian ~ $ fw_printenv | tail -n 5
partition=nand0,0
set_bootargs_owrt=setenv bootargs console=ttyS0,115200 $mtdparts
bootcmd_owrt=run set_bootargs_owrt; nboot 60500000 0 440000; bootm
bootcmd=run bootcmd_uenv; run scan_disk; run set_bootargs; run bootcmd_exec; run bootcmd_owrt
mtdparts=mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)
root@debian ~ $

Maybe this is also of interest. I will now check what this does in OpenWrt. And, to answer my own question, the result is the same as above (Bootloader command line (ignored) ).

---
Standart ist der Standardfehler



Edited 1 time(s). Last edit at 08/02/2022 05:49PM by chessplayer.
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 05:51PM
Hi chessplayer,

> to perform the second test at all

No need to!

root@OpenWrt:~# dmesg | grep -i 0x0
[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Bootloader command line (ignored): console=ttyS0,115200 mtdparts=41000000.nand:0x100000@0x0(u-boot),0x20000@0x100000(u-boot-env-main),0x80000@0x3c0000(u-boot-env),0x440000@0x9C0000(kernel),-(ubi)
[    0.858752] 0x000000000000-0x000000040000 : "stage1"
[    0.865351] 0x000000040000-0x0000003c0000 : "u-boot"
[    0.872015] 0x0000003c0000-0x000000440000 : "u-boot-env"
[    0.878963] 0x000000440000-0x000000e00000 : "kernel"
[    0.885488] 0x000000e00000-0x000008000000 : "ubi"

> I believe doing the first test is enough already,
> since the problem seems to be in the bootloader
> command line being ignored (according to dmesg):

I see. I don't recall seeing this in your log at all! did you post this or a full dmesg before?

I was hoping otherwise, but this is apparently built in to the OpenWrt kernel config.

Now we'll go to plan B. Let me think about it a bit.

In the mean time, could you post the listing of the envs OpenWrt see. Meaning the envs at

[    0.872015] 0x0000003c0000-0x000000440000 : "u-boot-env"

If there is anything there at all after sysupgrade? (This is not the envs for my u-boot)

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 2 time(s). Last edit at 08/02/2022 05:56PM by bodhi.
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 05:59PM
Ok I think I have a good solution.

But I still would like to know if sysupgrade writes to OpenWrt "u-boot-env" mtd partition first to make sure the solution is sound.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 06:26PM
bodhi Wrote:
-------------------------------------------------------
> Ok I think I have a good solution.
>
> But I still would like to know if sysupgrade
> writes to OpenWrt "u-boot-env" mtd partition first
> to make sure the solution is sound.


Well, without trying this again, this would just be at /dev/mtd2 0x0, reflecting the original fw_env.config from OpenWrt which gave the Bad CRC (this would be the only way I know how to address that or is there a way to read from it in debian?).

What I did (in debian) is:

echo '/dev/mtd0 0x3c0000 0x20000 0x20000' > /etc/fw_env.config_owrt

(should be the right numbers, I hope) and then I also run into this Bad CRC issue:

root@debian ~ $ cp /etc/fw_env.config_owrt /etc/fw_env.config
root@debian ~ $ fw_printenv
Warning: Bad CRC, using default environment
bootcmd=run distro_bootcmd
bootdelay=2
baudrate=115200
stdin=serial,cros-ec-keyb,usbkbd
stdout=serial,vidconsole
stderr=serial,vidconsole
ethaddr=00:00:11:22:33:44
eth1addr=00:00:11:22:33:45
eth3addr=00:00:11:22:33:46
eth5addr=00:00:11:22:33:47
ipaddr=1.2.3.4
host_boot=if host dev ${devnum}; then setenv devtype host; run scan_dev_for_boot_part; fi
sata_boot=if sata dev ${devnum}; then setenv devtype sata; run scan_dev_for_boot_part; fi
scsi_init=if ${scsi_need_init}; then setenv scsi_need_init false; scsi scan; fi
scsi_boot=run scsi_init; if scsi dev ${devnum}; then setenv devtype scsi; run scan_dev_for_boot_part; fi
virtio_boot=if virtio dev ${devnum}; then setenv devtype virtio; run scan_dev_for_boot_part; fi
boot_prefixes=/ /boot/
boot_scripts=boot.scr.uimg boot.scr
boot_script_dhcp=boot.scr.uimg
boot_targets=host1 host0 
boot_syslinux_conf=extlinux/extlinux.conf
boot_extlinux=sysboot ${devtype} ${devnum}:${distro_bootpart} any ${scriptaddr} ${prefix}${boot_syslinux_conf}
scan_dev_for_extlinux=if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${boot_syslinux_conf}; then echo Found ${prefix}${boot_syslinux_conf}; run boot_extlinux; echo SCRIPT FAILED: continuing...; fi
boot_a_script=load ${devtype} ${devnum}:${distro_bootpart} ${scriptaddr} ${prefix}${script}; source ${scriptaddr}
scan_dev_for_scripts=for script in ${boot_scripts}; do if test -e ${devtype} ${devnum}:${distro_bootpart} ${prefix}${script}; then echo Found U-Boot script ${prefix}${script}; run boot_a_script; echo SCRIPT FAILED: continuing...; fi; done
scan_dev_for_boot=echo Scanning ${devtype} ${devnum}:${distro_bootpart}...; for prefix in ${boot_prefixes}; do run scan_dev_for_extlinux; run scan_dev_for_scripts; done;
scan_dev_for_boot_part=part list ${devtype} ${devnum} -bootable devplist; env exists devplist || setenv devplist 1; for distro_bootpart in ${devplist}; do if fstype ${devtype} ${devnum}:${distro_bootpart} bootfstype; then run scan_dev_for_boot; fi; done
bootcmd_host1=setenv devnum 1; run host_boot
bootcmd_host0=setenv devnum 0; run host_boot
distro_bootcmd=setenv scsi_need_init; for target in ${boot_targets}; do run bootcmd_${target}; done
bootm_size=0x10000000
kernel_addr_r=0x1000000
fdt_addr_r=0xc00000
ramdisk_addr_r=0x2000000
scriptaddr=0x1000
pxefile_addr_r=0x2000
root@debian ~ $

Cheers,

chessplayer

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 06:44PM
chessplayer,

I meant forget about Debian for a moment. Run the sysupgrade to flash "kernel' and "ubi' partitions inside OpenWrt.

After that, you will run OpenWrt from flash. So if the "u-boot-env" partition was touched by the sysupgrade process, it will list the envs OK (without CRC error)

If using OpenWrt /etc/fw_config and the section "config ubootenv" as they were defined out of the box (after sysupgrade), can the OpenWrt envs be listed? or does fw_printenv give you CRC error?

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 06:49PM
Actually, this is what I did during installation, but I can, of course, just do this once more.

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 07:10PM
bodhi Wrote:
-------------------------------------------------------
> chessplayer,
>
> I meant forget about Debian for a moment. Run the
> sysupgrade to flash "kernel' and "ubi' partitions
> inside OpenWrt.
>
> After that, you will run OpenWrt from flash. So if
> the "u-boot-env" partition was touched by the
> sysupgrade process, it will list the envs OK
> (without CRC error)
>
> If using OpenWrt /etc/fw_config and the section
> "config ubootenv" as they were defined out of the
> box (after sysupgrade), can the OpenWrt envs be
> listed? or does fw_printenv give you CRC error?

Ok, I updated to 22.03-rc6, but that did not change a thing. If I try to read from mtd2, I get the CRC error and with the calculated file I can read from, but not write to uBoot-env.

Cheers,

chessplayer

---
Standart ist der Standardfehler
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 07:10PM
Hi chessplayer,

> Actually, this is what I did during installation,
> but I can, of course, just do this once more.

Please do. OTOH, I think it is best that you run sysupgrade a few times to double check if the sysuprade has no surprise :) i.e. hidden activities that we cannot see in the log.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 07:11PM
> Ok, I updated to 22.03-rc6, but that did not
> change a thing. If I try to read from mtd2, I get
> the CRC error and with the calculated file I can
> read from, but not write to uBoot-env.

Cool!

Are you still fully awake :) or you want to continue tomorrow?

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: HowTo: openWrt on OXNAS boxes
August 02, 2022 07:13PM
Hold on.

By " with the calculated file I can read from, but not write to uBoot-env."

Do you mean the envs from my u-boot installation?

-bodhi
===========================
Forum Wiki
bodhi's corner
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: