Installing permanently to remove usb

Hi all,

what would be the steps to get the system installed in the "real" boot partition. E.g. for my NSA310 I want the system to be independent of the usb flash drive and instead have it boot from internal flash/HDD. With my limited understanding, given my partitions:

lsblk 
NAME      MAJ:MIN RM  SIZE RO TYPE   MOUNTPOINT
sda         8:0    0  1.8T  0 disk   
|-sda1      8:1    0  502M  0 part   
`-sda2      8:2    0  1.8T  0 part   
  `-md0     9:0    0  1.8T  0 linear 
sdb         8:16   1  7.6G  0 disk   
`-sdb1      8:17   1  7.6G  0 part   /
mtdblock0  31:0    0    1M  0 disk   
mtdblock1  31:1    0  512K  0 disk   
mtdblock2  31:2    0  512K  0 disk   
mtdblock3  31:3    0  512K  0 disk   
mtdblock4  31:4    0   10M  0 disk   
mtdblock5  31:5    0   10M  0 disk   
mtdblock6  31:6    0 47.8M  0 disk   
mtdblock7  31:7    0   10M  0 disk   
mtdblock8  31:8    0 47.8M  0 disk

I am assuming sda1 was created by NSA310 since it contains

l -h tmp/
total 479M
drwxr-xr-x 3 root root 1.0K Mar 13  2016 .
drwxr-xr-x 3 root root 4.0K Oct 15 00:13 ..
drwxr-xr-x 2 root root  12K Mar 13  2016 lost+found
-rwxrwxrwx 1 root root 341M Mar 13  2016 swap_ul6545p
-rw-r--r-- 1 root root 136M Mar 13  2016 sysdisk.img

which would make for sysdisk.img being the stock OS. I cannot mount the mtdblockX partitions so I would venture to guess I could overwrite the stuff in /dev/sda1 and modify uboot to load my kernel from there.
But how???

Thanks!

randomDude,

It helps repeating what your current configuration is. You're booting stock u-boot with Debian-5.13.6-kirkwood-tld-1-rootfs-bodhi.tar.bz2 on USB rootfs?

Most of the time I'd recommend installing the new u-boot so it'll make easier to run both USB or HDD rootfs.

However, the 2017.07-tld-1 NSA310 u-boot lacks Ethernet, so you cannot run tftp from u-boot after you install it. It'll give you lots of other capabilities over stock u-boot, but lacking Ethernet might be a drawback.

So if you keep stock u-boot then you need to

1. Wipe the HDD, create an Ext3 partition, and copy the USB rootfs over.

Wiki thread

Quote

Backup and Cloning rootfs

Back up a live rootfs using rsync
CrashPlan 4.3.0
Backup/Restore rootfs using tar command
Adjust udev rules after cloning rootfs
Stock Pogoplug rootfs
How to clone a rootfs from one Kirkwood box to another: Step 1 and Step 2
How to clone SATA rootfs to USB rootfs

Or create a brand new rootfs with Debian-5.13.6-kirkwood-tld-1-rootfs-bodhi.tar.bz2 the same way you did for USB.

2. Adjust the u-boot envs to boot HDD rootfs.

USB rootfs

setenv usb_set_bootargs 'setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 earlyprintk=serial'
setenv load_uimage 'ext2load usb 0:1 0x800000 /boot/uImage'
setenv load_uinitrd 'ext2load usb 0:1 0x2100000 /boot/uInitrd'
setenv usb_boot 'mw 0x800000 0 1; run load_uimage; run load_uinitrd; bootm 0x800000 0x2100000'
setenv usb_bootcmd 'run usb_set_bootargs; run usb_boot'
setenv bootcmd 'usb reset; run usb_bootcmd; usb stop; reset'

HDD rootfs

setenv sata_set_bootargs 'setenv bootargs console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 earlyprintk=serial'
setenv load_uimage 'ext2load ide 0:1 0x800000 /boot/uImage'
setenv load_uinitrd 'ext2load ide 0:1 0x2100000 /boot/uInitrd'
setenv sata_boot 'mw 0x800000 0 1; run load_uimage; run load_uinitrd; bootm 0x800000 0x2100000'
setenv sata_bootcmd 'run sata_set_bootargs; run sata_boot'
setenv bootcmd 'ide reset; run sata_bootcmd; reset'

Do it in serial console but don't save it until after you can do a few boots.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi,

sorry, I will include more info next time. Apart from that, your assumption is spot on, I am running

Quote
bohdi
Debian-5.13.6-kirkwood-tld-1-rootfs-bodhi.tar.bz2 on USB rootfs

The problem with uboot is that I cannot upgrade it to a newer version, from stock, due to bad sectors. The tutorials advise against updating uboot if that occurs below block 7, which for me is the case with blocks 3 and 4:

Quote
randomDude
# dmesg | grep -i bad
Scanning device for bad blocks
Bad eraseblock 2 at 0x000000040000
Bad eraseblock 3 at 0x000000060000

Regarding your step 1, wiping the HDD. Do you mean the partition with the stock OS, or the complete HDD?
If only the current OS partition, then I think the uboot commands are what I was looking for.

Thanks and regards,
rD


Edit: and another question: you wrote

Quote
bohdi
Do it in serial console but don't save it until after you can do a few boots.

If I have wiped the HDD how can I boot without these uboot envs? It would be impossible to start the old stock kernel due to the wipe...?

Edit 2: I just checked the block devices:

lsblk 
NAME      MAJ:MIN RM  SIZE RO TYPE   MOUNTPOINT
sda         8:0    0  1.8T  0 disk   
|-sda1      8:1    0  502M  0 part   
`-sda2      8:2    0  1.8T  0 part   
  `-md0     9:0    0  1.8T  0 linear /mnt/share
sdb         8:16   1  7.6G  0 disk   
`-sdb1      8:17   1  7.6G  0 part   /
mtdblock0  31:0    0    1M  0 disk   
mtdblock1  31:1    0  512K  0 disk   
mtdblock2  31:2    0  512K  0 disk   
mtdblock3  31:3    0  512K  0 disk   
mtdblock4  31:4    0   10M  0 disk   
mtdblock5  31:5    0   10M  0 disk   
mtdblock6  31:6    0 47.8M  0 disk   
mtdblock7  31:7    0   10M  0 disk   
mtdblock8  31:8    0 47.8M  0 disk

So I am assuming /dev/sda1 contains the stock kernel. However, the partition is only 502MB and the currently booted USB rootsfs around 800MB. So this does not fit together anyways. I am reluctant to wipe the HDD to set everything up again since there is around 1TB worth of data there. Even on a gigabit network that is annoying to transfer...



Edited 8 time(s). Last edit at 10/15/2021 04:20PM by randomDude.

randomDude,

> The problem with uboot is that I cannot upgrade it
> to a newer version, from stock, due to bad
> sectors. The tutorials advise against updating
> uboot if that occurs below block 7, which for me
> is the case with blocks 3 and 4:
>

Quote
randomDude
> # dmesg | grep -i bad
> Scanning device for bad blocks
> Bad eraseblock 2 at 0x000000040000
> Bad eraseblock 3 at 0x000000060000
>

That are serious bad blocks. You're right to stop there.

However, there are only 2 bad blocks. So it is still possible to install u-boot. Because u-boot image is 4 blocks.

Here is an important warning: you should attempt this only if you can boot with kwboot (loading the new u-boot over UART). And as usual with something risky like this, if you are OK with stock u-boot (don't care or need to boot Ext4 rootfs, or boot 3TB HDD using GPT,....) then stay with stock u-boot for now.

In the future if you decide that you need to run new u-boot to have more capabilities, then I'll help you do it.

> Regarding your step 1, wiping the HDD. Do you mean
> the partition with the stock OS, or the complete
> HDD?

If there are data on other partition that you want to keep, then you should reuse the stock OS partition only. Here it is possible further minor tweaks in the booting envs will be needed.

> If only the current OS partition, then I think the
> uboot commands are what I was looking for.


> If I have wiped the HDD how can I boot without
> these uboot envs? It would be impossible to start
> the old stock kernel due to the wipe...?

You would do this in Debian after booting with USB rootfs:

- Log in as root user, wipe the HDD partition, copy the USB rootfs over, or untar the Debian-5.13.6-kirkwood-tld-1-rootfs-bodhi.tar.bz2 to that partition.
- Get the HDD partition layout info (fdisk -l)
- Label it rootfs.
- Shutdown, remove USB rootfs.
- Power up, in serial console ennter the envs to boot the HDD rootfs.
- Here you might need further small tweak. The partition number of the HDD rootfs is important. You got this number in the 2nd bullet point above.

> I am reluctant
> to wipe the HDD to set everything up again since
> there is around 1TB worth of data there. Even on a
> gigbit network that is annoying to transfer...

Right. We only want to find the partition that contains the stock OS, not the stock kernel.

But to mess with the HDD like this, you should do a backup of the 1TB data before you do anything further.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I took a look back.

`-sda2      8:2    0  1.8T  0 part   
  `-md0     9:0    0  1.8T  0 linear

Now the above info stops me in the track :) there are RAID involved, so I think you should not attempt to put the rootfs on HDD yet.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi,

thank you for your explanations!

I don't really need a new uboot for now, the changes are not important at the moment.

Regarding moving the rootfs from USB to HDD. Is there a way to reduce the size (significantly)? As I wrote in an edit, I have only ~500MB on that partition but the USB rootfs is ~800MB...

And, yes, for some reason, back when I was a lot younger, I decided to try a JBOD raid on the nas :)

rD



Edited 1 time(s). Last edit at 10/15/2021 04:50PM by randomDude.

randomDude,

> Regarding moving the rootfs from USB to HDD. Is
> there a way to reduce the size (significantly)? As
> I wrote in an edit, I have only ~500MB on that
> partition but the USB rootfs is ~800MB...

500MB is too small for Debian. You might be able to shrink it, but then you will not be able to upgrade it with too little space to spare.

You could run USB rootfs on a hub. I do this for all boxes that have USB rootfs and one USB port. I just use a small (half of a credit card size) portable USB 3.0 hub that hangs by its cable. And take precaution to not wearing it out.

Wiki thread:

Quote

Flash drives issues

How to prolong the life of your rootfs USB thumb drive

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi bodhi,

I think I understand what you did with the uboot commands. In the end you load a file called '/boot/uEnv.txt', which was not included in the tar file. Is there more important info in there?

rD

Never mind, I found the info in uboot article...
:)



Edited 1 time(s). Last edit at 10/18/2021 04:52PM by randomDude.

bodhi,
. I am wondering how you would copy data (e.g. the kernel) there. Do you have a wiki article for that?

Thanks!
rD



Edited 1 time(s). Last edit at 10/18/2021 04:51PM by randomDude.

deleted



Edited 2 time(s). Last edit at 10/18/2021 04:53PM by randomDude.

randomDude,

> . I am wondering how you would copy data (e.g. the
> kernel) there. Do you have a wiki article for
> that?

Not sure I understood the question! please describe what you want to do.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I think he wants to embed Debian in nand. And as we all know, that's not very possible since kernel 2 days.

Yes, that is what I was thinking. What changed since then that it is no longer possible?
rD

rD,

The kernel and rootfs got larger. Emdebian was the small-embedded Linux distro, and it ceased development long ago. Nowadays, NAS manutacturers only store the kernel files on NAND, but the rootfs is usually on HDD.

The distro that has a setup with everything on flash is OpenWrt.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 10/20/2021 04:15AM by bodhi.

bodhi Wrote:
-------------------------------------------------------
> rD,
>
> The distro that has a setup with everything on
> flash is OpenWrt.

rD,

bodhi has come up with clever ways of dual booting Debian and Openwrt for other platforms. Not sure if your device has Openwrt support but there are a number of posts on this forum that discuss this capability, in particular Pogoplug Pro and V4/Mobile.

LME

edit: Nas310 Openwrt info here: https://openwrt.org/toh/zyxel/nsa310b so it is supported.



Edited 1 time(s). Last edit at 10/20/2021 12:53PM by LeggoMyEggo.

Thanks bohdi and LeggoMyEggo for the info,

but if Openwrt can do it all in flash, why can't we? :)

Sorry if the questions are dumb, but couldn't the kernel (which would fit in NAND) load rootfs (or 'init') from HDD?
The workflow being something like this:
the kernel and a "mini" init are on one smaller partition (on HDD or NAND or wherever); the kernel starts this "mini" init, which is only capable of mounting the rootfs partition, then the kernel switches over to the rootfs' init and normal bootup starts?

rD

randomDude,

> Sorry if the questions are dumb, but couldn't the
> kernel (which would fit in NAND) load rootfs (or
> 'init') from HDD?

I think I've mentioned this before, It is posiible to store the kernel files on flash. But the Debian rootfs must be on USB or HDD. And this is what most NAS stock configuration look like.

OTOH, OpenWrt store everything on flash (as a router, their rootfs is very small). That's the difference.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 10/20/2021 04:30PM by bodhi.

rD,

If you were under the impression NAND was much faster for booting the kernel and rootfs, that's not always true and in fact some NAND is dog ass slow. So you won't suffer any serious speed penalties by putting the rootfs on HDD as the processor speed, amount of RAM and storage transfer rates are more crucial.

That is what bodhi has patiently taught us :)

LME



Edited 2 time(s). Last edit at 10/20/2021 05:04PM by LeggoMyEggo.

Sorry if this muddies the water, but you can reduce the size of your mdadm raid to get space for a set of ext3 boot and root partitions.

Process is reasonably straightforward, however any disk modification comes with risks.

Howto