Debian on Synology RS816 (Armada 385)

Hi Richard,

As with headless NAS box like this, we should approach the installation in phases. We need to to mess with u-boot envs, so there is always a great risk for users to perform a complicated installation procedure. So I think the installation steps should go like this.

I. Installation with USB rootfs

1. Create the USB rootfs using Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2 following the instruction from the kernel/rootfs release thread.

<Note whether step 4 in the rootfs installation is needed>

2. After the USB rootfs preparation is done. Bring it to the RS816 and plug in to the USB port

3. Power up, interrupt serial console and at the u-boot prompt.

3.a. Execute the following commands to set up u-boot envs.
....
3.b.
3.c.
....

<so on about u-boot envs to load kernel files, the subsections above are to separate the subtopics. It makes it easier for users to see the overview and ask question>

3.x. Done section I


II. Installion with HDD rootfs.

1. Create the HDD rootfs using Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2 following the instruction from the kernel/rootfs release thread.

1a. Prepare the kernel files in raw sectors.
....
1.b. Write the kernel files to raw sectors.
....
1.c
1.d.
....

2. After the HDD rootfs preparation is done. Bring it to the RS816 and plug in to the SATA port

3. Power up, interrupt serial console and at the u-boot prompt.

3.a. Execute the following commands to set up u-boot envs
....
3.b.
3.c.
.....
<so on about u-boot envs to load kernel files from raw sectors>

3.x. Done section II


III. Installation with RAID rootfs

1. Backup the HDD or USB rootfs. As the result of Section I or II, this rootfs is Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2. Now it is the reference rootfs when something goes wrong in RAID installation.

2. Boot up the system with Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2. rootfs

3. Adjust the u-boot envs to boot with RAID.

4.
<describe any steps needed to be done in the rootfs to activate RAID, if at all>

....
x. Done section III

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi Richard,

I was writing at the same time :) see above.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hi Richard,

I should also add that it is your installation procedure! you can do whatever you'd like :) (I'd only suggest an approach that I think is best for the community).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

@bohdi

My preference would be to do what is best for the community because this is my way of saying thank you for the help I received so I will have a go at constructing the tutorials along the lines you suggest.

My thoughts generally had been that with a rack server most people would just want to install as a 4 disk raid and that as I would have to type all the commands in to produce a tutorial anyway I might as well list them out in order that install was as simple as "copy paste your way through the tutorial" (I have tested doing exactly that) However having given it more thought I do acknowledge that these forums are not a place for linux newbies and that actually I may inadvertently be making everyone here's life a misery when someone who probably shouldn't be attempting this makes a simple copy paste error then bombards the forums with questions the answers to which they do not understand and hence in hindsight keeping tutorials to be only manageable by reasonably advanced linux users is probably the best way to go

Regarding extra tools in the rootfs I would ask you to reconsider for mdadm at least.

On my first try with your rootfs below is what happened to me. I'm sure a modified installation procedure would get around it but my point is why make things harder than necessary ?

1) Prepare USB rootfs
2) Boot NAS to USB rootfs
3) Install mdadm & parted from within rootfs
4) Create raids for installation
5) Install rootfs to new raids (from the copy of the rootfs my tutorial placed in USB /root)
5) chroot into new raids remembering there is now no mdadm there.
6) Install mdadm
7) update-initramfs -u etc and reboot

Problem on reboot is both rootfs and swap raids started degraded, no problem I thought to myself.

1) mdadm --manage /dev/md0 --add /dev/{the missing drive} (raid rebuilds) do same for swap raid.
2) mdadm --detail --scan >> /etc/mdadm/mdadm.conf (then edit out previous entries)
3) update-initramfs -u
4) re-create uImage and re-flash
5) reboot

>> Go back to start raid is degraded again!!.

There were uuid errors in the console log and I guess I could have used gptdisk to create new uuid's but decided not to try and debug I simply re-copied the install rootfs (which now had mdadm in it) and installed again with no issues this time.

Your rootfs and so your call but I would ask you to at least consider adding mdadm.

Richard

@bodhi

For the tutorials I am going to link into your USB preparation tutorial as you requested but to make a logical read of my tutorials I think we need to make a couple of changes.

1) For the purposes of my tutorials we need to generate uInitrd as well as uImage as we use both in the initial USB tftp boot. (lets us use LABEL=)

2) In my tutorials I copy the rootfs tarball into the prepared USB which I use for extraction onto HDD. I could just add this extra step to my tutorial its just I just think, if at all possible, we should have one complete USB preparation source and not broken up across various tutorials.

3) There is currently a numbering issue on your USB preparation tutorial which would be good to fix (currently listed as steps 1, 2, 4, 3, 4)

I have no issue if you have reasons not to want to do 1) and 2) but if so I think I should add the full USB preparation to my tutorials in order to avoid fragmentation.

I will wait for your thoughts before pressing on.

Richard

Hi Richard,

> My thoughts generally had been that with a rack
> server most people would just want to install as a
> 4 disk raid and that as I would have to type all
> the commands in to produce a tutorial anyway I
> might as well list them out in order that install
> was as simple as "copy paste your way through the
> tutorial" (I have tested doing exactly that)
> However having given it more thought I do
> acknowledge that these forums are not a place for
> linux newbies

A lot users don't care about RAID at all, even running rack server. If they do want RAID, they can always install it after the box has been booted with USB or HDD rootfs. Besides, my purpose for basic rootfs is to be a learning vehicle too. I could have provided a heavy rootfs, but then it will become another Ubuntu-like distro, no force opportunity for newcomers to learn Linux. In fact, the target audience of this forum are experienced Linux users who dislike heavy rootfs (large and bloated with stuff they don't care).

> Regarding extra tools in the rootfs I would ask
> you to reconsider for mdadm at least.

Even though I don't think it is necessary, I do agree that it will make it easier. This mdadm has been on my TODO note for 3,4 years :) it got lost in the log many activities after, and I forgot all about it.

> On my first try with your rootfs below is what
> happened to me. I'm sure a modified installation
> procedure would get around it but my point is why
> make things harder than necessary ?
>
> 1) Prepare USB rootfs
> 2) Boot NAS to USB rootfs
> 3) Install mdadm & parted from within rootfs
> 4) Create raids for installation
> 5) Install rootfs to new raids (from the copy of
> the rootfs my tutorial placed in USB /root)
> 5) chroot into new raids remembering there is now
> no mdadm there.
> 6) Install mdadm
> 7) update-initramfs -u etc and reboot
>
> Problem on reboot is both rootfs and swap raids
> started degraded, no problem I thought to myself.

This degradation should not occur.

>
> 1) mdadm --manage /dev/md0 --add /dev/{the missing
> drive} (raid rebuilds) do same for swap raid.
> 2) mdadm --detail --scan >> /etc/mdadm/mdadm.conf
> (then edit out previous entries)
> 3) update-initramfs -u
> 4) re-create uImage and re-flash
> 5) reboot
>
> >> Go back to start raid is degraded again!!.
>
> There were uuid errors in the console log

That sounds like an important hint.

> and I
> guess I could have used gptdisk to create new
> uuid's but decided not to try and debug I simply
> re-copied the install rootfs (which now had mdadm
> in it) and installed again with no issues this
> time.
>
> Your rootfs and so your call but I would ask you
> to at least consider adding mdadm.

If you can't figure out why it would be degraded then a new rootfs with mdadm preinstalled would be the solution. I could prepare and upload another rootfs.

In the mean time I think we can have the Installation Instruction. Section III should be written with the notice that it is still a draft and will be completed in another 4,5 weeks after you come back.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Richard,

> 1) For the purposes of my tutorials we need to
> generate uInitrd as well as uImage as we use both
> in the initial USB tftp boot. (lets us use
> LABEL=)
>
> 2) In my tutorials I copy the rootfs tarball into
> the prepared USB which I use for extraction onto
> HDD. I could just add this extra step to my
> tutorial its just I just think, if at all
> possible, we should have one complete USB
> preparation source and not broken up across
> various tutorials.
>

I think the USB rootfs creation should be referenced to my released post. Even though it is short, It has many caveats and concised instruction that will make your instruction cluttered up if you repeat it.

Give it a shot, make the reference (I always do this in tutorials here) and we'll review it to see if it would read better or not.

OTOH, If you intend the tutorial to be self-contained so it can be used elsewhere, then by all means, copy the rootfs creation steps to this tutorial and modify it. I have no objection at all (I agree it should not point people to this forum just to read it :).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

@bodhi
I have no plans to post the tutorial anywhere else, it is specific to your rootfs / kernel.
Remember I use my own rootfs / kernel for my purposes and if I were to post elsewhere (which I'm not) it would be with those.

>> .....Linux users who dislike heavy rootfs (large and bloated with stuff they don't care)......

I understand your "bloat" comment I feel the same way which is why I run my own fork of Arch Linux, but for the record my Debian rootfs includes build-essential in addition to mdadm, parted etc and is only 2mb bigger than yours, this was achieved by ripping stuff out of your rootfs which I considered bloat, i.e. bloat is in the eye of the beholder :-)

>> There were uuid errors in the console log
> That sounds like an important hint.

I'm just speculating but I think at the point we chroot the absence of mdadm triggers some auto config but it's also possible my experience was a one off. I will run it again but have no time to debug beyond noting if it occurs again at which point installing mdadm in the rootfs is the answer.

>Give it a shot, make the reference (I always do this in tutorials here) and we'll review it to see if it would read better or not.

I will give that a go.

My wife is giving me a seriously hard time to spend at least some of the public holiday with her so I will not start till tomorrow but will try to get as much done as I can.

Richard

Installation instructions for Synology RS816/DS116


I. Installation with USB rootfs

Prerequisites:

a. All HDD drives must be removed from the RS816/DS116 while this installation is performed.
b. tftp server is already set up in your network environment (we will use it for the kernel initial booting before it can be set up permanently in flash).
c. Serial console must be connected during installation.

Steps:

1. Create the USB rootfs using Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2 following the instruction from Linux Kernel 5.2.9 MVEBU package and Debian armhf rootfs..

Note: Step 3 (Create uImage with DTB appended) in the rootfs should look like below with armada-385-synology-rs816.dtb as the RS816 DTB file name.

cd /media/sdb1/boot
cp -a zImage-5.2.9-mvebu-tld-1 zImage.fdt 
cat dts/armada-385-synology-rs816.dtb  >> zImage.fdt
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-mvebu-tld-1 -d zImage.fdt uImage

2. Copy the 2 kernel files uImage and uInitrd (in the USB rootfs ./boot folder) to the root of your tftp server.

3. Attach the USB rootfs to the RS816. Power up, interrupt serial console and at the u-boot prompt,

(192.168.1.19 is the IP address of the RS816, and 192.168.1.20 is the IP address of the tftp server, please substituing appropriate IP addresses for you own network environment).

3.a. Execute the following commands to set up u-boot envs

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;'                
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv load_image 'echo loading uImage from tftpserver ...; tftpboot $load_image_addr uImage'
setenv load_initrd 'echo loading uInitrd from tftpserver ...; tftpboot $load_initrd_addr uInitrd'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts earlyprintk=serial"'
setenv bootcmd_exec 'echo Booting Debian . . .; run usb_setup;  run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; run load_initrd; bootm $load_image_addr $load_initrd_addr'
setenv bootcmd 'run bootcmd_exec; run bootspi'
boot

4. Watch the serial console logging, the Debian prompt will appear when the box finishes booting. Login with root/root credential.

In Debian, back up current SPI mtds for future restoration if you wish to return to stock.

dd if=/dev/mtd0 of=mtd0.stock.rs816 bs=4096k conv=sync
dd if=/dev/mtd1 of=mtd1.stock.rs816 bs=4096k conv=sync

Keep these somewhere safe!

At this point, you have successfully booted the Debian rootfs on USB. But since we have relied on tftp to load kernel files, Step 5 below will redefine the mtd partitions layout, set up the envs, and flash the kernel uImage to SPI mtd2 for permanent booting to this USB rootfs.

5. Power up (or reboot), interrupt serial console and at the u-boot prompt.

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;' 
setenv mtdparts 'mtdparts=spi0.0:896k(u-boot),64k(dtb),7104k(kernel),-(env)'               
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv load_image 'echo loading uImage from tftpserver ...; tftpboot $load_image_addr uImage'
setenv load_initrd 'echo loading uInitrd from tftpserver ...; tftpboot $load_initrd_addr uInitrd'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=LABEL=rootfs rootdelay=10 $mtdparts earlyprintk=serial"'
setenv bootcmd_exec 'echo Booting Debian . . .; run usb_setup;  run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; run load_initrd; bootm $load_image_addr $load_initrd_addr'
setenv bootcmd 'run bootcmd_exec; run bootspi'
boot

5.a. Watch the serial console logging, the Debian prompt will appear when the box finishes booting. Login with root/root credential.

In Debian, check the new mtd partitions definition is as expected.

cat /proc/mtd 
dev: size erasesize name 
mtd0: 000e0000 00001000 "u-boot" 
mtd1: 00010000 00001000 "dtb" 
mtd2: 006f0000 00001000 "kernel" 
mtd3: 00020000 00001000 "env"

5.b. Flash uImage to mtd2

cd /boot
dd if=uImage of=rs816.uImage.mtd2.kwb bs=7104k conv=sync
flashcp -v rs816.uImage.mtd2.kwb /dev/mtd2

5.c. Reboot, interrupt serial console. And then set up u-boot envs permanently.

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;'                
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv mtdparts 'mtdparts=spi0.0:896k(u-boot),64k(dtb),7104k(kernel),-(env)' 
setenv boot_config_spi 'setenv image_addr_spi 0x0F0000; setenv image_size_spi 0x006f0000; sf probe 0 50000000' 
setenv load_image_spi 'echo loading uImage from flash ...; sf read $load_image_addr $image_addr_spi $image_size_spi' 
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=/dev/sda1 rootdelay=10 $mtdparts earlyprintk=serial"' 
setenv bootcmd_exec 'echo Booting Debian ...; run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run boot_config_spi; run load_image_spi; bootm $load_image_addr -' 
setenv bootcmd 'run bootcmd_exec; run bootspi' 
saveenv 
reset

Allow the box to boot automatically, and Enjoy!

6. Backup USB rootfs

To backup the USB rootfs follow the wiki tutorial Backup/Restore rootfs using tar command.

End Installation

bodhi edited: for clarity and added DS116



Edited 9 time(s). Last edit at 01/02/2021 03:39PM by bodhi.

Hi Richard.

Looks good so far! I'll post comments tomorrow.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi edited: Please see this post for final HDD installation instruction.


Draft II. Installation with HDD rootfs.

DRAFT DO NOT FOLLOW UNTILL TRANSFERRED TO WIKI

Installation instructions for Synology RS816

II. Installion with HDD rootfs.

In this section we will install uImage and uInitrd to raw sectors on a HDD this method was originally contributed to the wiki by forum members eine HERE and jdwl101 HERE

Due to this method utilising uInitrd there are no limitations in using LABEL= or UUID when referencing drives

1. Follow section I. Installation with USB rootfs up to step 4. there is no need to backup the mtd's because we will not be touching or using them.

2. Install a single HDD into drive bay 1 it will show up as /dev/sdb

3. If this is a previously used disk clear existing partitions and ensure first 50MiB are clear of data

dd if=/dev/zero of=/dev/sdb bs=1MiB count=50

3.a Create your desired partition table ensuring the start is > 50MiB

3.b Create filesystem the RS816 is fine with ext3 or ext4 and create a label I suggest not using "rootfs" to avoid a naming conflict with the USB created in Section I in a recovery scenario.

mkfs.ext4 -L ROOT /dev/sdb1

3.c mount the partition and Install the rootfs tarball

mount /dev/sdb1 /mnt
tar -C /mnt -xjvf Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2
sync

3.d chroot into environment and setup your system (fstab etc)

mount -t proc proc /mnt/proc 
mount --rbind /sys /mnt/sys 
mount --rbind /dev /mnt/dev 
chroot /mnt /bin/bash
vi /etc/fstab (and anything else you want to do)

3.e Update boot images uImage & uInitrd and write to raw sectors

cd /boot
update-initramfs -u
cp -a zImage-5.2.9-mvebu-tld-1 zImage.fdt
cat dts/armada-385-synology-rs816.dtb >> zImage.fdt
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-mvebu-tld-1 -d zImage.fdt uImage
mkimage -A arm -O linux -T ramdisk -C gzip -a 0x00000000 -e 0x00000000 -n initramfs-5.2.9-mvebu-tld-1 -d initrd.img-5.2.9-mvebu-tld-1 uInitrd
dd if=uImage of=/dev/sdb bs=1MiB seek=10
dd if=uInitrd of=/dev/sdb bs=1MiB seek=20
exit
poweroff

4. Remove USB boot and interrupt at u-boot (modify bootargs to suit your boot device)

setenv load_image_addr 0x02000000
setenv load_initrd_addr 0x3000000
setenv load_image 'echo loading uImage from raw HDD ...; scsi device 0; scsi read $load_image_addr 0x5000 0x2F00'
setenv load_initrd 'echo loading uInitrd from raw HDD ...; scsi device 0; scsi read $load_initrd_addr 0xA000 0x4F00'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=LABEL=ROOT rootdelay=20 $mtdparts earlyprintk=serial"'
setenv bootcmd_exec 'echo Booting Debian . . .; run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; run load_initrd; bootm $load_image_addr $load_initrd_addr'
setenv bootcmd 'scsi init; run bootcmd_exec; run bootspi'
saveenv
reset

Allow to reboot and Enjoy!

5. Backup HDD rootfs
To backup your HDD rootfs see HERE

End section



Edited 10 time(s). Last edit at 01/01/2021 08:30PM by bodhi.

Draft III. Installation with RAID rootfs.

DRAFT DO NOT FOLLOW UNTILL TRANSFERRED TO WIKI

Installation instructions for Synology RS816

III. Installion with RAID rootfs.

In this section we will install uImage into NAND and create a 4 drive raid. This method is very similar to the stock Synology installation but with the benefit of being native Debian.

This tutorial is only interested in creating the raids for rootfs and swap which I suggest you make 30GiB and 2GiB respectively, you could then use the remainder of your HDD capacity at you preference but maybe for /home or /data etc.

1. Follow section I. Installation with USB rootfs up to and including step 4.

2. Install 4 HDD's into the drive bays (these will show up as /dev/{sdb, sdc, sde}

3. Create the partitions for rootfs & swap and mark the partitions as raid. I use parted which I use as an example below. (you would do this for all 4 drives)

parted /dev/sdb
(parted) mklabel msdos
(parted) mkpart primary 0% 30GiB 
(parted) mkpart primary 30Gib 32Gib  
(parted) set 1 raid on 
(parted) set 2 raid on

3.a Currently the default rootfs tarball does not contain mdadm so you will need to install it twice once now and again after you chroot into the new raid. Also due to errors encountered in testing it seems necessary to reboot again now. Interupt to u-boot and boot with the same envs as from step3.a in Section I

3.b Now create the raids with mdadm (the important bit here is --meta MUST be 0.9)

mdadm --create /dev/md0 --level=1 --meta=0.9 --raid-devices=4 /dev/sd[b-e]1
mdadm --create /dev/md1 --level=1 --meta=0.9 --raid-devices=4 /dev/sd[b-e]2

3.c Create filesystem the RS816 is fine with ext3 or ext4 and mount to /mnt

mkfs.ext4 /dev/md0
mkswap /dev/md1
mount /dev/md0 /mnt
swapon /dev/md1

3.d Extract rootfs to new raid

tar -C /mnt -xjvf Debian-5.1.11-mvebu-tld-1-rootfs-bodhi.tar.bz2 
sync

3.e chroot into environment and setup your system (mdadm.conf fstab etc)

mount -t proc proc /mnt/proc 
mount --rbind /sys /mnt/sys 
mount --rbind /dev /mnt/dev 
chroot /mnt /bin/bash
mdadm --detail --scan >> /etc/mdadm/mdadm.conf
vi /etc/fstab (and anything else you want to setup)

3.f Update boot images uImage

cd /boot
update-initramfs -u
cp -a zImage-5.2.9-mvebu-tld-1 zImage.fdt
cat dts/armada-385-synology-rs816.dtb >> zImage.fdt
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-mvebu-tld-1 -d zImage.fdt uImage
exit
poweroff

4. Create flash images and flash NAND

4.a Boot and interrupt to u-boot adjust serverip & addr to match your tftp

setenv ipaddr 192.168.1.19 
setenv serverip 192.168.1.1
setenv mtdparts 'mtdparts=spi0.0:896k(u-boot),64k(dtb),7104k(kernel),-(env)'
setenv load_image_addr 0x02000000 
setenv load_image 'echo loading uImage from tftpserver ...; tftpboot $load_image_addr uImage'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=/dev/md0 rootdelay=20 $mtdparts earlyprintk=serial"' 
setenv bootcmd_exec 'echo Booting Debian . . .; run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; bootm $load_image_addr -' 
setenv bootcmd 'run bootcmd_exec; run bootspi' 
boot

4.b Check mtd's look right

cat /proc/mtd 
dev: size erasesize name 
mtd0: 000e0000 00001000 "u-boot" 
mtd1: 00010000 00001000 "dtb" 
mtd2: 006f0000 00001000 "kernel" 
mtd3: 00020000 00001000 "env"

4.c Create & flash images

cd /boot
cp -a /boot/zImage-5.2.9-mvebu-tld-1 zImage.fdt
cat /boot/dts/armada-385-synology-rs816.dtb >> zImage.fdt
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-mvebu-tld-1 -d zImage.fdt uImage
dd if=uImage of=rs816.uImage.mtd2.kwb bs=7104k conv=sync
flashcp -v rs816.uImage.mtd2.kwb /dev/mtd2
poweroff

5. Remove usb reboot ant interrupt to u-boot

setenv mtdparts 'mtdparts=spi0.0:896k(u-boot),64k(dtb),7104k(kernel),-(env)' 
setenv boot_config_spi 'setenv image_addr_spi 0x0F0000; setenv image_size_spi 0x006f0000; sf probe 0 50000000' 
setenv load_image_spi 'echo loading uImage from flash ...; sf read $load_image_addr $image_addr_spi $image_size_spi' 
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=/dev/md0 rootdelay=20 $mtdparts earlyprintk=serial"' 
setenv bootcmd_exec 'echo Booting Debian ...; run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run boot_config_spi; run load_image_spi; bootm $load_image_addr -' 
setenv bootcmd 'run bootcmd_exec; run bootspi' 
saveenv
reset

Allow to reboot and Enjoy!

End section



Edited 7 time(s). Last edit at 09/29/2019 11:37PM by richjn.

@bodhi
All three tutorials are now complete and have been tested to work. There are no doubt quite a few edits required to tidy it up but the basics are sound.

The workaround for no mdadm in rootfs was to insert an additional reboot.

I'm out of time and need to return the NAS to stock now, hopefully any changes to the tutorial from here on will not require testing.

As sole judge I have decided to award myself a bottle of shiraz :-)

Richard

Hi Richard,

I've modified Section I to make it clearer, add missing envs setup, and in some instances remove redundant or too detailed information.

Once you have confirmed with no objection, I will copy and paste (to preserve formatting) this edited section back to your Section I post.


=====

Installation instructions for Synology RS816

I. Installation with USB rootfs

Prerequisites:

a. All HDD drives must be removed from the RS816 while this installation is performed.
b. tftp server is already set up in your network environment (we will use it for the kernel initial booting before it can be set up permanently in flash).

Steps:

1. Create the USB rootfs using Debian-5.2.9-mvebu-tld-1-rootfs-bodhi.tar.bz2 following the instruction from Linux Kernel 5.2.9 MVEBU package and Debian armhf rootfs..

Note: Step 3 (Create uImage with DTB appended) in the rootfs should look like below with armada-385-synology-rs816.dtb as the RS816 DTB file name.

cd /media/sdb1/boot
cp -a zImage-5.2.9-mvebu-tld-1 zImage.fdt 
cat dts/armada-385-synology-rs816.dtb  >> zImage.fdt
mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-5.2.9-mvebu-tld-1 -d zImage.fdt uImage

2. Copy the 2 kernel files uImage and uInitrd (in the USB rootfs ./boot folder) to the root of your tftp server.

3. Attach the USB rootfs to the RS816. Power up, interrupt serial console and at the u-boot prompt,

(192.168.1.19 is the IP address of the RS816, and 192.168.1.20 is the IP address of the tftp server, please substituing appropriate IP addresses for you own network environment).

3.a. Execute the following commands to set up u-boot envs

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;'                
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv load_image 'echo loading uImage from tftpserver ...; tftpboot $load_image_addr uImage'
setenv load_initrd 'echo loading uInitrd from tftpserver ...; tftpboot $load_initrd_addr uInitrd'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=LABEL=rootfs rootdelay=20 $mtdparts earlyprintk=serial"'
setenv bootcmd_exec 'echo Booting Debian . . .; run usb_setup;  run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; run load_initrd; bootm $load_image_addr $load_initrd_addr'
setenv bootcmd 'run bootcmd_exec; run bootspi'
boot

4. Watch the serial console logging, the Debian prompt will appear when the box finishes booting. Login with root/root credential.

In Debian, back up current SPI mtds for future restoration if you wish to return to stock.

dd if=/dev/mtd0 of=mtd0.stock.rs816 bs=4096k conv=sync
dd if=/dev/mtd1 of=mtd1.stock.rs816 bs=4096k conv=sync

Keep these somewhere safe!

At this point, you have successfully booted the Debian rootfs on USB. But since we have relied on tftp to load kernel files, Step 5 below will redefine the mtd partitions layout, set up the envs, and flash the kernel uImage to SPI mtd2 for permanent booting to this USB rootfs.

5. Power up (or reboot), interrupt serial console and at the u-boot prompt.

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;' 
setenv mtdparts 'mtdparts=spi0.0:896k(u-boot),64k(dtb),7104k(kernel),-(env)'               
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv load_image 'echo loading uImage from tftpserver ...; tftpboot $load_image_addr uImage'
setenv load_initrd 'echo loading uInitrd from tftpserver ...; tftpboot $load_initrd_addr uInitrd'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=LABEL=rootfs rootdelay=20 $mtdparts earlyprintk=serial"'
setenv bootcmd_exec 'echo Booting Debian . . .; run usb_setup;  run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; run load_initrd; bootm $load_image_addr $load_initrd_addr'
setenv bootcmd 'run bootcmd_exec; run bootspi'
boot

5.a. Watch the serial console logging, the Debian prompt will appear when the box finishes booting. Login with root/root credential.

In Debian, check the new mtd partitions definition is as expected.

cat /proc/mtd 
dev: size erasesize name 
mtd0: 000e0000 00001000 "u-boot" 
mtd1: 00010000 00001000 "dtb" 
mtd2: 006f0000 00001000 "kernel" 
mtd3: 00020000 00001000 "env"

5.b. Flash uImage to mtd2

cd /boot
dd if=uImage of=rs816.uImage.mtd2.kwb bs=7104k conv=sync
flashcp -v rs816.uImage.mtd2.kwb /dev/mtd2

5.c. Reboot, interrupt serial console. And then set up u-boot envs permanently.

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;'                
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv mtdparts 'mtdparts=spi0.0:896k(u-boot),64k(dtb),7104k(kernel),-(env)' 
setenv boot_config_spi 'setenv image_addr_spi 0x0F0000; setenv image_size_spi 0x006f0000; sf probe 0 50000000' 
setenv load_image_spi 'echo loading uImage from flash ...; sf read $load_image_addr $image_addr_spi $image_size_spi' 
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=/dev/sda1 rootdelay=20 $mtdparts earlyprintk=serial"' 
setenv bootcmd_exec 'echo Booting Debian ...; run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run boot_config_spi; run load_image_spi; bootm $load_image_addr -' 
setenv bootcmd 'run bootcmd_exec; run bootspi' 
saveenv 
reset

Allow the box to boot automatically, and Enjoy!

6. Backup USB rootfs

To backup the USB rootfs follow the wiki tutorial Backup/Restore rootfs using tar command.

7. End section I

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 09/30/2019 01:43AM by bodhi.

@bodhi
No issues with amendments your version is clearer.

EDITED my previous post as incomplete

In the final boot envs in Step 5.c I had missed out

setenv load_image_addr 0x02000000

It didn't show up on my test run because its the default anyway.

Extra boot env's added that are not needed / used are

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv load_initrd_addr 0x3000000

Richard



Edited 1 time(s). Last edit at 09/30/2019 06:20AM by richjn.

@ bodhi
Draft IV. Recovery to stock Synology

DRAFT DO NOT FOLLOW UNTILL TRANSFERRED TO WIKI

Installation instructions for Synology RS816

IV. Recovery to stock Synology

Prerequisites:

a. All HDD drives must be removed from the RS816 while this recovery is performed.
b. tftp server is already set up in your network environment.
c. Serial console must be connected during installation.

1. Using the USB rootfs prepared in Section 1
Attach the USB rootfs to the RS816. Power up, interrupt serial console and at the u-boot prompt,

(192.168.1.19 is the IP address of the RS816, and 192.168.1.20 is the IP address of the tftp server, please substituing appropriate IP addresses for you own network environment).

1.a. Execute the following commands to set up u-boot envs

setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv mtdparts 'mtdparts=spi0.0:4m(boot),-(spi-rootfs)'
setenv usb_setup 'setenv usbActive 1; setenv usbType 3; usb start;'                
setenv load_initrd_addr 0x3000000
setenv load_image_addr 0x02000000
setenv load_image 'echo loading uImage from tftpserver ...; tftpboot $load_image_addr uImage'
setenv load_initrd 'echo loading uInitrd from tftpserver ...; tftpboot $load_initrd_addr uInitrd'
setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=LABEL=rootfs rootwait $mtdparts earlyprintk=serial"'
setenv bootcmd_exec 'echo Booting Debian . . .; run usb_setup;  run set_bootargs; setenv fdt_skip_update yes; setenv initrd_high 0xffffffff; run load_image; run load_initrd; bootm $load_image_addr $load_initrd_addr'
setenv bootcmd 'run bootcmd_exec; run bootspi'
boot

1.b. Watch the serial console logging, the Debian prompt will appear when the box finishes booting. Login with root/root credential.

2. In Debian, check the new mtd partitions definition is as expected.

cat /proc/mtd 
dev: size erasesize name 
mtd0: 00400000 00001000 "boot" 
mtd1: 00400000 00001000 "spi-rootfs"

3. Flash backup mtd partions

flashcp -v mtd0.stock.rs816.kwb /dev/mtd0
flashcp -v mtd1.stock.rs816.kwb /dev/mtd1

4. Reboot and interupt to u-boot

setenv mtdparts 'mtdparts=spi0.0:4m(boot),-(spi-rootfs)'
setenv bootcmd 'mw 0xf10a2620 0x251;mw 0xf10a2e20 0x251;usb xhciinit; run bootspi'
saveenv
reset

Your NAS is now discoverable by Synology discovery

EDIT
Decided not comfortable with resetenv, it worked here but only tested once, reverting to known good recovery method.



Edited 7 time(s). Last edit at 10/01/2019 05:37AM by richjn.

Hi Richard,

Good works :) I'm a bit busy right now, so still reviewing the Section II and III.

Section IV was nicely done! I have no comment.

====

One prerequisite (item c) that I forgot to added:

Prerequisites:

a. All HDD drives must be removed from the RS816 while this installation is performed.
b. tftp server is already set up in your network environment (we will use it for the kernel initial booting before it can be set up permanently in flash).
c. Serial console must be connected during installation.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 09/30/2019 05:23PM by bodhi.

@bodhi

I have added the extra prerequisite to Section IV but added baud as well I can remove that bit if you think its unnecessary.

c. Serial console must be connected during installation. (baud 115200)

No problem time wise I think we are in the home straight on this one :-)

I probably need to concentrate on my day job for the next few days as I'm getting long stares from work colleagues who feel I should be "pulling my weight" Nice!, lets have a show of hands for all those working over a public holiday weekend ........ Hmm why is my hand the only one up ?

Richard

Richard,

No need for serial com details. That is basic stuff one could find out easily.

Yes, please don't worry too much about this hobby :) unless somebody is having problem booting something (that needs my helps), I don't spend too much time each day in this forum.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Quote

setenv load_image_addr 0x02000000
It didn't show up on my test run because its the default anyway.

Extra boot env's added that are not needed / used are
setenv ipaddr 192.168.1.19
setenv serverip 192.168.1.20
setenv load_initrd_addr 0x3000000

They are needed for completeness. The 2 load addresses are really a reference to go back to when someone messes up their u-boot envs later.

The 2 IP addresses are quite important to have that set up in any network environment for future troubleshooting (when serial console is not available anymore).

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

@bodhi

Baud rate removed.

Not sure I understand how the IP addresses help if you have no serial console but I guess I don't need to know :-)

Richard

@bodhi

Between doing real work I have been thinking about this mdadm degraded issue as it bugs me not really knowing what the cause of it is. The current "workaround" was to introduce an otherwise unnecessary reboot and although it seemed to fix it I'm not really comfortable with it.

Up to the recent tutorial runs I have been using 2TB drives or SSD's for my own purposes but I didn't have any spare on the day so used 8TB 7,200rpm drives. We currently have a rootdelay of 20 seconds and with soc sata ports taking a while to come up and these larger / faster drives taking longer to spin up I'm just wondering if during my tests the real issue was there wasn't always enough time to bring the second sata port drives up.

Its a shame I'm in no position to check this out at the moment but perhaps the answer is to bump rootdelay up to 40 seconds, apart from another 20 seconds missing from peoples lives it wouldn't do any harm.

Thoughts ?

Richard

Richard,

> Not sure I understand how the IP addresses help if
> you have no serial console but I guess I don't
> need to know :-)

It is something everybody might be interested in :)

This is just one example of many about the usefulness of having the box network set up properly in u-boot (which the manufacterers don't think users need to know).

With the IP address defined correctly, you can tell if u-boot has booted to to point where it is visible in the network, and the entire boot progress could be guessed as it goes.

- Continuously pinging the box ipaddr, and then power up the box, if you receive positive response then you know u-boot is running and your ethernet is working.
- When the box stops responding then the kernel has started booting. And u-boot has terminated.
- When it starts responding again, you would see the box system LED turn solid green/white.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 2 time(s). Last edit at 10/01/2019 03:09AM by bodhi.

Richard,


> Its a shame I'm in no position to check this out
> at the moment but perhaps the answer is to bump
> rootdelay up to 40 seconds, apart from another 20
> seconds missing from peoples lives it wouldn't do
> any harm.
>

This is when you want to use rootwait (indefinitely) instead of rootdelay. The array might not coming up quickly enough.

setenv set_bootargs 'setenv bootargs "console=ttyS0,115200 root=/dev/md0 rootwait $mtdparts earlyprintk=serial"'

Use rootwait and a stop watch to see if the array coming up fully after how long.

In a system such as this (rootfs on RAID ) and you basically don't care to have the benefit if it drops to initramfs, use rootwait.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

@bodhi

>> It is something everybody might be interested in :)
Ah I see, interesting stuff! It's the real joy of "playing" with this stuff there is always something new to learn every day.

>Use rootwait and a stop watch to see if the array coming up fully after how long.
As soon as I can run a test I will see how long it takes but the more I think about it the more confident I am it was the issue all along.

I think dropping to a initramfs is generally very desirable so maybe we should go 40 or even 60 seconds rootdelay until I can get this confirmed ?

Richard

Richard,

> I think dropping to a initramfs is generally very
> desirable so maybe we should go 40 or even 60
> seconds rootdelay until I can get this confirmed
> ?

Dropping to initramfs is useful if you have console access and can see that it stopped there.

However, most of the time, the most useful thing that you can do is fixing a corrupted rootfs that has not been possible to be fixed automatically (fstab option). Other than that, you would just do a reboot and hope that it will boot successfully. For some rootfs that has initramfs modified to do some fancy stuff then dropping to it is more useful.

I did not suggest of this option before (probably not paying attention), but I would prefer rootwait for RAID array, and for SD card rootfs, too. Unless you know you can do more in initramfs.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

@bodhi

>However, most of the time, the most useful thing that you can do is fixing a corrupted rootfs

Fair point, I guess there is serial over ethernet or just using a laptop with a USB serial adaptor to connect locally but overall I think you are correct, you are already having a bad day by this point so may as well rip it out of the rack to sort out in comfort somewhere.

Agreed lets go rootwait for RAID & SD. I will change Section IV now but leave the others as you are working on those.

Richard

@bodhi

Actually I'm not sure rootwait is the right answer for RAID because the rootfs WILL be seen as soon as its partially assembled i.e. the kernel will continue even on a degraded array.

I will see if I can "engineer" some time on the RS816 I think we need to do actual tests before deciding.

Richard

Richard,

> >> It is something everybody might be interested
> in :)
> Ah I see, interesting stuff! It's the real joy of
> "playing" with this stuff there is always
> something new to learn every day.

That's just one of many capabilities that you can have with a proper network setup in u-boot.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

@bodhi

Testing of rootwait

richjn Wrote:
-------------------------------------------------------
> I will see if I can "engineer" some time on the RS816

I got a short period of time to test and found the following:

1) Sata port A (sda & sdb) start to spin up almost immediately

2) Sata port B (sdc & sdd) start to spin up after ~8 - 10 seconds

3) With SSD's & small HDD's our current 20 second rootdelay is sufficient but on larger drives it takes ~24 - 34 seconds (based on the four runs from power off)

4) In a reboot scenario where the drives are still spinning a bit the spin up dalay was down to ! 17 - 25 seconds (based on four runs from reboot)

5) I tried inserting a "scisi init" to bootcmd but although it delayed u-boot a bit it made no difference to the spin up times.

6) Running with rootwait results in virtually no boot pause because we are using raid1 and as soon as any member of the raid is up boot continues.

7) Once boot gets to rootfs seen the drives which have not spun up seem to be just marked as 'missing' and need to be manually added once boot is complete. (I would need to dig into Debian config files to confirm this but it does appear to be the case from these trials and previous observations with 8TB 7200rpm drives)

Bottom line is it appears rootwait wont work for us on raid although I think it still makes sense for USB or SD on other boxes which have that.

I think to allow for slow drives we should just play safe and go for rootdalay=40 for RAID

On a plus note the apparent issue with mdadm not being in rootfs has been confirmed as not valid.


Richard