Welcome! Log In Create A New Profile

Advanced

Debian Stretch on Sata disk in WD MyCloud EX2 Ultra

Posted by LinAdmin 
Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 06, 2018 06:40AM
In this forum I have found several interesting threads about modifications of this NAS.

I have an USB stick attached where initrd and kernel 4.9.y are stored and a serial cable connected where I can boot from USB with the rest of Debian in a file system on Disk.

This system is pretty small and has excellent benchmarks for reading and writing via samba and nfs. The small footprint leaves enough RAM for efficient disk caching without need for swap.

Now, I would like to get a permanent solution where initrd and kernel no longer are on USB but on a small boot partition on disk.

Unfortunately the original uBoot from WD seems not to support Sata and also lacks saveenv which in the long term might be useful.

Therefore I would like to generate a new uBoot which would read kernel and initrd from bootP on sda1. This could be tested first using kwboot and then flashed once into mtd0: (An even more sophisticated approach would also change the primary boot order and other $bootargs).

This brings the important advantage that future updates of the LTS kernel 4.9.xx which is supported for many years can be installed without any further writing to the flash memory.
Do you think that this approach is possible and how should I proceed?
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 06, 2018 02:08PM
LinAdmin,

> Unfortunately the original uBoot from WD seems not
> to support Sata and also lacks saveenv which in
> the long term might be useful.
>
> Therefore I would like to generate a new uBoot
> which would read kernel and initrd from bootP on
> sda1. This could be tested first using kwboot and
> then flashed once into mtd0: (An even more
> sophisticated approach would also change the
> primary boot order and other $bootargs).

> Do you think that this approach is possible and
> how should I proceed?

It is possible. The quickest way to get there is to rebuild stock u-boot to enable saveenv (GPL is available from WD). As for SATA, it is most likely that it's just not powered up by stock WD u-boot. Not because it is not supported. This can be solved by poking the GPIO with u-boot mw.

So I think all it'd take is getting the GPL, change a few lines of code, rebuild, kwboot it, and flash to mtd0.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 07, 2018 11:35AM
Many thanks bodhi for your motivating reply.

I built u-boot from latest WDMyCloud_Ex2Ultra_GPL_v2.30.193_20180502 running xbuild.
This produces the file u-boot-a38x-GrandTeton_2014T3_PQ-nand-uart_512M which I renamed to u-boot.bin.

The MyColud EX2 got one disk partitioned with fdisk holding a small bootP with stored uImage and initrd as used before on the USB stick.

When giving the commands at the console to init scsi and load both files and then booting: Heureka it booted!

However, the command "scsi init" also produced the unexpected error message "Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found". This 1B4B address can be found in cmd_scsi.c (line 58) probably as a default and in armada_38x.h (line 249) in a multiple list.

Do you have an idea what must be modified to get rid of that error message?

I have two 8 TB red disks to be used and when i formatted them using gdisk and GPT, then "scsi init "after the expected output
Error: SCSI Controller(s) 1B4B:9215 1B4B:9235 not found
scanning bus for devices...
the uboot hangs forever. In armada_38x.h there is
#define CONFIG_SYS_64BIT_LBA	/* Support disk over 2TB */
#define CONFIG_LBA48
so I hoped that GPT would be supported?
What must I do in order to use the 8TB disks?

Looking forward to your advice.
LinAdmin



Edited 1 time(s). Last edit at 09/07/2018 11:56AM by LinAdmin.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 07, 2018 01:43PM
LinAdmin,

> However, the command "scsi init" also produced the
> unexpected error message "Error: SCSI
> Controller(s) 1B4B:9215 1B4B:9235 not found".

We've always seen this error during boot. But it never seem to affect the disk spin up. Did you issue a mw command to power up SATA?

For example, on the NAS326 with u-boot
uboot_version=U-Boot 2013.01-svn48797 (Jan 18 2016 - 03:10:12) Marvell version: 2014_T3.0p6

We needed to do this (one of the 2 commands powered up SATA, but I don't remember which one).
setenv sata_init 'mw.l f1018100 20420000; mw.l f1018140 003ca800; sleep 3; scsi init'

Note that the GPIO base addresss such as f1018100 and f1018140 are the same for all these Armada boxes, but the GPIO bits might be different between Zyxel and WD implementations.

And BTW, I recall that these u-boot versions do support GPT.

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 1 time(s). Last edit at 09/07/2018 01:50PM by bodhi.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 08, 2018 02:15AM
I have further investigated the problem:

When u-boot waits on the serial console, the sata hardware is already powered up without giving any 'mw' command.

However, 'scsi init' some times reports the disk id's with erroneous texts and and only after 'scsi reset' it eventually brings the correct text. After that, booting the small disk with partition table in mbr works as expected.

I then ran 'gdisk /dev/sda' on that disk and it reported the correct partitioning in mbr. Pressing 'w' wrote the GPT tables on that disk saying that it kept a protective mbr. Starting again gdisk and 'fdisk -l' both reported the GPT partitioning.

After that I again entered 'scsi init' and the command hung forever, the same way as it was before with the 8 TB disk initially partitioned with GPT without protective mbr.

Therefore I come to the conclusion that this version of u-boot does not support GPT.

(WD avoids this problem by storing kernel and initrd in flash ram. When the kernel runs the support for GPT is loaded.)

I am really disappointed that so close at the goal there seems to be a huge problem. What can be done?

PS: I have now verified in the code: include/part.h does not know of GPT.



Edited 3 time(s). Last edit at 09/10/2018 11:57AM by LinAdmin.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 08, 2018 02:15PM
LinAdmin,

I'm not convinced that this u-boot can't handle GPT. From what you said, this u-boot is the same as other WD MyCould and Zyxel Armada boxes. It is circa 2013 and I don't recall GPT was a problem.

Do you have a small HDD that you can format with a single GPT partition and test it? boot with USB and try to spin it up at serial console.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 08, 2018 03:28PM
I had described before that an old (750 G) disk partitioned with fdisk did work as excepted and when the partitioning was changed to GPT the 'scsi init' with the same disk hung. Other disks (small and up to 8 TB) partitioned with GPT show the same behavior.

As said before, in part.h of WD u-boot there is no sign of GPT code visible. GPT is mentioned only in the context of efi and I do not think that this helps booting a GPT disk with scsi.

PS1: I have just now learned about hybrid GPT and fully hope that this will work in my EX2...

PS2: I was able to create a hybrid partition. When the MBR table has an additional protective entry, u-boot command 'scsi init' hangs. When I remove that 'unnecessary' protective entry, booting works. However the linux system only uses the MBR information and the last big partition in the GPT table is not used. Proof: partprobe -s /dev/sda only shows MBR entries.

So the question remains: How to create a hybrid partitioning that satifyes u-boot but still forces the kernel to use the GPT data???

PS3: @bodhi: With a 750G disk partitioned with one GPT entry 'scsi init' hangs.



Edited 5 time(s). Last edit at 09/10/2018 12:00PM by LinAdmin.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 10, 2018 03:56AM
Conclusion about this u-boot and GPT:

a) With GPT this u-boot does not work at all.

b) With hybrid GPT, but without protective entry in the MBR table, this u-boot works. Linux however does not see the GPT and we need a trick to force Linux to use GPT although the protective entry in MBR table is missing. I did not find any such hints, but I still could imagine that some magic in the boot args or in initrd might be able to help.

c) With hybrid GPT including a protective entry in the MBR table, this u-boot does not work. We would need a trick to get it working and I have not yet tested if Linux would use GPT with such a disk.

(An intriguing side mark: WD MyCloud Gen 2 with single disk has one GPT disk with rootFS and boot directory in GPT sda3 and ext4 files system. Unfortunately they violate GPL and do not distribute source code of that u-boot.)
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 10, 2018 01:46PM
LinAdmin,

> PS3: @bodhi: With a 750G disk partitioned with one
> GPT entry 'scsi init' hangs.

You meant you wiped out the disk with parted, and create 1 GPT partition? i.e. no protective MBR remained?

Perhaps this u-boot version is different from other WD NAS boxes u-boot.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 10, 2018 03:13PM
bodhi, the MBR of that disk was filled with zeros to make sure nothing was left from hybrid partitioning and then using gdisk one partition created.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 10, 2018 03:42PM
LinAdmin Wrote:
-------------------------------------------------------
> bodhi, the MBR of that disk was filled with zeros
> to make sure nothing was left from hybrid
> partitioning and then using gdisk one partition
> created.

Have you tried parted/gparted?

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 11, 2018 06:43AM
I have wiped the disk and using gparted created a single GPT-partition.
It would have been too surprising if 'scsi init' would not hang as with every GPT :-(

Questions regarding Zyxel NAS326:

- With their u-boot and without anything on USB, what does 'scsi init' with 1 and 2 disk(s) partitioned with several GPT entries show?

- If their u-boot does correctly handle it, where can I download sources of their u-boot (Zyxel seems very complicated about that?)

- If have red something about a timeout value having been patched from 3 to 200 ms in order to allow detecting 2 sata disks?

Thx!!
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 11, 2018 03:09PM
LinAdmin,

> Questions regarding Zyxel NAS326:
>
> - With their u-boot and without anything on USB,
> what does 'scsi init' with 1 and 2 disk(s)
> partitioned with several GPT entries show?

I can't recall whether I have tested this in the NAS326. When I have a chance (found a spare drive) I will test it.

>
> - If their u-boot does correctly handle it, where
> can I download sources of their u-boot (Zyxel
> seems very complicated about that?)

Not really. The actually responded to my email request within a couple days. They just need the serial number to make sure you actually own the box. If you still have problem getting GPL sources, send me a PM.

> - If have red something about a timeout value
> having been patched from 3 to 200 ms in order to
> allow detecting 2 sata disks?

Yes, I've found that reading the GPL code, when some users had problem booting with 2 HDDs. These Marvell u-boot versions (all of these boxes u-boot derived from the same uboot version) use an old ACHI driver which sets the timeout at 3ms. The current (2017 ish) u-boot AHCI driver uses 200ms.

In the mean time, you could keep the rootfs on a GPT drive, and load the kernel from USB. I know it is a workaround only, but it is good work around when you will have solved this problem, you only need to adjust u-boot envs to boot completley from HDD.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 26, 2018 06:18AM
Hybrid partitioning did not succeed and I have given up. So the question remains how WD MyCloud Gen2 (single drive) with only a small flash storage can boot with a GPT disk. In the GPL source from WD u-boot is not included, but I can not imagine how the Grub source included should do the job?

So it still would be very helpful to know if NAS326 can boot on GPT?
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 26, 2018 09:43AM
LinAdmin Wrote:
-------------------------------------------------------
> This system is pretty small and has excellent benchmarks for reading and writing via samba and nfs.
>
I am interested to know what kind of excellent benchmarks performance you get from this setup.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 26, 2018 12:36PM
LinAdmin,

> So the question remains how WD MyCloud
> Gen2 (single drive) with only a small flash
> storage can boot with a GPT disk.

> but I can not imagine how the Grub source included should do the job?

Usually very simple. The kernel files are stored in flash. After booted, U-Boot (not Grub) stars the kernel. The kernel mounts the rootfs from the HDD. Similarly, a USB drive can be set up to store kernel files only and u-boot will boot from there.

Power up interrupt the count down and,

printenv


> So it still would be very helpful to know if
> NAS326 can boot on GPT?

Will see if I can.

-bodhi
===========================
Forum Wiki
bodhi's corner



Edited 1 time(s). Last edit at 09/26/2018 04:05PM by bodhi.
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
September 27, 2018 10:54PM
LinAdmin ,

> > So it still would be very helpful to know if
> > NAS326 can boot on GPT?

I've looked, but I don't have any spare HDD right now. So I can't check that for you.

-bodhi
===========================
Forum Wiki
bodhi's corner
Re: Debian Stretch on Sata disk in WD MyCloud EX2 Ultra
October 21, 2018 11:51AM
I finally found a work around for using the 8 TB disk with the old u-boot which does not understand GPT:

Normally, when a disk is partitioned with a GPT table, the first entry in the MBR table gets a so called "protective entry" allocating the maximum space of 2 TB of the old addressing scheme as a type "0xec". This entry also instructs Linux to use GPT instead of MBR partition information.

For some special cases gdisk does allow to have so called "hybrid partitioning": You first create a new GPT with all your smaller partitions below the 2 TB limit and as last entries the partitions above 2 TB. After that using the 'recovery menu' -> 'hybrid entries' you report max. 3 partitions to the MBR and in the last entry again type "0xec".

This last entry is important to instruct Linux to use the GPT, but it also results in u-boot not to accept the partitioning of such disks :(

By patching the code of u-boot I managed to get it discarding that protective entry and using all MBR entries in front of it.

That new u-boot with save_env enabled allows to have initrd and root file system on the first disk partitions.
The Debian kernel armmp from stretch works as expected when the module sdhci_pxav3 is blacklisted. This is necessary in order to avoid unnecessary high CPU load when idle.

If anybody is interested in the patches I can download them. In a later phase I still hope to find a suitable u-boot that understands GPT!
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: