Welcome! Log In Create A New Profile

Advanced

Debian on WD MyCloud EX2100

Posted by hmartin 
Debian on WD MyCloud EX2100
August 07, 2017 02:30PM
I thought I would finally get around to permanently using Debian on my EX2100. This means fixing up the DTB to support the dual gigabit NICs, and getting the mcm-daemon running to make the fan quiet.

Unfortunately, I cannot get either to work! Here the is networking config on the EX2100:
Load WD Yosemite Serdes Config:
board SerDes lanes topology details:
 | Lane #  | Speed |  Type       |
 --------------------------------
 |   0    |  06   |  SATA0	|
 |   1    |  05   |  PCIe0	|
 |   2    |  06   |  SATA1	|
 |   3    |  05   |  USB3 HOST1	|
 |   4    |  05   |  USB3 HOST0	|
 |   5    |  00   |  SGMII2	|
 --------------------------------
PCIe, Idx 0: detected no link
High speed PHY - Ended Successfully
...(later)
Net:   
|  port  | Interface | PHY address  |
|--------|-----------|--------------|
| egiga0 |   RGMII   |     0x00     |
| egiga1 |   RGMII   |   In-Band    |
| egiga2 |   SGMII   |     0x01     |

The EX2U DTB posted here works with the SGMII NIC, but neither RGMII NIC works if you change the status from "<disabled>" to "<okay>":

send_packet: Network is down
dhclient.c:2255: Failed to send 300 byte long packet over eth1 interface.
receive_packet failed on eth1: Network is down

Also, it seems that the MCU is not connected via serial@12100? mcm-daemon doesn't work at all:
# /usr/bin/mcm-daemon -f -c /etc/mcm-daemon.ini 
mcm-daemon[2737]: Server startup success on port 57367
mcm-daemon[2737]: Got no stop magic, but read 1 bytes!
mcm-daemon[2737]: Got no stop magic, but read 1 bytes!
mcm-daemon[2737]: Got no stop magic, but read 1 bytes!
mcm-daemon[2737]: Got no stop magic, but read 1 bytes!
mcm-daemon[2737]: Error sending DeviceReady command, exit!

Doesn't really make much sense, since it seems that WD are writing to serial 1 in u-boot to communicate with the MCU.

board/mv_ebu/a38x/mv_main_a38x.c:
        printf("Enable HD1\n");
        serial2_putc(0xfa);
        serial2_putc(0x6);
        serial2_putc(0x1);
        serial2_putc(0x0);
        serial2_putc(0x1);
        serial2_putc(0x1);
        serial2_putc(0xfb);

board/mv_ebu/common/USP/mv_serial.c:
#if 1   //ALPHA_CUSTOMIZE
void serial2_putc(const char c)
{
#if defined(CONFIG_MV_SMP) || (defined(MV78XX0) && defined(MV78200))
        if (c == '\n')
                mvUartPutc((whoAmI())%2, '\r');

        mvUartPutc((whoAmI())%2, c);
#else
        if (c == '\n')
                mvUartPutc(1, '\r');

        mvUartPutc(1, c);
#endif
}
#endif

I've confirmed that uart 0 is connected to the debug header on the board, writing to that kills your uart session. It seems that uart 1 might not be connected to the MCU, or the MCU isn't responding?

@bodhi: any ideas about the above?

I've also re-built u-boot to save environment to 0x100000 and to not overwrite bootcmd on boot as WD's source so kindly does. Find the kwboot and nand versions attached.

Edit: looks like my mcm-daemon woes might be because the MCU has lost its firmware :-(

Even the stock WD firmware cannot communicate to it:
root@WDMyCloudEX2100 / # up_send_ctl PowerStatus
root@WDMyCloudEX2100 / # up_send_ctl MCUVer

That would explain why the power LED is always flashing blue and the fan never shuts up even in the stock firmware.

Does anyone know how to update/restore the MCU firmware?



Edited 2 time(s). Last edit at 08/07/2017 02:41PM by hmartin.
Attachments:
open | download - u-boot-a38x-Yosemite_2014T3_PQ.tar.gz (919.2 KB)
Re: Debian on WD MyCloud EX2100
August 07, 2017 02:57PM
Interesting. Perhaps the commands for this MCU are different from the ones noted for the mcm-daemon.

I can't explain why it isn't working quite as expected, but I modified u-boot with the following commands just after "Enable HD1" and "Enable HD2":
  printf("Make Power LED flash orange\n");
  serial2_putc(0xfa);
  serial2_putc(0x26);
  serial2_putc(0x00);
  serial2_putc(0x22);
  serial2_putc(0x00);
  serial2_putc(0x01);
  serial2_putc(0xfb);
  for ( i=0;i<5;i++)
                udelay(1000000);

  printf("Make the fan shut up\n");
  serial2_putc(0xfa);
  serial2_putc(0x02);
  serial2_putc(0x00);
  serial2_putc(0x50);
  serial2_putc(0x00);
  serial2_putc(0x00);
  serial2_putc(0xfb);
  for ( i=0;i<5;i++)
                udelay(1000000);

The power button light does not change colour, but the fan does shut up. So it seems the MCU is alive after all...

Also bodhi here is the relevant part about NICs during the WD firmware boot:
port #0: is_sgmii=0, is_rgmii=1, phy_addr=0
  o Loading network interface(s) for port #0: cpu_mask=0x3, tx_csum_limit=9800

  o Port 0 is connected to Linux netdevice
    o egiga0, ifindex = 2, GbE port = 0
	giga p=0: mtu=1500, mac=00:50:43:XX:XX:XX (platform)

port #2: is_sgmii=1, is_rgmii=0, phy_addr=1
  o Loading network interface(s) for port #2: cpu_mask=0x3, tx_csum_limit=2048

  o Port 2 is connected to Linux netdevice
    o egiga1, ifindex = 3, GbE port = 2
	giga p=2: mtu=1500, mac=00:50:43:XX:XX:XX (platform)



Edited 1 time(s). Last edit at 08/07/2017 02:59PM by hmartin.
Re: Debian on WD MyCloud EX2100
August 07, 2017 05:40PM
Re: Debian on WD MyCloud EX2100
August 07, 2017 11:40PM
Thanks for the tips, I think there is something wrong with my u-boot build for EX2100 though. It seems that Linux cannot communicate with the MCU.

The Western Digital firmware is not correctly reading the system temperature, cannot set the fan speed, and does not properly shut off:
sysinfod: Disk2(sda) detected.
sysinfod: inform: Disk1 failed on.
sysinfod: inform: Disk1 non-healthy on.
sysinfod: inform: all drives status=Fault.
UBIFS: un-mount UBI device 0, volume 0
UBIFS: background thread "ubifs_bgt0_0" stops
MODEL:WDMyCloudEX2100
egiga0: link down
egiga0: link up, full duplex, speed 1 Gbps
egiga1: link down
egiga1: link up, full duplex, speed 1 Gbps
send cmd to micro-p to shutdown
The system is going down NOW!
Sent SIGTERM to all processes
Requesting system poweroff
sd 0:0:0:0: [sda] Synchronizing SCSI cache
sd 0:0:0:0: [sda] Stopping disk
xhci-hcd f10f8000.usb3: remove, state 1
usb usb5: USB disconnect, device number 1
xhci-hcd f10f8000.usb3: USB bus 5 deregistered
xhci-hcd f10f8000.usb3: remove, state 1
usb usb4: USB disconnect, device number 1
xhci-hcd f10f8000.usb3: USB bus 4 deregistered
xhci-hcd f10f0000.usb3: remove, state 1
usb usb3: USB disconnect, device number 1
xhci-hcd f10f0000.usb3: USB bus 3 deregistered
xhci-hcd f10f0000.usb3: remove, state 1
usb usb2: USB disconnect, device number 1
xhci-hcd f10f0000.usb3: USB bus 2 deregistered
Shutting Down Marvell Ethernet Driver
Shutting Down Marvell Ethernet Driver
System halted.

The unit does not power off. Power light stays flashing, fan at full blast, but the WD firmware is off.

I could imagine the DTB for the EX2U being incorrect and leading to problems on the EX2100, but the fact that the Western Digital firmware is not working properly anymore rules that out. I can only guess that there is some issue in the u-boot sources Western Digital has provided, which prevents the OS from communicating with the MCU.

Unfortunately I was an idiot and lost the backup of the original u-boot that was on my unit, so I cannot restore it. But I do recall that the fan used to quiet down when I booted into WD's firmware, and around the time I started kwbooting this behaviour stopped.

I will have to dig more into my modifications to u-boot to see if I have somehow managed to mess up MCU communications. The strange thing is that it works from within u-boot, I can make the fan run at 50% speed by sending the command to the MCU via serial2_putc()
Re: Debian on WD MyCloud EX2100
August 08, 2017 02:43AM
hmartin,

> lost the backup o
> f the original u-boot that was on my unit, so I ca
> nnot restore it.

Ouch! I guess you can get it from the WD FW release tarball and flash that version back. And this u-boot has a USB restore mechanism. What I got from reading the code: If you put the FW release in USB and press a button (not sure where) during boot, it will restore the OS to stock. I don't know what's in that FW release blob, whether it also upgrades u-boot.

And it is not unsual for the GPL to be different source than the one they flashed in the FW release. They are not legally obligated to release the same exact source code GPL as the binary they install in the FW. I've seen this before.


> I will have to dig more into my modifications to u
> -boot to see if I have somehow managed to mess up
> MCU communications. The strange thing is that it w
> orks from within u-boot, I can make the fan run at
> 50% speed by sending the command to the MCU via se
> rial2_putc()

I think, perhaps it means that we are not done with the DTS. It probably does not have enough to set the box MCU to a default state when the kernel comes up.

-bodhi
===========================
Wiki
latest Kirkwood kernel builds and rootfs
latest u-boot-kirkwood builds
latest Oxnas kernel builds and rootfs
latest u-boot-oxnas builds
latest MVEBU Armada kernel builds and rootfs
U-Boot & Kernel Booting process
bodhi's u-boot GitHub
bodhi's corner
Re: Debian on WD MyCloud EX2100
August 08, 2017 10:45AM
Quote
bodhi
And it is not unsual for the GPL to be different source than the one they flashed in the FW release. They are not legally obligated to release the same exact source code GPL as the binary they install in the FW. I've seen this before.

Really? I thought the GPL required them to release the source code of the firmware version running on the device. Otherwise what's the point?

Quote
bodhi
I think, perhaps it means that we are not done with the DTS. It probably does not have enough to set the box MCU to a default state when the kernel comes up.

I had that thought as well, but it doesn't explain why the stock firmware running from flash also cannot communicate with the MCU anymore. If it was an issue with the DTS, you would expect it to function in the stock firmware but not in Debian. This is sadly not the case.
Re: Debian on WD MyCloud EX2100
August 08, 2017 11:38AM
There are some additional commands in the u-boot source which are not documented in the mcm-daemon.

Here they are (transferred into the format of mcm.h):
char EnableHd1Cmd[]		= {0xfa, 0x06, 0x01, 0x00, 0x01, 0x01, 0xfb};
char EnableHd2Cmd[]		= {0xfa, 0x06, 0x01, 0x01, 0x01, 0x01, 0xfb};

There is a strange comment about powering off the NAS ("power off nas") which I don't understand why this would be in u-boot and definitely does not result in the NAS powering off...
char PwrOffCmd[]		= {0xfa, 0x03, 0x11, 0x00, 0x00, 0x00, 0xfb};



Edited 1 time(s). Last edit at 08/08/2017 11:50AM by hmartin.
Re: Debian on WD MyCloud EX2100
August 08, 2017 01:32PM
I fixed the EX2100 dts to work with both onboard Ethernet interfaces! MCU communications still broken in Linux.
Attachments:
open | download - armada-385-wd-ex2100.tar.gz (5.8 KB)
Re: Debian on WD MyCloud EX2100
August 08, 2017 03:43PM
hmartin,

Quote

I fixed the EX2100 dts to work with both onboard Ethernet interfaces!

Cool! I think this should be applied to the EX4100 too.

Quote

Really? I thought the GPL required them to release the source code of the firmware version running on the device. Otherwise what's the point?

Only the Linux kernel needs to be. They have a whole bunch other userland binaries running in stock OS that are important, but those are not required to provide source code. They are treated like graphics binary blobs that we have in the kernel.

Quote

I had that thought as well, but it doesn't explain why the stock firmware running from flash also cannot communicate with the MCU anymore. If it was an issue with the DTS, you would expect it to function in the stock firmware but not in Debian. This is sadly not the case.

I think it means some MCU states are persistent. So even stock OS did not reset it correctly. Have you tried removing the RTC battery?

-bodhi
===========================
Wiki
latest Kirkwood kernel builds and rootfs
latest u-boot-kirkwood builds
latest Oxnas kernel builds and rootfs
latest u-boot-oxnas builds
latest MVEBU Armada kernel builds and rootfs
U-Boot & Kernel Booting process
bodhi's u-boot GitHub
bodhi's corner
Re: Debian on WD MyCloud EX2100
August 09, 2017 10:44AM
Quote
bodhi
I think it means some MCU states are persistent. So even stock OS did not reset it correctly. Have you tried removing the RTC battery?

I have not removed the RTC battery yet. As far as I know it's buried somewhere deep inside the EX2100, which I haven't disassembled.

I'm not convinced this is an issue with the MCU, given that u-boot is still able to successfully communicate with the MCU to change the fan speed and power up the hard drives.
Re: Debian on WD MyCloud EX2100
August 13, 2017 05:58AM
Quote
hmartin
I have not removed the RTC battery yet. As far as I know it's buried somewhere deep inside the EX2100, which I haven't disassembled.

I was able to remove the RTC battery. It doesn't make any difference. The issue is not that the MCU is stuck in some invalid state.
Re: Debian on WD MyCloud EX2100
October 01, 2017 08:38AM
I have more details to add. I found the uart debug header of the Welltrend microcontroller. It's JP3 on the EX2100.

The pinout is as follows (pin 1 is facing the front of the device, toward the RTC coin cell):
1: Tx (connect to uart Rx)
2: NC
3: VCC?
4: GND
5: Rx? (doesn't seem to listen)

The UART baud rate seems to be 115200.

Here is the output through a stock boot to WD firmware:
Yosemite Uart test
nick 2222
nick pwr on
Check_SYSTEM_Command=11
PwrOnCause=10
Check_SATA_POWER_Command=0001
Check_SATA_POWER_Command=0101
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16
Check_SYSTEM_Command=16

Unfortunately, commands inside the WD firmware still don't work after boot. BUT if you run up_modify_baudrate again (it's already run during boot by the script hardware_init.sh) then all of a sudden lots of things start happening!

Check_SYSTEM_Command=16
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=05
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
 1 read SPI=01
 1 read SPI=01
 1 read SPI=01
 1 read SPI=01
 1 read SPI=01
 1 read SPI=01
 1 read SPI=01
 1 read SPI=01
-------------FF
Check_SATA_STATUS=0F
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=02
Check_SYSTEM_Command=08
TemperatureValue_MAIN=6A
Check_SYSTEM_Command=01
Check_SYSTEM_Command=05
Check_SYSTEM_Command=06
POWER_LED=0100
Check_SYSTEM_Command=06
POWER_LED=0000
Check_SYSTEM_Command=06
POWER_LED=0201
Check_SYSTEM_Command=06
POWER_LED=0100

The power LED changes colour and parts of fan_control work (like reading the system temperature):
root@WDMyCloudEX2100 / # fan_control -g 0
Current temperature is 34
hd0 temperature=0
hd1 temperature=0
CPU temperature=63

Unfortunately, there is no source code published for up_modify_baudrate. It's provided as a binary by Western Digital. Additionally, it seems to be armel so I cannot run it on bodhi's Debian firmware (which is armhf):
up_modify_baudrate: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 2.6.16, stripped

I don't have IDA Pro, so I cannot disassemble ARM binaries. I don't know if this binary is only changing the baudrate of ttyS1, or if it is also sending a special command to the microcontroller. Once the baudrate is changed, I believe up_read_daemon and up_send_daemon handle communication with the microcontroller.

So it seems the issue is not within u-boot after all, which is great because WD's u-boot source is really terrible to work on.

If someone can figure out what up_modify_baudrate does, maybe we can get the microcontroller supported in Debian too.



Edited 1 time(s). Last edit at 10/01/2017 08:41AM by hmartin.
Attachments:
open | download - up_modify_baudrate.gz (2.1 KB)
Re: Debian on WD MyCloud EX2100
October 01, 2017 08:45AM
Oh also I noticed that the assignment of ethernet ports in the dts are reversed from what is written on the back.

eth0: port labelled "2" (bottom port)
eth1: port labelled "1" (top port)

Sorry for any confusion this caused people.
Author:

Your Email:


Subject:


Spam prevention:
Please, enter the code that you see below in the input field. This is for blocking bots that try to post this form automatically. If the code is hard to read, then just try to guess it right. If you enter the wrong code, a new image is created and you get another chance to enter it right.
Message: