Howto: Setup a LUKS encrypted root file system on GoFlex/Dockstar and unlock it during the boot process via SSH

Here's a howto on using a LUKS encrypted Debian rootfs on a GoFlex/Dockstar. The rootfs is unlocked on boot by a password and that you must supply via SSH.

The working principle in a nutshell: We have a small (200 MB) unencrypted partition that contains the kernel (uImage) and initramfs (uInitrd). The root partition itself is LUKS encrypted using aes-cbc-essiv:256. The initramfs starts a dropbear SSH server that can accept the LUKS password and use it to open the root partition. After this the boot process continues as usual.

Advantages:

• If someone steals your device, this person will not be able to acess the data on it.
• Embrace your inner paranoiac ;)

Disadvantages:

• Even though nobody can access rootfs without knowing the password, it is perfectly possible to compromise the unencrypted boot partition and sniff the LUKS password. You can prevent this by creating MD5 sums of all the files on the boot partition, storing them in a safe place (e.g. another machine) and comparing those sums against the content of the boot partition every time you reboot the system.
• Encryption costs performance. The consequences of those speed penalties heavily depend on what you use your GoFlex/Dockstar for. If it has to serve or recieve big files (over 1GB) the slowness might be really annoying. On the other hand, certain tasks like mail server, print server, or UPNP server for music streaming don't require that much IO performance.
• If in the area where you live power blackouts are rather common, you will have to unlock your device everytime it restarts. For a Dockstar/GoFlex used as a VPN-server or personal cloud storage, this can be very undesirable.

Let's get to business. I will describe how to migrate a GoFlex/Dockstar with an unencrypted Debian squeeze image to LUKS encrypted root partition without data loss. Here I assume that you use a patched Debian 3.0.0 kernel from my other howto. You might try to get this working with other kernels, too. You will need a Linux machine with cryptsetup installed and an additional USB flash drive. I'll refer to it as "auxiliary flash drive". The USB flash drive that you use in you GoFlex/Dockstar will be called "original flash drive".

First we need to create a backup of the original flash drive. For that, power down your device and plug the original flash drive to your Linux system (let's call it /dev/sdb). Mount the rootfs (in my case it's /dev/sdb1) and create a tar backup file.

sudo -s
cd ~
mkdir /media/original_rootfs
mount /dev/sdb1 /media/original_rootfs
cd /media/original_rootfs
tar --one-file-system -cpf ~/original_rootfs.tar *

Connect the auxiliary flash drive and (let's call it /dev/sdc) and run gparted to create a single ext3 partition (/dev/sdc) on it. Mount that partition and copy the content of the original flash drive onto it.

mkdir /media/auxiliary_rootfs
mount /dev/sdc1 /media/auxiliary_rootfs
cd /media/auxiliary_rootfs
tar -xpf ~/original_rootfs.tar
cd ~
umount /media/auxiliary_rootfs
umount /media/original_rootfs
tune2fs -L "rootfs" /dev/sdc1

Disconnect the auxiliary flash drive and use it to boot your GoFlex/Dockstar. It should boot flawlessly. At this point we're going to create encrypted rootfs using our Linux system. But first we need to wipe the original flash drive properly. Open gparted, delete all partitions on the original flash drive and create a single ext3 partition (/dev/sdb1).
Use shred to make sure that no valuable data can be recovered from the original flash drive.

shred -v -n3 /dev/sdb1

This will take some time. After this is done, open gparted once again and use the following schema for creating partitions

/dev/sdb1: An ext3 partition about 200MB large
/dev/sdb2: An ext4 partiton that will become our encrypted rootfs. Needs to be at least 2-3GB
/dev/sdb3: A swap partition. The experience shows that 512MB are more than enough

The rest can be a separate data partition or you can simply make your encrypted rootfs as large as possible. Let's encrypt /dev/sdb2 and make it an ext4 partition

cryptsetup -c aes-cbc-essiv:sha256 -y -s 256 luksFormat /dev/sdb2
cryptsetup luksOpen /dev/sdb2 rootfs_crypt
mkfs.ext4 /dev/mapper/rootfs_crypt
tune2fs -L "rootfs" /dev/mapper/rootfs_crypt
cryptsetup luksClose /dev/mapper/rootfs_crypt

Disconnect the original flash drive and connect it to your GoFlex/Dockstar that was booted using the auxiliary flash drive.
Now ssh into your device and install the necessary packages:

apt-get install cryptsetup busybox dropbear

Be sure to save the id_rsa key as you will need it to unlock the partition via ssh.

cat /etc/initramfs-tools/root/.ssh/id_rsa

I just pasted the output to "/home/my_usual_username/.ssh/myserver_rsa" on my Linux laptop that I use to unlock my GoFlex

nano /home/my_usual_username/.ssh/myserver_rsa
-> paste the key, save and exit
chmod 600 /home/my_usual_username/.ssh/myserver_rsa
chown my_usual_username:my_usual_username /home/my_usual_username/.ssh/myserver_rsa
touch /home/my_usual_username/.ssh/myserver_known_hosts
chmod 600 /home/my_usual_username/.ssh/myserver_known_hosts
chown my_usual_username:my_usual_username /home/my_usual_username/.ssh/myserver_known_hosts

Of course, "my_usual_username" is the name of the non-root user who will use ssh to unlock the device.
Back to the GoFlex/Dockstar. Edit "/etc/initramfs-tools/initramfs.conf"

nano /etc/initramfs-tools/initramfs.conf

and replace "DEVICE=" by "DEVICE=eth0" to make sure that the initramfs has network access. Futhermore, we need to find out the UUID of the encrypted partition. To do thus run

blkid

and look for something like

/dev/sdb2: UUID="bbb2c58e-1084-4d1a-b9f8-79179d25743c" TYPE="crypto_LUKS"

Now edit /etc/crypttab

nano /etc/crypttab

and paste a line looking like this (depends on what your UUID is)

rootfs_crypt UUID=bbb2c58e-1084-4d1a-b9f8-79179d25743c none luks

Also edit /etc/fstab

nano /etc/fstab

and make it look like this

# <file system> <mount point>   <type>  <options>       <dump>  <pass>
/dev/mapper/rootfs_crypt /     ext4    rw,noatime,nouser_xattr,noacl,errors=remount-ro 0 1
tmpfs          /tmp            tmpfs   defaults          0       0

Mount the encrypted rootfs so that update-initramfs can determine the cipher

cryptsetup luksOpen /dev/sdb2 rootfs_crypt

This should be enough to create a proper initramfs, so let's do it

mkdir -p /lib/modules/3.0.0-goflex/kernel/arch
update-initramfs -c -k 3.0.0-goflex -t

It's very important that this works without any warnings or errors. Especially warnings or errors containing the words "device-mapper" or "cryptsetup" indicate that you did something wrong. It's even more important that our initrd knows, where to look for the encrypted rootfs. So we'll apply my ugly initrd hack. This hack works for all devices independently of their bootloader and doesn't involve changing anything via fw_setenv. Run

cd /boot
mkdir initrd.img-new &&  cp initrd.img-3.0.0-goflex initrd.img-new/
cd initrd.img-new/ && gzip -dc initrd.img-3.0.0-goflex | cpio -id
rm -r /boot/initrd.img-new/lib/modules/3.0.0-goflex/kernel/drivers/crypto
rm initrd.img-3.0.0-goflex
nano init

replace

if [ -n "${noresume}" ]; then
export noresume
unset resume
else
resume=${RESUME:-}
fi

by

if [ -n "${noresume}" ]; then
export noresume
unset resume
else
resume=${RESUME:-}
fi
ROOT="/dev/mapper/rootfs_crypt"

and do

find ./ | cpio -H newc -o > ../initrd.img-3.0.0-goflex_arc
gzip ../initrd.img-3.0.0-goflex_arc
mv ../initrd.img-3.0.0-goflex_arc.gz ..//initrd.img-3.0.0-goflex_mod
cd ..
rm -r initrd.img-new
mkimage -A arm -O linux -T ramdisk -C gzip -a 0 -e 0 -n Linux-3.0.0  -d /boot/initrd.img-3.0.0-goflex_mod /boot/uInitrd

Of course, we also need to create the kernel image

mkimage -A arm -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n Linux-3.0.0 -d /boot/vmlinuz-3.0.0-goflex /boot/uImage

Fine, now shutdown your GoFlex/Dockstar via

cryptsetup luksClose /dev/mapper/rootfs_crypt
shutdown -h now

and disconect both flash drives. Connect them to your Linux box. Once again I assume that the original flash drive is /dev/sdb, whereas the auxiliary flash drive is /dev/sdc. For the sake of safety let's create a backup of the rootfs on the auxiliary flash drive

mount /dev/sdc1 /media/auxiliary_rootfs
cd /media/auxiliary_rootfs
tar --one-file-system -cpf ~/auxiliary_rootfs.tar *

Now we are going the copy the auxiliary rootfs to the original flash drive.

mkdir /media/rootfs_crypt
cryptsetup luksOpen /dev/sdb2 rootfs_crypt
mount /dev/mapper/rootfs_crypt /media/rootfs_crypt
cp -a /media/auxiliary_rootfs/* /media/rootfs_crypt
rm -r /media/rootfs_crypt/boot/*

We cleared the boot folder, because our uImage and uInitrd will be located not there, but on a separate unencrypted partition (the small 200MB ext3 partition we created before)

mkdir /media/unenc_boot
mount /dev/sdb1 /media/unenc_boot
cp -a /media/auxiliary_rootfs/boot/*  /media/unenc_boot
mkdir /media/unenc_boot/boot
cd /media/unenc_boot/boot
ln -s ../uInitrd uInitrd
ln -s ../uImage uImage
cd ~
umount /media/unenc_boot/
umount /media/rootfs_crypt
umount /media/auxiliary_rootfs
cryptsetup luksClose rootfs_crypt

Let's clean up the mess

rmdir /media/auxiliary_rootfs
rmdir /media/unenc_boot
rmdir /media/rootfs_crypt
rmdir /media/original_rootfs

That's it. Now connect the original flash to GoFlex/Dockstar and boot it. If you monitor the boot process via netconsole, you'll observe the following. After "Starting kernel ..." you have to wait for about 20 seconds before the kernel messages appear. When you see something like this

[   29.683821] device-mapper: uevent: version 1.0.3
[   29.690027] device-mapper: ioctl: 4.20.0-ioctl (2011-02-02) initialised: dm-devel@redhat.com

then the initramfs is waiting for you to supply the LUKS password. Remember the id_rsa key we saved to "~/.ssh/myserver_rsa"? Well, it's on time to use it. Futhermore the usual Linux' OpenSSH client might become a bit confused, because the fingerprint of the dropbear SSH server running on the initramfs is different from the usual OpenSSH fingerprint of your GoFlex/Seagate. So we should better use a special known_hosts file for that. What about the LUKS password? Let's suppose you password is

debian_rules

Then the command to unlock your device will be

ssh -o "UserKnownHostsFile=~/.ssh/myserver_known_hosts" -i ~/.ssh/myserver_rsa root@ip-of-your-device "echo -ne \"debian_rules\" > /lib/cryptsetup/passfifo"

The slashes and quotation marks should look exactly like this. For more information on the syntax check out "/usr/share/doc/cryptsetup/README.remote.gz" on your GoFlex/Dockstar. You can read the manual via

zless /usr/share/doc/cryptsetup/README.remote.gz

After having submitted the password, on the netconsole you will see something like

[   53.805822] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
[   55.327002] udev[256]: starting version 164
[   56.909229] EXT4-fs (dm-0): re-mounted. Opts: (null)
[   57.093194] EXT4-fs (dm-0): re-mounted. Opts: nouser_xattr,noacl,errors=remount-ro
...
[   58.490091] kjournald starting.  Commit interval 5 seconds
...

After a few seconds you should be able to ssh into you device the usual way. Having done this, you should do some further modifications. If your kernel contains mv_cesa remove the module. mv_cesa doesn't work properly with 3.0.x kernels and will give you segfaults.

mv /lib/modules/3.0.0-goflex/kernel/drivers/crypto/mv_cesa.ko /lib/modules/3.0.0-goflex/kernel/drivers/crypto/mv_cesa.old

Use blkid to find the boot partition. First search for crypto_LUKS. For example if crypto_LUKS is located on /dev/sdc2, then the boot partition will be on /dev/sdc1
Copy the UUID of the boot partition and add a line looking like (use your UUID!)

UUID=393ac665-f5c2-488d-b601-b59ba1d5675b /boot ext3 defaults 0 1

to your /etc/fstab. We also want to activate the encrypted swap partition. It's very easy to find the swap partition since you already know what is your root partition. Since swap is the third primary partition, you just have to replace one number. For example, if your crypto_LUKS is located on /dev/sdc2, then swap will be /dev/sdc3. Another way is to look for the id of your usb flash

ls /dev/disk/by-id/

For instance, in my case the id looks like this

/dev/disk/by-id/usb-SanDisk_Cruzer_Blade_20051740220F36703394-0\:0-part3

Note the appropriate location and add a line looking like this (use your id!)

swap_crypt /dev/sdc3 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256

or like this

swap_crypt /dev/disk/by-id/usb-SanDisk_Cruzer_Blade_20051740220F36703394-0\:0-part3 /dev/urandom swap,cipher=aes-cbc-essiv:sha256,size=256

to /etc/crypttab. Finally add

/dev/mapper/swap_crypt          none swap sw 0 0

to /etc/fstab
Run

/etc/init.d/cryptdisks start
mount -a
swapon -a
free

The output of free should look like this

             total       used       free     shared    buffers     cached
Mem:        125404     123836       1568          0       3012      76560
-/+ buffers/cache:      44264      81140
Swap:       524284       5772     518512

Here are also some tweaks for kernel/ext4 to reduce the number of writes and thus prolong the life of your flash drive. By default, if kernel wants to write something to rootfs, it will wait for about 5 seconds to gather all the write operation and then perform them simultaneously. Futhermore, ext4 ususally flushes its journal every 5 second too. We can slightly increase this intervals, say to 15 seconds. The problem about making the intervals too big is that if your GoFlex/Dockstar suddenly looses power, all the data that was written in the last x seconds will be lost. While it sounds reasonable to loose data of the last 5 or 15 seconds, it probably wouldn't be that nice to loose the last 5 or 10 minutes. That's why we would rather go with 15 seconds, although higher intervals are possible as well.

Edit /etc/fstab and replace

/dev/mapper/rootfs_crypt /     ext4    rw,noatime,nouser_xattr,noacl,errors=remount-ro 0 1

by

/dev/mapper/rootfs_crypt /     ext4    rw,noatime,nouser_xattr,noacl,commit=15,errors=remount-ro 0 1

Also run

echo "vm.dirty_writeback_centisecs = 1500" >> /etc/sysctl.conf

Reboot your device for the tweaks to take effect. For more information on the ext4 mount parameters you might want to check out the documentation

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/filesystems/ext4.txt;h=7be02ac5fa36d7f4c07856fe9cf89391e08986f7;hb=HEAD

By the way, if you use bash to do the ssh unlocking, you probably don't want your password to appear in the bash history. What you can do is open a new terminal window and run

unset HISTFILE
ssh -o "UserKnownHostsFile=~/.ssh/myserver_known_hosts" -i ~/.ssh/myserver_rsa root@ip-of-your-device "echo -ne \"debian_rules\" > /lib/cryptsetup/passfifo"
exit

this way the ssh command will not be added to the history

What about hardware offloading via mv_cesa?
Update Forget mv_cesa. It doesn't work with the 3.0.0 kernel. Used it for swap and got unexpected segfaults. After shutting it down the problems where gone. Tried it again and the segfault where there.

I tried to make it work for about a week, experimenting with different kernel versions and kernel parameters, but to no avail.
Although mv_cesa itself works fine, if you try to load the module before unlocking your LUKS encrypted root partition, you always get a kernel panic. As soon as you remove mv_cesa from your initrd, everything works fine again. Just in case someone is interested. The kernel panic looks like this

[   29.778658] mv643xx_eth_port mv643xx_eth_port.0: eth0: link up, 1000 Mb/s, full duplex, flow control disabled
[   29.789733] console [netcon0] enabled
[   29.793438] netconsole: network logging started
[   29.867747] NET: Registered protocol family 10
[   29.914938] device-mapper: uevent: version 1.0.3
[   29.921044] device-mapper: ioctl: 4.20.0-ioctl (2011-02-02) initialised: dm-devel@redhat.com
[   54.681398] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
[   54.878703] Kernel panic - not syncing: Attempted to kill init!
[   54.884759] [<c0035be8>] (unwind_backtrace+0x0/0xec) from [<c02e35f0>] (panic+0x54/0x18c)
[   54.893031] [<c02e35f0>] (panic+0x54/0x18c) from [<c00491cc>] (do_exit+0xb8/0x6dc)
[   54.900657] [<c00491cc>] (do_exit+0xb8/0x6dc) from [<c004987c>] (do_group_exit+0x8c/0xc0)
[   54.908917] [<c004987c>] (do_group_exit+0x8c/0xc0) from [<c0057b40>] (get_signal_to_deliver+0x390/0x3d0)
[   54.918482] [<c0057b40>] (get_signal_to_deliver+0x390/0x3d0) from [<c0032598>] (do_signal+0xb0/0x5ac)
[   54.927776] [<c0032598>] (do_signal+0xb0/0x5ac) from [<c0032aac>] (do_notify_resume+0x18/0x60)
[   54.936466] [<c0032aac>] (do_notify_resume+0x18/0x60) from [<c0030334>] (work_pending+0x24/0x28)

It happens exactly the moment when run-init is performed in the init script. Somehow switching to the rootfs breaks mv_cesa and since the rootfs encryption is handled via mv_cesa, the kernel panics. If mv_cesa is not loaded, the encryption is handled by the kernel itself (no hardware offloading). In this case everything works well. Note, that although mv_cesa doesn't "accelerate" root, it does work for the swap partition. That's because swap is mounted after switching to the rootfs, so at that moment it's safe to use mv_cesa. If you use aes-cbc-essiv:256 to encrypt some other partitions, hardware offloading will work for them too. It's only the rootfs where mv_cesa fails. This is why it's better to create a rather small root partition containing only the bare minimum and keep all other stuff like backups, music, documents, etc. on another encrypted partition. In this case that data partition will benefit from mv_cesa.

Hope this helps!



Edited 2 time(s). Last edit at 10/15/2011 03:12PM by Vlad.

Wow! I don't have an immediate need to do this, but I'm definitely bookmarking this thread. Thanks for taking the time to do that masterful job of documenting your work, Vlad!

Thanks, but unfortunately it doesn't work quite as expected. By this I mean the aes offloading. I thought that I made it working my compiling mv_cesa against the kernel, but in fact it works only on occasions, giving segfaults otherwise. However, if you remove mv_cesa completely everything works perfectly. I believe that that module is still too buggy and not properly maintained. Anyway, even without hardware offloading, the performance is sufficient for my needs (mostly postfix and minidlna).

So if you want to get this working, just remove the mv_cesa module when configuring the kernel. Alternatively you can keep it, but perform a

rm -r /boot/initrd.img-new/lib/modules/3.0.0-goflex/kernel/drivers/crypto

while performing the initrd hack part.

Just in case someone is interested. The kernel panic looks like this

[   29.778658] mv643xx_eth_port mv643xx_eth_port.0: eth0: link up, 1000 Mb/s, full duplex, flow control disabled
[   29.789733] console [netcon0] enabled
[   29.793438] netconsole: network logging started
[   29.867747] NET: Registered protocol family 10
[   29.914938] device-mapper: uevent: version 1.0.3
[   29.921044] device-mapper: ioctl: 4.20.0-ioctl (2011-02-02) initialised: dm-devel@redhat.com
[   54.681398] EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: (null)
[   54.878703] Kernel panic - not syncing: Attempted to kill init!
[   54.884759] [<c0035be8>] (unwind_backtrace+0x0/0xec) from [<c02e35f0>] (panic+0x54/0x18c)
[   54.893031] [<c02e35f0>] (panic+0x54/0x18c) from [<c00491cc>] (do_exit+0xb8/0x6dc)
[   54.900657] [<c00491cc>] (do_exit+0xb8/0x6dc) from [<c004987c>] (do_group_exit+0x8c/0xc0)
[   54.908917] [<c004987c>] (do_group_exit+0x8c/0xc0) from [<c0057b40>] (get_signal_to_deliver+0x390/0x3d0)
[   54.918482] [<c0057b40>] (get_signal_to_deliver+0x390/0x3d0) from [<c0032598>] (do_signal+0xb0/0x5ac)
[   54.927776] [<c0032598>] (do_signal+0xb0/0x5ac) from [<c0032aac>] (do_notify_resume+0x18/0x60)
[   54.936466] [<c0032aac>] (do_notify_resume+0x18/0x60) from [<c0030334>] (work_pending+0x24/0x28)

It happens exactly the moment when run-init is performed in the init script. Somehow switching to the rootfs breaks mv_cesa and since the rootfs encryption is handled via mv_cesa, the kernel panics. If mv_cesa is not loaded, the encryption is handled by the kernel itself (no hardware offloading). In this case everything works well.

Damn, I'm so frustrated :(

I've just updated the howto to reflect the current situation. To make it straight: The howto is correct, both encryption and SSH unlocking work very well. The only thing that doesn't work is hardware offloading (mv_cesa) for the root partition. Hardware offloading for all other encrypted partitions (including swap) except for the root partition works. The thing is, that when you open an encrypted partition via "cryptsetup luksOpen" you must have mv_cesa loaded in order to get hardware offloading. If at that moment mv_cesa is not loaded, then the encryption will be handled entirely by the CPU.

Now if you unlock a partition without mv_cesa and now want to enable hardware offloading, the only thing you can do it is unmount the partition, lock it (cryptsetup luksClose), load mv_cesa and then unlock the partition again.

Our problem is that if you load mv_cesa before unlocking the root partition, the kernel crashes. After the root partition has been unlocked you can still load mv_cesa, but the hardware offloading for this partition will not work (see above). The only solution I can think of would be to let kernel switch to the rootfs, then unmount rootfs, lock it, load mv_cesa and then mount rootfs again. But since it's not possible to unmount rootfs of a working system, unless you want it to crash, I currently see no possibility to find any workaround for this problem.

The only thing we can hope is that someone will create a kernel patch to fix it.

The latest news. mv_cesa is not working properly with the 3.0.0 kernel. I tried using it for swap partition and got sudden segfaults. Recovered my rootfs from a backup twice only to realize that the segfaults started every time I activated the swap. Then I disabled mv_cesa and the segfaults were gone. Enabled it and segfault came again. What can I say: forget about hardware offloading :(