Trying to squeeze Debian rootfs into 128 MB NAND storage.

So I've been trying to create a Debian rootfs tiny enough to fit into the limited NAND storage found on the Pogoplug (E02/Mobile) which is only 128 MB. The goal being to produce an embedded Debian rescue system now that Embedian is no longer around. My first attempt using

debootstrap --arch=armel --variant=minbase stretch . http://SomeDebianMirror

Created a rootfs that was still much too big. Somewhere near 250 MB+ if memory serves. After some research I found that Debian produces a slimmed down rootfs for docker that is small enough, but lacks several components to run on actual hardware as there's no init system, no kernel, and no networking, After still more hours of work/research I added back the missing components and was surprised the whole rootfs was roughly ~85 MB total uncompressed (24 MB compressed). I've taken the liberty of uploading it here:

Debian Stretch - Mini Rootfs Armel No Kernel

As the name implies there's no kernel installed in the system because unfortunately once installed, the kernel brings the total rootfs size to ~200 MB. This should not be an issue for devices with larger NAND storage but I'm currently still working on compiling a small enough kernel to fit in the remaining NAND flash. That said I figured someone else might benefit from the rootfs and this project in its current state. This is the list of the total installed packages:

apt list --installed

Listing... Done
adduser/now 3.115 all [installed,local]
apt/now 1.4.10 armel [installed,local]
base-files/now 9.9+deb9u13 armel [installed,local]
base-passwd/now 3.5.43 armel [installed,local]
bash/now 4.4-5 armel [installed,local]
bsdutils/now 1:2.29.2-1+deb9u1 armel [installed,local]
busybox/now 1:1.22.0-19+b3 armel [installed,local]
coreutils/now 8.26-3 armel [installed,local]
cpio/now 2.11+dfsg-6 armel [installed,local]
dash/now 0.5.8-2.4 armel [installed,local]
debconf/now 1.5.61 all [installed,local]
debian-archive-keyring/now 2017.5+deb9u1 all [installed,local]
debianutils/now 4.8.1.1 armel [installed,local]
dialog/now 1.3-20160828-2 armel [installed,local]
diffutils/now 1:3.5-3 armel [installed,local]
dpkg/now 1.18.25 armel [installed,local]
dropbear-bin/now 2016.74-5+deb9u1 armel [installed,local]
dropbear-initramfs/now 2016.74-5+deb9u1 all [installed,local]
dropbear-run/now 2016.74-5+deb9u1 all [installed,local]
e2fslibs/now 1.43.4-2+deb9u2 armel [installed,local]
e2fsprogs/now 1.43.4-2+deb9u2 armel [installed,local]
findutils/now 4.6.0+git+20161106-2 armel [installed,local]
gcc-6-base/now 6.3.0-18+deb9u1 armel [installed,local]
gpgv/now 2.1.18-8~deb9u4 armel [installed,local]
grep/now 2.27-2 armel [installed,local]
gzip/now 1.6-5+b1 armel [installed,local]
hostname/now 3.18+b1 armel [installed,local]
ifupdown/now 0.8.19 armel [installed,local]
init-system-helpers/now 1.48 all [installed,local]
initramfs-tools/now 0.130 all [installed,local]
initramfs-tools-core/now 0.130 all [installed,local]
initscripts/now 2.88dsf-59.9 armel [installed,local]
insserv/now 1.14.0-5.4+b1 armel [installed,local]
iproute2/now 4.9.0-1+deb9u1 armel [installed,local]
isc-dhcp-client/now 4.3.5-3+deb9u1 armel [installed,local]
isc-dhcp-common/now 4.3.5-3+deb9u1 armel [installed,local]
klibc-utils/now 2.0.4-9 armel [installed,local]
kmod/now 23-2 armel [installed,local]
libacl1/now 2.2.52-3+b1 armel [installed,local]
libapt-pkg5.0/now 1.4.10 armel [installed,local]
libattr1/now 1:2.4.47-2+b2 armel [installed,local]
libaudit-common/now 1:2.6.7-2 all [installed,local]
libaudit1/now 1:2.6.7-2 armel [installed,local]
libblkid1/now 2.29.2-1+deb9u1 armel [installed,local]
libbz2-1.0/now 1.0.6-8.1 armel [installed,local]
libc-bin/now 2.24-11+deb9u4 armel [installed,local]
libc6/now 2.24-11+deb9u4 armel [installed,local]
libcap-ng0/now 0.7.7-3+b1 armel [installed,local]
libcomerr2/now 1.43.4-2+deb9u2 armel [installed,local]
libdb5.3/now 5.3.28-12+deb9u1 armel [installed,local]
libdebconfclient0/now 0.227 armel [installed,local]
libdns-export162/now 1:9.10.3.dfsg.P4-12.3+deb9u7 armel [installed,local]
libelf1/now 0.168-1 armel [installed,local]
libfdisk1/now 2.29.2-1+deb9u1 armel [installed,local]
libgcc1/now 1:6.3.0-18+deb9u1 armel [installed,local]
libgcrypt20/now 1.7.6-2+deb9u3 armel [installed,local]
libgpg-error0/now 1.26-2 armel [installed,local]
libisc-export160/now 1:9.10.3.dfsg.P4-12.3+deb9u7 armel [installed,local]
libklibc/now 2.0.4-9 armel [installed,local]
libkmod2/now 23-2 armel [installed,local]
liblz4-1/now 0.0~r131-2+b1 armel [installed,local]
liblzma5/now 5.2.2-1.2+b1 armel [installed,local]
libmnl0/now 1.0.4-2 armel [installed,local]
libmount1/now 2.29.2-1+deb9u1 armel [installed,local]
libncurses5/now 6.0+20161126-1+deb9u2 armel [installed,local]
libncursesw5/now 6.0+20161126-1+deb9u2 armel [installed,local]
libpam-modules/now 1.1.8-3.6 armel [installed,local]
libpam-modules-bin/now 1.1.8-3.6 armel [installed,local]
libpam-runtime/now 1.1.8-3.6 all [installed,local]
libpam0g/now 1.1.8-3.6 armel [installed,local]
libpcre3/now 2:8.39-3 armel [installed,local]
libprocps6/now 2:3.3.12-3+deb9u1 armel [installed,local]
libselinux1/now 2.6-3+b3 armel [installed,local]
libsemanage-common/now 2.6-2 all [installed,local]
libsemanage1/now 2.6-2 armel [installed,local]
libsepol1/now 2.6-2 armel [installed,local]
libsmartcols1/now 2.29.2-1+deb9u1 armel [installed,local]
libss2/now 1.43.4-2+deb9u2 armel [installed,local]
libssl1.0.2/now 1.0.2u-1~deb9u2 armel [installed,local]
libstdc++6/now 6.3.0-18+deb9u1 armel [installed,local]
libsystemd0/now 232-25+deb9u12 armel [installed,local]
libtinfo5/now 6.0+20161126-1+deb9u2 armel [installed,local]
libudev1/now 232-25+deb9u12 armel [installed,local]
libustr-1.0-1/now 1.0.4-6 armel [installed,local]
libuuid1/now 2.29.2-1+deb9u1 armel [installed,local]
linux-base/now 4.5 all [installed,local]
linux-image-4.20.6-kirkwood-tld-1/now 1.0 armel [installed,local]
login/now 1:4.4-4.1 armel [installed,local]
lsb-base/now 9.20161125 all [installed,local]
mawk/now 1.3.3-17+b3 armel [installed,local]
mount/now 2.29.2-1+deb9u1 armel [installed,local]
multiarch-support/now 2.24-11+deb9u4 armel [installed,local]
ncurses-base/now 6.0+20161126-1+deb9u2 all [installed,local]
ncurses-bin/now 6.0+20161126-1+deb9u2 armel [installed,local]
net-tools/now 1.60+git20161116.90da8a0-1 armel [installed,local]
netbase/now 5.4 all [installed,local]
passwd/now 1:4.4-4.1 armel [installed,local]
perl-base/now 5.24.1-3+deb9u7 armel [installed,local]
procps/now 2:3.3.12-3+deb9u1 armel [installed,local]
sed/now 4.4-1 armel [installed,local]
sensible-utils/now 0.0.9+deb9u1 all [installed,local]
startpar/now 0.59-3.1 armel [installed,local]
sysv-rc/now 2.88dsf-59.9 all [installed,local]
sysvinit-core/now 2.88dsf-59.9 armel [installed,local]
sysvinit-utils/now 2.88dsf-59.9 armel [installed,local]
tar/now 1.29b-1.1 armel [installed,local]
tzdata/now 2020a-0+deb9u1 all [installed,local]
u-boot-tools/now 2016.11+dfsg1-4 armel [installed,local]
udev/now 232-25+deb9u12 armel [installed,local]
util-linux/now 2.29.2-1+deb9u1 armel [installed,local]
zlib1g/now 1:1.2.8.dfsg-5 armel [installed,local]

In case you'd like to create your own rootfs, i'll outline the steps i took
---------------------------------------------------------------------------------------------
On another linux machine, use docker to pull a recent "slim" Debian container. This is the one I used:

docker pull debian@sha256:ee521e00a735696b1d44cb15620e924bd6f2e9b69553ce873f8efc0a567f376b

Pull up the Image ID of the container you just downloaded:

sudo docker images -a

Save it as a tar archive:

docker save -o SomeName.tar IMAGE-ID

Once you extract the archive, there should be a file called "layer.tar" inside. That's the base rootfs archive that's used (~53 MB). This will need to be copied over to an Armel based system.

Now on an ARMV5/Armel system:
Create a small img that we'll loopback mount:

dd if=/dev/zero of=ImageName.img bs=1M count=200 status=progress
mkfs.ext3 ImageName.img
mount -o loop ImageName.img SomeMountPoint/

copy/extract layer.tar into mounted image. As root:

cp layer.tar SomeMountPoint/
cd SomeMountPoint/
tar -xvf layer.tar

Copy these debs into SomeMountPoint/tmp for later install:

busybox_1.22.0-19+b3_armel.deb
cpio_2.11+dfsg-6_armel.deb
dialog_1.3-20160828-2_armel.deb
dropbear-bin_2016.74-5+deb9u1_armel.deb
dropbear-initramfs_2016.74-5+deb9u1_all.deb
dropbear-run_2016.74-5+deb9u1_all.deb
ifupdown_0.8.19_armel.deb
initramfs-tools_0.130_all.deb
initramfs-tools-core_0.130_all.deb
initscripts_2.88dsf-59.9_armel.deb
insserv_1.14.0-5.4+b1_armel.deb
iproute2_4.9.0-1+deb9u1_armel.deb
isc-dhcp-client_4.3.5-3+deb9u1_armel.deb
isc-dhcp-common_4.3.5-3+deb9u1_armel.deb
klibc-utils_2.0.4-9_armel.deb
kmod_23-2_armel.deb
libdns-export162_9.10.3.dfsg.P4-12.3+deb9u7_armel.deb
libelf1_0.168-1_armel.deb
libisc-export160_9.10.3.dfsg.P4-12.3+deb9u7_armel.deb
libklibc_2.0.4-9_armel.deb
libkmod2_23-2_armel.deb
libmnl0_1.0.4-2_armel.deb
libncurses5_6.0+20161126-1+deb9u2_armel.deb
libprocps6_3.3.12-3+deb9u1_armel.deb
libssl1.0.2_1.0.2u-1~deb9u2_armel.deb
linux-base_4.5_all.deb
netbase_5.4_all.deb
net-tools_1.60+git20161116.90da8a0-1_armel.deb
procps_3.3.12-3+deb9u1_armel.deb
startpar_0.59-3.1_armel.deb
sysvinit-core_2.88dsf-59.9_armel.deb
sysv-rc_2.88dsf-59.9_all.deb
u-boot-tools_2016.11+dfsg1-4_armel.deb
udev_232-25+deb9u12_armel.deb

chroot into the rootfs (as root):

In the root of SomeMountPoint:

mkdir dev/pts
touch dev/urandom

mount -t proc /proc/ proc/
mount devpts dev/pts/ -t devpts
mount --bind /dev/urandom dev/urandom

chroot . /bin/bash

Now that you're in the chroot system you just need to install the deb's we copied over in a previous step. Also create/edit the following config files:

/etc/network/interfaces
/etc/fw_env.config
/etc/fstab

These can be copied over from Bodhi's rootfs without issue.

Don't forget to change the default password:

passwd

Now you can choose to install one of Bodhi's kernel's which works perfectly on this rootfs but will take the total installed size to ~200 MB. Regardless when you're ready exit and umount proc,dev/pts, and dev/urandom as well as the mounted image:

exit
umount proc/
umount dev/pts
umount dev/urandom
cd ..
umount SomeMountPoint

Now you should have an img you can mount/copy/backup which you can use to create a tar.gz backup/rootfs as outlined in Bodhi's wiki post:
https://forum.doozan.com/read.php?2,12096,24034#msg-24034



Edited 2 time(s). Last edit at 11/01/2020 04:14PM by alexr.

Why do you need to host the kernel in your rootfs?

If the kernel uimage is in the mtds or on a USB I’d think you could leave it out? As long as you don’t need any other modules loaded.

There's actually no real reason beyond wanting to have a self contained, read only recovery system to boot into when no drives are connected. That coupled with the fact that I'm still pretty ignorant as to the workings of u-boot and working with the mtd partitions in general (mostly this). Honestly, if I can keep everything small enough I might just be able to stick everything in the 91MB mtd3 "data" partition and be done with it.

Ahh understood, well you could still just host the uimage and uinitrd in your 128mb rootfs and remove all the kernel items.

As it’s a reduce system you won’t be compiling anything or need to update the kernel too often.

I’m unsure how you would get uboot to load a kernel stored this way, but it’s so flexible there’s sure to be a way, just need to know which blocks of the mtds to access.

alexr,

I actually had done this back in kernel 4.0.x (~ 5years ago), but never released this Kirkwood rootfs (84MB). I did it as an experiment (using both UBIFS and SquashFS to see I can do a pros and cons).

With multiple rescue systems we have available in Rescue System subforum, I did not see the need to add another one. However, it is still a good approach, because it is so clean and simple to manage. We'll just have a u-boot and a rootfs on NAND, no other moving parts.

Let me see when I have time, I can revisit this rootfs building and hope I can still keep it under 100MB with the latest kernel.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Hey bodhi,

Thanks for the reply! I agree, having it all in one small self contained little package simplifies things greatly. A small Debian rescue system would allow the user to fix most if not all issues that might arise right from their own device and provides a familiar system for them to work within. I've had moderate success with my tiny kernel builds so far. They work, and the overall install size is under 100MB but they output strange errors during the boot process due to disabling far too many things I think. I'd very much appreciate your expertise on the matter when you have time. Thanks again.