Basic steps in building Linux modules

I have updated the build procedure for the released kernels. So from kernel linux-5.3.2-kirkwood-tld-1 and kernel linux-4.4.201-oxnas-tld-1 on, there will be no need to work around some problems building module as seen in the past kernels installation. You would just install the kernel header, and start building modules natively on your box.

Quote

Updated 16 Nov 2019:

Kernel linux-4.4.201-oxnas-tld-1 package was uploaded.

New/Update features:

- Pickup security patches from mainline.
- This kernel header now can be installed and used to build modules, without any special tweaks (this issue was described here).

Quote

Updated 06 Oct 2019:

Kernel linux-5.3.2-kirkwood-tld-1 package has been uploaded. The following features were added/updated:

- This kernel is now configured to run with Kirkwood boxes up to 2GB RAM.
- Updated Dell Kace M300 DTS.
- This kernel header is now can be installed and used to build cryptodev module, without any special tweaks.

========================

Old Info, keep for posterity


I have this post written up so it can be linked to when users have problem building modules with my released kernel. A few users have got around a minor problem caused by apt-get and/or dpkg to build modules successfully. Below is a more detailed explanation.


In general, here is what we need to do to build modules. I'm using linux-headers-5.2.9-kirkwood-tld-2 kernel as an example.

apt-get install build-essential
apt-get install linux-headers-5.2.9-kirkwood-tld-2_2.0_armel.deb

Now at this point, the kernel header has been installed:

ls -l /usr/src

drwxr-xr-x 23 root root 4096 Sep  2 19:25 linux-headers-5.2.9-kirkwood-tld-2

Note that the build folder is not created by apt-get install command above. This is a quirk with apt-get/dpkg. The build folder will be created only if you install the kernel and headers at the same time.

For example, this command will install both kernel and headers, and then create a symlink to the headers folder at the same time.

dpkg -i linux-image-5.2.9-kirkwood-tld-1_1.0_armel.deb linux-headers-5.2.9-kirkwood-tld-2_2.0_armel.deb

But when you install the kernel only, and then install the headers later, it will not create that symlink. So we have to do it manually:

ln -s /usr/src/linux-headers-5.2.9-kirkwood-tld-2/ /lib/modules/5.2.9-kirkwood-tld-2/build

Double check it:

ls -l /lib/modules/5.2.9-kirkwood-tld-2/

lrwxrwxrwx  1 root root     44 Sep  2 19:28 build -> /usr/src/linux-headers-5.2.9-kirkwood-tld-2/

At this point you can proceed and build modules. The build enviroment has been set up properly.


When you build a particular module, that will also have its own build environment setup requirements. Things such as makefile or scripts, will be specific to that source code. If the build fails, you will need to investigate that particular source and its makefile/script to see if the build paths were set up propoerly.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 5 time(s). Last edit at 11/16/2019 07:14PM by bodhi.

Hi,

Thanks for the great tutorial. It's indeed the basic setup for building a module.



However in many cases. It's seems we missed something more than just a soft link.
https://forum.doozan.com/read.php?2,76709,87984
https://forum.doozan.com/read.php?2,18152,51209#msg-51209

Somehow these two different modules are all missed

#include <sys/socket.h>

And as racic said

Quote
racic
The sys/socket.h however sits within /usr/include/arm-linux-gnueabi folder.
If I modify Makefile to include that, the features.h is missing, when I include the /usr/include folder I get tons of messages warning of types redefinitions.

Did you have any clue about what cause it?

Thank you!

yuko1225,

I would look at the makefile or the build script of the module source being built. It might be too old. To see more details, I would need actually build it myself.

This is a basic Debian rootfs. So anything more than Debian build-essential, you might need a particular module build set up script or a particular makefile.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> yuko1225,
>
> I would look at the makefile or the build script
> of the module source being built. It might be too
> old. To see more details, I would need actually
> build it myself.
>
> This is a basic Debian rootfs. So anything more
> than Debian build-essential, you might need a
> particular module build set up script or a
> particular makefile.

The cryptodev module has just released v1.10 this year. It doesn't looks like an abandoned project.
https://github.com/cryptodev-linux/cryptodev-linux
The install instruction is also pretty straight forward. make > make install

In fact, TheFeaR in that thread mentioned if cross-complied it has no problem to build. So something must be missing whatever it is...

I will try to cross-compile it if there is really no other solution.That's gonna be a headache

> The cryptodev module has just released v1.10 this
> year. It doesn't looks like an abandoned project.
> https://github.com/cryptodev-linux/cryptodev-linux
> The install instruction is also pretty straight
> forward. make > make install

> In fact, TheFeaR in that thread mentioned if
> cross-complied it has no problem to build. So
> something must be missing whatever it is...

Sounds to me the author has made a bad assumption about the build environment. It should be OK to build it natively with a basic Debian debootstrap system where build-essential package was installed. If it is not enough, then there should be a script or some extra steps documented.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi Wrote:
-------------------------------------------------------
> > The cryptodev module has just released v1.10
> this
> > year. It doesn't looks like an abandoned
> project.
> >
> https://github.com/cryptodev-linux/cryptodev-linux
> > The install instruction is also pretty straight
> > forward. make > make install
>
> > In fact, TheFeaR in that thread mentioned if
> > cross-complied it has no problem to build. So
> > something must be missing whatever it is...
>
> Sounds to me the author has made a bad assumption
> about the build environment. It should be OK to
> build it natively with a basic Debian debootstrap
> system where build-essential package was
> installed. If it is not enough, then there should
> be a script or some extra steps documented.


I just cross-compiled it with the kernel source tree on a freshly debootstrap system. No special setup and no issue at all.

The debian on my plug has no issue to build non-kernel related package either. I've already rebuild the openssl and openssh on it.

yuko1225,

> The debian on my plug has no issue to build
> non-kernel related package either. I've already
> rebuild the openssl and openssh on it.

Please be more specific! which Debian, running which kernel, and which plug?

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

Sorry for that.

Pogoplug V4 Mobile. Debian buster. 5.2.9-kirkwood-tld-1.

The debootstrap system was also debian buster, build on debian jessie kernel. The kernel source tree was 5.2.9 applied with the patch in your kernel package.

I tried both way to compile it. Out of the tree and within the tree. Both succeed. The in-tree Makefiles configuration was:

obj-$(CONFIG_CRYPTO_OCF_CRYPTODEV) += cryptodev.o
cryptodev-objs := ioctl.o main.o cryptlib.o authenc.o zc.o util.o

With its own headers file put in the include path.

The out of tree is just pretty much straight forward. Edit the Makefiles to reflect the real kernal path

KERNEL_DIR ?= /usr/src/linux-5.2.9

Set the cross-complie variables

ARCH=arm
CROSS_COMPILE=arm-linux-gnueabi-

Then "make" it.

I'm probably gonna make a new thread about the HW/Crypto on the new debian buster, and share my pre-compiled binary. But it would be great if I can just compile it on device.



Edited 6 time(s). Last edit at 09/03/2019 08:02PM by yuko1225.

yuko1225,

Thanks very much for the info!

I think mieki test case, and alexr test case, and your test cases are very useful to track down the problem.

Hope I will have sometime to do some further tests.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

I’ve tried building crytodev to see why it would not work building natively. I see the reason now.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

So I've tracked this down to a bug in one of kernel-package scripts.

The headers generated by make-kpkg has an extra set of include files.

When we build a typical module like mieki's test case above, it is fine. A little bit complicate to explain this, but when built with cryptodev code, GCC would resolve to a wrong include path.

And the reason cross-compiling works is because the cross compiler only looks for the headers in /usr/include/arm-linux-gnueabi/.

Too bad, because make-kpkg has been a really good tool, but it is now EOL in mainline. So I have switched my builds back to regular make.

This will be fixed in the next kernel release. If anybody has a more urgent need to do a native build of cryptodev, then please post the request.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 09/08/2019 04:46PM by bodhi.

I have updated the 1st post in this thread to include the notice:

Quote

I have updated the build procedure for the released kernels. So from kernel linux-5.3.2-kirkwood-tld-1 on, there will be no need to work around some problems building module as seen in the past kernels installation. You would just install the kernel header, and start building modules natively on your box.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)



Edited 1 time(s). Last edit at 10/06/2019 01:10AM by bodhi.

I'm trying to build a kernel module zd1211rw

I followed this post then executed these steps:

extraversion="-"$(uname -r|sed 's/[0-9].[0-9].[0-9]-//' ) # extraversion in 3.5.0-17-generic is -17-generic
rm -rf ~/tmp
mkdir -p ~/tmp/build/
cd ~/tmp/build/
cp /boot/config-`uname -r`  .config
cp /usr/src/linux-headers-5.2.9$extraversion/Module.symvers ./
cd /usr/src/linux-headers-5.2.9$extraversion
make EXTRAVERSION=$extraversion O=~/tmp/build oldconfig
make EXTRAVERSION=$extraversion O=~/tmp/build prepare

#make EXTRAVERSION=$extraversion O=~/tmp/build outputmakefile
#make EXTRAVERSION=$extraversion O=~/tmp/build archprepare
#make EXTRAVERSION=$extraversion O=~/tmp/build modules SUBDIRS=scripts

but it failed at "prepare" step

with this error:

make[2]: *** No rule to make target '/usr/src/linux-headers-5.2.9-kirkwood-tld-1/arch/arm/tools/syscall.tbl', needed by 'arch/arm/include/generated/uapi/asm/unistd-common.h'.  Stop.

Google didn't get me useful results :/

any hints?

pepe,

Have you updated the kernel to 5.3.2-kirkwood-tld-1, and instal that kernel header?

In the 1st post:

Quote

So from kernel linux-5.3.2-kirkwood-tld-1 and kernel linux-4.4.201-oxnas-tld-1 on, there will be no need to work around some problems building module as seen in the past kernels installation. You would just install the kernel header, and start building modules natively on your box.

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)

bodhi,

Nop, I'm working with 5.2.9 without updating. Do you recommend updating the kernel in general?
I installed debian on NSA325 v1 following your great work and guides! (Thanks a lot!)
It is working without noticeable issues for some weeks now...so I didn't feel the need for kernel updates...please correct me if I'm missing something.

I found the issue, the headers package I installed didn't include the code, so I got 5.2.9 sources and the makes above went fine.

FTR, module zd1211rw works with NSA325 v1.

pepe,

-------------------------------------------------------
> bodhi,
>
> Nop, I'm working with 5.2.9 without updating. Do
> you recommend updating the kernel in general?
> I installed debian on NSA325 v1 following your
> great work and guides! (Thanks a lot!)
> It is working without noticeable issues for some
> weeks now...so I didn't feel the need for kernel
> updates...please correct me if I'm missing
> something.

Best to install new kernel. It has a fix for the module builds folder location. But other than that, no need to.

> I found the issue, the headers package I installed
> didn't include the code, so I got 5.2.9 sources
> and the makes above went fine.
>
> FTR, module zd1211rw works with NSA325 v1.

Cool!

-bodhi
===========================
Forum Wiki
bodhi's corner (buy bodhi a beer)