It’s really common for people to make mistakes. Some of us make just a lot of them. And when we do, the question of course is whether or not we can do something about it. Now, I don’t know about you, but I always prefer solving my problems after understanding what went wrong.
In a recent case, what happened had terrific consequences. Without knowing much about the support of Reiser4 filesystem in the kernel of Ubuntu 11.04, I stupidly went ahead and formatted a 2TB external hard drive with said filesystem from my Arch-based laptop. Then I happily put tons of data on it, since my laptop’s hard drive had already failed several times. Imagine what: Very soon after, the same hard drive failed again.
I was really excited I had a backup of my data on an external hard drive and that time around I didn’t lose a bit (yes, this has happened to me before). So the day came and I wanted to watch the fourth episode of Rozen Maiden (an epic Anime). But after plugging the drive into my Ubuntu box I discovered that it just wouldn’t mount. I tried hard with Nautilus, pmount etc. but to no avail. At some point I looked at /proc/filesystems. What I saw in there didn’t make me happy at all: Reiser4 was absent! There was also no such module in /lib/modules/`uname -r`/kernel/fs, so by that time I thought I was dead in the water. A moment later -it really seemed like an eternity- I managed to pull myself together and immediately looked up for Reiser4 support in the Linux kernel. Soon I discovered that the mainline kernel needs a patch in order to support Reiser4, so do the majority of the Linux distributions out there.
Since I was left without my Arch laptop I had to improvise in the -relatively unknown for me- land of Ubuntu. I was pretty sure I could make it compile a kernel. I mean Ubuntu is no Arch but it’s still a Linux-based OS, darn it, so in reality there was no excuse for failure. And as it turned out, failure and yours truly didn’t meet that day.
Preparing for the operation
This is the not-so-painful part of the process. You only have to install some packages that are needed in order to compile a kernel. Now, I’m not a newbie, I know my way around a Linux system. In spite of that I admit I had at first some problems with the package names – but Google really helped. So there you go:
$ sudo apt-get -y install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge git-core libncurses5 libncurses5-dev libelf-dev libdw-dev binutils-dev ... $
If you are a developer or have messed with similar stuff, then you most probably have some or even all of those packages already installed. But it doesn’t hurt to execute this command anyways. You also have to make sure that all the dependencies for the Linux build are satisfied. To that end, just type the following:
$ sudo apt-get build-dep kernel-package linux-meta ... $
Bringing on the patient
By now you should have everything you need to recompile your kernel. But wait. Where are you going to operate on? Why, on the Linux kernel’s source code, of course! There are many ways to get the patient on the surgery table. Personally, I prefer having a clean git clone of the latest kernel for the version of Ubuntu I’m using and keep a backup of it in case I destroy something, so I won’t have to re-download the source (believe me, it’s *huge*).
Since you installed git a bit earlier, now create a directory to have as a workplace:
$ mkdir -p ~/dev/kernel $ cd ~/dev/kernel/ $
Then clone the repository in it:
$ git clone git://kernel.ubuntu.com/ubuntu/ubuntu-natty.git source Cloning into source... remote: Counting objects: 1980747, done. remote: Compressing objects: 100% (329901/329901), done. Receiving objects: 100% (1980747/1980747), 496.08 MiB | 1.32 MiB/s, done. Resolving deltas: 100% (1655091/1655091), done. $
If you’re catchy you’ll notice we just downloaded the Natty (11.04) kernel source. Needless to say that in case you’re using a different version of Ubuntu you’ll have to get the corresponding source tree.
The cloning process should take some time. When it’s done you’ll see that a new folder named source has been created. It’s contents are obvious, I believe.
The time for patching has come
First thing you should do is take a note of the exact kernel version your Ubuntu installation is currently using:
$ uname -r 2.6.38-8-generic $
After knowing the exact version string it was much easier for me to get the right patch for proper Reiser4 support. Make sure you also select the right patch for your own kernel version. Please note that Linux stable versions appear like flies and if you’re reading this article long after it’s written you will almost surely have to make some adjustments here and there.
$ mkdir patches $ cd patches/ $ wget http://www.kernel.org/pub/linux/kernel/people/edward/reiser4/reiser4-for-2.6/reiser4-for-2.6.38.patch.gz --2011-04-28 12:32:23-- http://www.kernel.org/pub/linux/kernel/people/edward/reiser4/reiser4-for-2.6/reiser4-for-2.6.38.patch.gz Resolving www.kernel.org... 149.20.4.69, 149.20.20.133 Connecting to www.kernel.org|149.20.4.69|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 609740 (595K) [application/x-gzip] Saving to: `reiser4-for-2.6.38.patch.gz' 100%[=========================================>] 609,740 271K/s in 2.2s 2011-04-28 12:32:26 (271 KB/s) - `reiser4-for-2.6.38.patch.gz' saved [609740/609740] $ gunzip reiser4-for-2.6.38.patch.gz $
Great! The patch is where it’s supposed to be, unpacked and ready to be incorporated into the Ubuntu kernel. Now, say we want to use another patch. This time it’s gonna be the CK patch by Con Kolivas, which, if used wisely, provides a great speed boost in multitasking scenarios. Again, be sure you download the right one for your particular kernel version.
$ wget http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.38/2.6.38-ck3/patch-2.6.38-ck3.gz --2011-04-28 12:36:33-- http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.38/2.6.38-ck3/patch-2.6.38-ck3.gz Resolving www.kernel.org... 149.20.20.133, 149.20.4.69 Connecting to www.kernel.org|149.20.20.133|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 73398 (72K) [application/x-gzip] Saving to: `patch-2.6.38-ck3.gz' 100%[=========================================>] 73,398 59.6K/s in 1.2s 2011-04-28 12:36:35 (59.6 KB/s) - `patch-2.6.38-ck3.gz' saved [73398/73398] $ gunzip patch-2.6.38-ck3.gz $
We’re now ready to apply the patches. Since we put them all in a single directory, it’s dead easy to do it.
$ cd ../source/ $ patch -p1 < ../patches/reiser4-for-2.6.38.patch $ patch -p1 < ../patches/patch-2.6.38-ck3 $
Get psyched for the configuration
Terrified? You shouldn’t be, as the configuration process is really easy. Supposing you’re not a customization maniac, the default config file of the standard Ubuntu kernel, i.e. the one describing the features of the currently running kernel, will suit you well. You’re going to make a few changes here and there, but when you’re using the default config as a base the whole process is pretty easy and totally safe.
$ cp -vi /boot/config-$(uname -r) .config $ make oldconfig $ make oldconfig HOSTCC scripts/basic/fixdep HOSTCC scripts/basic/docproc HOSTCC scripts/kconfig/conf.o HOSTCC scripts/kconfig/kxgettext.o SHIPPED scripts/kconfig/zconf.tab.c SHIPPED scripts/kconfig/lex.zconf.c SHIPPED scripts/kconfig/zconf.hash.c HOSTCC scripts/kconfig/zconf.tab.o HOSTLD scripts/kconfig/conf scripts/kconfig/conf --oldconfig Kconfig * * Restart config... * * * General setup *
If everything went well you will presented with the following question. Agree by pressing the [Y] key.
BFS cpu scheduler (SCHED_BFS) [Y/n/?] (NEW)
Immediately after that you’re going to be asked about the timer frequency. Go for a low one if you have a laptop or a not-so-powerful computer. I think my dual core desktop is powerful enough so I pressed [4], which is 1000MHz.
Timer frequency 1. 100 HZ (HZ_100) 2. 250 HZ (HZ_250_NODEFAULT) (NEW) 3. 300 HZ (HZ_300) > 4. 1000 HZ (HZ_1000) 5. 1500 HZ (HZ_1500) (NEW) 6. 2000 HZ (HZ_2000) (NEW) 7. 3000 HZ (HZ_3000) (NEW) 8. 4000 HZ (HZ_4000) (NEW) 9. 5000 HZ (HZ_5000) (NEW) 10. 7500 HZ (HZ_7500) (NEW) 11. 10000 HZ (HZ_10000) (NEW) choice[1-11?]:
Next, just press [Enter] to all the other questions until you see this one:
Reiser4 (EXPERIMENTAL) (REISER4_FS) [N/m/y/?] (NEW)
Instead of incorporating Reiser4 support into our kernel we choose to build it as a dynamically loadable module instead, so we press the [M] key. We don’t want debugging support for Reiser4, so in the following question we press [N].
Enable reiser4 debug mode (REISER4_DEBUG) [N/y/?] (NEW)
For any other question just select the default answer by pressing [Enter]. In case you’d like to configure any special options, you can also type
make menuconfig
Build it like a real builder
We’re almost done. We just have to build our new kernel, after all this hard work.
$ make-kpkg clean
exec make kpkg_version=12.036+nmu1 -f /usr/share/kernel-package/ruleset/minimal.mk clean
====== making target minimal_clean [new prereqs: ]======
This is kernel package version 12.036+nmu1.
test ! -f .config || cp -pf .config config.precious
test ! -e stamp-building || rm -f stamp-building
test ! -f Makefile || \
make ARCH=i386 distclean
make[1]: Entering directory `/home/gtklocker/dev/kernel/source'
CLEAN scripts/basic
CLEAN scripts/kconfig
CLEAN include/config include/generated
CLEAN .config .config.old
make[1]: Leaving directory `/home/gtklocker/dev/kernel/source'
test ! -f config.precious || mv -f config.precious .config
rm -f modules/modversions.h modules/ksyms.ver scripts/cramfs/cramfsck scripts/cramfs/mkcramfs
$
For the next command you’ll most definitely have to make two adjustments. First, give to the –append-to-version parameter a name you’d like your new kernel to have. Second, change the value of –overlay-dir to the full path of the directory the kernel source resides.
$ nice fakeroot make-kpkg --initrd --append-to-version=-lockard --overlay-dir=/home/gtklocker/dev/kernel/source kernel-image kernel-headers ... $
Be patient and wait for the compilation to finish. Depending on the raw processing power -or lack thereof- of your computer’s CPU, the whole process may take up to an hour or even more. In any case, a gazillion of messages will fly by your terminal window. Sit back, grab a slice of pizza or two and enjoy.
Installation
If everything went well -and I really can’t imagine a reason they don’t-, after the compilation finishes you’ll find a couple of DEB files in ~/dev/kernel.
$ cd .. $ ls -lh total 46M -rw-r--r-- 1 gtklocker gtklocker 7.4M 2011-04-28 17:17 linux-headers-2.6.38.4-ck3-lockard+_2.6.38.4-ck3-lockard+-10.00.Custom_i386.deb -rw-r--r-- 1 gtklocker gtklocker 39M 2011-04-28 17:12 linux-image-2.6.38.4-ck3-lockard+_2.6.38.4-ck3-lockard+-10.00.Custom_i386.deb drwxr-xr-x 1 gtklocker gtklocker 80 2011-04-28 12:39 patches drwxr-xr-x 1 gtklocker gtklocker 1.2K 2011-04-28 17:13 source $
Go ahead and install them:
$ sudo dpkg -i linux-*.deb [sudo] password for gtklocker: Selecting previously deselected package linux-headers-2.6.38.4-ck3-lockard+. (Reading database ... 133602 files and directories currently installed.) Unpacking linux-headers-2.6.38.4-ck3-lockard+ (from linux-headers-2.6.38.4-ck3-lockard+_2.6.38.4-ck3-lockard+-10.00.Custom_i386.deb) ... Selecting previously deselected package linux-image-2.6.38.4-ck3-lockard+. Unpacking linux-image-2.6.38.4-ck3-lockard+ (from linux-image-2.6.38.4-ck3-lockard+_2.6.38.4-ck3-lockard+-10.00.Custom_i386.deb) ... Done. Setting up linux-headers-2.6.38.4-ck3-lockard+ (2.6.38.4-ck3-lockard+-10.00.Custom) ... Examining /etc/kernel/header_postinst.d. run-parts: executing /etc/kernel/header_postinst.d/nvidia-common 2.6.38.4-ck3-lockard+ /boot/vmlinuz-2.6.38.4-ck3-lockard+ Setting up linux-image-2.6.38.4-ck3-lockard+ (2.6.38.4-ck3-lockard+-10.00.Custom) ... Running depmod. Examining /etc/kernel/postinst.d. run-parts: executing /etc/kernel/postinst.d/initramfs-tools 2.6.38.4-ck3-lockard+ /boot/vmlinuz-2.6.38.4-ck3-lockard+ update-initramfs: Generating /boot/initrd.img-2.6.38.4-ck3-lockard+ run-parts: executing /etc/kernel/postinst.d/nvidia-common 2.6.38.4-ck3-lockard+ /boot/vmlinuz-2.6.38.4-ck3-lockard+ run-parts: executing /etc/kernel/postinst.d/pm-utils 2.6.38.4-ck3-lockard+ /boot/vmlinuz-2.6.38.4-ck3-lockard+ run-parts: executing /etc/kernel/postinst.d/update-notifier 2.6.38.4-ck3-lockard+ /boot/vmlinuz-2.6.38.4-ck3-lockard+ run-parts: executing /etc/kernel/postinst.d/zz-update-grub 2.6.38.4-ck3-lockard+ /boot/vmlinuz-2.6.38.4-ck3-lockard+ Generating grub.cfg ... Found linux image: /boot/vmlinuz-2.6.38.4-ck3-lockard+ Found initrd image: /boot/initrd.img-2.6.38.4-ck3-lockard+ Found linux image: /boot/vmlinuz-2.6.38-8-generic Found initrd image: /boot/initrd.img-2.6.38-8-generic Found memtest86+ image: /memtest86+.bin done $
And now it’s one of those rare times where in order for the changes to take effect you need to reboot your Linux box. Do that and after the system comes up check the version of the running kernel:
$ uname -r 2.6.38.4-ck3-lockard+ $
Great! Also take a look into the directory where the active kernel’s filesystem modules reside:
$ ls -dlh /lib/modules/2.6.38.4-ck3-lockard+/kernel/fs/reiser* drwxr-xr-x 1 root root 20 2011-04-28 20:58 /lib/modules/2.6.38.4-ck3-lockard+/kernel/fs/reiser4 drwxr-xr-x 1 root root 22 2011-04-28 20:58 /lib/modules/2.6.38.4-ck3-lockard+/kernel/fs/reiserfs $
Congratulations! You’re now running your own, custom-made kernel.
Conclusion
All in all, the kernel building process is easy for everyone and does not require any sophisticated or even advanced skills. So, if you want a feature and there’s a patch for it, go ahead and recompile! In the worlds of Linux in particular and Open Source / Free Software in general, nothing is impossible ;)


this guide is fantastic! while not a total linux noob i didnt have a clue where to start and having natty and trying to follow a hardy guide wasnt going to well for me
OVERALL a great, up-to-date, detailed guide.
*bookmark* ;)
Great guide :) I’d like to ask a few things though:
1) Where’s a good place to get a list of patches like those you mentioned and why are they in the default kernel already – especially if they improve multitasking performance?
2) Why did you choose to build Reiser4 support into the kernel as a dynamically loadable module? That is to say, why M and not Y?
3) 1000Hz seems to be the value most users have chosen – but what happens if you opt for 100 or even 10000? Is 1000 the ‘safe bet’ on not breaking something? What could go wrong here? lol
Thanks again!
Thanks for your comment :)
1) I don’t think there really is a list of kernel patches. Individuals make them and reputation gets people to know them, but I’d bet there are some really good ones that even Linus Torvalds himself doesn’t know about. As for the inclusion or not of certain patches in the default kernel, there are many reasons. Reiser4, for example, is kinda unstable at this point (and I guess it will remain unmaintained for long because of its lead developer being imprisoned). As for bfs/ck, it’s highly possible to fail on older machines. Also, the distribution’s kernel maintainers can’t really know what CONFIG_HZ you want the kernel to be in, so it will either be high or low.
2) Well, if you look at the default configuration file you’ll see that all the filesystems are built as *modules*. This happens mostly because the filesystems are loaded exactly when there’s need to mount a partition. That being said, when you try to mount your /home partition in xfs, the xfs module is loaded and *then* the partition gets mounted. This is the way things work and it would be kind of stupid to not do it the standard way.
3) As I said, low frequencies lead to lower performance. Also 1000Hz is good enough for the most modern systems; this is why I like it. I wouldn’t advocate the use of bfs/ck on a laptop, though, as it’s not kind at all to battery. For more info read the BFS FAQ.
Happy hacking ;)
When I say BFS FAQ, I mean this:
http://ck.kolivas.org/patches/bfs/bfs-faq.txt
Well lower frequencies lead to higher latency, but better throughput. So if you don’t need the low latency it’s worth it to go with lower values for HZ.
Basically you don’t do context switches as often with lower HZ, if you’re running a webserver or something you probably don’t need low latency, but the boost in throughput might be helpful, depending on your workload.
the way i’ve been working it out (that has given me amazing performance boost already) is to do roughly 1000/no. of cores
for example my dual core is set at 300 (thats the closest)
actually the reason i was on this page again is for reference as i fiddle with the timings and such
That’s an interesting rule – and I’m glad you can back it up with real tests. Thanks for your feedback :)
well i was going to test it: is there a way to change the scheduler frequency without a full recompile?
None that I know of :S
Sorry, on 1), i meant aren’t*
Just a question, why would you use a filesystem that is not in the kernel mainline, that has nobody working on it anymore (since Hans Reiser was arrested for murder and his company sort of went away), for your backups?
Especially since you seem to be aware of some of the issues with reiser4…
As you said you made a mistake, I’m just curios what lead to it…
Concerning your first question, my first guess is something along the lines of “because you can” :) But besides that, the Reiser4 case was just an example. The purpose of this post was to demonstrate how one can re-compile his or her Linux kernel. The discussion on the kinds of patches a user may actually incorporate to the kernel is a totally different one! (And oh BTW, thanks for the idea you just gave me :D)
Il manque le paquet kernel-package, pour avoir make-kpkg.
Sorry but this is an English-speaking site, so you’ll have to translate your comment…
Sorry, I read english everyday and sometime I’m comfuse about what language I just read !
Just told you that it looks like the package kernel-package was missing on your apt command, in order to have the make-kpkg command.
Don’t worry about the language thing :) As for the kernel-package package, it’s included in the second apt-get command.
Another add:
Since I use my new custom kernel, eclipse (and some RCP tool like rssowl) didn’t work anymore.
java.lang.IllegalArgumentException: invalid qualifier: 4-ck3-core2+
Very nasty thing. I’m investigating…
Found that “Kernel release name with a ‘+’ causes Eclipse to fail to start with an “invalid qualifier” error”
Do you know how to suppress this +?
Hm, not really, no. I’ll have to talk to the author about this…
I had a problem with this line:
sudo apt-get -y install build-dep kernel-package linux-meta
The following worked for me:
sudo apt-get build-dep kernel-package linux-meta
You’re absolutely right, the “-y install” part has been put there by mistake. In reality, the build-dep command is all that’s needed to cause apt-get to install or remove all the packages necessary in an attempt to satisfy the build dependencies for a source package. I’m gonna correct the error right now – thanks again for pointing it out!
some one help me to install prism54 on natty.
i’ve found this prism54-cvs-latest.tar.bz2 .please help me on how to install.
I’m sorry my friend, but I don’t have time for tech support related questions :S
make-kpkg has useful option ‘-j’ for compilation parallelization (for example, I used -j4 for 4-core CPU)
nice guide! I was able to complete the whole building and installation of the custom kernel after two days. ;)
my setup was ubuntu 10.04(lts) upgraded the kernel to 2.6.38-13, then patch it with ck and reiser4. the only blocker i got was the initrd.img. i wasn’t able to find it after the compilation but executing update-initramfs -c -k made it. maybe we can add it in the steps >:)
Btw, thanks a lot!