For a nice pdf of document of the same click here
Download links are provided at the bottom of this article along with SHA256SUMS
Again thanks to Linux Mint 11, I am enjoying it
Introduction
Download links are provided at the bottom of this article along with SHA256SUMS
Again thanks to Linux Mint 11, I am enjoying it
Introduction
I faced the following with earlier kernel builds
- Symlink errors while installing kernel debs in another system where the kernel build did not happen and lots of comments in the blog also reported the same
- DKMS modules did not work in different systems where the kernel build did not happen
- Cross compiling 32 bit kernels from my 64 bit PC was cumbersome
This kernel tries to address the above problems and give a kernel as similar to Ubuntu but with optimization. Towards the end of this document I have explained a step by step way of building the kernel Ubuntu way . Also I have made the same as a shell script which can be rerun again and again to build kernels Ubuntu way
This document is divided into two major sections, the first one is enough info about deb files and the second one is about how to build them
Info about the deb files
Requirements
This package is created in Linux Mint 11 and should install fine in all versions of Linux Mint 11 (as well Mint 10) and Ubuntu 11.04 (as well 10.10) (should as well in debian squeeze but not sure about dkms).
The build is done with march=core2 and mtune=native
This ensures the build will run on all core2 and above but will run with optimum speed on i3, i5 and i7 processors, so
The build is for 64bit i3,i5 and i7 processors
The following are the configuration settings
- Processor is selected as CORE2
- Makefile and Makefile_32.cpu modified to reflect march settings to core2 and mtune settings to native (my processor is i7 920)
- Configuration of oneiric kernel is taken as base, http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.0.1-oneiric/
- Kernel Timer Frequency is 300 HZ
How to install
To install
sudo dpkg -i linux-headers-3.0.1-030001_3.0.1-030001.201108060905_all.deb linux-headers-3.0.1-030001-i7_3.0.1-030001.201108060905_amd64.deb linux-image-3.0.1-030001-i7_3.0.1-030001.201108060905_amd64.deb
After this reboot into the newly installed kernel from grub (use update-initramfs in case of 10.04 or older if it is not building initramfs)
Kernel Source
I downloaded the kernel source tarball from kernel.org
Version: 3.0.1
Ubuntu Oneiric Patches
How to build yourself
This kernel follows Ubuntu way of building kernel and has all ubuntu oneiric patches
The following is a documentation of building it the Ubuntu way, I used my article about Ubuntu Maverick kernel compilation as a base for this, with some modifications. Here is a step by step of the same (and this explains the shell script commands which I created for my own reference)
Install dependencies for building Kernel
Here are the apt-get commands to install the dependencies required for building kernel the Ubuntu way
sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge libncurses5 binutils-dev libelf-dev libdw-dev libnewt-dev libncurses5-dev
sudo apt-get --no-install-recommends install asciidoc xmlto
Get the Kernel source and Ubuntu Patches
Create a directory for the building kernel source
#Enter into Home directory of the user
cd ~
#create a kernel directory if it is not there
mkdir -p kernel3.0.1
#enter into kernel directory
cd kernel3.0.1
The following wget commands is used to fetch the Kernel source and Ubuntu Patches
#get the kernel sources and ubuntu patches
#kernel 3.0.1
#Ubuntu Oneiric Patches from kernel ppa
Create a soft link of extracted kernel folder
#unpack linux kernel 3.0.1
tar xjf linux-3.0.1.tar.bz2
#create a soft link for the kernel directory
ln -s linux-3.0.1 linux
#enter into unpacked directory
cd linux
Install Ubuntu patches we downloaded earlier
#install the ubuntu/debian patches
patch -p1 < ../0001-base-packaging.patch
patch -p1 < ../0002-debian-changelog.patch
patch -p1 < ../0003-default-configs.patch
Create and configure a new kernel flavor, here I create it as i7
After the patches are applied, we come here to create a new flavor
Before this we need to give execute permissions to debian scripts and debian rules
Give execute permissions to debian/rules and debian/scripts
#make debian/rules as executable
chmod -Rv +x debian/rules
#make debian/scripts directory executable
chmod -Rv +x debian/scripts/
Copy existing generic flavor (or use existing flavor to create new flavor, here i7)
#create i7 flavor
cp debian.master/config/amd64/config.flavour.generic debian.master/config/amd64/config.flavour.i7
Configure the newly created flavor i7
#clean
fakeroot debian/rules clean
fakeroot debian/rules updateconfigs
When calling editconfigs, say n to avoid editing any other flavor except the created flavor, here i7. When a prompt asks us to edit i7 (or any other flavor you created) give y, this will open menuconfig of the kernel, feel free to change kernel configurations to hearts wish and dont forget to save while exiting the menuconfig screen. This is the place where I gave CORE2 as the processor type and changed kernel timer frequency to 300 HZ (or whichever you prefer)
fakeroot debian/rules editconfigs
Edit debian related abi information for newly created flavor, here i7
#copy the abi entries for i7 flavor and add i7 entries in vars and amd64.mk
cp debian.master/abi/3.0.0-7.9/amd64/generic debian.master/abi/3.0.0-7.9/amd64/i7
cp debian.master/abi/3.0.0-7.9/amd64/generic.modules debian.master/abi/3.0.0-7.9/amd64/i7.modules
sed -i s/getall\ amd64\ generic\ server\ virtual/getall\ amd64\ generic\ server\ virtual\ i7/g debian.master/etc/getabis
sed -i s/\=\ generic\ server\ virtual/\=\ generic\ server\ virtual\ i7/g debian.master/rules.d/amd64.mk
cp debian.master/control.d/vars.generic debian.master/control.d/vars.i7
sed -i s/\"Generic\"/\"core\ i7\"/g debian.master/control.d/vars.i7
Edit architecture dependent Makefiles to finetune gcc flags for compilation
Architecture dependent Makefiles for kernel compilation will reside in arch/
For x86 architecture, the following are the two Makefiles
arch/x86/Makefile and arch/x86/Makefile_32.cpu
#edit arch/x86/Makefile and arch/x86/Makefile_32.cpu if you want to tweak the march and mtune params of gcc
vi arch/x86/Makefile
vi arch/x86/Makefile_32.cpu
After opening the Makefile search for CORE2 and change march=core2 and mtune=native
In Makefile the following will be the change
cflags-$(CONFIG_MCORE2) += \
$(call cc-option,-march=core2,$(call cc-option,-mtune=native))
In Makefile_32.cpu the following will be the change
cflags-$(CONFIG_MCORE2) += -march=core2 $(call tune,native)
Build
Clean before building
#clean
fakeroot debian/rules clean
Build source and headers debian package which is independent of architecture or flavor, hence indep
#build headers/source deb files
skipabi=true skipmodule=true fakeroot debian/rules binary-indep
Set the CONCURRENCY_LEVEL for concurrent builds which will speed up the kernel compilation tremendously
#set the CONCURRENCY_LEVEL to the number of processor cores in system to do parallel compilation and thus speed up the kernel compilation
#`cat /proc/cpuinfo | grep "cpu cores" | head -1 | cut -d":" -f2 | cut -c2-` is used to get the number of cores
export CONCURRENCY_LEVEL=`cat /proc/cpuinfo | grep "cpu cores" | head -1 | cut -d":" -f2 | cut -c2-`
cat /proc/cpuinfo gives all info about cpu in the system
grepping for “cpu cores” gives multiple lines one fore each core
head -1 takes the first line of output
cut -d”:” -f2 uses : as delimiter and gives the second field, which is the number of corescut -c2- skips leading space
To test the CONCURRENCY_LEVEL, use echo $CONCURRENCY_LEVEL
Build the flavor
It is nice to time a build, just prefix time command before any other command, here the kernel build command. While calling debian/rules, we are invoking the newly created flavor which is binary-i7 in this case
#build the actual kernel flavor i7 (and time it)
time skipabi=true skipmodule=true fakeroot debian/rules binary-i7
Wait for some time, have a coffee, relax and wait for your system to finish build. My system took 20 minutes to finish the build
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/BuildKernel3.0.1.sh
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/config-3.0.1-030001-i7
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/linux-headers-3.0.1-030001_3.0.1-030001.201108060905_all.deb
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/linux-headers-3.0.1-030001-i7_3.0.1-030001.201108060905_amd64.deb
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/linux-image-3.0.1-030001-i7_3.0.1-030001.201108060905_amd64.deb
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/Readme.odt
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/Readme.pdf
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/SHA256SUMS
SHA256SUMS
1291a52bbcbb74f6b044171ae10d4efe399cff2a615ac03ff8b4ff2c64983e53 BuildKernel3.0.1.sh
202e4bef97a05f1df4c93fb6821036018f1a4ccd25745eb16f21aca02c8e6be8 config-3.0.1-030001-i7
7ef4a2f7e22335498535e980bfed3afe58999922120ff2ae735970994d1d1318 linux-headers-3.0.1-030001_3.0.1-030001.201108060905_all.deb
05ba8bc60980868ac299a473c832768e4d185b3764e00b9012527578449afc71 linux-headers-3.0.1-030001-i7_3.0.1-030001.201108060905_amd64.deb
48bb53f0e50b6da024f40189bf859d686daad6801b76464829737a69ac93583e linux-image-3.0.1-030001-i7_3.0.1-030001.201108060905_amd64.deb
f0cd271d2d4413a29b08963a9b9d99ef966513ecf815def2fed2c63a3638eb8b Readme.odt
e56ef8600f25f99ea6707d87b3a48f793327111d6579031779acc54a8e61ea7c Readme.pdf
Kernel 3.0.1 did not build dkms modules for 270 series nvidia drivers, I updated nvidia drivers to 280 series using the following x updates ppa
ReplyDeletesudo apt-add-repository ppa:ubuntu-x-swat/x-updates
sudo apt-get update
After this it built dkms modules for kernel 3 properly and I could boot with nvidia latest graphics drivers
After doing sudo apt-get update, install the updated nvidia drivers from update manager (here from mint update manager)
ReplyDeleteHi,
ReplyDeleteThis kernel compilation does ott work for me. Only black screen appears and ALT - SYSRQ + B solved it ;-)
vbox 4.1 did not compiled properly within linux-header.
let me know how i can investigate why it is not starting properly?
Martin
Lots of old sources or packages not ready for kernel 3.0 will fail. I faced a similar thing when broadcom did not work. Then I downloaded latest broadcom driver from packages.ubuntu.com in oneiric section (http://packages.ubuntu.com/oneiric/i386/bcmwl-kernel-source/download)
ReplyDeleteMaybe you need to download virtualbox 4.1 from oneiric repository and try. Also regarding black screen is due to nvidia or proprietary graphics driver not able to build with 3.0
Get the updated proprietary driver from oneiric before installing kernel 3.0
Old packages may expect kernel version to look like 2.6.x whereas new kernel is 3.0 or 3.0.1 and it may break their dependency check. The reason Fedora 15 opted to use 2.6.40 as its kernel version instead of 3.0 though there is no kernel 2.6.40 out there
ReplyDeleteHi,
ReplyDeleteMy pre-previous post (last day) has been lost somewhere again :-)
There is more information about installation of this kernel:
I am running now on ubuntu kernel 3.0.0-8 (3 independent instances) (and one ubuntu 3.0.0-7) with catalyst 11.7 without any problem (one frozen with 'stress' test).
Catalyst 11.7 has been properly compiled with your kernel and vbox 4.1 has been compiled properly partly. I mean with "linux-image..." compilation succeeded but NOT with "linux-headers...". So, the problem should not be the proprietary graphic driver and virtualbox does not mind me now even if I am strongly dependent on it.
Overview within your compiled kernel:
-------------------------------------
proprietary catalyst 11.7 graphic driver - compiled OK !!!
virtualbox 4.1 - "headers..." compilation failed
virtualbox 4.1 - "image..." compilation ok
but what is the main problem is that system is not going to start (after restarting), even if catalyst graphic driver has been compiled properly. So, it should not be problem within proprietary graphic driver - Please let me know if I am wrong and where?
And I do not know, if there are some log/s files to investigate what went wrong?
My intention is now to investigate and properly make compilation of kernel within steps described here. But if this compiled kernel is not working for me I would like to know how to compile the kernel to make it work
I have no clue where to look to start investigate to properly compile kernel.
If you have any suggestions , please I really much appreciate it to share knowledge.
Thanks,
Martin
Hi again,
ReplyDeleteI forgot to ask. Should I install new gcc/g++ compilators in ubuntu 11.04 to follow these steps for compilation of this kernel?
The original compilator is: gcc/g++ (Ubuntu/Linaro 4.5.2-8ubuntu4) 4.5.2
Thanks for answer,
Martin
Couple of minutes ago I sent comment - it had appeared and disappeared again ???
ReplyDeleteNo buddy, it was in spam, I checked and ticked it as not spam.
ReplyDeleteI used Linux Mint 11 and default gcc 4.5.2 (as my ubuntu went down due to no graphics and maybe btrfs)
If you are not going to use your compiled kernel away from your box, use make-kpkg way to compile (which is not the ubuntu way)
Can you remove catalyst and add it again. Sometimes nvidia drivers also does the same. Regarding dkms failures, you can see them in /var/lib/dkms/ folder
You should see catalyst folder and inside it some log file of your make file will be there
If you find errors, please share it, I can see if I missed something while building kernel
According to kernel ppa documentation here
ReplyDeleteinstalling kernel is done by
sudo dpkg -i *.deb
Maybe the order in which the kernel packages installed is not correct, maybe, I donno just guessing
Maybe if you try like above it may build vbox
Hi,
ReplyDeleteI have big trouble to post something here. It is not going through :-(
-- the order of installation is ok.
-- I cannot put logs here - it is refusing to do it.
-- Logs, for catalyst 11.7 and vbox 4.1 are ok. No errors.
I start to believe that there has to be something important in config within compilation kernel to make it work.
I have no clue where and what to investigate?
If you have any idea what logs could be important to find what went wrong, please let me know, maybe some log when it is starting the kernel modules or something.
Martin
Can you upload logs and refer me the links to the uploaded link (like dropbox public link)
ReplyDeleteIt is using the oneiric kernel with only 2 changes
Processor type changed to CORE2 and HZ to 300
I can try building with a configuration, say el6 configuration and upload for you to try
You can use
dmesg
command to read the logs
All logs will be stored in /var/log/messages
which requires su permission to open and read
Also, did you verify the SHA256SUM after downloading, maybe a corrupted download ?
Hi,
ReplyDeletelog files are here:
http://dl.dropbox.com/u/38489690/UBUEXT4_1/var/lib/dkms/fglrx/8.872/3.0.1-030001-i7/x86_64/log/make.log
http://dl.dropbox.com/u/38489690/UBUEXT4_1/var/lib/dkms/vboxhost/4.1.0/3.0.1-030001-i7/x86_64/log/make.log
(I installed this kernel on my old laptop with core 2 processor and nvidia proprietary driver and it works.)
I also installed tree different versions of oneiric kernel and none of them work.
Maybe it could be the oneiric patches which causes not to work.
I have a looked into logs by this command:
egrep -i '(3.0.1-030001-i7)' /var/log/*
but I did not find anything wrong.
and
egrep -i '(fail|denied|segfault|segmentation|reject|oops|warn)' /var/log/*
but I did not find anything relevant as far as I understand what informations meant.
dmesg found errors but related to unsuccessfully mount my btrfs partition.
I do not know what is el6 configuration but I am willing to do anything to reveal why oneiric kernel compilations does not work on my computer.
I have checked SHA256SUM for all three kernel files (even after installation) and they are fine.
thanks for any help,
Martin
Hi,
ReplyDeleteMay last post has been lost in your spam folder probably :-)
Martin
Yeah, checked it was in spam, ticked as not spam
ReplyDeleteSankaran,
ReplyDeleteLooking ar "cpu cores" gives the number of cores on the current processor, however most systems have more the one physical processor.
You can call 'nproc' to get the number of cores on the system or via 'grep -c processor /proc/cpuinfo'
Alex.
Very strange with procinfo! All i7 cpu's have 4 physical and 4 logical (hyperthreading) cores. Why then does cpuinfo and grepping, output only 4?
ReplyDeleteI think that exporting concurrency for 4 cpu's really slows things down (it's still fast I think but it would be better by exporting 7). I would pick 7, leaving one left over for the system. Any thoughts on this? Appreciate any feedback. Thanks.
Core I7 microprocessors actually have 8 cores. 4 physical and 4 logical. Will what is suggested here for the kernel compile, actually use all 8? I would prefer 7, leaving one hyperthread left for the system, and the rest for kernel compilation. Any thoughts on this?
ReplyDeleteThanks in advance.
BTW, this blog is probably THE only coherent, step by step guide to do such a thing! The "kernelcheck" utility comes close, but it's not nearly as thorough, with arch, mtune, flavors and the like. absolutely excellent. Thanks. Compiling as I type this :) BTW, this is a core i7-950 first-gen. Very, very fast!
Just wondering, where's the shell script you said you made? Can't find it here within these instructions?
ReplyDeleteThanks in advance.
Marsh
http://dl.dropbox.com/u/22900905/MyKernelBuilds/3.0.1-030001-i7/BuildKernel3.0.1.sh is the shell script
ReplyDeleteAll the scripts and the built deb are in the bottom of the article. Hope I did not miss any thing in haste
ReplyDeleteSomeone commented about using the following to get the count of cpu, here it is, use it to get the count (usually i use 7 on my i7 like you bingo, one for my browsing habits)
ReplyDeletecat /proc/cpuinfo | grep -c "cpu cores"
Recent find from gentoo, use -pipe in compiling to speed up if you have lots of RAM, how great
ReplyDeleteI know about that and did that as you instructed but my machine still only reports a 4 cpu machine, when in fact, the i7-950 cpu is a hyperthreaded chip, making 4 physical and 4 logical cores. Interesting that cpuinfo would return 4, but the system monitor reports 8!!! Right?
ReplyDeleteThanks for the script. I can tweak the hell out of my kernel, stripping it of all the junk, like wireless cards and Adaptec SCSI cards. Eggcellent. Thanks again Sankaran. BTW, where did you learn how to do all of this? I'd love to really master the Linux command shell and system, any hints at what books I should study? Thanks in advance.
ReplyDeleteActually I am a newbie in kernel compiling and not mastered anything.
ReplyDeleteThere is no one book. Read as many as possible, as for me, I am a heavy surfer and tried various ways of building kernel in Fedora, Debian, Arch, Gentoo and now Ubuntu
The best resource would be Arch Linux or Gentoo
Gentoo is meant for building everything from source, I have tried Gentoo four times and failed three times. It is fun if you can spend a lots of time. System will run extremely fast, but even to get firefox, it will start building, so I moved away from Gentoo to Ubuntu and Windows
ReplyDeleteThe only other linux which comes close or even better than Ubuntu is Archlinux. There everything is provided as binary, but building is just a step away and it is very easy. Here again the problem is internet speed in India is poor and I dont/cant wait get every source with the horrible broadband speed.
I set up Fedora 8 or Fedora 7 with all the rebuilt kernels and one day when I was sleepy, I booted into live cd and was testing, I erased 500 GB fedora partition thinking it is my pendrive which was 4 GB. After that I lost fedora completely and did not get inspiration to go again with Fedora as setting Fedora is not as easy as Ubuntu. lol
Debian is as good or better than Ubuntu, but I am impatient to wait for new software. Linux mint I love very much except the fact it brings me nausea of green color everywhere
you are awsome man i just did a newb install and it already runs great on ubuntu 11.04
ReplyDelete