Home page of this blog

Sunday, October 14, 2012

Build script for kernel 3.6.2 optimized native for Ubuntu

Here is the script to build ubuntu (debian) packages optimized for the native processor

https://docs.google.com/open?id=0B_nTbpzPbbNJU0h1RzFLMG1lR2s


The following tries to dissect the script and how it works

Build dependencies

The portion below is used to install build environment in ubuntu, if the following are already installed, script just does not install it again(thanks to apt)



sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge libncurses5 binutils-dev libelf-dev libdw-dev libnewt-dev libncurses5-dev

sudo apt-get install flex

sudo apt-get install bison

sudo apt-get  --no-install-recommends install asciidoc xmlto


Downloading kernel source and patching it with ubuntu patches to build ubuntu package



mkdir -p ~/Builds/Kernel362 
cd ~/Builds/Kernel362 


wget -c http://www.kernel.org/pub/linux/kernel/v3.0/linux-3.6.2.tar.bz2
wget -c http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.6.2-quantal/0001-base-packaging.patch
wget -c http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.6.2-quantal/0002-debian-changelog.patch
wget -c http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.6.2-quantal/0003-default-configs.patch

rm -rf linux-3.6.2
rm -rf linux

tar xvjf linux-3.6.2.tar.bz2
ln -s linux-3.6.2 linux

cd linux

patch -p1 < ../0001-base-packaging.patch 
patch -p1 < ../0002-debian-changelog.patch 
patch -p1 < ../0003-default-configs.patch




Creating new flavor and giving execute permissions to debian build scripts



chmod -Rv +x debian/rules

chmod -Rv +x debian/scripts/

cp debian.master/config/amd64/config.flavour.generic debian.master/config/amd64/config.flavour.native

fakeroot debian/rules clean
fakeroot debian/rules updateconfigs




Changing kernel config based on stable config



#copy existing config from rock stable 12.04 kernel using uname -r, do not save it in .config, as this file is heavily modified when calling editconfigs
cp /boot/config-`uname -r` stableconfig
#cp /boot/config-3.2.0-31-generic stableconfig

#give y to native config
#Load alternate config and choose stableconfig which is untouched by editconfigs, save alternate config and save .config after changing processor type and kernel frequency
fakeroot debian/rules editconfigs





This step needs a bit more explanation on how to base kernel config based on stable config, to do this I just put screenshots explaining briefly how this is done

When the script is running and comes to asking if we want to edit config, choose Y for native flavor

see below (I am giving n for every other flavor)


When the kernel config screen comes, scroll down and select Load an Alternate Configuration File


In the dialog, erase .config using backspace and type stableconfig



Following screens show, that I change processor type and kernel frequency, the old and new

Processor type old

Processor type new



Kernel frequency old


Kernel frequency new


After finishing all configuration changes, do not just exit!

Scroll down to Save an Alternation Configuration File


In the popped up dialog, erase stableconfig and type .config


Now choose exit in the main configuration screen


ABI  entries for new flavor and makefile changes



ABI_DIR=`ls debian.master/abi/`

cp debian.master/abi/$ABI_DIR/amd64/generic debian.master/abi/$ABI_DIR/amd64/native
cp debian.master/abi/$ABI_DIR/amd64/generic.modules debian.master/abi/$ABI_DIR/amd64/native.modules

sed -i s/getall\ amd64\ generic/getall\ amd64\ generic\ native/g debian.master/etc/getabis
grep native debian.master/etc/getabis

sed -i s/\=\ generic/\=\ generic\ native/g debian.master/rules.d/amd64.mk
grep native debian.master/rules.d/amd64.mk

cp debian.master/control.d/vars.generic debian.master/control.d/vars.native
sed -i s/\"Generic\"/\"core\ native\"/g debian.master/control.d/vars.native
grep native debian.master/control.d/vars.native


sed -i s/\=\ gcc/\=\ gcc\ \-march\=native\ \-pipe/g Makefile
sed -i s/\=\ g++/\=\ g++\ \-march\=native\ \-pipe/g Makefile
grep "march=native" Makefile


sed -i s/core2/native/g arch/x86/Makefile
sed -i s/core2/native/g arch/x86/Makefile_32.cpu
grep native arch/x86/Makefile_32.cpu
grep native arch/x86/Makefile





you can see above, I have given march=native you can choose whichever is your processor architecture

Building headers and kernel flavor



fakeroot debian/rules clean
skipabi=true skipmodule=true fakeroot debian/rules binary-headers
time skipabi=true skipmodule=true no_dumpfile=yes fakeroot debian/rules binary-native

The above builds kernel headers and kernel image for our new flavor

Just install the newly built packages using 

sudo dpkg -i *.deb






Core i7 optimized Ubuntu Kernel 

Kernel Headers All
Kernel Header corei7
Kernel Image


SHA256SUMS of the above in order


537e87a7c3200aab82624810526d25031ffd5093f406b32bc0f2e053e6213477  linux-headers-3.6.2-030602_3.6.2-030602.201210121823_all.deb

adf8fc79df2974622762f7e7ff84b8804f0b91f4927a70e3115d6326a2872bc2  linux-headers-3.6.2-030602-corei7_3.6.2-030602.201210121823_amd64.deb

001b05ffd6377b1f18d58704f850128f6b73525a9fb5c2744c30583b84b639e0  linux-image-3.6.2-030602-corei7_3.6.2-030602.201210121823_amd64.deb

12 comments:

  1. I do not understand what you are doing here. What exactly is being done here ?

    ReplyDelete
    Replies
    1. Trying to explain the script which I have created, well not everyone could understand script ...

      Delete
    2. OH OK. Great!

      I still dont understand what/how each individual command works. I just copy whatever you have written here. But its more fun to copy from an explanatory post, rather than just executing a script. :)

      Delete
  2. Interesting article on Phoronix :

    http://www.phoronix.com/scan.php?page=article&item=gcc_47_optimizations&num=1

    and the related forum discussion. A few posts mention some "low jitter" kernels etc.

    http://phoronix.com/forums/showthread.php?74409-The-Performance-Between-GCC-Optimization-Levels

    ReplyDelete
  3. I remember you always built with i7 optimized scripts. I still have the old ones and I can certainly just change a few things here and there to make it work right? What happened to you did you get rid of you i7 processor or does this work just as well? Also I remember you used to boost the Kernel frequency to 1000mhz have you found that not to be good anymore?

    Thanks,
    Rob

    ReplyDelete
    Replies
    1. Yeah, I used to build i7 optimized kernels, but I am building kernels using native optimization, so I thought I will not share the built debs.

      I have not sold my i7 system, I am extracting every ounce of its power by building, running multiple OS, using virtualization, watching high def, streaming, downloading, playing games ...

      You can use this script as is without changing anything!!, because native optimization chooses the best optimization flags for your system and should give you much better kernel for your system (and only your system)

      1000HZ I feel is best for performance, but I want best performance and long life, so I won't go down to 100 HZ, nor 250 HZ, best compromise is 300 HZ just following the arch linux way!

      Delete
    2. Native means that the GCC compiler will detect youe exact CPU and choose the optimizations that are based on that specific CPU architecture. This is the best method and will build the fastest possible kernels.

      Delete
    3. Sorry I haven't checked back in awhile. Thanks for the reply and explanations Sankaran and mayankleoboy1. excellent.

      Thanks again!

      Delete
  4. I run your script, but it always hangs giving me the following error:

    time: cannot run skipabi=true: No such file or directory
    Command exited with non-zero status 127
    0.00user 0.00system 0:00.00elapsed ?%CPU (0avgtext+0avgdata 368maxresident)k
    0inputs+0outputs (0major+79minor)pagefaults 0swaps

    anything I sould change in the script?

    thanks for your help

    A

    ReplyDelete
    Replies
    1. cSorry to hear it hangs for you, which version of Ubuntu are you using, can you give a bit more details?

      Could you also remove time from the last line in the script and run it directly, it may work

      change last line to look like below and try

      skipabi=true skipmodule=true no_dumpfile=yes fakeroot debian/rules binary-native

      Delete
    2. I am on Quantal
      your suggestion worked and I could compile my kernel succesfully
      with a couple of edits in the script I could use it also to compile 3.6.3
      Thank you very much, it is a great script!

      Delete