Reinstalling grub after installing windows or after losing it

Recently my friend lost his grub after installing windows and he could not boot into Ubuntu. Long back we refered grub2 documentation from Ubuntu site and followed it to reinstall grub2. He asked me if I remembered it and I do

He asked me to post this as an article for his or anyone's easy reference and here it is for easy reference, if you lost your grub2 and unable to boot into any one of the following OS

 Ubuntu 10.04, Ubuntu 10.10, Linux Mint 9, Linux Mint 10, debian squeeze

His root ubuntu partition is in /dev/sda8 so I assume the reader of this article will take care to replace it with his root partition

Here are the steps to restore grub2 after your windows installation messes it up

Boot Ubuntu live cd (if you had installed 64 bit Ubuntu, then boot into 64 bit, otherwise use 32 bit, if u use a 32bit livecd to restore 64 bit or vice versa wont work!)

goto command line (use Applications --> Accessories --> Terminal or use Alt + F2 and gnome-terminal)

sudo mount /dev/sda<root partition number> /mnt

in my friends case it is sudo mount /dev/sda8 /mnt

sudo mount --bind /dev /mnt/dev

sudo mount --bind /dev/pts /mnt/dev/pts

sudo mount --bind /proc /mnt/proc

sudo mount --bind /sys /mnt/sys

sudo chroot /mnt

sudo update-grub

sync

grub-install /dev/sda

sync

exit

sync

sudo umount /mnt/sys

sudo umount /mnt/proc

sudo umount /mnt/dev/pts

sudo umount /mnt/dev

sudo umount /mnt

sudo reboot

Enterprise Kernel 6 has SSD TRIM support

Today morning I successfully compiled Red Hat Enterprise Kernel 6 from source into a deb. The kernel is running excellently in Ubuntu. As an added bonus, I found that it already has SSD TRIM support

Now that I have an enterprise kernel as a debian binary and running, I tested SSD TRIM functionality with EL6 kernel

SSD TRIM is a wear levelling feature and supported from kernel versions 2.6.33 and above in ext4 filesystems. Though Enterprise Kernel 6 is in version 2.6.32 it has this TRIM functionality out of box for ext4. I think it has been backported into EL6 (exciting ....) and now I have it in Ubuntu (wow wow wow)

This is how I tested it

Before testing the SSD partition should be mounted with discard option,

Mount the SSD partition with a discard option, that is after the default options of SSD partition in /etc/fstab, say / for root SSD partition in /dev/sda1, it should look like below



/dev/sdb1 /  ext4    errors=remount-ro,noatime,discard  0       1

Use hdparm to test if discard is actually using TRIM to clear the bits in hardware

Here is how

1. Create a temporary file inside the SSD partition as root. Important change the directory to the SSD partition, from home partition if home is a harddisk partition

dd if=/dev/urandom of=tempfile count=100 bs=512k oflag=direct

2. Get the device sector of the tempfile generated and copy the first sector information after value 0

hdparm --fibmap tempfile

In the above example, the sector info I copied for testing is 802816 this will be a different value for the tempfile you generate

3. Read the sector using hdparm before deleting. Here I used the value I copied from previous step and gave as input to hdparm to read the sector info to see if it is non zero. Change the number after --read-sector to the value you got for hdparm --fibmap tempfile, dont blindly give 802816. /dev/sdb is my SSD harddisk, so I used that at the end, replace with the correct harddisk name

hdparm --read-sector 802816 /dev/sdb

4. Remove the tempfile and sync

rm tempfile && sync

5. Retry step 3 and the values displayed should all be zeroes, if TRIM is in action

hdparm --read-sector 802816 /dev/sdb

Kudos, if you came to this level, you probably tried, if your kernel supported TRIM for your SSD or know that Red Hat Enterprise 6  Kernel has TRIM feature in place already!

If you thought Ubuntu Long Term Support 10.04, kernel 2.6.32 has TRIM already, sorry, it does not have it yet. But EL6 kernel has TRIM support for SSD and it rocks!!

I have built the kernel deb of EL6 from source and you can follow my earlier blog to build it yourself as I have not yet hosted it

RH Enterprise Linux 6 kernel in Ubuntu

I want to taste what an enterprise kernel will look like. I have downloaded the Red Hat Enterprise linux kernel source rpm and extracted the source code. I am compiling the kernel as a debian package. Will update in next article about my experiments with enterprise linux kernel running in Ubuntu (if it runs)

Want to build your own kernel from source for Ubuntu, then follow my earlier blog about kernel compilation in ten steps

Update 1: I successfully compiled the debian packages of enterprise linux 6 kernel in Ubuntu. I got some compilation errors, but I used/merged the different config given by enterprise linux directory and spec file and got rid of the compilation errors. After the correct config, the kernel built successfully. The config I used to build the el6 kernel is here

Kernel compilation in ten steps -- Ubuntu

Introduction: Last article I mentioned about how I did kernel compilation from source code. Here is a step by step how to.

This kernel compilation article will work for debian and debian based distros (and of course Ubuntu)

Step 1: Fetch kernel source

Browse http://www.kernel.org and choose a stable version and click full source

For e.g 2.6.34.7

To know more about the stability and features of a certain kernel version visit http://kernelnewbies.org  http://www.h-online.com and http://www.phoronix.com

Decide on the stable version based on your requirements and stability

Step 2: Install kernel build dependencies in debian/ubuntu

sudo apt-get install kernel-package libncurses5 libncurses5-dev


If you are a debian user, become root and use apt-get/aptitude directly to install kernel-package and libncurses5. After installing become normal user and follow steps below

Step 3:  Extract the source code

Unzip/untar the zip file of the kernel source code. Now enter into the directory of the extracted source code.

Step 4: Prepare config for kernel compilation

Step 4.1: make clean && make mrproper
Step 4.2: make menuconfig


This will open the kernel configuration, change/hack settings to your hearts content. By default make menuconfig copies your currently running kernel configuration. If you want a different configuration, copy that to .config file before launching make menuconfig. If you want to start fresh, use make defconfig before calling make menuconfig

Step 5: Make the kernel deb files after cleaning

Step 5.1: make-kpkg clean

Step 5.2:  do a build, based on your processor power, this will take some time. For example, my system compiles within 15 minutes since it is a quad core with hyperthreading. use CONCURRENCY_LEVEL=n, where n is the processor count

Core 2 duo/dual core processor owners
CONCURRENCY_LEVEL=2 fakeroot make-kpkg  --initrd --append-to-version=-shortdescription kernel_image kernel_headers


Quad core processor owners

CONCURRENCY_LEVEL=4 fakeroot make-kpkg  --initrd --append-to-version=-shortdescription kernel_image kernel_headers

Quad core with hyperthreading

CONCURRENCY_LEVEL=8 fakeroot make-kpkg  --initrd --append-to-version=-shortdescription kernel_image kernel_headers

Hexa core

CONCURRENCY_LEVEL=6 fakeroot make-kpkg  --initrd --append-to-version=-shortdescription kernel_image kernel_headers

Hexa core with hyperthreading

CONCURRENCY_LEVEL=12 fakeroot make-kpkg  --initrd --append-to-version=-shortdescription kernel_image kernel_headers

Once the compilation, linking is complete proceed to next step

Step 6: Install the debian packages

Change to previous directory, where the kernel packages are built as debian binary. There should be a header and a image deb file. Just use the following command to install the deb

cd .. && sudo dpkg -i *.deb

Step 7: Create a initrd

sudo update-initramfs -c -k <kernel version>

for e.g since I compiled 2.6.34.7, I used 

sudo update-initramfs -c -k 2.6.34.7

Now after this step, update your grub

Step 8: Update grub with new initrd

sudo update-grub

Step 9: Reboot

sudo reboot

Step 10: Enjoy!!

200 line linux patch backport to ubuntu 2.6.32

Today I tried to backport the 200 line linux patch for improving responsiveness

Here is the patch




diff -urN linux-2.6.32/Documentation/kernel-parameters.txt linux/Documentation/kernel-parameters.txt
--- linux-2.6.32/Documentation/kernel-parameters.txt    2010-10-31 16:13:19.000000000 +0530
+++ linux/Documentation/kernel-parameters.txt    2010-11-18 10:05:50.027037654 +0530
@@ -1618,6 +1618,9 @@
     noapic        [SMP,APIC] Tells the kernel to not make use of any
             IOAPICs that may be present in the system.


+    noautogroup    Disable scheduler automatic task group creation.
+
+
     nobats        [PPC] Do not use BATs for mapping kernel lowmem
             on "Classic" PPC cores.


diff -urN linux-2.6.32/drivers/char/tty_io.c linux/drivers/char/tty_io.c
--- linux-2.6.32/drivers/char/tty_io.c    2010-10-31 16:13:19.000000000 +0530
+++ linux/drivers/char/tty_io.c    2010-11-18 10:06:51.637609197 +0530
@@ -3029,6 +3029,8 @@
     put_pid(tsk->signal->tty_old_pgrp);
     tsk->signal->tty = tty_kref_get(tty);
     tsk->signal->tty_old_pgrp = NULL;
+
+    sched_autogroup_create_attach(tsk);
 }


 static void proc_set_tty(struct task_struct *tsk, struct tty_struct *tty)
diff -urN linux-2.6.32/include/linux/sched.h linux/include/linux/sched.h
--- linux-2.6.32/include/linux/sched.h    2010-10-31 16:13:19.000000000 +0530
+++ linux/include/linux/sched.h    2010-11-18 09:45:08.517579562 +0530
@@ -556,6 +556,8 @@
     spinlock_t lock;
 };


+struct autogroup;
+
 /*
  * NOTE! "signal_struct" does not have it's own
  * locking, because a shared signal_struct always
@@ -622,6 +624,10 @@


     struct tty_struct *tty; /* NULL if no tty */


+#ifdef CONFIG_SCHED_AUTOGROUP
+    struct autogroup *autogroup;
+#endif
+
     /*
      * Cumulative resource counters for dead threads in the group,
      * and for reaped dead child processes forked by this group.
@@ -1939,6 +1945,22 @@


 extern unsigned int sysctl_sched_compat_yield;


+#ifdef CONFIG_SCHED_AUTOGROUP
+extern unsigned int sysctl_sched_autogroup_enabled;
+
+extern void sched_autogroup_create_attach(struct task_struct *p);
+extern void sched_autogroup_detach(struct task_struct *p);
+extern void sched_autogroup_fork(struct signal_struct *sig);
+extern void sched_autogroup_exit(struct signal_struct *sig);
+#else
+static inline void sched_autogroup_create_attach(struct task_struct *p) { }
+static inline void sched_autogroup_detach(struct task_struct *p) { }
+static inline void sched_autogroup_fork(struct signal_struct *sig) { }
+static inline void sched_autogroup_exit(struct signal_struct *sig) { }
+#endif
+
+
+
 #ifdef CONFIG_RT_MUTEXES
 extern int rt_mutex_getprio(struct task_struct *p);
 extern void rt_mutex_setprio(struct task_struct *p, int prio);
diff -urN linux-2.6.32/init/Kconfig linux/init/Kconfig
--- linux-2.6.32/init/Kconfig    2010-10-31 16:13:19.000000000 +0530
+++ linux/init/Kconfig    2010-11-18 10:04:56.430328910 +0530
@@ -608,6 +608,19 @@


 endif # CGROUPS


+config SCHED_AUTOGROUP
+    bool "Automatic process group scheduling"
+    select CGROUPS
+    select CGROUP_SCHED
+    select FAIR_GROUP_SCHED
+    help
+      This option optimizes the scheduler for common desktop workloads by
+      automatically creating and populating task groups.  This separation
+      of workloads isolates aggressive CPU burners (like build jobs) from
+      desktop applications.  Task group autogeneration is currently based
+      upon task tty association.
+
+
 config MM_OWNER
     bool


diff -urN linux-2.6.32/kernel/fork.c linux/kernel/fork.c
--- linux-2.6.32/kernel/fork.c    2010-10-31 16:13:19.000000000 +0530
+++ linux/kernel/fork.c    2010-11-18 10:10:19.796963164 +0530
@@ -908,6 +908,8 @@


     tty_audit_fork(sig);


+    sched_autogroup_fork(sig);
+
     sig->oom_adj = current->signal->oom_adj;


     return 0;
diff -urN linux-2.6.32/kernel/sched_autogroup.c linux/kernel/sched_autogroup.c
--- linux-2.6.32/kernel/sched_autogroup.c    1970-01-01 05:30:00.000000000 +0530
+++ linux/kernel/sched_autogroup.c    2010-11-17 22:43:28.726509000 +0530
@@ -0,0 +1,140 @@
+#ifdef CONFIG_SCHED_AUTOGROUP
+
+unsigned int __read_mostly sysctl_sched_autogroup_enabled = 1;
+
+struct autogroup {
+    struct kref        kref;
+    struct task_group    *tg;
+};
+
+static struct autogroup autogroup_default;
+
+static void autogroup_init(struct task_struct *init_task)
+{
+    autogroup_default.tg = &init_task_group;
+    kref_init(&autogroup_default.kref);
+    init_task->signal->autogroup = &autogroup_default;
+}
+
+static inline void autogroup_destroy(struct kref *kref)
+{
+    struct autogroup *ag = container_of(kref, struct autogroup, kref);
+    struct task_group *tg = ag->tg;
+
+    kfree(ag);
+    sched_destroy_group(tg);
+}
+
+static inline void autogroup_kref_put(struct autogroup *ag)
+{
+    kref_put(&ag->kref, autogroup_destroy);
+}
+
+static inline struct autogroup *autogroup_kref_get(struct autogroup *ag)
+{
+    kref_get(&ag->kref);
+    return ag;
+}
+
+static inline struct autogroup *autogroup_create(void)
+{
+    struct autogroup *ag = kmalloc(sizeof(*ag), GFP_KERNEL);
+
+    if (!ag)
+        goto out_fail;
+
+    ag->tg = sched_create_group(&init_task_group);
+    kref_init(&ag->kref);
+
+    if (!(IS_ERR(ag->tg)))
+        return ag;
+
+out_fail:
+    if (ag) {
+        kfree(ag);
+        WARN_ON(1);
+    } else
+        WARN_ON(1);
+
+    return autogroup_kref_get(&autogroup_default);
+}
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg)
+{
+    int enabled = ACCESS_ONCE(sysctl_sched_autogroup_enabled);
+
+    enabled &= (tg == &root_task_group);
+    enabled &= (p->sched_class == &fair_sched_class);
+    enabled &= (!(p->flags & PF_EXITING));
+
+    if (enabled)
+        return p->signal->autogroup->tg;
+
+    return tg;
+}
+
+static void
+autogroup_move_group(struct task_struct *p, struct autogroup *ag)
+{
+    struct autogroup *prev;
+    struct task_struct *t;
+    struct rq *rq;
+    unsigned long flags;
+
+    rq = task_rq_lock(p, &flags);
+    prev = p->signal->autogroup;
+    if (prev == ag) {
+        task_rq_unlock(rq, &flags);
+        return;
+    }
+
+    p->signal->autogroup = autogroup_kref_get(ag);
+    __sched_move_task(p, rq);
+    task_rq_unlock(rq, &flags);
+
+    rcu_read_lock();
+    list_for_each_entry_rcu(t, &p->thread_group, thread_group) {
+        sched_move_task(t);
+    }
+    rcu_read_unlock();
+
+    autogroup_kref_put(prev);
+}
+
+void sched_autogroup_create_attach(struct task_struct *p)
+{
+    struct autogroup *ag = autogroup_create();
+
+    autogroup_move_group(p, ag);
+    /* drop extra refrence added by autogroup_create() */
+    autogroup_kref_put(ag);
+}
+EXPORT_SYMBOL(sched_autogroup_create_attach);
+
+/* currently has no users */
+void sched_autogroup_detach(struct task_struct *p)
+{
+    autogroup_move_group(p, &autogroup_default);
+}
+EXPORT_SYMBOL(sched_autogroup_detach);
+
+void sched_autogroup_fork(struct signal_struct *sig)
+{
+    sig->autogroup = autogroup_kref_get(current->signal->autogroup);
+}
+
+void sched_autogroup_exit(struct signal_struct *sig)
+{
+    autogroup_kref_put(sig->autogroup);
+}
+
+static int __init setup_autogroup(char *str)
+{
+    sysctl_sched_autogroup_enabled = 0;
+
+    return 1;
+}
+
+__setup("noautogroup", setup_autogroup);
+#endif
diff -urN linux-2.6.32/kernel/sched_autogroup.h linux/kernel/sched_autogroup.h
--- linux-2.6.32/kernel/sched_autogroup.h    1970-01-01 05:30:00.000000000 +0530
+++ linux/kernel/sched_autogroup.h    2010-11-17 22:39:46.575555000 +0530
@@ -0,0 +1,18 @@
+#ifdef CONFIG_SCHED_AUTOGROUP
+
+static void __sched_move_task(struct task_struct *tsk, struct rq *rq);
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg);
+
+#else /* !CONFIG_SCHED_AUTOGROUP */
+
+static inline void autogroup_init(struct task_struct *init_task) {  }
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg)
+{
+    return tg;
+}
+
+#endif /* CONFIG_SCHED_AUTOGROUP */
diff -urN linux-2.6.32/kernel/sched.c linux/kernel/sched.c
--- linux-2.6.32/kernel/sched.c    2010-10-31 16:13:19.000000000 +0530
+++ linux/kernel/sched.c    2010-11-18 10:01:01.377697053 +0530
@@ -76,6 +76,7 @@
 #include

 #include "sched_cpupri.h"
+#include "sched_autogroup.h"

 #define CREATE_TRACE_POINTS
 #include
@@ -346,19 +347,31 @@
 /* return group to which a task belongs */
 static inline struct task_group *task_group(struct task_struct *p)
 {
-    struct task_group *tg;

 #ifdef CONFIG_USER_SCHED
+    struct task_group *tg;
+
     rcu_read_lock();
     tg = __task_cred(p)->user->tg;
     rcu_read_unlock();
+
+    return tg;
 #elif defined(CONFIG_CGROUP_SCHED)
-    tg = container_of(task_subsys_state(p, cpu_cgroup_subsys_id),
-                struct task_group, css);
+    struct task_group *tg;
+     struct cgroup_subsys_state *css;
+
+    css = task_subsys_state(p, cpu_cgroup_subsys_id);
+
+    tg = container_of(css, struct task_group, css);
+
+    return autogroup_task_group(p, tg);
 #else
+    struct task_group *tg;
+
     tg = &init_task_group;
-#endif
+
     return tg;
+#endif
 }

 /* Change a task's cfs_rq and parent entity if it moves across CPUs/groups */
@@ -1826,6 +1839,7 @@
 #include "sched_idletask.c"
 #include "sched_fair.c"
 #include "sched_rt.c"
+#include "sched_autogroup.c"
 #ifdef CONFIG_SCHED_DEBUG
 # include "sched_debug.c"
 #endif
@@ -9558,6 +9572,8 @@
     init_task_group.parent = &root_task_group;
     list_add(&init_task_group.siblings, &root_task_group.children);
 #endif /* CONFIG_USER_SCHED */
+
+    autogroup_init(&init_task);
 #endif /* CONFIG_GROUP_SCHED */

 #if defined CONFIG_FAIR_GROUP_SCHED && defined CONFIG_SMP
@@ -10097,15 +10113,11 @@
 /* change task's runqueue when it moves between groups.
  *    The caller of this function should have put the task in its new group
  *    by now. This function just updates tsk->se.cfs_rq and tsk->se.parent to
- *    reflect its new group.
+ *    reflect its new group.  Called with the runqueue lock held.
  */
-void sched_move_task(struct task_struct *tsk)
+void __sched_move_task(struct task_struct *tsk, struct rq *rq)
 {
     int on_rq, running;
-    unsigned long flags;
-    struct rq *rq;
-
-    rq = task_rq_lock(tsk, &flags);

     update_rq_clock(rq);

@@ -10128,6 +10140,15 @@
         tsk->sched_class->set_curr_task(rq);
     if (on_rq)
         enqueue_task(rq, tsk, 0);
+}
+
+void sched_move_task(struct task_struct *tsk)
+{
+    struct rq *rq;
+    unsigned long flags;
+
+    rq = task_rq_lock(tsk, &flags);
+    __sched_move_task(tsk, rq);

     task_rq_unlock(rq, &flags);
 }
diff -urN linux-2.6.32/kernel/sysctl.c linux/kernel/sysctl.c
--- linux-2.6.32/kernel/sysctl.c    2010-10-31 16:13:19.000000000 +0530
+++ linux/kernel/sysctl.c    2010-11-18 10:03:11.533818737 +0530
@@ -379,6 +379,18 @@
         .mode        = 0644,
         .proc_handler    = &proc_dointvec,
     },
+#ifdef CONFIG_SCHED_AUTOGROUP
+    {
+        .ctl_name    = CTL_UNNUMBERED,
+        .procname    = "sched_autogroup_enabled",
+        .data        = &sysctl_sched_autogroup_enabled,
+        .maxlen        = sizeof(unsigned int),
+        .mode        = 0644,
+        .proc_handler    = proc_dointvec,
+        .extra1        = &zero,
+        .extra2        = &one,
+    },
+#endif
 #ifdef CONFIG_PROVE_LOCKING
     {
         .ctl_name    = CTL_UNNUMBERED, 

200 line linux kernel patch backport to 2.6.34.7

Today morning I read about 200 line linux kernel patch that does wonders here. I found that the patch is for 2.6.37 and increases responsiveness of desktop by 10 times max to 60 times average

It got my curiosity aroused and I wanted to see it in action.

I downloaded the patch from here and tried to  backport it to 2.6.34.7 (downloaded from here)

My maiden attempt to backport the patch succeeded and I compiled and installed the deb file

Here is the backport patch content



diff -urN linux-2.6.34.7/Documentation/kernel-parameters.txt linux/Documentation/kernel-parameters.txt
--- linux-2.6.34.7/Documentation/kernel-parameters.txt    2010-09-13 22:24:58.000000000 +0530
+++ linux/Documentation/kernel-parameters.txt    2010-11-17 22:49:10.082513013 +0530
@@ -1643,6 +1643,9 @@
     noapic        [SMP,APIC] Tells the kernel to not make use of any
             IOAPICs that may be present in the system.

+    noautogroup    Disable scheduler automatic task group creation.
+
+
     nobats        [PPC] Do not use BATs for mapping kernel lowmem
             on "Classic" PPC cores.

diff -urN linux-2.6.34.7/drivers/char/tty_io.c linux/drivers/char/tty_io.c
--- linux-2.6.34.7/drivers/char/tty_io.c    2010-09-13 22:24:58.000000000 +0530
+++ linux/drivers/char/tty_io.c    2010-11-17 22:38:38.750591065 +0530
@@ -3049,6 +3049,7 @@
     put_pid(tsk->signal->tty_old_pgrp);
     tsk->signal->tty = tty_kref_get(tty);
     tsk->signal->tty_old_pgrp = NULL;
+    sched_autogroup_create_attach(tsk);
 }

 static void proc_set_tty(struct task_struct *tsk, struct tty_struct *tty)
diff -urN linux-2.6.34.7/include/linux/sched.h linux/include/linux/sched.h
--- linux-2.6.34.7/include/linux/sched.h    2010-09-13 22:24:58.000000000 +0530
+++ linux/include/linux/sched.h    2010-11-17 21:40:45.286509459 +0530
@@ -514,6 +514,8 @@
     spinlock_t lock;
 };

+struct autogroup;
+
 /*
  * NOTE! "signal_struct" does not have it's own
  * locking, because a shared signal_struct always
@@ -580,6 +582,9 @@

     struct tty_struct *tty; /* NULL if no tty */

+#ifdef CONFIG_SCHED_AUTOGROUP
+    struct autogroup *autogroup;
+#endif
     /*
      * Cumulative resource counters for dead threads in the group,
      * and for reaped dead child processes forked by this group.
@@ -1904,6 +1909,21 @@

 extern unsigned int sysctl_sched_compat_yield;

+#ifdef CONFIG_SCHED_AUTOGROUP
+extern unsigned int sysctl_sched_autogroup_enabled;
+
+extern void sched_autogroup_create_attach(struct task_struct *p);
+extern void sched_autogroup_detach(struct task_struct *p);
+extern void sched_autogroup_fork(struct signal_struct *sig);
+extern void sched_autogroup_exit(struct signal_struct *sig);
+#else
+static inline void sched_autogroup_create_attach(struct task_struct *p) { }
+static inline void sched_autogroup_detach(struct task_struct *p) { }
+static inline void sched_autogroup_fork(struct signal_struct *sig) { }
+static inline void sched_autogroup_exit(struct signal_struct *sig) { }
+#endif
+
+
 #ifdef CONFIG_RT_MUTEXES
 extern int rt_mutex_getprio(struct task_struct *p);
 extern void rt_mutex_setprio(struct task_struct *p, int prio);
diff -urN linux-2.6.34.7/init/Kconfig linux/init/Kconfig
--- linux-2.6.34.7/init/Kconfig    2010-09-13 22:24:58.000000000 +0530
+++ linux/init/Kconfig    2010-11-17 22:48:16.986827765 +0530
@@ -614,6 +614,18 @@

 endif # CGROUPS

+config SCHED_AUTOGROUP
+    bool "Automatic process group scheduling"
+    select CGROUPS
+    select CGROUP_SCHED
+    select FAIR_GROUP_SCHED
+    help
+      This option optimizes the scheduler for common desktop workloads by
+      automatically creating and populating task groups.  This separation
+      of workloads isolates aggressive CPU burners (like build jobs) from
+      desktop applications.  Task group autogeneration is currently based
+      upon task tty association.
+
 config MM_OWNER
     bool

diff -urN linux-2.6.34.7/kernel/fork.c linux/kernel/fork.c
--- linux-2.6.34.7/kernel/fork.c    2010-09-13 22:24:58.000000000 +0530
+++ linux/kernel/fork.c    2010-11-17 22:35:27.942763006 +0530
@@ -886,6 +886,8 @@
     posix_cpu_timers_init_group(sig);

     tty_audit_fork(sig);
+    sched_autogroup_fork(sig);
+

     sig->oom_adj = current->signal->oom_adj;

diff -urN linux-2.6.34.7/kernel/sched_autogroup.c linux/kernel/sched_autogroup.c
--- linux-2.6.34.7/kernel/sched_autogroup.c    1970-01-01 05:30:00.000000000 +0530
+++ linux/kernel/sched_autogroup.c    2010-11-17 22:43:28.726509874 +0530
@@ -0,0 +1,140 @@
+#ifdef CONFIG_SCHED_AUTOGROUP
+
+unsigned int __read_mostly sysctl_sched_autogroup_enabled = 1;
+
+struct autogroup {
+    struct kref        kref;
+    struct task_group    *tg;
+};
+
+static struct autogroup autogroup_default;
+
+static void autogroup_init(struct task_struct *init_task)
+{
+    autogroup_default.tg = &init_task_group;
+    kref_init(&autogroup_default.kref);
+    init_task->signal->autogroup = &autogroup_default;
+}
+
+static inline void autogroup_destroy(struct kref *kref)
+{
+    struct autogroup *ag = container_of(kref, struct autogroup, kref);
+    struct task_group *tg = ag->tg;
+
+    kfree(ag);
+    sched_destroy_group(tg);
+}
+
+static inline void autogroup_kref_put(struct autogroup *ag)
+{
+    kref_put(&ag->kref, autogroup_destroy);
+}
+
+static inline struct autogroup *autogroup_kref_get(struct autogroup *ag)
+{
+    kref_get(&ag->kref);
+    return ag;
+}
+
+static inline struct autogroup *autogroup_create(void)
+{
+    struct autogroup *ag = kmalloc(sizeof(*ag), GFP_KERNEL);
+
+    if (!ag)
+        goto out_fail;
+
+    ag->tg = sched_create_group(&init_task_group);
+    kref_init(&ag->kref);
+
+    if (!(IS_ERR(ag->tg)))
+        return ag;
+
+out_fail:
+    if (ag) {
+        kfree(ag);
+        WARN_ON(1);
+    } else
+        WARN_ON(1);
+
+    return autogroup_kref_get(&autogroup_default);
+}
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg)
+{
+    int enabled = ACCESS_ONCE(sysctl_sched_autogroup_enabled);
+
+    enabled &= (tg == &root_task_group);
+    enabled &= (p->sched_class == &fair_sched_class);
+    enabled &= (!(p->flags & PF_EXITING));
+
+    if (enabled)
+        return p->signal->autogroup->tg;
+
+    return tg;
+}
+
+static void
+autogroup_move_group(struct task_struct *p, struct autogroup *ag)
+{
+    struct autogroup *prev;
+    struct task_struct *t;
+    struct rq *rq;
+    unsigned long flags;
+
+    rq = task_rq_lock(p, &flags);
+    prev = p->signal->autogroup;
+    if (prev == ag) {
+        task_rq_unlock(rq, &flags);
+        return;
+    }
+
+    p->signal->autogroup = autogroup_kref_get(ag);
+    __sched_move_task(p, rq);
+    task_rq_unlock(rq, &flags);
+
+    rcu_read_lock();
+    list_for_each_entry_rcu(t, &p->thread_group, thread_group) {
+        sched_move_task(t);
+    }
+    rcu_read_unlock();
+
+    autogroup_kref_put(prev);
+}
+
+void sched_autogroup_create_attach(struct task_struct *p)
+{
+    struct autogroup *ag = autogroup_create();
+
+    autogroup_move_group(p, ag);
+    /* drop extra refrence added by autogroup_create() */
+    autogroup_kref_put(ag);
+}
+EXPORT_SYMBOL(sched_autogroup_create_attach);
+
+/* currently has no users */
+void sched_autogroup_detach(struct task_struct *p)
+{
+    autogroup_move_group(p, &autogroup_default);
+}
+EXPORT_SYMBOL(sched_autogroup_detach);
+
+void sched_autogroup_fork(struct signal_struct *sig)
+{
+    sig->autogroup = autogroup_kref_get(current->signal->autogroup);
+}
+
+void sched_autogroup_exit(struct signal_struct *sig)
+{
+    autogroup_kref_put(sig->autogroup);
+}
+
+static int __init setup_autogroup(char *str)
+{
+    sysctl_sched_autogroup_enabled = 0;
+
+    return 1;
+}
+
+__setup("noautogroup", setup_autogroup);
+#endif
diff -urN linux-2.6.34.7/kernel/sched_autogroup.h linux/kernel/sched_autogroup.h
--- linux-2.6.34.7/kernel/sched_autogroup.h    1970-01-01 05:30:00.000000000 +0530
+++ linux/kernel/sched_autogroup.h    2010-11-17 22:39:46.575555847 +0530
@@ -0,0 +1,18 @@
+#ifdef CONFIG_SCHED_AUTOGROUP
+
+static void __sched_move_task(struct task_struct *tsk, struct rq *rq);
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg);
+
+#else /* !CONFIG_SCHED_AUTOGROUP */
+
+static inline void autogroup_init(struct task_struct *init_task) {  }
+
+static inline struct task_group *
+autogroup_task_group(struct task_struct *p, struct task_group *tg)
+{
+    return tg;
+}
+
+#endif /* CONFIG_SCHED_AUTOGROUP */
diff -urN linux-2.6.34.7/kernel/sched.c linux/kernel/sched.c
--- linux-2.6.34.7/kernel/sched.c    2010-09-13 22:24:58.000000000 +0530
+++ linux/kernel/sched.c    2010-11-17 22:04:34.054508727 +0530
@@ -77,6 +77,7 @@
 #include

 #include "sched_cpupri.h"
+#include "sched_autogroup.h"

 #define CREATE_TRACE_POINTS
 #include
@@ -311,13 +312,20 @@
 {
     struct task_group *tg;

+
 #ifdef CONFIG_CGROUP_SCHED
-    tg = container_of(task_subsys_state(p, cpu_cgroup_subsys_id),
-                struct task_group, css);
+     struct cgroup_subsys_state *css;
+
+    css = task_subsys_state(p, cpu_cgroup_subsys_id);
+
+    tg = container_of(css, struct task_group, css);
+
+    return autogroup_task_group(p, tg);
 #else
     tg = &init_task_group;
-#endif
+
     return tg;
+#endif
 }

 /* Change a task's cfs_rq and parent entity if it moves across CPUs/groups */
@@ -1935,6 +1943,7 @@
 #include "sched_idletask.c"
 #include "sched_fair.c"
 #include "sched_rt.c"
+#include "sched_autogroup.c"
 #ifdef CONFIG_SCHED_DEBUG
 # include "sched_debug.c"
 #endif
@@ -7738,7 +7747,7 @@
 #ifdef CONFIG_CGROUP_SCHED
     list_add(&init_task_group.list, &task_groups);
     INIT_LIST_HEAD(&init_task_group.children);
-
+    autogroup_init(&init_task);
 #endif /* CONFIG_CGROUP_SCHED */

 #if defined CONFIG_FAIR_GROUP_SCHED && defined CONFIG_SMP
@@ -8257,15 +8266,11 @@
 /* change task's runqueue when it moves between groups.
  *    The caller of this function should have put the task in its new group
  *    by now. This function just updates tsk->se.cfs_rq and tsk->se.parent to
- *    reflect its new group.
+ *    reflect its new group.  Called with the runqueue lock held.
  */
-void sched_move_task(struct task_struct *tsk)
+void __sched_move_task(struct task_struct *tsk, struct rq *rq)
 {
     int on_rq, running;
-    unsigned long flags;
-    struct rq *rq;
-
-    rq = task_rq_lock(tsk, &flags);

     update_rq_clock(rq);

@@ -8288,6 +8293,15 @@
         tsk->sched_class->set_curr_task(rq);
     if (on_rq)
         enqueue_task(rq, tsk, 0, false);
+}
+
+void sched_move_task(struct task_struct *tsk)
+{
+    struct rq *rq;
+    unsigned long flags;
+
+    rq = task_rq_lock(tsk, &flags);
+    __sched_move_task(tsk, rq);

     task_rq_unlock(rq, &flags);
 }
diff -urN linux-2.6.34.7/kernel/sysctl.c linux/kernel/sysctl.c
--- linux-2.6.34.7/kernel/sysctl.c    2010-09-13 22:24:58.000000000 +0530
+++ linux/kernel/sysctl.c    2010-11-17 22:45:38.558806407 +0530
@@ -354,6 +354,17 @@
         .mode        = 0644,
         .proc_handler    = proc_dointvec,
     },
+#ifdef CONFIG_SCHED_AUTOGROUP
+    {
+        .procname    = "sched_autogroup_enabled",
+        .data        = &sysctl_sched_autogroup_enabled,
+        .maxlen        = sizeof(unsigned int),
+        .mode        = 0644,
+        .proc_handler    = proc_dointvec,
+        .extra1        = &zero,
+        .extra2        = &one,
+    },
+#endif
 #ifdef CONFIG_PROVE_LOCKING
     {
         .procname    = "prove_locking",

Ubuntu Maverick 64bit Kernel Compilation

Introduction

After installing Ubuntu Maveric release candidate, I tried to play some movies. The movies were getting stuck. I tried to open windows and alt-tab windows. Everything seems to get stuck!

Why? The reason I believe lies in the way Ubuntu 64 bit kernel is built. 64 bit ubuntu kernel timer frequency is just 100 HZ

100 HZ is excellent for running inside cloud or virutalized environments, but 100 HZ  is unsuitable for desktop activity or games. Graphics drivers demand a very high timer frequency eventhough kernel has moved on to tickless. Nvidia proprietary kernel module requires 1000 HZ to play games or render graphics without a lag (this is based on my observation of how the movies lag though I have a very powerful processor i7 and a very powerful graphics card nvidia gtx 460).

So I wanted to get a kernel which has 1000 HZ kernel frequency. I did not find one and I am doing compilation myself.

I found the following blog extremely useful for kernel compilation

http://blog.avirtualhome.com/2010/07/14/how-to-compile-a-ubuntu-2-6-35-kernel-for-lucid/

Unfortunately, the article is using git and I don’t have patience to download 700+ MB of git content and package content just to compile a 88 MB kernel.  The answer to this comes from the same blog which explains kernel compilation without using git, but written for ubuntu intrepid which is nearly 2 years back

http://blog.avirtualhome.com/2008/10/28/how-to-compile-a-custom-kernel-for-ubuntu-intrepid/

The current method I used, fuses both the methods and compiles without using git.

Here is a how to of kernel compilation which I did exactly without using git for compiling. This may help someone with a similar problem, hence this document

This article only tells how to compile 64bit kernel and does not bother to explain 32 bit kernel compilation.

Step 1: Install all dependencies to compile kernel

The following command will install all the dependent packages needed for compiling a linux kernel.



sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge libncurses5 binutils-dev libelf-dev libdw-dev libnewt-dev libncurses5-dev



Install the following separately (this avoids pulling all recommended packages, which are huge)



sudo apt-get  --no-install-recommends install asciidoc xmlto



Ensure none of the dependent packages are missing by running the following command.

sudo apt-get build-dep linux

Step 2: Get the kernel source code

Create a directory called Builds and change into the created directory. This will make the task of handling the kernel builds pretty easy.

mkdir Builds
cd Builds

The following command helps the kernel source to be fetched and extracted in the current directory. In this case this is Builds directory. See that the command is executed as a normal user only (without sudo prefix)

apt-get source linux-image-$(uname -r)

Step 3: Create a copy before compiling

cp linux-2.6.35 linux -r

Always use the copy instead of the original source directory. In case there is an error and not recoverable, delete the directory linux and recreate it using above command.

Step 4: Create a new config

Change into the copied linux directory.

cd linux

Copy the generic flavour of the kernel and give the name of your processor for the new flavour. See that I am giving i7

cp debian.master/config/amd64/config.flavour.generic debian.master/config/amd64/config.flavour.i7

fakeroot debian/rules clean

Change the scripts permissions to make them executable

chmod -Rv +x debian/scripts/

fakeroot debian/rules updateconfigs

The following will bring up the kernel configuration screen for whichever configuration we give Y to edit. In this case I gave Y to config.flavour.i7

fakeroot debian/rules editconfigs

The following are the important configuration changes I made

• Changed
• Processor type and features --> Timer frequency --> 1000 HZ
• Processor type and features --> Processor family --> Core 2/newer Xeon
• Unticked(deselected)
• Processor type and features --> Support for extended (non-PC) x86 platforms
• Device drivers --> Staging drivers --> Nouveau (nVidia) cards

Step 5: Changing abi files before compilation

Copy the generic abi for the custom flavour. In this case it is as follows

cp debian.master/abi/2.6.35-22.34/amd64/generic debian.master/abi/2.6.35-22.34/amd64/i7
cp debian.master/abi/2.6.35-22.34/amd64/generic.modules debian.master/abi/2.6.35-22.34/amd64/i7.modules

Add i7 (or whatever flavour name we gave) to the getabis file under debian.master/etc/

sed -i s/getall\ amd64\ generic\ server\ virtual/getall\ amd64\ generic\ server\ virtual\ i7/g debian.master/etc/getabis

Add i7 (or whatever flavour name we gave) to amd64.mk file under debian.master/rules.d/

sed -i s/\=\ generic\ server\ virtual/\=\ generic\ server\ virtual\ i7/g debian.master/rules.d/amd64.mk

Make compiler aware of the flavour. change i7 to the flavour name we gave

cp debian.master/control.d/vars.generic debian.master/control.d/vars.i7

Edit the supported processor name in the above file. For e.g I changed to Core i7 from Generic

sed -i s/\"Generic\"/\"core\ i7\"/g debian.master/control.d/vars.i7

Step 6: Kernel Compilation

Since I wanted to optimize for core2 microarchitecture, I edited the following two files

arch/x86/Makefile

arch/x86/Makefile_32.cpu

In the file arch/x86/Makefile I made the following change

        cflags-$(CONFIG_MCORE2) += \

                $(call cc-option,-march=core2,$(call cc-option,-mtune=core2))

See the bolded mtune=core2, it was generic and I changed to core2

In the file arch/x86/Makefile_32.cpu I made the following change

cflags-$(CONFIG_MCORE2) += -march=core2 $(call tune,core2)

See the bolded march=core2, it was i686 and I changed to core2

Here is how I compiled for my flavour i7

Clean before building

fakeroot debian/rules clean

Independent package build

skipabi=true skipmodule=true fakeroot debian/rules binary-indep

Flavoured build

Build now, change CONCURRENCY_LEVEL=4 for quad cores and CONCURRENCY_LEVEL=2 for dual cores

CONCURRENCY_LEVEL=8 skipabi=true skipmodule=true fakeroot debian/rules binary-i7

Step 7: Installing newly built kernel

Change to previous directory, where all the build output as .deb files are created

cd ..

For e.g

sudo dpkg -i linux-headers-2.6.35-22-i7_2.6.35-22.35_amd64.deb linux-image-2.6.35-22-i7_2.6.35-22.35_amd64.deb

This will install all dkms modules which are already installed ( including nvidia proprietary driver if already installed from additional drivers)

restart (there should be a red colored power button in the top right corner of ubuntu, indicating that a restart is in due)

Enjoy