Benchmark of Windows 7 vs Ubuntu comparision may be flawed under uneven environments

I read the following interesting benchmark between Windows 7 vs Ubuntu 10.04

http://www.phoronix.com/scan.php?page=article&item=linux_windows_part1&num=1

I agree, for games Windows 7 is one of the best, but still

The benchmarks may be  flawed due to the following assumptions which is not ideal setup for Ubuntu

Reasons

1. Compiz is enabled by default in Ubuntu, whereas the games in Windows disable Aero to gain that extra FPS. I doubt if the games in Ubuntu  disable compiz before running

2. Biggest flaw is cpufrequency is controlled by Ubuntu to run as ondemand instead of utilizing full processor power. The ideal Ubuntu Setup would be to run all the cores with cpufrequency governor performance. Windows 7 by default runs in full gear and kudos to it. But that does not answer this question, why the benchmarks were done when running ondemand governor in Ubuntu Lucid?

How do I assume cpufrequency has something to do with performance, well I will try to prove it ( though you may try for yourself by changing cpufrequency to run with different performance governors and check simple benchmarks)

I run three benchmarks on google-chrome when cpufrequency is running ondemand and ran the same when cpufrequency is performance (you can see the result visually)

First screenshot shows cpufrequency is 1.60 GHz though core i7 920 frequency is 2.66 GHz due to ondemand governor

See the line

 current CPU frequency is 1.60 GHz (asserted by call to hardware)


this is due to ondemand governor

Under ondemand governor (all 4 cores (8 threads of core i7) running under ondemand governor)

By default ubuntu runs with ondemand governor, but to change it manually use

for i in 0 1 2 3 4 5 6 7; do sudo cpufreq-set -c $i -g ondemand; done

here change 0 1 2 3 for quad core and 0 1 for core 2/atom

[Ubuntu by default does not have cpufrequtils installed, to install it click the link

cpufrequtils

or goto command line and install it using the following command

sudo apt-get install cpufrequtils

cpufrequtils provides the commands cpufreq-set and cpufreq-info]

Google v8 benchmark result  is 5622 (higher is better)
Sunspider benchmark score is 443.6 milliseconds (lower is better) Sunspider benchmark results detailed link
Peacekeeper benchmark score is 8129 (higher is better)

Images of results under ondemand governor

Google v8

Sunspider

Peacekeeper Browser Benchmark (rank and in detail)

Under performance governor (all 4 cores (8 threads of core i7) running under performance governor)


To change all cores to performance I used the following commandline


for i in 0 1 2 3 4 5 6 7; do sudo cpufreq-set -c $i -g performance; done

here change 0 1 2 3 for quad core and 0 1 for core 2/atom

Google v8 benchmark result  is 5641 (higher is better)

Sunspider benchmark score is 323.3 milliseconds (lower is better) Sunspider benchmark results detailed link

Servicemark peacekeeper browser benchmark score is 8365 (higher is better)

Images of results under performance governor

Google v8

Sunspider

Peacekeeper Browser Benchmark (rank and in detail)

Here are charts I created for easy viewing of the comparision when running in ondemand and performance

3. By default gnome/Ubuntu is bent towards power saving like spinning down hard disk and cpu frequency, whereas windows 7 professional is geared towards performance

4. The kernel used by Ubuntu 64 bit is a fancy 100 Hz kernel which is unsuitable for any game with high frame rate. Why not use a different kernel suitable for game? Maybe it should have used preempt kernel which is 1000 Hz with kernel preemption or an rt kernel(http://packages.ubuntu.com/lucid/linux-image-2.6.32-21-preempt) linux-preempt

Ask any gamer, he would undergo the pain to recompile kernel even above 1000Hz for getting high FPS and he would not use a 100 Hz kernel