Code Race, Virtual Machine vs Native Execution

For two reasons I wrote here much less than I expected, the first one is that I had a pretty stressing semester, the second, and more important, is that I want to have good texts and not many texts here, so I will only write when I have something to say.

For Easter I bought 2 books, Code Complete 2 and CLR via C# third edition. Since they are pretty different (the first is more general and high level than the second), I am reading them in parallel. Code Complete is extremely interesting, but I am at the beginning, so I (still) do not have too much to say about it, vie CLR via C# really get directly to the point, so I may have something to say.
I have always heard that code that uses virtual machines (like Java or C#) are slower than code that is directly compiled to native code (C++, for instance). Jeffrey Richter, the guy who wrote the CLR via C# book, may disagree, and he has some interesting points about it, but before I get to it, I would like to explain how the 2 phase compilation happens in the .NET Framework.

Once you write your code and click "Compile", the compiler will compile your code in IL code (Intermediate Language). This is a language that is used by the CLR to generate the native code at runtime. The generation of native code happens only once per execution, in the first time the method is called, then the native code is stored in memory and is used in all other method calls.
It is clear that in the first time a method is called, there will be a cost for runtime compiling. After that, it seems that the code generated by the CLR is the same as generated by a C++ compiler. But that is wrong. Because of the fact that the CLR compile happens on runtime, it has much more information about the hardware that the C++ compiler, it can make better tuning in the native code, taking advantage of this knowledge.

Some examples given in the book:

• The CLR Compiler (also called JIT – Just In Time – Compiler) can detect that the code is running on a specific CPU and use special instructions offered by this CPU.

• The compiler can detect that some tests are always true or always false and avoid doing it. For instance: if (numberOfCPUs > 1). It won’t change during execution, it doesn’t have to be tested, and if it is always false, the inner code of this if statement does not have to be compiled.

• The compiler can recompile the IL code into native code to correct some wrong branch predictions, depending on observed execution patterns. (The current version does not do that, but in the future they might).

I had never thought about it. For me, they seem very plausible and strong arguments to believe that code that uses VMs CAN have better performance than code compiled direct to native code, but it always depend on how well the VM is built, of course. The first argument is the most interesting, since a C++ compiler has to use the lowest-common-denominator CPU Instruction set, it cannot tune the native code, so knowing the CPU can bring a huge enhancement.

I am pretty convinced that it is true, that a VM execution can be at least as fast as a native code execution. Unfortunately testing that can be pretty hard, since many things can speed-up or slow-down an execution (for instance, how well the code is written).

And you, what had you ever thought about it?
Do you think it can be true or is it only marketing?


Ps: The .Net framework has a tool to generate native code in a separate file and to use it, to avoid compilation at runtime, called NGEN.exe. But it has its issues and you should use it only when you really need it. It will avoid the JIT compiler, but you will still need the CLR to execute your assembly.