Adaptive compilation for for-functions #2126

The issues

There are two intertwined issues in the way IronPython's interpreter/compiler handles functions with a for statement.

Issue 1: _forceCompile with LightLambdaExpression

The first one is about handling of _forceCompile in ScopeStatement. My expectation is that it forces a function (or any scope that has this field set to true) into immediate compilation on the first call, never trying to run it through the interpreter. This is supported by comments in the code as well as the DLR, which also has _forceCompile meaning exactly that. The issue is that _forceCompile in IronPython does not always force immediate compilation: it does so if the code is in a form of LambdaExpression. But if the code is LightLambdaExpression, Compile means adaptive compile with the hard-coded default call threshold. The DLR defines the default to be 32 (what a round number) so a function with for loops, despite being set with _forceCompile, is still interpreted for the first 32 calls, like any other function. Each loop inside it is also compiled after 32 iterations, so a hot loop is compiled during the first call.

The issue is broader than _forceCompile: the same code path is taken whenever ScopeStatement.ShouldInterpret is false, and -X NoAdaptiveCompilation makes it false for every function, not just ones with for. So with that option, no function is compiled up front.

Issue 2: Adaptive compilation for for-functions or not

The second issue is that I do not see any reason for functions with for loops to avoid interpreter altogether. Those functions have been interpreted for years and nobody ever noticed any negative side effects. Sure, the interpreter of for loops can be optimized further, but even now it works quite okay. When interpreted, a for loop is still slightly faster than a while loop. OTOH, a compiled for is noticeably faster than a compiled while, so the gain from compilation is more pronounced for for loops. Nevertheless, to always force-compile functions with for would increase the startup time by about 10 to 20%.

I couldn't find any reasoning in the existing code for the status quo: it happened before the first git commit of IronPython to the main repo (now archived). I guess that in the old times, there were some inefficiencies in the interpreter for for loops (or maybe outright roadblocks) so the Ancients decided to force compilation and forego the adaptive compilation. If it was so, it is no longer the case.

Both issues together

So the two issues sort-of cancel each other out. Perhaps that's why for-loop containing functions flew under the radar when #2113 was being investigated. _forceCompile is disregarded by the bug. But the problem appears when options -X CompilationThreshold and -X NoAdaptiveCompilation are used. For instance, setting -X CompilationThreshold=2147483647 makes all functions always interpreted except functions containing for, which get compiled after 32 calls. Setting -X CompilationThreshold=0 makes all functions never interpreted, except functions containing for, which still get compiled after 32 calls. Setting -X NoAdaptiveCompilation compiles no function up front; all are interpreted with threshold 32.

Summary of current behaviour

How a function's delegate looks after its first call:

"interpreted, N" means that the function is compiled to IL after N calls. Each loop inside it is also compiled separately after N iterations.
Bold marks results that contradict the option.
Imported modules are not affected by -X NoAdaptiveCompilation (but is it correct behaviour?)
Functions defined by exec behave exactly like the imported-module columns.
Traced functions are build from LambdaExpressions, so Issue 1 does not affect them.