meetings/2024/LDM-2024-04-01.md
async void"https://github.com/dotnet/runtimelab/blob/e69dda51c7d796b8122d0f55b560bc44094a4bec/docs/design/features/runtime-handled-tasks.md
Presentation
Today, we looked at a proposal for improving async in .NET. The runtime has been experimenting with several different approaches here. Their first prototype, green threading, was designed to try
and solve the metastability issue where sync-over-async can cause threadpool exhaustion. While this was an interesting experiment, their ultimate conclusion was to not move forward with this
experiment, and instead focus on improving the overall performance of async methods; their results can be viewed here.
To that end, we're looking at what language or compiler changes may be required if we were to update the async state machine generation. For the purposes of discussion, we refer to this new format as
async2; this does not mean that we expect to introduce a literal async2 keyword, it's simply a shorthand. The current runtime experiment moves the state machine generation directly into the runtime,
rather than having the C# compiler do it.
This can be done entirely without breaking changes in the behavior of code, but there is one major concern; SyncContexts and AsyncLocals. The saving of the execution context today ensures that, when
an async method returns, its modifications to an AsyncLocal are not observed by callers, because of the compiler's saving and restoring of execution contexts. It is possible to have the async2 machinery
behave in the same way as async here, but doing so is a potential perf hit, of ~30% or more of the gain in some cases. These scenarios will still be faster than the existing async mechanisms, but if we
can take a breaking change here, we can make the performance gains with async2 even greater. There's some argument that the current behavior of these methods is actually very unexpected, but even if
that's the case, it's a potential break. One idea that was floated repeatedly was making this configurable; we could opt for behavior-preserving semantics by default, and let users opt-in to the breaking
change if they chose to do so.
Another point we considered is how to trigger the new generation strategy. The current prototype adjusted the compiler to simply put async2 into the signature of the method; we don't think this is
something we want to do long term. In fact, we think that specific syntax is likely the wrong solution to triggering the new generation strategy. We like our current async syntax, and think that it should
continue to be the main syntax for the future. The goal with this change is that there's little-to-no semantic change in the meaning of async methods, and is instead just a code generation strategy change.
This isn't something we want users to have to opt-in to, it should just be an improvement that they get as they move forward with the platform. There's also some concern about users who multi-target
between various platforms; would users who target both .NET Standard 2.0 and modern .NET need to #ifdef their async methods so that they can take advantage of new features where they're available?
All of this leads us to consider a very rarely used strategy; a configuration flag. We have very few of these in C#, intentionally, as we don't want to create language dialects. The good thing for us here
is that we're not actually creating a language dialect; the emit strategy may be different, but the code inside the async method body means the same thing, semantically, whether the flag is on or off.
There will be a cost to pay in compiler testing; we will forever have to test both async code generation strategies, which adds a lot of new tests to the test matrix. But it seems like a cost we'd be
willing to pay for the improvements we're looking at here. Further, we don't think we have to concern most users with the existence of a switch here. Instead, the compiler can simply look for a RuntimeFeature
flag, and turn on the new strategy if it's available. The switch could be limited to simply being an escape hatch for users who need to switch back to the old generation strategy for some reason.
The only wrinkle with this strategy is the potential SyncContext behavior change. This isn't something that really reflects in C# code itself, or in the code generated by the compiler at all. Instead,
it affects the code that the JIT generates for the async method at runtime, as it converts the body to the async state machine. We therefore think it would be possible to leave this behavior change to
a configuration option at runtime, either through some attribute on the assembly, or through runtime flags.
Another topic we discussed briefly was ConfigureAwait. This issue may end up being the tipping point that forces an assembly-wide configuration solution, as we don't want to force developers to realize
the Task return by calling ConfigureAwait. The async2 mechanism avoids realizing Tasks when not necessary; this is where most of the performance gain comes from, as most async callstacks actually
have very few true suspension points. If we have ConfigureAwait calls throughout the stack, it could force us to materialize the Tasks where we otherwise wouldn't have to, costing a large part of the
performance gains. It's possible that we could have the JIT recognize the scenario and elide the Task allocation, but that may end up being somewhat expensive to implement, so we will need to do the
exercise of costing to determine what we do there.
Finally, there were a few other topics mentioned:
async or task computation expression, or Kotlin's CoroutineContext.
This would require new APIs from the runtime, and potentially change the way that asynchronous workflows are constructed, but if we're going to make such a change, perhaps now is the time. Moving to
such a model may obviate the main use case of https://github.com/dotnet/csharplang/issues/6300 as well, as the cancellation could be implicitly checked during asynchronous calls by the runtime.