• Shortcuts : 'n' next unread feed - 'p' previous unread feed • Styles : 1 2

» Publishers, Monetize your RSS feeds with FeedShow:  More infos  (Show/Hide Ads)


Date: Thursday, 04 Apr 2013 17:26

The .NET Framework blog published this morning a guest post from yours truly on .NET Memory Allocation Profiling with Visual Studio 2012.  As you're trying to improve the performance, throughput, and memory usage of code that uses Tasks, the described profiler in Visual Studio can be a valuable tool in your tool belt (of course, the example I use in that post to highlight the profiler's capabilities is one that uses Task and async/await).  I hope you find the post helpful.

Author: "Stephen Toub - MSFT" Tags: ".NET 4.5, Visual Studio"
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 03 Apr 2013 17:53

A few years back, Wes Dyer wrote a great post on monads, and more recently, Eric Lippert wrote a terrific blog series exploring monads and C#. In that series, Eric alluded to Task<TResult> several times, so I thought I’d share a few related thoughts on Task<TResult> and the async/await keywords.

As both Wes and Eric highlight, a monad is a triple consisting of a type, a Unit function (often called Return), and a Bind function. If the type in question is Task<T>, what are its Unit and Bind functions?

The Unit operator takes a T and “amplifies” it into an instance of the type:

public static M<T> Unit<T>(this T value);

That’s exactly what Task.FromResult does, producing a Task<T> from a T, so our Unit method can easily be implemented as a call to FromResult:

public static Task<T> Unit<T>(this T value)
{
    return Task.FromResult(value);
}

What about Bind? The Bind operator takes the instance of our type, extracts the value from it, runs a function over that value to get a new amplified value, and returns that amplified value:

public static M<V> Bind<U, V>(
    this M<U> m, Func<U, M<V>> k);

If you squint at this, and if you’ve read my previous blog post Implementing Then with Await, the structure of this declaration should look eerily like the last overload of Then discussed:

public static Task<TNewResult> Then<TResult, TNewResult>(
    this Task<TResult> task, Func<TResult, Task<TNewResult>> continuation);

In fact, other than the symbols chosen, they’re identical, and we can implement Bind just as we implemented Then:

public static async Task<V> Bind<U, V>(
    this Task<U> m, Func<U, Task<V>> k)
{
    return await k(await m);
}

This is possible so concisely because await and async are so close in nature to the monadic operators. When you write an async function that returns Task<V>, the compiler expects the body of the method to return a V, and it lifts (or amplifies) that V into the Task<V> that’s returned from the method call; this is basically Unit (async, of course, also handles the creation and completion of the returned Task for the case where an exception propagates out of the body of the async method). Further, a core piece of the Bind operator is in extracting the value from the supplied instance, and that’s nicely handled by await.

In Eric’s last post on monads, he talks about some of the C# LINQ operators, and how they can easily be implemented on top of types that correctly implement a Unit and a Bind method:

static M<C> SelectMany<A, B, C>(
    this M<A> monad,
    Func<A, M<B>> function,
    Func<A, B, C> projection)
{
    return monad.Bind(
        outer => function(outer).Bind(
            inner => projection(outer, inner).Unit()));
}

Sure enough, with our Bind and Unit implementations around Task<T>, we can substitute in “Task” for “M”, and it “just works”:

static Task<C> SelectMany<A, B, C>(
    this Task<A> monad,
    Func<A, Task<B>> function,
    Func<A, B, C> projection)
{
    return monad.Bind(
        outer => function(outer).Bind(
            inner => projection(outer, inner).Unit()));
}

What does it mean here to “just work”? It means we can start writing LINQ queries using the C# query comprehension syntax with operators that rely on SelectMany, e.g.

int c = await (from first in Task.Run(() => 1)
               from second in Task.Run(() => 2)
               select first + second);
Console.WriteLine(c == 3); // will output True

Of course, we can implement SelectMany without Bind and Unit, just using async/await directly:

static async Task<C> SelectMany<A, B, C>(
    this Task<A> task,
    Func<A, Task<B>> function,
    Func<A, B, C> projection)
{
    A a = await task;
    B b = await function(a);
    return projection(a, b);
}

In fact, we can implement many of the LINQ query operators simply and efficiently using async/await. The C# specification section 7.16.3 lists which operators we need to implement to support all of the C# query comprehension syntax, i.e. all of the LINQ contextual keywords in C#, such as select and where. Some of these operators, like OrderBy, make little sense when dealing with singular values as we have with Task<T>, but we can easily implement the others. This enables using most of the C# LINQ query comprehension syntax with Tasks:

public static async Task<V> SelectMany<T, U, V>(
    this Task<T> source, Func<T, Task<U>> selector, Func<T, U, V> resultSelector)
{
    T t = await source;
    U u = await selector(t);
    return resultSelector(t, u);
}

public static async Task<U> Select<T, U>(
    this Task<T> source, Func<T, U> selector)
{
    T t = await source;
    return selector(t);
}

public static async Task<T> Where<T>(
    this Task<T> source, Func<T, bool> predicate)
{
    T t = await source;
    if (!predicate(t)) throw new OperationCanceledException();
    return t;
}

public static async Task<V> Join<T, U, K, V>(
    this Task<T> source, Task<U> inner,
    Func<T, K> outerKeySelector, Func<U, K> innerKeySelector,
    Func<T, U, V> resultSelector)
{
    await Task.WhenAll(source, inner);
    if (!EqualityComparer<K>.Default.Equals(
        outerKeySelector(source.Result), innerKeySelector(inner.Result)))
            throw new OperationCanceledException();
    return resultSelector(source.Result, inner.Result);
}

public static async Task<V> GroupJoin<T, U, K, V>(
    this Task<T> source, Task<U> inner,
    Func<T, K> outerKeySelector, Func<U, K> innerKeySelector,
    Func<T, Task<U>, V> resultSelector)
{
    T t = await source;
   
return resultSelector(t,
        inner.Where(u => EqualityComparer<K>.Default.Equals(
            outerKeySelector(t), innerKeySelector(u))));
}

public static async Task<T> Cast<T>(this Task source)
{
    await source;
    return (T)((dynamic)source).Result;
}

Interestingly, Task<TResult> already has the members necessary to be considered “comonadic.” As Brian Beckman discusses in his precis, a comonad is the dual of a monad, a triple consisting of the type and two operators: Extract (the flip of Unit/Return) and Extend (the flip of Bind). Here I’ve taken a few liberties with the signatures from what Brian outlines, such as swapping the order of some of the parameters:

public T Extract<T>(this W<T> self);
public W<U> Extend<T, U>(this W<T> self, Func<W<T>, U> func);

Task<TResult> already supports Extract, it’s just called Result:

public TResult Result;

And it already supports Extend, it’s just called ContinueWith:

public Task<TNewResult> ContinueWith<TNewResult>(
    Func<Task<TResult>, TNewResult> func);

(In truth, to correctly implement all of the comonadic laws Brian outlines, we’d likely want to tweak both of these with a thin layer of additional code to modify some corner cases around exceptions and cancellation, due to how Result propagates exceptions wrapped in AggregateException and how ContinueWith tries to match thrown OperationCanceledExceptions against the CancellationToken supplied to ContinueWith. But the basic idea stands.)

Most of the posts I write on this blog are about practical things. So, is any of this really useful in everyday coding with Tasks and async/await? Would you actually want to implement and use the LINQ surface area directly for Tasks? Eh, probably not. But it’s a fun to see how all of these things relate.

(Update 4/3/2013: Thanks to Aaron Lahman for pointing out the bug I had in my GroupJoin implementation.  Fixed.)

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 13 Mar 2013 17:08

I can be a bit sensitive when it comes to language and how concepts are conveyed.  I think it’s important to be accurate, even if not precise, when describing what something is or how to use it, as otherwise the folks to whom you’re communicating can easily form the wrong mental model for that thing.  Having a good mental model for something is important in then being able to reason about the thing in question and to correctly infer other uses and behaviors.

That’s why I frequently cringe when I hear someone say something like “invoke the method using 'await'”.  The communicator is usually trying to convey that a statement like:

FooAsync();

should instead be:

await FooAsync();

with the await keyword prefixed to the method invocation.  Now, maybe that statement of “invoke the method using 'await'” is sufficient to communicate to the audience the desired result.  So why do I cringe? Because the statement doesn’t make any sense!

The 'await' keyword has nothing to do with invoking methods in this regard… nothing.  It doesn’t influence how a method is invoked, nor is it somehow associated by the compiler with a subsequent method call.  It’d be like someone showing the code:

return Foo();

and saying “I’m using the 'return' keyword to invoke the method”.  In both this case and in the 'await' case, the keyword isn’t operating on the method or somehow influencing the method’s invocation.  Rather, it’s operating on the result of the method's invocation.  The statement:

await FooAsync();

is functionally identical to:

var t = FooAsync();
await t;

just as:

return Foo();

is functionally identical to:

var t = Foo();
return t;

If you hear someone say “invoke the method using 'await'”, that might lead you to believe that 'await' does in fact somehow relate to method invocation, which might in turn shield you from the fact that you can await anything that’s awaitable, and conversely that you can invoke an async method without awaiting its result.  There are a whole host of valuable scenarios that come about from doing such things.  For example, the ability to launch multiple asynchronous operations to run concurrently and then wait for them all to complete relies on the ability to not immediately await for an operation after launching it, e.g.

await Task.WhenAll(from url in urls select DownloadAsync(url));

I recently came across another such example in my own personal weekend-coding project.  I was working on a Windows Store app** that has play and pause buttons which influence an audio loop implemented with async/await.  Each iteration through the loop, the code will check to see if a pause request has been issued, using a construct very much like that described at http://blogs.msdn.com/b/pfxteam/archive/2013/01/13/cooperatively-pausing-async-methods.aspx, e.g.

foreach(var sample in decoder.GetSamples())
{
    …
    await pauseToken.WaitWhilePausedAsync();
    …
}

This works, but due to the way the audio system functions, at the point the loop notices there’s a pause request there are already several buffers worth of audio data queued up to the underlying audio system to be played, which means that there is a several second delay between the time a pause is requested and the time music stops playing.  To fix that, I needed to actually stop the underlying audio system while paused, e.g.

foreach(var sample in decoder.GetSamples())
{
    …
    player.Stop();
    await pauseToken.WaitWhilePausedAsync();
    player.Start();
    …
}

There’s a new problem with this, however.  By stopping and starting the audio like this, it can result in audible hiccups in playback, since we're stopping and starting even if pausing wasn't requested.  I really only wanted to do this stopping and starting if a pause was requested, so that the only time stoppage is induced is when the user has actually requested them. And for that, it’s very helpful that await isn’t tied to a method’s invocation, because I was able to do something like this:

foreach(var sample in decoder.GetSamples())
{
    …
    var t = pauseToken.WaitWhilePausedAsync();
    if (t.Status != TaskStatus.RanToCompletion)
    {
        player.Stop();
        await t;
        player.Start();
    }
    …
}

In short, once you think about await as operating on awaitables rather than operating on method invocations, you gain a good deal of freedom in how you can write your asynchronous operations.

---------------------

**Personal plug about the aforementioned app: For anyone interested in karaoke, Vocals Be Gone (http://apps.microsoft.com/windows/en-US/app/vocals-be-gone/783df35d-2d8a-4f0f-bace-56c873ec671c) is a free app for Windows 8 that provides a simple UI for music playback (mp3, wma, and wav), supports background audio, snapped usage, and all such goodness, but most importantly enables attempting to remove the primary vocals from the audio.  It also supports pitch shifting, so that regardless of whether you’re using the vocal removal feature, you can play back the music in a different key than in what it was recorded.  The vocal removal today employs a very simple algorithm, and won’t work well with all recordings (nor will it work with DRM-protected content), so it may not work great with whatever song you pick first.  Try a few professional recordings, enjoy, and let me know what you think.  And as long as you’re downloading apps, you can also download the free Sudoku Classic app (http://apps.microsoft.com/windows/en-us/app/sudoku-classic/c41fec40-7bf8-4a8c-a0fa-53ecd4563596) for Windows 8.  It provides a very nice Sudoku experience (if I do say so myself :-), including a good puzzle generator with a variety of controls over how the puzzles are generated; support for playing with touch, pen, keyboard, or mouse; and a host of other goodies.  Enjoy!

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 20 Feb 2013 18:51

Lucian Wischik and I presented an "async clinic" at the MVP Summit in Bellevue this week.  The async/await keywords in C# and Visual Basic drastically simplify asynchronous programming, but that of course doesn't mean that using them is without any gotchas: the goal of the discussion was to highlight some of the key areas in which we see developers struggling with asynchronous development and to help provide guidance on avoiding and overcoming those roadblocks.  Attached are the slides from the presentation.

Attached Media: application/octet-stream (15 31 ko)
Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 29 Jan 2013 01:34

These days it’s not uncommon for me to receive an email or read a forum post from someone concerned about a problem they’re experiencing with an async method they’ve written, and they’re seeking help debugging the issue.  Sometimes plenty of information about the bug is conveyed, but other times the communication is void of anything more than a root problem statement.  That’s when I engage my powers of psychic debugging to suggest what the root cause might be, without actually knowing more about the developer’s codebase. 

Here are four of the more common issues I’ve heard raised, along with their likely culprits.  If you experience one of these problems, look at one of these causes first, as there’s a very good chance it’s to blame.

1. “I converted my synchronous method to be asynchronous using ‘async’, but it still runs synchronously.”

As explained in the Async/Await FAQ, marking a method as ‘async’ does not force the method to run asynchronously, e.g. it doesn’t queue a work item to run the method, it doesn’t spawn a new thread to run the method, etc.  Marking a method as ‘async’ really just tells the compiler to allow usage of ‘await’ inside the body of the method, and to handle completion/results/exceptions from the method in a special way (i.e. putting them into a task that’s returned from the method call).  When you invoke a method that’s marked as ‘async’, the method is still invoked synchronously, and it continues to run synchronously until either a) the method completes, or b) the method awaits an awaitable (e.g. a task) that’s not yet complete.  If the method doesn’t contain any awaits (which will generate a compiler warning) or if it only ever awaits awaitables that are already completed, the method will run to completion synchronously, and the task returned to the caller will be completed by the time it’s handed back.  This is by design.

If your goal in using ‘async’ isjust to offload some synchronous work to another thread, you can instead use Task.Run to do this rather than using the async/await language features, e.g. if you have the synchronous method:

int DoWork()
{
    …
}

instead of converting that to the following (which isn’t doing what you wanted, as the method will run entirely synchronously due to a lack of awaits):

async Task<int> DoWorkAsync()
{
    … // same code as in the synchronous method
}

you can convert it to:

Task<int> DoWorkAsync()
{
    return Task.Run(() =>
    {
        … // same code as in the synchronous method
    });
}

Task.Run will run the supplied delegate on a ThreadPool thread, immediately returning a Task that represents the eventual completion of that work.

(This is a fine thing to do inside of your application, such as for offloading some compute-intensive work from your UI thread to the ThreadPool.  For predominantly philosophical reasons, however, I don’t recommend exposing such asynchronous wrappers publicly for others to consume.)

2. “I made my method asynchronous using ‘async’ and ‘await’, but I can’t await it.”

My psychic powers tell me that you have a void-returning synchronous method, and that when you applied the ‘async’ keyword, you didn’t change the return type from ‘void’ to ‘Task’ (and subsequently ignored the warning and friendly recommendation the compiler made when you tried to await the method’s result: “error CS4008: ‘MethodAsync’ does not return a Task and cannot be awaited. Consider changing it to return Task.”)

Methods marked as ‘async’ can return void, Task, or Task<T>, but void should really only be reserved for top-level entry points to your program, e.g. UI event handlers.  If you have a synchronous library method that you’re converting to be asynchronous with async/await, and if you think it should stay returning void, think again: unless you have a really good reason, asynchronous operations exposed from libraries should return Task or Task<T>, not void.

3. “My async method never completes.”

The problem statement here is that an async method returns a Task that never completes.  Often the description also includes a statement that one of the awaits inside of the async method never completed.  This is behavior is typically due to one of two things, or variations off of these:

A) The caller had a non-null SynchronizationContext, ConfigureAwait(false) was not used on the await of a task, and the SynchronizationContext is blocked or no longer pumping.

If there is a current SynchronizationContext when a task is awaited, and if that task has not yet completed by the time it’s awaited, by default that SynchronizationContext will be used to run the continuation (the code that comes next after the await) once the task completes.  As this continuation represents part of the async method, the async method can’t be considered complete (and the Task returned to represent this async method’s invocation can’t complete) until that continuation is run.  And when/where/how that continuation is run is entirely up to the SynchronizationContext to which the continuation is Post’d.

Often this SynchronizationContext will be a single-threaded context, meaning that only one thread at a time is able to process queued work (in the case of a GUI context, like DispatcherSynchronizationContext for WPF, WinRTSynchronizationContext for Windows Store apps, or WindowsFormsSynchronizationContext, that one thread is likely to be a dedicated “UI thread”; in the case of a server environment, like AspNetSynchronizationContext for ASP.NET, that thread is just whatever thread is currently the one allowed to process queued work for a given HTTP request). 

If that one thread gets blocked, no other thread will be allowed to process the queued work, which means the aforementioned continuation won’t be run, which means the async method’s Task won’t complete.  And, if it’s that Task that’s being waited on to block the thread, then we have ourselves a case of deadlock.  Look for places where you’re synchronously blocking, e.g. using Task.Wait or Task.WaitAll or other such forms of waiting (you shouldn’t be doing such synchronous blocking on such precious resources as a UI thread); there’s a good chance that’s where you’ll find your culprit.

Of course, there are reasons other than explicit blocking that UI thread might not be processing.  For example, the message pump that had been running on that thread might have shut down, e.g. in WPF you were inside of a call to Dispatcher.PushFrame which was told to exit by setting the frame’s Continue property to false.  As with the deadlock example, if the context isn’t able to run the continuation, the method will never complete.

Now, as an implementer of library code that uses await, you can help protect against such issues by using “await task.ConfigureAwait(false)” instead of just “await task” whenever you await tasks.  This tells the system not to force the continuation back to the original SynchronizationContext.  And since most libraries are agnostic to the environment in which they run, most library-based asynchronous methods shouldn’t need to be SynchronizationContext-aware. 

This good citizenry on your part will also pay dividends to you, in that it’ll often make your code faster.  If your method is invoked when there’s a current SynchronizationContext, all of your method’s continuations will be forced back to that context, incurring costs in the form of thread hops and object allocations and the like, all of which are very likely unnecessary.  Using “.ConfigureAwait(false)” will help avoid those costs. 

So, if you the code in your method doesn’t need to run the continuations back on the original SynchronizationContext, go the extra step of specifying ConfigureAwait(false) whenever you await a task in that method; you’ll be happy you did.

B) An async method used in this method never completes.

If you have a synchronous method that enters an infinite loop or blocks indefinitely or some other such operation that never completes, the invocation of the synchronous method will never return to its caller, e.g.

public void MethodThatNeverCompletes()
{
    …
    var tcs = new TaskCompletionSource<int>();
    tcs.Task.Wait(); // the Task is never completed, so this blocks forever
    …
}

Similarly, a direct async port of this method will result in the returned Task never completing:

public async Task MethodThatNeverCompletesAsync()
{
    …
    var tcs = new TaskCompletionSource<int>();
    await tcs.Task; // the Task is never completed, so this awaits forever
    …
}

And as a result, since the Task returned from MethodThatNeverCompletesAsync will never complete, any awaits on the Task returned from this method will also never complete, e.g.

await MethodThatNeverCompletesAsync(); // awaits forever

This may seem like a silly example, but I’ve seen issues related to this a non-trivial number of times.  The moral of the story is: don’t forget to complete your tasks

Of course, the issues are almost never as simple as I’ve demonstrated above.  Often they occur when a developer has forgotten to complete a TaskCompletionSource task on a failure or cancellation path, e.g.

public Task<int> SomeLibraryMethodAsync()
{
    var tcs = new TaskCompletionSource<int>();
    BeginLibraryMethod(ar =>
    {
        try
        {
            int result = EndLibraryMethod(ar);
            tcs.SetResult(result);
        }
        catch(Exception exc) {} // oops! Forgot tcs.SetException(exc);
    });
    return tcs.Task;
}

Here the developer should have completed the task even if an error occurred, and in neglecting to do so, anyone awaiting the Task<int> returned from SomeLibraryMethodAsync will end up awaiting forever in the event of an exception.

Examples can get much more complicated than this. For example, one system I saw involved code that was maintaining a queue of waiters, and when one part of the system requested cancellation, all of the waiters were dropped, e.g.

private readonly object m_syncObj = new object();
private readonly Queue<T> m_data = new Queue<T>();
private Queue<TaskCompletionSource<T>> m_waiters =
    new Queue<TaskCompletionSource<T>>();

public void Add(T data)
{
    TaskCompletionSource<T> next = null;
    lock(m_syncObj)
    {
        if (m_waiters.Count > 0) 
           
next = m_waiters.Dequeue();
        else
            m_data.Enqueue(data);
    }
    if (next != null)
        next.SetResult(data);
}

public Task<T> WaitAsync()
{
    lock(m_syncObj)
    {
        if (m_data.Count > 0)
            return Task.FromResult(m_data.Dequeue());

        var tcs = new TaskCompletionSource<T>();
        m_waiters.Enqueue(tcs);
        return tcs.Task;
    }
}

public void Reset()
{
    lock(m_syncObj)
    {
        m_data.Clear();
        m_waiters.Clear(); // uh oh!
    }
}

This implementation might look ok, but there’s a dastardly bug lurking: calling Reset will simply dump the TaskCompletionSource instances for anyone currently waiting for data, which will likely cause anyone waiting on the associated tasks to wait forever.  The developer of this code probably should have iterated through each of the waiters and explicitly canceled or faulted them, rather than leaving each incomplete, e.g.

public void Reset()
{
    TaskCompletionSource<T> oldWaiters;
    lock(m_syncObj)
    {
        m_data.Clear();
        oldWaiters = m_waiters;
        m_waiters = new Queue<TaskCompletionSource<T>>();
    }
    foreach(var waiter in oldWaiters)
        waiter.SetCanceled();
}

Long story short: if you ever write or review code that just drops a stored reference to a TaskCompletionSource without first completing it, ask yourself whether there might be a bug lurking there.

4. “The Task returned from my async method completes before the method is done.”

This symptom is oftem caused by one of three things, or variations off of these:

A) An async lambda is passed to a method accepting an Action.

I see this one bite people over and over again.  For a thorough explanation, see pitfalls to avoid when passing around async lambdas.

In short, whenever you pass an async lambda or an async anonymous method to a method, be sure to take a look at the signature of the method you’re calling, and in particular to the type of the delegate parameter to which you’re passing the async lambda/anonymous method.  If the delegate type is a Func<Task>, Func<Task<T>>, or some other delegate type returning a Task or a Task<T>, be happy!  If, however, the delegate type is an Action, or an Action<T>, or any other void-returning delegate type, be afraid, be very very afraid!

The whole point of an asynchronous method returning a Task or some other future/promise type is that it gives the caller an object that represents the eventual completion of that operation.  Just the method returning to its synchronous caller is insufficient to signal the asynchronous operation’s completion: the returned task must also complete.  In the case of a void-returning asynchronous method, there is no such object handed back, and thus there isn’t a good way for a caller to know that the asynchronous operation has completed (in making this statement, I’m largely ignoring the advanced possibility of using a custom SynchronizationContext for its OperationStarted/OperationCompleted methods, which will be invoked by the infrastructure backing an “async void” method… I feel comfortable ignoring that because I don’t consider such an ambient mechanism to be “a good way” for the caller to know).  Without such an indication, many callers will erroneously assume that the method’s synchronous return does in fact mean the operation has completed.

For example, consider a recent misuse I saw: passing an async lambda to Parallel.ForEach.  Here’s the basic overload of Parallel.ForEach (there are longer overloads, but they don’t matter to this discussion):

public static ParallelLoopResult ForEach<TSource>(
    IEnumerable<TSource> source,
    Action<TSource> body);

This overload takes two arguments: an enumerable of data to iterate through, and an action to invoke for each element of data.  The caller of the Action<TSource> in this case is the ForEach implementation, and the ForEach implementation won’t return to its caller until it’s invoked the body delegate for each element of data (I’m ignoring for this discussion the possibility of an exception).  That’s all fine and dandy when a synchronous implementation is passed as the body, but consider the following:

var sw = Stopwatch.StartNew();
Parallel.ForEach(Enumerable.Range(0, 1000), async i => // uh oh!
{
    await Task.Delay(TimeSpan.FromSeconds(1));
});
Console.WriteLine(sw.Elapsed);

You might expect this to take ~17 minutes to run (1000 iterations, each of which should be waiting for a second), but it actually completes almost immediately, outputting a sub-second time.  Why?  Because the delegate is void-returning, so the async lambda is being mapped into an “async void” method, one that returns void.  And as soon as the body of the delegate hits an await for an awaitable that’s not yet completed, it returns to its caller.  It's caller, in this case, is the ForEach code, which has little choice but to assume that the delegate has completed its work.  As such, the ForEach very quickly enumerates through all 1000 elements, launching the async method for each, but not waiting for each to complete.  (In this particular case of Parallel.ForEach, the developer could have implemented a ForEachAsync, one that used a Func<TSource,Task> for the delegate rather than an Action<TSource>.)

B) A Task<Task> or a Task<Task<T>> is used as if it were just a Task.

Consider this code:

static async Task ReproAsync()
{
    var sw = Stopwatch.StartNew();
    await Task.Factory.StartNew(async delegate
    {
        await Task.Delay(TimeSpan.FromSeconds(1000));
    });
    Console.WriteLine(sw.Elapsed);
}

How much time do you think this outputs, a value close to 1000 seconds, or a value under 1 second?  If you guessed the latter, congratulations.  If you guessed the former, sorry, read on.  This example is an interesting variation on the previous discussion about passing around async lambdas / async anonymous methods. 

The StartNew method has multiple overloads, including ones that accept an Action and ones that accept a Func<TResult>.  The overload that accepts an Action returns a Task, and the overload that accepts a Func<TResult> returns a Task<TResult>.  Makes sense.  Now, the C# compiler is going to use its overload resolution logic to pick the best matching overload given the supplied arguments, and it’s going to end up matching the above to Func<TResult>, where TResult is a Task.  Great, so our async delegate is getting mapped to a Func<Task>… all’s well, right?  Wrong.  As far as StartNew is concerned, that Func<Task> is just a Task<TResult>, and StartNew isn’t going to give any special treatment to a particular TResult type, i.e. it’ll treat a TResult of Task just as if it were an Int32 or a String or an Object or a TimeSpan or anything else.  And that means that StartNew will be returning a Task<Task>.  When the above code is awaiting the result of the StartNew, it’s awaiting that outer task to complete, not the inner Task that was returned from the invocation of the asynchronous delegate.  To do this correctly, the code should either be awaiting the outer task to get the inner task, and then awaiting that inner task, or it should be using the Unwrap method, or it should be using Task.Run instead of Task.Factory.StartNew.

Mindbending?  If so, read this post on Task.Run vs Task.Factory.StartNew, and it should shed more light on the subject.

C) An awaitable object isn’t awaited in a method marked with the async keyword.

If you have the following method:

public async Task PauseOneSecondAsync() // buggy
{
    Task.Delay(1000);
}

The compiler will issue a warning on the call to Task.Delay:

warning CS4014: Because this call is not awaited, execution of the current method continues before the call is completed. Consider applying the 'await' operator to the result of the call.

That’s because invoking a Task-returning method (or other methods that return other future-like or promise-like types) just initiates the asynchronous operation, but doesn’t wait or block for it to complete.  Rather, the returned object is the representation for the eventual completion of the operation, and to prevent forward progress until that operation has completed, the returned object must be awaited.

Of course, this warning can be suppressed, either at the project level, more localized with a “#pragma warning disable”, or more commonly just by doing something with the result of the call, including storing it into a variable (even if that variable is then never used):

public async Task PauseOneSecondAsync() // buggy
{
    var t = Task.Delay(1000); // no warning
}

The compiler won’t issue any warning in this case, as it’s been suppressed due to the Task returned from Task.Delay being stored into a local, and yet this method will complete before the developer intended for it to, because of the missing await:

public async Task PauseOneSecondAsync()
{
    await Task.Delay(1000);
}

(As a complete aside about performance, if you actually wanted to implement PauseOneSecondAsync, it doesn’t need to be marked as ‘async’ and use ‘await’.  As described in the post “when at last you await”, this could be rewritten as:

public Task PauseOneSecondAsync()
{
    return Task.Delay(1000);
}

That code will be identical functionally but with better performance characteristics.)

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Sunday, 13 Jan 2013 22:27

Recently I was writing an app that processed a bunch of files asynchronously.  As with the Windows copy file dialog, I wanted to be able to provide the user with a button that would pause the processing operation.

To achieve that, I implemented a simple mechanism that would allow me to pass a “pause token” into the async method, which the async method could asynchronous wait on at appropriate points. 

public async Task ProcessFiles(
    IEnumerable<StorageFile> files, PauseToken pauseToken)
{
    foreach(var file in files)
    {
        await pauseToken.WaitWhilePausedAsync();
        await ProcessAsync(file);
    }
}

My pause token follows a similar design to that of CancellationToken and CancellationTokenSource.  I have a PauseToken instance that I can pass to any number of operations (synchronous or asynchronous), and those operations can monitor that token to be alerted to pause requests.  Separately, a PauseTokenSource is responsible for creating the PauseToken to be handed out and for issuing the pause requests.

public class PauseTokenSource
{
    public bool IsPaused { get; set; }
    public PauseToken Token { get; }
}

public struct PauseToken
{
    public bool IsPaused { get; }
    public Task WaitWhilePausedAsync();
}

We’ll start by implementing PauseTokenSource, which is the meat of the implementation; as with CancellationToken and CancellationTokenSource, PauseToken is just a thin value-type veneer on top of PauseTokenSource that just delegates most calls to the underlying reference type.  PauseTokenSource has one instance field:

private volatile TaskCompletionSource<bool> m_paused;

The m_paused field is the TaskCompletionSource<bool> that can be used to complete the Task we’ll hand out to waiters when the instance is paused (such that when we’re un-paused, we’ll set the Task to wake up all the waiters): if m_paused is null, we’re not paused, and if it’s non-null, we’re currently paused.

The bulk of the implementation is then in PauseTokenSource.IsPaused.  Its getter just returns whether m_paused is not null, but its setter is more complicated:

public bool IsPaused
{
    get { return m_paused != null; }
    set
    {
        if (value)
        {
            Interlocked.CompareExchange(
                ref m_paused, new TaskCompletionSource<bool>(), null);
        }
        else
        {
            while (true)
            {
                var tcs = m_paused;
                if (tcs == null) return;
                if (Interlocked.CompareExchange(ref m_paused, null, tcs) == tcs)
                {
                    tcs.SetResult(true);
                    break;
                }
            }
        }
    }
}

If IsPaused is being set to true, then we simply need to transition m_paused from null to a new TaskCompletionSource<bool>; we do this with an interlocked compare-exchange so that we only do the transition if m_paused is null, regardless of what other threads we might be competing with.

If IsPaused is being set to false, we need to do two things: transition m_paused from non-null to null, and complete the Task from the TaskCompletionSource<bool> that was stored in m_paused.  We do this with another Interlocked.CompareExchange, and as we need to tell the CompareExchange operation what exact value we expect to find in m_paused, and as another thread could be changing it out from under us, we need to do this in a standard compare-exchange loop: grab the current value, do the compare exchange assuming that value, and if the value actually changed between the time we grabbed it and the time we did the compare-exchange, repeat.

To shield these implementation details from PauseToken, we’ll add an internal WaitWhilePauseAsync method to PauseTokenSource that PauseToken can then access.

internal Task WaitWhilePausedAsync()
{
    var cur = m_paused;
    return cur != null ? cur.Task : s_completedTask;
}

This method just grabs m_paused, and if it’s non-null returns its Task.  If it is null, then we’re not paused, so we can hand back an already completed Task in order to avoid unnecessary allocations (since we should expect it to be very common that WaitWhilePauseAsync is called when not actually paused):

internal static readonly Task s_completedTask = Task.FromResult(true);

The last member we need on PauseTokenSource is the Token property that will return the associated PauseToken:

public PauseToken Token { get { return new PauseToken(this); } }

Now for implementing PauseToken.  Its implementation is very simple, as it’s just a wrapper over the PauseTokenSource from which its constructed:

public struct PauseToken
{
    private readonly PauseTokenSource m_source;
    internal PauseToken(PauseTokenSource source) { m_source = source; }

    public bool IsPaused { get { return m_source != null && m_source.IsPaused; } }

    public Task WaitWhilePausedAsync()
    {
        return IsPaused ?
            m_source.WaitWhilePausedAsync() :
            PauseTokenSource.s_completedTask;
    }
}

PauseToken’s IsPaused property only has a getter and not a setter, since our design requires that all transitioning from un-paused to paused, and vice versa, is done via the PauseTokenSource (that way, only someone with access to the source can cause the transition).  PauseToken’s IsPaused getter just delegates to the source’s IsPaused; of course, as this PauseToken is a struct, it’s possible it could have been default initialized such that m_source would be null… in that case, we’ll just return false from IsPaused.

Finally, we have our PauseToken’s WaitWhilePauseAsync method.  If we’re paused, we simply delegate to the source’s WaitWhilePausedAsync implementation we already saw.  If we’re not paused (which could include not having a source), we just return our cached already-completed Task.

That’s it: our implementation is now complete, and we can start using it to pause asynchronous operations.  Here’s a basic console-based example of using our new PauseToken type:

class Program
{
    static void Main()
    {
        var pts = new PauseTokenSource();
        Task.Run(() =>
        {
            while (true)
            {
                Console.ReadLine();
                pts.IsPaused = !pts.IsPaused;
            }
        });
        SomeMethodAsync(pts.Token).Wait();
    }

    public static async Task SomeMethodAsync(PauseToken pause)
    {
        for (int i = 0; i < 100; i++)
        {
            Console.WriteLine(i);
            await Task.Delay(100);
            await pause.WaitWhilePausedAsync();
        }
    }
}

As a final thought, for those of you familiar with various kinds of synchronization primitives, PauseTokenSource might remind you of one in particular: manual reset events.  In fact, that’s basically what it is, just with a different API set (for comparison, see this blog post on building an AsyncManualResetEvent).  Setting IsPaused to false is like setting/signaling a manual reset event, and setting it to true is like resetting one.

Enjoy!

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Friday, 14 Dec 2012 20:32

Igor Ostrovsky is one of the minds behind the parallel programming support in the .NET Framework.  Igor's recently written a great set of articles for MSDN Magazine to cover "The C# Memory Model in Theory and Practice".  Part 1 is available now in the December 2012 issue, and it's a great read.

Author: "Stephen Toub - MSFT" Tags: "Parallel Extensions, .NET 4, Article Sum..."
Comments Send by mail Print  Save  Delicious 
Date: Saturday, 17 Nov 2012 01:19

In both .NET 4 and .NET 4.5, PLINQ supports enumerables with up to Int32.MaxValue elements.  Beyond that limit, PLINQ will throw an overflow exception.  LINQ to Objects itself has this limitation with certain query operators (such as the indexed Select operator which counts the elements processed), but PLINQ has it with more.

This limitation impacts so few scenarios that it’s relatively benign and I rarely hear about it.  That said, it does come up now and again, and I was in fact asked about it earlier this week.  So, to help in case anyone else runs into this…

There is a relatively straightforward workaround you can apply if you do run up against this limit: ensure the enumerable passed to PLINQ has no more than Int32.MaxValue.  That answer might sound flippant and impractical, but in reality it’s often easily accomplished by batching a longer enumerable into multiple shorter enumerables, where none of the shorter enumerables exceed the limit.

Consider a basic query like the following:

IEnumerable<Output> outputs =
    from input in inputs.AsParallel()
    where Filter(input)
    select Map(input);

If the inputs enumerable has more than Int32.MaxValue elements, this will result in an overflow.  To workaround this, imagine if we had a Batch extension method for IEnumerable<T> that would partition the IEnumerable<T> into multiple sequential IEnumerable<T> instances, each of which had no more than Int32.MaxValue elements.  With that, we could rewrite this as follows:

IEnumerable<Output> outputs = inputs.Batch(Int32.MaxValue).SelectMany(batch =>
    from input in batch.AsParallel()
    where Filter(input)
    select Map(input)
);

The original query remains intact, except that instead of operating on the original inputs enumerable, it’s operating on the new batch enumerable, which is a subset provided by the Batch method.  We’ll now be parallelizing our processing of each sequential batch, moving on to the next batch to be processed in parallel once we finish the previous one.

Such a Batch method doesn’t actually exist built-in to .NET 4.5, but writing one is relatively straightforward.  Here’s one implementation which you could customize as your needs demand:

static class MyLinqExtensions
{
    public static IEnumerable<IEnumerable<T>> Batch<T>(
        this IEnumerable<T> source, int batchSize)
    {
        using (var enumerator = source.GetEnumerator())
            while (enumerator.MoveNext())
                yield return YieldBatchElements(enumerator, batchSize - 1);
    }

    private static IEnumerable<T> YieldBatchElements<T>(
        IEnumerator<T> source, int batchSize)
    {
        yield return source.Current;
        for (int i = 0; i < batchSize && source.MoveNext(); i++)
            yield return source.Current;
    }
}

Author: "Stephen Toub - MSFT" Tags: "PLINQ, Parallel Extensions, .NET 4, .NET..."
Comments Send by mail Print  Save  Delicious 
Date: Friday, 05 Oct 2012 16:29

This is a question I hear relatively frequently:

“I have an async operation that’s not cancelable.  How do I cancel it?”

The construction of the question often makes me chuckle, but I understand and appreciate what’s really being asked.  The developer typically isn’t asking how to cancel the operation itself (if they are asking that, the answer is simple: it’s not cancelable!), but rather they’re asking how to allow their program’s execution to continue upon a cancellation request even if the operation being waited on hasn’t completed yet.  That’s a different ball game.

When someone talks about canceling async operations, they typically mean one of three things:

  1. Requesting that the async operation itself cancel.  The async operation may still run for some period of time after the cancellation request comes in, either because the operation’s implementation doesn’t respect cancellation, or because the implementation isn’t very aggressive about noticing and responding to such requests, or because there’s cleanup work that needs to be done even after a cancellation request is observed, or because the operation just isn’t at a good place in its execution to be canceled.
  2. Requesting that the code waiting for the async operation to complete stop waiting.  This has nothing to do with canceling the operation itself, and in fact the operation may still execute for quite some time.  This is entirely about changing the control flow of the program such that code (which was waiting for the operation to complete before going on to do other things) stops waiting and progresses to do those other things even though the operation hasn’t completed.
  3. Both #1 and #2.  Request the async operation to cancel, but also cancel the wait on the async operation so that we may continue running sooner than the async operation might complete.

In .NET, #1 is enabled by passing a CancellationToken to the async operation in question.  For example, here I’m passing a token to a Stream operation:

FileStream fs = …;
byte [] b = …;
CancellationToken token = …;
await fs.ReadAsync(b, 0, b.Length, token);

When the token has cancellation requested, the ReadAsync operation in flight may observe that request and cancel its processing before it otherwise would have completed.  By design, the Task returned by ReadAsync won’t complete until all processing has quiesced one way or another, and the system won’t continue to execute any code after the await until it does; this is to ensure that when the await completes, you know there’s no relevant work still in flight, that you can safely manipulate any data (like the byte[] buffer) that was provided to the async operation, etc.

Ok, so what about #2… is it possible to implement #2 in .NET?  Of course… but we didn’t make it super easy.  For example, you might expect an overload of ConfigureAwait that accepts a CancellationToken, e.g.

FileStream fs = …;
byte [] b = …;
CancellationToken token = …;
await fs.ReadAsync(b, 0, b.Length).ConfigureAwait(token); // overload not in .NET 4.5

Such an overload would allow the await itself to be canceled and execution to continue after the await even if the async operation on the stream was still in flight.  But there’s the rub… this can lead to unreliability.  What if the async operation eventually completes and returns an object that should be disposed of or otherwise acted upon?  What if the async operation fails with a critical exception that gets ignored?  What if the async operation is still manipulating reference arguments provided to it? And so on.  That’s not to say a developer couldn’t cope with some such issues, e.g.

FileStream fs = …;
byte [] b = …;
CancellationToken token = …;
Task op = fs.ReadAsync(b, 0, b.Length);
try
{

    await op.ConfigureAwait(token); // overload not in .NET 4.5
}
catch(OperationCanceledException)
{
    op.ContinueWith(t => /* handle eventual completion */);
    … // whatever you want to do in the case of cancellation, but
      // be very careful if you want to use the byte[], which
      // could be modified concurrently by the async operation
      // still in flight…
}

but doing so is non-trivial.  As such, for better or worse, in .NET 4.5 such an overload of ConfigureAwait doesn’t exist, out of concern that it would become a crutch too quickly used without thinking through the ramifications, which are subtle. 

Of course, that doesn’t prevent you from implementing such functionality yourself if you believe it’s the right thing for your needs.  In fact, you can implement this #2 functionality with just a few lines of code.  Here’s one approach:

public static async Task<T> WithCancellation<T>(
    this Task<T> task, CancellationToken cancellationToken)
{
    var tcs = new TaskCompletionSource<bool>();
    using(cancellationToken.Register(
                s => ((TaskCompletionSource<bool>)s).TrySetResult(true), tcs))
       
if (task != await Task.WhenAny(task, tcs.Task))
            throw new OperationCanceledException(cancellationToken);
    return await task;
}

Here we’re using a Task.WhenAny to wait for either the task to complete or for a cancellation request to arrive (which we do by creating another task that will complete when cancellation is requested).  With that function, I now can achieve #2 as outlined previously (subject to the same caveats), e.g.

FileStream fs = …;
byte [] b = …;
CancellationToken token = …;
Task op = fs.ReadAsync(b, 0, b.Length);
try
{

    await op.WithCancellation(token);
}
catch(OperationCanceledException)
{
    op.ContinueWith(t => /* handle eventual completion */);
    … // whatever you want to do in the case of cancellation
}

Of course, once I can do #1 and #2, doing #3 is straightforward, since it’s just a combination of the other two (passing the CancellationToken to the operation in addition to passing it to a WithCancellation-like function), e.g.

FileStream fs = …;
byte [] b = …;
CancellationToken token = …;
Task op = fs.ReadAsync(b, 0, b.Length, token);
try
{

    await op.WithCancellation(token);
}
catch(OperationCanceledException)
{
    if (!op.IsCompleted)
        op.ContinueWith(t => /* handle eventual completion */);
    … // whatever you want to do in the case of cancellation
}

So, can you cancel non-cancelable operations? No.  Can you cancel waits on non-cancelable operations?  Sure… just be very careful when you do.

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Saturday, 22 Sep 2012 15:57

Astute users of the Task Parallel Library might have noticed three new options available across TaskCreationOptions and TaskContinuationOptions in .NET 4.5: DenyChildAttach, HideScheduler, and (on TaskContinuationOptions) LazyCancellation.  I wanted to take a few minutes to share more about what these are and why we added them.

DenyChildAttach

As a reminder, when a Task is created with TaskCreationOptions.AttachedToParent or TaskContinuationOptions.AttachedToParent, the creation code looks to see what task is currently running on the current thread (this Task’s Id is available from the static Task.CurrentId property, which will return null if there isn’t one).  If it finds there is one, the Task being created registers with that parent Task as a child, leading to two additional behaviors: the parent Task won’t transition to a completed state until all of its children have completed as well, and any exceptions from faulted children will propagate up to the parent Task (unless the parent Task observes those exceptions before it completes).  This parent/child relationship and hierarchy is visible in Visual Studio’s Parallel Tasks window.

If you’re responsible for all of the code in your solution, you have control over whether the tasks you create try to attach to a parent task.  But what if your code creates some Tasks, and from those Tasks calls out to code you don’t own?  The code you call might use AttachedToParent and attach children to your tasks.  Did you expect that?  Is your code reliable against that?  Have you done all of the necessary testing to ensure it?

For this situation, we introduced DenyChildAttach.  When a task uses AttachedToParent but finds there is no current Task, it just doesn’t attach to anything, behaving as if AttachedToParent wasn’t supplied.  If there is a parent task, but that parent task was created with DenyChildAttach, the same thing happens: the task using AttachedToParent won’t see a parent and thus won’t attach to anything, even though technically there was a task to which it could have been attached.  It’s a slight of hand or Jedi mind trick: “this is not the parent you’re looking for.”

With 20/20 hindsight, if we could do .NET 4 over again, I personally would chosen to make both sides of the equation opt-in.  Today, the child task gets to opt in to being a child by specifying AttachedToParent, but the parent must opt out if it doesn’t want to be one.  In retrospect, I think it would have been better if both sides had the choice to opt in, with the parent specifying a mythical flag like AllowsChildren to opt in rather than DenyChildAttach to opt out.  Nevertheless, this is just a question of default.  You’ll notice that the new Task.Run method internally specifies DenyChildAttach when creating its Tasks, in affect making this the default for the API we expect to become the most common way of launching tasks.  If you want explicit control over the TaskCreationOptions used, you can instead use the existing Task.Factory.StartNew method, which becomes the more advanced mechanism and allows you to control the options, object state, scheduler, and so on.

HideScheduler

With code written in .NET 4, we saw this pattern to be relatively common:

private void button_Click(…)
{
    … // #1 on the UI thread
   
Task.Factory.StartNew(() =>
    {
        … // #2 long-running work, so offloaded to non-UI thread
    }).ContinueWith(t =>
    {
        … // #3 back on the UI thread
    }, TaskScheduler.FromCurrentSynchronizationContext());
}

In other words, Tasks and continuations became a way to offload some work from the UI thread, and then run some follow-up work back on the UI thread.  This was accomplished by using the TaskScheduler.FromCurrentSynchronizationContext method, which looks up SynchronizationContext.Current and constructs a new TaskScheduler instance around it: when you schedule a Task to this TaskScheduler, the scheduler will then pass the task along to the SynchronizationContext to be invoked.

That’s all well and good, but it’s important to keep in mind the behavior of the Task-related APIs introduced in .NET 4 when no TaskScheduler is explicitly provided.  The TaskFactory class has a bunch of overloaded methods (e.g. StartNew), and when you construct a TaskFactory class, you have the option to provide a TaskScheduler.  Then, when you call one of its methods (like StartNew) that doesn’t take a TaskScheduler, the scheduler that was provided to the TaskFactory’s constructor is used.  If no scheduler was provided to the TaskFactory, then if you call an overload that doesn’t take a TaskScheduler, the TaskFactory ends up using TaskScheduler.Current at the time the call is made (TaskScheduler.Current returns the scheduler associated with whatever Task is currently running on that thread, or if there is no such task, it returns TaskScheduler.Default, which represents the ThreadPool).  Now, the TaskFactory returned from Task.Factory is constructed without a specific scheduler, so for example when you write Task.Factory.StartNew(Action), you’re telling TPL to create a Task for that Action and schedule it to TaskScheduler.Current.

In many situations, that’s the right behavior.  For example, let’s say you’re implementing a recursive divide-and-conquer problem, where you have a task that’s supposed to process some chunk of work, and it in turn subdivides its work and schedules tasks to process those chunks.  If that task was running on a scheduler representing a particular pool of threads, or if it was running on a scheduler that had a concurrency limit, and so on, you’d typically want those tasks it then created to also run on the same scheduler.

However, it turns out that in other situations, it’s not the right behavior.  And one such situation is like that I showed previously.  Imagine now that your code looked like this:

private void button_Click(…)
{
    … // #1 on the UI thread
   
Task.Factory.StartNew(() =>
    {
        … // #2 long-running work, so offloaded to non-UI thread
    }).ContinueWith(t =>
    {
        … // #3 back on the UI thread
        Task.Factory.StartNew(() =>
        {
            … // #4 compute-intensive work we want offloaded to non-UI thread (bug!)
        });

    }, TaskScheduler.FromCurrentSynchronizationContext());
}

This seems logical: we do some work on the UI thread, then we offload some work to the background, when that work completes we hop back to the UI thread, and then we kick off another task to run in the background.  Unfortunately, this is buggy.  Because the continuation was scheduled to TaskScheduler.FromCurrentSynchronizationContext, that scheduler is TaskScheduler.Current during the execution of the continuation.  And in that continuation we’re calling Task.Factory.StartNew using an overload that doesn’t accept a TaskScheduler.  Which means that this compute-intensive work is actually going to be scheduled back to the UI thread! Ugh.

There are of course already solutions to this.  For example, if you own all of this code, you could explicitly specify TaskScheduler.Default (the ThreadPool scheduler) when calling StartNew, or you could change the structure of the code so that the StartNew became a continuation off of the continuation, e.g.

private void button_Click(…)
{
    … // #1 on the UI thread
   
Task.Factory.StartNew(() =>
    {
        … // #2 long-running work, so offloaded to non-UI thread
    }).ContinueWith(t =>
    {
        … // #3 back on the UI thread

    }, TaskScheduler.FromCurrentSynchronizationContext()).ContinueWith(t =>
    {
        … // #4 compute-intensive work we want offloaded to non-UI thread
    });
}

But neither of those solutions are relevant if the code inside of the continuation is code you don’t own, e.g. if you’re calling out to some 3rd party code which might unsuspectingly use Task.Factory.StartNew without specifying a scheduler an inadvertently end up running its code on the UI thread.  This is why in production library code I write, I always explicitly specify the scheduler I want to use.

For .NET 4.5, we introduced the TaskCreationOptions.HideScheduler and TaskContinuationOptions.HideScheduler values.  When supplied to a Task, this makes it so that in the body of that Task, TaskScheduler.Current returns TaskScheduler.Default, even if the Task is running on a different scheduler: in other words, it hides it, making it look like there isn’t a Task running, and thus TaskScheduler.Default is returned.  This option helps to make your code more reliable if you find yourself calling out to code you don’t own. Again with our initial example, I can now specify HideScheduler, and my bug will be fixed:

private void button_Click(…)
{
    … // #1 on the UI thread
   
Task.Factory.StartNew(() =>
    {
        … // #2 long-running work, so offloaded to non-UI thread
    }).ContinueWith(t =>
    {
        … // #3 back on the UI thread
        Task.Factory.StartNew(() =>
        {
            … // #4 compute-intensive work we want offloaded to non-UI thread (bug!)
        });

    }, CancellationToken.None,
       TaskContinuationOptions.HideScheduler,
       TaskScheduler.FromCurrentSynchronizationContext());
}

One additional thing to note is around the new Task.Run method, which is really just a simple wrapper around Task.Factory.StartNew.  We expect Task.Run to become the most common method for launching new tasks, with developers falling back to using Task.Factory.StartNew directly only for more advanced situations where they need more fine-grained control, e.g. over which scheduler to be targeted.  I already noted that Task.Run specifies DenyChildAttach, so that no tasks created within a Task.Run task can attach to it.  Additionally, Task.Run always specifies TaskScheduler.Default, so that Task.Run always uses the ThreadPool and ignores TaskScheduler.Current.  So, even without HideScheduler, if I’d used Task.Run(Action) instead of Task.Factory.StartNew(Action) in my initially buggy code, it would have been fine.

LazyCancellation

Consider the following code:

Task a = Task.Run(…);
Task b = a.ContinueWith(…, cancellationToken);

The ContinueWith method will create Task ‘b’ such that ‘b’ will be scheduled when ‘a’ completes.  However, because a CancellationToken was provided to ContinueWith, if cancellation is requested before Task ‘a’ completes, then Task ‘b’ will just immediately transition to the Canceled state.  So far so good… there’s no point in doing any work for ‘b’ if we know the user wants to cancel it.  Might as well be aggressive about it.

But now consider a slightly more complicated variation:

Task a = Task.Run(…);
Task b = a.ContinueWith(…, cancellationToken);
Task c = b.ContinueWith(…);

Here there’s a second continuation, off of Task ‘b’, resulting in Task ‘c’.  When Task ‘b’ completes, regardless of what state ‘b’ completes in (RanToCompletion, Faulted, or Canceled), Task ‘c’ will be scheduled.  Now consider the following situation: Task ‘a’ starts running.  Then a cancellation request comes in before ‘a’ finishes, so ‘b’ transitions to Canceled as we’d expect.  Now that ‘b’ is completed, Task ‘c’ gets scheduled, again as we’d expect.  However, this now means that Task ‘a’ and Task ‘c’ could be running concurrently.  In many situations, that’s fine.  But if you’d constructed your chain of continuations under the notion that no two tasks in the chain could ever run concurrently, you’d be sorely disappointed.

Enter LazyCancellation.  By specifying this flag on a continuation that has a CancellationToken, you’re telling TPL to ignore that CancellationToken until the antecedent has already completed.  In other words, the cancellation check is lazy: rather than the continuation doing the work to register with the token to be notified of a cancellation request, it instead doesn’t do anything, and then only when the antecedent completes and the continuation is about to run does it poll the token and potentially transition to Canceled. In our previous example, if I did want to avoid ‘a’ and ‘c’ potentially running concurrently, we could have instead written:

Task a = Task.Run(…);
Task b = a.ContinueWith(…, cancellationToken,
     TaskContinuationOptions.LazyCancellation, TaskScheduler.Default);
Task c = b.ContinueWith(…);

Here, even if cancellation is requested early, ‘b’ won’t transition to Canceled until ‘a’ completes, such that ‘c’ won’t be able to start until ‘a’ has completed, and all would be right in the world again.

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, .NET 4, .NET 4.5"
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 11 Sep 2012 15:55

Given that .NET 4.5 has only recently been released in its final form, it’s not surprising that many folks are still very new to the async/await keywords and have misconceptions about what they are and what they do (I’ve tried to clarify some of these in this Async/Await FAQ).  One of the more common misconceptions is that a method marked as async forces asynchrony, and that’s not true: if there are no awaits in a method marked as async, or if none of the awaits in the async method ever result in suspensions (because they awaited awaitables that were already complete by the time the await occurred), then the method will run synchronously, meaning its code will run on the same thread as the caller and will run to completion before returning to its caller.

Still, there is an interesting boundary established when calling an async method.  When an async method does suspend for the first time, it returns control flow to its caller, effectively forking the computation.  When a call to a synchronous method, e.g. “Foo();”, is replaced by directly awaiting its asynchronous counterpart, e.g. “await FooAsync();”, this forking isn’t obvious, because the join with the forked operation occurs at the await point, immediately after the fork.  If there’s no code between the time the fork occurs and the join occurs, from the perspective of this method, there is no concurrency (I say from the perspective of this method because the thread is being released to do other things during the await, so at a level higher than this method, there could still very well be concurrency).  If, however, there is code between the fork and the join, the opportunity for concurrency from the perspective of this method exists:

var t = FooAsync();
… // code here that may run concurrently with the FooAsync operation
await t;

Not only can the concurrency be achieved between the calling method and the called method, but also between multiple called methods.  The gap between the fork and the join can be used to fork other operations (one of the primary reasons Task.WhenAll exists, to support joining with multiple forked operations at the same time):

var t1 = FooAsync();
… // code
var t2 = BarAsync();
… // code
await Task.WhenAll(t1, t2);

Of course, this concurrency can only be achieved if the async operation actually forks something.  As suggested at the beginning of this post, the async keyword is itself not enough to fork.  For example, there is no forking in the following method:

async Task FooAsync()
{
}

FooAsync();

nor is there any here:

async Task FooAsync()
{
    for(int i=0; i<100000; i++);
}

FooAsync();

nor is there any here:

async Task FooAsync()
{
    for(int i=0; i<100000; i++) 
        await SomeMethodThatNeverSuspendsAsync();
}

FooAsync();

In all of these cases, the call to FooAsync() will start executing the code synchronously, and will finish executing the body of FooAsync synchronously. For a method implemented with async/await to result in the caller having the potential for concurrency, some await inside of the method needs to suspend, such that the async method doesn’t complete synchronously; the async method will at that point return to the caller, enabling the continuation that will continue running the method to run concurrently with the remainder of the caller.

Interestingly, await doesn’t enable a fork with respect to the body of the async method it’s in, only in regards to whatever method is calling the async method.  This is because control flow from the perspective of any individual async method’s execution is serialized; when you await, the rest of that method won’t execute until the awaited awaitable completes, and thus you can’t have multiple awaits currently suspending in the same method at the same time.

To address this, some folks have suggested an additional language construct that would enable this kind of situation, an “async block”, e.g.

async Task FooAsync()
{
    Console.WriteLine(“Starting FooAsync”);
    Task t1 = async {
        Console.WriteLine(“Starting first async block”);
        await Task.Delay(1000);
        Console.WriteLine(“Done first block”);
    };
    Task t2 = async {
        Console.WriteLine(“Starting second async block”);
        await Task.Delay(2000);
        Console.WriteLine(“Done second block”);
    };
    await Task.WhenAll(t1, t2);
    Console.WriteLine(“Done FooAsync”);
}

With this theoretical language feature, the expected output here would be something like:

Starting FooAsync
Starting first async block
Starting second async block
Done first block
Done second block
Done FooAsync

In other words, the first async block began executing synchronously but then yielded when it encountered the first await on a not-yet-completed awaitable (the delay task), at which point the block returned control to the caller, providing a task representing that block’s eventual completion.  Then the second block began executing synchronously, and so on.  This is exactly the behavior you’d expect if this had been rewritten with the blocks extracted into separate methods, e.g.

async Task FooAsync()
{
    Console.WriteLine(“Starting FooAsync”);
    Task t1 = Foo1Async();
    Task t2 = Foo2Async();
    await Task.WhenAll(t1, t2);
    Console.WriteLine(“Done FooAsync”);
}

async Task Foo1Async()
{
    Console.WriteLine(“Starting first async block”);
    await Task.Delay(1000);
    Console.WriteLine(“Done first block”);
}

async Task Foo2Async()
{
   Console.WriteLine(“Starting second async block”);
   await Task.Delay(2000);
   Console.WriteLine(“Done second block”);
}

This mythical language construct is in fact mythical; it doesn’t exist.  However, it doesn’t take much to achieve the same thing (beyond separating out the code into separate named methods, which you can of course do).  In Visual Basic, you can get very close to this, due to VB’s forgiving nature when it comes to using lambdas and automatically inferring an appropriate type for them even when no delegate type is ever mentioned, e.g.

Async Function FooAsync() As Task
    Console.WriteLine("Starting FooAsync")
    Dim t1 = (Async Function()
                  Console.WriteLine("Starting first async block")
                  Await Task.Delay(1000)
                  Console.WriteLine("Done first block")
              End Function)()
    Dim t2 = (Async Function()
                  Console.WriteLine("Starting second async block")
                  Await Task.Delay(2000)
                  Console.WriteLine("Done second block")
              End Function)()
    Await Task.WhenAll(t1, t2)
    Console.WriteLine("Done FooAsync")
End Function

Here I’m able to declare the asynchronous lambda and then immediately invoke it, providing a syntax extremely close to that of the mythical async block feature.  In C#, the same thing can be done, but we need to declare the type for the delegate, making the syntax a bit more cumbersome, e.g.

async Task FooAsync()
{
    Console.WriteLine("Starting FooAsync");
    Task t1 = ((Func<Task>)(async delegate
    {
        Console.WriteLine("Starting first async block");
        await Task.Delay(1000);
        Console.WriteLine("Done first block");
    }))();
    Task t2 = ((Func<Task>)(async delegate
    {
        Console.WriteLine("Starting second async block");
        await Task.Delay(2000);
        Console.WriteLine("Done second block");
    }))();
    await Task.WhenAll(t1, t2);
    Console.WriteLine("Done FooAsync");
}

For cases where I want to use this sort of async block-like construct, I prefer to use a few very simple helper methods:

static void AsyncBlockVoid(Action asyncMethod) { asyncMethod(); }

static Task AsyncBlock(Func<Task> asyncMethod) { return asyncMethod(); }

static Task<T> AsyncBlock<T>(Func<Task<T>> asyncMethod) { return asyncMethod(); }

Each of these methods just accepts a delegate as a parameter and invokes that delegate, but in doing so it provides to the C# compiler enough information for it to infer an appropriate delegate type for the async delegate, e.g.

async Task FooAsync()
{
    Console.WriteLine(“Starting FooAsync”);
    Task t1 = AsyncBlock(async delegate {
        Console.WriteLine(“Starting first async block”);
        await Task.Delay(1000);
        Console.WriteLine(“Done first block”);
    });
    Task t2 = AsyncBlock(async delegate {
        Console.WriteLine(“Starting second async block”);
        await Task.Delay(2000);
        Console.WriteLine(“Done second block”);
    });
    await Task.WhenAll(t1, t2);
    Console.WriteLine(“Done FooAsync”);
}

This gives me a clean and understandable way of achieving the same thing, and without a special language feature.  It also maps nicely to other methods that take a delegate, return a task, but add additional behaviors.  For example, I previously described how these async block constructs resulted in the delegate being invoked synchronously.  What if I wanted it to be invoked asynchronously?  I can just change “AsyncBlock” above to “Task.Run”, and I get the same behavior except with the async delegate being invoked asynchronously rather than synchronously, e.g.

async Task FooAsync()
{
    Console.WriteLine(“Starting FooAsync”);
    Task t1 = Task.Run(async delegate {
        Console.WriteLine(“Starting first async block”);
        await Task.Delay(1000);
        Console.WriteLine(“Done first block”);
    });
    Task t2 = Task.Run(async delegate {
        Console.WriteLine(“Starting second async block”);
        await Task.Delay(2000);
        Console.WriteLine(“Done second block”);
    });
    await Task.WhenAll(t1, t2);
    Console.WriteLine(“Done FooAsync”);
}

Enjoy.

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 22 Aug 2012 00:19

As just announced on the Base Class Libraries (BCL) team blog, the RTM release of TPL Dataflow is now available.

Enjoy :)

Author: "Stephen Toub - MSFT" Tags: "Parallel Extensions, Async, Dataflow, .N..."
Comments Send by mail Print  Save  Delicious 
Date: Wednesday, 15 Aug 2012 23:00

In a post a while ago, I talked about sequential composition of asynchronous operations.  Now that we have the async/await keywords in C# and Visual Basic, such composition is trivial, and async/await are indeed the recommended way to achieve such composition with these languages.

However, in that post I also described a few “Then” methods (similar methods are now available in JavaScript and VC++), and I thought it’d be fun to quickly show how Then can be implemented in terms of await. We’d probably want to support Then methods that operate on either Task or Task<TResult>, and then run a continuation delegate that returns either void or TNewResult.  This leads to four overloads, each of which can be implemented with just one or two lines of code:

public static async Task Then(
    this Task antecedent, Action continuation)
{
    await antecedent;
    continuation();
}

public static async Task<TNewResult> Then<TNewResult>(
    this Task antecedent, Func<TNewResult> continuation)
{
    await antecedent;
    return continuation();
}

public static async Task Then<TResult>(
    this Task<TResult> antecedent, Action<TResult> continuation)
{
    continuation(await antecedent);
}

public static async Task<TNewResult> Then<TResult,TNewResult>(
    this Task<TResult> antecedent, Func<TResult,TNewResult> continuation)
{
    return continuation(await antecedent);
}

Simple, right?  In each case, we just await the supplied task and then run the supplied continuation delegate (potentially passing in the result of the antecedent task).  Exception handling is all handled correctly here as well, which is extra exciting since we didn’t have to do anything special with regards to exceptions: any exceptions from either a non-successful antecedent task or from invoking the continuation delegate are automatically stored into the task returned from Then.

There are a few additional overloads we might also want.  These previous overloads work with continuation functions that return void or TNewResult, rather than Task or Task<TNewResult>.  If you want to use await within the continuation functions, or if you otherwise want to return a Task from the delegate, additional overloads could help to automatically unwrap the task.  As before, these Then overloads are simple to write, with just a few tweaks from the previously shown overloads:

public static async Task Then(
    this Task task, Func<Task> continuation)
{
    await task;
    await continuation();
}

public static async Task<TNewResult> Then<TNewResult>(
    this Task task, Func<Task<TNewResult>> continuation)
{
    await task;
    return await continuation();
}

public static async Task Then<TResult>(
    this Task<TResult> task, Func<TResult,Task> continuation)
{
    await continuation(await task);
}

public static async Task<TNewResult> Then<TResult, TNewResult>(
    this Task<TResult> task, Func<TResult, Task<TNewResult>> continuation)
{
    return await continuation(await task);
}

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Code Samples, Par..."
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 02 Aug 2012 14:58

Recently I’ve had several folks ask me about how to process the results of tasks as those tasks complete.

A developer will have multiple tasks representing asynchronous operations they’ve initiated, and they want to process the results of these tasks, e.g.

List<Task<T>> tasks = …;
foreach(var t in tasks) {
    try { Process(await t); }
    catch(OperationCanceledException) {}
    catch(Exception exc) { Handle(exc); }
}

This approach is fine for many situations.  However, it enforces an additional constraint on the processing that wasn’t actually implied by the initial problem statement: this code ends up processing the tasks in the order they were supplied rather than in the order they complete, which means some tasks that have already completed might not be made available for processing, because an earlier task in the order may not have completed yet.

There are many ways to implement a solution like this.  One approach involves simply using Task’s ContinueWith method, e.g.

List<Task<T>> tasks = …;
foreach(var t in tasks)
    t.ContinueWith(completed => {
        switch(completed.Status) {
            case TaskStatus.RanToCompletion: Process(completed.Result); break;
            case TaskStatus.Faulted: Handle(completed.Exception.InnerException); break;
        }
    }, TaskScheduler.Default);

This solution actually alleviates an additional constraint, which is that the original solution forced the processing of all of the continuations to run serially (and on the original SynchronizationContext if there was one), whereas this allows the processing to run in parallel and on the ThreadPool.  If you wanted to go with this approach but also wanted to force the tasks to run serially, you could do so by supplying a serializing scheduler to ContinueWith (i.e. a scheduler that forced the continuations to run exclusively with regards to each other).  For example, you could use a ConcurrentExclusiveSchedulerPair, e.g.

List<Task<T>> tasks = …;
var scheduler = new ConcurrentExclusiveSchedulerPair().ExclusiveScheduler;
foreach(var t in tasks)
    t.ContinueWith(completed => {
        switch(completed.Status) {
            case TaskStatus.RanToCompletion: Process(completed.Result); break;
            case TaskStatus.Faulted: Handle(completed.Exception.InnerException); break;
        }
    }, scheduler);

or if, for example, you were on the UI thread of your application, you could supply a scheduler that represents that UI thread, e.g.

var scheduler = TaskScheduler.FromCurrentSynchronizationContext();

Even so, this ContinueWith-based approach does force you into a callback-based model to handle your processing.  If you wanted the process-serially-as-the-tasks-complete behavior, but with using async/await rather than using ContinueWith, that’s also possible.

There are a few ways to achieve this.  One relatively simple way is by using Task.WhenAny.  WhenAny accepts a set of tasks and will asynchronously provide the first one that completes.  As such, you can repeatedly invoke WhenAny, each time removing the previously completed task such that you’re asynchronously waiting for the next to complete, e.g.

List<Task<T>> tasks = …;
while(tasks.Count > 0) {
    var t = await Task.WhenAny(tasks);
    tasks.Remove(t);

    try { Process(await t); }
    catch(OperationCanceledException) {}
    catch(Exception exc) { Handle(exc); }
}

Functionally, this is fine, and as long as the number of tasks is small, the performance of this should be fine, too.  However, if the number of tasks is large here, this could result in non-negligible performance overheads.  What we’ve effectively created here is an O(N2) algorithm: for each task, we search the list for the task to remove it, which is an O(N) operation, and we register a continuation with each task, which is also an O(N) operation.  For example, if we had 10,000 tasks, over the span of this whole operation we’d end up registering and unregistering upwards of 50 million continuations as part of the WhenAny calls.  Now, that’s not quite as bad as it sounds, as WhenAny is smart about how it manages its resources, for example not registering continuations with tasks that are already completed, stopping as soon as it finds a completed task, reusing the same continuation object across all of the tasks, etc.  Still, there’s work here we can avoid if profiling deems it to be problematic.

An alternate approach is to create a new “combinator” method specifically geared towards this purpose.  When working with a collection of Task<T> instances, WhenAny returns a Task<Task<T>>; this is a task that will complete when the first supplied task has completed, and that Task<Task<T>>’s Result will be that first completed task.  In our case, we don’t just want the first task, but rather we want all of the tasks, ordered in how they’ll complete.  We can represent this with a Task<Task<T>>[].  Think of this as an array of buckets where we’ll place the input tasks as they complete, one task per bucket.  So, if you want to get the first task to complete, you can await the first bucket of this array, and if you want to get the sixth task to complete, you can await the sixth bucket of this array.

public static Task<Task<T>> [] Interleaved<T>(IEnumerable<Task<T>> tasks)
{
    var inputTasks = tasks.ToList();

    var buckets = new TaskCompletionSource<Task<T>>[inputTasks.Count];
    var results = new Task<Task<T>>[buckets.Length];
    for (int i = 0; i < buckets.Length; i++) 
    {
        buckets[i] = new TaskCompletionSource<Task<T>>();
        results[i] = buckets[i].Task;
    }

    int nextTaskIndex = -1;
    Action<Task<T>> continuation = completed =>
    {
        var bucket = buckets[Interlocked.Increment(ref nextTaskIndex)];
        bucket.TrySetResult(completed);
    };

    foreach (var inputTask in inputTasks)
        inputTask.ContinueWith(continuation, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);

    return results;
}

So what’s happening here?  First we force our enumerable of tasks into a List<Task<T>>; this is to ensure that any tasks that might be produced lazily by enumerating the enumerable are reified once.  Next, we create TaskCompletionSource<Task<T>> instances to represent the buckets, one bucket per each of the tasks that will eventually complete.  Then we hook up a continuation to each input task: this continuation will get the next available bucket and store the newly completed task into it.  With this combinator, we can now rewrite our original code as follows:

List<Task<T>> tasks = …;
foreach(var bucket in Interleaved(tasks)) {
    var t = await bucket;
    try { Process(await t); }
    catch(OperationCanceledException) {}
    catch(Exception exc) { Handle(exc); }
}

To close out this discussion, let’s look at an example of what this actually does in practice.  Consider the following:

var tasks = new[] { 
    Task.Delay(3000).ContinueWith(_ => 3),
    Task.Delay(1000).ContinueWith(_ => 1), 
    Task.Delay(2000).ContinueWith(_ => 2),
    Task.Delay(5000).ContinueWith(_ => 5),
    Task.Delay(4000).ContinueWith(_ => 4),
};
foreach (var bucket in Interleaved(tasks)) {
    var t = await bucket;
    int result = await t;

    Console.WriteLine("{0}: {1}", DateTime.Now, result);
}

Here we have an array of Task<int>, each of will complete after N seconds and return the integer N (e.g. the first task in the array will complete after 3 seconds and return the number 3).  We then loop through these tasks using our handy Interleaved method, printing out results as we get them.  When I run this code, I see the following output:

8/2/2012 7:37:48 AM: 1
8/2/2012 7:37:49 AM: 2
8/2/2012 7:37:50 AM: 3
8/2/2012 7:37:51 AM: 4
8/2/2012 7:37:52 AM: 5

and this is exactly the behavior we wanted.  Note the times that each element was output. All of the tasks were started at the same time, so their timers are all running concurrently.  As each task completes, our loop is able to process it, and as a result we get one line of output each second. 

In contrast, consider the same example but not using Interleaved:

var tasks = new[] { 
    Task.Delay(3000).ContinueWith(_ => 3),
    Task.Delay(1000).ContinueWith(_ => 1), 
    Task.Delay(2000).ContinueWith(_ => 2),
    Task.Delay(5000).ContinueWith(_ => 5),
    Task.Delay(4000).ContinueWith(_ => 4),
};
foreach (var t in tasks) {
    int result = await t;
    Console.WriteLine("{0}: {1}", DateTime.Now, result);
}

When I run this variation, I see:

8/2/2012 7:42:08 AM: 3
8/2/2012 7:42:08 AM: 1
8/2/2012 7:42:08 AM: 2
8/2/2012 7:42:10 AM: 5
8/2/2012 7:42:10 AM: 4

Now look at the times.  Because we’re now processing the tasks in order, we couldn’t print out the results for the 1s task or the 2s task until the 3s task had completed (since it was before them in the array).  Similarly, we couldn’t print out the result for the 4s task until the 5s task completed.

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Friday, 15 Jun 2012 18:10

I’ve been asked a few times recently various questions about ExecutionContext and SynchronizationContext, for example what the differences are between them, what it means to “flow” them, and how they relate to the new async/await keywords in C# and Visual Basic.  I thought I’d try to tackle some of those questions here.

WARNING: This post goes deep into an advanced area of .NET that most developers never need to think about.

What is ExecutionContext, and what does it mean to flow it?

ExecutionContext is one of those things that the vast majority of developers never need to think about.  It’s kind of like air: it’s important that it’s there, but except at some crucial times (e.g. when something goes wrong with it), we don’t think about it being there.  ExecutionContext is actually just a container for other contexts.  Some of these other contexts are ancillary, while some are vital to the execution model of .NET, but they all follow the same philosophy I described for ExecutionContext: if you have to know they’re there, either you’re doing something super advanced, or something’s gone wrong.

ExecutionContext is all about “ambient” information, meaning that it stores data relevant to the current environment or “context” in which you’re running.  In many systems, such ambient information is maintained in thread-local storage (TLS), such as in a ThreadStatic field or in a ThreadLocal<T>.  In a synchronous world, such thread-local information is sufficient: everything’s happening on that one thread, and thus regardless of what stack frame you’re in on that thread, what function is being executed, and so forth, all code running on that thread can see and be influenced by data specific to that thread.  For example, one of the contexts contained by ExecutionContext is SecurityContext, which maintains information like the current “principal” and information about code access security (CAS) denies and permits.  Such information can be associated with the current thread, such that if one stack frame denies access to a certain permission and then calls into another method, that called method will still be subject to the denial set on the thread: when it tries to do something that needs that permission, the CLR will check the current thread’s denials to see if the operation is allowed, and it’ll find the data put there by the caller.

Things get more complicated when you move from a synchronous world to an asynchronous world.  All of a sudden, TLS becomes largely irrelevant.  In a synchronous world, if I do operation A, then operation B, and then operation C, all three of those operations happen on the same thread, and thus all three of those are subject to the ambient data stored on that thread. But in an asynchronous world, I might start A on one thread and have it complete on another, such that operation B may start or run on a different thread than A, and similarly such that C may start or run on a different thread than B.  This means that this ambient context we’ve come to rely on for controlling details of our execution is no longer viable, because TLS doesn’t “flow” across these async points.  Thread-local storage is specific to a thread, whereas these asynchronous operations aren’t tied to a specific thread.  There is, however, typically a logical flow of control, and we want this ambient data to flow with that control flow, such that the ambient data moves from one thread to another.  This is what ExecutionContext enables.

ExecutionContext is really just a state bag that can be used to capture all of this state from one thread and then restore it onto another thread while the logical flow of control continues.  ExecutionContext is captured with the static Capture method:

// ambient state captured into ec
ExecutionContext ec = ExecutionContext.Capture();

and it’s restored during the invocation of a delegate via the static run method:

ExecutionContext.Run(ec, delegate
{
    … // code here will see ec’s state as ambient
}, null);

All of the methods in the .NET Framework that fork asynchronous work capture and restore ExecutionContext in a manner like this (that is, all except for those prefixed with the word “Unsafe,” which are unsafe because they explicitly do not flow ExecutionContext).  For example, when you use Task.Run, the call to Run captures the ExecutionContext from the invoking thread, storing that ExecutionContext instance into the Task object. When the delegate provided to Task.Run is later invoked as part of that Task’s execution, it’s done so via ExecutionContext.Run using the stored context.  This is true for Task.Run, for ThreadPool.QueueUserWorkItem, for Delegate.BeginInvoke, for Stream.BeginRead, for DispatcherSynchronizationContext.Post, and for any other async API you can think of.  All of them capture the ExecutionContext, store it, and then use the stored context later on during the invocation of some code.

When we talk about “flowing ExecutionContext,” we’re talking about exactly this process of taking the state that was ambient on one thread and restoring that state onto a thread at some later point while that thread executes a supplied delegate.

What is SynchronizationContext, and what does it mean to capture and use it?

We in software development love abstractions.  We’re rarely happy hardcoding to a particular implementation; rather, when writing higher-level systems, we abstract away the details of a particular implementation so that we can plug in a different implementation later without having to change our higher-level system.  This is why we have interfaces, this is why we have abstract classes, this is why we have virtual methods, and so on.

SynchronizationContext is just an abstraction, one that represents a particular environment you want to do some work in.  As an example of such an environment, Windows Forms apps have a UI thread (while it’s possible for there to be multiple, for the purposes of this discussion it doesn’t matter), which is where any work that needs to use UI controls needs to happen.  For cases where you’re running code on a ThreadPool thread and you need to marshal work back to the UI so that this work can muck with UI controls, Windows Forms provides the Control.BeginInvoke method.  You give a delegate to a Control’s BeginInvoke method, and that delegate will be invoked back on the thread with which that control is associated.

So, if I’m writing a component that needs to schedule some work to the ThreadPool and then continue with some work back on the UI thread, I can code my component to use Control.BeginInvoke.  But now what if I decide I want to use my component in a WPF app? WPF has the same UI thread constraint that Windows Forms has, but it has a different mechanism for marshaling back to the UI thread: rather than using Control.BeginInvoke on a control associated with the right thread, you use Dispatcher.BeginInvoke (or InvokeAsync) on the Dispatcher instance associated with the right thread. 

We now have two different APIs for achieving the same basic operation, so how do I write my component to be agnostic of the UI framework?  By using SynchronizationContext.  SynchronizationContext provides a virtual Post method; this method simply takes a delegate and runs it wherever, whenever, and however the SynchronizationContext implementation deems fit.  Windows Forms provides the WindowsFormSynchronizationContext type which overrides Post to call Control.BeginInvoke.  WPF provides the DispatcherSynchronizationContext type which overrides Post to call Dispatcher.BeginInvoke.  And so on.  As such, I can now code my component to use SynchronizationContext instead of tying it to a specific framework.

If I was writing my component specifically to target Windows Forms, I might implement my go-to-the-ThreadPool-and-then-back-to-the-UI-thread logic something like the following:

public static void DoWork(Control c)
{
    ThreadPool.QueueUserWorkItem(delegate
    {
        … // do work on ThreadPool
        c.BeginInvoke(delegate
        {
            … // do work on UI
        });
    });
}

If I was instead writing my component to use SynchronizationContext, I might instead write it as:

public static void DoWork(SynchronizationContext sc)
{
    ThreadPool.QueueUserWorkItem(delegate
    {
        … // do work on ThreadPool
        sc.Post(delegate
        {
            … // do work on UI
        }, null);
    });
}

Of course, it’s annoying (and prohibitive for certain desired programming models) to need to pass around the target context to come back to, so SynchronizationContext provides the Current property, which allows you to discover from the current thread the context that will let you get back to the current environment, if there is one.  This allows you to “capture it” (i.e. read the reference from SynchronizationContext.Current and store that reference for later usage):

public static void DoWork()
{
    var sc = SynchronizationContext.Current;
    ThreadPool.QueueUserWorkItem(delegate
    {
        … // do work on ThreadPool
        sc.Post(delegate
        {
            … // do work on the original context
        }, null);
   });
}

Flowing ExecutionContext vs Using SynchronizationContext

Now, we have a very important observation to make: flowing ExecutionContext is semantically very different than capturing and posting to a SynchronizationContext. 

When you flow ExecutionContext, you’re capturing the state from one thread and then restoring that state such that it’s ambient during the supplied delegate’s execution.  That’s not what happens when you capture and use a SynchronizationContext.  The capturing part is the same, in that you’re grabbing data from the current thread, but you then use that state differently.  Rather than making that state current during the invocation of the delegate, with SynchronizationContext.Post you’re simply using that captured state to invoke the delegate.  Where and when and how that delegate runs is completely up to the implementation of the Post method.

How does this apply to async/await?

The framework support behind the async and await keywords automatically interacts with both ExecutionContext and SynchronizationContext.

Whenever code awaits an awaitable whose awaiter says it’s not yet complete (i.e. the awaiter’s IsCompleted returns false), the method needs to suspend, and it’ll resume via a continuation off of the awaiter.  This is one of those asynchronous points I referred to earlier, and thus, ExecutionContext needs to flow from the code issuing the await through to the continuation delegate’s execution.  That’s handled automatically by the Framework.  When the async method is about to suspend, the infrastructure captures an ExecutionContext.  The delegate that gets passed to the awaiter has a reference to this ExecutionContext instance and will use it when resuming the method.  This is what enables the important “ambient” information represented by ExecutionContext to flow across awaits.

The Framework also has support for SynchronizationContext.  The aforementioned support for ExecutionContext is built into the “builders” that represent async methods (e.g. System.Runtime.CompilerServices.AsyncTaskMethodBuilder), and these builders ensure that ExecutionContext is flowed across await points regardless of what kind of awaitable is being used.  In contrast, support for SynchronizationContext is built into the support for awaiting Task and Task<TResult>, specifically.  Custom awaiters could add similar logic themselves, but they don’t get it automatically; that’s by design, as being able to customize when and how the continuation gets invoked is part of why custom awaiters are useful.

When you await a task, by default the awaiter will capture the current SynchronizationContext, and if there was one, when the task completes it’ll Post the supplied continuation delegate back to that context, rather than running the delegate on whatever thread the task completed or rather than scheduling it to run on the ThreadPool.  If a developer doesn’t want this marshaling behavior, it can be controlled by changing the awaitable/awaiter that’s used.  Whereas this behavior is always employed when you await a Task or Task<TResult>, you can instead await the result of calling task.ConfigureAwait(…).  The ConfigureAwait method returns an awaitable that enables this default marshaling behavior to be suppressed.  Whether it’s suppressed is controlled by a Boolean passed to the ConfigureAwait method.  If continueOnCapturedContext is true, then you get the default behavior; if it’s false, the awaiter doesn’t check for a SynchronizationContext, pretending as if there wasn’t one. (Note that when the awaited task completes, regardless of ConfigureAwait, the runtime may check the context that’s current on the resuming thread to determine whether it’s ok to synchronously run the continuation there or whether the continuation must be scheduled asynchronously from that point.)

Note that while ConfigureAwait provides explicit, await-related programming model support for changing behavior related to SynchronizationContext, there is no await-related programming model support for suppressing ExecutionContext flow.  This is on purpose.  ExecutionContext is not something developers writing async code should need to worry about; it’s infrastructure level support that helps to simulate synchronous semantics (i.e. TLS) in an asynchronous world.  Most folks can and should completely ignore that it’s there (and should avoid using the ExecutionContext.SuppressFlow method unless they really know what you’re doing).  In contrast, where code runs is something developers should be cognizant of, and thus SynchronizationContext rises to level of something that does deserve explicit programming model support.  (In fact, as I’ve stated in other posts, most library implementers should consider using ConfigureAwait(false) on every await of a task.)

Isn’t SynchronizationContext part of ExecutionContext?

I’ve glossed over some details up until this point, but I can’t avoid them any further.

The main thing I glossed over is that of all the contexts ExecutionContext is capable of flowing (e.g. SecurityContext, HostExecutionContext, CallContext, etc.), SynchronizationContext is actually one of them.  This is, I personally believe, a mistake in API design, one that’s caused a few problems since it was instituted in .NET many versions ago.  Nevertheless, it’s the design we have and have had for a long time, and changing it now would be a breaking change.

When you call the public ExecutionContext.Capture() method, that checks for a current SynchronizationContext, and if there is one, it stores that into the returned ExecutionContext instance.  Then, when the public ExecutionContext.Run method is used, that captured SynchronizationContext is restored as Current during the execution of the supplied delegate.

Why is this problematic?  Flowing SynchronizationContext as part of ExecutionContext changes the meaning of SynchronizationContext.Current.  SynchronizationContext.Current is supposed to be something you can access to get back to the environment that you're currently in at the time you access Current, so if SynchronizationContext flows to be current on another thread, you can’t trust what SynchronizationContext.Current means.  In such a case, it could either be the way to get back to the current environment, or it could be the way to get back to some environment that occurred at some point previously in the flow.

As one example of how this can be problematic, consider the following code:

private void button1_Click(object sender, EventArgs e)
{
    button1.Text = await Task.Run(async delegate
    {
        string data = await DownloadAsync();
        return Compute(data);
    });
}

Here’s what my mental model tells me will happen with this code.  A user clicks button1, causing the UI framework to invoke button1_Click on the UI thread.  The code then kicks off a work item to run on the ThreadPool (via Task.Run).  That work item starts some download work and asynchronously waits for it to complete.  A subsequent work item on the ThreadPool then does some compute-intensive operation on the result of that download, and returns the result, causing the Task that was being awaited on the UI thread to complete.  At that point, the UI thread processes the remainder of this button1_Click method, storing the result of the computation into the button1’s Text property.

My expectation is valid if SynchronizationContext doesn’t flow as part of ExecutionContext.  If it does flow, however, I will be sorely disappointed.  Task.Run captures ExecutionContext when invoked, and uses it to run the delegate passed to it.  That means that the UI SynchronizationContext which was current when Task.Run was invoked would flow into the Task and would be Current while invoking DownloadAsync and awaiting the resulting task.  That then means that the await will see the Current SynchronizationContext and Post the remainder of asynchronous method as a continuation to run back on the UI thread.  And that means my Compute method will very likely be running on the UI thread, not on the ThreadPool, causing responsiveness problems for my app.

The story now gets a bit messier: ExecutionContext actually has two Capture methods, but only one of them is public.  The internal one (internal to mscorlib) is the one used by most asynchronous functionality exposed from mscorlib, and it optionally allows the caller to suppress the capturing of SynchronizationContext as part of ExecutionContext; corresponding to that, there’s also an internal overload of the Run method that supports ignoring a SynchronizationContext that’s stored in the ExecutionContext, in effect pretending one wasn’t captured (this is, again, the overload used by most functionality in mscorlib).  What this means is that pretty much any asynchronous operation whose core implementation resides in mscorlib won’t flow SynchronizationContext as part of ExecutionContext, but any asynchronous operation whose core implementation resides anywhere else will flow SynchronizationContext as part of ExecutionContext.  I previously mentioned that the “builders” for async methods were the types responsible for flowing ExecutionContext in async methods, and these builders do live in mscorlib, and they do use the internal overloads… as such, SynchronizationContext is not flowed as part of ExecutionContext across awaits (this, again, is separate from how task awaiters support capturing the SynchronizationContext and Post’ing back to it).   To help deal with the cases where ExecutionContext does flow SynchronizationContext, the async method infrastructure tries to ignore SynchronizationContexts set as Current due to being flowed.

In short, SynchronizationContext.Current does not “flow” across await points.

Author: "Stephen Toub - MSFT" Tags: ".NET 4, Async, .NET 4.5"
Comments Send by mail Print  Save  Delicious 
Date: Friday, 15 Jun 2012 00:25

Several weeks ago, I wrote a post for the Windows 8 app developer blog that was all about using await and AsTask to consume WinRT async operations.  I've now published a follow-up post that's all about exposing .NET tasks as WinRT async operation.  In a sense, you can think about the first post as showing how to convert from WinRT async operations to .NET tasks, and you can think about the second post as showing how to convert from .NET tasks to WinRT async operations.  I hope you find them helpful.

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Friday, 01 Jun 2012 06:59

In the previous "What's New for Parallelism in Visual Studio 2012 RC" blog post, I mentioned briefly that for the .NET 4.5 Release Candidate, StreamReader.ReadLineAsync experienced a significant performance improvement over Beta.  There's an intriguing story behind that, one I thought I'd share here.

It has to do with some interesting interactions between certain optimizations in the BCL and how the C# and Visual Basic compilers compile async methods.  Before I describe the changes made, a bit of experimentation will be useful to help set the stage. 

Consider the following code. The code is simply timing how long it takes to run two different async methods (which don’t actually await anything, and so will run to completion synchronously).  Each method increments an integer 100 million times.  The first method does so by incrementing a field 100 million times, and the second method does so by copying the field into a local, incrementing that local 100 million times, and then storing the local back to the field.

using System;
using System.Diagnostics;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        var sw = new Stopwatch();
        var obj = new MyObj();
        while (true)
        {
            foreach (var m in new Func<Task>[] { 
                new MyObj().Foo1, new MyObj().Foo2 })
            {
                sw.Restart();
                m();
                sw.Stop();
                Console.WriteLine("{0}: {1}", m.Method.Name, sw.Elapsed);
            }
            Console.WriteLine();
        }
    }
}

class MyObj
{
    const int ITERS = 100000000;
    private int m_data;

    public async Task Foo1()
    {
        for (int i = 0; i < ITERS; i++) m_data++;
    }

    public async Task Foo2()
    {
        int localData = m_data;
        for (int i = 0; i < ITERS; i++) localData++;
        m_data = localData;
    }
}

When I run this on the laptop on which I’m writing this blog post, I get output numbers like the following:

Foo1: 00:00:00.2289705
Foo2: 00:00:00.0755818

In other words, this particular microbenchmark implies that accessing a local (which in this case is actually compiled to be a field on the stack-allocated struct representing the async state machine) is approximately three times faster than accessing a class field.  Now, let’s make a one line change to our example.  Instead of:

class MyObj

we’ll change that to be:

class MyObj : MarshalByRefObject

Now when I run the sample, I see much different output for Foo1:

Foo1: 00:00:05.4411074
Foo2: 00:00:00.0757737

Woah!  Notice that our Foo2 method took basically the same amount of time as it did previously, but the amount of time it took to execute Foo1 skyrocketed in comparison to its previous cost.  Whereas previously it was only 3x more expensive to use the field than the local, now it’s ~72x more expensive.  What happened?

The issue here has to do with .NET Remoting.  According to MSDN documentation, MarshalByRefObject “enables access to objects across application domain boundaries.”  What this means is that you can construct a MarshalByRefObject in one AppDomain, where it will live, and then hand out to other AppDomains references to that object.   In those other AppDomains, the developer’s code will still be accessing the object via its type, but that AppDomain doesn’t actually have a direct reference to the object; rather, the CLR has played some magic under the covers, such that the other AppDomain actually holds onto a proxy object.  You can see this with a modification to our earlier sample by using the System.Runtime.Remoting.RemotingServices.IsTransparentProxy method:

var obj1 = new MyObj();
Console.WriteLine(RemotingServices.IsTransparentProxy(obj1));

var ad = AppDomain.CreateDomain("myDomain");
var obj2 = (MyObj)ad.CreateInstanceAndUnwrap(
    typeof(MyObj).Assembly.FullName, typeof(MyObj).FullName);
Console.WriteLine(RemotingServices.IsTransparentProxy(obj2));

This will output ‘false’ for obj1 but ‘true’ for obj2.

Ok, so the CLR supports proxy objects… how is that relevant?  When you access a field on a MarshalByRefObject, the CLR needs to check whether the object being accessed lives in the current domain or whether it actually lives in the remote domain, in which case it needs to marshal the request across the AppDomain boundary.  That check, which is inserted by the JIT compiler, can be relatively expensive (at least when compared to the normal cost of accessing a field).  As such the CLR has some optimizations in place to mitigate these costs.  In particular, it special cases access to fields on ‘this’, since if you’re accessing ‘this’, it means you’re already running in a method on the target object, which means you must already be in the same AppDomain as the field.  This optimization has served the CLR very well, as most fields are private and thus are only accessed by that object.  It’s worked so well that very few people ever noticed these costs… that is, until async methods entered the picture.

Remember how async methods are compiled.  If you have an async method BarAsync:

public class Foo
{
    private int m_data;

    public async Task BarAsync()
    {
        ...
        m_data = 42;
        ...
    }
}

the compiler will generate a state machine type for BarAsync, and the contents of the BarAsync method will be compiled into a MoveNext method on that type, e.g.

public class Foo
{
    private int m_data;

    public async Task BarAsync()
    {
        var builder = AsyncTaskMethodBuilder.Create();
        var sm = new BarAsyncStateMachine();
        sm.<>this = this;
        …
        builder.Start(ref sm);
        return sm.Task;
    }

    private struct BarAsyncStateMachine : IAsyncStateMachine
    {
        private Foo <>this;
        ...
        public void MoveNext()
        {
            ...
            <>this.m_data = 42;
            ...
        }
    }
}

Note that the MoveNext method is no longer accessing m_data on ‘this’, as the m_data field is now on a separate instance.  As such, when the JIT compiles this method, it’s unable to apply the aforementioned optimization, and we end up getting the cross-AppDomain checks.  This is what’s causing the significant difference in our earlier microbenchmark.

In general, this shouldn’t be a big deal, as very few types actually inherit from MarshalByRefObject, and of those few that do, very few will have async methods on them, and of those that do, very few of those async methods will make frequent enough accesses to fields for this to be a measurable problem.  Unfortunately, there is an important set of types and methods that do meet all of these conditions: System.IO.  The Stream, TextReader, and TextWriter classes all derive from MarshalByRefObject, which means that any async method implemented on these or a derived type is susceptible to this problem.  Further, they deal with I/O, which means there are often potentially long-latency operations involved that warrant using asynchrony.  And there’s often some amount of data buffering done in order to limit the number of long-latency calls made, which means potentially frequent access to fields storing the buffered data.

We analyzed all of the known cases in the BCL where this could potentially be a problem, and we fixed those.  The most high profile and most impacted example of this was the StreamReader.ReadLineAsync method.  ReadLineAsync internally has a tight loop which was making multiple accesses to fields on the StreamReader, and thus this particular issue was very noticeable; by fixing this in ReadLineAsync, we saw a several fold increase in ReadLineAsync’s performance between Beta and the Release Candidate.

Note that this issue wasn’t addressed by adding new optimizations to JIT, but rather by running performance tests, finding methods that weren’t meeting performance goals, measuring to determine the cost, and in the offending cases, addressing the issue by minimizing the number of times the fields were accessed.  If you have any custom Stream, StreamReader, or StreamWriter types on which you’ve implemented async methods and that access fields on the type very frequently, you might also want to measure and consider whether you’re impacted by this.  If you are, you have a few possible workarounds.  One is to use a local, as I did in my original MyObj example.  If that works for you, that’s the best solution.  If that isn’t feasible given your code, a second workaround is to create a simple property on the MarshalByRefObject-derived type; that property’s getter should return the value of the field and its setter should set it, and then you can use that property from your async method as a stand-in for the local.   The JIT has additional optimizations in place for dealing with properties on MarshalByRefObjects, because it’s much more likely that a property will be exposed publicly than it is for a field to be exposed publicly.  If I go back to my original repro and add the following additional implementation:

private int Data { get { return m_data; } set { m_data = value; } }

public async Task Foo3()
{
    for (int i = 0; i < ITERS; i++) Data++;
}

I then see this output:

Foo1: 00:00:05.1012831
Foo2: 00:00:00.0758750
Foo3: 00:00:00.8242660

Note that the approach which uses the property is still noticeably slower than the local-based approach, but it’s also 6x faster than the original flawed version. Again, though, only consider making such changes to your code if measurements actually show this issue to be impactful… the majority of the cases we looked at in the BCL were not impacted, as the other costs in the method dwarfed any overheads incurred from accessing fields.

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 31 May 2012 19:57

In September, I blogged about what was new for parallelism and asynchrony in the Visual Studio 2012 Developer Preview, and in February I followed that up with a post on what was new in the Beta.  Now that Visual Studio 2012 Release Candidate is out, I want to share a few thoughts on what’s new in the Release Candidate.

Most new features for a release of Visual Studio and the .NET Framework show up prior to a Release Candidate, which is typically focused on polish, performance, and the like.  That’s true for our work around asynchrony and parallelism in Visual Studio 2012, with a fair amount of work being done under the covers to improve stability and performance.  However, there are still some visible features you’ll see show up for the first time in the Release Candidate.

Compilation Changes

Between the Developer Preview and the Beta, there were some significant changes made to how the C# and Visual Basic compilers compiled async methods.  We pushed to get as many such necessary changes made by the Beta so as to minimize that kind of churn for the Release Candidate and beyond.  As such, the patterns employed by the compiler remain the same since Beta.

However, there is one new addition that will be of most value to tool developers.  If you use a tool like ildasm.exe to look at a compiled async method, you’ll notice a new attribute is now used.  For example, the method:

public static async Task SomeMethodAsync() { … }

will end up getting compiled something like the following:

[AsyncStateMachine(typeof(<SomeMethodAsync>d__d))]
[DebuggerStepThrough]
public static Task SomeMethodAsync() { … }

The employed AsyncStateMachineAttribute contains a reference to the state machine type the compiler generated for this async method.  This allows a tool to easily navigate from an async method stub method to the state machine type that backs it.

Performance

As was true from the Async CTP to the Developer Preview and from the Developer Preview to the Beta, for the Release Candidate a lot of work has gone into tuning the performance of the new async story.  Here are a few examples of ways in which we’ve tweaked the system. (As is typically true of performance improvements, this is all implementation detail and could of course change in the future.)

At the runtime level, we’ve focused a lot on the performance of the Framework components that back the code generated by the compiler when you use the async/await keywords.  For the Release Candidate, we’ve made multiple changes here that further reduce the number and size of allocations used by the system, bringing the overhead down to a bare minimum.  We’ve also looked at allocations in systems closely related to Tasks and asynchrony, such as CancellationToken; as an example, we reduced the typical memory usage of CancellationTokenSources created by CreateLinkedTokenSource by around 40%. Such memory improvements also help with improved throughput, but in addition we focused specifically on improving throughput by trimming out as much code as possible from key async-related fast paths throughout the Task Parallel Library.  For example, we improved the lifecycle performance of creating and completing a TaskCompletionSource<TResult> by around 25%.

Additional async-related performance improvements were made throughout the base class libraries (BCL).  For example, StreamReader.ReadLineAsync has seen significant performance improvements, in some cases improving throughput by as much as 300%. BufferedStream’s implementations of ReadAsync and WriteAsync were further optimized for the common case of reading already-buffered data from the stream or writing to the stream an amount of data that would fit entirely in the buffer; both of these cases result in the operations completing synchronously, and those cases were optimized to achieve significant performance boosts.  Related, the AsStream, AsStreamForRead, and AsStreamForWrite extension methods in System.Runtime.WindowsRuntime.dll now use BufferedStream internally, yielding performance gains due to avoiding repeated interop costs.  And so on.

ASP.NET

ASP.NET has made several useful improvements around asynchrony for the Release Candidate.

One important addition has to do with async methods on an ASP.NET Page.  It was natural in the Beta to expect that you could just add the async keyword to a page-level method, e.g.

async void Button1_Click(object sender, EventArgs e)
{
    … // code that uses await
}

However, code that did this was potentially quite buggy.  The issue here is that async void methods don’t give back a handle that ASP.NET can use to monitor the operation for completion, and thus it relies on its SynchronizationContext (and notifications from an async void method to that SynchronizationContext) to know when all of the async work is completed.  In Beta, ASP.NET only had a single synchronization point which checked and asynchronously waited for all async operations to complete, and this synchronization point occurred fairly late in the page lifecycle.  Work like data binding and generating the control tree occur before that synchronization point.  Thus, if the async void method tried to do anything whose output would affect data binding or generating the control tree, it was non-deterministic whether those changes would actually have an effect, due to the race between the asynchronous operations completing and the page’s lifecycle progressing.

The recommended approach has been and continues to be using the Page.RegisterAsyncTask method with “async Task” methods rather than using “async void”.  This allows the developer to be very clear on their intent, giving ASP.NET a delegate that can be used both to invoke the operation at a specific point and to await the operation's completion (via the task returned by the delegate), e.g. instead of writing:

async void Button1_Click(object sender, EventArgs e)
{
    … // code that uses await
}

you'd write:

void Button1_Click(object sender, EventArgs e)
{
    RegisterAsyncTask(new PageAsyncTask(async () =>
    {
        … // code that uses await
    }));
}

However, since it was so easy to assume different behavior for “async void” methods, for the Release Candidate, ASP.NET now does allow you to write “async void” methods and have them behave how you expect, aligning better with how asynchronous event handlers behave in UI frameworks like Windows Forms, WPF, and Metro style apps.

Another async-related improvement in ASP.NET shows up on the HttpRequest and HttpResponse objects and has to do with cancellation.  Since ASP.NET’s creation, ASP.NET has supported page timeout, where if a page takes too long to complete (by default, more than 110 seconds), the thread processing the page will be aborted.  This behavior, while well intended and sometimes helpful, has also led to reliability issues due to the dangers of aborting threads at arbitrary locations (ASP.NET has the benefit here that it can spin up new AppDomains and spin down old ones to help work around state corruption problems); it also doesn’t work well with asynchrony, where there may not be a thread to abort.  A more reliable approach to timeouts is via a cooperative cancellation mechanism.  Imagine that ASP.NET provided a CancellationToken that developers could pass around in their code to all cancelable async operations, and then if the page takes too long to complete, that token could have cancellation requested.  This is exactly what ASP.NET now provides in the form of HttpRequest.TimedOutToken.  TimedOutToken returns a CancellationToken that will have cancellation requested if the page takes “too long” to process.

Another related condition occurs when the client that was connected to the server to cause the page to be processed then prematurely disconnects.  In that case, you’d like for asynchronous operations to be able to end early so as not to continue doing work that’s no longer necessary.  In the Release Candidate, ASP.NET now also addresses this case, by providing the HttpResponse.ClientDisconnectedToken property.  ClientDisconnectedToken returns a CancellationToken that will have cancellation requested when the client disconnects.

Dataflow

In the Developer Preview and Beta, the new System.Threading.Tasks.Dataflow.dll library shipped as part of the full .NET Framework package.  It was, however, not available to Metro style apps, and we got feedback from developers that the ability to use the dataflow library in a portable fashion, across both their desktop and Metro style apps, was important.  For the Release Candidate, we’re taking advantage of a new distribution approach from the BCL team, which is to support shipping Framework assemblies on NuGet.  Starting with the Release Candidate, you can now get a portable dataflow library (one that works both with the full .NET Framework and with .NET for Metro style apps) via NuGet.  For more information, see the BCL Team blog.

Conclusion

As always, we continue to march forward thinking a lot about these scenarios and how we can improve them, both from a programmability perspective and from a performance perspective.  If you have any feedback, we’re always interested in hearing it.

Thanks, and enjoy!

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Parallel Extensio..."
Comments Send by mail Print  Save  Delicious 
Date: Tuesday, 08 May 2012 17:43

Since .NET 4’s release, I’ve received several questions about a peculiar behavior of ConcurrentQueue<T> having to do with memory management.

With Queue<T>, List<T>, and other such data structures in the .NET Framework, when you remove an element from the collection, the collection internally wipes out its reference to the stored element, e.g.

class MyQueue<T>
{
    private T[] m_data;
    private int m_head;
    ...
    public T Dequeue()
    {
        ...

        T value = m_data[m_head];
        m_data[m_head] = default(T);
        m_head = (m_head + 1) % m_data.Length;
        ...
        return value;
    }
}

This ensures that the data structure won’t hold on to the dequeued data in the structure after the data has been removed, such that if the consumer also drops all references, any referenced data can be garbage collected. 

This behavior is intuitive, which is a key reason it’s concerned some folks that ConcurrentQueue<T> in .NET 4 doesn’t behave exactly like this.  For its implementation in .NET 4 (as with all discussions of implementation, these details can change in the future), ConcurrentQueue<T> is made up of a linked list of array segments, each of which can store up to 32 elements.  As elements are enqueued, they fill up the tail array segment, and when it’s full, a new tail segment is tacked on to the linked list.  When elements are dequeued, they’re taken from the head segment, and when all elements have been taken from the segment, the segment is removed from the linked list and all references to it dropped.  Since the queue only stores a reference to an enqueued item in one segment, dropping that segment removes all references to the enqueued item (for the purposes of this description, if the same item is enqueued twice, I’m treating that as two distinct items).  However, multiple items are stored per segment, so dequeuing an element doesn’t cause the segment to be removed from the linked list unless the element is the last remaining one in the segment.  And in .NET 4, ConcurrentQueue<T> doesn’t null out individual items in the segment as they’re dequeued.  All of this means that the queue might hold on to the last <= 31 dequeued elements.

If the elements are small, you’ll probably never notice this.  If, however, the elements hold on to large resources (e.g. each element is a huge image bitmap), it’s possible you could see the impact of this (one workaround is to queue a wrapper object, e.g. have a ConcurrentQueue<StrongBox<T>> rather than a ConcurrentQueue<T>, and null out the wrapper’s reference to the T value after the wrapper has been dequeued).

For better or worse, this behavior in .NET 4 is actually “by design.”  The reason for this has to do with enumeration semantics.  ConcurrentQueue<T> provides “snapshot semantics” for enumeration, meaning that the instant you start enumerating, ConcurrentQueue<T> captures the current head and tail of what’s currently in the queue, and even if those elements are dequeued after the capture or if new elements are enqueued after the capture, the enumeration will still return all of and only what was in the queue at the time the enumeration began.  If elements in the segments were to be nulled out when they were dequeued, that would impact the veracity of these enumerations.

For .NET 4.5, we’ve changed the design to strike what we believe to be a good balance.  Dequeued elements are now nulled out as they’re dequeued, unless there’s a concurrent enumeration happening, in which case the element isn’t nulled out and the same behavior as in .NET 4 is exhibited.  So, if you never enumerate your ConcurrentQueue<T>, dequeues will result in the queue immediately dropping its reference to the dequeued element.  Only if when the dequeue is issued someone happens to be enumerating the queue (i.e. having called GetEnumerator on the queue and not having traversed the enumerator or disposed of it yet) will the null’ing out not happen; as with .NET 4, at that point the reference will remain until the containing segment is removed.

Author: "Stephen Toub - MSFT" Tags: "Parallel Extensions, Coordination Data S..."
Comments Send by mail Print  Save  Delicious 
Date: Thursday, 26 Apr 2012 21:10

We’re happy to announce that you can now download an Async Targeting Pack for Visual Studio 11 that lets you target .NET 4 and Silverlight 5.  The included DLLs address the previously discussed issue of the Visual Studio 11 Beta compilers being incompatible with the AsyncCtpLibrary* DLLs from the Async CTP; with this targeting pack, you can use the async/await keywords with Visual Studio 11 Beta to compile for .NET 4 and Silverlight 5.  The DLLs are available via a NuGet package, with installation details available here. Enjoy!

Author: "Stephen Toub - MSFT" Tags: "Task Parallel Library, Code Samples, Par..."
Comments Send by mail Print  Save  Delicious 
Next page
» You can also retrieve older items : Read
» © All content and copyrights belong to their respective authors.«
» © FeedShow - Online RSS Feeds Reader