Ensuring thread safety of a ReactiveDerivedCollection

[Scooty Code]

Hi all,

I feel like I have a solved question, but I can't seem to put my finger on the solution.  I've been using ReactiveUI for probably six months or so, and though I was a complete Rx newbie (in fact, new to WPF, MVVM, Rx, and Linq), I feel like I've come at least some way towards understanding what's going on.  I've regularly pestered my coworkers with "come check this out" and proceeded to show them a line that I'm sure looked like absolute gibberish to them, but I think they believed my repeated assertions that it was in fact, awesome.  Over time, they've come to an understanding as well, though they still regard some of it as "magic code".  One of those things is the derived collection.

My model objects implement INotifyPropertyChanged, and expose ReactiveCollections so that we properly get updates in our viewmodels, of course.  Now, we immediately hit the problem where our Model data comes in on a non-dispatcher thread.  So I created a ThreadedReactiveCollection, which invokes changes to itself to the dispatcher.  This, while probably a hack (feel free to tell me a better way, there's a LOT of this I don't know), has worked fine.  ThreadedReactiveCollection derives from ReactiveCollection, but overrides Add, Remove, etc and only invokes if it is required.  That all works properly, if slightly inelegantly.

However, the derived collection ALMOST works.  Because of the invoke occurring in the ThreadedReactiveCollection, itemAdded and itemRemoved appear to also fire in the dispatcher thread, and items are added and removed from the derived collection without any threading issues.  However, if I turn on change tracking on the ReactiveCollection, the item change notifications are NOT on the dispatcher, causing an exception.  I tried creating the derived collection from the Changed observable of the ReactiveCollection (so that I could ObserveOn), but doesn't that lose the extra "magic" that you get from creating the derived collection from a ReactiveCollection in the first place?

So I'm trying to figure out where I've gone wrong.  Is it my design, my implementation, or is there a magic trick I don't know?  I feel like this has to be a solved problem.  I have Reactive model objects so I know when they change, I have reactive collections and derived collections in my viewmodels which are bound to my UI.  My data processing occurs in worker threads, and my UI thread stays "free" of blocking calls.  I can't override ReactiveDerivedCollection (it is a sealed class, tricksy Mr. Betts!), so I can't force changes to it onto the dispatcher.

I admit there is a lot I don't know about these frameworks and while I've found documentation for the basic stuff and I've found documentation for the knowledgeable user, I haven't found too much for in the middle.  A part of me feels like the answer to this question isn't even in ReactiveUI, but rather in Rx and Linq.  (For example, I haven't learned the magic of SelectMany)

Cheers!  Sorry to bother you with what I know is a solved problem.  Particularly when I KNOW there are massive holes in what I know, but I'm trying to learn.  :-)

Lastly, while I understand that ReactiveCollection is derived from ObservableCollection, and thus has the same problems that ObservableCollection has, what does deriving from ObservableCollection buy you?  Why not roll your own that does have thread safety?  Is it that there are extension methods on ObservableCollection that are being preserved?

~Scooty

[Paul Betts]

Hi Scooty,

> So I created a ThreadedReactiveCollection, which invokes changes to itself
> to the dispatcher.

This is fundamentally a Bad Idea™ though many people want it, and let me tell
you why.

Consider the following scenario, that you want to write your own version of
CreateDerivedCollection - you want to take the initial state + the
notifications and mirror the current state of the collection.

If you look at the definition of NotifyCollectionChangedEventArgs, it is
effectively a "delta" on the current state. Meaning, that to get the new
state, you must combine the old state + the delta. This implies though, that
you must receive these deltas **in order** so that you can reconsitute the new
state.

Running these change notifications through a scheduler fundamentally violates
that requirement - there is no guarantee that scheduler items will arrive in
the same order that they happen. This is a dealbreaker for any code that wants
to use these notifications.

> So I'm trying to figure out where I've gone wrong. Is it my design, my
> implementation, or is there a magic trick I don't know?

So, in general, the idea is that collections that you bind to in the UI (even
indirectly via CreateDerivedCollection) should only be changed on the UI
thread.

So, how can you solve this problem? So, say we have our network method to
fetch new Models:

IObservable<List<Model>> GetTheModels();

and our ReactiveCollection:

ReactiveCollection<Model> VisibleModels;

GetTheModels()
.ObserveOn(RxApp.DeferredScheduler)
.Subscribe(x = >{
using (VisibleModels.SuppressChangeNotifications()) {
VisibleModels.Clear();
VisibleModels.AddRange(x);
}
});

This can result in significantly better perf as well, as this strategy
generates the minimum amount of change notifications (=> the minimum number of
XAML layouts / loading new controls / etc).

> Lastly, while I understand that ReactiveCollection is derived from
> ObservableCollection

It's actually not anymore, it's fully reimplemented. While this was a ton of
work, it got us a lot of benefits around correctly issuing notifications in
certain scenarios.

--
Paul Betts <pa...@paulbetts.org>