pureComputed and automatic promise unwrapping

[Daniel Earwicker]

Hello, I am a Knockout addict!

One thing I do a lot of in my code is based on this pattern:

https://github.com/knockout/knockout/wiki/Asynchronous-Dependent-Observables

In fact that's my blog linked to at the bottom of the page (thanks!)

But I'm wondering if there is any more recent advice for how to implement this correctly, in light of the introduction of pureComputed.

I'm hoping that switching to pureComputed will alleviate some of our problems with memory retention. As I understand it, a pureComputed will "go to sleep" when not wired into the DOM, but can "wake up" again when rewired. In the sleeping state, it is more apt to be GC-ed because it is not kept alive by subscriptions to the data it depends on. It sounds ideal.

The problem is that not all of the (approx 1700) computeds in our code base are pure (THE SHAME OF IT!)

One major case is that we use the "async" extender, which (as described in the above linked page) internally uses a impure computed.

The issue here is not really that it's impure, but that it creates a dangling/orphan ko.computed which becomes dependent on external data, and cannot transition to a "sleeping" state like a pureComputed can. So its lifetime is that of the data it depends on. Meaning that if it transitively were to become dependent on something that lives forever, then it too will live forever, unless manually disposed. Which it never is, because it's an orphan created inside some helper code.

So I guess to solve this what I need to do is make a computed that goes to sleep when another observable has no subscribers.

Is that possible? Or is there a new better way to do this kind of thing?

Thanks

--Daniel

[Michael Best]

This is an interesting question. I think there is a solution using ko.pureComputed and maybe some of the new features coming out in 3.3. When I have a little time, I'll see if I can put something together.

-- Michael

[Daniel Earwicker]

I see what you mean, those new events seem ideal. Here's what I have so far:

https://github.com/danielearwicker/knockout-promiseComputed

Internally I still use a plain ko.computed but ensure that it is disposed and you can see that happening in the debug console logging.

[Daniel Earwicker]

I've updated it to include a simple stress test that can be started from the debug console, and also a way to switch between the old and new implementations of promiseComputed.

NB. I'm favouring a separate ko.promiseComputed API rather than using `extend` because I am almost as crazy-in-love for TypeScript as I am for KO, and unfortunately extend makes it practically impossible to write a helpful type description.

[Daniel Earwicker]

Another way of phrasing the same thoughts, in a more general form:

In practice there are sometimes uses for standalone or "orphan" computeds, where they return nothing but have side-effects:

ko.computed(function() {

    var v = someOtherObservable();

    // have some side-effect using v

});

In any view model code that is setup and torn down repeatedly, the above example is woefully incomplete because it doesn't get disposed.

It would be ideal if such an orphan could be associated with an "owner", which would be an observable supporting awake/asleep events. This would take care of building/disposing the actual computed based on the wakefulness of the owner:

var xo = {}; // for knockout extensions

xo.execute = function(owner, evaluator, thisObj) {

    var orphan, extenders = [];

    owner.subscribe(function() {

        if (!orphan) {

            orphan = ko.computed(evaluator, thisObj);

            extenders.forEach(function(e) {

                orphan.extend(e);

            });

        }

    }, null, "awake");

    owner.subscribe(function() {

        if (orphan) {

            orphan.dispose();

            orphan = null;

        }

    }, null, "asleep");

    return {

        extend: function(extender) {

            if (orphan) {

                orphan.extend(extender);

            }

            extenders.push(extender);

        }

    };

};

(NB. not sure about the extenders part but I guess something like that is needed).

The idea is that the side-effects (whatever they are) would only be needed to occur when the owner is awake. When the owner is asleep, the orphan computed is internally torn down.

The only caveat is that ordinary observables don't emit asleep/awake events, so only pureComputeds can serve as owners for others.

It would be great if asleep/awake events could be integrated into plain observables, but obviously that adds complexity to the most heavily used feature of KO so may not be desirable. If so, it can be defined by another external wrapper:

xo.observable = function(initVal) {

    var observable = ko.observable(initVal);

    var changeSubscriptionCount = 0;

    

    observable.beforeSubscriptionAdd = function (event) {

        if (event === 'change') {

            if (changeSubscriptionCount++ === 0) {

                observable.notifySubscribers(observable.peek(), "awake");

            }

        }

    };

    observable.afterSubscriptionRemove = function (event) {

        if (event === 'change') {

            if (--changeSubscriptionCount === 0) {

                observable.notifySubscribers(observable.peek(), "asleep");

            }

        }

    };

    return observable;

};

Out of these we get a "leak-proof" pattern for using KO:

• Use xo.execute instead of standalone or "orphaned" ko.computed
  
• Use xo.observable instead of ko.observable if you want it to serve as owner of an xo.execute
  
• Everywhere else, use ko,pureComputed. Never use a raw ko.computed
  
• The promiseComputed I was originally asking about can be implemented easily on top of xo.execute/xo.observable. It is very nearly the original code 

This way, you are safe from memory retention or weird side-effects that keep happening after their associated view-model has been abandoned, and you don't have to manually dispose anything.

Does this make sense?

Also, is there a way to ask a Knockout subscribable what events it can emit? That would allow xo.execute to valid the owner passed to it, rather than subscribing to awake and never hearing anything. I wondered about the possibility of making subscribables throw an exception if you try to subscribe to an event they don't ever emit, but of course that would be a breaking change.

[Michael Best]

Daniel,

I like what you've done. I think this is pretty much what I was envisioning.

-- Michael

[Michael Best]

One caution for making such a general-purpose utility is, as you mentioned, that not all observables emit awake/asleep events. Also you assume that the pure computed is currently asleep, but that might not be the case. There's not currently an API for determining either of these conditions, although the latter can be inferred using getSubscriptionsCount('change').

-- Michael

[Daniel Earwicker]

Agreed. I'm thinking something simpler and adhering to standard KO would be much more appropriate. My "rules" are:

• Never, ever use ko.computed. Period.
  
• Where you want to use a void-returning ko.computed (to get side-effects) instead just use pureComputed and "publish" it in your view model and use an "execute" binding (see below) to keep it awake.
  
• Where you can't use the execute binding, use ko.execute (see below)
  

The execute binding is trivial:

    ko.bindingHandlers.execute = {

        init: function() {

            return { 'controlsDescendantBindings': true };

        },

        update: function (element, valueAccessor) {

            ko.toJS(valueAccessor());

        }

    };

    ko.virtualElements.allowedBindings.execute = true;

Its only purpose is to keep pure computeds awake, e.g.:

    <!-- ko execute: mySideEffects --><!-- /ko -->' +

It uses ko.toJS to observe everything you pass it, whatever the structure, so you can give it an array, etc.

And ko.execute now insists that its "owner" is a ko.pureComputed, so it's harder to use it incorrectly. The way it checks this is delightfully fragrant, I'm sure you'll agree! :)

    ko.isPureComputed = function(obs) {

        // There are hacks, and then there are hacks

        return !!(obs && obs.beforeSubscriptionAdd &&

            obs.beforeSubscriptionAdd.toString().match(/"awake"/));

    }

It looks as if ko.execute also needs this to make sure it starts awake if necessary:

    if (pureComputed.isActive()) {

        wake();

    }

Finally, ko.promiseComputed serves as a exemplar of ko.execute usage: it's a utility that returns data, so what it returns is a pureComputed. Hence it can use that as the owner for ko.execute.

By following these rules (which are easy to check for, because you just search your app code for ko.computed!) and using well-behaved bindings, memory leaks ought to be a thing of the past.

I've updated the demo at https://github.com/danielearwicker/knockout-promiseComputed with these pieces.

Thanks for your invaluable help with this!

--Daniel

[Michael Best]

Daniel,

Great advice. Perhaps we can incorporate some of these ideas into future versions of Knockout.

Your isPureComputed function will only work when using the Knockout debug build, since in the minified build, beforeSubscriptionAdd has a minified name. Another option for this is to wrap the ko.pureComputed function to add a special property to the returned object.

Also isActive() doesn't tell you whether the computed is sleeping or not. You can use getSubscriptionsCount('change') to determine whether it's awake.

-- Michael

[Daniel Earwicker]

Awesome. With those changes, I think I'm ready start grepping for ko.computed in my code! Will write it up soon also.

[Daniel Earwicker]

Small library with plentiful documentation: https://github.com/danielearwicker/knockout.clear/

(Have made good progress on my 100K lines of JS/TS/KO.)

[Daniel Earwicker]

Follow-on question. I just noticed that ko.execute was STILL dependent on the debug version of KO - D'oh. So I've fixed that.

But in any case, it looks like the thing I wanted to do isn't going to be possible, so this is turning into a feature request. :)

There's a rule that has to be followed, where ko.execute's first argument must not be depended upon by its second. Otherwise, it sets up a dependency that keeps it awake indefinitely and effectively turns it back into a ko.computed from a clean-up perspective.

The problem is, this rule is easy to break by accident, because the chain of dependencies could be any length.

Is there a non-intrusive way to validate something like that so ko.execute can throw an error if the rule is broken?

[Michael Best]

Daniel,

First, your plugin looks really nice. Your feature request is certainly valid, and you can open an issue for it on Github. A sort-of workaround is to check getSubscriptionCount('change') on the pureComputed before and after calling the evaluator function, verifying that the value hasn't changed. This isn't fool-proof since, by definition, ko.execute is used to run code with side effects, and one of those side effects could result in a subscription to the pureComputed.

-- Michael

[Michael Best]

I've read through your blog post at https://smellegantcode.wordpress.com/2015/02/21/knockout-clear-fully-automatic-cleanup-in-knockoutjs-3-3/ and found it quite informative and interesting. I did find one mistake that you might like to fix--the code for ko.execute doesn't close the wake function at the right time.

-- Michael

[Daniel Earwicker]

Thank you! Fixed. That'll teach me not to change my code in the wordpress editor...