Updates since 2010?

[Nick Myers]

This project seems ideal for many webapps!  Why hasn't this been touched since 2010?  I would love to get involved.

[Joshua Harrison]

Honestly, there just didn't seem to be much interest. If you are interested in giving it a shot, I'd be more than willing to resurrect my involvement.

I will move the project over to github.com and update all of the documentation. Maybe it will drum up some more interest over there. How'd you find it?

[Nick Myers]

I'm working on a project that involves several million students, many of which are at schools with very slow internet connections.  They all log into the site simultaneous in the morning, and the average load time of the site can be around 5 minutes after we deploy new code.  So I was researching a way to do patches, instead of re-downloading the entire codebase.  We use the closure compiler to put the entire codebase into a single JavaScript file, and it seemed to me that there should be a way to do simple patches to that file.

It seems to me that your project is something that pretty much every large webapp could benefit from, but your project seems to be literally the only attempt at the concept.  

I haven't dug into your project yet, but what I'm envisioning is including a single JavaScript library in index.html.  It will do the following:

    *Check local storage for a particular specified file

    *Check the version number of that file 

    *Communicate to the server what the version of that file is

    *The server will have to do a little bit of work here.  It checks the user's version number, does a diff of that version with the current version, creates a "patch" file of the differences, and sends the patch file to the user

    *Back on the client side, the JavaScript library takes that patch file and creates a new file in local storage by inserting the differences into the old version

All of a sudden that 5 minute load time is cut down to just a few seconds.  It would be awesome if 100% of the logic could be in the JavaScript library, so that it just works out of the box without any server-side logic.  But I haven't figured out how that would be possible yet.  Imagine just including a <script> tag, and then your app's download time is slashed by orders of magnitude without any custom code whatsoever.  I feel that if we come up with a solution that is easy to use like this, it would really take off.  And frankly, I'm surprised it doesn't exist yet!

--
You received this message because you are subscribed to a topic in the Google Groups "Diffable" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/diffable/xN1E3cWr17w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to diffable+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Nick Myers

[Joshua Harrison]

Wow - I would love to help get diffable deployed in this type of environment and see how it performs!

The steps that you describe are exactly what the project does, and we have been experimenting with it, off and on, inside Google since I originally built it. However, we were mostly interested in trying to use it to lower the page load time of desktop sites from 10s of seconds to single digit seconds. We had not played with it in a very low bandwidth setting like the one you are describing. The main issues we have run into during our various experiments are local storage implementations and the added latency of the boot sequence.

In many cases where users have poor connections, they also seem to have poor local storage performance. Reading from local storage was adding significant time to the boot sequence, and in a significant portion of the cases, otherwise fast users (had they been using the network only) were slowed down by this initial access of local storage. However, given the scenario you are describing, I don't think this would be an issue given how long it takes to download over the network. The second major problem was the case when there was no version in storage, or, for whatever reason, the version in storage was no longer valid or patchable. You can imagine that, in production, you could end up with exponential patches as each new version requires a patch from every older version and so putting a limit on how old the library can be was a way to manage cost. But in these cases, having to first read from local storage, and then access the network for the full download, was a big latency hit compared to them just having downloaded the resource directly. Again, this seems like a problem you would not have given how long the download takes.

In summation, I think what you are describing would be a perfect candidate for why I initially created this. As I mentioned, the version in source control at the moment does everything you describe and is absolutely functional, though it's been about 4 years since I've worked with it. I would need to familiarize myself with the code again. However, I would love to do so if it were going to be receiving this kind of trial by fire! Out of curiosity, what environment are you using on the server side?

Per your client-only concept, I don't think this would be possible given that the server would always have to be the one generating patches. One thing I could imagine is that all patches are only between successive versions, i.e. v1 -> v2 -> v3 -> v4. If someone loaded the base page and saw the v4 was the current and v2 was in local storage, they would first have to download and patch v2 -> v3 and then v3 -> v4. This would mean all diffs could be generated at build time, on the server, and pretty much all logic from that point forward could reside in the client. But given the network conditions you describe, I imagine adding these extra round-trips could be quite costly.

Thoughts?

And thanks again for reaching out! It's so cool to think that it could be used to help so many people, and I would love to help in any way I can!

Joshua

You received this message because you are subscribed to the Google Groups "Diffable" group.
To unsubscribe from this group and stop receiving emails from it, send an email to diffable+u...@googlegroups.com.

[Nick Myers]

Do you have a way that I can get in touch with you privately?  My gmail account is N I C K A M 9 9 2