unhosting web apps themselves

[Michiel de Jong]

ok, so the unhosted web architecture is about separation between application and data. So to then treat the application as data again, might be complicating things into a blur. Or... it might actually be a powerful idea.

The simplest version of doing this is to keep a bootloader on a domain name, to function as the URN of the application. That bootloader then pulls in the trusted source code from unhosted accounts of the application developers.

A next step would be to have a generic bootloader for unhosted web apps, and then run the web apps on there using google-caja.

The main reason to research this angle are software freedom (if software can move around, it will be a small step to allow users to apply patches to it), as well as taking away the necessity to register a domain name everytime you develop an app. i found this annoying myself, every time i write a quick app, i have to register a domain name, or point a subdomain somewhere, setup a vhost, etcetera. it would be nice to remove this barrier, especially if we think about web apps as remixes of smaller snippet-apps, so to speak.

it would also be nice to do an IDE, and automatically publish revisions/versions of the apps we develop. i think this could make app development easier, quicker, and more fun. that could be key to the success of the unhosted web architecture, because i still think that the key challenge will be having good apps that people can use with unhosted accounts.

cheers!

Michiel

[Ya Knygar]

I think - making a kind of
distributed GitHub(in terms of GH usability and fun)
is a great idea, Micheil!

>as well as taking away the necessity to register a domain name everytime you
>develop an app.

certainly, nowadays many people could just use a GH page or kind for
it,
but it - binds the community to a pretty closed social network,
so you have to choose BB/Gitor/etc. and once have chosen - live and
work in particular - to maintain the projects/favorites/mates/watching
lists,
messaging also. I'v seen many projects - trying to use two social
rep's at a time,
and haven't seen any success with it. Numerous of asks for, at least,
cross-watching feature.

I think - it's obvious that in next 5 years the bandwidth of average
Internet user would
be highly increased around the world - by that distributive hosting
could make it, into reality,
especially - given the webrtc.org kind of initiatives that are so fast
- growing from
http://www.readwriteweb.com/archives/his_could_be_big_decentralized_web_standard_under.php
promises.

>it would also be nice to do an IDE, and automatically publish
>revisions/versions of the apps we develop.

we could use https://mozillalabs.com/skywriter/ and http://ace.ajax.org/
projects achievements
and p2p https://secure.wikimedia.org/wikipedia/en/wiki/Distributed_revision_control
like http://www.monotone.ca/ ,

actually - I am a designer and PR manager of independent Wave social
server http://pyofwave.info/
from the part that are working on Open Augmented Reality initiative -
http://primarypad.com/rHfNNWzVng

we are in exactly the same need of unhosted hosting for web AR apps,
distributed collaboration on editing
and, basically - all what unhosted project is aimed for.

Also - we are, as PyOfWave, or other members of http://primarypad.com/OeMj2ZnZqo
pad (we are all - distributed systems, BTW)
will likely to make a p2p kind of wiki (by the way:)),
that likely to be based on some ultra-modern p2p database and nothing
prevents from including p2p version control, as i know.

PS: unhosted project and all - interested in decentralized social
systems:
help are highly needed and appreciated - on that working pads and in
repositories,
we are all for open-source systems, privacy and security both for data
and programs.

[Thad Guidry]

Hmm... this ideology that your referring to Tony begins to remind me less of unhosted ideas and more in line with Ian Clarke's - Swarm http://code.google.com/p/swarm-dpl/

But I dunno.  You tell me.

On Fri, Jul 1, 2011 at 12:08 PM, Tony Garnock-Jones <to...@ccs.neu.edu> wrote:

On 2011-07-01 12:46 PM, Tony Garnock-Jones wrote:

A tiny boot-loader program - from some well-known branch URL or blob
URL - would be given a URL list, and would retrieve, link, and run the
resulting code.

It'd be good if the list could contain sublists, too, so that libraries could specify their own internal dependencies.

It'd also be important to make blobs contain hints about what their human-readable mutable branch URL might be, so that given a blob URL you can retrieve the headers and follow the pointer to the latest version of the code, and all the metadata etc.

Finally, say I load library X, which depends on library Y at version 1, and then not knowing that X depends on Y, I introduce a dependency on Y at version 2. In situations like this the linker needs to be aware that there might be a clash and that it might need to step in. Lots of different strategies, from the simple-but-stupid to the complex-and-possibly-incomprehensible, are possible.

If a capability-secure language (caja?) were used it'd make it easier to load a copy of Y v1 for X and a copy of Y v2 for the main app, and to keep them separate.

Regards,
 Tony

--
-Thad
http://www.freebase.com/view/en/thad_guidry

[Tony Garnock-Jones]

On 2011-07-01 2:01 PM, Thad Guidry wrote:
> Hmm... this ideology that your referring to Tony begins to remind me
> less of unhosted ideas and more in line with Ian Clarke's - Swarm
> http://code.google.com/p/swarm-dpl/
> But I dunno. You tell me.

It doesn't seem too similar: IIUC, swarm is all about mobile running
code, whereas what I have in mind is orthogonal to that. I was thinking
more of how to stitch together program text from documents containing
source code that are distributed about the web. I was imagining using
unhosted for the source code snippets, like one might for other kinds of
data.

Regards,
Tony

[Tony Garnock-Jones]

On Jun 30, 8:56 am, Michiel de Jong <mich...@unhosted.org> wrote:
> ok, so the unhosted web architecture is about separation between application
> and data. So to then treat the application as data again, might be
> complicating things into a blur. Or... it might actually be a powerful idea.

Perhaps it's both :-) Metaprogramming requires careful attention to
phase distinctions, but is phenomenally powerful.

> The main reason to research this angle are software freedom (if software can
> move around, it will be a small step to allow users to apply patches to it),

This is a wonderful and important goal, very near to the heart of free
software. Making it not just technically possible, but *easy* for
users (or friendly developers they know) to make changes to running
software could have profound (hopefully positive) effects.

> a vhost, etcetera. it would be nice to remove this barrier, especially if we
> think about web apps as remixes of smaller snippet-apps, so to speak.

Exactly. A program these days tends to be a library of app-specific
code combined with a handful of generic open-source libraries
downloaded from somewhere else. It'd be nice to reify this arrangement
and unhost it.

Have you seen the site http://www.codecatalog.net/? It's an
interesting little library of code snippets for various languages. I
think the idea is that, by identifying the snippets within the source
code, they can be updated as needed when bug-fixes or improvements are
uploaded to the site. It's one approach to fluid semi-unhosted
application assembly.

Here's how I've been thinking:

- Git is great. It has an immutable content addressable store for
code, and mutable human-readable pointers into the content-addressed
space providing a naming indirection.
- What if we unhosted JS blobs, indexed by their hash?
- Well-known URLs http://example.org/mypackage would be like git
branches, resolving (perhaps via HTTP redirect) to URLs of the
underlying hash/content-addressed blobs,
http://example.org/blobs/5f1a7c5a62097d9623902cb174931d58a7af6c5b. At
the blob URL would be... the blob.
- A program, then, would be a list of URLs. Linking and loading would
be retrieval, string-append, and eval.

A tiny boot-loader program - from some well-known branch URL or blob
URL - would be given a URL list, and would retrieve, link, and run the
resulting code.

Pardon me if I'm using unhosted terminology and ideas wrongly :-) I've
only just joined the group and need to catch up on past discussions
and how people here are thinking of things. Hopefully this message
offers a few ideas people are interested in.

Regards,
Tony
--
Tony Garnock-Jones
tonygarn...@gmail.com
http://homepages.kcbbs.gen.nz/tonyg/

[Tony Garnock-Jones]

On 2011-07-01 12:46 PM, Tony Garnock-Jones wrote:

> A tiny boot-loader program - from some well-known branch URL or blob
> URL - would be given a URL list, and would retrieve, link, and run the
> resulting code.

It'd be good if the list could contain sublists, too, so that libraries

[Tony Garnock-Jones]

On Jun 30, 8:56 am, Michiel de Jong <mich...@unhosted.org> wrote:

> ok, so the unhosted web architecture is about separation between application
> and data. So to then treat the application as data again, might be
> complicating things into a blur. Or... it might actually be a powerful idea.

Perhaps it's both :-) Metaprogramming requires careful attention to
phase distinctions, but is phenomenally powerful.

> The main reason to research this angle are software freedom (if software can
> move around, it will be a small step to allow users to apply patches to it),

This is a wonderful and important goal, very near to the heart of free
software. Making it not just technically possible, but *easy* for
users (or friendly developers they know) to make changes to running
software could have profound (hopefully positive) effects.

> a vhost, etcetera. it would be nice to remove this barrier, especially if we
> think about web apps as remixes of smaller snippet-apps, so to speak.

Exactly. A program these days tends to be a library of app-specific
code combined with a handful of generic open-source libraries
downloaded from somewhere else. It'd be nice to reify this arrangement
and unhost it.

Have you seen the site http://www.codecatalog.net/? It's an
interesting little library of code snippets for various languages. I
think the idea is that, by identifying the snippets within the source
code, they can be updated as needed when bug-fixes or improvements are
uploaded to the site. It's one approach to fluid semi-unhosted
application assembly.

Here's how I've been thinking:

- Git is great. It has an immutable content addressable store for
code, and mutable human-readable pointers into the content-addressed
space providing a naming indirection.
- What if we unhosted JS blobs, indexed by their hash?
- Well-known URLs http://example.org/mypackage would be like git
branches, resolving (perhaps via HTTP redirect) to URLs of the
underlying hash/content-addressed blobs,
http://example.org/blobs/5f1a7c5a62097d9623902cb174931d58a7af6c5b. At
the blob URL would be... the blob.
- A program, then, would be a list of URLs. Linking and loading would
be retrieval, string-append, and eval.

A tiny boot-loader program - from some well-known branch URL or blob
URL - would be given a URL list, and would retrieve, link, and run the
resulting code.

[Siddhant Sanyam]

Yes, remember when we were discussing about how can we trust the app? If
they are not being changed by the provider? Well this might be the
answer. You can have application.com deploy the application in
yourunhostedhive.com . But now my question is that we have two ways of
doing it:
1. We never ever visit application.com again, we just go to
yourunhostedhive.com/username/application and run the application
2. We still go to application.com and it fetches data from
yourunhostedhive.com like it would do to the data.

The picture is still unclear. I guess we should be more specific as to
what has to be done.

[Thad Guidry]

On Thu, Jun 30, 2011 at 8:35 AM, Siddhant Sanyam <siddh...@gmail.com> wrote:

Yes, remember when we were discussing about how can we trust the app? If they are not being changed by the provider? Well this might be the answer. You can have application.com deploy the application in yourunhostedhive.com .  But now my question is that we have two ways of doing it:
1. We never ever visit application.com again, we just go to yourunhostedhive.com/username/application and run the application
2. We still go to application.com and it fetches data from yourunhostedhive.com like it would do to the data.

The picture is still unclear. I guess we should be more specific as to what has to be done.

 

As a user, I would want some notice or level of trust or affirmation that Scenario 2's application.com has not changed it's policies or the sharing of my data.

I have not heard the mention of a sandboxed environment (outside of a javascript engine itself of course).  The software I use in my browser is still rendered via a JS engine.

--
-Thad
http://www.freebase.com/view/en/thad_guidry

[Siddhant Sanyam]

But then why make different applications.coms? We can simply have appsapp.com to have all the applications hosted and you can just "Install to my Unhosted Account" And then let your unhostedhive.com give you the code.

I'll try to explain why it is not a good idea to treat applications as data and to store them in your unhosted account (I'll call this flower-in-hive).
1. It breaks the concept of unhosted.  Unhosted's main principle is have application and data in separate sever and completely decoupled. Now you can argue that with the flower-in-hive scheme, the application is still decoupled with the data, but actually it is getting served from the same domain.
2. It facilitates redundancy. This is the most important part. With this scheme, I'm duplicating the application code on several Unhosted accounts. For a popular app, this will mean that each  Unhosted user will have its own redundant copy of the same data. This is very ugly. The beauty of unhosted app was that it minimized code duplication by separating the application from the data. Data is what is user specific, not the application. Hence it makes less sense to have individual copy of the same copy.
3. It facilitates centralization. As I said, then it becomes very motivating to have one domain for all the apps (like the app store). This gives the idea of centralization rather than distributiveness.

[Tony Garnock-Jones]

On 2011-06-30 9:35 AM, Siddhant Sanyam wrote:
> Yes, remember when we were discussing about how can we trust the app? If
> they are not being changed by the provider? Well this might be the
> answer.

Yep. Specifying the code you want to run by naming it by its
cryptographically-strong hash has the nice property of knowing for
absolute sure that you're running exactly the code you wanted :-)

Tony

[Tony Garnock-Jones]

On 2011-07-01 6:50 PM, Siddhant Sanyam wrote:
> For a popular app, this will mean that each Unhosted user will
> have its own redundant copy of the same data.

If you name content by its hash, then as long as the server *knows*
you're doing this, it can be doing totally transparent deduplication and
nobody need ever know. Furthermore, hashes being what they are, they can
be used as capabilities to the data. (And of course the blobs themselves
can be hosted on any machine, since you have the means of checking you
get the right thing back.)

Tony

[Ya Knygar]

i understand that talk is essentially about particular scheme of
unhosted hosting but i just need to add 2c here ..

>For a popular app, this will mean that each Unhosted user

>will have its own redundant copy of the same data. This is very ugly.
>The beauty of unhosted app was that it minimized code duplication by
>separating the application from the data. Data is what is user specific,
>not the application. Hence it makes less sense to have individual copy
>of the same copy.

since this talk is about decentralized apps i'll free myself and post
about apps rather then particular unhosted scheme's
i'll double one X3C(that Pad lists) member- "it is the popular content
that is the second-class citizen when it comes time to make room in a
full block cache. After all, the most popular blocks are the mostly
likely to be available elsewhere, and therefore, the safest to
delete." from - http://thisisimprobable.tumblr.com/post/4977027269/the-freenet-effect-vs-viral-survival

that article is about p2p decentralized network named ConcurrentTree
(the link is on pad)

so, further :
in case of old non-p2p Wave we had App Engine storage, and only App
Engine, it was completely ok for basic non-websockets apps as Google
provides free storage for app until popularity exceeds some limit, and
if it get enormous popularity you could mirror it or ask for donations
for hosting, kind of.
But:
1. it's proprietary cloud
2. that's enough ;)

Other variants like home hosting doesn't guarantee the conditions for
the survival of the written apps, according to article and what i'v
seen.
You can imagine how many apps got ditched after Wave closed, they kind
of make it somewhere with some google team regrets i don't want to
talk here about, but in overall we got the system where "write once
run everywhere" and "not invented here" isn't working not because of
languages or platform (at least). But because of nobody knows of that
little poor apps, for that we have GitHub and kinds, now - but "1.
it's proprietary cloud"

so - only other variant we see like realistic now is - some p2p
storage scheme for the apps themselves
and "It facilitates centralization. As I said, then it becomes very

motivating to have one domain for all the apps (like the app store).
This gives the idea of centralization rather than distributiveness. "

goes here but in the way of having list of apps (if wouldn't leverage
push tech in apps so they, finally could get searched)

Pushing is a very beautiful idea comparing other variants and it
could, certainly help the problem of redundancy and centralization.

We'll adapt it anyway in most standardized way we could (probably XMPP
pubsub or hubbub, oh,well) -- for Augmented Reality servers,
because of AR apps nature,
but i think - same benefits of pushing and (probably - or, but see
again that 1.) p2p distributing is what web apps missing now and is -
why we'll get centralized markets and clouds for awhile, even for web
apps,
if won't bring on market something really clever and still - working.
i think - apps.mozillalabs.com is a move in direction of freedom but
without p2p, pushing.. unhosting it still will just compete with dozen
of other % driven markets, and the worst in that - these markets will,
probably be - that proprietary clouds of code and data.


Moreover - if we won't get help with unbound, reusable apps strategy,
some will probably introduce a really ugly schemes of apps that stuck
into social platform, best if - like FB or old Wave apps - with an
ugly iframes.

[Michiel de Jong]

This conversation is super interesting, i think. Thank you Tony and others for coming over to the mailing list to help to discuss this.

What if the snippet identifier contains not only the hash but also the user address of its developer? Of course, it would make many web apps fail if the author of, say, jQuery closes his account, because that snippet is used everywhere. But copying into everybody else's account as soon as they use it might also not be the solution, because it creates unnecessary duplication. Deduplication on the server-side would put big storage providers ahead of small ones, so it would be nice if we could find something better.

Apart from that, i think we should never forget to aim for the case where an end-user visits a normal website, and signs up the normal way. the 'zero user experience' case, where the end-user does not notice (apart from a label-of-approval maybe) whether they are using a hosted or unhosted account on a website. discovery should still be possible in the usual ways - clicking a link, typing a url, searching in a search engine, from a portal, from a bookmark, from an app manifest. Whatever user experience people have on web2.0, should be what we aim to give them as a minimum (although not necessarily a maximum) on the unhosted web we build.

and we should make sure it's easy to convert a standard website to an unhosted one - not create an entirely separate closed ecosystem, but rather diffuse small changes into the web that already exists.

I'm not so afraid that appsapp.org would become a centralized thing, because it would be entirely replaceable. we would just define an 'apps' scope or a 'snippets' scope, and different websites would be viewers for this scope.

I think a first step might be to put the MyFavouriteSandwich app into a google-caja sandbox, maybe putting the end-to-end encryption outside the caja. I'm still a bit concerned that caja-cajoling has to be done server-side, but i'm sure there's a solution for that.

So then second step would be to make what is inside the caja sandbox editable, and load it as modular snippets, and check the hashes of what you're loading. To the user, this would be like a 'Fork me' button on https://myfavouritesandwich.org/ If you click it, obviously to save your changes the URL has to change. or maybe you log in to the bootloader of https;//myfavouritesandwich.org/, and the sandbox content (the part you can edit) is what you see when you're already logged in. This would have the downside that you can't edit the login screen, but maybe that's OK. Login screens are one of the web's architecture bugs anyway.

Third step would be to cache/survive/deduplicate snippets, so you don't bring down 56 different web apps if the author of one snippet doesn't pay their hosting. But i'm hesitant to bring this into the architecture in a fundamental way and/or at this early stage, because it's so easy to get it wrong. Maybe it's as easy as specifying various alternative sources in the pointer. and have the bootstrap randomly select a source, so that traffic gets load balanced, and we don't all load the jQuery library from John Resig's personal unhosted storage account.

What do people think of putting the duplication/deduplication on a higher layer that way, instead of making it a feature of a lower layer? It's a bit like tcp/ip i (would hope to) think, where addressing is done at a low layer, and data integrity on the higher one.

Cheers

Michiel