[Spec Proposal] OpenSocial Templates

[Evan Gilbert]

I've incorporated feedback from a number of discussions and now am looking for +1s on the high-level design for OpenSocial templates.

Spec proposal:  http://groups.google.com/group/opensocial-and-gadgets-spec/web/opensocial-templates
Single document version:  http://docs.google.com/Doc?id=dg6csr2s_3gttjwjzz

The definition of <os:> tags is not covered yet - we will cover each tag as a separate spec enhancement, and I expect there to be a fair number clarifications and minor revisions to the proposal over the course of implementation.

Thanks for all the feedback, and looking forward to getting templates up and running!

Evan

[Kevin Brown]

My biggest concern is that it seems that there won't be any practical way to do template processing server-side, and you'll be forced to do the processing in js, which will lead to performance problems.

I'd like to do a sanity check to make sure that templates can be efficiently processed server-side as well (it would also be nice if you could have a first-class content type; <Content type="template">). Specifying a MIME type (text/os-template, though I believe that should probably be x-os-template) so that <Preload> and makeRequest could make use of the results would complete the picture.

[Ian Boston]

One observation on reading the template spec, it looks quite simple,
with variables, loops and conditionals and a xml friendly way of
expressing them, so it would be possible to convert them into a
template that could be processed efficiently on the server or the
client. So any of the off the shelf template libraries that have the
necessary concepts could be used for server site.

The thing that worries me about where the template is rendered, is not
so much the template or its semantics, but the model that populates
the variables driving the templates.

However, that being said, if the model boundaries are constrained so
that it can be created and/or exists on either client or server then
processing can take place in either place. In the work we have been
doing elsewhere we have been using client side templates using
trimpath (on google code, not xml friendly), which has a format close
to smarty and velocity, and so could be processed with relative ease
on the server. We have not found the need to process on the server...
mainly because trimpath compiles the template on the client using
string manipulation rather than dom manipulation and so is lightning
fast. It uses eval on the client, but this compilation could be
performed on the server with a dom-less js implementation, so that a
template library could be compiled once on the server and delivered
for rendering in the client... building on the first-class content
type Kevin suggested (if I read that part right).

I can think of a number of ways of delivering client and/or server
side rendering of templates that dont have the performance penalty of
slow operations on the client or the server..... but its very possible
I have missed a real blocker in my reading of the template spec
proposed.

Ian

2008/6/10 Kevin Brown <et...@google.com>:

[Kevin Brown]

On Tue, Jun 10, 2008 at 1:46 AM, Ian Boston <ianb...@googlemail.com> wrote:

One observation on reading the template spec, it looks quite simple,
with variables, loops and conditionals and a xml friendly way of
expressing them, so it would be possible to convert them into a
template that could be processed efficiently on the server or the
client. So any of the off the shelf template libraries that have the
necessary concepts could be used for server site.

The thing that worries me about where the template is rendered, is not
so much the template or its semantics, but the model that populates
the variables driving the templates.

Yeah, that's actually my concern. Populating the variables server-side isn't quite so straight forward because it's not clear that you can make assumptions about the values of the variables during processing. Once you have the variables, actual processing isn't bad at all. Client side, the data is clearly available at all times (albeit with the potential overhead of additional, blocking opensocial.* js calls).

However, that being said, if the model boundaries are constrained so
that it can be created and/or exists on either client or server then
processing can take place in either place. In the work we have been
doing elsewhere we have been using client side templates using
trimpath (on google code, not xml friendly), which has a format close
to smarty and velocity, and so could be processed with relative ease
on the server. We have not found the need to process on the server...
mainly because trimpath compiles the template on the client using
string manipulation rather than dom manipulation and so is lightning
fast. It uses eval on the client, but this compilation could be
performed on the server with a dom-less js implementation, so that a
template library could be compiled once on the server and delivered
for rendering in the client... building on the first-class content
type Kevin suggested (if I read that part right).

That's the idea. It would be really great if this was a valid gadget:

<Module>
  <Content view="canvas" type="template" href="canvas.tpl"/>
  <Content view="profile" type="template" href="profile.tpl"/>
  <Content view="home" type="template" href="home.tpl"/>
</Module>

The primary advantage of processing server side is that what gets sent to the client would require no additional transformation, and would not need to have the template library (which could easily grow to tens of KB or more) downloaded if there is no dynamic template use.
 

[Ian Boston]

2008/6/10 Kevin Brown <et...@google.com>:

So what gets delivered to the client is a single compiled template
library for that gadget as a block of text/javascript and the end of a
URL.

In the client templates get invoked as JS calls, though a wrapper
class to isolate the Gadget writer from the compiled template.

On the server the compiled template library is cached with all the
other artifacts associated with the gadget, only recompiled when the
gadget is update or the cache expires, but the same library served to
all used of the gadget.

In terms of processing, the server is compiling rarely and mainly
serving bytes. And the client is processing native code binding to the
open social apis and not interpreting.

Is that what you were thinking of ?

Ian

[Evan Gilbert]

Fast client-side processing should be possible with the proposal as it stands. Per Ian's note, as long as we avoid DOM manipulation and can precompile the templates, the display can be as fast as we want it to be. <Preload> adds the missing piece.

I think we're going to end up with <Content view="canvas" type="template" href="canvas.tpl"/>, but I don't think we should lock this down yet - there are too many cases where developers need to re-render templates on the client, and I'd like to see usage patterns first. We likely will need a syntax for declaring which external template libraries are used - this should probably go into the <Require> tag as a param.

Regarding server-side processing - there are a few things we'd need to add to make this work:
- Declarative query syntax - you'd need to be able to specify the parameters to a call to newFetchPeopleRequest() declaratively. Chris at MySpace has some good ideas in this area
- Reserved variable namespace for variables that are commonly available, i.e. ${OpenSocial.Viewer}, ${OpenSocial.CurView}, etc.
- <Content view="canvas" type="template" href="canvas.tpl"/> seems a likely syntax for this,
- Anything else?

One note - we're anticipating many additions before we lock down the 1.0 spec, so I'd like to encourage people to focus on discussing the parts of this proposal that are incompatible with future needs. I'm hoping to get agreement on the overall shape of templates so that people can start working while we finalize the details.

Evan

[Kevin Brown]

On Tue, Jun 10, 2008 at 7:52 AM, Evan Gilbert <uid...@google.com> wrote:

Fast client-side processing should be possible with the proposal as it stands. Per Ian's note, as long as we avoid DOM manipulation and can precompile the templates, the display can be as fast as we want it to be. <Preload> adds the missing piece.

I think we're going to end up with <Content view="canvas" type="template" href="canvas.tpl"/>, but I don't think we should lock this down yet - there are too many cases where developers need to re-render templates on the client, and I'd like to see usage patterns first. We likely will need a syntax for declaring which external template libraries are used - this should probably go into the <Require> tag as a param.

Regarding server-side processing - there are a few things we'd need to add to make this work:
- Declarative query syntax - you'd need to be able to specify the parameters to a call to newFetchPeopleRequest() declaratively. Chris at MySpace has some good ideas in this area
- Reserved variable namespace for variables that are commonly available, i.e. ${OpenSocial.Viewer}, ${OpenSocial.CurView}, etc.

The current spec seems to use $Top for this. Would $OpenSocial, or maybe $os be better for that?

[Graham Spencer]

I agree with Evan -- in the interest of making progress, let's identify the parts of the spec that *preclude* server-side processing and make sure we flag the things where we'll need to add detail (either now or later).

Kevin, is your concern addressed by clarifying which variables the server would need to know in order to process the template?

--g

On Tue, Jun 10, 2008 at 7:52 AM, Evan Gilbert <uid...@google.com> wrote:

[Kevin Brown]

On Tue, Jun 10, 2008 at 4:14 PM, Graham Spencer <g...@google.com> wrote:

I agree with Evan -- in the interest of making progress, let's identify the parts of the spec that *preclude* server-side processing and make sure we flag the things where we'll need to add detail (either now or later).

Kevin, is your concern addressed by clarifying which variables the server would need to know in order to process the template?

I think so. If the variables are in context on the server then it's really no different from client side processing.
 

[Ian Boston]

Please tell me if I am being dumb or disruptive here..... (and I will
listen and learn)

If the aim is to compile the template on the server so that no
template processing libraries are needed on the client, then variables
that are resolvable and deterministic on the server offer an
opportunity for optimization of the compiled template; but those
variables that are only resolvable on the client or non deterministic
over multiple requests can remain as variables in the compiled
template that is delivered to the client.

Hence the lack of a variable on the server is unlikely to prevent the
compilation of the template, although it could conceivably result in a
compiled template being less efficient.

As long as not expensive or bulky template library is present on the
client, then its all is Ok. I am naively thinking a compiled template
is a javascript function suitably packaged.

Ian

2008/6/11 Kevin Brown <et...@google.com>:

[Kevin Brown]

On Tue, Jun 10, 2008 at 4:58 PM, Ian Boston <ianb...@googlemail.com> wrote:

Please tell me if I am being dumb or disruptive here..... (and I will
listen and learn)

If the aim is to compile the template on the server so that no
template processing libraries are needed on the client, then variables
that are resolvable and deterministic on the server offer an
opportunity for optimization of the compiled template; but those
variables that are only resolvable on the client or non deterministic
over multiple requests can remain as variables in the compiled
template that is delivered to the client.

Hence the lack of a variable on the server is unlikely to prevent the
compilation of the template, although it could conceivably result in a
compiled template being less efficient.

As long as not expensive or bulky template library is present on the
client, then its all is Ok. I am naively thinking a compiled template
is a javascript function suitably packaged.

Probably -- the ideal solution would be for the server to process as much as it could, leaving placeholders for the client if it can't.

If you do any processing on the client, though, you may as well do all of it (at least for the initial request). If the server can determine that there is dynamic content and defer template processing to the client, that would work out just fine.

[Evan Gilbert]

Ian & Kevin answered this well - you can evaluate what's possible on the server and send the rest down to the client.

Also, if the server doesn't support templates or if you are in active dev and want to iterate quickly, you can render only client side - server is mostly for performance and security.

Evan

[Kevin Brown]

On Wed, Jun 11, 2008 at 11:26 PM, Evan Gilbert <uid...@google.com> wrote:

Ian & Kevin answered this well - you can evaluate what's possible on the server and send the rest down to the client.

Also, if the server doesn't support templates or if you are in active dev and want to iterate quickly, you can render only client side - server is mostly for performance and security.

Of course -- supporting client side will probably be a requirement anyway in the long term,  if we want some sane way to allow innerHTML alternatives to be used with caja. I'm fine with initial support being client side only, I just want to be sure that server side is an option for the sake of performance.

[Evan Gilbert]

Hi everyone - I believe we resolved the open issues on the previous thread... all in favor, please +1.

Spec proposal:  http://groups.google.com/group/opensocial-and-gadgets-spec/web/opensocial-templates
Single document version:  http://docs.google.com/Doc?id=dg6csr2s_3gttjwjzz

Evan

[Chris Bissell]

+1

[Mike Austin]

+1 for high-level design

 

We’re very interested in is the <os:Message> tag, which makes it a whole lot easier to write localized apps.  Of course, you can simply pass the current language to your server and handle the message bundles on the backend, but for beginning OpenSocial developers, it provides a standard way of localizing.

 

Evan, I forget if there is a way to specify a div that your template will process in, similar to Ajax.Updater?  For example, opensocial.template.render(‘my-div’, ‘my-template’, myData).

 

We also mentioned the ability to implicitly use tags in attributes with some sort of “set” tag, for example:

 

<os:Set var=”tooltip” value=”<os:Message key=”background-color-help”/>

<img title=”${tooltip}” alt=”${tooltip}”/>

 

And compared to all the injected HTML that is included in an open social app, I don’t think rendering time is much of a challenge, unless you have one huge template.  You can break them up and display them as your data comes in, and you update certain sections of your pages like Ajax driven sites.  This is possible, correct?

 

Mike Austin

hi5.com

---

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Evan Gilbert
Sent: Monday, June 09, 2008 6:04 PM
To: opensocial-an...@googlegroups.com
Subject: [Spec Proposal] OpenSocial Templates

 

I've incorporated feedback from a number of discussions and now am looking for +1s on the high-level design for OpenSocial templates.

[Vivian (李卓茹)]

+1.

--
Meet OpenSocial developers
irc://irc.freenode.net/opensocial

[Ian Boston]

+1
Ian

[Krishna Sankar (ksankar)]

+1

 

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Evan Gilbert
Sent: Wednesday, June 18, 2008 1:17 PM
To: opensocial-an...@googlegroups.com
Subject: [Spec Proposal] OpenSocial Templates

 

Hi everyone - I believe we resolved the open issues on the previous thread... all in favor, please +1.

[Graham Spencer]

+1

--g

[Evan Gilbert]

On Wed, Jun 18, 2008 at 2:58 PM, Mike Austin <mau...@hi5.mi8.com> wrote:

+1 for high-level design

 

We're very interested in is the <os:Message> tag, which makes it a whole lot easier to write localized apps.  Of course, you can simply pass the current language to your server and handle the message bundles on the backend, but for beginning OpenSocial developers, it provides a standard way of localizing.

 

Evan, I forget if there is a way to specify a div that your template will process in, similar to Ajax.Updater?  For example, opensocial.template.render('my-div', 'my-template', myData).

We definitely need to support this functionality, the specific API signature isn't specified. There are a number of these JS API details that I hope can be fleshed out and proposed back into the spec as we work through the implementation.

 

We also mentioned the ability to implicitly use tags in attributes with some sort of "set" tag, for example:

 

<os:Set var="tooltip" value="<os:Message key="background-color-help"/>

<img title="${tooltip}" alt="${tooltip}"/>

Definetely want to support a <Set> tag, not positive we need it in first iteration.

The idea I was thinking for localizable attributes was to use on of  the following mechanisms, hidden under nicer API:

Load using a message ID:

<img oncreate="this.title=opensocial.template.renderMessage('background-color-help');/>,

Or if you need default text, Load from contents of another element:

<img oncreate="this.title = document.getElementById('img-text').innerHTML"/>
<div style="display-none" id="img-text"><os:Message key="background-color-help">Hello</os:Message></div>

For common use cases, we can hide these under OpenSocial tags:

<os:Image src="...">
  <os:Title msg="background-color-help">Some default text</os:Title>
</os:Image>

Is this sufficient to handle the messaging use cases?

 

And compared to all the injected HTML that is included in an open social app, I don't think rendering time is much of a challenge, unless you have one huge template.  You can break them up and display them as your data comes in, and you update certain sections of your pages like Ajax driven sites.  This is possible, correct?

Yes, app developer has flexibility to do this. Depending on how the data comes in, it may make sense to wait for all the data and render once, or if there are straggler requests to render in stages.

The rendering time should be good in JS but needs to be watched - creating HTML using JavaScript can be shockingly slow
 

[Evan Gilbert]

We have the requisite number of +1s here.

However, there are still are two separate group threads I'd like to touch base on before claiming consensus:

• Reinoud had a concern about undefined escaping semantics. Resolution was that we will escape strings based on context, and add full details on escaping semantics before the spec is complete. I updated http://groups.google.com/group/opensocial-and-gadgets-spec/web/expressions with details.
• There also is an active markup discussion, covering a number of topics:
• My key takeaway was that we need to support a model in OpenSocial where developers dynamically create HTML / markup from their own servers. I believe that we can define a subset of the full templating spec that is useable for these cases.
• This thread also touched on alternative templating syntax choices (XSL, PHP were mentioned). I couldn't tell if these concerns were strong enough to delay the consensus here... I'd encourage people to read through the proposal in detail @http://groups.google.com/group/opensocial-and-gadgets-spec/web before suggesting alternatives. Specifically, we need to make sure that the goals can be met (the need to render in JavaScript as well as in multiple server languages is a particularly tough one). If there are alternate choices that we think more easily meet these goals, I'm happy to discuss in more detail.

Thanks,
Evan

[Chris Chabot]

That's a good summary of the situation and i agree with it completely.

However the one thing that still puzzles me a bit is -how- we are
going to do that. Meaning: how are we going to get these pages to the
gadget?

Either it's a gadget view with a content type='url', however then
there's no social information present on their server, or you pull in
content through makeRequest, but then it's still no simple 'get your
pages (with template tags in it) to the server', they have to be
documents without any head, body etc tags in it (problem 1 when
'hosting your app' on googlepages.com), and probably people who do use
makeRequest will probably want to disable caching since otherwise
nothing is 'generated on their server', which would be a real blow for
the fighting gadget latency perspective.

Perhaps it would be an idea to pull the template out of the view
block, and put it in it's own name space, and have a type=url option
for them too? That would solve at least one of the problems (how are
we going to get 'pages' into the gadget)

That could look something like:

<?xml version="1.0" encoding="UTF-8"?>
<Module>
<ModulePrefs title="Hello World" height="100">
<Require feature="opensocial-0.7"/>
<Require feature="opensocial-templates"/>
</ModulePrefs>
<Templates>
<template name="example-one">
<div style="font-size: 20px">Hello world!</div>
</template>
<template url="http://myhost.com" name="example-two"/>
<template url="http://myhost.com" name="example-three"
authz="signed"/>
</Templates>
<Content type="html">
<![CDATA[
<script type="text/javascript">
// loadSocialData() is a function that calls OpenSocial APIs
and returns your data
loadSocialData(function(socialData) {
opensocial.template.processAll(socialData);
});
</script>
]]>
</Content>
</Module>

It wouldn't be a complete solution, but atleast brings us a step
closer to our goal of simplicity for (ex) FBML users :)

-- Chris

[Mike Samuel]

2008/6/25 Evan Gilbert <uid...@google.com>:

We have the requisite number of +1s here.

However, there are still are two separate group threads I'd like to touch base on before claiming consensus:

• Reinoud had a concern about undefined escaping semantics. Resolution was that we will escape strings based on context, and add full details on escaping semantics before the spec is complete. I updated http://groups.google.com/group/opensocial-and-gadgets-spec/web/expressions with details.
• There also is an active markup discussion, covering a number of topics:
• My key takeaway was that we need to support a model in OpenSocial where developers dynamically create HTML / markup from their own servers. I believe that we can define a subset of the full templating spec that is useable for these cases.
• This thread also touched on alternative templating syntax choices (XSL, PHP were mentioned). I couldn't tell if these

There were also alternative existing JS client-side templates, such as MJT.
 

[Kevin Brown]

Sorry for top posting -- couldn't you just include type="html" href="foo" (as added in 0.8) for importing the template files? This assumes the "semi-static" use case of today, of course, not the "proxied" solution that some of us are calling for.

[Chris Chabot]

Yea that would be a more consistent naming indeed :)

People could have 'static' html files included with a normal type="html" href="foo", which already makes live a lot easier for the hobbyists with a great idea and googlepages.com.

However if you also add a signed option to it, you would force dynamically loaded / non cached files (signed requests are never cached) .. at least you would have the container, owner and viewer available on your gadget's server (ie: the gadget developers server), and they could output content including the opensocial template tags which then get filled in by the gadget server.. and you have a pretty much working system that allows a more classic development style?

For a lot of use-cases that i imagine that should provide a working solution, without having to change the spec to much. (except for the moving the template tags into global space, and adding the type="html", href="foo" and possible name="template_name" tags)

-- Chris

[Kevin Brown]

On Wed, Jun 25, 2008 at 5:02 PM, Chris Chabot <cha...@xs4all.nl> wrote:

Yea that would be a more consistent naming indeed :)

People could have 'static' html files included with a normal type="html" href="foo", which already makes live a lot easier for the hobbyists with a great idea and googlepages.com.

However if you also add a signed option to it, you would force dynamically loaded / non cached files (signed requests are never cached) .. at least you would have the container, owner and viewer available on your gadget's server (ie: the gadget developers server), and they could output content including the opensocial template tags which then get filled in by the gadget server.. and you have a pretty much working system that allows a more classic development style?

That's exactly my proposal, actually:

<Content type="html" href="app.php" authz="signed"/>

Viola, mainstream web development.
 

[Chris Chabot]

On Jun 26, 2008, at 2:09 AM, Kevin Brown wrote:

On Wed, Jun 25, 2008 at 5:02 PM, Chris Chabot <cha...@xs4all.nl> wrote:

Yea that would be a more consistent naming indeed :)

People could have 'static' html files included with a normal type="html" href="foo", which already makes live a lot easier for the hobbyists with a great idea andgooglepages.com.

However if you also add a signed option to it, you would force dynamically loaded / non cached files (signed requests are never cached) .. at least you would have the container, owner and viewer available on your gadget's server (ie: the gadget developers server), and they could output content including the opensocial template tags which then get filled in by the gadget server.. and you have a pretty much working system that allows a more classic development style?

That's exactly my proposal, actually:

<Content type="html" href="app.php" authz="signed"/>

Viola, mainstream web development.

Hmm so content type="url" is a header location, content type="html" href="foo" is external template ... hmm i think i like it!

So if we have the html downloaded (signed) by the gadget server & rendered there (os template tags), and output it from there (important for origin policy), the javascript API would still function too! Brilliant :)

content type="url" already includes a &libs=/gadgets/js/foo:bar.js param, we could do the same with content type="html" url="foo", so that the page knows that javascripts to include to pull in the open social javascripts.

This could actually work really well!

-- Chris

[Kevin Brown]

We could always just add the scripts to the head of the document as well -- no need to make the developer do any extra work or risk any errors.

[Chris Chabot]

On Jun 26, 2008, at 2:38 AM, Kevin Brown wrote:
>> That's exactly my proposal, actually:
>>
>> <Content type="html" href="app.php" authz="signed"/>
>>
>> Viola, mainstream web development.
>
> Hmm so content type="url" is a header location, content type="html"
> href="foo" is external template ... hmm i think i like it!
>
> So if we have the html downloaded (signed) by the gadget server &
> rendered there (os template tags), and output it from there
> (important for origin policy), the javascript API would still
> function too! Brilliant :)
>
> content type="url" already includes a &libs=/gadgets/js/foo:bar.js
> param, we could do the same with content type="html" url="foo", so
> that the page knows that javascripts to include to pull in the open
> social javascripts.
>
> We could always just add the scripts to the head of the document as
> well -- no need to make the developer do any extra work or risk any
> errors.

As long as we parse the document and inject it into it ... yep that
would work great and save some error margin.

I'm actually really excited about this, this sounds like a great
solution for all the cases we discussed, and has the potential to be
really, really fast too.

If we could make the gadget server smart enough to recognize the
social information it required to complete the template (by looking at
the tags it needs to process) and request those from the social
server, it could even fill those in tags server sided, and lead to the
'server sided processing' people have been clamoring for.

-- Chris

[Louis Ryan]

On a related note. The markup fetched by <Content type="html" href="app.php" authz="signed"/> will be cached by Shindig as per the HTTP headers. An earlier proposal mentions exposing a gadget + RESTful API to allow for programmatic flushing & pushing changes to this cache. I believe this is a win-win for containers and gadget developers alike compared to how FBML is fetched by Facebook...

[Chris Chabot]

You know the only thing that currently worries me is the way the proposed templating spec works, it is still very 'Ajaxie' in it's nature, something i thought we were hoping to solve, or well at least improve :-)

See there's 3 use cases i have in mind, the hobbyist, the hobbyist web developer and the 'professional'. Since hardly anyone reads emails all the way to the end i'll start with the conclusion:

What i'm afraid of is that the proposed template spec still depends on people being 'expert developers' who know javascript .. they need to know how to create a gadget xml file, put javascript in it to load some context, parse the data to a template, and understand how to fetch various fields, that Top.Name is an owner in one case and the viewer in another, and sometimes an activity... it requires insight and 'skills'.

Most of the people who advocate a "FBML' alike solution want things to be simple ... they create a html document, put in a magic <insertNameHere> tag in the html .. and presto they have a basic social app. No javascript, no magic incantations, no big API to learn and explore ... they don't have the ambition to become a 'software engineer', they just want to make something that works and have some time left to spend with the wife and kids in the evening.

With that we'd miss out on 2 of the 3 use cases here, and i think the essence on what the long threads asking for a 'FBML type solution' have been about. 

----------- The Hobbyist ---------

The hobbyist has a great idea, it will be a great app even though the technical foundations are non existent.. he just has a great concept or great content, and he wants to create a simple html page on googlepages.com, and that's his 'gadget'.

He has read in the documentation he can add a bit of personal flair to it by putting in a special 'tag' to greet the viewer by name; He is unaware of what a tag is but it turns out it's a weird bit of text,: <os:${Viewer}.Name> ... ok he types that in his static document, and publishes it to the social network sites.

Users love so much that he becomes rich from ad income alone, and he's able to hire real programmers to improve on his app! :-)

------ The Hobbyist Web Developer ------

Ok now this guy knows a bit of {PHP,Python,Perl,RoR,whichever}, and he has a great idea.. He creates a web app using some MVC framework, and creates a local database that matches the owner and viewer ID to his local data (book lists, pictures, products, whatever).

He fills in the templates on his server with his local info, like some pictures of the books the owner user is reading, and he puts in some dynamic tags to fill in the other information, he choses to use <os:${Owner}.Name}>, <os:${Owner}.Age}> and <os:${Owner}.ThumbnailUrl}> in some nice layout he created.

Now this developer has no knowledge of Javascript, he taught him self a tiny bit of server side scripting by reading <MyLanguage> For Dummies, but that's ok .. his app is a hit since it hits all the right social notes and his idea rocks

------- The Professional Developer -------

Now this guy is paid to make magic happen, he doesn't 'do' static html, he wants optimized preloaders, he wants tuned caching headers, sub second gadget rendering time, cdn's where required, and all the interaction with the user should be animated, smooth moving, desktop like and created using javascript pretty much all the way to have that edge in the user experience. He (or his boss) has a great idea and by putting the extra sugar on top it'll truly rock!

Soon they reach the 'top 10 apps' on all the popular social network sites, since they had the idea, and the tools to make it great.

[Kevin Brown]

Yes.

[Evan Gilbert]

Chris / Kevin - I really like the spec extensions you're proposing -  we definitely need to support a model in OpenSocial where developers dynamically create HTML / markup from their own servers.

Part of the reason the proposal needs to make things work in an AJAXy model is that's how *all* OpenSocial apps are built today. We need a bridge to a new model instead of having people throw away all of the code they just built because we told them that this is how OpenSocial works. A solution that didn't address this pain point would negligent (the examples at http://docs.google.com/Presentation?id=dg6csr2s_5cgx7bkhf seem pretty compelling to me).

Even for advanced developers, having templates that can be rendered without going to 3rd party servers may be important for performance:
- Containers may require cached markup for profile views. This can't be flexibly personalized without templating
- Developers may want faster canvas views by using app data + templating. Even if a developer has their own servers, they might push data into the container so that canvas views don't make a round trip.
- Note: the round trip is *very* slow, because it goes from browser -> container -> developer server -> RESTful API to container -> developer server -> container -> browser (6 hops).

I agree that the proposal is a superset of what is needed (on the templating/markup side) for the use case where developers create HTML/markup on their own servers. However I think it supports this use case with the extensions you both are proposing as well as supporting the existing AJAXy model.

Evan

[Kevin Brown]

On Thu, Jun 26, 2008 at 2:21 PM, Evan Gilbert <uid...@google.com> wrote:

Chris / Kevin - I really like the spec extensions you're proposing -  we definitely need to support a model in OpenSocial where developers dynamically create HTML / markup from their own servers.

Part of the reason the proposal needs to make things work in an AJAXy model is that's how *all* OpenSocial apps are built today.

Not really. Most opensocial apps are just being lame and calling makeRequest to simulate the page-driven approach to work around the current model entirely.
 

We need a bridge to a new model instead of having people throw away all of the code they just built because we told them that this is how OpenSocial works. A solution that didn't address this pain point would negligent (the examples at http://docs.google.com/Presentation?id=dg6csr2s_5cgx7bkhf seem pretty compelling to me).

You don't have to change the existing code to work with the existing setup. Actually, since the current proposal doesn't include makeRequest template processing (directly, anyway), it would be *harder* to support templating in the current model unless you significantly altered your gadget.
 

Even for advanced developers, having templates that can be rendered without going to 3rd party servers may be important for performance:
- Containers may require cached markup for profile views. This can't be flexibly personalized without templating
- Developers may want faster canvas views by using app data + templating. Even if a developer has their own servers, they might push data into the container so that canvas views don't make a round trip.

That sounds like a lot of maybes to me though -- we still see that developers aren't making much use of app data and are overwhelmingly favoring connections to their own site. The most common case of not doing this are legacy igoogle gadgets, not gadgets that use social data.
 

- Note: the round trip is *very* slow, because it goes from browser -> container -> developer server -> RESTful API to container -> developer server -> container -> browser (6 hops).

You don't necessarily have to hit the REST API to do processing on your own server -- that depends on what's available in the markup / templating language.

I agree that the proposal is a superset of what is needed (on the templating/markup side) for the use case where developers create HTML/markup on their own servers. However I think it supports this use case with the extensions you both are proposing as well as supporting the existing AJAXy model.

You have to support both anyway -- that's not really the issue.
 

[Chris Chabot]

I won't re-iterate what kevin wrote and just chime in that i agree with what he said.

I think the only or main discord between the Ajaxy model is the way in which the data is retrieved. 

In the Ajaxy model you have to do the following incantation to make something happen with 'social data':

<script type="text/os-template">

hello <os:ShowPerson person="${Top.Viewer}"/>

</script>

<script type="text/javascript">

        // loadSocialData() is a function that calls OpenSocial APIs and returns your data

        loadSocialData(function(socialData) {

          opensocial.template.processAll(socialData);

        });

</script>

Now that doesn't translate to the simple web development model that people are hoping for, what they want to do is:

<html>

<body>

Hello ${Viewer}.Name

</body>

</html>

If they really want to mix it up they want to do (using a signed request, and yes this was written quick-and-very-dirty on purpose to demonstrate how people will probably use this):

<html>

<body>

Hello ${Viewer.name}. these are the books ${Owner.name} is currently reading

<?php

$owner_id = $ouath['owner_id']; // retrieve owner id from oauth

$res = mysql_select("select titles from books_this_person_is_reading where id = $owner_id");

while (list($book) = mysql_fetch_row($res)) {

echo "$book<br>";

}

?>

To perform the same stuff on the current API, even with the proposed template spec, would take 4x as much code and -more- round trips (fetch gadget xml, fetch social data, and then in the gadget fetch the books using gadget.io.makeRequest).

However this can be cut down to: fetch gadget content, fill in variables on the server, and, well there is no step 3..

It would require some magic to happen on the gadget server though, it would have to do a simple regex to extract all the social variables it touches (owner, viewer, friends, activities, etc) and fetch those and fill them in in the content. But that's not very technically challenging to create so i don't see a problem in that.

So the challange is, how do we marry those 2 types of template usage? If they used different verbiage and variable names, it would be -incredibly- confusing for the open social gadget developers, so it's in our interest to try to find a model that can be applied to both situations.

-- Chris

[Evan Gilbert]

Kevin kicked off a very productive discussion on another thread, quick summary is that we want to support a "proxied" mode that sends a request through to gadget developer's servers every time.

I think the template proposal as it stands works for "proxied" mode. We will have a choice to make whether proxied mode supports full flow control (i.e. if/repeat statements), but I'd like to defer that discussion to the proxied-mode thread if possible. The core syntax and extensibility model seem to work well both for proxied and non-proxied mode.

Chris - for your example I think some form of data request syntax is appropriate, i.e. <os:PreloadData key="Viewer" type="VIEWER" params="..."/>. Although "Viewer" could be automatically determined, as soon as we make requests for friends or app data we need the developer to specify the input parameters, so I don't think this is onerous. Chris Bissell (@ MySpace)  has been thinking about this issue I think should have a proposal ready for the group soon.

So... have we resolved open issues on the high-level design? We can tweak the syntax and tags as we go, but I'd like to lock on the high-level agreement so we can start iterating in code to find the real issues.

Evan

[Dan Peterson]

+1 -- let's get to the code!

-Dan

[Chris Chabot]

+1

Sounds reasonable and correct. So a 'template' could probably look like:

<html><body>

<os:PreloadData key="Viewer" type="VIEWER" params="..."/>

<os:PreloadData key="Owner" type="OWNER" params="..."/>
Hello <os:ShowPerson person="${Viewer.Name.UNSTRUCTURED}"/> welcome on
<os:ShowPerson person="${Owner.Name.UNSTRUCTURED}"/>'s hello world
gadget.
</body></html>

If so, that sounds great tbh. Plus this can be implemented both server
and client side, so hits all the right notes for me!

-- Chris

[Chris Chabot]

Obviously that was meant to be:

Hello <os:ShowPerson person="${Viewer}"/> welcome on
<os:ShowPerson person="${Owner}"/>'s hello world

of course.

[Krishna Sankar (ksankar)]

Yep, let us code. +1

[Kevin Brown]

On Mon, Jun 30, 2008 at 10:05 AM, Evan Gilbert <uid...@google.com> wrote:

Kevin kicked off a very productive discussion on another thread, quick summary is that we want to support a "proxied" mode that sends a request through to gadget developer's servers every time.

I think the template proposal as it stands works for "proxied" mode. We will have a choice to make whether proxied mode supports full flow control (i.e. if/repeat statements), but I'd like to defer that discussion to the proxied-mode thread if possible. The core syntax and extensibility model seem to work well both for proxied and non-proxied mode.

Chris - for your example I think some form of data request syntax is appropriate, i.e. <os:PreloadData key="Viewer" type="VIEWER" params="..."/>. Although "Viewer" could be automatically determined, as soon as we make requests for friends or app data we need the developer to specify the input parameters, so I don't think this is onerous. Chris Bissell (@ MySpace)  has been thinking about this issue I think should have a proposal ready for the group soon.

So... have we resolved open issues on the high-level design? We can tweak the syntax and tags as we go, but I'd like to lock on the high-level agreement so we can start iterating in code to find the real issues.

I do wonder if certain elements are necessary at all if we support templates as a more "first class" part of the spec -- specifically:

- Do we need the JavaScript element? (why not just script tags?)
- Is the js bootstrapping necessary? (it could juts be a "standard" thing that gets done automatically)

Also, I'm still unconvinced about the need for two different ways to do many of the operators (flow control + localization). Can you explain why we need both attributes and tags for this? It's my understanding that you went with attributes in the interest of making a cleaner schema, but the tags are in there now anyway, so it seems like that's not an important issue anymore. Do we really need the attributes?

 

[Evan Gilbert]

responses below...

On Mon, Jun 30, 2008 at 3:05 PM, Kevin Brown <et...@google.com> wrote:

On Mon, Jun 30, 2008 at 10:05 AM, Evan Gilbert <uid...@google.com> wrote:

Kevin kicked off a very productive discussion on another thread, quick summary is that we want to support a "proxied" mode that sends a request through to gadget developer's servers every time.

I think the template proposal as it stands works for "proxied" mode. We will have a choice to make whether proxied mode supports full flow control (i.e. if/repeat statements), but I'd like to defer that discussion to the proxied-mode thread if possible. The core syntax and extensibility model seem to work well both for proxied and non-proxied mode.

Chris - for your example I think some form of data request syntax is appropriate, i.e. <os:PreloadData key="Viewer" type="VIEWER" params="..."/>. Although "Viewer" could be automatically determined, as soon as we make requests for friends or app data we need the developer to specify the input parameters, so I don't think this is onerous. Chris Bissell (@ MySpace)  has been thinking about this issue I think should have a proposal ready for the group soon.

So... have we resolved open issues on the high-level design? We can tweak the syntax and tags as we go, but I'd like to lock on the high-level agreement so we can start iterating in code to find the real issues.

I do wonder if certain elements are necessary at all if we support templates as a more "first class" part of the spec -- specifically:

- Do we need the JavaScript element? (why not just script tags?)

For standalone XML tag libraries, this is needed. If you're embedding templates, you can just use <script type="text/javascript"> in a separate block on the page.
 

- Is the js bootstrapping necessary? (it could juts be a "standard" thing that gets done automatically)

There are two bootstrap pieces, I think we should support both but also work towards an XML-only variant.

The bootstrap pieces:
- <script type="text/os-template">. It is a strong goal for templating to be possible in standalone HTML pages with just the inclusion of a JS library. For this to work, all XML tags need to be inside an unprocessable HTML element (<script> or <xmp> were the two we found).
- opensocial.template.processAll(). We definitely need the ability to render templates from JavaScript (for AJAXy solutions and just for testing), and currently there isn't an <os:PreloadData> call. Seemed to be a good starting point to attach templates to the existing JS API

The XML variant (roughly)
- <Content type="text/os-template"> with only XML content. This would both avoid the <script> tag and processAll() call.
- Note that this requires equivalent of<os:PreloadData key="Viewer" type="VIEWER" params="..."/>

Also, I'm still unconvinced about the need for two different ways to do many of the operators (flow control + localization). Can you explain why we need both attributes and tags for this? It's my understanding that you went with attributes in the interest of making a cleaner schema, but the tags are in there now anyway, so it seems like that's not an important issue anymore. Do we really need the attributes?

There are enough people passionate on both sides that it seemed to make sense to just support both. We had a breakout session on OpenSocial templates at the OpenSocial API summit, and everyone seemed to think this was a fair compromise and it wasn't too confusing.

Here's the quick list of pros on each side:

Attributes
- Less typing
- Don't require an extra level of nesting
- May allow for schema validation

Elements
- People are more familiar with elements
- Close to necessary for a couple of use cases.

Having written a lot of XSL code, I would not be in favor of an element-only solution (happy to paste some examples in if this would help clarify :))

[Kevin Brown]

On Mon, Jun 30, 2008 at 3:57 PM, Evan Gilbert <uid...@google.com> wrote:

responses below...

On Mon, Jun 30, 2008 at 3:05 PM, Kevin Brown <et...@google.com> wrote:

On Mon, Jun 30, 2008 at 10:05 AM, Evan Gilbert <uid...@google.com> wrote:

Kevin kicked off a very productive discussion on another thread, quick summary is that we want to support a "proxied" mode that sends a request through to gadget developer's servers every time.

I think the template proposal as it stands works for "proxied" mode. We will have a choice to make whether proxied mode supports full flow control (i.e. if/repeat statements), but I'd like to defer that discussion to the proxied-mode thread if possible. The core syntax and extensibility model seem to work well both for proxied and non-proxied mode.

Chris - for your example I think some form of data request syntax is appropriate, i.e. <os:PreloadData key="Viewer" type="VIEWER" params="..."/>. Although "Viewer" could be automatically determined, as soon as we make requests for friends or app data we need the developer to specify the input parameters, so I don't think this is onerous. Chris Bissell (@ MySpace)  has been thinking about this issue I think should have a proposal ready for the group soon.

So... have we resolved open issues on the high-level design? We can tweak the syntax and tags as we go, but I'd like to lock on the high-level agreement so we can start iterating in code to find the real issues.

I do wonder if certain elements are necessary at all if we support templates as a more "first class" part of the spec -- specifically:

- Do we need the JavaScript element? (why not just script tags?)

For standalone XML tag libraries, this is needed. If you're embedding templates, you can just use <script type="text/javascript"> in a separate block on the page.
 

- Is the js bootstrapping necessary? (it could juts be a "standard" thing that gets done automatically)

There are two bootstrap pieces, I think we should support both but also work towards an XML-only variant.

The bootstrap pieces:
- <script type="text/os-template">. It is a strong goal for templating to be possible in standalone HTML pages with just the inclusion of a JS library. For this to work, all XML tags need to be inside an unprocessable HTML element (<script> or <xmp> were the two we found).
- opensocial.template.processAll(). We definitely need the ability to render templates from JavaScript (for AJAXy solutions and just for testing), and currently there isn't an <os:PreloadData> call. Seemed to be a good starting point to attach templates to the existing JS API

Sorry, I was just implying that opensocial.template.processAll() should be something containers were expected to call automatically -- similar to gadgets.util.runOnLoadHandlers. I have no issue with custom template blocks being specified with script tags.
 

The XML variant (roughly)
- <Content type="text/os-template"> with only XML content. This would both avoid the <script> tag and processAll() call.
- Note that this requires equivalent of<os:PreloadData key="Viewer" type="VIEWER" params="..."/>

Also, I'm still unconvinced about the need for two different ways to do many of the operators (flow control + localization). Can you explain why we need both attributes and tags for this? It's my understanding that you went with attributes in the interest of making a cleaner schema, but the tags are in there now anyway, so it seems like that's not an important issue anymore. Do we really need the attributes?

There are enough people passionate on both sides that it seemed to make sense to just support both. We had a breakout session on OpenSocial templates at the OpenSocial API summit, and everyone seemed to think this was a fair compromise and it wasn't too confusing.

Here's the quick list of pros on each side:

Attributes
- Less typing
- Don't require an extra level of nesting
- May allow for schema validation

Elements
- People are more familiar with elements
- Close to necessary for a couple of use cases.

Having written a lot of XSL code, I would not be in favor of an element-only solution (happy to paste some examples in if this would help clarify :))

Ok, that sounds like a plausible explanation. From an implementation standpoint, can we safely say that we can always transform

<div if="${Top.YourScore == Top.HighScore}">Good job!</div>

Into:

<If expr="${Top.YourScore == Top.HighScore}">
 <div...
</If>

That seems ok.
 

[Ignacio Blanco]

Hi,

I would like to know, why the spec should be taking care of templates, when they are already implemented in some JS toolkits as Prototype for example.

I'm ok on providing helpers for the UI, but I think that the spec should be maintainted as minimal as possible.

My 2 cents.

--
"If you have an apple and I have an apple and we exchange these apples
then you and I will still each have one apple.
But if you have an idea and I have an idea and we exchange these
ideas, then each of us will have two ideas." Bernard Shaw

[Evan Gilbert]

response below...

On Tue, Jul 1, 2008 at 10:11 AM, Ignacio Blanco <blan...@gmail.com> wrote:

Hi,

I would like to know, why the spec should be taking care of templates, when they are already implemented in some JS toolkits as Prototype for example.

For OpenSocial, a couple of requirements led to needing a spec:
- We need a standard set of <os:*> tags that are supported by all containers (with different implementations), and to define common tags you have to define how templating works
- We need to support templating both in JavaScript and on the server
- OpenSocial containers may not support arbitrary JS includes in all views. For PROFILE/HOME_PAGE views, the container often will have performance goals that may lead to limiting the use of <script> src. We still want templating to be available in these cases.

The template spec won't prevent developers from using their own client-side JS templating (although as mentioned above, containers might).

[Evan Gilbert]

OK - I think we have answered the major concerns (picking up a couple of +1's along the way) and we're at consensus.

Please speak up if you're a -1... otherwise I think we have an official spec enhancement!

Evan

[Evan Gilbert]

I believe we have consensus.

Expect follow up threads soon to start the discussion on the first set of <os:*> tags.

There is already a thread on Shindig for getting a client-side implementation working in JS, and a cool prototype has already been posted. I'll forward this to the spec list.

Evan

[Lane LiaBraaten]

Hi Folks,

We voted on OpenSocial Templates back in June so for completeness I've added OpenSocial Templates to the list of approved proposals for v0.9.  A few useful links if you want to look at the details again:

The wiki page with summary: http://wiki.opensocial.org/index.php?title=OpenSocial_Templates

The spec: http://wiki.opensocial-templates.org/index.php?title=OpenSocial_Templating

The thread where we voted: http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/21e0e6848d4134b7/b54ea47139d078da?lnk=gst&q=%2B1+for+high+level+design#b54ea47139d078da

-Lane

On Fri, Jun 20, 2008 at 8:42 AM, Evan Gilbert <uid...@google.com> wrote:

On Wed, Jun 18, 2008 at 2:58 PM, Mike Austin <mau...@hi5.mi8.com> wrote:

+1 for high-level design

 

We're very interested in is the <os:Message> tag, which makes it a whole lot easier to write localized apps.  Of course, you can simply pass the current language to your server and handle the message bundles on the backend, but for beginning OpenSocial developers, it provides a standard way of localizing.

 

Evan, I forget if there is a way to specify a div that your template will process in, similar to Ajax.Updater?  For example, opensocial.template.render('my-div', 'my-template', myData).

We definitely need to support this functionality, the specific API signature isn't specified. There are a number of these JS API details that I hope can be fleshed out and proposed back into the spec as we work through the implementation.

 

We also mentioned the ability to implicitly use tags in attributes with some sort of "set" tag, for example:

 

<os:Set var="tooltip" value="<os:Message key="background-color-help"/>

<img title="${tooltip}" alt="${tooltip}"/>

Definetely want to support a <Set> tag, not positive we need it in first iteration.

The idea I was thinking for localizable attributes was to use on of  the following mechanisms, hidden under nicer API:

Load using a message ID:

<img oncreate="this.title=opensocial.template.renderMessage('background-color-help');/>,

Or if you need default text, Load from contents of another element:

<img oncreate="this.title = document.getElementById('img-text').innerHTML"/>
<div style="display-none" id="img-text"><os:Message key="background-color-help">Hello</os:Message></div>

For common use cases, we can hide these under OpenSocial tags:

<os:Image src="...">
  <os:Title msg="background-color-help">Some default text</os:Title>
</os:Image>

Is this sufficient to handle the messaging use cases?

 

And compared to all the injected HTML that is included in an open social app, I don't think rendering time is much of a challenge, unless you have one huge template.  You can break them up and display them as your data comes in, and you update certain sections of your pages like Ajax driven sites.  This is possible, correct?

Yes, app developer has flexibility to do this. Depending on how the data comes in, it may make sense to wait for all the data and render once, or if there are straggler requests to render in stages.

The rendering time should be good in JS but needs to be watched - creating HTML using JavaScript can be shockingly slow
 

 

Mike Austin

hi5.com

---

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Evan Gilbert
Sent: Monday, June 09, 2008 6:04 PM
To: opensocial-an...@googlegroups.com

Subject: [Spec Proposal] OpenSocial Templates

 

I've incorporated feedback from a number of discussions and now am looking for +1s on the high-level design for OpenSocial templates.

Spec proposal:  http://groups.google.com/group/opensocial-and-gadgets-spec/web/opensocial-templates
Single document version:  http://docs.google.com/Doc?id=dg6csr2s_3gttjwjzz

The definition of <os:> tags is not covered yet - we will cover each tag as a separate spec enhancement, and I expect there to be a fair number clarifications and minor revisions to the proposal over the course of implementation.

Thanks for all the feedback, and looking forward to getting templates up and running!

Evan

[Dan Peterson]

Hey Lane, thanks for catching this and adding it -- templates is an important piece of OpenSocial 0.9.

-Dan

[Dave]

On Thu, Nov 27, 2008 at 1:36 AM, Dan Peterson <dpet...@google.com> wrote:
> I noticed that you put up a screenshot of the app gallery you built using
> OpenSocial templates:Would you be up for sharing your experiences
> (and code?) with the spec list? It'd be great to get your feedback on how
> it all went down.

Thanks for the (2nd) prompt. I've been using OpenSocial templates for
over a month now and have nothing but good things to say about the
current spec and implementation. Thanks to the folks behind the
templates effort for some very useful tech.

> Is it using templates inside a gadget? Is it using the templates
> as per the spec below?

Yes. Except for our Admin Console, all UI in SocialSite is implemented
as OpenSocial Gadgets. Our newest Gadgets use OpenSocial templates and
YUI components and I've been very happy with the results. Our Gadget
code is much easier to write, understand and maintain now. Here are
some examples.

First, our Dashboard, which allows a user to browse and interact with
friends, groups, activities and messages. You can send messages,
create groups, request and accept friendships. This is all implemented
via the SocailSite API, OpenSocial templates and YUI.

More information about the Dashboard and screenshots here:
http://blogs.sun.com/socialsite/entry/a_little_more_about_socialsite
http://blogs.sun.com/socialsite/resource/dashboard-activities.png
http://blogs.sun.com/socialsite/resource/dashboard-people.png

Code for the Dashboard here:
dashboard.xml - http://tinyurl.com/5zms2g
dashboard.js - http://tinyurl.com/56w295

And second, we have a Gadget Directory, which allows you to browse
available Gadgets and install them for either a user or for a group
(we support Gadgets "owned" by a group via the SocialSite API). This
is templates and YUI too:

Screenshot of the Gadget Directory
http://www.flickr.com/photos/snoopdave/3059025664

Code for the Gadget Directory
gadget_directory.xml - http://tinyurl.com/5blb47
gadget_directory.js - http://tinyurl.com/69o4c4

I haven't used any template library features yet, so I can't comment
on them, but I suspect they'll be very helpful as we rewrite more of
our Gadgets to use templates. Same goes for I18N.

- Dave

> ---------- Forwarded message ----------
> From: Lane LiaBraaten <llia...@google.com>
> Date: Wed, Nov 26, 2008 at 10:20 AM
> Subject: [opensocial-and-gadgets-spec] Re: [Spec Proposal] OpenSocial
> Templates