Allow JavaScript objects to be stored in AppData

4 views
Skip to first unread message

Arne Roomann-Kurrik

unread,
Mar 7, 2008, 8:42:15 PM3/7/08
to opensocial-an...@googlegroups.com
Currently, calls to newUpdatePersonAppDataRequest only accept data in the form of strings.  If I wish to store complex data structures, I need to convert them using gadgets.json.stringify and store the result.  To retrieve the data, I must retrieve the object and run gadgets.json.parse on the returned string. 

Since mandatory escaping was introduced to keep developers from blindly dumping unescaped data into the innerHTML of a page element, another step has been added to my flow - before running json.parse, I must gadgets.util.unescapeString on the returned data first.  This has a side effect of unescaping any untrusted data contained in my object.

For an example, check out http://graargh.returnstrue.com/buh/escape_data.xml - it stores a link containing javascript in a JSON object that is:
  1.) Converted to a string
  2.) Stored in appData
  3.) Requested by the app
  4.) Escaped by the container and passed back to the app
  5.) Unescaped by the app
  6.) Parsed back into an object
  7.) Dumped into the innerHTML of a page element

The javascript remains intact.  Of course, I could just change the following line:

     books.innerHTML = mydata.books_i_like;
to
     books.innerHTML = gadgets.util.escapeString(mydata.books_i_like);

But I still need to know to manually escape data that I'm dumping back to the page, so no net benefit has come from the auto-escape.

If the container's updateAppData call were made to be able to accept objects, the container could handle the conversion to and from JSON.  When returning an object, the container could then iterate over each key in the parsed result and escape string values appropriately.  The result would be a native JS object with escaped fields, which would not need to be unescaped to be usable in my application.  The incentive to unescape drops dramatically.  As a side effect, app data becomes easier to use, since I don't have to put all the JSON encode/decode logic into my code.

Thoughts?
~Arne

John Hjelmstad

unread,
Mar 7, 2008, 10:01:24 PM3/7/08
to opensocial-an...@googlegroups.com
Arne:

To me this sounds like an extension of the notion that all strings passed in AppData are escaped as a preventive measure. It essentially closes what's likely to be the most commonly-exploited (likely without any malicious intent) hole in the content-escaping strategy. Since we're doing escaping already for AppData, it doesn't seem like too large a burden on developers that all string values in JSON objects be treated likewise.

Cost: My assumption is that most JSON objects stored as AppData will be reasonably small, and iterating over their contents to escape strings should represent a modest cost for the associated security gains.

What API would you suggest be used for this? We could introduce a new JSON-only API, or specify that newUpdatePersonAppDataRequest auto-detect input data type. I'm assuming from your text you prefer the latter, and I would as well.

That said, unless I'm missing something it seems this imposes an additional burden on OpenSocial API servers to store data by type, due to the introduction of ambiguity between strings that begin with "{" (or "[", if arrays are to be supported in JSON-object-storage style per this proposal) and JSON objects.

Consider:
var storeReq=opensocial.newDataRequest();
storeReq.add(req.newUpdatePersonAppDataRequest(...VIEWER, "one", "{ foo: 1 }");
storeReq.add(req.newUpdatePersonAppDataRequest(...VIEWER, "two", { foo: 1 });
storeReq.send(...);
...later...

var getReq=opensocial.newDataRequest();
getReq.add(req.newFetchPersonAppDataRequest(...VIEWER, [ "one", "two" ]);
getReq.send(handleGetReq);

function handleGetReq(data) {
  var oneData = data.get("one");
  var twoData = data.get("two");
  ...use data...
}

In retrieving oneData and twoData in the latter function, the server needs to determine their type for proper unescaping, because escaped, they are identical. On the server side, storing only untyped strings/blobs has simplicity benefits. True, the implementation could be as simple as prefixing a stored string with a type-identifier (eg "S{ foo: 1 }" and "J{ foo: 1 }"). However, such an implementation would presumably be backward-incompatible with those that exist now.

Backward-compatible implementations could do something like reserve an unusual prefix (eg "{ foo: 1 }" -> string, "_x_x_x__{ foo: 1 }" -> JSON), which could work in real-world use but should be standardized for consistent behavior across sites.

Thoughts?
John

Brian Stoler

unread,
Mar 7, 2008, 10:05:37 PM3/7/08
to opensocial-an...@googlegroups.com
How about extending to allow the value to not be a string at all?


storeReq.add(req.newUpdatePersonAppDataRequest(...VIEWER, "one", { foo: 1 });

Implementations could check typeof to decide whether to escape (string) or JSON-ify (object).

-brian

John Hjelmstad

unread,
Mar 7, 2008, 10:10:15 PM3/7/08
to opensocial-an...@googlegroups.com
I'm not sure I see how this gets around the issue.

In either case "{ foo: 1 }" and { foo: 1 } serialize identically. When writing data that's fine; when consuming it, in order to know what type to emit you need to know what type was stored as input.

Am I missing something?

--John

Kevin Brown

unread,
Mar 8, 2008, 2:28:58 PM3/8/08
to opensocial-an...@googlegroups.com
You could still store a type on the backend, you just wouldn't require app developers to be explicit about what type they're storing. I think I like Brian's proposal.
--
~Kevin

Sergio Marti

unread,
Mar 8, 2008, 3:42:07 PM3/8/08
to opensocial-an...@googlegroups.com
this could be implemented as JS library utility functions that wrap
the String-only API. When writing values a JS function marshals it to
string, adding a type indicator at the start or something similar.
Then a matching utility function unmarshals the string to the
appropriate type by looking at the type indicator. These would be
optional utility methods that would sit on top of the existing API and
thus not require any container specific implementation. Or it
could be embedded into the JS API.
The one complication from optional client-side type marshaling may
come when we have a RESTful API that needs to get and set data in a
way that is compatible with the JS API..

Kevin Brown

unread,
Mar 8, 2008, 4:02:29 PM3/8/08
to opensocial-an...@googlegroups.com
That doesn't quite work, because escapeString() is ignorant of data type, and (by design) idempotent.

When it's JSON, you still want to escape the actual keys / values, but you don't want to escape the double quotes. What you're proposing would wind up with something like this:

JSON{"foo":&#34bar"}

unescapeString() produces:

JSON{"foo":"bar"}

So far so good, right?

What if the JSON data looks like this?

{"title":"Hello world!", "content": "Kevin <b>thinks</b> that you have a really good idea so far!"}

The stored form is:

"JSON{&#34;title&#34;:&#34;Hello world!&#34;,&#34;content&#34;:&#34;Kevin &#60;b&#62;thinks&#60;/b&#62; that you have a really good idea so far!&#34;}"

Unescape produces:

JSON{"title":"Hello world!", "content": "Kevin <b>thinks</b> that you have a really good idea so far!"}

Still looking good, right?

But what happens when the actual value of "content" needs to be escaped? This requires going back to requiring gadget authors manually escaping their data, and that means that the entire auto-escaping solution doesn't add any extra security at all.
--
~Kevin

John Panzer

unread,
Mar 8, 2008, 4:53:18 PM3/8/08
to opensocial-an...@googlegroups.com
On Sat, Mar 8, 2008 at 12:42 PM, Sergio Marti <sma...@google.com> wrote:
>
> this could be implemented as JS library utility functions that wrap
> the String-only API. When writing values a JS function marshals it to
> string, adding a type indicator at the start or something similar.
> Then a matching utility function unmarshals the string to the
> appropriate type by looking at the type indicator. These would be
> optional utility methods that would sit on top of the existing API and
> thus not require any container specific implementation. Or it
> could be embedded into the JS API.
> The one complication from optional client-side type marshaling may
> come when we have a RESTful API that needs to get and set data in a
> way that is compatible with the JS API..

You could just assume that the content is always JSON and use
enclosure in "" as the signal that it's just a string.

That's effectively what the current RESTful API proposal does.

Alternatively, there should be a way to indicate that the content is
text/plain or application/json :)

Kevin Brown

unread,
Mar 8, 2008, 6:37:31 PM3/8/08
to opensocial-an...@googlegroups.com
I think John's proposal makes sense. Always treat it as JSON, and then just make various retrieval calls aware of this fact and escape / unescape appropriately. Existing sites such as hi5 or orkut would have to migrate the existing stored data (or come up with some other way to deal with data that has already been stored).
--
~Kevin

Arne Roomann-Kurrik

unread,
Mar 10, 2008, 12:45:54 PM3/10/08
to opensocial-an...@googlegroups.com
+1 to this idea - it seems like it would have the least negative impact on existing applications and solves the need to re-encode object keys.  Could existing data be ported just by wrapping it with enclosing quotes, treating it like a JSON-encoded string?

~Arne

Kevin Brown

unread,
Mar 10, 2008, 3:33:00 PM3/10/08
to opensocial-an...@googlegroups.com
Probably, but that's up to the containers to decide.

Could we get some feedback from other containers that have already launched with app data support on this issue? It sounds like Orkut will be willing to do this (and we'll figure out what to do with existing data), but what about others? hi5?
--
~Kevin

Paul Lindner

unread,
Mar 10, 2008, 3:37:07 PM3/10/08
to opensocial-an...@googlegroups.com
On Mon, Mar 10, 2008 at 12:33:00PM -0700, Kevin Brown wrote:
> Probably, but that's up to the containers to decide.
>
> Could we get some feedback from other containers that have already launched
> with app data support on this issue? It sounds like Orkut will be willing to
> do this (and we'll figure out what to do with existing data), but what about
> others? hi5?

Our app data support is very weak at the moment. We'll try to take
whatever's current prior to our launch at the end of the month.

--
Paul Lindner ||||| | | | | | | | | |
lin...@inuus.com

John Hjelmstad

unread,
Mar 10, 2008, 3:54:24 PM3/10/08
to opensocial-an...@googlegroups.com
John's proposal sounds reasonable to me as well -- my main point is that unless I'm missing something, any addition of typing implies non-backwards-compatibility with existing implementations for "legacy" data stored before typing existed. There are few such implementations now, so that's likely not an issue (and certainly makes this the right time to add typing if we're ever going to do so), but I thought worth mentioning so that migration paths can be devised where needed.

John

Kevin Marks

unread,
Mar 10, 2008, 4:00:08 PM3/10/08
to opensocial-an...@googlegroups.com
JSON for the persisted data seems a very natural thing for developers.
The forced HTML escaping on write seems like doing things in the wrong place to me; I think the right answer there would be a sanctioned (and Cajoling-friendly) setInnerHtmlByID call, that escapes by default.

If the JSON serializing/unserializing is done in the Data APIs, the HTML escaping is much less necessary.

Brian Eaton

unread,
Mar 20, 2008, 7:58:19 PM3/20/08
to opensocial-an...@googlegroups.com
*bump*

This thread seemed to be going somewhere useful. Were we approaching
consensus on anything?

Can we find some way to make the APIs safe for novice developers
without inconveniencing the more advanced types who are sticking JSON
in appdata?

Cassie

unread,
Apr 1, 2008, 8:01:41 AM4/1/08
to opensocial-an...@googlegroups.com
Okay, so I see a couple possible spec changes to address this issue:

1. all input comes is treated as json. all the strings that are currently in the data store are fine, as they are just strings still. escaping will be done on all the sub fields of the json. which, for all the original data means that nothing will change (as the entire string is a field). however, apps going forward can take advantage of the new type and won't need to use stringify, unescapeString and parse on the json object (html within the json will still need to be unescaped though)
pros: invisible to developer
cons: everything has to be valid json? (a regular string is valid json right?)

2. you can specify the type of data. this type will need to be stored in backend data servers. all original data will have the String type and the default will still be String. escaping will take place in the same was as #1.
pros: We could add many more complex types as time goes on. These types could be used to make auto uis (like user pref editing) or other things I can't think of.
cons: The datastore needs to know about types

3.  we leave it as it is
pros: nothing changes in code, backends or gadgets
cons: apps are still unescaping blindly


A separate issue is a setHtml wrapper that perhaps unescapes a string for you and sanitizes or something. We can address that after the first part is resolved.

Okay, please help me clarify those issues as I'm sure I either got something wrong or left something out.
And... vote!

- Cassie

Reinoud Elhorst

unread,
Apr 1, 2008, 8:16:21 AM4/1/08
to opensocial-an...@googlegroups.com
Cassie wrote:
> Okay, so I see a couple possible spec changes to address this issue:
>
> 1. all input comes is treated as json. all the strings that are currently in
> the data store are fine, as they are just strings still. escaping will be
> done on all the sub fields of the json. which, for all the original data
> means that nothing will change (as the entire string is a field). however,
> apps going forward can take advantage of the new type and won't need to use
> stringify, unescapeString and parse on the json object (html within the json
> will still need to be unescaped though)
> pros: invisible to developer
> cons: everything has to be valid json? (a regular string is valid json
> right?)
I think that all current strings in store need to be changed, to add
quotes and escaping:
hello --> "hello"
that's right --> "that\'s right"
They are valid JSON then.

I like this first solution best, but it's extra work to already existing
stores. I don't like the whole HTML escaping at all, I don't believe
getting free gloves with every knife you buy; you just need to know not
to cut yourself in the finger. It reminds me way too much of PHP's magic
quotes....

Then again, that issue has been discussed before and we decided in favor
of escaping then...

Reinoud Elhorst

unread,
Apr 1, 2008, 8:19:43 AM4/1/08
to opensocial-an...@googlegroups.com
Reinoud Elhorst wrote:
> Cassie wrote:
>> Okay, so I see a couple possible spec changes to address this issue:
>>
>> 1. all input comes is treated as json. all the strings that are currently in
>> the data store are fine, as they are just strings still. escaping will be
>> done on all the sub fields of the json. which, for all the original data
>> means that nothing will change (as the entire string is a field). however,
>> apps going forward can take advantage of the new type and won't need to use
>> stringify, unescapeString and parse on the json object (html within the json
>> will still need to be unescaped though)
>> pros: invisible to developer
>> cons: everything has to be valid json? (a regular string is valid json
>> right?)
> I think that all current strings in store need to be changed, to add
> quotes and escaping:
> hello --> "hello"
> that's right --> "that\'s right"
> They are valid JSON then.
Sorry, that's right should be "that's right" (the ' is not quoted in
json). Some other characters ("\) are though.

Oleg Pylnev

unread,
Apr 1, 2008, 12:26:50 PM4/1/08
to opensocial-an...@googlegroups.com

I like the second solution as it allows to explicitly support JSON as a data type while providing less overhead to simple data types.

 

-Oleg Pylnev

Myspace

Cassie

unread,
Apr 14, 2008, 7:27:51 AM4/14/08
to opensocial-an...@googlegroups.com
Anybody else have an opinion on how we should handle the json data?
We have 1 in favor of #2.

- Cassie

Cassie

unread,
Apr 14, 2008, 7:29:27 AM4/14/08
to opensocial-an...@googlegroups.com
Oh sorry, and 1 in favor of #1.

Arne Roomann-Kurrik

unread,
Apr 14, 2008, 1:14:58 PM4/14/08
to opensocial-an...@googlegroups.com
+1 for the first option.  You can currently only store strings, which are valid JSON (when wrapped in quotes) so this shouldn't be a breaking change, and it improves the security and usability of the Persistence API without requiring developers to learn about custom data types (will we ever need anything more complex than Number, String, Object, and Array?).

~Arne
--
OpenSocial IRC - irc://irc.freenode.net/opensocial

Kevin Brown

unread,
Apr 14, 2008, 3:12:42 PM4/14/08
to opensocial-an...@googlegroups.com
I'm in favor of 1 or 2; 2 is a bit more flexible, but requires a more extensive change than 1 would.
--
~Kevin

Zhen Wang

unread,
Apr 14, 2008, 4:38:21 PM4/14/08
to opensocial-an...@googlegroups.com
I favor option 1 for its simplicity.

Kevin Marks

unread,
Apr 14, 2008, 6:47:19 PM4/14/08
to opensocial-an...@googlegroups.com
I'm in favour of 1 too - by specify JSON we are using a well-defined subset of JavaScript that is well-tested, and inherently mroe compatible wiht Caja. Crude HTML-escaping is a poor protection against injection attacks in any case.

Graham Spencer

unread,
Apr 14, 2008, 9:01:02 PM4/14/08
to opensocial-an...@googlegroups.com
I liked the earlier proposal from Kevin Marks -- don't escape at all, because developers will just call unescape anyway; instead make it easier to safely insert HTML into the document. Arguments against escaping the entire JSON subtree:
  1. It's surprising (IMO) to pass in an elaborate structure and then get it back with lots of child data escaped.
  2. String data can have many different purposes; for all of the cases where I'm *not* about to insert the data into HTML, it seems onerous (again IMO) to force me to unescape that data all the time.
  3. I believe that developers who don't understand security will just immediately unescape the escaped data, making forced escaping pointless anyway.
  4. Apps will get data from many other places, and that data won't be escaped by default, so it's still in our best interests to offer easy ways to safely insert content that might be malicious. In other words, forced escaping of app data doesn't solve the whole problem.
--g

Zhen Wang

unread,
Apr 14, 2008, 9:29:10 PM4/14/08
to opensocial-an...@googlegroups.com
+1!

Automatically escaping app data prevents developers from XSS attacks
as ineffectively as php's magic_quotes function against SQL injection.

Reinoud Elhorst

unread,
Apr 15, 2008, 6:20:12 AM4/15/08
to opensocial-an...@googlegroups.com
+ one billion on not html-escaping the data!

I was affraid I had missed the boat on this discussion, glad to see that so many other people dislike this idea.

Chris Chabot

unread,
Apr 15, 2008, 6:29:37 AM4/15/08
to opensocial-an...@googlegroups.com
+1 as well, false sense of security often leads to a whole lot of insecurity and bug\\\'s

On Apr 15, 2008, at 12:20 PM, Reinoud Elhorst wrote:
+ one billion on not html-escaping the data!


Reinoud, no cheating allowed here! :)

Arne Roomann-Kurrik

unread,
Apr 15, 2008, 12:33:31 PM4/15/08
to opensocial-an...@googlegroups.com
On Tue, Apr 15, 2008 at 3:29 AM, Chris Chabot <cha...@xs4all.nl> wrote:
+1 as well, false sense of security often leads to a whole lot of insecurity and bug\\\'s

I see what you did there.  Touché ;)

Lane LiaBraaten

unread,
Apr 15, 2008, 12:48:58 PM4/15/08
to opensocial-an...@googlegroups.com
+1

Brian Eaton

unread,
Apr 15, 2008, 6:50:24 PM4/15/08
to opensocial-an...@googlegroups.com
I may be misunderstanding how appdata is used, so let me double check.

1) any user can write to any of their friend's app data
2) the typical way a gadget uses app data is to pull data out and then
assign that data to innerHTML

Without automatic escaping, that's completely insecure. XSS worms
will spread through app data, infecting every user who has the
application installed.

I love the idea of providing an HTML sanitizer to make it easy for
developers to deal with rich text coming from untrusted sources. I
think the one from Caja is suitable, we just need to decide on a
function name for the opensocial spec.

Cheers,
Brian

On Mon, Apr 14, 2008 at 6:01 PM, Graham Spencer <g...@google.com> wrote:

Graham Spencer

unread,
Apr 15, 2008, 6:55:43 PM4/15/08
to opensocial-an...@googlegroups.com
This is correct, except that I disagree that automatic escaping solves this problem. My assertion is that developers will simply wrap every appdata call in unescape() before assigning to innerHTML, removing any security benefits. So IMO it's better to tell developers "always use safeSetInnerHTML()" insead of telling developers "never use unescape()". And safeSetInnerHTML helps with data from other sources, not just appdata.

--g

Arne Roomann-Kurrik

unread,
Apr 15, 2008, 6:56:57 PM4/15/08
to opensocial-an...@googlegroups.com
On Tue, Apr 15, 2008 at 3:50 PM, Brian Eaton <bea...@google.com> wrote:

I may be misunderstanding how appdata is used, so let me double check.

1) any user can write to any of their friend's app data

A user can only write to VIEWER data.  Users can retrieve data for OWNER_FRIENDS, OWNER, VIEWER_FRIENDS (maybe), VIEWER (maybe).  See http://code.google.com/apis/opensocial/articles/persistence.html for everything you ever wanted to know about Persistence ;)

2) the typical way a gadget uses app data is to pull data out and then
assign that data to innerHTML

Yes, after running it through escapeString if they're doing JSON encoding and coding responsibly.
 

Brian Eaton

unread,
Apr 15, 2008, 7:04:00 PM4/15/08
to opensocial-an...@googlegroups.com
Automatic escaping does not solve the problem 100%, but it's extremely
rare to solve any kind of software security problem for 100% of cases.
The current escaping solves the problem for the simplest case where
gadgets are storing plain text in app data and assuming that they are
getting plain text back. If we don't do that escaping, the most
simple 'hello world' use of app data will have a serious security
problem.

Maybe a function like 'setAppData(html, AppDataType.HTML)' that
automatically does HTML sanitization would improve the usability of
the API while still leaving it safe by default?

Cheers,
Brain

Reinoud Elhorst

unread,
Apr 16, 2008, 5:26:06 AM4/16/08
to opensocial-an...@googlegroups.com
I think it's not in the spec which (whose) data can be written and read, but above is probably the most likely implementation for most containers.


On Wed, Apr 16, 2008 at 1:04 AM, Brian Eaton <bea...@google.com> wrote:

Automatic escaping does not solve the problem 100%, but it's extremely
rare to solve any kind of software security problem for 100% of cases.
 The current escaping solves the problem for the simplest case where
gadgets are storing plain text in app data and assuming that they are
getting plain text back.  If we don't do that escaping, the most
simple 'hello world' use of app data will have a serious security
problem.

Maybe a function like 'setAppData(html, AppDataType.HTML)' that
automatically does HTML sanitization would improve the usability of 
the API while still leaving it safe by default?

 Shouldn't the escaping be done when getting the data, not when setting it, for it to be secure.

Anyway, I think it's dangerous to make assumptions on how gadgets are typically going to use the appdata. Imagine a digg-like application that stores urls, should we then check that the url doesn't start with "javascript:"?

I think most single developers expect any kind of storage to behave in the way that what you put in is what you get out. I wouldn't like a database that automatically escapes data, I wouldn't expect that if I wrote data to a file and read it later it would be escaped in any way. Any developer is (or should be) always concerned on where the data came from, and to what respect it can be trusted.

I'm just wondering, how would a hello world application be more complicated if it used element.safeSetInnerHtml() ?

Brian Eaton

unread,
Apr 16, 2008, 10:51:24 AM4/16/08
to opensocial-an...@googlegroups.com
On Wed, Apr 16, 2008 at 2:26 AM, Reinoud Elhorst <rei...@hyves.nl> wrote:
> Shouldn't the escaping be done when getting the data, not when setting it,
> for it to be secure.

Yeah, that's probably a good idea. Cleaning usually belongs where
data is used, not where it is stored.

> I think most single developers expect any kind of storage to behave in the
> way that what you put in is what you get out.

I really wish app data worked that way, but it doesn't. No matter
what a gadget puts in to app data, an attacker can modify the data
that people will see when they view the attackers page. Developers
who assume that the app data they read is the same as what they wrote
are going to have serious vulnerabilities in their gadgets.

"Serious vulnerability" is a very subjective term, so let me explain
the risk: app data can be used to spread XSS worms. If an attacker
discovers an XSS vulnerability that can be exploited through app data,
they will do so in such a way that

a) the attack payload spreads rapidly to all users of the app (The
Samy worm spread to 1 million users in under a day.)
b) the payload includes browser exploits that attempt to install
malware. (These attacks are very popular, because they can be
monetized.)

Can folks have a look at how the gadgets they are running are actually
using app data, and what portion of gadget developers are assuming
that the content they wrote to app data is the same as they are
getting back?

Cheers,
Brian

Arne Roomann-Kurrik

unread,
Apr 16, 2008, 12:34:01 PM4/16/08
to opensocial-an...@googlegroups.com

Since automatic escaping was introduced on orkut, I don't believe that any application developer on that platform (myself included) now expects to get back exactly what they put in.



Cheers,
Brian


Reinoud Elhorst

unread,
Apr 17, 2008, 5:04:37 AM4/17/08
to opensocial-an...@googlegroups.com
I think it makes sense to get this thread somewhat on track again. I propose that we use this thread to discuss how to change the appstore (whether or not to allow JSON; whether we need to specify it as type of data, or just allow inserting it) and a separate thread on whether or not to have auto escaping ( http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/974695cf0d0140ff?hl=en )

Quote Cassie's proposals:
==============

Okay, so I see a couple possible spec changes to address this issue:

1. all input comes is treated as json. all the strings that are currently in the data store are fine, as they are just strings still. escaping will be done on all the sub fields of the json. which, for all the original data means that nothing will change (as the entire string is a field). however, apps going forward can take advantage of the new type and won't need to use stringify, unescapeString and parse on the json object (html within the json will still need to be unescaped though)
pros: invisible to developer
cons: everything has to be valid json? (a regular string is valid json right?)

2. you can specify the type of data. this type will need to be stored in backend data servers. all original data will have the String type and the default will still be String. escaping will take place in the same was as #1.
pros: We could add many more complex types as time goes on. These types could be used to make auto uis (like user pref editing) or other things I can't think of.
cons: The datastore needs to know about types

3.  we leave it as it is
pros: nothing changes in code, backends or gadgets
cons: apps are still unescaping blindly

================

As I sum up, we have in favour:
1) Kevin Brown, Reinoud Elhorst, Arne Roomann-Kurrik, Zhen Wang, Kevin Marks
2) Oleg Pylnev, Kevin Brown
3) None

Cassie

unread,
Apr 20, 2008, 12:29:20 PM4/20/08
to opensocial-an...@googlegroups.com
Thanks Reinoud!
The other thread has now been added to the spreadsheet.

Oleg - you are the main supporter for the second option. Do you feel strongly for #2 over #1? If so, it would be great it you could try to convince some of the #1 supporters why #2 is better. Thanks.

- Cassie

Oleg Pylnev

unread,
Apr 20, 2008, 2:48:56 PM4/20/08
to opensocial-an...@googlegroups.com

Thanks Cassie,

 

#2 Pros:

1)      Expandability

2)      Compact serialization

3)      Potential support for default values.

 

#2 Cons:

1)      Data Store needs to be modified

2)      Container needs to be aware of data types

 

What about making the JSON the default format for option #2?

 

-Oleg

 

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Cassie
Sent: Sunday, April 20, 2008 9:29 AM
To: opensocial-an...@googlegroups.com
Subject: Re: Allow JavaScript objects to be stored in AppData

 

Thanks Reinoud!

Cassie

unread,
Apr 28, 2008, 7:41:33 AM4/28/08
to opensocial-an...@googlegroups.com
fyi: escaping of data can now be turned off with a flag to fetch person app data method.

with that in mind do we care about this proposal anymore?

if we do, then there are still two options here and we haven't agreed on what we want to do.
we can either 1. assume all input is json or 2. have developers specify the type of their data.

here are some questions I have:

a) what types would we want to support for #2?
b) if we did #2 then could we theoretically finally implement user prefs as app data? (i think the answer here is yes)
c) how hard is it myspace, hi5, and orkut to add types to your data store? how about doing a migration of data? (ie jsonifying strings)

Thanks.
- Cassie

Brian Eaton

unread,
Apr 28, 2008, 10:56:15 AM4/28/08
to opensocial-an...@googlegroups.com
On Mon, Apr 28, 2008 at 4:41 AM, Cassie <do...@google.com> wrote:
> fyi: escaping of data can now be turned off with a flag to fetch person app
> data method.
>
> with that in mind do we care about this proposal anymore?

I still care. Arne originally wrote:

"When returning an object, the container could then iterate over each
key in the parsed result and escape string values appropriately. The
result would be a native JS object with escaped fields, which would
not need to be unescaped to be usable in my application."

That's a good idea, I'd like to see it happen.

Cheers,
Brian

Arne Roomann-Kurrik (Google)

unread,
Apr 28, 2008, 1:05:43 PM4/28/08
to OpenSocial and Gadgets Specification Discussion
I still care, too... the whole json.stringify/json.parse cycle is
annoying and something that could be taken care of by the container.
I'm still in favor of treating all appData as JSON-encoded, but if the
difference is dumping this functionality or changing the Persistence
API to take a data type, then I'd be in favor of using the data types.

Incorporating the escape toggle parameter from the other thread, I'd
like to see functionality where I can store a JavaScript object using
an "object" data type (if needed) and then pull out an object with
escaped fields (unless I set the escaping parameter to false).

~Arne

On Apr 28, 7:56 am, "Brian Eaton" <bea...@google.com> wrote:

Cassie

unread,
Apr 30, 2008, 2:15:55 PM4/30/08
to opensocial-an...@googlegroups.com
Okay, so the opinions now stand like this:


1) Reinoud Elhorst, Zhen Wang, Kevin Marks
2) Oleg Pylnev
3) None 

4) (Either 1 or 2) Arne, Brian Eaton, Kevin Brown


So, Reinoud, Zhen and Kevin Marks either all need to go for door #2. Oleg needs to come over to door #1 or some other compromise needs to be reached.
Thanks.

- Cassie

Oleg Pylnev

unread,
Apr 30, 2008, 5:02:08 PM4/30/08
to opensocial-an...@googlegroups.com

Another point for having specific data type is the potential expansion of data store:

Allow to sort friends app data for a specific key. Imagine if developer wants to execute following query: ‘select top 10 game scores from my friends’. Having specific data type will allow proper sorting.

 

I do see the need for a compromise and if none reached I would be fine with door #1.

 

-Oleg

 

 

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Cassie
Sent: Wednesday, April 30, 2008 11:16 AM
To: opensocial-an...@googlegroups.com
Subject: Re: Allow JavaScript objects to be stored in AppData

 

Okay, so the opinions now stand like this:

Louis Ryan

unread,
Apr 30, 2008, 6:52:55 PM4/30/08
to opensocial-an...@googlegroups.com
+1 for #1 with the autoescape toggle option.

Cassie

unread,
May 1, 2008, 6:29:01 AM5/1/08
to opensocial-an...@googlegroups.com
Okay, so based on everybody's thoughts here is what will happen:

1. For 0.8, we change the default type for all app data from String to Json.
2. For 0.9, we can discuss the pros and cons of having explicit types that developers can use. Thankfully, the 0.8 change doesn't prevent this from happening.. it just changes the default type.

Doing things in this order seems like it satisfies all the requirements of this thread.
Thanks.

- Cassie
Reply all
Reply to author
Forward
0 new messages