Proposal: Messaging changes

[bobby]

Hello all,

This is a proposal for some messaging changes that I've come up with
while working on our own container at Project SocialSite (I'm also
robert....@sun.com on the shind...@incubator.apache.org list).
There are three changes I'd like to suggest, and as background I refer
you to Ropu's writeup:
http://docs.google.com/View?docid=dfjz793x_187gm3prnch

Maybe these should be broken up into separate threads, but we can do
that if there is enough conversation on any of them to warrant it.

1. Changes to the OpenSocial unique ID format. This is currently the
container domain followed by a user's ID, for instance:
"example.com:abc123" This works well enough for people, but in
SocialSite we can send messages to groups (our definition of group
that is a collection of people with its own group profile page) as
well. In the future, there could be more types of recipients than just
these two.

I don't have a specific recommendation, but I'd like to see this made
more general. One idea is to change the format from "domain:id" to
"domain:optional_type:id" (or put the type first, so it looks more
like a URI scheme). Because colons are not allowed in usernames, there
shouldn't be any ambiguity in the string. Thus, a recipient could be
"person:example.org:abc123." Another idea is to simply use URIs as
identifiers -- since every person has a unique profile URL in the
OpenSocial API, this is an option. (In our project, a user and a group
could have the same ID, but they have separate URLs for their profile
pages.)

2. A sender field in the message. The current proposal, along with
some proposed standard methods, allow for sending messages to a
recipient. We have services for retrieving these messages as well, and
so we need to get the sender information somehow to present to the
user. While this could simply be an ID, I'd like to see more
information in the message, for example including a Person object in
the Message. That would allow showing a nice UI to the end user with
the sender's full name and a link to his/her profile without having to
parse the messages, grab the IDs, and make another request back to the
server for the senders' full info. Of course, the sender isn't needed
when sending since that information is already available.

I'm currently implementing this by adding a JSONObject sender field to
my message impl class, and it's passed over the wire by Shindig
through inspecting and calling all public getters. However, I'm
processing the JSON manually on the client end, and it would be nice
to have a jsonmessage.js object on the client to handle this (I sent a
beginning version of jsonmessage.js in a proposed patch as dicussed on
shindig-dev, but it doesn't include all of these ideas).

3. A 'data' (or 'payload,' 'attachment,' etc.) field in a message for
gadget-specific information. As an example, a request to join a group
in SocialSite needs to contain information about the group that the
sender is asking to join. While in this case there is no body of the
message and so the group info could be placed here, we could in the
future allow personal messages within a group request ("please let me
in! I'll bring pizza!").

Also, information in the body of the message is a simple string, so
the gadget displaying the message would have to parse this string for
data. If the data field were JSON instead, then this skips a step
while leaving the 'body' field open for human-readable information.
I've implemented this in our project in the same way I've added a
'sender' field -- by simply providing the info in a public getter
method and parsing the JSON in the client.

Cheers,
Bobby

[anand sharma]

Hi Bobby
Nice documentation on Shindig message service,
Actually i m trying to implement message service class and not getting idea from where i should start, so would u please help me so that i can able to write message service class .


Thx
Anand
PHP | PERL | Linux
anandkuma...@gmail.com

[bobby]

> Actually i m trying to implement message service class and not getting idea
> from where i should start, so would u please help me so that i can able to
> write message service class .

Rodrigo Gallardo has started working on the message service in
Shindig, so
that code might be a good place to look:

http://codereview.appspot.com/5702

I saw this on the shind...@incubator.apache.org email list, so that
might
also be a good place to ask. I'm implementing it in our own container
for
Project SocialSite, which you're welcome to browse, but the code is
pretty
specific to our container except for where it plugs into the Shindig
service.
You can see my code here if you'd like: http://tinyurl.com/46yszc

Cheers,
Bobby

[Scott Seely]

We have 1 +1 (submitter). Any other votes/comments?

On Oct 13, 9:53 am, bobby <bbiss...@gmail.com> wrote:
> > Actually i m trying to implement message service class and not getting idea
> > from where i should start, so would u please help me so that i can able to
> > write message service class .
>
> Rodrigo Gallardo has started working on the message service in
> Shindig, so
> that code might be a good place to look:
>
> http://codereview.appspot.com/5702
>

> I saw this on the shindig-...@incubator.apache.org email list, so that

[Ropu]

+1

but im also a submitter,

--
.-. --- .--. ..-
R  o  p  u

[Scott Seely]

Does anyone have anything extra to add to this discussion? It seems
like no one is against the idea but that the original proponent has
dropped off the thread.

I will call the thread dead in Wednesday (11-5) if we don't see some
more discussion.

[bobby]

> Does anyone have anything extra to add to this discussion? It seems
> like no one is against the idea but that the original proponent has
> dropped off the thread.

Hi,

Didn't mean to give the impression that I had dropped off the thread.
I thought your "1 +1 (submitter)" covered me. Am still in favor of the
proposal.

Cheers,
Bobby

[snoopdave]

+1 for getting these changes into OpenSocial v0.9.

- Dave

[Paul Lindner]

I went through and merged the two proposals here:

http://wiki.opensocial.org/index.php?title=Messaging_API_Changes

Comments?

On Nov 8, 6:55 am, snoopdave <snoopd...@gmail.com> wrote:
> +1 for getting thesechangesinto OpenSocial v0.9.

[Louis Ryan]

Im a general +1 on this.

Some notes:

- Message does not contain a  msg collection id, is this intentional as it seems useful
- In the examples the message ids are URLs not URNs?
- Is anyone actually using body_id and title_id ? Maybe we can just drop these
- What is the body of the response to POST /messages/{guid}/@outbox when item was created? The entity itself?

[bobby]

> I went through and merged the two proposals here:
>
> http://wiki.opensocial.org/index.php?title=Messaging_API_Changes
>
> Comments?

Hi Paul,

Can we keep the original URIs that include inbox/outbox as we had
before?

--- begin ---
Recipients can request messages by requesting them from their 'inbox'
collection for received messages or 'outbox' collection for sent
messages. If {msgid} is omitted, all messages are returned.

GET /messages/{guid}/inbox/{msgid}
GET /messages/{guid}/outbox/{msgid}

Recipients can remove messages by sending a DELETE request to either
the inbox or outbox. A message may also be updated with a PUT request
(i.e., marking the messages as 'read'). In a DELETE or PUT request,
{msgid} is required.
--- end ---

If you'd prefer not using "inbox" and "outbox," then I'd like to see
some other comments at least.

The status strings that we had used before were "READ" and "UNREAD" --
do you think those should be standardized? I can see a potential
problem with something like "SPAM" since a message could be both read
and spam at the same time. We could have I guess any number of status
fields (e.g., one specifically for read/unread), but I don't want to
propose making too big a change now.

Thanks,
Bobby

[Paul Lindner]

On Nov 12, 2008, at 11:28 AM, bobby wrote:

>
>> I went through and merged the two proposals here:
>>
>> http://wiki.opensocial.org/index.php?title=Messaging_API_Changes
>>
>> Comments?
>
>
> Hi Paul,
>
> Can we keep the original URIs that include inbox/outbox as we had
> before?

inbox semantics are available as @all
outbox is available as @outbox

This conforms to the existing opensocial norms that designate special
collection names with an @ prefix.

> --- begin ---
> Recipients can request messages by requesting them from their 'inbox'
> collection for received messages or 'outbox' collection for sent
> messages. If {msgid} is omitted, all messages are returned.
>
> GET /messages/{guid}/inbox/{msgid}
> GET /messages/{guid}/outbox/{msgid}
>
> Recipients can remove messages by sending a DELETE request to either
> the inbox or outbox. A message may also be updated with a PUT request
> (i.e., marking the messages as 'read'). In a DELETE or PUT request,
> {msgid} is required.
> --- end ---
>
> If you'd prefer not using "inbox" and "outbox," then I'd like to see
> some other comments at least.

I got rid of that syntax once I added support for generic message
collections. I'm willing to be flexible here, I just want to insure
that opensocial required collections are distinct from container
optional collections.

> The status strings that we had used before were "READ" and "UNREAD" --
> do you think those should be standardized? I can see a potential
> problem with something like "SPAM" since a message could be both read
> and spam at the same time. We could have I guess any number of status
> fields (e.g., one specifically for read/unread), but I don't want to
> propose making too big a change now.

Correct this is a messy area.

Long term goal would be to use atompub style categorization to use
defined terms tagged to each message.

http://atompub.org/rfc4287.html#element.category

For now I'm happy to stick with READ/UNREAD/DELETED and ignore the
spam aspect.

> Thanks,
> Bobby
>
> >

Paul Lindner
plin...@hi5.com

[Paul Lindner]

On Nov 12, 2008, at 9:04 AM, Louis Ryan wrote:

Im a general +1 on this.

Some notes:

- Message does not contain a  msg collection id, is this intentional as it seems useful

Sounds fine.  It would need to be an array, since messages can live in multiple collections.

- In the examples the message ids are URLs not URNs?

I'll go over this section.  I actually have the code running in the sample container, so it's getting easier to get samples.

- Is anyone actually using body_id and title_id ? Maybe we can just drop these

We use them to provide internationalized and templatized messages.  In my opinion these are only of interest for message submission.

- What is the body of the response to POST /messages/{guid}/@outbox when item was created? The entity itself?

Right now it's just a simple 201 created response, I believe.  Since an outbox can have multiple recipients there's no sense in returning what you posted.  The IDs are going to be different for each user.

On Mon, Nov 10, 2008 at 1:16 PM, Paul Lindner <plin...@hi5.com> wrote:

I went through and merged the two proposals here:

http://wiki.opensocial.org/index.php?title=Messaging_API_Changes

Comments?

On Nov 8, 6:55 am, snoopdave <snoopd...@gmail.com> wrote:
> +1 for getting thesechangesinto OpenSocial v0.9.

>
> - Dave
>
> On Nov 3, 3:32 pm, bobby <bbiss...@gmail.com> wrote:
>
>
>
> > > Does anyone have anything extra to add to this discussion? It seems
> > > like no one is against the idea but that the original proponent has
> > > dropped off the thread.
>
> > Hi,
>
> > Didn't mean to give the impression that I had dropped off the thread.
> > I thought your "1 +1 (submitter)" covered me. Am still in favor of the
> > proposal.
>
> > Cheers,
> > Bobby

Paul Lindner

plin...@hi5.com

[bobby]

> > Can we keep the original URIs that include inbox/outbox as we had
> > before?
>
> inbox semantics are available as @all
> outbox is available as @outbox
>
> This conforms to the existing opensocial norms that designate special  
> collection names with an @ prefix.

That sounds good, and we're fine with the @xyz semantics. One minor
thing: could we change "@all" to "@inbox" or something else? I don't
want someone to be confused that "all" should return, well, all
messages, including sent ones. I think I see what you're going for: in
my email reader, I have only one "box" for sent messages and many
"boxes" for all my incoming stuff (they could be boxes, folders,
etc.). I'm just wary of the connotations of "all."

> > The status strings that we had used before were "READ" and "UNREAD" --
> > do you think those should be standardized? I can see a potential
> > problem with something like "SPAM" since a message could be both read
> > and spam at the same time. We could have I guess any number of status
> > fields (e.g., one specifically for read/unread), but I don't want to
> > propose making too big a change now.
>
> Correct this is a messy area.
>
> Long term goal would be to use atompub style categorization to use  
> defined terms tagged to each message.
>
> http://atompub.org/rfc4287.html#element.category
>
> For now I'm happy to stick with READ/UNREAD/DELETED and ignore the  
> spam aspect.

That sounds good to me.

Cheers,
Bobby

[Ropu]

- What is the body of the response to POST /messages/{guid}/@outbox when item was created? The entity itself?

Right now it's just a simple 201 created response, I believe.  Since an outbox can have multiple recipients there's no sense in returning what you posted.  The IDs are going to be different for each user.

its possible to return an array that states the result of each message?

since when sending it to multiple recipients, not necessary all will success or fail

so someting like this

{
  {"id1" : "Success"},
  {"id2" : "Fail"},
...
  {"idN" : "Success"}
}


besides that, im +1

[Scott Seely]

Several edits have been made to the thread. By my count, the only post
mod votes are from Paul Lindner (he's editing) and Ropu.

+2

(please correct me if I made a mistake)

[Ropu]

bobby?

[bobby]

> bobby?

I'd like to change @all to @inbox (or anything else that couldn't be
mistaken for implying "every message"), but if others like the "all"
then it's just a string and doesn't materially change the proposal.

So I'm +1

Cheers,
Bobby

[Paul Lindner]

Okay, @inbox it is.

Suggest wording is:

@inbox contains all messages inbound to user
@outbox contains all messages sent by the user

Posting to @outbox appends/sends new messages.

Spec change updated here:

http://wiki.opensocial.org/index.php?title=Messaging_API_Changes

Paul Lindner
plin...@hi5.com

[Paul Lindner]

This is a difficult issue depending on how pure you want your REST
semantics.

Submission through an @outbox endpoint results in multiple entities
being created, which means that you can't return the location of the
newly created entity in a Location: header or as a the actual
representation of the item in the post response.

Also some systems might queue up requests and immediately return a
success code, which complicates matters.

I think we need to give some thought to how this might work, because
I'd like to see the same response format used for activity creation as
well.

So for 0.9 I'd propose simple 201/403 responses for POST requests, and
try to come up with a standardized way to respond to write requests.

Paul Lindner
plin...@hi5.com

[Paul Lindner]

BTW, specification on the wiki is updated for @inbox/@outbox naming..

Paul Lindner
plin...@hi5.com

[Scott Seely]

Another option would be to return the URL for the sent message in
/messages/{guid}/[moral equivalent of sent items]

And each failure would come back to the sender as separate messages.

This is how e-mail systems work today. Is there a reason this doesn't
work for OpenSocial?

[Paul Lindner]

I think we should spend our time working out an extensible way to
return error responses. The OpenSocial spec covers collections in
exacting detail, but relegates error responses to section 13 HTTP
response codes...

The JSON-RPC protocol is much better in this respect as we can attach
error metadata with the response. Something like this:

"error" : {
"code" : "409",
"message" : "Delivery Error",
"data" : {
"deliveryStatus" : { "id1" : "200", "id2": "403", "id3":
"500"}
}
}

is fairly straightforward. A similar response format for REST would
be very useful for solving these issues.

So let's get our error responses sorted out, then add the appropriate
error response formats for each request type.

Paul Lindner
plin...@hi5.com

[Scott Seely]

In the case of sending a message, it will be sufficient to queue the
message to the e-mail system (or equivalent) and then return. I don't
think we should assume that containers will block on sending a message
to a large group.

Consider the case of sending a message to a group of 250 users. To make
sure that the site performs well, the messaging system will submit the
request to send to that 250 messages and then return a response. The
requests will then be processed through some mechanism that accepts a
throttle so that burstiness doesn't cause the system to fall apart. In
this case, returning status on all 250 messages isn't just hard, it's
impossible (ok, it is at least untenable). Delivery error would have to
be sent to the user later, probably as a message from the system to the
user about the errors.

That said, v.NEXT should get some treatment on error handling. Paul-- do
you want to kick off that thread?

[Paul Lindner]

I certainly can get something going for error handling for v.NEXT.

I'd like to get some feedback for some potential small tweaks to the
Mesaging API. At hi5 we've put the draft protocol into production and
are working with one partner who is developing against it.

Here's the smallish set of changes:

* Add new messaging types -- FRIEND_REQUEST, MEDIA_COMMENT
* Add support for media comments at the message collection and message
level.
** For message collections and messages allow MEDIAITEM fields as a
singular ID that links with the Media/Albums proposal.
** Also recommend that url type 'photo' is a link to an originating
photo, and type 'thumbnail' a link to the thumbnail.

* Add descriptive text stating that:
-- While each container can support any number of message
collections each should support retrieving specific types of messages
from the @inbox and @outbox message collections. This will allow
application authors to support the differing message types in a
standard way. To get specific messages filtered by type use the
collection options as follows:

filterBy=type filterOp=equals filterValue={publicMessage|
privateMessage|notification|...}

* Question -- should we modify the JS API for newMessage(BODY,
OPT_PARAMS) to newMessage(PARAMS) ?

* Question -- right now we have a standard urls parameter for People
that I've adopted for messages and message collections. This maps
quite cleanly to Atom <link rel="xxx" href="yy"/> It's missing the
type attribute, which could specify the mime type. For message
collections we want to support a the equivalent of

rel="alternate" type="application/vnd.xhtml+xml" href="url..."



FYI -- If anyone wants to see this API in action you can check out the
following unauthenticated URLs for hi5's CTO:

List of message collections:

http://api.hi5.com/social/rest/messages/1000?format=xml

PhotoComments -- aggregate, with a count limit

http://api.hi5.com/social/rest/messages/1000/photoComments?count=2&format=xml

Profile Comments:

http://api.hi5.com/social/rest/messages/1000/profile?format=xml

Profile Comments, specific Ids:

http://api.hi5.com/social/rest/messages/1000/profile/3700597152,3423129381?format=xml

Profile Comments, with updatedSince parameter:

http://api.hi5.com/social/rest/messages/1000/profile?format=xml&updatedSince=2008-10-15T10:23:56.292-07:00

If you grab a security token from Firebug you can add that to the URL
and request private messages like this:

http://api.hi5.com/social/rest/messages/XXXXX/inbox?st=YYYYYYYY

where XXXXX is the hi5 uid and YYYYYY is the security token

We also support notification, and a bulk inbox. Soon we'll have
friend requests and other features in this test bed.

Would appreciate further comments and +1s if people think this is the
correct way to go.

Paul Lindner
plin...@hi5.com

[Louis Ryan]

Seems pretty reasonable. One question about message types vs. groups

On Tue, Dec 2, 2008 at 12:54 PM, Paul Lindner <plin...@hi5.com> wrote:

I certainly can get something going for error handling for v.NEXT.

I'd like to get some feedback for some potential small tweaks to the
Mesaging API.  At hi5 we've put the draft protocol into production and
are working with one partner who is developing against it.

Here's the smallish set of changes:

* Add new messaging types -- FRIEND_REQUEST, MEDIA_COMMENT

How far do we want to standardize the set of types, Orkut for example would like to expose in addition to the two you listed birthday notifications. It seems like we need to choose between using messageType and group to associate type information. E.g.

/messages/{guid}/@birthday    vs  /messages/{guid}/@all?filterBy=messageType&filterOp=equals&filterValue=@birthday.

One advantage of using groups is it makes it simpler to create a consolidated list of different notifications. E.g

/messages/{guid}/@all?filterBy=messageType&filterOp=equals&filterValue=@notification

 

* Add support for media comments at the message collection and message
level.
  ** For message collections and messages allow MEDIAITEM fields as a
singular ID that links with the Media/Albums proposal.
  ** Also recommend that url type 'photo' is a link to an originating
photo, and type 'thumbnail' a link to the thumbnail.

* Add descriptive text stating that:
   -- While each container can support any number of message
collections each should support retrieving specific types of messages
from the @inbox and @outbox message collections.  This will allow
application authors to support the differing message types in a
standard way.  To get specific messages filtered by type use the
collection options as follows:

  filterBy=type filterOp=equals filterValue={publicMessage|
privateMessage|notification|...}

* Question -- should we modify the JS API  for newMessage(BODY,
OPT_PARAMS) to newMessage(PARAMS) ?

I would be in favor of not adding them to the API to encourage folks to move to the lite JS API
 

[Paul Lindner]

I think we can support both constructs.  Each container can designate it's own list of message collection that make sense for that container.  Some containers may even allow custom user message collections (like Imap folders, for example)

However for cross-platform compatibility we will want to perform the queries as you described in your latter option and also as described in my 'descriptive text' below.

Paul Lindner

plin...@hi5.com

[Rodrigo]

+1 (I'm a submitter for the original proposal, but I don't know if I'd
been counted. Anyhow, this vote is for the proposal as it stands now.)

On Dec 2, 10:47 pm, Paul Lindner <plind...@hi5.com> wrote:
> I think we can support both constructs.  Each container can designate  
> it's own list of message collection that make sense for that  
> container.  Some containers may even allow custom user message  
> collections (like Imap folders, for example)
>
> However for cross-platform compatibility we will want to perform the  
> queries as you described in your latter option and also as described  
> in my 'descriptive text' below.
>
> On Dec 2, 2008, at 1:40 PM, Louis Ryan wrote:
>
>
>
> > Seems pretty reasonable. One question about message types vs. groups
>

> > On Tue, Dec 2, 2008 at 12:54 PM, Paul Lindner <plind...@hi5.com>  

> >  http://api.hi5.com/social/rest/messages/1000/photoComments?count=2&fo...

>
> > Profile Comments:
>
> >  http://api.hi5.com/social/rest/messages/1000/profile?format=xml
>
> > Profile Comments, specific Ids:
>

> >  http://api.hi5.com/social/rest/messages/1000/profile/3700597152,34231...

>
> > Profile Comments, with updatedSince parameter:
>

> >  http://api.hi5.com/social/rest/messages/1000/profile?format=xml&updat...

> > >>>> plind...@hi5.com
>
> > >> Paul Lindner
> > >> plind...@hi5.com
>
> > Paul Lindner
> > plind...@hi5.com
>
> Paul Lindner
> plind...@hi5.com

[Ropu]

with rodrigo we have +4?

Any one else so we can close the proposal?

ropu

[Louis Ryan]

Im +1.

I'd like to see containers put some effort into standardizing the set of groups/types for interoperability but that isn't blocking prototyping IMO.

[Paul Lindner]

Do we still need another +1 on this proposal?  I've lost count... 

Paul Lindner

plin...@hi5.com

[Scott Seely]

Given the level of change on this, I only count +3 since last edit. Please post if you have a different count.

[Ropu's iPhone]

It seems we don't

In fact is also actualized on the wiki too as accepted

Ropu

Sent from Ropu's iPhone

[Bess Ho]

+1
Be sure all containers be consistent in supporting group and type. Group is another untapped viral channel. Group tied to event and calendar, sharing updates and status.

--
Bess Ho

[bobby]

> I certainly can get something going for error handling for v.NEXT.

> [...]

> Would appreciate further comments and +1s if people think this is the  
> correct way to go.

Just to be clear, are you asking for comments for a proposal post-this-
one?

I think I count five +1 votes above, but at various points during the
discussion and things have been fluid. For the existing proposal:
http://wiki.opensocial.org/index.php?title=Messaging_API_Changes

Can we get people who at one point approved above to check in? I see:
- Bobby (myself)
- Ropu
- Paul (not explicitly, but he's an author)
- Dave
- Louis

Thanks,
Bobby

[Scott Seely]

Please see http://wiki.opensocial.org/index.php?title=Spec_Changes.

This item passed (I miscounted).

> -----Original Message-----
> From: opensocial-an...@googlegroups.com [mailto:opensocial-
> and-gadg...@googlegroups.com] On Behalf Of bobby
> Sent: Thursday, December 04, 2008 6:44 AM
> To: OpenSocial - OpenSocial and Gadgets Specification Discussion
> Subject: [opensocial-and-gadgets-spec] Re: Proposal: Messaging changes
>
>