attached objects, and Google Buzz

[Will Norris]

I'm spending a considerable portion of my time right now working on
the Activity Streams output for Google Buzz.  There have been a couple
places where concepts in Buzz haven't mapped into the AS format as
cleanly as I would have liked, and we've been debating the best way to
handle this.  I'd love to get others thoughts on this... whether or
not it proves to be a general enough problem to justify inclusion in
the spec, but at least to make sure we're not doing anything TOO
gross.

So I'll just assume people are familiar with how Buzz works... you
have a free-form message, and are able to attach various kinds of
media to your post, such as photos, photo albums, videos, links, etc.
The first challenge was in identifying the actual activity.  Given a
short Buzz post with an attached image, that could be interpreted two
ways:

 - Joe posted a note, with an attached image
 - Joe shared an image, with an attached annotation[0]

This is trickier when you have multiple attached images or videos,
since activities are defined as containing only a single object.  For
right now, we're leaning toward the first option, since it avoids the
multiple object problem.  But then we still have the problem of how to
identify these attached media objects.  Of course, if all you care
about is linking to it, you can simply use:

  <link rel="related" type="image/jpeg" href="..." />

But we are wanting to be a bit more expressive, along the lines of the
various activity object types.  So the best thing I could come up with
is to express the note as the object of the activity, and to identify
the attachments in a custom element, using the same schema as an
activity object.  So you would end up with:

<entry>
<author> ... </author>
<activity:verb> .../post </activity:verb>
<activity:object>
<activity:object-type> .../note </activity:object-type>
...
</activity:object>
<link rel="related" type="image/jpeg" href="foo.jpg" />
<buzz:attachment>
<activity:object-type> .../photo </activity:object-type>
<title>My Photo</title>
<link rel="alternate" type="image/jpeg" href="foo.jpg" />
<link rel="preview" type="image/jpeg" href="foo-thumbnail.jpg" />
</buzz:attachment>
</entry>

So here you have the attached image represented twice. First as a
simple <link> for those consumers that understand Atom (and maybe
Activity Streams). And second, as a buzz:attachment, with all the
richness of an activity:object, such as title and thumbnail image. An
alternate approach would be to include the photo object inside the
note object somehow, but that starts to lead down a slippery slope
that I'm not sure we want to venture toward.

So I guess my first question is, does anyone else have similar use
cases, where you want to provide richer metadata about an object, but
that object is neither the formal object or target of the given
activity? If so, how are you expressing that in Activity Streams, if
at all? Secondly, how does the use of buzz:attachment in the example
above strike people? Should this kind of use case be handled with
custom elements, or do we think there should be a common way to
express this without needing custom elements?

thanks for any thoughts, and would love to discuss this this weekend
if time permits.

-will

[0]: http://wiki.activitystrea.ms/Annotations

[Martin Atkins]

Will,

In TypePad we just do this by embedding the images in the atom:content,
which means that they get rendered in the correct places inside the
entry but it does mean you need to start scraping the HTML if you want
to pull the images out.

In the TypePad API, which started off looking roughly like a JSON
serialization of activity streams but has deviated over time, we
invented a specialized "image link" construct to deal with the problem
of linking to multiple images while providing multiple representations
of each. This object type is defined here:

http://www.typepad.com/services/apidocs/objecttypes/ImageLink

Now obviously the stuff about URL templates is TypePad-specific, but the
notion of including both the original image URL and the URL of its
preview together is the important thing.

Here's one way you could translate this into Atom:

<link rel="enclosure" href="..." type="image/jpeg"
media:width="..." media:height="...">
<link rel="variant" href="" type="image/jpeg"
media:width="..." media:height="..."
/>
</link>

This remains compatible with the basic Atom enclosures mechanism while
extending it with multiple variants of the same enclosure with different
characteristics. Here I used the AtomMedia extensions to describe the
various sizes available.

Of course, this doesn't represent the image as a full-fidelity Activity
Streams object. For the TypePad API we decided that providing the
information necessary to render the image was most important and
therefore focused on that for the inline representation. However, we do
offer a separate API endpoint where you can get the full object
representations for the images; I'm not sure how you'd translate *that*
into Activity Streams, where generally we've always aspired to stuff
everything into the feed to avoid additional round-trips.

[Charlie Cauthen]

We have the same issue at cliqset.  We try to attach as much data to the entry as possible, in the form of multiple objects.  We note the primary object/verb as a category on the entry.  So, if there is a "blog posted" activity, it could have a weblog entry object, an image object (if there was an image in the blog entry), and a category like this:

<category scheme="http://schemas.cliqset.com/activity/categories/1.0" term="WeblogEntryPosted" label="WeblogEntry Posted"></category>

It is not so tough to determine and specify the primary action when we are consuming a feed from a specific type of provider, but a lot harder when entries could come in via salmon from potentially anyone.

I would be in favor of including something in the spec to signify the primary object or intent of the user.  Looking forward to discussing this this weekend.

Charlie

--
You received this message because you are subscribed to the Google Groups "Activity Streams" group.
To post to this group, send email to activity...@googlegroups.com.
To unsubscribe from this group, send email to activity-strea...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/activity-streams?hl=en.

[Will Norris]

The idea of multiple objects was also mentioned here, but I'm not sure if that is consistent with how the spec is written today.  I know we've gone back and forth a little on this, so I'll let Mart or Panzer correct me, but at one time a single atom:entry with multiple objects was simply shorthand for multiple activities.  So if you had:

atom:entry

 - atom:author = Joe

 - activity:verb = post

 - activity:object = { weblog }

 - activity:object = { photo }

then you're actually saying two things:

 - Joe posted a weblog entry

 - Joe posted a photo

There is technically no correlation between the weblog entry and the photo, except for the fact that they just happened to be coalesced into a single entry.

-will

[Charlie Cauthen]

Yeah, I think you are right.  This implementation was done at least 6 months ago, I do believe the semantics have changed since then.   

My point is that the problem you pointed out is something we experience too.  Not suggesting the official solution should be what we hacked together, just that we should have a solution that is "official".

I would prefer the solution to be machine friendly, so parsing of markup isn't required.

[Martin Atkins]

On 04/14/2010 05:01 PM, Will Norris wrote:
> The idea of multiple objects was also mentioned here, but I'm not sure
> if that is consistent with how the spec is written today. I know we've
> gone back and forth a little on this, so I'll let Mart or Panzer correct
> me, but at one time a single atom:entry with multiple objects was simply
> shorthand for multiple activities. So if you had:
>
> atom:entry
> - atom:author = Joe
> - activity:verb = post
> - activity:object = { weblog }
> - activity:object = { photo }
>
> then you're actually saying two things:
> - Joe posted a weblog entry
> - Joe posted a photo
>
> There is technically no correlation between the weblog entry and the
> photo, except for the fact that they just happened to be coalesced into
> a single entry.
>

Yes, that is what the specification says today.

Even if it did not, it seems like your use-case is about objects
embedded inside other objects, so it would probably be more natural to
have them nested inside one another at the XML level.

[Chris Messina]

I've gone ahead and documented my impression of this issue, as a reaction to the conversation I had with Will about this, and after doing some basic research:

http://wiki.activitystrea.ms/Attachments

I'd be interested in feedback on this, and see if we can drive this to consensus.

Chris

--

You received this message because you are subscribed to the Google Groups "Activity Streams" group.
To post to this group, send email to activity...@googlegroups.com.
To unsubscribe from this group, send email to activity-strea...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/activity-streams?hl=en.

--
Chris Messina
Open Web Advocate, Google

Personal: http://factoryjoe.com
Follow me on Buzz: http://buzz.google.com/chrismessina
...or Twitter: http://twitter.com/chrismessina

This email is:   [ ] shareable    [X] ask first   [ ] private

[Rob Dolin]

Thanks for putting this together Chris.  I like this direction of using child elements of <link rel=”related” …/> for the attachement(s.)

 

I’ve added the example + screenshots of MySpace’s UX for attaching a photo, link, or video to a status post.

 

Thanks again—

--Rob

[John Panzer]

It looks good to me. Any thoughts on how to tie attachments back to
embedded references in HTML content or vice versa? Thinking of
editing use cases in future, but also hovercards on content etc.

Same problem pops up for embedded @mentions in OStatus/Salmon. Would
like to attach explicit attention/mention metadata but also tie back
to relevant bits of content.

Like Cid: cross references in MIME.

Not blocking for 1.0 of course.

On Sunday, April 18, 2010, Rob Dolin <robd...@microsoft.com> wrote:
> Thanks for putting this together Chris.  I like this direction of using child elements of <link rel=”related” …/> for the attachement(s.) I’ve added the example + screenshots of MySpace’s UX for attaching a photo, link, or video to a status post. Thanks again—--Rob From: activity...@googlegroups.com [mailto:activity...@googlegroups.com] On Behalf Of Chris Messina
> Sent: Saturday, April 17, 2010 4:38 PM
> To: activity...@googlegroups.com

> Subject: Re: attached objects, and Google Buzz I've gone ahead and documented my impression of this issue, as a reaction to the conversation I had with Will about this, and after doing some basic research: http://wiki.activitystrea.ms/Attachments I'd be interested in feedback on this, and see if we can drive this to consensus. ChrisOn Wed, Apr 14, 2010 at 5:58 PM, Martin Atkins <ma...@degeneration.co.uk> wrote:On 04/14/2010 05:01 PM, Will Norris wrote:The idea of multiple objects was also mentioned here, but I'm not sure

--
--
John Panzer / Google
jpa...@google.com / abstractioneer.org / @jpanzer

[Chris Messina]

Well, there's a slight distinction in the use case... while you might use rel-enclosure for content embedded in prose, the purpose of rel-related with child elements is simply to provide additional payload elements, which might not actually be semantically or meaningfully related to the content of the post.

For example in Buzz, you can share a link, and in doing so, choose a series of (up to four) images to include with your post. These images just happen to be on the page — but may not actually be altogether related to the post.

Similarly, I might want to post a zip file publicly, and include a message "hey guys, here're my latest mockups". While the ideal activity would be "Chris posted some photos" or "Chris uploaded a file", in the case of some service, that kind of clarify of intention is impossible to detect.

The best thing we can do is use an activity like "Chris posted a note" ... "with an attachment."

I think this actually works pretty well and covers a large number of cases that we might not otherwise be able to cover.

Chris

[Martin Atkins]

Where we ended up today:

For the purpose of marking up "attachments" you can use a link of either
rel="related" or rel="enclosure" with extension elements inside its
content which follow the same content model as an activity streams
object construct serialized as an activity:object element.

The goal of allowing both rel="related" and rel="enclosure" is to allow
publishers to decide between two different behaviors for
non-activity-aware consumers. If you use rel="related" then existing
consumers, if they do anything at all, are more likely to link to the
target. If you use rel="enclosure" and an appropriate MIME type,
consumers are more likely to render the indicated resource alongside the
content or download it for the user to consume separately. Activity
Streams consumers don't care which of these you use... publishers can
just do whatever achieves their desired behavior in non-activity-aware
clients.

We also adapted the in-reply-to element from Atom Threading Extensions
in the same way: you can nest inside of it elements describing an
activity streams object that the object is a response to. The concept of
something being in response to something else will now be defined as a
core property of all objects regardless of type.

I believe the rationale behind these decisions has already or will
ultimately be documented on some wiki page.

On 04/18/2010 11:35 AM, Chris Messina wrote:
> Well, there's a slight distinction in the use case... while you might
> use rel-enclosure for content embedded in prose, the purpose of
> rel-related with child elements is simply to provide additional payload
> elements, which might not actually be semantically or meaningfully
> related to the content of the post.
>
> For example in Buzz, you can share a link, and in doing so, choose a
> series of (up to four) images to include with your post. These images
> just happen to be on the page — but may not actually be altogether
> related to the post.
>
> Similarly, I might want to post a zip file publicly, and include a
> message "hey guys, here're my latest mockups". While the ideal activity
> would be "Chris posted some photos" or "Chris uploaded a file", in the
> case of some service, that kind of clarify of intention is impossible to
> detect.
>
> The best thing we can do is use an activity like "Chris posted a note"
> ... "with an attachment."
>
> I think this actually works pretty well and covers a large number of
> cases that we might not otherwise be able to cover.
>

[Chris Toomey]

This looks great and is an important addition that I'd been meaning to
ask about for a while. So this is going into the 1.0 spec., correct?

thx,
Chris

On Apr 18, 7:36 pm, Martin Atkins <m...@degeneration.co.uk> wrote:
> Where we ended up today:
>
> For the purpose of marking up "attachments" you can use a link of either
> rel="related" or rel="enclosure" with extension elements inside its
> content which follow the same content model as an activity streams
> object construct serialized as an activity:object element.
>
> The goal of allowing both rel="related" and rel="enclosure" is to allow
> publishers to decide between two different behaviors for
> non-activity-aware consumers. If you use rel="related" then existing
> consumers, if they do anything at all, are more likely to link to the
> target. If you use rel="enclosure" and an appropriate MIME type,
> consumers are more likely to render the indicated resource alongside the
> content or download it for the user to consume separately. Activity
> Streams consumers don't care which of these you use... publishers can
> just do whatever achieves their desired behavior in non-activity-aware
> clients.
>
> We also adapted the in-reply-to element from Atom Threading Extensions
> in the same way: you can nest inside of it elements describing an
> activity streams object that the object is a response to. The concept of
> something being in response to something else will now be defined as a
> core property of all objects regardless of type.
>
> I believe the rationale behind these decisions has already or will
> ultimately be documented on some wiki page.
>

[Monica Keller]

That is what we agreed to in the meetup. Its an important use case for
most services which have a user stream
Does anyone have objections ?