Distributed sharing

27 views
Skip to first unread message

Les Orchard

unread,
Nov 5, 2011, 8:16:17 PM11/5/11
to shar...@googlegroups.com
So, I just posted this over on the diaspora list, but if I had one
feature request for a new hosted feed reader it would be this:

Don't limit sharing to users who just happen to be members of the same
service. Instead, accept subscriptions to feeds of shared links from
both local and off-site users. There are features of both RSS feeds and
the new JSON Activity Streams format that can help facilitate this.

For a hosted feed reader, the cool thing would be to allow "share"
subscriptions to both local users and "share" feeds from off-site. That
way, the sharing can be decentralized and even inclusive of competing
feed reader services.

Here's the quick sketch I posted over on the other list:

* Take a feed, call it a "source" feed (eg. from CNN, blogs, etc)

* Take someone's feed containing shared links, call it a "share" feed.
(eg. generated internally from local users, from a self-hosted feed
reader like Tiny Tiny RSS, from pinboard.in, etc)

* Allow subscriptions to both "source" and "share" feeds. Maybe
something in the "share" feed identifies it as such for auto-detection,
or maybe the subscription UI helps out.

* Whenever "share" feed items are found whose links match an item in a
"source" feed, those "share" feed items get collated and displayed as
comments on that "source" feed item. Think of it as a relational join:
Each "source" item gets joined on "share" items, if any.

--
l.m.o...@pobox.com
http://decafbad.com
{web,mad,computer} scientist

Alex Chaffee

unread,
Nov 7, 2011, 9:15:06 PM11/7/11
to shar...@googlegroups.com
Don't limit sharing to users who just happen to be members of the same
> service. Instead, accept subscriptions to feeds of shared links from
both local and off-site users. There are features of both RSS feeds and
the new JSON Activity Streams format that can help facilitate this.

I agree.

The first pass on lipsumarium.com and the upcoming first pass on sharebro.org will probably only support feeds hosted thereon, but delegating or referring or whatever is a no-brainer. Meaning not that it's trivial, but it's important.

I don't think your distinction between source feeds and share feeds is useful except at a semantic level -- like how a person would categorize their own feed to other people, but not technically. E.g. most blogs these days contain links to and excerpts from other blogs and news sources, and that doesn't make them any less or more sources in and of themselves.

* Whenever "share" feed items are found whose links match an item in a
> "source" feed, those "share" feed items get collated and displayed as
>  comments on that "source" feed item. Think of it as a relational join:
Each "source" item gets joined on "share" items, if any.

Now that's an interesting idea. But again, it could apply to any feed. 

By the way, I think you just described "trackbacks" which have been around in several incarnations for years.

 - A

Emmanuel Pire

unread,
Nov 7, 2011, 9:25:13 PM11/7/11
to shar...@googlegroups.com
Totally agree on the fact that source feeds and share feeds are useful at a semantic level, not technical.
A good thing to keep in mind indeed.
--
Emmanuel Pire
Web development
http://lipsumarium.com/

Les Orchard

unread,
Nov 7, 2011, 9:43:54 PM11/7/11
to shar...@googlegroups.com
On 11/7/11 9:15 PM, Alex Chaffee wrote:
>> Don't limit sharing to users who just happen to be members of the same
>> service. Instead, accept subscriptions to feeds of shared links from
>> both local and off-site users. There are features of both RSS feeds and
>> the new JSON Activity Streams format that can help facilitate this.
>
> I agree.
>
> The first pass on lipsumarium.com and the upcoming first pass on
> sharebro.org will probably only support feeds hosted thereon, but
> delegating or referring or whatever is a no-brainer. Meaning not that
> it's trivial, but it's important.

I'll probably get around to building at least a prototype of what I'm
thinking of. So, as long as sharebro.org spits out feeds of shared
items, it's still keen as far as I'm concerned.

> I don't think your distinction between source feeds and share feeds is
> useful except at a semantic level -- like how a person would categorize
> their own feed to other people, but not technically.

It's useful at a presentation level and has data model implications.

"Source" feed items are what would appear in the main list / river /
stream. They're the primary social objects.

"Share" feed items are what would appear as notes after source feed
items - ala old-school Google Reader. They're annotations on the primary
social objects.

In my scenario, the "share" feed items would rarely, if ever, appear in
the main reading stream. In fact, I'd say if you ran into a "share" item
without a corresponding "source" subscription, you'd go out and fetch
the "source" feed on the spot to get the corresponding item.

> E.g. most blogs
> these days contain links to and excerpts from other blogs and news
> sources, and that doesn't make them any less or more sources in and of
> themselves.

Sure, but some feeds are specifically for sharing. Like my pinboard.in
feeds, or the old-school Google Reader shared items, or maybe whatever
feeds sharebro.org produces.

These can be treated differently than "source" feeds and marked as
"share" feeds in subscriptions, and so presented differently.

Alternatively, where the <source> element appears in any RSS feed, you
could find the corresponding source item and do the display-as-note join
there. That would work for mixed purpose feeds - ie. blogs with both
original content and shares.

>> * Whenever "share" feed items are found whose links match an item in a
>> "source" feed, those "share" feed items get collated and displayed as
>> comments on that "source" feed item. Think of it as a relational join:
>> Each "source" item gets joined on "share" items, if any.
>
> Now that's an interesting idea. But again, it could apply to any feed.

That's the main idea. It could apply to any feed, especially given the
<source> element. But, indicating that some feeds are primarily meant
for sharing can help make implementation easier.

> By the way, I think you just described "trackbacks" which have been
> around in several incarnations for years.

It's a lot like "trackbacks". The key difference, though, is that
"trackbacks" go back to the original site. The relation between "source"
and "share" items gets joined at any reader's end - that's the
distributed part.

--
m...@lmorchard.com
http://decafbad.com
{web,mad,computer} scientist

Emmanuel Pire

unread,
Nov 7, 2011, 10:03:26 PM11/7/11
to shar...@googlegroups.com
I understand more the "source" and "share" feeds concept now.
How do comments integrate in that ?

As I saw it first, it was: source feed provides items with a unique id, as specified in RSS standards. Share service then aggregate comments in some way, all linked to a unique item from a source feed, and provide it to reader clients. In my first definition, source feeds are also public feeds (provided by sharebro.org or any RSS supplier).

My 2 cents.

Les Orchard

unread,
Nov 7, 2011, 10:27:16 PM11/7/11
to shar...@googlegroups.com
For what it's worth, I don't want to totally flood the list with this
idea. So, this'll be my last post on it for awhile :) I'll just take my
show over to decafbad.com when or if I start actually hacking.

On 11/7/11 10:03 PM, Emmanuel Pire wrote:
> I understand more the "source" and "share" feeds concept now.
> How do comments integrate in that ?

The items in share feeds are the comments or notes on source feed items.
So, you might have something like:

* Source feed
* http://rss.cnn.com/rss/cnn_topstories.rss

* Share feeds
* http://feeds.pinboard.in/rss/u:deusx/
* http://static.reallysimple.org/users/dave/linkblog.xml
* http://sharebro.org/users/example_user/rss

Now, say the CNN feed publishes up a headline like:
"Life found on Mars!"

Then, the users behind the share feeds post the URL to the CNN article
with notes:
pinboard.in - "Finally!"
reallysimple.org - "I'm still a skeptic."
sharebro.org - "I'm not surprised"

In a feed reader with distributed sharing, say newsblur.com for example,
you'd see something like:

* CNN.com: "Life found on Mars!"
* deusx: "Finally!"
* dave: "I'm still a skeptic."
* example_user: "I'm not surprised"

News and notes, distributed between 5 different sites.

> As I saw it first, it was: source feed provides items with a unique id,
> as specified in RSS standards. Share service then aggregate comments in
> some way, all linked to a unique item from a source feed, and provide it
> to reader clients. In my first definition, source feeds are also public

> feeds (provided by sharebro.org <http://sharebro.org> or any RSS supplier).
>
> My 2 cents.

Well, in what I'm thinking, the share service just publishes the feed
for individual users' shared items. A feed reader can aggregate notes
from share feeds.

But, a single site could be both a sharing service (as output) and a
feed reader (as input).

Alex Chaffee

unread,
Nov 7, 2011, 10:36:19 PM11/7/11
to shar...@googlegroups.com
On Mon, Nov 7, 2011 at 7:27 PM, Les Orchard <l.m.o...@pobox.com> wrote:
For what it's worth, I don't want to totally flood the list with this
idea. So, this'll be my last post on it for awhile :) I'll just take my
show over to decafbad.com when or if I start actually hacking.


I'm okay with the flood, especially since it's confined to this thread. And since it is a pretty cool idea :-)

Your idea is very interesting but I'm not sure it's anthropologically valid, i.e. corresponds to the way people do use or want to use existing tech. For instance, many "notes" are miles long and contain links in themselves, often several links, and to different sources. 

Slightly off topic, the difference between "comment" and "note" in Reader has always seemed artificial to me. Like, ok, maybe it makes sense to put the first comment on top, but not really; the privilege of being the one who shared this thing this time should be enough for you to be FIRST!!! without reifying Note over Comment. 

--
Alex Chaffee - al...@stinky.com
http://alexchaffee.com
http://twitter.com/alexch

Les Orchard

unread,
Nov 7, 2011, 10:45:03 PM11/7/11
to shar...@googlegroups.com
On 11/7/11 10:36 PM, Alex Chaffee wrote:
> I'm okay with the flood, especially since it's confined to this thread.
> And since it is a pretty cool idea :-)

Cool. I know I can ramble on about an idea when I get enthusiastic. That
gets annoying if it turns out to be a cruddy idea. :)

> Your idea is very interesting but I'm not sure it's anthropologically
> valid, i.e. corresponds to the way people do use or want to use existing
> tech. For instance, many "notes" are miles long and contain links in
> themselves, often several links, and to different sources.

Yeah, that's true. I think this is mostly suited to feeds with quick
one-liner or at least < 3 paragraph responses. But, that's more cultural
than technological.

Someone could totally share content longer than the original source item
if they wanted to, and it would be up to the feed reader UI to decide
how to display it. (eg. like Google+ truncates and offers "more" links
to reveal)

> Slightly off topic, the difference between "comment" and "note" in
> Reader has always seemed artificial to me. Like, ok, maybe it makes
> sense to put the first comment on top, but not really; the privilege of
> being the one who shared this thing this time should be enough for you
> to be FIRST!!! without reifying Note over Comment.

I think in the scheme I described, the main difference is you really
only get one note per item per person.

It doesn't really lend itself to threaded comments with replies. I mean,
you could probably get that by repeatedly sharing the same item to
indicate subsequent replies over time - but that just seems annoying.

Otherwise, we'd probably be talking about an extension to RSS to
indicate additional comments in the same share. Though, having said
that, I'm pretty sure I was on some mailing list threads back around
2002 or so talking about that exact thing :)

Lucas Wiman

unread,
Nov 7, 2011, 10:46:38 PM11/7/11
to shar...@googlegroups.com
Slightly off topic, the difference between "comment" and "note" in Reader has always seemed artificial to me. Like, ok, maybe it makes sense to put the first comment on top, but not really; the privilege of being the one who shared this thing this time should be enough for you to be FIRST!!! without reifying Note over Comment. 

IIRC, they added "notes" at the same time they added the "note in reader" bookmarklet and added commenting later.  I suspect that it was a historical accident, it probably had no strong UX case for it.

OTOH, I think there is a good reason to separate the behaviors:  some comments I try to highlight things to look out for in the share.  Sort of pre-comments.  Others, I'm commenting on the content of the article to someone who has already read it.

- Lucas

Emmanuel Pire

unread,
Nov 8, 2011, 4:36:05 AM11/8/11
to shar...@googlegroups.com
> Slightly off topic, the difference between "comment" and "note" in Reader has always seemed artificial to me.

Yes and no. Yes because whether i comment or start a note in reader, it will end up in my shared feed. No, because Note was much more like a post (so a source feed, you could actually write a small blog post) and comments were just simple comments on source feeds. Your fiends could actually comment on an item you made up yourself using the bookmarklet for example.

I like this idea that as soon as you have a public feed, you become somehow like a blog that have an RSS feed. Commenting on the other side is another business and should be, in my sense, more than short notes. I used to discuss some items with several peoples, days after the original post.

Alex Chaffee

unread,
Nov 8, 2011, 10:37:59 AM11/8/11
to shar...@googlegroups.com
I think you kind of restated my point:

a. "add an item to my feed"
B. "add a 'top comment'"
c. "add a comment"

"Share" meant "A"
"Share with note" meant "A and B"
"Comment" meant "C"

I think they could have done away withe the atom of "B" and left it up to the UI to put the first comment on top if it was made my the owner of the feed.

But I also agree with the "historical accident" hypothesis of Lucas.

 - A

Emmanuel Pire

unread,
Nov 10, 2011, 2:40:00 AM11/10/11
to shar...@googlegroups.com
I was wondering about this "top comment" gReader used to display in a bubble on top of the item content.
As I see things, this is the problem of readers to choose to show the first comment (from original sharer) on top or not.
Because after all, *they* decide how to display content, not the sharer service. Sharer service is only providing data.

Saliency

unread,
Nov 16, 2011, 6:03:51 PM11/16/11
to shar...@googlegroups.com
agree. 

alex kessinger

unread,
Nov 16, 2011, 6:26:14 PM11/16/11
to shar...@googlegroups.com
I have been looking at activity streams, and it might actually be what
we are looking for vis distributed sharing. If any one service
generated activity streams, a user could use an activity stream
consumer, or aggregator.

So in this hypotehtical world there could be feed readers, and comment
aggrigators. The comment aggregators could just consume activity
streams.

So you and all your friends would be on a given comment aggregator,
and people could use whatever feed reader they wanted.

On Wed, Nov 16, 2011 at 3:03 PM, Saliency <sali...@gmail.com> wrote:
> agree.
>
> --
> The Sharebro Google Group: for http://sharebro.org and related development
> http://groups.google.com/group/sharebro for archives and options
>

Saliency

unread,
Nov 17, 2011, 10:27:43 AM11/17/11
to shar...@googlegroups.com
"So you and all your friends would be on a given comment aggregator,
and people could use whatever feed reader they wanted."

I like it better if everyone is on their own comment feed.

I like the idea of having just having my friends subscribe to my comment feed in the same way you subscribe to a RSS feed.

alex kessinger

unread,
Nov 17, 2011, 11:47:55 AM11/17/11
to shar...@googlegroups.com
How would people respond to your comment feed?

Saliency

unread,
Nov 17, 2011, 1:22:48 PM11/17/11
to shar...@googlegroups.com
To respond to a person you publish a comment to your feed.  If they are following your feed their rss links your comment to their comment.

When you respond to a comment you just put in the comments url instead of a news stories url.

Saliency

unread,
Nov 17, 2011, 1:23:38 PM11/17/11
to shar...@googlegroups.com
their rss reader that is.

Emmanuel Pire

unread,
Nov 17, 2011, 3:15:09 PM11/17/11
to shar...@googlegroups.com
So if I understand Saliency, I have 1 public feed that contains my items, and another one that contains my comments, each linked to the source item.
My followers will see their own comments attached to items + comments from me and other they follow.
If they want to comment, they add an item in their comment feed, so if i follow them back i see their comments.

hey, that's good isn't it ?



On Thu, Nov 17, 2011 at 7:23 PM, Saliency <sali...@gmail.com> wrote:
their rss reader that is.

--
The Sharebro Google Group: for http://sharebro.org and related development
http://groups.google.com/group/sharebro for archives and options

Emmanuel Pire

unread,
Nov 18, 2011, 7:43:16 AM11/18/11
to shar...@googlegroups.com
I see one potential problem though:

John and Jack are pals. They follow each other, share a lot and discuss through comments.
Biney is just a normal guy using gReader, not following anyone.

Now what if Biney starts to follow John but NOT Jack. Biney will then see only a part of the discussion between John and Jack, only the posts from John.

This is a problem. How did Google did originally to avoid this ? Or was it like this before ?

Alex Chaffee

unread,
Nov 18, 2011, 12:10:41 PM11/18/11
to Emmanuel Pire, shar...@googlegroups.com
On Thu, Nov 17, 2011 at 12:15 PM, Emmanuel Pire <pirem...@gmail.com> wrote:
> So if I understand Saliency, I have 1 public feed that contains my items,
> and another one that contains my comments, each linked to the source item.
> My followers will see their own comments attached to items + comments from
> me and other they follow.
> If they want to comment, they add an item in their comment feed, so if i
> follow them back i see their comments.
> hey, that's good isn't it ?

I think not. I want the old Reader back! In the old Google Reader a
separate comment *thread* was made for each item+sharer dyad.

I think you just described Notes, not Comments. Comments are a
conversation. Notes are all from a single person.

Or... if others could post their comments to "my" comment feed then
the system you described would be a complete mess, mixing comments
from different people on different items into a single "thread" (more
like a ball of yarn, since you couldn't tell who was replying to
what), but *not* including comments made by the same people on items
shared by other people...

And how, in an RSS Reader, would you subscribe to one of these comment threads?

Slipping into sharbro dev mode now... I'm learning a lot about the
Google Reader API. Most of it is not going away. I believe that *if*
there were a way to add a comment from inside Reader, I could create a
new feed and then subscribe users to it and put that feed in each
user's subscriptions list. But where would I put it? In a new folder
called "Comments" (or "Comment View") that's far beneath the "Shares"
(or "People you follow") folder, and thus far removed from the actual
item that has the comments on it?

Also each subscription would need a good title... would the title
include the full name of the original item? e.g.

Comments on "Why Strawberries Are Objectively Superior To
Raspberries, Part 3" from 'Berry Blog'

Problems like these are why I've been punting on comments for now. I
think the only solution will end up being a browser plugin/userscript
so that each dyad feed can be spliced in beneath the text of the item
in Reader. And I think, like Alex K, that the server-side data format
of comments will look more like an Activity Stream and less like RSS.

Server-side I think we're making good brainstorming progress.
Client-side we've still barely restored People You Follow (1000+), let
alone put comments back in.

- A

Saliency

unread,
Nov 18, 2011, 2:51:55 PM11/18/11
to shar...@googlegroups.com
I am pretty sure that all this can be handled by the RSS reader.  I think anything can be done with a decentralized system.  I just might not know the design pattern of the top of my head :)  Also something to keep in mind that in this model RSS readers become more like search engines.  All the data is available to everyone.  The service, a reader in this case not a search engine, able to provide the best service wins business.  No company has the webs data all to themselves but companies may choose to create derived data for performance or added value.  Google for instance does not crawl the web each time we make a request they work off of some internal datasets they have.


Back to the question.  First off if we both use the same web based RSS reader the company could derive all the relations based off of who everyone in their system subscribes to.  This is better then the current centralized system but not ideal.  We can all move to a new product and not lose any data but the system is subject to network effects.

What I think you do so that your friends are not forced to use the same reader is to add FOAF tags to a profile section on your domain.  I'm not sure exactly were you would store your profile data but I guess that conventions for each platform people are using to store their data would emerge.  Perhaps on blogger the convention would be that you put your profile data on an about page.  Perhaps in tumblr you tag you profile data "profile" so that it is exposed in a new feed on the same domain.

Saliency

unread,
Nov 18, 2011, 3:05:28 PM11/18/11
to shar...@googlegroups.com, Emmanuel Pire

"I think not. I want the old Reader back! In the old Google Reader a
separate comment *thread* was made for each item+sharer dyad.

I think you just described Notes, not Comments. Comments are a
conversation. Notes are all from a single person."


Nope inside of the RSS reader it is conversation.  The RSS reader does need to do the work of assembling the conversation. This is why you need to subscribe to your friends comment feeds.

alex kessinger

unread,
Nov 18, 2011, 3:38:36 PM11/18/11
to shar...@googlegroups.com
I guess I don't see how you would be able to follow a thread in your
system or RSS feed only comments.

Alex Chaffee

unread,
Nov 18, 2011, 7:13:57 PM11/18/11
to alex kessinger, shar...@googlegroups.com
On Fri, Nov 18, 2011 at 12:38 PM, alex kessinger <void...@gmail.com> wrote:
> I guess I don't see how you would be able to follow a thread in your
> system

...without a lot of smarts built in to the feed reader. Which is fine
if you're building a reader but we're not doing that... yet. Though
Francis is -- see http://yfrog.com/h2aziip -- but I'd like to try to
make something that works inside Google Reader.

And ideally through many reader apps if they want to use our API
instead of their own database. (And speaking as a developer of apps
that is a tough sell.)

The user base of Google Reader is going to be (1000+) times more than
of any 3rd party feed reader, especially since I believe Peak RSS may
have happened around 2008 or so.

> or RSS feed only comments.

...especially since RSS 2.0 is still kind of behind the times in terms
of source and identity metadata. Atom may be better. JAS definitely
is.

Alex Chaffee

unread,
Nov 18, 2011, 8:06:57 PM11/18/11
to shar...@googlegroups.com
The Google Reader API backend is pretty smart, and has actually solved many of the problems you're waving your hands at.

From http://code.google.com/p/pyrfeed/wiki/GoogleReaderAPI#The_three_layers_for_feed_aggregators :

When you're writting a feed aggregator, you need to write three different layers :

Layer 1 : The layer that parse feeds. It's not the easiest job. "But, it's just xml, it should be easy". It's not. It's just xml. It's just 10 differents and incompatible xml formats (9 RSS formats according to Mark Pilgrim and 1 Atom format). You also perhaps need to understand all non standard feeds that mix some features from differents standards.

Layer 2 : The database layer. Once you've parsed your feed, you need to store it in a database, and and interesting things like "items read", etc.

Layer 3 : The user interface.

Comment threads are pretty dang easy design wise... but only once you have a standard namespace or other system for uniquely identifying feeds, items, and users.

And that's not counting all the annoying crap the Google Reader team (not to mention Feedburner, who got bought by Google, and several others) solved back in the 2001-2005 era. What if I post an update to my original item? The system needs to realize that that "new" item is actually the same as the old item for comment purposes. What if my new RSS file doesn't include an item that was there an hour ago? The system needs to decide if the item was deleted, or just old. What if some feed has a bug where it reuses a guid, or doesn't provide one in the first place?

If an item isn't shared publicly, who can see it? Who can see the comments on it? Who can add comments and who can just read them? The system now needs a friend network. How do we identify people in that network? Is @alexch the same as ale...@gmail.com? Even if he is, does he want everyone to know that?

I'm sorry if this sounds like I'm piling on. I appreciate your optimism and drive. I'm just afraid you saying "RSS readers become more like search engines" means that you don't realize that they already pretty much are like search engines, only solving a different and overlapping set of problems.

If all you're saying is "it'd be nice if all the item and comment data were distributed" well then sure, I agree, but that opens another thousand cans of worms. Is it a federation or a caching proxy network? If a federation, then what happens when a node goes away? What's the API and schema and are they even theoretically performant and scalable? And what problem are we trying to solve anyway?

Diaspora is also trying to solve some of these problems, at least at a buzzword level. I know the people (they are IRL FOAFs) but I still don't really know what their underlying design solutions are. I need to buy some people a beer.(*)


BTW RSS 2.0 already has a "comments" element: 


It sounds good except uh-oh, it's just a URL to a web page somewhere and I honestly don't know if anyone who wrote an RSS reader has ever used it. Wordpress appends an <a href> link to it into the "content:encoded" section of its feed items along with a little image saying "Add a comment" in a crappy font [ look familiar?]  that I bet nobody in history has ever clicked on when they see it inside Reader. If they did they'd end up back at the wordpress blog source page at "http://blog.mycrappycrap.com/2011/11/17/my-crappy-thoughts/#comments".

 - A

P.S. I'm in a pissy mood right now so please forgive my pissiness.

P.P.S. (*) Actually this week I need to buy them as much whiskey as they feel like and not ask stupid technical questions: http://techcrunch.com/2011/11/13/diaspora-co-founder-ilya-zhitomirskiy-passes-away-at-21/


> --
> The Sharebro Google Group: for http://sharebro.org and related development
> http://groups.google.com/group/sharebro for archives and options
>



--

Saliency

unread,
Nov 19, 2011, 2:37:30 AM11/19/11
to shar...@googlegroups.com
"I'm in a pissy mood right now so please forgive my pissiness."
No problem your the one writing code not me :)  I'm the one being bossy ;)

Reply all
Reply to author
Forward
0 new messages