Feed Handling Experience and Firefox 2
Ben Goodger
Feeds have existed on the web for some years, but higher market
penetration continues to be difficult as the vehicle for feed discovery
(the browser) too often stands in the way of easy subscription. While
syndicated content is not for everyone, it is very useful for people who
like to read news and receive lots of information every day.
The goal of the Feed Handling changes proposed here are to make a useful
experience for the ~15% or so of folk I predict will find feeds very
useful, and not provide a horrible experience to the remainder.
Technical Background:
- Sites provide web feeds so that their content can be dynamically
syndicated to users seeking a uniform reading experience.
- Feeds are also used as a transport to data of other types, e.g. media
and pictures
- Many sites link to this content using a <link> tag in the <head>
section of their HTML, but they also link to it directly using <a>
tags.
- Apple defines a "feed" pseudo-protocol-scheme which maps to http that
tells their client (Safari) that it is loading a feed.
Firefox 1.x's approach to dealing with this:
- Firefox 1.x detects <link> tags and shows an icon in the location bar,
offering users the ability to subscribe using Live Bookmarks.
- When the user clicks on a link to feed content, it displays either raw
XML, a text/plain version of the content, or some sampling of the raw
data, depending on how the server is configured
- When the user clicks on a feed: link, the application on the user's
system registered to handle that protocol is invoked. This is
different from Live Bookmarks, and any reader the user may prefer to
use.
Problems with this approach:
- They all offer inconsistent experiences:
* Live Bookmarks are not suitable for every user, and for some feeds,
even if they are for others.
* Web readers are very popular (Bloglines, My Yahoo, iGoogle, etc) and
are not represented here. The experience to subscribe to one of
these services is very bad. The user either has to find the URL of
the feed by reading the HTML source or copying a link on the page,
or rely on the site author to provide a subscription link to the
site. This is not scalable as there are very many subscription
sites. The site a user prefers to use may not be listed.
* Having a different subscription experience just because a URL used
feed: instead of http: is bad. All pathways to subscription should
follow the same conduit, regardless of protocol scheme.
- Showing garbage when the user clicks on a document of a type that is
as common as feed formats is fundamentally wrong.
Ideal Experience:
- When users click on a link to feed content, it should show something
meaningful, irrespective of how the content is served, and using a
feed: protocol scheme or not. People's definitions of "meaningful" can
vary. It is not our goal to provide reading functionality through this
view, since generally people click on these links from the page
content itself. Rather, it is primarily a vehicle for subscription.
- The subscription experience within the browser should recognize that
users have diverse preferences with respect to their preferred reader:
the reader could be a client application, a web service or the
internal Live Bookmarks handler (or even another Firefox extension[1])
Proposed System Requirements/Solution:
The Proposed System has several components:
* Inline Feed Preview & Subscription
When the user clicks on a link to a feed, rather than displaying the
raw data contained within the feed, a page should be shown that
displays some metadata about the feed, and some of its content. Front
and center should be the ability to subscribe to the feed.
Since feeds are a relatively new concept to the majority of users, the
page should succinctly but clearly explain what a feed is.
Because a site might offer multiple feeds for subscription, some
content from the feed should also be shown to allow the user to know
that they are subscribing to the correct feed (since it can take some
work to remove it from their reader once they have added it) (the
argument against this is that the reader should provide this preview,
but this not consistently implemented across all readers).
* Reader Selection
The user should be able to choose their favorite reader to subscribe
in. This can be a web service, a client application, or an internal
component (e.g. Live Bookmarks). The preview page should present these
options.
I think that showing the different major types of options (client vs.
web reader vs. Live Bookmarks) may be worth breaking out into separate
lists, since it may not occur to the user that they can change the
value of a single menu list, and the method for changing the selection
from one client app to another (using a file picker - OS maintains a
list) is very different to choosing another web service (menulist -
browser maintains a list).
At the same time I am hesitant to make the page appear too
"heavyweight", so perhaps the detailed subscribe options could be
hidden by default until the user decides to actually subscribe
(remembering expand/collapse state the next time they view a feed, of
course).
The user should be able to ask the browser to remember the decision
they made and never ask again. When they choose this option, when
opening feed content they are taken directly to their reader
automatically, regardless of what the reader is.
-> A note about podcasts, photocasts
Feeds are beginning to be used as a transport for other data, e.g.
multimedia. We had a problem with feeds where the variety of types
feeds are incorrectly served as means that we need to perform content
sniffing of all data that comes through. Since these new data formats
("podcast" and "photocast") do not have a mime type that is
distinguishable from a regular feed, we have no way of telling them
apart. However they have functionality that requires a very different
reading experience than news or blog postings. As a result we do not
support "automatic handling" where the preview page is by-passed for
these multimedia feeds. The user must always visit this page and
choose the appropriate reader. We will do our best at that time to
detect the implicit "content type" from the data in the feed and offer
intelligent choices.
* Subscribe Button (Browser Toolbar)
When the user browses to a site that exposes a feed through the <link>
tag, a button appears adjacent to the URL bar that offers them the
ability to subscribe.
When there is only one feed linked, clicking the button will load that
feed in the content area (thus showing the Inline Feed Preview which
gives the option to subscribe).
When there are more than one feed (feeds that look like the same data
with the only difference being a change in syndication format - e.g.
RSS vs. Atom are collapsed), a menu is shown that lets the user pick
one to display.
If the user has already selected a handler and asked not to be asked
again, clicking the Subscribe button automatically subscribes them in
their selected reader.
Right clicking on the button should show a context menu option for
viewing the feed content and subscribing using a different reader (as
an override, analogous to "Open With...")
* Web Service Registration
An API is to be exposed to web services so that they can offer to be
added to the set of available choices in the Subscription experience.
A description of this API[2] can be found in the WhatWG Web
Applications 1.0 specification.
When the user browses to their reader web service, the site can
provide a link that uses a special javascript call on the navigator
object to add themselves to the list of choices in the Reader
Selection UI. This is analogous to adding a Search Engine to the
search box.
There are various security considerations here:
1 First of all, this javascript API cannot add without prompting the
user first. The user must be shown the name of the reader, and the
URI it is proposing to add.
2 We cannot have the javascript function make the reader the "default"
action, since it's too easy for the user's experience to be hijacked
this way (given the number of users who invariably choose the wrong
option when presented with a dialog box). (This is actually
analogous to the search engine experience, since when you add an
engine it does not become the default, only another choice).
After you add a reader how do you switch to it? We cannot easily
determine when someone installs a new reader application on their
computer. We can however detect when someone registers a new web
service using the javascript API. When this happens we could force the
Preview Page again, with the subscription box opened and some
indication that a new item had been added to the list, akin to the
Start Menu's "New Programs have been Installed".
The last option is configuration...
* Configuration
The "Download Actions" window should be modified to have two views:
Basic and Advanced. Advanced shows all download actions for registered
types and plugins, as with Firefox 1.x. Basic shows common types and
how they are handled, such as feeds, mailto: links, etc.
[1] - The scope of this work is not to explicitly implement such an
extension but it should aim to not preclude the addition of
additional subscription options by extensions.
[2] - http://www.whatwg.org/specs/web-apps/current-work/#browser
Tracking:
I am tracking this feature through:
https://bugzilla.mozilla.org/show_bug.cgi?id=325081
-Ben
Daniel Schierbeck
This suggestion looks very nice, and I'm sure the added functionality
will be a boost to the use of web syndication.
I'm curious to know if any changes to Firefox' own feed handling has
been discussed - currently, feeds can only be presented as Live Bookmarks.
There are, in my opinion, two problems with the current handling.
1. The lack of an update notification mechanism means
that extensive use of Live Bookmarks becomes painful,
as each bookmark must be manually scanned for updates.
Both RSS and Atom have the meta information needed to
implement a simple notification mechanism, e.g. showing
a different icon for updated feeds.
2. The one-to-one relationship between a web feed and a
Live Bookmark. Many feeds are not updated very often,
and may fall within the same category. Having Live
Bookmarks that combined several feeds would simplify
the user interface. A UI must be designed to let the
user know which feed a Live Bookmark entry belongs to.
These are but extensions of the behavior you have already sketched out.
Cheers,
Daniel Schierbeck
Jeff Schiller
>
> 1. The lack of an update notification mechanism means
> that extensive use of Live Bookmarks becomes painful,
> as each bookmark must be manually scanned for updates.
> Both RSS and Atom have the meta information needed to
> implement a simple notification mechanism, e.g. showing
> a different icon for updated feeds.
>
I agree, although I like to spend my mornings flicking through my Live
Bookmark fly-out menus, this does get to be a long chore eventually.
Here's my proposal for this:
- Firefox should remember an "unread" flag per Live Bookmark (per
profile, obviously), unread should be set to true if there is at least
one item from the Live Bookmark has not been visited in the browser
- any unvisited items will show the Live Bookmark as bold in the
Bookmark menu
- this "bolding" should propagate up the bookmark hierarchy (in other
words, folders that contain Live Bookmarks with unread items will also
appear bold, this continues for folders within folders)
- a right-click menu option for a Live Bookmark would be to "Mark all
items as read" (put it underneath Reload Live Bookmark)
NOTE: The bolding only happens for Live Bookmarks and folders that
contain Live Bookmarks (due to their syndicated nature), not for
regular bookmarks or folders that only contain regular bookmarks...
Ben Goodger
> 2. The one-to-one relationship between a web feed and a
> Live Bookmark. Many feeds are not updated very often,
> and may fall within the same category. Having Live
> Bookmarks that combined several feeds would simplify
> the user interface. A UI must be designed to let the
> user know which feed a Live Bookmark entry belongs to.
With Places, you will shortly be able to perform searches like this:
place:&annotation=livemark/bookmarkFeedURI&folder=47&folder=52&sort=4&maxResults=20
...and then save them into your bookmarks, where they will show up as
live-updating folders... In this case this means "show me the most
recent 20 feed posts from folders 47 and 52" (let's say 47 and 52 are
livemark folders)
Of course, the Query Builder will have the capability of providing a GUI
for constructing queries like this without having to remember the wacky
place: syntax!
-Ben
Daniel Schierbeck
Very nice!
If only there was time to ship Places with Bon Echo...
Cheers,
Daniel
hoy...@gmail.com
a couple of considerations. A lot OS X users are using NetNewsWire.
Safari is actually easier to use with NetNewsWire than Firefox because
of its inability to drag the syndication icon. Arguably, you can't
drag the RSS icon in Safari either, but you can drag a "feed://" URL
once clicking on the RSS button. So, would it be possible to list a
drop down box of draggable feed items rather than only giving the
option to Livebookmark?
Another consideration about Livebookmarks is OPML export which I don't
see mentioned here. It'd be trivial to allow an OPML of all
Livebookmarks retaining any existing folder hierarchy. While not
optimal, this would enable some compatibility with some of the web
enabled RSS readers. Clearly, a web service or API that could be
hooked in to by a web-based reader would be a better solution which I
think is what you're proposing.
Another troubling concern are these feed extension proposals from
Microsoft (SSE) and Google (GData). It seems that both companies are
moving toward more intelligent feeds. Implied in the Microsoft SSE
standard is their intention of using request variables to request the
_new_ items which would be used for syncing. If Firefox doesn't
support GData and SSE it could fall behind Internet Explorer 7. Also,
it sounds to me like MS is going to allow Internet Explorer to sync
bookmarks to other computers running IE e.g. your work computer can
sync your home computers bookmarks probably via a web service. I think
a lot of users are willing to eat the inherent hassle of web based
aggregators as a stopgap solution of keeping a "master" bookmark file
available. Automatic bookmark syncing could erode the signifigance of
web based bookmarking sites like Bloglines, etc. Does Mozilla/Firefox
intend go leverage Microsoft Feed Platform or invent a new one? Also,
are you aware of any plans for feed synchronization within Firefox?
hoy...@gmail.com
Robert Sayre
> So, would it be possible to list a
> drop down box of draggable feed items rather than only giving the
> option to Livebookmark?
The design allows for other feed programs to register, not just live
bookmarks.
>
> Another consideration about Livebookmarks is OPML export which I don't
> see mentioned here. It'd be trivial to allow an OPML of all
> Livebookmarks retaining any existing folder hierarchy.
I wrote the OPML export code in Thunderbird, and I feel that would be a
nice-to-have for Firefox, since most other feed readers import/export to
that format, and it's considered good citizenship. Import is less of an
immediate concern IMO, since I don't think many people will be importing
reading lists into live bookmarks.
> If Firefox doesn't
> support GData and SSE it could fall behind Internet Explorer 7.
Disagree. The feed parser has a pretty open design. It should be
possible to add support for that stuff later, or let extension authors
do it.
> Does Mozilla/Firefox
> intend go leverage Microsoft Feed Platform or invent a new one? Also,
> are you aware of any plans for feed synchronization within Firefox?
I agree that feed synchronization is going to be necessary on the
Mozilla platform, but I don't think it's a realistic feature for FF2. If
anyone is lost here, Tony is referring to the various HTTP-based
synchronization protocols for feeds (think IMAP, read/unread, etc). The
biggest example right now is Newsgator/NetNewsWire/FeedDemon, but I'm
sure Microsoft, Yahoo, Google and others are working on it. Kind of old
news for Forumzilla users, since I gather Myk has had IMAP sync working
for years. ;)
-Rob
Adam Kowalczyk
mainly on the subscription mechanism and didn't cover any planned
improvements to the Live Bookmarks themselves. Some excellent points
where made by the previous posters but I still don't know if the
developers see it the same way. I would like to elaborate why I think
notifications are so important.
I know that the approach is to support only basic feed reading. However,
in my opinion notifications are one of those basic functionalities that
are required for the feature to be usable. Let's split the analysis
between a couple of use cases:
1) Regular reading of the feed. An example would be daily news and other
sources that are read on a regular basis, for instance once a day. In
such case, when user checks the Live Bookmark folder he knows that there
are some new messages and can spot them rather easily as he can remember
what he read the last time. It would still be nice to have the unread
items marked, though.
2) Irregularly updated feeds. A typical example is a blog. User has to
remember to manually scan the Live Bookmark folder for new messages -
often wasting time since there are none. The fact that he has to
remember which items he has already read, which may have been quite some
time ago, makes the chore double hard. This way you can follow a couple
of feeds tops or it becomes easier to just visit the websites.
3) Frequent checking for updates. While I am at my computer doing some
other stuff I'd like to be notified of any breaking news at Reuters or
new posts at /. instead of wasting time quick-peaking every half an hour.
Every of those use cases would benefit greatly from a notification
system, like the one described by Jeff. Let's make it clear. The primary
goal of feed aggregation is to reduce the effort to follow various
information sources. When you have to manually scan for updates and
remember the already read ones, this goal is hardly achieved any better
than when manually visiting the websites with which the feed is associated.
- Adam
Daniel Schierbeck
Daniel
Ben Goodger
> The original post by Ben Goodger, although very insightful, focused
> mainly on the subscription mechanism and didn't cover any planned
> improvements to the Live Bookmarks themselves. Some excellent points
> where made by the previous posters but I still don't know if the
> developers see it the same way. I would like to elaborate why I think
> notifications are so important.
Most of the improvements that affect live bookmarks are tied to Places.
We could back-port some of them to the old bookmarks system but I don't
think it'd be worth the effort.
For Firefox 2, we'd at least like to improve the subscription experience
for _any_ reader, since that is probably the biggest hurdle faced by
users at present.
-Ben
Mike Connor
On 27-Apr-06, at 1:59 PM, Adam Kowalczyk wrote:
> Every of those use cases would benefit greatly from a notification
> system, like the one described by Jeff. Let's make it clear. The
> primary goal of feed aggregation is to reduce the effort to follow
> various information sources. When you have to manually scan for
> updates and remember the already read ones, this goal is hardly
> achieved any better than when manually visiting the websites with
> which the feed is associated.
You are correct that the goal of feed aggregation is to reduce that
effort. However, I don't believe Live Bookmarks fills the
aggregation role in any significant fashion. They're great for a lot
of things (i.e. I use them for bugzilla buglists, which is a
ridiculously wonderful feature of Bugzilla) but if you want to read
everything on a feed, especially a rarely-updated feed, a real
aggregator will beat Live Bookmarks handily, as well it should.
Trying to be a good aggregator is not a useful goal, as there are
great ones out there that can focus for more on providing the right
UI for that type of task (I'm especially partial to FeedLounge, myself).
This isn't to say that we shouldn't change the live bookmark icon
color to indicate an updated state. I could even make a case that we
want to use the orange color for new/updated content, and find
another color that's a touch more subtle for the default. (And
possibly use the same scheme to indicate that the feed for a page is
already a live bookmark? Added is the default, not added is orange?)
-- Mike