Microformat support in future versions of Firefox
Alex Faaborg
microformat support (http://en.wikipedia.org/wiki/Microformats) in
future versions of Firefox. Open questions include topics like:
-Is it a good idea?
-What is the user experience?
-How should Mozilla create a developer ecosystem for microformats?
I'll start the conversation with some emails sent back and forth
between me and Sherman Dickman.
-Alex
Alex Faaborg
in Firefox]
There's a lot of really exciting ideas in there. In particular I
thought one of the most important ideas was: "The parsing algorithms
themselves could be plug-able so that users may replace the default
parsers or add new parsers to find specific content."
If you look at the most commonly used data detectors currently on the
market, Microsoft Smart Tags (Office XP and later) and Google AutoLink
(Google toolbar 3.0 and later), both systems are closed and controlled,
designed to redirect users to each company's various online properties.
In fact, Microsoft even pulled Smart Tags as being a feature from IE 6
when the press accused them of illegally leveraging their monopoly.
In 2001, before the release of IE 6, Walt Mossberg wrote in the Wall
Street Journal:
"Using the browser to plant unwanted and unplanned content on these
pages -- especially links to Microsoft's own sites -- is the equivalent
of a printing company adding its own editorial and advertising messages
to the margins of a book it has been hired to print....Microsoft
executives have done the right thing this week in killing the feature."
But then in 2005 Google added data detection to IE and Firefox through
their toolbar. And despite Google's incredibly different public image
(the whole "don't be evil" thing), the press reacted in exactly the
same way to AutoLink as they did to Smart Tags. Leslie Walker wrote in
the Washington Post:
"There must be limits to Google's power to steer us around the Internet
-- otherwise Google might morph into a meaner monopolist than
Microsoft."
By making the parsing algorithms plug-able, Mozilla could finally do
for data detection what Microsoft and Google have thus far been unable
to do. Mozilla could create an open system that is designed to benefit
end users, as opposed to one that is designed to further the business
interests of a particular company.
This is important because there are a lot of user scenarios that aren't
possible until a plug-able architecture for parsing algorithms is
created. For instance, linking all of the foods in a recipe to the
user's online grocery store. In the past this hasn't be possible
because (1) Microsoft and Google don't have a business interest in the
user going to a particular online grocery store, and (2) each user is
likely to use a different online grocery store.
My masters thesis was actually about solving this problem, enabling
users to link any kind of data to any service on the Web. The system
doesn't rely on the page having any type of microformat or semantic
markup, and it is able to understand the meaning of the words and
phrases on the Web page using MIT's ConceptNet knowledge base and
Stanford's TAP knowledge base. If you are interested you can see a
video of the prototype here (H.264, plays in Quicktime 7):
http://alumni.media.mit.edu/~faaborg/files/thesis/draft/complete/movies/3_miro_1_scan.mov
You can learn more about the project here:
http://agents.media.mit.edu/projects/semanticsearch/
-Alex
Alex Faaborg
I think the critical questions become... how do we stage something
like this out over time? Could we start with micro-formated contacts
and events, knock those out of the park, and then add support for new
data types over time? How would we do this in the most massively
scalable way possible?
Best,
Sherman
Alex Faaborg
> I think the critical questions become... how do we stage something
> like this out over time?
Probably the best approach would be to start by supporting a small
number of microformats (less than 10), and for each microformat, be
sure to support the most common applications for that data type. This
would require some market analysis, figuring out the most popular
calendar applications, mapping applications, etc. It's important that
application support is prioritized by market share to maximize the
usefulness of the feature.
However the most important aspect of this initial entry into
microformats is not to have great coverage, since a lot of sites won't
have data encoded yet. Instead, the most important aspect of the
initial microformat support is to demonstrate to Web developers and
extension developers that Firefox is serious about the idea. It's
important to get these two groups involved with early beta releases so
that they can start to get their sites and extensions ready. It might
be a good idea to wait 6 months from announcing the idea to releasing a
version of Firefox with microformat support, so that Web developers and
extension developers have some time to prepare.
When I say extensions, I mean associations between microformats and
services that are created by application developers. So these are not
like traditional Firefox extensions in that Firefox should provide a
consistent user experience for data detection, regardless of the type
of data. These extensions should also be easier to add (never require
that the application restart), and be managed in an area of Firefox's
preferences that is dedicated to data detection.
> How would we do this in the most massively
> scalable way possible?
This limited support for microformats should then be launched
simultaneously with a Web site that has sections to focus on the
specific needs of end users, Web developers, and extension developers.
End users:
(1) Tutorial about using Firefox to associate data with services
(2) List of extensions that detect particular types of data
(3) [power users] ability to vote on what types of data they would like
to see detected in the future
Web developers:
(1) Tutorial about microformat detection in Firefox, with examples of
how the data is expressed
(2) View a list of the most popular microformats on the Web (get data
from Google?)
(3) Online tool that makes encoding a particular type of data in a
microformat as easy as filling out a form (should also ship as a built
in feature of Firefox in the tools menu near the JavaScript console)
Extension developers:
(1) View a list of the most popular microformats on the Web (get data
from Google?)
(2) View a list of the most requested data types by actual end users
(3) Search to see if a particular type of data has a microformat
associated with it yet
(4) Define new a microformat, and publicize it on the site
(5) Learn how to associate a client or Web-based application with a
particular microformat
(6) Create a new extension, and publicize it on the site
When particular formats and extensions reach a critical mass of
popularity amongst the small percentage of users who install
extensions, consider integrating them directly into the browser.
But before launching any of this, it is important to get the user
experience perfect. I think microformat detection would be a great
prototype to come out of Mozilla Labs. It would be good to see how
users react to the idea, and to do some usability testing.
-Alex
Benjamin Smedberg
I think a related question is, do we support microformats natively at all?
Or do we provide hooks so that extensions (and other applications) can
recognize and process microformat data? (Probably the answer is "both", but
I think that the massively scalable solution involves leveraging our vast
extension development community to add support for new/popular microformats
over time.
--BDS
Alex Faaborg
>do we support microformats natively at all?
I agree that scalability can only be achieved by leveraging the
extension development community.
However, I think one advantage of initially supporting some
microformats natively is that it will help them quickly reach the
critical mass needed to begin appearing on the Web.
For instance, imagine it's your job to create the schedule for a
conference attended by 3000 people, and put it up on the Web. You know
the conference will be attended by a pretty technical crowd, so about
60% of them will be using Firefox. You also know that encoding each
session into a microformat is as easy as filling out a form that
generates the data with the correct semantics. So, do you go to the
extra work to make it easy for these people to add the sessions they
are interested in to their calendars?
I would argue that if you know you are going to help 1,800 people, you
will go to the extra work. But if you are only going to help Firefox
users who also use an extension (5% of 60% of 3,000, so 90 people), you
won't bother.
It is still possible for XML standards to reach a critical mass of
adoption without any native Web browser support, RSS is a great
example. However, in the case of microformats, I think we are looking
at classic chicken and egg problem: Web developers are not going to
semantically encode data unless they have a compelling reason to do so.
-Alex
Alex Faaborg
the topic of what is the best way to handle the user experience of
microformat detection.
Here are some screencasts of Microsoft's copy and paste interface for
microformats, called Live Clipboard:
http://spaces.live.com/editorial/rayozzie/demo/liveclip/screencast/liveclipdemo.html
Ray Ozzie's concept development team prototypes with Firefox?
I don't like the copy-paste model for several reasons:
1) After copying a piece of data, it is not immediately clear to the
user what they should do next.
2) It also isn't clear to the user what is actually possible. If the
user can't copy data into a particular application, they are likely to
blame the browser, even though it is the application's fault for not
supporting the microformat. Things might work well for common types
like contact information, calendar events and addresses, but this
problem would increase if a wide variety of specialized and domain
specific microformats started to emerge on the Web.
3) It is not clear from the videos if the browser is injecting the
clipboard button into the page or if the Web page's creator provided
the icon. If it is the former, people are going to be up in arms about
the browser modifying their pages. And if it is the later, the user is
going to have a really inconsistent experience between different sites
(like all those different RSS buttons that blogs currently have).
4) The clipboard icon is small, but the semantically encoded
information can sometimes take up a large section of the page. Because
of this, it is not always clear exactly what information the clipboard
icon maps to.
Potential Solutions:
One way to solve the first two problems is to dynamically generate
"send to" menus (along with an option to change the defaults), instead
of using a copy and paste model. For instance, after the user hovered
over an event, the browser would display "Send to Google Calendar."
The second two problems could be solved by overlaying what appear to be
colored transparent panes of glass over detected information on the
page. The advantages of this are 1) the user experience of detecting
semantically encoded data is consistent between sites, 2) it doesn't
appear as if the browser is directly modifying the page, and 3)
transparent overlays can map better to the information being acted on.
3D accelerating the appearance of these panes of glass could provide
some cool eye candy.
Sherman Dickman
Great ideas, and I most definitely agree that we should respect the
page creator's work (and perhaps copyright) and not alter the default
appearance of the page.
The Tails/Tails Export extensions do a good job of listing
microformats via pop up window or side panel, but I also feel that
there should be a way to recognize microformats directly in the
content window, individually or in aggregate. Perhaps this can be
achieved by mouse over for individual elements, and by toggle key or
toggle toolbar button for all elements.
For operations, handling individual elements via contextual menu
selection seems most intuitive to me, but I also see a need to batch
process multiple elements (for example, add 10 out of 15 recognized
hCards to my address book). Perhaps in the later case, a sidebar
would be more appropriate to facilitate faster selection of multiple
elements?
Sherman