A strawman proposal and implementation for a simplified WebFinger

[DeWitt Clinton]

Hi all,

I'm excited that this discussion is getting underway.  I remember the first conversations about WebFinger and thinking what a great idea it was, and how easily it could be built using existing web technologies, like HTTP, HTML, and XFN.

That said, I've been wondering if somewhere along the way that simplicity might get lost, so I wanted to throw out an even simpler proposal for discovery and response formats that could potentially cover the 80% case of email-like identifier to service lookup.

Note that this isn't saying anything bad about XRD or anything, I just wanted to do something lightweight that reused our existing investment and successes with formats like HTML and XFN.  Nothing here precludes XRD, of course.

But rather than make an alternative proposal in a vacuum, I coded up a working demo first.  To see this simplified WebFinger-ish client in action, please visit:

  http://webfingerclient-dclinton.appspot.com/

Type in an email-like identifier.  My email address, for example, works well:  "dew...@unto.net".  In a moment, a list of services that are associated with the account should appear.  You can also try your @gmail.com address, and @friendfeed.com addresses should work (more on why/how in a sec*).  If you follow the instructions below you can enable this for your own site with just a few steps.

The interesting part is how the lookup is performed.  The steps are simple:

1. The client is given an email-like identifier
2. The identifier is parsed into a local-part and a domain per RFC 2822
3. The domain is converted into a host URL at http://{domain}/
4. The host URL is fetched, following redirects, to retrieve an HTML page
5. The HTML page is parsed for the existence of a rel="finger" link element
6. The href target of the rel="finger" is parsed as a URL template, substituting in the local-part or addr-spec as necessary
7. The resultant URL is fetched, following redirects, and parsed for rel="me" anchors and links
8. The links are returned as services associated with the identifier

In practice, this approach can be exceeding simple for a provider to implement.  In my own case, I simply added the following line of HTML to my homepage:

  <link rel="finger" href="http://finger.unto.net/search?q={addr-spec}" title="WebFinger lookup service"/>

Finger templates can include either 'local-part' (e.g., 'dewitt') or 'addr-spec' (e.g. 'dew...@unto.net') as they prefer.

I could have implemented a new service at finger.unto.net to return a full document with all of the services associated with the account.  But in my case, I already have a XFN based profile at Google, so all I needed to do was redirect to it.  I did this with nothing more than a RedirectMatch in my httpd.conf.

  RedirectMatch ^/search\?addr-spec=dew...@unto.net$ http://www.google.com/profiles/dclinton

*I mentioned that @gmail.com and @friendfeed.com addresses also work with the client.  I did this by cheating.  Both of those domains accept incoming email and expose XFN formatted profiles with a well-known mapping, so I hardcoded those by hand.  I wouldn't recommend that clients do this by default, but I did want to show some other semi-working examples.  Your site can work by following the instructions for link rel="finger" as above.

Advantages to this simplified model are a) it really is that easy, both on the client side, and particularly on the server side, and b) it reuses work we've already invested in HTTP, HTML, and XFN.

This client was implemented in Python on Google App Engine, source code here:

  http://code.google.com/p/webfingerclient-dclinton/

Anyway, just wanted to throw this idea out there.  I look forward to reading your thoughts and comments.  (And if if people hate it, the client code could certainly be reused for an XRD-based client as well.)

-DeWitt

[Santosh Rajan]

Great piece of work!

Folks will give you the usual arguments we have heard before, against this.

But I guess you made your point.

--
http://santrajan.blogspot.com
http://twitter.com/santoshrajan
http://www.facebook.com/santosh.rajan

[Eran Hammer-Lahav]

Two observations. First, from the experience of getting OpenID support
at Yahoo! (and I believe Google as well), this is a no go since it
means adding stuff to insanely frequented pages when less than
0.00001% will actually care about those links. Second:

On Aug 16, 7:50 pm, DeWitt Clinton <dclin...@gmail.com> wrote:

>    6. The href target of the rel="finger" is parsed as a URL template,

>    substituting in the local-part or addr-spec as necessary

This violates HTML since href is defined as a URI.

EHL

[Will Norris]

If nothing else, this helps demonstrate that perhaps we have two
different use cases here...

1) Mimic the `finger` protocol for email-like identifiers --
discover basic user data (hcard), service profiles (xfn rel="me"),
perhaps contacts (xfn), etc. As Dewitt has demonstrated, this
certainly doesn't the complexity of XRD.

2) Use email-like identifiers to identify users in protocols that
rely on XRD for discovery. OpenID of course is the most obvious, but
there can and most certainly will be more... OAuth, Portable Contacts,
Activity Streams, etc. In order to make email-like identifiers work
in an XRD world, then we absolutely have to figure out how an XRD
descriptor is discovered for that identifier.

I guess the question really is, what is the desired scope of this
particular 'webfinger' project? If the goal is simply to accomplish
#1, that's fine, and Dewitt's proposal will likely work just fine.
But #2 is going to need to be done one way or the other. There has
been talk about using email addresses for OpenID for quite a long
time. The solution is likely going to be profiling LRDD To work with
email-like identifiers. If that work gets rolled into the webfinger
project, then great. If not, that's fine too.

For what it's worth, I'll note that I am not terribly interested in
#1. While it's cool and all, I can't help but see it as a single
service that does something only moderately useful. I'm interested in
seeing #2 happen, since that is building the infrastructure to use
email-like identifiers in countless identity related applications
going forward.

-will

On Aug 16, 2009, at 8:32 PM, Santosh Rajan wrote:

> Great piece of work!Folks will give you the usual arguments we have

>> 1. The client is given an email-like identifier
>> 2. The identifier is parsed into a local-part and a domain per
>> RFC 2822
>> 3. The domain is converted into a host URL at http://{domain}/
>> 4. The host URL is fetched, following redirects, to retrieve an
>> HTML
>> page
>> 5. The HTML page is parsed for the existence of a rel="finger" link
>> element
>> 6. The href target of the rel="finger" is parsed as a URL template,

>> substituting in the local-part or addr-spec as necessary

>> 7. The resultant URL is fetched, following redirects, and parsed

>> for
>> rel="me" anchors and links

>> 8. The links are returned as services associated with the

>> identifier
>>
>> In practice, this approach can be exceeding simple for a provider to
>> implement. In my own case, I simply added the following line of
>> HTML to my
>> homepage:
>>
>> <link rel="finger" href="http://finger.unto.net/search?q={addr-

>> spec}<http://finger.unto.net/search?q=%7Baddr-spec%7D>

[DeWitt Clinton]

Responses inline.

On Sun, Aug 16, 2009 at 10:23 PM, Eran Hammer-Lahav <er...@hueniverse.com> wrote:

Two observations. First, from the experience of getting OpenID support
at Yahoo! (and I believe Google as well), this is a no go since it
means adding stuff to insanely frequented pages when less than
0.00001% will actually care about those links. Second:

But for the 99% case of non-Google, non-Yahoo websites -- okay, we're just making up numbers : ) -- adding discovery mechanisms to HTML is possible, and easy.

 

On Aug 16, 7:50 pm, DeWitt Clinton <dclin...@gmail.com> wrote:

>    6. The href target of the rel="finger" is parsed as a URL template,

>    substituting in the local-part or addr-spec as necessary

This violates HTML since href is defined as a URI.

Mea culpa here.  I sincerely thought link href was under-specified in HTML5 and invalid URIs could squeak through.

But I will challenge the assertion that Will's use-case #1 of looking up services by email is uninteresting.  Given the sudden flood of traffic to the test client by people plugging their gmail addresses and/or adding (invalid!) link rel headers to their page for identity lookup, I think this very simple XFN-based approach may indeed be of interest to some people.

Either way, I will modify my client to support well-known and XRD once the spec is a little more polished.  It's a fine starting place for a test client, and I don't regret at all trying to make a point about use-case #1.  And I'll also continue to push for reuse of XFN (and maybe FOAF) because they already exist in the wild.

-DeWitt

[Kevin Marks]

On Mon, Aug 17, 2009 at 12:05 AM, DeWitt Clinton<dcli...@gmail.com> wrote:
> Responses inline.
>
> On Sun, Aug 16, 2009 at 10:23 PM, Eran Hammer-Lahav <er...@hueniverse.com>
> wrote:
>>
>> Two observations. First, from the experience of getting OpenID support
>> at Yahoo! (and I believe Google as well), this is a no go since it
>> means adding stuff to insanely frequented pages when less than
>> 0.00001% will actually care about those links. Second:
>
> But for the 99% case of non-Google, non-Yahoo websites -- okay, we're just
> making up numbers : ) -- adding discovery mechanisms to HTML is possible,
> and easy.

as for hard-coding mappings from big sites, can't you use the existing
SGNodeMapper to go from mailto: to known profile URLs?

http://code.google.com/p/google-sgnodemapper/

>
>>
>> On Aug 16, 7:50 pm, DeWitt Clinton <dclin...@gmail.com> wrote:
>>
>> >    6. The href target of the rel="finger" is parsed as a URL template,
>> >    substituting in the local-part or addr-spec as necessary
>>
>> This violates HTML since href is defined as a URI.
>
> Mea culpa here.  I sincerely thought link href was under-specified in HTML5
> and invalid URIs could squeak through.
> But I will challenge the assertion that Will's use-case #1 of looking up
> services by email is uninteresting.  Given the sudden flood of traffic to
> the test client by people plugging their gmail addresses and/or adding
> (invalid!) link rel headers to their page for identity lookup, I think this
> very simple XFN-based approach may indeed be of interest to some people.
> Either way, I will modify my client to support well-known and XRD once the
> spec is a little more polished.  It's a fine starting place for a test
> client, and I don't regret at all trying to make a point about use-case #1.
>  And I'll also continue to push for reuse of XFN (and maybe FOAF) because
> they already exist in the wild.

I think the two can coexist, certainly

[Ben Laurie]

On Mon, Aug 17, 2009 at 6:23 AM, Eran Hammer-Lahav<er...@hueniverse.com> wrote:
>
> Two observations. First, from the experience of getting OpenID support
> at Yahoo! (and I believe Google as well), this is a no go since it
> means adding stuff to insanely frequented pages when less than
> 0.00001% will actually care about those links.

This is easily fixed by adding back in SRV support (and illustrates
why you really can't do without a DNS lookup).

[DeWitt Clinton]

One thing I liked about having built a WebFinger-like client on Google App Engine is that it serves as a useful litmus test for what can be accomplished in degraded/sandboxed environments.  Since socket access is likely out of the question in such environments, even if WebFinger does support something like SRV discovery (which in general I'd support) we'd still want some web-only fallback for even SRV.

I realize full well that this circular logic.  I'm not sure how to achieve both goals.

-DeWitt

[Ben Laurie]

On Mon, Aug 17, 2009 at 3:32 PM, DeWitt Clinton<dcli...@gmail.com> wrote:
> One thing I liked about having built a WebFinger-like client on Google App
> Engine is that it serves as a useful litmus test for what can be
> accomplished in degraded/sandboxed environments.  Since socket access is
> likely out of the question in such environments, even if WebFinger does
> support something like SRV discovery (which in general I'd support) we'd
> still want some web-only fallback for even SRV.
> I realize full well that this circular logic.  I'm not sure how to achieve
> both goals.

I don't think you can.

The correct conclusion here, IMO, is that the sandbox is too
restrictive. It should either provide DNS lookup services, or it
should support webfinger itself.

If you argue from the position that we should only ever do things that
fit inside current sandboxes, we are doomed to never move beyond
existing protocols. Which seems like a pretty sucky thing to be
arguing for.

[Ben Laurie]

On Mon, Aug 17, 2009 at 3:54 PM, Ben Laurie<be...@google.com> wrote:
> On Mon, Aug 17, 2009 at 3:32 PM, DeWitt Clinton<dcli...@gmail.com> wrote:
>> One thing I liked about having built a WebFinger-like client on Google App
>> Engine is that it serves as a useful litmus test for what can be
>> accomplished in degraded/sandboxed environments.  Since socket access is
>> likely out of the question in such environments, even if WebFinger does
>> support something like SRV discovery (which in general I'd support) we'd
>> still want some web-only fallback for even SRV.
>> I realize full well that this circular logic.  I'm not sure how to achieve
>> both goals.
>
> I don't think you can.

Actually, I'll take that back. The easy answer is to stand up a look
up service that can be accessed over HTTP somewhere.

I still stand by the below, though.

[Josh Patterson]

I think any real code is good discussion. And you are right, the
masses will bury a discussion in minutia. This is pretty slick and is
basically an instantiation of a Personal Services Index. I'm a fan of
XRD [blog: 1,2], but I think real working examples are the most
critical primordial evolutionary blocks any web tech can have. It
needs to exist at its Simplest Possible Instatiation --- and then move
from there.

Josh Patterson

[1] http://jpatterson.floe.tv/index.php/2009/04/19/the-data-ecology/
[2] http://jpatterson.floe.tv/index.php/2008/11/30/a-road-sign-in-a-digital-world/

>    1. The client is given an email-like identifier
>    2. The identifier is parsed into a local-part and a domain per RFC 2822
>    3. The domain is converted into a host URL at http://{domain}/
>    4. The host URL is fetched, following redirects, to retrieve an HTML page
>    5. The HTML page is parsed for the existence of a rel="finger" link
>    element
>    6. The href target of the rel="finger" is parsed as a URL template,

>    substituting in the local-part or addr-spec as necessary

>    7. The resultant URL is fetched, following redirects, and parsed for

>    rel="me" anchors and links

>    8. The links are returned as services associated with the identifier

>
> In practice, this approach can be exceeding simple for a provider to
> implement.  In my own case, I simply added the following line of HTML to my
> homepage:
>
>   <link rel="finger"

> href="http://finger.unto.net/search?q={addr-spec}<http://finger.unto.net/search?q=%7Baddr-spec%7D>

> " title="WebFinger lookup service"/>
>
> Finger templates can include either 'local-part' (e.g., 'dewitt') or
> 'addr-spec' (e.g. 'dew...@unto.net') as they prefer.
>
> I could have implemented a new service at finger.unto.net to return a full
> document with all of the services associated with the account.  But in my
> case, I already have a XFN based profile at Google, so all I needed to do
> was redirect to it.  I did this with nothing more than a RedirectMatch in my
> httpd.conf.
>

>   RedirectMatch ^/search\?addr-spec=dew...@unto.net$http://www.google.com/profiles/dclinton

[John Panzer]

(A third observation is that going beyond an 80% correct solution for reliably parsing out link@rel="finger" elements from arbitrary HTML with no potential for security issues would probably be more work than the rest of the project put together.)

--
John Panzer / Blogger
jpa...@google.com / abstractioneer.org / @jpanzer

On Sun, Aug 16, 2009 at 10:23 PM, Eran Hammer-Lahav <er...@hueniverse.com> wrote:

[Joseph A Holsten]

DeWitt Clinton supposedly wrote:
...

This sounds very much in line with the thinking for web-linking[1]. I
feel a little funny using a URI-template in an HTML or Atom link.
That's the main advantage to XRD right now. It's interesting that HTML
forms don't have a rel attribute. I'd really love to know if you
think this is the right way to resolve a discovery document after
reviewing the web-linking draft.

If you're going to use a rel value in HTML, please define a profile.
If in XHTML, please give it a full URI.

> Finger templates can include either 'local-part' (e.g., 'dewitt') or
> 'addr-spec' (e.g. 'dew...@unto.net') as they prefer.

+1 to the terms 'local-part' and 'addr-spec'

> I could have implemented a new service at finger.unto.net to return
> a full document with all of the services associated with the
> account. But in my case, I already have a XFN based profile at
> Google, so all I needed to do was redirect to it. I did this with
> nothing more than a RedirectMatch in my httpd.conf.
>
> RedirectMatch ^/search\?addr-spec=dew...@unto.net$ http://www.google.com/profiles/dclinton
>
> *I mentioned that @gmail.com and @friendfeed.com addresses also work
> with the client. I did this by cheating. Both of those domains
> accept incoming email and expose XFN formatted profiles with a well-
> known mapping, so I hardcoded those by hand. I wouldn't recommend
> that clients do this by default, but I did want to show some other
> semi-working examples. Your site can work by following the
> instructions for link rel="finger" as above.

...

Practically, I'm most interested in the final page you're resolving
too. Since finger was designed to be a user-visible tool, not just an
underlying protocol, HTML seems to make a lot of sense. It might be
worth investigating mapping XRD into a microformat so there's some
equivalence between human optimized and tool optimized formats. XRD is
mostly just links and relations anyway.

The open question here seems to be: should webfinger resolve to a
discovery document which has a <link rel="profile"/> or should it
point to a profile with a <link rel="describedby"/>? Either should
solve both of the problems Will Norris mentioned.

1: http://tools.ietf.org/html/draft-nottingham-http-link-header
--
http://JosephHolsten.com

[Brad Fitzpatrick]

Cute, but I can't get behind this.  I love microformats, but parsing them is a nightmare.  (and I speak from some experience, having done the Social Graph API....)

Like John Panzer said, documenting exactly how to format & parse them would be a considerably bigger spec than what we've got now.  Microformats don't tend to exactly specify the large number of edge cases, many of which are inherent to randomly-formatted HTML itself.

Plus UI designers are always breaking sites' microformats when they're copy/pasting template code around which some microformat proponent in the company inserted.

That said, if people start putting XRDs in Microformats or something, I'm sure Google (er, me?) will parse them, because data's data, but I really don't want to advocate that, as I don't wish that pain on anybody else.

Don't worry -- we'll get the webfinger stuff simple enough that given an email address you'll be able to draw a path to an microformat-enabled HTML page pretty easily.  I just don't want to depend on that.

[John Panzer]

(A third observation is that going beyond an 80% correct solution for reliably parsing out link@rel="finger" elements from arbitrary HTML with no potential for security issues would probably be more work than the rest of the project put together.)

--
John Panzer / Blogger
jpa...@google.com / abstractioneer.org / @jpanzer

On Sun, Aug 16, 2009 at 10:23 PM, Eran Hammer-Lahav <er...@hueniverse.com> wrote:

[John Panzer]

My desire is not to have an email vs. url identifier split, but rather to enlarge the domain of identifiers so that they encompass both emails and urls (and potentially other things, like telephone numbers, in the future).  In a way such that it's simple for end users to understand, it's easy for software to deal with, and we share parts of discovery mechanisms where reasonable rather than re-inventing wheels.  Because it'd be silly if we had totally separate, unrelated discovery mechanisms for j...@example.org and example.org/joe -- they're both going to show up as "Joe" in my email and friends lists, and the details of how someone gets a unique identifier, and how software deals with it, should just be part of the common base web infrastructure.

--
John Panzer / Blogger
jpa...@google.com / abstractioneer.org / @jpanzer

 <link rel="finger" href="http://finger.unto.net/search?q={addr-spec}<http://finger.unto.net/search?q=%7Baddr-spec%7D>

[Eran Hammer-Lahav]

This calls for representing this email-like identifier as a URI. URI is the generic identify architecture of the web and we should obey it, not try to create another construct next to it.

 

EHL

[John Panzer]

Agreed that any unified identifier should be a URI.  With the proviso that it should be usable by normal people (the way browsers work properly if people leave off the http: prefix).  A new URI scheme would certainly solve the problem easily:

identifier:j...@example.org
identifier:example.org/joe
identifier:+12125551212

There are merely political issues with actually registering a new URI scheme with IANA, but you don't have to do that to start using one for internal purposes.  If it proves useful and gains traction, then one could go and register it to prevent future collisions.  (http://tools.ietf.org/html/rfc4395.)  But really you don't need to to be useful (see http://en.wikipedia.org/wiki/URI_scheme#Unofficial_but_common_URI_schemes.)

[Eran Hammer-Lahav]

There is no need to replace tel: or http:… we should focus on the local-id/domain pair use case.

[DeWitt Clinton]

Update:

I just updated and republished the client at http://webfingerclient-dclinton.appspot.com/.

This version drops support for the proposed XFN-based format and HTML-based discovery.  Instead it now reads from well-known/host-meta and parses XRD according my best-effort reading of the specifications.

If your host publishes XRD at host-meta then this client should print something out for your account identifier.  Both my and Brad's gmail addresses return something when used as account identifiers.

I'll send feedback about XRD itself in a separate thread.

-DeWitt

[DeWitt Clinton]

There's also a command-line client included in the source distribution.

% svn checkout http://webfingerclient-dclinton.googlecode.com/svn/trunk/ webfingerclient

% cd webfingerclient

% python webfinger.py acct:dcli...@gmail.com

subject {

value: "acct://dcli...@gmail.com" } [...snipped...]

And a protobuf-based python XRD parser:

% python2.5

>>> import urllib >>> import xrd >>> content = urllib.urlopen('http://gmail.com/.well-known/host-meta').read() >>> parser = xrd.Parser() >>> description = parser.parse(content) >>> str(description) 'subject {\n value: "http://gmail.com/"\n}\nlinks {\n relations {\n value: "http://webfinger.info/rel/service"\n }\n uri_templates {\n value: "http://www.google.com/s2/webfinger/?q={%id}"\n }\n}\nlinks {\n relations {\n value: "describedby"\n }\n uri_templates {\n value: "http://www.google.com/s2/webfinger/?q={%id}"\n }\n}\n'

 

Enjoy,

-DeWitt

[Steven Livingstone-Perez]

Nice one.

 

Updated my site so you can now put acct:goldf...@livz.org  into http://webfingerclient-dclinton.appspot.com and get some stuff back about me.

 

Still figuring out what all the endpoints are to do but getting there.

 

Still feel i should be able to enter just http://livz.org as is is *my* site and my host-meta could be found. Equally http://twitter.com/weblivz (if they supported it) could return the same information.

 

steven

acct:goldf...@livz.org

http://livz.org

[Brad Fitzpatrick]

Nice! :-)

And I see I don't need the "acct:" prefix and you do the right thing
and assume email-like is acct:. Perfect!

Simple, readable code too! And nice seeing protos used. :-)