Hashing standard for URLs to find the Twitter version of shortened URLs

[Bjoern]

Hi,

this is maybe a bit random, but I feel like throwing the idea out
there for fun. It was suggested in a recent discussion thread that to
get the Twitter variant of an URL, one could just post the URL to
Twitter and see what Twitter makes of it.

Since it is infeasible to generate a lot of URLs that way, here is a
variant: what if along with posting the URL to twitter, one would also
post a short hash of the URL. The hash function would be a standard
everybody agrees on. Then to find the Twitter variant of a shortened
URL, one could search Twitter for the hash of that URL. So you would
not have to post all URLs yourself, you could also benefit from other
people having "Twitter-Shortened" the URL before. (Searching for the
hash might bring up multiple results, as Twitter does not always
shorten the URL - sometimes multiple tries might be necessary).

In fact if such a scheme was in place, it would also give people a way
to "officially" link to a site. They could add the hash of the
destination URL in their tweet and become searchable. I realize that
would probably be too geeky for widespread adaption, but in theory I
like the idea ;-)

Björn

[Matt Sanford]

Hi Björn,

Your proposal works if everybody plays by the rules but I think
email spam has taught us that's an unrealistic expectation. Think of
shortening malwareurl.com via Bit.ly and then including the hash for
the URL to a popular YouTube video. Applications searching for the
YouTube video would find the tweet, provide it to users, and infect
them. Shorteners have the general problem of not knowing the
destination but I think a hash created by the same person who might be
trying to trick you in the first place is unreliable. Just a thought.

Thanks;
— Matt Sanford / @mzsanford

[Bjoern]

On Jul 17, 4:44 pm, Matt Sanford <m...@twitter.com> wrote:
>      Your proposal works if everybody plays by the rules but I think  
> email spam has taught us that's an unrealistic expectation. Think of  
> shortening malwareurl.com via Bit.ly and then including the hash for  
> the URL to a popular YouTube video. Applications searching for the  
> YouTube video would find the tweet, provide it to users, and infect  
> them. Shorteners have the general problem of not knowing the  

Good call, however it would be necessary in any case to compare the
URLs found via the scheme with the original URL. So the process would
be to search Twitter for the URLs via the hash tag, then resolve those
URLs to their final destination and check if it is indeed the right
URL. That would be necessary anyway because a hashing standard for
URLs could not guarantee uniqueness of hashes. URL shorteners have it
easier because they can take of uniqueness via their database.

Björn

[Nick Arnett]

On Fri, Jul 17, 2009 at 5:50 AM, Bjoern <bjoe...@googlemail.com> wrote:

In fact if such a scheme was in place, it would also give people a way
to "officially" link to a site. They could add the hash of the
destination URL in their tweet and become searchable. I realize that
would probably be too geeky for widespread adaption, but in theory I
like the idea ;-)

This issue goes well beyond Twitter.  Those of us who have created any sort of URL tracking and measurment application would benefit from it. There's great value, I am certain, in being able to identify, as close to real-time as possible, URLs that are being cited by a lot of people (or by influencers/opinion leaders, etc.)  Each cite is a signifcant "vote" for the page and when it occurs in real-time media (v. static web pages), it provides a relevance metric that Google and its competitors aren't touching yet.

This seemed to be worth a blog post:

http://www.nickarnett.net/2009/07/17/whats-really-wrong-with-url-shorteners/

Nick