link wrapping on the API

689 views
Skip to first unread message

Raffi Krikorian

unread,
Jun 8, 2010, 6:57:21 PM6/8/10
to twitter-deve...@googlegroups.com, twitter-ap...@googlegroups.com
hi all.

twitter has been wrapping links in e-mailed DMs for a couple months now. with that feature, we're trying to protect users against phishing and other malicious attacks. the way that we're doing this is that any URL that comes through in a DM gets currently wrapped with a twt.tl URL -- if the URL turns out to be malicious, Twitter can simply shut it down, and whoever follows that link will be presented with a page that warns them of potentially malicious content. in a few weeks, we're going to start slowly enabling this throughout the API for all statuses as well, but instead of twt.tl, we will be using t.co.

practically, any tweet that is sent through statuses/update that has a link on it will have the link automatically converted to a t.co link on its way through the Twitter platform. if you fetch any tweet created after this change goes live, then its text field will have all its links automatically wrapped with t.co links. when a user clicks on that link, Twitter will redirect them to the original URL after first confirming with our database that that URL is not malicious.  on top of the end-user benefit, we hope to eventually provide all developers with aggregate usage data around your applications such as the number of clicks people make on URLs you display (it will, of course, be in aggregate and not identifiable manner). additionally, we want to be able to build services and APIs that can make algorithmic recommendations to users based on the content they are consuming. gathering the data from t.co will help make these possible.

our current plan is that no user will see a t.co URL on twitter.com but we still have some details to work through. the links will still be displayed as they were sent in, but the target of the link will be the t.co link instead. and, we want to provide the same ability to display original links to developers. we're going to use the entities attribute to make this possible.

let's say i send out the following tweet: "you have to check out http://dev.twitter.com!"

a returned (and truncated) status object may look like:

{
  "text" : "you have to check out http://t.co/s9gfk2d4!",
  ...
  "user" : {
    "screen_name" : "raffi",
    ...
  },
  ...
  "entities" : {
    "urls" : [
      {
        "url" : "http://t.co/s9gfk2d4",
        "display_url" : "http://dev.twitter.com",
        "indices" : [23, 43]
      }
    ],
    ...
  },
  ...
}

two things to note: the text of the returned status object doesn't have the original URL and instead it has a t.co URL, and the entities block now has a display_url attribute associated with it. what we're hoping is that with this data, it should be relatively easy to create a UI where you replace the http://t.co/s9gfk2d4 in the text with the equivalent of


this means the user would not see the t.co link, but we all can still provide the protection and gather data from the wrapped link. for the applications that don't choose to update, the t.co link will be shown (and the goal to protect users will be met). i just want to emphasize -- we really do hope that you all render the original URL, but please send the user through the t.co link.   if you do choose to prefetch all the URLs on a timeline, then, when a user actually clicks on one of the links, please still send him or her through t.co. We will be updating the TOS to require you to check t.co and register the click.

related to this: the way the Twitter API counts characters is going to change ever so slightly. our 140 characters is now going to be defined as 140 characters after link wrapping. t.co links are of a predictable length -- they will always be 20 characters. after we make this live, it will be feasible to send in the text for a status that is greater than 140 characters. the rule is after the link wrapping, the text transforms to 140 characters or fewer. we'll be using the same logic that is in twitter-text-rb to figure out what is a URL.

look for an update to dev.twitter.com where we'll have a best practices document on how to use these t.co links.

what's the timeline?  "soon" we'll enable this on @twitterapi, @rsarver, @raffi, and a few other test accounts so you all have live data to play with.  on the timescale of weeks (to potentially a month or two), we'll roll this out to everybody.

of course, if there are any questions, just feel free to direct them to @twitterapi!


--
Raffi Krikorian
Twitter Platform Team
http://twitter.com/raffi
Reply all
Reply to author
Forward
0 new messages