Changing domains for image hosting

[Doug Williams]

Folks --
We are going to be moving images to a new domain (twimg.com) to streamline our image hosting and offer better performance. We hope this will have limited impact as will only change the image URL. Example URLs include:

Profile images:

http://s3.amazonaws.com/twitter_production/profile_images/35240332/2929920.gif

http://a0.twimg.com/profile_images/35240332/2929920.gif

Background images:

http://s3.amazonaws.com/twitter_production/profile_background_images/18156348/jessica_tiled.jpg.jpeg

http://a0.twimg.com/profile_background_images/18156348/jessica_tiled.jpg.jpeg

Thanks,
Doug

[Wynn Netherland]

Hi, Doug,

Thanks for the heads up and all the hard work to keep us in the loop.

Any plans to offer something like:

http://twimg.com/pengwynn so people can always have my latest avatar regardless of the filename?

Thanks,

Wynn Netherland

@pengwynn

--
Wynn Netherland
twitter: pengwynn

[Lachlan Hardy]

Got a rough timeline for this change, Doug?

[Doug Williams]

At the moment there is no internal interest in offering a programmatic URL scheme to locate user images. One of the easiest cache breaking strategies is to change the URL for the image which works across browsers, applications, etc., which is why we like the unique URLs. Additionally, we want to ensure that clients have to go through the API to get image data; we do not want to end up a harvesting target.

We recommend clients serve images through a local cache when possible. As new API data enters the system, the client should periodically perform a lookup on the URL-based hash in the image cache. In the event that the URL-based hash is not valid cache key, the client can assume the image has not been loaded or has changed. This should trigger the client to update the cached version of the avatar or background image. This straight forward strategy to allows clients to control the quality of service and ensures timely image updates.

The change should come within the next day or two. It is owned by another team and they are being pretty aggressive about moving forward with the deploy.

Thanks,
Doug

[Doug Williams]

Andrew Stone asked a great question off list and I wanted to share the answer here.

Images will be hosted from a number of sub-domains of twimg.com. Therefore you should not expect all images to be served from a0.twimg.com as in the examples provided, but instead should expect an aribitrary sub-domain (*.twimg.com).

Thanks,

Doug

[Clint Shryock]

I'm still having issues with the profile_image_url attribute of users/show updating in a timely fashion, any chance this issue be resolved with this update?

+Clint

[Hayes Davis]

Are you going to shut off the S3 URLs immediately or will they still be accessible for a time? I understand that newly updated avatar images would not be updated at S3 after the switch but I'd like to know when we can expect any S3 URLs we may have in our caches to all go dark.

Thanks.

Hayes

[Doug Williams]

I've asked the operation's teams to leave the old S3 hosted images available for 2 weeks after the deploy. This should give you time to update your caches.

Cheers,
Doug

[roamlog]

it seems there are some issues with the new pics server
the "https" is failed, please check it..

e.g. :
http://a0.twimg.com/profile_background_images/18156348/jessica_tiled.jpg.jpeg
is ok

but

https://a0.twimg.com/profile_background_images/18156348/jessica_tiled.jpg.jpeg
is failed..

[Hayes Davis]

I really appreciate it, Doug. Thanks.

Hayes

[gotwalt]

Any chance you'll enable SSL access of these images? At @cotweet we
rely on s3's SSL when pages are being served via HTTPS (login, account
changes, permissions, etc) in order to prevent security zone issues in
IE. I'd imagine that other web-based clients may face similar issues.

Aaron

[Toucan]

Same here and it's already breaking our app today. We either give
users the browser mixed-ssl error (bad) or re-write to SSL which makes
images spin forever which also seems to break some javascript loads on
FF (also bad). Since Toucan runs inside salesforce, we are always on
SSL for every page.

Can twitter turn on SSL for twimg, even if to buy time while we figure
it out? One day notice really isn't a lot of time to develop a fix and
try to get everyone to re-install their app....especially since for
those of us that must serve every page with SSL, there is no good fix.

We really appreciate your help!
Diane

[Abraham Williams]

A temporary fix is to run a proxy on your own SSL domain and pull the photos through there.

Abraham

--
Abraham Williams | Community Evangelist | http://web608.org
Hacker | http://abrah.am | http://twitter.com/abraham
Project | http://fireeagle.labs.poseurtech.com
This email is: [ ] blogable [x] ask first [ ] private.

[Matt Sanford]

Hi all,

    I'm working with our ops folks now to get SSL fixed on twimg.com. No need for running proxies … we're working on it.

Thanks;

 – Matt Sanford / @mzsanford

     Twitter Dev

[Doug Williams]

All --
We are rolling back the twimg.com change because a number of issues became apparent when we moved it into production -- one being the lack of SSL support in the caching layer that many have noted. We do hope to make this change rather soon, but we have problems to fix before moving forward with this change.

Updates to come as we learn about them. Thanks for your patience, here. Twitter devs rock.

Doug