Upcoming changes to the way status IDs are sequenced

333 views
Skip to first unread message

Taylor Singletary

unread,
Mar 26, 2010, 8:41:30 PM3/26/10
to twitter-development-talk, twitter-api-announce
Hi Developers,

It's no secret that Twitter is growing exponentially. The tweets keep coming with ever increasing velocity, thanks in large part to your great applications.

Twitter has adapted to the increasing number of tweets in ways that have affected you in the past: We moved from 32 bit unsigned integers to 64-bit unsigned integers for status IDs some time ago. You all weathered that storm with ease. The tweetapoclypse was averted, and the tweets kept flowing.

Now we're reaching the scalability limit of our current tweet ID generation scheme. Unlike the previous tweet ID migrations, the solution to the current issue is significantly different. However, in most cases the new approach we will take will not result in any noticeable differences to you the developer or your users.

We are planning to replace our current sequential tweet ID generation routine with a simple, more scalable solution. IDs will still be 64-bit unsigned integers. However, this new solution is no longer guaranteed to generate sequential IDs.  Instead IDs will be derived based on time: the most significant bits being sourced from a timestamp and the least significant bits will be effectively random. 

Please don't depend on the exact format of the ID. As our infrastructure needs evolve, we might need to tweak the generation algorithm again.

If you've been trying to divine meaning from status IDs aside from their role as a primary key, you won't be able to anymore. Likewise for usage of IDs in mathematical operations -- for instance, subtracting two status IDs to determine the number of tweets in between will no longer be possible.

For the majority of applications we think this scheme switch will be a non-event. Before implementing these changes, we'd like to know if your applications currently depend on the sequential nature of IDs. Do you depend on the density of the tweet sequence being constant?  Are you trying to analyze the IDs as anything other than opaque, ordered identifiers? Aside for guaranteed sequential tweet ID ordering, what APIs can we provide you to accomplish your goals?

Taylor Singletary
Developer Advocate, Twitter
Reply all
Reply to author
Forward
0 new messages