Snowflake, it's almost 9007199254740992 time.

[Matt Harris]

Hey everyone,

You may remember a few weeks ago we launched Snowflake having encouraged you all to check your code to make sure you were able to handle the larger numbers it will generate. For those of you whose code couldn't handle the longer numbers we created String versions of the IDs in our JSON responses, identified by an "_str" at the end of their name - for example the Tweet ID in the JSON response exists twice: once as a number (id) and once as a string (id_str). For API requests which returned arrays of IDs we added the parameter "stringify_ids" to force all IDs to Strings. For example:

    https://api.twitter.com/1/followers/ids.json?cursor=-1&stringify_ids=1

We're sending this reminder because at 2.14pm PDT (10.14pm UTC) this Sunday, 28th November 2010 Snowflake IDs will reach 53bits.

Only Tweet IDs are generated by Snowflake. This means only Tweets, Retweets, Mentions and Replies are affected this weekend. Things like Saved Searches, Users and Direct Messages are not Snowflaked.

If you haven't converted your code to use the String version you should do so immediately. Once the IDs reach 53bits Javascript, and some other languages, misrepresent the numbers. As an example: 2**53 = 9007199254740992

Representing this in Javascript gives....

> (9007199254740992).toString()

"9007199254740992"

> (9007199254740993).toString()

"9007199254740992"

> (9007199254740994).toString()

"9007199254740994"

> (9007199254740995).toString()

"9007199254740996"

> (9007199254740999).toString()

"9007199254741000"

You can see in this example that the Tweet IDs are being misrepresented in their converted state. We've provided String versions of all of our IDs, even those which are not using Snowflake IDs. We've done this to make it easier for you to convert your code. Even if your code can handle the longer numbers you may want to convert to Strings anyway. Doing so will reduce the risk of problems should you extend your code with a language or library that doesn't support >53bit numbers.

If you are using Javascript you may find the following code samples helpful. They were put together by our web team as an example approach to the problem of capturing the String version of the IDs, and sorting them.

The first gist looks for the new *_str field and uses it if it's there.  If it's missing, the original field is used but stringified first. This doesn't make IDs >53bit safe for you but but does mean you can use String IDs for all other attributes without having to check for them first.

The second code sample using the library natcompare.js which performs 'natural order' comparisons of strings in JavaScript. It was written by Kristof Coomans of the SCK-CEN (Belgian Nucleair Research Centre). 

    http://gist.github.com/637624

There has been some great discussion about this in the developer forums, including some questions and answers about the change. You can read more here:

    http://groups.google.com/group/twitter-development-talk/browse_thread/thread/6a16efa375532182/c30b408a6c09e3d1

Best,

@themattharris

Developer Advocate, Twitter

[Kevin Watters]

Beware the natural order comparison code linked above.
natcompare(10705970246197248, 9999625058521088) returns -1
incorrectly. See https://gist.github.com/727383 for an example.

> http://groups.google.com/group/twitter-development-talk/browse_thread...

[Matt Harris]

Thanks for pointing this out Kevin. I've passed this onto the engineers for review.

Best
@themattharris
Developer Advocate, Twitter
http://twitter.com/themattharris

--
Twitter developer documentation and resources: http://dev.twitter.com/doc
API updates via Twitter: http://twitter.com/twitterapi
Issues/Enhancements Tracker: http://code.google.com/p/twitter-api/issues/list
Change your membership to this group: http://groups.google.com/group/twitter-development-talk

[themattharris]

Hi Kevin,

Thanks again for pointing this out. We've updated the gist with a fix
for the issue you identified:
https://gist.github.com/637624

Best,
@themattharris

On Dec 3, 3:23 pm, Matt Harris <thematthar...@twitter.com> wrote:
> Thanks for pointing this out Kevin. I've passed this onto the engineers for
> review.
>
> Best
> @themattharris

> Developer Advocate, Twitterhttp://twitter.com/themattharris

>
> On Fri, Dec 3, 2010 at 11:10 AM, Kevin Watters <kevinwatt...@gmail.com>wrote:
>
>
>
>
>
>
>
> > Beware the natural order comparison code linked above.
> > natcompare(10705970246197248, 9999625058521088) returns -1

> > incorrectly. Seehttps://gist.github.com/727383for an example.