Status ID closing in on maximum unsigned integer

[Matt Sanford]

Hi all,

Just a quick heads up. Our status id is fast approaching the
maximum unsigned integer value. Based on all of the feedback on
scaling I've received from this list I'm assuming everyone will be
fine. One thing to check is your table definitions … no amount of
interpreted language will save you from that. I think the tipping
point will be this afternoon sometime but with the planned maintenance
it might end up being later in the day.

Thanks;
— Matt Sanford / @mzsanford

[Matt Sanford]

Make that *signed* integer. Writing too many emails at once, sorry.

[Chad Etzel]

I'm assuming this is just a friendly warning to developers... and that
the status IDs will continue to monotonically increase past (2^31 -
1), i.e. not rollover?

-Chad

[Dossy Shiobara]

When you say "integer" what are you referring to? I'm _assuming_ 32-bit
integers, not 64-bit integers, because ILP-32 is so commonplace.

When you're about to breach the 2^64 in your ID space, please give us a
heads-up around 2^63 so we can upgrade our systems. :)

--
Dossy Shiobara | do...@panoptic.com | http://dossy.org/
Panoptic Computer Network | http://panoptic.com/
"He realized the fastest way to change is to laugh at your own
folly -- then you can let go and quickly move on." (p. 70)

[Matt Sanford]

Hi Chad,

They will continue to increment. This was, as you assumed, just a
friendly reminder.

Thanks;
– Matt Sanford / @mzsanford

Twitter Dev

[Matt Sanford]

Hi Dossy,

I did indeed mean 32-bit signed integer. 64 bit is increasingly
common but so are people defining table using INTEGER or INT(10) or
whatever their database calls it. When we breach 2^64 I'm not telling
anyone … it seems to create more work for me when I send out friendly
warnings :)

Thanks;
– Matt Sanford / @mzsanford

Twitter Dev

See Also: http://twitter.com/dougw/status/1741587950

[Nick Arnett]

On Wed, May 13, 2009 at 7:40 AM, Matt Sanford <ma...@twitter.com> wrote:

Make that *signed* integer. Writing too many emails at once, sorry.

That would be the MySQL (and any 32-bit OS) maximum signed integer, or 2,147,483,647, I'm assuming.  So if we're using an unsigned INT in our tables, we're fine for a while (months?) - status IDs will always be positive, right?  And BIGINT, especially unsigned, is safe for a long, long time?

Just curious... does this mean Twitter really is closing in on 2 billion tweets, or were some IDs skipped?  Is there a prize for whoever posts tweet number 2 billion?  ;-)

Nick

[Matt Sanford]

Hi Nick,

    As mentioned to Dossy (probably still wending its way thought Google Groups) I did mean 32-bit signed integers. Unsigned ints in MySQL will be fine but the rails default is int(10) if I recall which would be a problem. As mentioned to Chad (Google Groups seems to be slow delivering my messages :/) status IDs will continue to increment as positive numbers. BIGINT and unsigned number should be fine for the time being, yes.

    You know we don't comment on the number of users and tweets … but you almost tricked me. Sneaky, sneaky.

Thanks;

 – Matt Sanford / @mzsanford

     Twitter Dev

[Chad Etzel]

On Wed, May 13, 2009 at 11:08 AM, Matt Sanford <ma...@twitter.com> wrote:
> When we breach 2^64 I'm not telling anyone … it seems to
> create more work for me when I send out friendly warnings :)

Well, if you *did* send out a warning at 2^64, it would mean either:

a) you have found the fountain of youth
b) you're a zombie
c) you wrote the most awesome cron job ever
d) you have a Delorean

-Chad

[Andrew Badera]

or e) Twitter becomes even more massively popular while the world also
undergoes a birth rate revoluion.

Thanks-
- Andy Badera
- and...@badera.us
- Google me: http://www.google.com/search?q=andrew+badera
- This email is: [ ] bloggable [x] ask first [ ] private

[Matt Sanford]

Have you not heard, we're tweeting from space these days. Birth rate
changes are now considered optional.

No Humans Required™
— Matt

[Nick Arnett]

On Wed, May 13, 2009 at 8:19 AM, Matt Sanford <ma...@twitter.com> wrote:

    You know we don't comment on the number of users and tweets … but you almost tricked me. Sneaky, sneaky.

Actually, I did not know that.  I might have known if there were an API call to get the total number of tweets, which returned NULL every time.  Perhaps you'll want to add that... ;-)

I still think there should be a prize.

I see that the product manager/APIs position is still on your site... does that mean the position is still open?  Does teasing the API team help as a qualification?  

Nick 

[Cameron Kaiser]

> I see that the product manager/APIs position is still on your site... does
> that mean the position is still open? Does teasing the API team help as a
> qualification?

It may get you an interview, but only so they can surreptitiously get a
photo and send it to the nice Italian men with the kneecappers.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- They told me I was gullible ... and I believed them. -----------------------

[Matt Sanford]

Quick update …

While looking at the code that reminded me of this error I see it
had some bugs of its own. We seem to have a matter of weeks rather
than days before this change. Mobile developers and other who deride
our lack of early notice take heed … now is the time to go unsigned.

Thanks;
– Matt Sanford / @mzsanford
Twitter Dev

[Craig Hockenberry]

Let me be the first to say THANK YOU for this advance notice. I found
and fixed some bugs in our iPhone client today because of it — and I'm
very happy to hear that we have some time to get the code approved in
the App Store.

We love to complain when the process goes awry, but it's also
important to remember to give credit when it's well and truly due.
Thanks again!

-ch

On May 13, 12:12 pm, Matt Sanford <m...@twitter.com> wrote:
> Quick update …
>
>      While looking at the code that reminded me of this error I see it  
> had some bugs of its own. We seem to have a matter of weeks rather  
> than days before this change. Mobile developers and other who deride  
> our lack of early notice take heed … now is the time to go unsigned.
>
> Thanks;
>   – Matt Sanford / @mzsanford
>       Twitter Dev
>
> On May 13, 2009, at 10:49 AM, Cameron Kaiser wrote:
>
>
>
>
>
> >> I see that the product manager/APIs position is still on your  
> >> site... does
> >> that mean the position is still open?  Does teasing the API team  
> >> help as a
> >> qualification?
>
> > It may get you an interview, but only so they can surreptitiously  
> > get a
> > photo and send it to the nice Italian men with the kneecappers.
>
> > --
> > ------------------------------------ personal:http://www.cameronkaiser.com/
> >  --

> >  Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com

[Martin Dufort]

And you can check on the progress of that Status ID on: www.twitpocalypse.com
Unless Twitter decides to blacklist us) for sending too many requests
to the public timeline API...

So Twitter, if you do so, please send me an email martin [insert at]
wherecloud [dot] com so we can work something out :-)

Martin - www.wherecloud.com


On May 13, 9:07 pm, Craig Hockenberry <craig.hockenbe...@gmail.com>
wrote:

[Patrick Burrows]

Hilarious!

--
Patrick Burrows
http://Categorical.ly (the Best Twitter Client Possible)
@Categorically

[mwm]

I'm using a bigint (20) with MySQL for TwitteReader so
18446744073709551615 available. Still many IDs to go ;)