How does account/rate_limit_status.json work?

[Reinier Zwitserloot]

#1: Every Xth minute of each hour, this counter resets back to
20/30/70/the limit du jour.

#2: It's the limit du jour minus 1 for each call you made in the 60
minutes prior to the call ('moving target').

if #1, it would be -really- nice if the # of milliseconds until the
counter resets is included.

[Reinier Zwitserloot]

I hit get Limit: 20
I hit get direect messages, then limit: 19
I hit direct messages again, then limit: 20. It's 16:57 or so, perhaps
the server reset its counters.
I hit direct messages twice, then hit limit: still 20.
A few minutes later, for good measure, another direct messages, and a
getReplies too just for good measure, then hit limit: Still 20.


What the heck? I can't actually use the value limit returns for
anything useful this way; I'd rather that it just reflect the hourly
limit instead instead of this mess.

[Ed Finkler]

Search the archives.

--
Ed Finkler
http://funkatron.com
AIM: funka7ron
ICQ: 3922133
Skype: funka7ron

[Reinier Zwitserloot]

The only relevant item I can find is this:

http://groups.google.com/group/twitter-development-talk/browse_thread/thread/fe3192a45bcbc3f5/c6abafa2f7c92f8b

which asks the same question - and hasn't been answered.

If you mean: Twitter is still reporting the hourly cap instead of the
remaining hits, where'd that 19 come from?

On Jun 24, 7:21 pm, "Ed Finkler" <funkat...@gmail.com> wrote:
> Search the archives.
>
> --

> Ed Finklerhttp://funkatron.com

> AIM: funka7ron
> ICQ: 3922133
> Skype: funka7ron
>
> On Tue, Jun 24, 2008 at 11:01 AM, Reinier Zwitserloot
>

[Ed Finkler]

On Wed, Jun 25, 2008 at 1:35 AM, Reinier Zwitserloot <rein...@gmail.com> wrote:
>
> The only relevant item I can find is this:
>
> http://groups.google.com/group/twitter-development-talk/browse_thread/thread/fe3192a45bcbc3f5/c6abafa2f7c92f8b
>
> which asks the same question - and hasn't been answered.

Indeed, it seems like the status hasn't changed.

--
Ed Finkler

[Reinier Zwitserloot]

"fixed" it by setting up a second dummy account and polling its rate
limit to future proof me against 'remaining_hits' ever does what it
suggests. Meh.

On Jun 25, 3:44 pm, "Ed Finkler" <funkat...@gmail.com> wrote:

> On Wed, Jun 25, 2008 at 1:35 AM, Reinier Zwitserloot <reini...@gmail.com> wrote:
>
> > The only relevant item I can find is this:
>

> >http://groups.google.com/group/twitter-development-talk/browse_thread...

>
> > which asks the same question - and hasn't been answered.
>
> Indeed, it seems like the status hasn't changed.
>
> --

> Ed Finklerhttp://funkatron.com

[Mark Armendariz]

It seems to be counting down correctly now (not sure about directs). I
Second the request for a "reset_timestamp" of some sorts so we can use
the remaining hits number properly.

[Horst Gutmann]

Would it perhaps be also possible to include the current absolute
ratelimit in that response? Something like

{
"remaining_hits":19,
"max_hits":20
}

Best regards, Horst

[Reinier Zwitserloot]

Running through various use case scenarios in my head, the most useful
set of return values is:

remaining_hits, and millis_until_reset (NOT reset_timestamp). Having
twitter respond with just the number of milliseconds until the next
reset avoids clock syncing issues - what if my server's clock is 3
minutes off? It would also cause universal havoc if twitter's clock is
off. You'll also run into big trouble if your timezones aren't
configured correctly. Using millis_until_reset avoids the mess
entirely. Also, millis_until_reset can safely be squashed into a 32-
bit int, whereas timestamps only fit in 64-bit IEEE floating points
and 64-bit integers. Okay, that last one isn't very important :)

Use Cases:

1. active counter in a client-side app using the API. Needs:
remaining_hits, millis_until_reset. max_hits could help, by allowing
clients to assume the next hour will have the same limit as the
current hour, at least until a call to rate_limit is made.

2. deciding poll frequency. This needs reset_timestamp and
remaining_hits as well: For servers or clients that want to expend,
say, 75% of the limit on polling, they would have to use the formula:
(remaining_hits * .75) / (millis_until_reset) to figure out how many
milliseconds to wait in between each call.

3. knowing when to stop bothering. Lets say there's a queue of tweets
to send via the API, then you don't really need rate_limit at all;
just keep sending until you hit the proper HTTP error, then try to
send every 15 minutes until it works again.

On Jun 27, 2:53 pm, "Horst Gutmann" <horst.gutm...@gmail.com> wrote:
> Would it perhaps be also possible to include the current absolute
> ratelimit in that response? Something like
>
>     {
>         "remaining_hits":19,
>         "max_hits":20
>     }
>
> Best regards, Horst
>

[Alex Payne]

How about something like this? In XML:

<?xml version="1.0" encoding="UTF-8"?>
<hash>
<remaining-hits type="integer">19</remaining-hits>
<hourly-limit type="integer">20</hourly-limit>
<reset-time-in-seconds type="integer">1214598988</reset-time-in-seconds>
<reset-time type="datetime">2008-06-27T13:36:28-07:00</reset-time>
</hash>

...and in JSON:

{
"remaining_hits":19,
"hourly_limit":20,
"reset_time_in_seconds":1214598988,
"reset_time":"Fri Jun 27 13:36:28 -0700 2008"
}

--
Alex Payne
http://twitter.com/al3x

[Marco Kaiser]

+1

Sounds good to me, all relevant information included.

Sent from my iPhone

[Folkert Hielema]

+1 That would be pretty cool, thanks.

Folkert Hielema

[Stut]

As Reinier said earlier it would be better if the reset-time-in-
seconds field gave the time until the reset rather than a timestamp.
However, I don't think that needs to be in milliseconds considering
the (normal) allowed requests is only just over 1 per minute.

-Stut

--
http://stut.net/

[Kevin Tronkowski]

Alex,

Super!

Thanks,
Kevin

> Alex Paynehttp://twitter.com/al3x- Hide quoted text -
>
> - Show quoted text -

[Mark Armendariz]

+1 Perfect!

On Jun 27, 3:48 pm, "Alex Payne" <a...@twitter.com> wrote:

[Mark Armendariz]

+1 Perfect!

On Jun 27, 3:48 pm, "Alex Payne" <a...@twitter.com> wrote:

[Alex Payne]

We'll try to get this shipped early next week.

--
Alex Payne
http://twitter.com/al3x

[Reinier Zwitserloot]

What's the point of listing time to next update as a formatted
timestamp and as a time in seconds? A formatted or in-seconds absolute
timestamp + a time remaining would be nice.

I mentioned milliseconds because most of the world works in epoch
millis, so its convenient. Seconds is just as good, really. Whatever's
convenient.

[Mark Armendariz]

Either one works for me. For twitter apps, I tend to work in GMT
timestamps - since tweets are sent with GMT stamps, but anything
denoting "time until next reset" will be great. Anything is better
than having to track the reset manually (which is how I'm currently
handling it). Well, not "manually", but through code and
remaining_hits tracking to figure out when it last reset and then an
approximate "next reset".

[twibble]

would it make sense to add that information in a response to a status
message? Might save some extra round trips and hits on the twitter API
given twitter clients will poll the values after requesting status
messages.
As for the parameters the reset-time should be in UTC to be consistent
across the API. Not sure if useful anyway since the reset-time-in-
seconds seems to be sufficient.

Best,
http://twitter.com/twibble

[Horst Gutmann]

Perfect, thank you very much :D

[Michael Pelz-Sherman]

This is definitely cool, but is it actually working?

I'm getting negative numbers for remaining hits (what does that mean?), and getting "Rate limit exceeded" errors from timeline requests even though my request rate is nowhere near 20/hr.

Right now I can't get timelines from the API, only from the web site.

Also: is that reset time really in seconds, or is it milliseconds?

- mps

[Gerard van Enk]

On 30 jun, 00:31, Michael Pelz-Sherman <mpelzsher...@gmail.com> wrote:
> This is definitely cool, but is it actually working?
>
> I'm getting negative numbers for remaining hits (what does that mean?), and getting "Rate limit exceeded" errors from timeline requests even though my request rate is nowhere near 20/hr.
>
> Right now I can't get timelines from the API, only from the web site.
>

I'm having the same problem, negative numbers, very large reset-time-
in-seconds (1214802098) and rate limit exceeded :(

Gerard

[Damon Clinkscales]

Alex said they would try to get it in this coming week, so what you
read earlier in this thread was just a spec. So no, it's not supposed
to be working as specified just yet.

Secondly, I think that it was supposed to be milliseconds, even though
it was referred to as seconds. Otherwise, none of us will be
programming the Twitter API by the time it resets. :)

-damon

--
http://twitter.com/damon

[Craig Hockenberry]

Damon, I think what Michael is describing is a bug with rate limiting
in general.

I've had users report that they are rate limited after a long period
of inactivity on Twitter. I've seen this issue on a test account --
one that had not been in use for several days. Was getting 403 on the
timeline when I definitely shouldn't have.

It took awhile (a couple hours?) for the situation to correct itself.
I reported it to Alex.

Unfortunately, only a small number of users are affected with this
transient problem, so it's very hard to track down...

-ch

[Damon Clinkscales]

Craig,

Ah, ok. I've seen this with my own account in Twitterific as well.

Thanks,
-damon

[Michael Pelz-Sherman]

Thanks Craig. Yes, there does indeed seem to be a bug with rate limiting. 

Anything we can do to help y'all track it down, twitter guys?

- Michael

http://twitter.com/mpelzsherman

----- Original Message ----
From: Craig Hockenberry <craig.ho...@gmail.com>
To: Twitter Development Talk <twitter-deve...@googlegroups.com>
Sent: Monday, June 30, 2008 10:36:46 AM
Subject: Re: How does account/rate_limit_status.json work?

Damon, I think what Michael is describing is a bug with rate limiting
in general.

I've had users report that they are rate limited after a long period
of inactivity on Twitter. I've seen this issue on a test account --
one that had not been in use for several days. Was getting 403 on the
timeline when I definitely shouldn't have.

It took awhile (a couple hours?) for the situation to correct itself.
I reported it to Alex.

Unfortunately, only a small number of users are affected with this
transient problem, so it's very hard to track down...

-ch

[Reinier Zwitserloot]

The 'reset time in seconds' thing is working fine, it's just useless
information.

Take jan 1st 1970, add the provided number of seconds to this, and
you'll end up at reset_time. Why this information is duplicated is
beyond me.

Alex and crew: Please remove rtis and add 'time_until_next_reset',
which is a number in seconds or milliseconds. Thanks!

On Jun 30, 5:11 pm, "Damon Clinkscales" <sca...@pobox.com> wrote:
> Craig,
>
> Ah, ok.  I've seen this with my own account in Twitterific as well.
>
> Thanks,
> -damon
>
> On Mon, Jun 30, 2008 at 9:36 AM, Craig Hockenberry
>

[Alex Payne]

Given the disparity of date-handling mechanisms in various languages
and libraries, I figured it would be handy to return the timestamp for
when the rate limit will be reset both in a human-readable format and
the lowest-common-denominator machine-friendly format, Unix epoch
time. That's what reset_time_in_seconds is.

I've rarely seen time/date libraries that can easily covert from
milliseconds without a bunch of multiplication or nested calls, but
most every language has a one-shot call from a epoch time integer to a
native time/date format.

Are we on the same page, or are you asking for something different?
I'm not sure why the number of seconds until the rate limit is reset
is more inherently useful than a timestamp. It seems like a client
could easily calculate the difference between the current time and the
reset-time timestamp if necessary.

--
Alex Payne
http://twitter.com/al3x

[Stut]

And what happens if the client's clock is totally wrong? It simply
won't work as expected.

I really think it needs to be "the number of seconds until the reset"
- that would be far more reliable for client use.

-Stut

--
http://stut.net/

[Alex Payne]

I can see both being useful. If it's okay with y'all, I'll provide
the timestamp in epoch time as well as the number of seconds until the
reset.

[Reinier Zwitserloot]

That'd be great. Thanks!

[Stut]

Sounds good, cheers.

-Stut

--
http://stut.net/

[Michael Pelz-Sherman]

Yeah, sure, that's great. Thanks! 

(You may want to consider renaming reset-time-in-seconds to "reset-epoch" or something like that.)

Now then, what about this business of some people (myself included) getting negative values for "remaining-hits", even after long inactive periods?

I'm also getting 0 for reset time

Here's the current response I'm getting (when I don't get a "whale" that is):

<?xml version="1.0" encoding="UTF-8"?>

<hash>

  <hourly-limit type="integer">20</hourly-limit>

  <reset-time-in-seconds type="integer">0</reset-time-in-seconds>

  <reset-time nil="true"></reset-time>

  <remaining-hits type="integer">-14</remaining-hits>

</hash>

And sure enough, my friends_timeline.xml requests are being denied.

What gives? Is there a better place than here for reporting API bugs? (Like maybe a bug tracking system?)

Thanks,

- Michael

----- Original Message ----
From: Alex Payne <al...@twitter.com>
To: twitter-deve...@googlegroups.com
Sent: Monday, June 30, 2008 7:01:35 PM
Subject: Re: How does account/rate_limit_status.json work?

[Alex Payne]

Contacting me off-list with detailed bug reports is the best way to
resolve issues like this. The rate limiting code has changed under
the hood since the last time developers reported a similar bug.

[Alex Payne]

There's a slight monkey-wrench in this plan. Due to the newfound
popularity of this method, we need a caching strategy for it.
Returning times relative to the time of request means that caching is
essentially impossible. So, I'm not sure if we'll be able to return
the extra data.

[Ed Costello]

Might consider setting the Expires: header to the same time returned in the request.
I tried doing a modified-GET request using the Etag: and found that I get a different Etag on each request.  Although the values in the returned JSON are the same, they are in a different order with each request.

e.g. I received:
{"hourly_limit":20,"reset_time_in_seconds":1215118965,"reset_time":"Thu Jul 03 21:02:45 +0000 2008","remaining_hits":10}
and
{"reset_time_in_seconds":1215118965,"reset_time":"Thu Jul 03 21:02:45 +0000 2008","remaining_hits":10,"hourly_limit":20}

in response to two requests about ten seconds apart (if it helps they appear to have come from different servers, twitter-web048.twitter.com for the first request and twitter-web001.twitter.com for the second.

-- 

-ed costello

[Ryan Rosario]

I am really confused. I have been doing queries to this file for the
past 20 mins using cURL and it keeps returning the same value for
time_until_next_reset. Is this a bug, or is this value only updated
every once in a while?

- Ryan

On Jun 30, 9:27 am, Michael Pelz-Sherman <mpelzsher...@gmail.com>
wrote:

[Evan Weaver]

Not a bug. As far as I understand it, it resets 60 minutes after the
first request since the last reset, and both time formats reported are
absolute, not relative.

Evan

--
Evan Weaver