API Limit of 150 is Obsolete
Eric Woodward
limit to properly implement Twitter within a desktop client
application given the addition of: 1) three retweets timelines; 2)
checking the account's saved searches; and 3) up to 10-20 Twitter
Lists timelines.
Twitter Lists alone are causing real problems if a user follows more
than 5 or so. We cant poll Twitter List subscriptions with one API
call that combines them altogether, which we could then split apart
client-side with some attached meta-data. That alone would have been a
big help, and without it we are left polling each List as if it was a
separate timeline, since that is what they are.
Implementing proper Lists management is a non-starter within this
limit, so is regular confirmation of a relationship between two users
when asked for by the user (on Lists or search results). There is
simply a lot of stuff I cannot do properly that is standard on
twitter.com, all because I am subject to the API limit while
twitter.com is not. Users simply do not understand this distinction in
possibilities.
I would like to formally ask on behalf of all client developers that
the API limit increase to 250, from 150, for all applications whether
they use OAuth or HTTP Basic Authentication. We are simply not able to
implement Twitter properly within a limit of 150, but dont need a lot
more, only another 100-200 API calls or so.
If Twitter can even technically contemplate a 10x API limit increase
to 1,500 for OAuth applications, surely an increase to 250 based on
the addition of core features like official retweets and Lists is a
reasonable request. A limit of 150 is simply obsolete, and has been
for a long time.
I do not want to wait for the UX repairs around OAuth for desktop
applications, and I dont like being forced into OAuth sooner than we
are ready just because we need the extra API hits just to do basic
features properly. And besides, that was announced as two weeks away
three weeks ago. I dont want to wait any longer. I want to properly
implement the basics, like Lists polling, now.
This is a considered email because I care about the quality of our
Twitter implementation and I care about the Twitter ecosystem. I would
appreciate a considered reply.
--ejw
Eric Woodward
Email: e...@nambu.com
Josh Roesslein
addition of new API features.
I would almost like a solution where twitter sets a "guaranteed"
hits/hour soft limit.
By soft limit I mean if you go above this limit you "may" be rate
limited if the twitter cluster
is currently under heavy load or you are being too rough with the API.
If the cluster has unused capacity,
why start limiting users? For non-whitelisted applications a guarantee
of 250 would be nice. Whiltelisted apps
would get a higher guaranteed limit still to meet their demands.
I'm sure twitter has floated this idea around. Not sure how big of a
technical hurdle it would be to implement.
Just my two cents on the subject of API rate limits.
Josh
M. Edward (Ed) Borasky
cursors within a "followers_ids" or "friends_ids" call, if that helps.
But I really think that increasing the API rate limit for basic HTML
auth is a bad idea - let's make oAuth work!
Jaanus
at this point, so everybody is capped at 150.
To inform the discussion, I wonder if Twitter could share any figures
like what's the actual API use distribution? Like what combination of
users/apps hit the cap regularly and cause massive load? If it was an
equal distribution (i.e most users/apps are around the same level)
that gives them heavy load, then I would see why they need to be
careful about raising limits (any increase would bring more fail
whales). But I suspect that it's highly asymmetrical... i.e there are
very few users/apps who actually cause any meaningful load.
Another hunch: desktop apps are negligible and the real load comes
from web apps who spider asynchronously 24/7. Should the load be
differentiated across client and web apps? Client apps are typically
only one user per device at a time, whereas the web app may be
spidering on behalf of who knows how many people.
Cameron Kaiser
> from web apps who spider asynchronously 24/7. Should the load be
> differentiated across client and web apps? Client apps are typically
> only one user per device at a time, whereas the web app may be
> spidering on behalf of who knows how many people.
The problem here is distinguishing the two. OAuth doesn't (and I was
told this by one of the people on the OAuth committee) specifically
allow you to unambiguously and securely identify an application just
because it has a certain app key, and Twitter's Basic Auth implementation
uses source keys pretty much purely cosmetically.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- BOND THEME NOW PLAYING: "Die Another Day" ----------------------------------
M. Edward (Ed) Borasky
On Jan 20, 4:50 pm, Cameron Kaiser <spec...@floodgap.com> wrote:
> The problem here is distinguishing the two. OAuth doesn't (and I was
> told this by one of the people on the OAuth committee) specifically
> allow you to unambiguously and securely identify an application just
> because it has a certain app key
Huh? Can you translate this into either English or pseudo-code? I fill
out a form. The app gets a name, which must be unique. And I choose
between a desktop exclusive-or server app (PIN workflow exclusive-or
callback workflow) with a radio button. I get a consumer key and
consumer secret, also, I'm assuming, unique.
So now I run that app. I send packets back and forth between the app /
my IP address and Twitter's servers / Twitter's IP addresses. Are you
saying Twitter can't distinguish my oAuth app running on my IP address
from another oAuth app running on a different IP address? You don't
know where I am and what I'm running? You don't know which of 30 users
of my app from different machines is acting abusively?
Abraham Williams
Abraham Williams | Moved to Seattle | May cause email delays
Project | Intersect | http://intersect.labs.poseurtech.com
Hacker | http://abrah.am | http://twitter.com/abraham
This email is: [ ] shareable [x] ask first [ ] private.
Sent from Seattle, WA, United States
Cameron Kaiser
> > > from web apps who spider asynchronously 24/7. Should the load be
> > > differentiated across client and web apps? Client apps are typically
> > > only one user per device at a time, whereas the web app may be
> > > spidering on behalf of who knows how many people.
> >
> > The problem here is distinguishing the two. OAuth doesn't (and I was
> > told this by one of the people on the OAuth committee) specifically
> > allow you to unambiguously and securely identify an application just
> > because it has a certain app key, and Twitter's Basic Auth implementation
> > uses source keys pretty much purely cosmetically.
>
> 20k IPs making 5 API hits each...
But it's not a guaranteed one either. Also, it doesn't allow you to
distinguish between equally popular services, one which might be kosher
and the other not.
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- BOND THEME NOW PLAYING: The James Bond Theme from "Dr. No" -----------------
Cameron Kaiser
> > told this by one of the people on the OAuth committee) specifically
> > allow you to unambiguously and securely identify an application just
> > because it has a certain app key
>
> Huh? Can you translate this into either English or pseudo-code? I fill
> out a form. The app gets a name, which must be unique. And I choose
> between a desktop exclusive-or server app (PIN workflow exclusive-or
> callback workflow) with a radio button. I get a consumer key and
> consumer secret, also, I'm assuming, unique.
Didn't we just get done with a thread where people complained, correctly,
that a desktop app containing a consumer key/secret can't keep those
things secret?
--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- Of course, what I really want is total world domination. -- Linus Torvalds -
Aki
possible. I am not able to roll out an update for my desktop Twitter
client, simply because the API limit is too low for the features I
have implemented. The new version of my client utilizes Lists related
APIs, which consumes a lot of API calls. Moreover, multiple API calls
are currently required to get a chain of replies (conversation), which
consumes significant number of API calls.
The previously discussed features were implemented, because there was
an official announcement to change the API limit to 1500. I know it is
expected to happen in "few weeks", but it would be good to know the
estimated date for this, so the developers can make appropriate
decisions on the application deployment timeline.
-- Aki