Connectivity to Twitter API completely gone for 1 of my apps

[JH]

As of yesterday my app, ticker-app, can no longer connect to Twitter's API.  It has worked fine for over a year.  As of yesterday any urlfetch call to twitter.com times out with DeadlineExceeded.  However, using other app ID's I have they can still urlfetch to twitter.  I have to assume it has something to do with which data center ticker-app is hosted in?

[Jeff Schnitzer]

You are most likely right. Given the shared pool of IPs, they could
get blacklisted at any time:

https://dev.twitter.com/docs/rate-limiting

This is going to become a progressively more serious problem as time
goes on (GAE becomes more successful, useful public services add rate
limiting). Star this issue:
http://code.google.com/p/googleappengine/issues/detail?id=6644

For now it's probably best to make calls to external services through
a proxy with a dedicated IP address you control. I'd love to hear
suggestions for "cheapest possible solution" - $11/mo for a 256MB
rackspacecloud instance is as good as I can come up with.

Jeff

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/eMHHC7P3PtUJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

[JH]

I don't want to get excited prematurely but it appears to be working again.  Also, note this is not a rate-limiting issue.  I am authenticating with oauth.  When Twitter rate limits your IP, which they do for all app engine IP's I've ever used, they will let you know you are rate limited with an error response.

[Jeff Schnitzer]

On Fri, Aug 24, 2012 at 12:19 PM, KC <demi...@gmail.com> wrote:
> We've been having exactly the same problem problem in one of our apps in GAE
> since yesterday.
> We are using Twitter4J to connect to Twitter, and receiving errors like:
> "Could not fetch URL: https://api.twitter.com/oauth/request_token"
> It worked perfectly for more than 1 year. Any clue how to tackle this?

You need to proxy any calls to services which have rate limits (or
might have rate limits in the future).

Jeff

[Jeff Schnitzer]

On Fri, Aug 24, 2012 at 1:00 PM, JH <ja...@mhztech.com> wrote:
> I don't want to get excited prematurely but it appears to be working again.
> Also, note this is not a rate-limiting issue. I am authenticating with
> oauth. When Twitter rate limits your IP, which they do for all app engine
> IP's I've ever used, they will let you know you are rate limited with an
> error response.

According to https://dev.twitter.com/docs/rate-limiting :

----
Blacklisting

We ask that you honor the rate limit. If you or your application
abuses the rate limits we will blacklist it. If you are blacklisted
you will be unable to get a response from the Twitter API.
----

Sounds like your problem.

Jeff

[Bryce Cutt]

Related to discussion on Twitter Dev board:

https://dev.twitter.com/discussions/3218