Help! Did something change with the API?

[DustyReagan]

Hi,

I have 2 apps http://FriendOrFollow.com (I haven't changed the code on
this site in weeks) and http://FeaturedUsers.com (uses the Zend
Framework to access Twitter). Both of these sites are using the same
authentication and are giving me the error "Unable to Connect to
tcp://twitter.com:80. Error #110: Connection timed out."

I've been checking my rate limit status quite a bit, and it doesn't
seem to shift below 20k for some unknown reason. My rate limit right
now is 19998 because I manually hit "http://twitter.com/statuses/
followers.xml" twice, just to see if the API was working.

Did I miss a vital update to the API or something? What could be
happening, that my apps are broken, but I can still manually hit the
API?

Thanks!

Dusty

[DustyReagan]

PS. I'm using Media Temple to server my sites. Could the IP Address be
blocked or something?

On Feb 11, 3:27 pm, DustyReagan <dustyrea...@gmail.com> wrote:
> Hi,
>
> I have 2 appshttp://FriendOrFollow.com(I haven't changed the code on
> this site in weeks) andhttp://FeaturedUsers.com(uses the Zend

[Matt Sanford]

Hi Dusty,

    The timeout error sounds suspiciously like a network problem and not a rate limit issue. When you say you tested the API manually, did you do it from your servers? Also, if you can let me know the IP address I can check if it is blocked for some reason.

Thanks;

  — Matt Sanford

[DustyReagan]

I *think* it's 72.47.224.154 (FriendOrFollow.com) & 72.47.224.157
(FeaturedUsers.com)

On Feb 11, 3:32 pm, Matt Sanford <m...@twitter.com> wrote:
> Hi Dusty,
>
>      The timeout error sounds suspiciously like a network problem and  
> not a rate limit issue. When you say you tested the API manually, did  
> you do it from your servers? Also, if you can let me know the IP  
> address I can check if it is blocked for some reason.
>
> Thanks;
>    — Matt Sanford
>
> On Feb 11, 2009, at 01:29 PM, DustyReagan wrote:
>
>
>
> > PS. I'm using Media Temple to server my sites. Could the IP Address be
> > blocked or something?
>
> > On Feb 11, 3:27 pm, DustyReagan <dustyrea...@gmail.com> wrote:
> >> Hi,
>

> >> I have 2 appshttp://FriendOrFollow.com(Ihaven't changed the code on
> >> this site in weeks) andhttp://FeaturedUsers.com(usesthe Zend

[DustyReagan]

Oh. I tested the API manually from home. Just typed the address in my
browser.

On Feb 11, 3:32 pm, Matt Sanford <m...@twitter.com> wrote:

> Hi Dusty,
>
>      The timeout error sounds suspiciously like a network problem and  
> not a rate limit issue. When you say you tested the API manually, did  
> you do it from your servers? Also, if you can let me know the IP  
> address I can check if it is blocked for some reason.
>
> Thanks;
>    — Matt Sanford
>
> On Feb 11, 2009, at 01:29 PM, DustyReagan wrote:
>
>
>
> > PS. I'm using Media Temple to server my sites. Could the IP Address be
> > blocked or something?
>
> > On Feb 11, 3:27 pm, DustyReagan <dustyrea...@gmail.com> wrote:
> >> Hi,
>

> >> I have 2 appshttp://FriendOrFollow.com(Ihaven't changed the code on
> >> this site in weeks) andhttp://FeaturedUsers.com(usesthe Zend

[Alex Payne]

Matt will be conctacting you off-list. For future reference if others
run into this issue:
http://apiwiki.twitter.com/FAQ#IsmyIPbannedorblacklisted

--
Alex Payne - API Lead, Twitter, Inc.
http://twitter.com/al3x

[DustyReagan]

That's for the quick feedback guys!

Is there any way to warn a poor guy when an IP range he's on is about
to get blocked? My sites are important to me, get a decent amount of
traffic, and make revenue. I got punished due to someone else's crime.
*I'm not trying to play the violin over here, but this kinda' sucks.*

Is the only safe course to host on a private dedicated server?

On Feb 11, 4:09 pm, Alex Payne <a...@twitter.com> wrote:
> Matt will be conctacting you off-list. For future reference if others
> run into this issue:http://apiwiki.twitter.com/FAQ#IsmyIPbannedorblacklisted
>
>
>
> On Wed, Feb 11, 2009 at 13:45, DustyReagan <dustyrea...@gmail.com> wrote:
>
> > Oh. I tested the API manually from home. Just typed the address in my
> > browser.
>
> > On Feb 11, 3:32 pm, Matt Sanford <m...@twitter.com> wrote:
> >> Hi Dusty,
>
> >>      The timeout error sounds suspiciously like a network problem and
> >> not a rate limit issue. When you say you tested the API manually, did
> >> you do it from your servers? Also, if you can let me know the IP
> >> address I can check if it is blocked for some reason.
>
> >> Thanks;
> >>    — Matt Sanford
>
> >> On Feb 11, 2009, at 01:29 PM, DustyReagan wrote:
>
> >> > PS. I'm using Media Temple to server my sites. Could the IP Address be
> >> > blocked or something?
>
> >> > On Feb 11, 3:27 pm, DustyReagan <dustyrea...@gmail.com> wrote:
> >> >> Hi,
>

> >> >> I have 2 appshttp://FriendOrFollow.com(Ihaven'tchanged the code on

[Matt Sanford]

Hi Dusty,

    We've seen a few different people on shared hosting services run into problems where they are blocked in the aftermath of some other application. Without your own IP address we really can't tell you apart so you do run the risk of being blocked if you happen to share an IP with a service attempting to spam us or crack passwords. We try to help everyone out but at the end of the day user security and keeping the system up out weigh everything else. It sucks that we have to block people, I'm in total agreement. Finding contacts for an IP range is difficult and waiting on a reply while being attacked isn't totally practical. The only way to be sure this doesn't effect you is to have a dedicated IP address.

Thanks;

  — Matt Sanford

"I will strike down upon thee with great vengeance and furious anger those who would attempt to poison and destroy my brothers"

[Jay]

Hi Alexa,
I have the same problem with Dusty. I am also using Media Temple
(mediatemple.net). I think the IP address is 72.47.224.142.
I try " curl http://twitter.com", there is no response, but ping
works.

Are all mt users blocked by twitter.com?

Jay

On Feb 11, 5:09 pm, Alex Payne <a...@twitter.com> wrote:
> Matt will be conctacting you off-list. For future reference if others
> run into this issue:http://apiwiki.twitter.com/FAQ#IsmyIPbannedorblacklisted
>
>
>
> On Wed, Feb 11, 2009 at 13:45, DustyReagan <dustyrea...@gmail.com> wrote:
>
> > Oh. I tested the API manually from home. Just typed the address in my
> > browser.
>
> > On Feb 11, 3:32 pm, Matt Sanford <m...@twitter.com> wrote:
> >> Hi Dusty,
>
> >>      The timeout error sounds suspiciously like a network problem and
> >> not a rate limit issue. When you say you tested the API manually, did
> >> you do it from your servers? Also, if you can let me know the IP
> >> address I can check if it is blocked for some reason.
>
> >> Thanks;
> >>    — Matt Sanford
>
> >> On Feb 11, 2009, at 01:29 PM, DustyReagan wrote:
>
> >> > PS. I'm using Media Temple to server my sites. Could the IP Address be
> >> > blocked or something?
>
> >> > On Feb 11, 3:27 pm, DustyReagan <dustyrea...@gmail.com> wrote:
> >> >> Hi,
>

> >> >> I have 2 appshttp://FriendOrFollow.com(Ihaven'tchanged the code on

[Jeremy Darling]

Try switching out to MOSSO instead of MediaTemple.  About the same price, better customer service, and its on the cloud instead of a grid system.  We have been using them for a while now and are quite happy.  In fact I'm busy moving all of our clients from MT over to MOSSO.  With all the problems MT has been having I wouldn't be surprised if its on their end and not on Twitters end.

 Jeremy

[JeffC]

The fact that you rate limit by IP address seems to be a fundamental
problem. Wouldn't this be alleviated by introducing some kind of API
key that uniquely identifies the actual application with each call ?
You could keep the existing structure for 'unsigned' calls and let
people who really care sign up, get a key, and use it with all their
API calls.

My apologies if this is naive, impractical, or already discussed
elsewhere in this group. I don't have much experience in this area but
it seems like an obvious solution.

Jeff Clark

> >>>>>> I have 2 appshttp://FriendOrFollow.com(Ihaven'tchangedthe code  

[Matt Sanford]

Hi Jeff,

    This error is unrelated to rate limiting and is instead a network level block to prevent the selection of attacks they were running. This is also a block of a range of IP addresses because the attacker was coming from multiple IPs in the same range. We have to deflect attacks with the tools we have, and right now a network block is that tool. We're waiting on the Media Template abuse group to get back to us before we can unblock it.

Thanks;

  — Matt Sanford

[Duane Storey]

Matt,

I wrote a popular WordPress plugin for Twitter, and currently the
Media Temple blocking is impacting us. All of our servers run on
Media Temple, and currently we do not have the ability to test the
plugin or to release new updates because our servers (which ironically
host the plugin for download) can't access your API due to mass
blocking of a media temple cluster from your side. Obviously we can
set up a test environment somewhere else, but I don't think your
solution to the problem is adequate, and it's hurting our ability to
release plugins which people use to interface with Twitter. If we
can find another solution to this problem, it would be appreciated, as
I don't think mass blocking IP addresses is a good way to go as it
results in issues like these for people on shared hosting.

Regards,
Duane Storey

[Matt Sanford]

Hi Duane,

    I'm sorry you've been caught in the crossfire but until Media Temple's abuse department let's us know they've handled this we can't unblock the IP range. The best thing I can suggest is to contact Media Temple and let them know that not taking action on this is affecting you.

    I think everyone agrees that IP range blocking like this is a bad thing to have to do. Unfortunately it's the only method that our operations team has found to be effective. I have an operations background and I've not seen any other solution to stop this type of behavior other than contacting the hosting provider's abuse department. Withstanding an attack while you wait on a hosting provider with whom you have no agreement is not really an ideal solution.

    Again, to Duane and everyone else caught in the crossfire, I'm sorry that we have to block IPs like this. I know this is hurting people's applications and development time but there does not seem to be a better solution at the moment. If you have a suggestion of a better way to block these types of attacks please email me off-list and I'll be happy to read them. In the mean time I am going to stop replying to individual messages in this thread. I will update everyone once we have some resolution.

Thanks;

  — Matt Sanford

[Jesse Stay]

I'm not affected, but I understand where these guys are coming from.  Could you guys at least put out an announcement or a blog post when these things happen?  Media Temple isn't just a small service - I'm sure there are still lots that don't even realize they're affected yet.

And I second the thought that when your IP address gets banned notifying those whitelisted users that their address was banned or disabled for whatever reason.  While I enjoy free service, I'd pay for that level of service, and would build a business model to support it.  Remember that when our apps are blocked, so are our users, and those users are also Twitter users.

Thanks,

Jesse Stay

[Rod]

Jeremy, based on what Matt has said, MOSSO-hosted apps would be just
as vulnerable to this sort of banning as an MT-hosted account. Though
perhaps MOSSO would be quicker to get on it from the abuse handling
side...

On Feb 11, 7:16 pm, Jeremy Darling <jeremy.darl...@gmail.com> wrote:
> Try switching out to MOSSO instead of MediaTemple.  About the same price,
> better customer service, and its on the cloud instead of a grid system.  We
> have been using them for a while now and are quite happy.  In fact I'm busy
> moving all of our clients from MT over to MOSSO.  With all the problems MT
> has been having I wouldn't be surprised if its on their end and not on
> Twitters end.
>
>  Jeremy
>

> On Wed, Feb 11, 2009 at 5:03 PM, Jay <jieqiw...@gmail.com> wrote:
>
> > Hi Alexa,
> > I have the same problem with Dusty. I am also using Media Temple
> > (mediatemple.net). I think the IP address is 72.47.224.142.

> > I try " curlhttp://twitter.com", there is no response, but ping

> > > >> >> I have 2 appshttp://FriendOrFollow.com(Ihaven'tchangedthe code on

[Gilles Frydman]

In view of the problem you/we experienced how about requiring all those who are whitelisted to connect via a secure socket layer, either via SSH or SSL?

[wrigh...@gmail.com]

Thanks for the info Matt!

I'm on Media Temple and this is affecting my app as well :(

It's good to know you guys are on it.

John

[Randy Tayler]

I've sent a message to MediaTemple to hound them a bit... I'd rather
not have to switch hosting services.

[Matt Sanford]

Hi all,

While we had heard nothing back from Media Temple before it seems
that having customers (you all) contact them has changed that. Thank
you economy 101, thank you. Our operations staff is now in touch with
the Media Temple staff to try and find out who the offender was that
caused all of this mess in the first place and stop them. I'll send
more updates and information becomes available.

Thanks;
— Matt

[Duane Storey]

Thanks Matt. Please let us know the outcome.

Regards,
Duane

[Rod]

Really, this can't be the first time this has happened. Is this really
the best, or only, way to deal with this problem?

[Abraham Williams]

A possible solution is to add a method to the API that lets
applications register their IP with Twitter. IPs could be run through
the list before Twitter blacklists them and contact the Twitter
account holder that authenticated when registering the IP. Application
could check once a day to see if their IP has changed and if so update
the IP registered with Twitter.

--
Abraham Williams | http://the.hackerconundrum.com
Web608 | Community Evangelist | http://web608.org
This email is: [ ] blogable [x] ask first [ ] private.

[M Handa]

Hello everyone. I work in the (mt) Media Temple Abuse Department. I
just wanted people to know that we were provided with very little
information to help us narrow down the "basic auth attempts" that were
causing Twitter problems and forced them to block a range of IPs
belonging to some web nodes. As of last night, at approximately 6:45pm
we were unable to find the source of the issue as it had most likely
subsided or stopped all together. These types of attacks are very
common, and as mentioned already, the quickest solution is to simply
ban the source. The attacks are generally an exploited server or site
that is running through a list of common user names and passwords,
then move on.
Generally, when these types of attacks occur, the source of the attack
is notified, and provided a snippet of logs that displays times the
attacks occur, source IP, useragent string, or any information of that
nature. Because we have a large client base that uses Twitter
legitimately, it makes it difficult to find a source based simply on
connections to twitter. We have created a tool that should help us
track down this type of problem in the future, if we know when it's
occurring. The checking that we have in place needs to be manually
implemented, so we will be making a conscious effort to be aware of
any notifications from Twitter moving forward.
A notification of our progress with the issue and request to remove
the blocked IP range was sent to multiple recipients at Twitter about
10:30 this morning. We have yet to hear anything in return regarding
this issue with them. We would like nothing more than to clear this up
as quickly as possible with them, people at (mt) use Twitter to. :)
M Handa