How are we supposed to be handeling HTML responses from the API?

2 views
Skip to first unread message

Andrew

unread,
Aug 18, 2009, 10:37:45 AM8/18/09
to Twitter Development Talk
I get this response quite frequently:

<!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd"> -->
<HTML>
<HEAD>
<META HTTP-EQUIV="Refresh" CONTENT="0.1">
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">
<META HTTP-EQUIV="Expires" CONTENT="-1">
<TITLE></TITLE>
</HEAD>
<BODY><P></BODY>
</HTML>

And other people are getting it as well (
http://groups.google.com/group/twitter-development-talk/browse_thread/thread/acdcb4baf76037c8/e0ac9745e4c9bfe9?lnk=gst&q=refresh#e0ac9745e4c9bfe9
).

What is the recommended way of handling this? I could do an immediate
refresh and hope for the best, which is my initial thought. But the
other question is, why is a request for JSON returning HTML?

The request that seems to be the most problematic for me in this
regard is:
http://twitter.com/statuses/friends.json

I am authenticating via OAuth and not using Curl.

Cameron Kaiser

unread,
Aug 18, 2009, 10:59:32 AM8/18/09
to twitter-deve...@googlegroups.com
> I get this response quite frequently:
>
> <!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"

In general, when TTYtter receives HTML when it expects JSON (and it can
identify it as HTML -- we had an incident last night where it passed a
JSON empty list, followed by an HTML page), it simply ignores it as a glitch
and refetches.

I'm wondering if this has to do with whatever load-balancing Twitter has
in place, but I can't confirm that.

--
------------------------------------ personal: http://www.cameronkaiser.com/ --
Cameron Kaiser * Floodgap Systems * www.floodgap.com * cka...@floodgap.com
-- "Endian Little Hate We" -- credits from Connectix Virtual PC 6 for Mac -----

Rich

unread,
Aug 18, 2009, 11:02:27 AM8/18/09
to Twitter Development Talk
I'm getting the same issue with XML, yet the API is responding with a
200 status OK

On Aug 18, 3:59 pm, Cameron Kaiser <spec...@floodgap.com> wrote:
> > I get this response quite frequently:
>
> > <!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
>
> In general, when TTYtter receives HTML when it expects JSON (and it can
> identify it as HTML -- we had an incident last night where it passed a
> JSON empty list, followed by an HTML page), it simply ignores it as a glitch
> and refetches.
>
> I'm wondering if this has to do with whatever load-balancing Twitter has
> in place, but I can't confirm that.
>
> --
> ------------------------------------ personal:http://www.cameronkaiser.com/--
>   Cameron Kaiser * Floodgap Systems *www.floodgap.com* ckai...@floodgap.com

Ryan Sarver

unread,
Aug 18, 2009, 11:08:48 AM8/18/09
to twitter-deve...@googlegroups.com
Andrew,

Thanks for bumping this up to us. Can you please also provide some
additional data to us (as much as you can) so we can help figure out
what is going on and where it is coming from?

1. The IP of the machine making requests to the Twitter API. If you're
behind NAT, please be sure to send us your *external* IP.
2. The IP address of the machine you're contacting in the Twitter
cluster. You can find this on UNIX machines via the "host" or
"nslookup" commands, and on Windows machines via the "nslookup"
command.
3. The Twitter API URL (method) you're requesting and any other
details about the request (GET vs. POST, parameters, headers, etc.).
4. Your host operating system, browser (including version), relevant
cookies, and any other pertinent information about your environment.
5. What kind of network connection you have and from which provider,
and what kind of network connectivity devices you're using.

Thanks in advance. Best, Ryan

Andrew C.

unread,
Aug 18, 2009, 11:32:01 AM8/18/09
to Twitter Development Talk
Ryan,

I'll put in some additional logging to get the answers to two. The
request I am making gets cached for 24 hours (as to not overload the
Twitter servers) so once it succeeds it could be a while before I hit
the error again. I can try to force it. As far as the rest of it goes:

1. I'll email you the IP address.
2. As above: I will get the answer next time the error happens.
3. The method is statuses/friends.json via GET
4. The host OS is CentOS 5. The connection is being made via a TCP/IP
socket. OAuth is used for user authentication. No cookies are stored.
5. The machine is in the Amazon Cloud. Since it is a VM I can't know
what the underlying network hardware is.

A side note regarding #2: one of the biggest problems with this error
is that it is sporadic. But the application loses significant
functionality when it does happen. So if the problem can't be fixed a
good alternative is to have a reliable way to recover from the problem
so that the app can return to it's functional state.

Thanks,
- Andrew

On Aug 18, 11:08 am, Ryan Sarver <rsar...@twitter.com> wrote:
> Andrew,
>
> Thanks for bumping this up to us. Can you please also provide some
> additional data to us (as much as you can) so we can help figure out
> what is going on and where it is coming from?
>
> 1. The IP of the machine making requests to the Twitter API. If you're
> behind NAT, please be sure to send us your *external* IP.
> 2. The IP address of the machine you're contacting in the Twitter
> cluster. You can find this on UNIX machines via the "host" or
> "nslookup" commands, and on Windows machines via the "nslookup"
> command.
> 3. The Twitter API URL (method) you're requesting and any other
> details about the request (GET vs. POST, parameters, headers, etc.).
> 4. Your host operating system, browser (including version), relevant
> cookies, and any other pertinent information about your environment.
> 5. What kind of network connection you have and from which provider,
> and what kind of network connectivity devices you're using.
>
> Thanks in advance. Best, Ryan
>
> On Tue, Aug 18, 2009 at 7:37 AM, Andrew<andrewcuri...@gmail.com> wrote:
>
> > I get this response quite frequently:
>
> > <!-- <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
> > "http://www.w3.org/TR/html4/strict.dtd"> -->
> > <HTML>
> > <HEAD>
> > <META HTTP-EQUIV="Refresh" CONTENT="0.1">
> > <META HTTP-EQUIV="Pragma" CONTENT="no-cache">
> > <META HTTP-EQUIV="Expires" CONTENT="-1">
> > <TITLE></TITLE>
> > </HEAD>
> > <BODY><P></BODY>
> > </HTML>
>
> > And other people are getting it as well (
> >http://groups.google.com/group/twitter-development-talk/browse_thread...
Reply all
Reply to author
Forward
0 new messages