Twitter search fails intermittently

Roger Ertesvag

unread,

Sep 8, 2009, 12:39:43 PM9/8/09

to twitter4r-users

Hi,
I have some problems using Twtter4R to search. Here is a method I have
in one of the models:

def self.test
client = Twitter::Client.from_config("#{RAILS_ROOT}/config/
twitter.yml")
messages = client.search(:q => "#test", :rpp => 50)
end

When I run this in the console on my server it will randomly work or
fail the first 3-4 times, after that it works fine. When it fails I
get the following:

/opt/local/lib/ruby/gems/1.8/gems/rails-2.3.4/lib/commands/runner.rb:
48: /opt/local/lib/ruby/gems/1.8/gems/json-1.1.9/lib/json/common.rb:
122:in `parse': 618: unexpected token at '<!DOCTYPE html PUBLIC "-//
W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/1999/REC-
html401-19991224/strict.dtd">
(JSON::ParserError)

<HTML>
<HEAD>
<META HTTP-EQUIV="Refresh" CONTENT="0.1">
<META HTTP-EQUIV="Pragma" CONTENT="no-cache">
<META HTTP-EQUIV="Expires" CONTENT="-1">
<TITLE></TITLE>
</HEAD>
<BODY><P></BODY>
</HTML>
' from /opt/local/lib/ruby/gems/1.8/gems/json-1.1.9/lib/json/common.rb:
122:in `parse'
from /opt/local/lib/ruby/gems/1.8/gems/mbbx6spp-twitter4r-0.4.0/lib/
twitter/client/search.rb:24:in `search'
from /home/gmn6mfaa/apps/ffnor/releases/20090908150950/app/models/
message.rb:14:in `search'
from (eval):1
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:
31:in `eval'
from /opt/local/lib/ruby/gems/1.8/gems/rails-2.3.4/lib/commands/
runner.rb:48
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:
31:in `gem_original_require'
from /opt/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:
31:in `require'
from /home/gmn6mfaa/apps/ffnor/current/script/runner:3

I realize that the problem is probably in mys server rather than the
gem itself, but hopefully someone can point me in the right direction
on where to start trouble shooting. Like for instance, it seems from
the error message that the request returns an empty HTML document,
what might that mean?

Regards
Roger

Susan Potter

unread,

Sep 9, 2009, 11:00:58 AM9/9/09

to twitter4r-users

Hi Roger,

I'll have to look into this further before I can be sure, but it looks
like a handful (or just one) of the servers on the twitter.com side
are responding with HTML instead of JSON when we send the Accept HTTP
header with json.

I'll look into it either tonight or tomorrow night after work and get
back to you.

Thanks for reporting it,
Susan

Susan Potter

unread,

Sep 9, 2009, 3:12:38 PM9/9/09

to twitter4r-users

Hi Roger,

This *might* also be the cause, but detail from Twitter still to come:
https://twitter.com/twitterapi/statuses/3869228579

Again once I get out of client work mode today, I will looking into
this further as well.

Best,
Susan

Roger Ertesvag

unread,

Sep 10, 2009, 1:54:23 AM9/10/09

to twitter4r-users

Hi Susan,
thanks for getting back to me.

I agree, it looks more like a twitter problem than a problem with the
gem, I'll keep an eye on their announcements.

So if I understand correctly, the empty HTML document must come from
Twitter, its not something that the gem could create internally if
there is a network problem?

Regards
Roger

S Potter

unread,

Sep 10, 2009, 10:44:13 AM9/10/09

to twitter...@googlegroups.com

Roger,

> So if I understand correctly, the empty HTML document must come from
> Twitter, its not something that the gem could create internally if
> there is a network problem?

That is correct. I actually expect JSON results from the server and
the only HTML in the codebase is in the web directory that I upload to
Rubyforge.

Susan

Reply all

Reply to author

Forward