handling timeout errors in mechanize

Skip to first unread message

converging rails

Feb 25, 2008, 6:38:41 AM2/25/08
to WWW::Mechanize users
Hi all,
For the past 2 days, I was stuck with the problem of
handling timeout exceptions in mechanize. I have read many pags over
google where I was suggested to use the Timeout::timeout module
method but that too doesnt work. Timeout::timeout() method is unable
to rescue these exceptions in Mechanize.

here goes my code:

require 'timeout'
require 'rubygems'
require 'mechanize'

agent = WWW::Mechanize.new
Timeout::timeout(10) do
agent.get('http://www.r-knowsys.com') #this url doesn't exist
rescue Timeout::Error
puts "timeout the page doesnt exist"

How do I handle such a case? Any help appreciated...


Kim Ryan

Jul 24, 2013, 1:13:44 AM7/24/13
to www-mecha...@googlegroups.com, bagam....@gmail.com
I'm getting the same issue. I have to make thousand of requests to a web site, and every 500 or so I get a time out. IS there a way of increasing the timeout periods. It would be great if mech->get could stope the program from dying on a timeout or GET error.
Also, I'm assuming Mechanize is just using the default 180 seconds that the underlying LWP get call makes. Would be nice if this could be altered/

Meir Guttman

Jul 24, 2013, 10:05:54 AM7/24/13
to www-mecha...@googlegroups.com

Hi Kim

Yes, it can be done and one way to do that is when you create a mechanize object:


  my $mech = WWW::Mechanize->new(

    conn_cache => $conn_cache,

    cookie_jar => $$cookie_jar_ref,

    onerror    => \&do_next,

    timeout    => 30,


There are more properties one can set this way. The doc to look all these is one of the LWP packages docs, since Mechanize inherits these from LWP.



You received this message because you are subscribed to the Google Groups "WWW::Mechanize users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to www-mechanize-u...@googlegroups.com.
To post to this group, send email to www-mecha...@googlegroups.com.
Visit this group at http://groups.google.com/group/www-mechanize-users.
For more options, visit https://groups.google.com/groups/opt_out.

No virus found in this message.
Checked by AVG - www.avg.com
Version: 2013.0.3349 / Virus Database: 3204/6515 - Release Date: 07/23/13

Reply all
Reply to author
0 new messages