403 forbidden by spamhaus

[Gil Vidals]

I've never seen a URL like this before that detect whether the search is coming from a desktop browser or from an automated query like wget or mechanize. Even setting the user-agent and cookies doesn't help. Does anyone know how to bypass the 403 error and retrieve the page using mechanize?

http://www.spamhaus.org/query/bl?ip=1.2.3.4

Thanks,
Gil

[Jon Hermansen]

Hi Gil,
 I would recommend recording the whole HTTP transaction with LiveHTTPHeaders, tcpdump or Wireshark and then manually reproducing it with telnet. If you can duplicate the results you want, start picking out parts of the request until you can automate it with Perl.

Once you try to write it in Perl, forget about Mechanize until you get it working in LWP::UserAgent. Then you can try to adapt it mech...

HTH,

Jon

--
You received this message because you are subscribed to the Google Groups "WWW::Mechanize users" group.
To post to this group, send email to www-mecha...@googlegroups.com.
To unsubscribe from this group, send email to www-mechanize-u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/www-mechanize-users?hl=en.

[Paul Miller]

I'm sure you can figure this out if you persist, but perhaps you
shouldn't figure this out. The reason the don't want to you to do
this from automated tools is that it's a much simpler matter for their
servers and for your computer if you use the DNS interfaces. Sure the
web interface is more familiar, but this system is intended as a DNS
interface.

127.0.0.2 is always on these dnsbls by design...

host 2.0.0.127.xbl.spamhaus.org

http://www.spamhaus.org/faq/answers.lasso?section=DNSBL%20Usage#202
http://search.cpan.org/~olaf/Net-DNS-0.66/lib/Net/DNS.pm