How do I make Ruby automatically follow redirects with Net/HTTP?

[Nathan Lane]

How do I make Ruby automatically follow redirects with Net/HTTP? Can anybody tell me?

Here's what I've got:

require 'net/http'

http = Net::HTTP.new('www.vehix.com ', 80)
path = '/'
resp, data = http.get(path, nil)

And then I get a redirect page, which is very evident if you follow this code through irb.

--
Nathan Lane

[Jeremy Nicoll]

 Hello Nathan,
    I am not sure if any native Ruby class supports automatic redirects, but it's fairly simple to do.  It seems that the path you are trying to retrieve is looping on it's self - probably because it expects a cookie to be sent with the request.  Here is my fairly simple solution to follow redirects:

#!/usr/bin/env ruby
require 'net/http'

url = URI.parse("http://www.vehix.com/")  # Make sure you put the trailing slash on!
# These variables are required because the location header does not always contain a complete URI.  They often come without the domain information if on the same site.
host = url.host
port = url.port

http = Net::HTTP.new(host, port)
resp, data = http.get(url.path, nil)

prev_redirect = ''

while resp.header['location']
  raise "Recursive redirect: #{resp.header['location']}" if prev_redirect == resp.header['location']
  prev_redirect = resp.header['location']
 
  url = URI.parse(resp.header['location'])
  host = url.host if url.host
  port = url.port if url.port
  http = Net::HTTP.new(host, port)
  resp, data = http.get(url.path, nil)
end

Jeremy Nicoll
www.gnexp.com
(801) 783-3831

Nathan Lane wrote:

--~--~---------~--~----~------------~-------~--~----~
Utah Ruby Users Group
ur...@googlegroups.com
http://groups.google.com/group/urug/
-~----------~----~----~----~------~----~------~--~---

[Jeremy Nicoll]

Just made the code much more concise, if anyone is still interested:

require 'net/http'

url = URI.parse('

http://www.vehix.com/') # Make sure you put the trailing slash on!

found = false
until found
  host, port = url.host, url.port if url.host && url.port
  req = Net::HTTP::Get.new(url.path)
  res = Net::HTTP.start(url.host, url.port) {|http|  http.request(req) }
  res.header['location'] ? url = URI.parse(res.header['location']) : found = true
end

puts res.body

Jeremy Nicoll
www.gnexp.com
(801) 783-3831

[Jeremy Nicoll]

Blast, I hate it when I make typos and post them to the list.  You need to take the "url." off of line 9 from both the variables.  Here's the updated code:

require 'net/http'

url = URI.parse('http://www.vehix.com/') # Make sure you put the trailing slash on!

found = false
until found
  host, port = url.host, url.port if url.host && url.port
  req = Net::HTTP::Get.new(url.path)

  res = Net::HTTP.start(host, port) {|http|  http.request(req) }

  res.header['location'] ? url = URI.parse(res.header['location']) : found = true
end

puts res.body

    I also happened across a standard library called "open-uri"  this might prove more useful, though I really don't feel like researching it at the moment.  Looks like it's easier to use, but I don't know if it does redirects.

[Nathan Lane]

Thanks Jeremy,

That is very helpful :)  I might take a look on open-uri, but this looks like it will serve my purposes.  Until now I have been doing something like this:

...
def handle_redirects()
  @resp.body.downcase !
       
  redirectLinkBegin = (@resp.body.split(/href="/))[1]       
  redirectLink = (redirectLinkBegin.split(/"/))[0]
       
  if !(redirectLink['://'].nil?)
    hostString = redirectLink.split (/:\/\//)[1]
    @host = (hostString.split(/\//))[0]
    pathOffset = hostString.index('/')
           
    if !(pathOffset.nil?)
      @path = hostString.slice(pathOffset, (hostString.length - 1))
    end # End: if pathOffset > -1
           
    if @debug
      puts 'Redirecting to: ' + @host
    end
           
    @http = Net::HTTP.new @host, @portNumber
  else
    if @debug
      puts 'Redirecting to: ' + redirectLink.to_s
    end
           
    @path = redirectLink
  end # End: if !(redirectLink['://'].nil?)       
end # End: def handle_redirects()

Which works for a particular flavor of redirects, but I don't know if it works for every kind of redirect.  I don't have many test resources, nor do I know of many sites that use redirects to test it on.

Thanks,

Nathan

Nathan Lane