Getting 403 errors from AWS using fog 1.11.1 and excon 0.22.0

[Lee Jones]

Hello Everyone,

I'm setting up some new servers today and some of my existing automation with fog broke. It looks like the recent release of excon 0.22.0 may be causing some issues talking to AWS. Here is an example of the error I'm seeing:

~$ irb -r fog

irb(main):001:0> Fog::Compute.new({:provider=>"aws", :aws_access_key_id=>"real-key-here", :aws_secret_access_key=>"real-secret-here"}).servers.first

Excon::Errors::Forbidden: Expected(200) <=> Actual(403 Forbidden)

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/expects.rb:10:in `response_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:355:in `response'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:249:in `request'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/idempotent.rb:12:in `error_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/base.rb:10:in `error_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:262:in `rescue in request'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:222:in `request'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/idempotent.rb:12:in `error_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/base.rb:10:in `error_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:262:in `rescue in request'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:222:in `request'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/idempotent.rb:12:in `error_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/middlewares/base.rb:10:in `error_call'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:262:in `rescue in request'

        from /usr/lib/ruby/gems/1.9.1/gems/excon-0.22.0/lib/excon/connection.rb:222:in `request'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/core/connection.rb:21:in `request'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/aws/compute.rb:384:in `_request'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/aws/compute.rb:379:in `request'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/aws/requests/compute/describe_instances.rb:76:in `describe_instances'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/aws/models/compute/servers.rb:64:in `all'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/core/collection.rb:141:in `lazy_load'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/core/collection.rb:22:in `empty?'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/core/collection.rb:86:in `block in inspect'

        from /usr/lib/ruby/gems/1.9.1/gems/formatador-0.2.4/lib/formatador.rb:92:in `indent'

        from /usr/lib/ruby/gems/1.9.1/gems/fog-1.11.1/lib/fog/core/collection.rb:79:in `inspect'

        from /usr/bin/irb:12:in `<main>'

I can work around the issue by removing excon 0.22.0 and installing version 0.21.0.

I was trying to understand what might be the issue. I looked at the diff from today's release of excon and possibly narrowed it down to the "Host" header not including the port?

headers sent in excon 0.21.0

=> {"User-Agent"=>"fog/1.11.1",

 "Content-Type"=>"application/x-www-form-urlencoded",

 "Host"=>"ec2.us-east-1.amazonaws.com:443"}

headers sent in excon 0.22.0

=> {"User-Agent"=>"fog/1.11.1",

 "Content-Type"=>"application/x-www-form-urlencoded",

 "Host"=>"ec2.us-east-1.amazonaws.com"}

I temporarily patched the new #port_string method in exxon 0.22.0 to confirm:

 

irb(main):001:0> module Excon

irb(main):002:1>  class Connection

irb(main):003:2>    def port_string(datum)

irb(main):004:3>      ':443'

irb(main):005:3>    end

irb(main):006:2>  end

irb(main):007:1> end

=> nil

irb(main):008:0> Fog::Compute.new({:provider=>"aws", :aws_access_key_id=>"real-key-here", :aws_secret_access_key=>"real-secret-here"}).servers.first

=> # successful response with usual data etc.

Not sure if I'm on the right track, but would be interested in understanding the issue more. Maybe someone more familiar with the recent changes could shed some light.

Thanks,

Lee

[geemus (Wesley Beary)]

Had a small logic error (missed a set of crucial parens). Should be fixed in 0.22.1, sorry for the bump in the road.

--
You received this message because you are subscribed to the Google Groups "ruby-fog" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ruby-fog+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Lee Jones]

No worries. Thanks for the fix!