OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server session ticket A

[Jeremy Aube]

Last night at 4:27am ET, we had an ad-hoc campaign performance report fail with the following error:

OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv3 read server session ticket A

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:799:in `connect'

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:799:in `block in connect'

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/timeout.rb:68:in `timeout'

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/timeout.rb:99:in `timeout'

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:799:in `connect'

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:755:in `do_start'

from /Users/<user>/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/1.9.1/net/http.rb:744:in `start'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/httpi-1.0.0/lib/httpi/adapter/net_http.rb:75:in `do_request'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/httpi-1.0.0/lib/httpi/adapter/net_http.rb:30:in `post'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/httpi-1.0.0/lib/httpi.rb:96:in `block in post'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/httpi-1.0.0/lib/httpi.rb:193:in `with_adapter'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/httpi-1.0.0/lib/httpi.rb:94:in `post'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-ads-common-0.8.0/lib/ads_common/http.rb:52:in `post_response'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-ads-common-0.8.0/lib/ads_common/auth/client_login_handler.rb:151:in `generate_token'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-ads-common-0.8.0/lib/ads_common/auth/client_login_handler.rb:119:in `create_token'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-ads-common-0.8.0/lib/ads_common/auth/base_handler.rb:50:in `get_token'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-ads-common-0.8.0/lib/ads_common/auth/client_login_handler.rb:67:in `auth_string'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-adwords-api-0.7.1/lib/adwords_api/report_header_handler.rb:53:in `headers'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-adwords-api-0.7.1/lib/adwords_api/report_utils.rb:177:in `get_report_request_headers'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-adwords-api-0.7.1/lib/adwords_api/report_utils.rb:153:in `make_adhoc_request'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-adwords-api-0.7.1/lib/adwords_api/report_utils.rb:139:in `get_report_response'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-adwords-api-0.7.1/lib/adwords_api/report_utils.rb:57:in `download_report'

from /Users/<user>/<dist>/<app>/shared/bundle/ruby/1.9.1/gems/google-adwords-api-0.7.1/lib/adwords_api/report_utils.rb:76:in `download_report_as_file'

We're using v201209. In the past we've had this error come up intermittently, sometimes occurring several times in a row before disappearing for a few weeks. Currently we're not doing anything to handle it--I'm not sure if we can just wait 30 seconds and try again--but everything did work fine when it ran the following hour. Also, based on the timing, it's possible that it spent a full 7 minutes trying to download this report before coming up with the error; normally it would complete nearly instantly.

[Danial Klimkin]

Hello Jeremy,

It appears you are getting network timeouts. From the stack I can see you are using the Ruby library with net_http adapter. I recommend trying httpclient instead.

All you need to do it to install httpclient gem like:

  gem install httpclient

-Danial, AdWords API Team.

[Jeremy Aube]

Hi Danial,

We installed the httpclient gem, but the SSL error still pops up now and then.

[AdWordsAPIAdvisor]

Hello Jeremy,


Can you please send me full debug output logs for such a request over
email (please make sure not to share it with the whole group as it may
contain sensitive information).


-Danial, AdWords API Team.


Original Message Follows:
------------------------
From: Jeremy Aube <ja...@roirevolution.com>
Subject: Re: OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5

errno=0
state=SSLv3 read server session ticket A

Date: Mon, 3 Dec 2012 05:05:36 -0800 (PST)

> --
> --
> =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
> Also find us on our blog and discussion group:
> http://adwordsapi.blogspot.com
> http://groups.google.com/group/adwords-api
> =~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~
>
> You received this message because you are subscribed to the Google
> Groups "AdWords API Forum" group.
> To post to this group, send email to adwor...@googlegroups.com
> To unsubscribe from this group, send email to
> adwords-api...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/adwords-api?hl=en
>
>
>

[Jeremy Aube]

Hi Danial,

We checked the logs and it looks like no requests are being made during the time when the errors occur. We had a 2 hour gap this morning where no requests or responses were recorded, which happened at the time time as all the errors. The errors also seem to be coinciding with increased network activity on our side, so it seems very likely at this point that it's an issue on our side rather than with the AdWords API or the ruby client library.

Thanks for your help!

[Jeremy Aube]

Right now we're wrapping the calls with code to keep retrying the calls until they succeed, or fail too many times. 

 
On Tuesday, December 11, 2012 5:34:56 AM UTC-5, Nilesh Avhad wrote:

Hi Jeremy,

I am facing a similar issue. How did you resolve this?

Regards,

Nilesh

[Jeremy Aube]

Just to follow up on this, updating to the latest version of OpenSSL seems to have fixed the issue. We haven't had any SSL errors for 2 days now.

[Jeremy Aube]

It looks like updating OpenSSL didn't work. We had SSL errors on the 25th as well as this morning (the 27th). 

I had left a few posts, but it looks like those were deleted for some reason, possibly because I posted the backtrace for the latest error. Rather than doing that again, I'll just point out the main differences:

1. The error is now 'read server hello A' instead of 'read server session ticket A'.

2. The error now occurs in session.rb, line 302, in 'connect', which is a file in the httpclient gem.

3. The next several lines are all from httpclient instead of net/http.

4. The google-ads-common lines are from 0.8.2 instead of 0.8.0.

5. The google-adwords-api lines are from 0.7.2 instead of 0.7.1.

I'm not sure what else to try at this point. The adwords_xml.log file contains a large gap while these are occurring, so either no SOAP requests/responses are being made when the errors occur or they're not being recorded.

Jeremy

[Danial Klimkin]

Hello Jeremy,

It sounds like a local configuration issue rather than a library issue to me, but let's keep investigating.

Can you please send full logs for the error to me over email?

Also, I suggest trying the curb backend as well. It is a native backend and may work better in mac env. To try it, install curb gem and set the configuration file as follows:

:service:

 [...]

[:authentication]

 [...]

:connection:

  :adapter: :curb

-Danial, AdWords API Team.

[Jeremy Aube]

Hi Danial,

The relevant logs are several gigabytes large, so I haven't found a good way to send them yet. It looks like the errors are being triggered by memory issues. They tend to only happen when all of the available RAM has been used up. I've been working on ways to reduce memory usage and that seems to have calmed down the errors significantly. 

I did try changing the adapter, but still had SSL errors:

Curl::Err::SSLConnectError: Curl::Err::SSLConnectError

Anyways, I think we can consider the issue closed and I can just continue to work on being more efficient with memory.

Thanks for all of your help!

Jeremy