Failing to connect to https URL during staging

[Orson]

Hi all.

Having an issue where staging an app attempts, at some point in the process, to connect to an http URL

The URL in question appears to be  https://0a394a455aee951daea4c969e47c79b5.[redacted domain]/staging_tasks/d8550f1ce3e2e72bc3f6f882dec59f03/file_path?hmac=dde29fa2ac079628f18380254634f4769948ccc3c0fe2e96908992c84521368fb6b3795ecfbe55d8777d5844f2ce4bfcdb4b560f753e80c70ba52f253c0e2364&path=%2Ftmp%2Fstaged%2Flogs%2Fstaging_task.log&timestamp=1377173789 

The problem is, that (as far as I know, corrections welcome) by default, the go router doesn't support https - and as a result, I get:

Errno::ECONNREFUSED: Connection refused - connect(2)

cat ~/.cf/crash # for more details

Running release 138.

Any and all input welcomed.

Cheers, 

Orson

[Orson]

Some additional information, for clarity:

That URL is returned as one of the Response Headers (x-app-staging-log:) to the call that's made here:

Preparing to start street>>>

REQUEST: PUT http://api.[redacted]/v2/apps/d392c084-bdea-4daf-81e9-f2cf93726067

O.

[Doug Davis]

I saw this on my system too, a workaround is to modify the directory_server in jobs/dea_next/config/dea.yml:
directory_server:
  protocol: "https"
to:
directory_server:
  protocol: "http"

thanks
-Doug
________________________________________________________
STSM |  Standards Architect  |  IBM Software Group
(919) 254-6905  |  IBM 444-6905  |  d...@us.ibm.com
The more I'm around some people, the more I like my dog.

Orson ---08/22/2013 08:32:53 AM---Some additional information, for clarity: That URL is returned as one of the Response Headers (x-app

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Dr Nic Williams]

Unless you have an ssl front end to gorouter (such as sslproxy-boshrelease), then all your https should be modified to be http.

--
Dr Nic Williams
Stark & Wayne LLC - the consultancy for Cloud Foundry
http://starkandwayne.com
+1 415 860 2185
twitter: drnic

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[Orson]

Thanks Doug, that solved it.

Is anyone able to suggest the best place for this to be filed as an issue or pull request?

O.

[James Bayer]

Orson,

I'm not sure a Pull Request makes sense as we recommend that the logs be retrieved with an SSL endpoint by default so that communication is secure if the logs should be secured. Perhaps documentation could be the answer. You can see the docs in the cf-docs repository.

--

Thank you,

James Bayer

[Orson]

Hi James

Just to make sure I understand here:

Retrieving logs over plain http is forbidden (without hackery), but authenticating via plain http is fine?

I'm looking for expert guidance here - my initial perception is that a deployment that doesn't allow you to stage apps or retrieve logs is broken-by-default - but I'm happy to accept that my particular default may be the broken (while something more generic may be perfectly fine).

Dr Nic made a reference to an SSL front end to the gorouter, is that something that's normally included by default, that I've somehow missed from the deployments I'm working on?

I'm happy to make updates to docs, and submit them as pull requests once I understand what the recommended process is - but, what you (James) seems to be saying is "use https for logs", and I have no information on the official/recommended way to set that up.

Cheers

Orson

[Dr Nic Williams]

James, for dev/test it might be consistent to continue to allow http everywhere; or nowhere?

--
Dr Nic Williams
Stark & Wayne LLC - the consultancy for Cloud Foundry
http://starkandwayne.com
+1 415 860 2185
twitter: drnic

[Dr Nic Williams]

If its to be https everywhere, perhaps bring in Ferdy's sslproxy-boshrelease into cf-release?

--
Dr Nic Williams
Stark & Wayne LLC - the consultancy for Cloud Foundry
http://starkandwayne.com
+1 415 860 2185
twitter: drnic

[James Bayer]

authenticated with plain http is also not fine. ryan morgan fixed part of that today for the login server and saml server [1]. ryan knows whether uaa is affected too. if you would like to override the default https requirements, then configuration changes can handle that in the bosh deployment manifest, which is not hackery but rather how bosh deployment default overrides are meant to be done.

i appreciate the case for local development to not have an SSL enabled load balancer. i believe dev env approaches like nise-bosh for cf and cf-vagrant-installer can consider that use case.

nic, i'm not familiar with sslproxy bosh release. i suggest that someone discuss it with mark kropf. it may also be possible to add the x-forwarded-proto: https value as a header when not running ssl termination. i haven't tried it.

[1] https://github.com/cloudfoundry/cf-release/pull/147

Thank you,

James Bayer

650.646.1684

[m...@matt-j.co.uk]

Hi All,

Glad this is an active discussion, we are also running into the same issue with a V2 CF proof of concept ontop of vsphere. (You have most likely seen many mailing list posts by my colleague Troy Astle).

From our investigations, it appears the gorouter does not listen for SSL/TCP443 at all (which seems to be backed up above).
This caused us the problem of cf push failing as discussed above, causing us to force cf to try HTTP instead of HTTPS which it was not doing.

Our answer to this (as we could not find any documentation suggesting we could alter the HTTPS use of the 'cf' client tool; was to edit the baseclient.rb in rubygem cfoundry to always use port 80 instead of 443, which allowed the cf push to work successfully.

We still have the remaining questions I'm hoping someone could answer;

1. Is there a configuration file for the cf client tools to force HTTP instead of hacking the code? (useful to know for future reference).

2. while trying to enable HTTPS (*this is our goal, HTTP is a problem for us with auth etc*) we tried the following UAA settings in our deployment.yml. This causes the UAA to listen on port 443, but still for HTTP only (openssl s_client confirms HTTP only) could anyone offer help on if we are using the wrong/missing options to enable UAA HTTPS? or is the functionality and configuration directives just broken?

435  uaa:
436     port: 443
437     no_ssl: false
439     require_https: true
441     dump_requests: true
442     # Use postgres (see line 12 in the file referenced below)
443     # https://github.com/cloudfoundry/cf-release/blob/master/jobs/uaa/templates/uaa.yml.erb
444     spring_profiles: postgresql
445     url: https://uaa.ourclouddomain.net
446     token:
447         url: https://uaa.ourclouddomain.net/oauth/token
448     login:
449         url: https://uaa.ourclouddomain.net/login.do
450     resource_id: account_manager
451     batch:
452       username: uaabatchuser
453       password: uaabatchpassword

Kind Regards,
Matt

[James Bayer]

1. Is there a configuration file for the cf client tools to force HTTP instead of hacking the code? (useful to know for future reference).

The recommended approach is to use a load balancer that supports SSL termination in-front of gorouter. Ferdy has provided one here:

https://github.com/cloudfoundry-community/sslproxy-boshrelease

We use ELBs in AWS currently and I believe we have tested with a vSphere virtual appliance in our vSphere environments. We have certainly also used off-the-shelf LBs like Zeus, F5, and anything that supports the x-forwarded-for and x-forwarded-for headers to pass along to gorouter. I will ask our team to confirm what we use.

2. while trying to enable HTTPS (*this is our goal, HTTP is a problem for us with auth etc*) we tried the following UAA settings in our deployment.yml. This causes the UAA to listen on port 443, but still for HTTP only (openssl s_client confirms HTTP only) could anyone offer help on if we are using the wrong/missing options to enable UAA HTTPS? or is the functionality and configuration directives just broken?

I can't tell for sure but it looks like the UAA job defaults are configured for HTTPS so it does not appear as though the deployment manifests we use have modified any of the https settings at all. I'll ask that identity team too. The UAA config is not expected to be broken.

[Dave Syer]

On 02/09/13 15:34, m...@matt-j.co.uk wrote:
> while trying to enable HTTPS (*this is our goal, HTTP is a problem for
> us with auth etc*) we tried the following UAA settings in our
> deployment.yml. This causes the UAA to listen on port 443, but still
> for HTTP only (openssl s_client confirms HTTP only) could anyone offer
> help on if we are using the wrong/missing options to enable UAA HTTPS?
> or is the functionality and configuration directives just broken?

The UAA sits behind the CF router (and any other front end you have in
place to so SSL termination). It never does SSL termination, even if you
ask it to listen on a different port. Changes in the manifest related to
https and the UAA/Login Server are purely for the clients of those
systems, to prevent them from using http by mistake. The way to fix it
for the foreseeable future is to use those manifest settings and also
put a front end router in place to terminate the SSL before it reaches
the CF router.