external domain support

[yaco...@gmail.com]

Hi all.

TLDR:
I have a CF env installed with nise bosh and I need users to access it via an external address (an ip forwarding was set up to forward all packets from the external address to the internal address of the CF server)
Currently, the CF is configured to accept URLs like: app-name.server-ip.xip.io, and I want CF to be able to accept URLs like app-name.externalURL.
What files do I need to change (and what do I need to change in them) in order to do that?

-------------------------------------------------
My attempts so far:
In: /var/vcap/jobs/cloud_controller_ng/config/cloud_controller_ng.yml I changed the external_domain to be api.externalURL
and changed everything that ends with server-ip.xip.io.
also changed: /var/vcap/bosh/state.yml (likewise).
The problem is that the gorouter doesn't seem to forward the request to the UAA (it doesn't forward uaa.externalURL), and if I try to login from within the server itself without changing the UAA URL in the cloud_controller_ng.yml, I succeed, but the loggregator refuses to show the content of the logs, complaining I'm not authorized (but I should be).

I tried another approach:
I have a Tomcat servlet that acts as a reverse proxy to the haproxy, but changes the response of /v2/info so that the uaa endpoint and loggregator endpoints are given the external address, and I added the following lines in the haproxy configuration:
    reqirep ^Host:\ uaa.externalURL      Host:\ uaa.server-ip.xip.io
    reqirep ^Host:\ api.externalURL       Host:\ api.server-ip.xip.io
    reqirep ^Host:\ loggregator.externalURL:4443 Host:\ loggregator.server-ip.xip.io:4443


The login from outside (via the reverse proxy) works, because it returns to the CF CLI : uaa.externalUR, but from some reason the last rule that's supposed to change the host header for the loggregator doesn't take effect and the request goes to the gorouter as it is (with the external host in the host header) and the gorouter returns to CF CLI that no such route exists....

Is there a way of adding another domain to the gorouter?


Many thanks in advance,

Yacov.

[Daniel Mikusa]

On Tue, Sep 30, 2014 at 10:34 AM, <yaco...@gmail.com> wrote:

Hi all.

TLDR:
I have a CF env installed with nise bosh and I need users to access it via an external address (an ip forwarding was set up to forward all packets from the external address to the internal address of the CF server)
Currently, the CF is configured to accept URLs like: app-name.server-ip.xip.io, and I want CF to be able to accept URLs like app-name.externalURL. 

What files do I need to change (and what do I need to change in them) in order to do that?

I don't think you need to change anything with your CF config.  You can add additional application domains with cf.  See `cf domains`, `cf create-domain` and `cf create-shared-domain`.  Just make sure DNS for that domain points to your CF environment.

Dan

 

-------------------------------------------------
My attempts so far:
In: /var/vcap/jobs/cloud_controller_ng/config/cloud_controller_ng.yml I changed the external_domain to be api.externalURL
and changed everything that ends with server-ip.xip.io.
also changed: /var/vcap/bosh/state.yml (likewise).
The problem is that the gorouter doesn't seem to forward the request to the UAA (it doesn't forward uaa.externalURL), and if I try to login from within the server itself without changing the UAA URL in the cloud_controller_ng.yml, I succeed, but the loggregator refuses to show the content of the logs, complaining I'm not authorized (but I should be).

I tried another approach:
I have a Tomcat servlet that acts as a reverse proxy to the haproxy, but changes the response of /v2/info so that the uaa endpoint and loggregator endpoints are given the external address, and I added the following lines in the haproxy configuration:
    reqirep ^Host:\ uaa.externalURL      Host:\ uaa.server-ip.xip.io
    reqirep ^Host:\ api.externalURL       Host:\ api.server-ip.xip.io
    reqirep ^Host:\ loggregator.externalURL:4443 Host:\ loggregator.server-ip.xip.io:4443


The login from outside (via the reverse proxy) works, because it returns to the CF CLI : uaa.externalUR, but from some reason the last rule that's supposed to change the host header for the loggregator doesn't take effect and the request goes to the gorouter as it is (with the external host in the host header) and the gorouter returns to CF CLI that no such route exists....

Is there a way of adding another domain to the gorouter?


Many thanks in advance,

Yacov.

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/b9a477d4-9bc5-4f20-b007-50995c6ff93e%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[yaco...@gmail.com]

I tried to create a shared domain, and it tells me "error code: 130001, message: The domain is invalid: name overlapping_domain"
I gave it the parameter of my external address, is that wrong?

בתאריך יום שלישי, 30 בספטמבר 2014 17:39:22 UTC+3, מאת Daniel Mikusa:

[James Bayer]

can you provide an example such as:

system domain is cloud.example.com

apps domain is apps.example.com

new domain you are trying to register is: foo.example.com

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/64ea75fc-8225-48ef-94f9-1d20a8d7db1f%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Thank you,

James Bayer

[yaco...@gmail.com]

yeah.
The system and apps domain is both an internal ip address suffixed with xip.io, which was auto-generated using one of the bosh-nise scripts during the installation (I think via these lines [1])
The new domain I'm trying to register is actually an external address of a server, for example: foo.example.com.

[1]
https://github.com/yudai/cf_nise_installer/blob/master/scripts/generate_deploy_manifest.sh#L12

בתאריך יום שלישי, 30 בספטמבר 2014 20:45:05 UTC+3, מאת jbayer:

[James Bayer]

ok, so the overlapping domain error likely means that there is already a route or domain somewhere in the system that is using either example.com or foo.example.com. this is not allowed to prevent multiple organizations from claiming the same domain ownership.

you might need to search through the CC DB to find the domain/route that it is complaining about. i haven't gone through this procedure myself.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/136c3c8c-1a6a-408f-9a03-b739f36688c0%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[yaco...@gmail.com]

1) Do you have any idea why the haproxy Host header renaming rule for the loggregator doesn't work? it works for the uaa, but not for the loggregator. I suspect it's because of the :4443 suffix, but I'm not sure. Are there any haproxy configuration expers in the crowd? :)

2) I issued the cf create-shared-domain command again, while less-ing the cloud_controller_ng.log to see what tables it accesses. then I deleted the first row from the table "domains" (the one with the external-domain)
id |                 guid                 |         created_at         |         updated_at                                   |       name                                   | wildcard | owning_organization_id
----+--------------------------------------+----------------------------+----------------------------+-------------------+----------+------------------------
  2 | e50a569b-3b0e-4164-bcc8-a284c6fd89c8 | 2014-09-30 15:25:41.027563 |                            | external-domain-name              | t            |                      1
  1 | 6b73fdb5-bfc3-4f3e-82d8-f0b934b40606 | 2014-09-08 08:17:34.262302 | 2014-09-30 16:09:33.598485 | internal-domain-name  | t            |                      

and then invoked cf-create-shared-domain again, and this time it successfully added the shared domain, and indeed in "cf domains" I see the 2nd domain but I don't see how that helps me.
when I try to login from outside of the CF environment via targetting api.external-domain-name, the cf-cli performs a GET /v2/info
and Cloud Foundry returns:
{"name":"vcap","build":"2222","support":"http://support.cloudfoundry.com","version":2,"description":"Cloud Foundry sponsored by Pivotal","authorization_endpoint":"https://uaa.internal-domain.xip.io","token_endpoint":"https://uaa.internal-domain.xip.io","api_version":"2.13.0","logging_endpoint":"wss://loggregator.internal-domain.xip.io:4443"}

I can't connect to the internal address, the whole point of what I'm trying to do here is have the CF environment exposable from outside of the server via the external WAN address....



Thanks in advance.


בתאריך יום שלישי, 30 בספטמבר 2014 21:10:16 UTC+3, מאת jbayer:

ok, so the overlapping domain error likely means that there is already a route or domain somewhere in the system that is using either example.com or foo.example.com. this is not allowed to prevent multiple organizations from claiming the same domain ownership.

you might need to search through the CC DB to find the domain/route that it is complaining about. i haven't gone through this procedure myself.

[Daniel Mikusa]

On Tue, Sep 30, 2014 at 2:46 PM, <yaco...@gmail.com> wrote:

1) Do you have any idea why the haproxy Host header renaming rule for the loggregator doesn't work? it works for the uaa, but not for the loggregator. I suspect it's because of the :4443 suffix, but I'm not sure. Are there any haproxy configuration expers in the crowd? :)

2) I issued the cf create-shared-domain command again, while less-ing the cloud_controller_ng.log to see what tables it accesses. then I deleted the first row from the table "domains" (the one with the external-domain)
id |                 guid                 |         created_at         |         updated_at                                   |       name                                   | wildcard | owning_organization_id
----+--------------------------------------+----------------------------+----------------------------+-------------------+----------+------------------------
  2 | e50a569b-3b0e-4164-bcc8-a284c6fd89c8 | 2014-09-30 15:25:41.027563 |                            | external-domain-name              | t            |                      1
  1 | 6b73fdb5-bfc3-4f3e-82d8-f0b934b40606 | 2014-09-08 08:17:34.262302 | 2014-09-30 16:09:33.598485 | internal-domain-name  | t            |                      

and then invoked cf-create-shared-domain again, and this time it successfully added the shared domain, and indeed in "cf domains" I see the 2nd domain but I don't see how that helps me.

Glad you got this fixed up.  Creating the shared domain just makes it available so that users can map apps to the domain.  You probably still need to create a route and map it to an app.

See this doc for the complete process.  

http://docs.cloudfoundry.org/devguide/deploy-apps/domains-routes.html

 

when I try to login from outside of the CF environment via targetting api.external-domain-name, the cf-cli performs a GET /v2/info
and Cloud Foundry returns:
{"name":"vcap","build":"2222","support":"http://support.cloudfoundry.com","version":2,"description":"Cloud Foundry sponsored by Pivotal","authorization_endpoint":"https://uaa.internal-domain.xip.io","token_endpoint":"https://uaa.internal-domain.xip.io","api_version":"2.13.0","logging_endpoint":"wss://loggregator.internal-domain.xip.io:4443"}

I can't connect to the internal address, the whole point of what I'm trying to do here is have the CF environment exposable from outside of the server via the external WAN address....

This is different than what you previously asked.  Previously you asked about application domains, and for that see the process above.  If you want to change the system domain that's different.  To do that, I'm pretty sure you'd need to change your configuration and redeploy with Bosh.  I'm not sure on the exact details off the top of my head though.

Dan

[yaco...@gmail.com]

But I can't even login.
I need a way to tell the CF to send me the external domain address when I cf target it with cf-cli through the external address.
I don't see how this tutorial helps me at all.

בתאריך יום שלישי, 30 בספטמבר 2014 21:55:53 UTC+3, מאת Daniel Mikusa:

[Daniel Mikusa]

I'm not entirely sure I understand what you're talking about here.  What I can say is that the tutorial link is for application domains.  This is like www.my-cool-app.com -> my-cool-app deployed on your CF.  It's different than your system domain (i.e. api.<system-domain> or login.<system-domain>), which is what it sounds like you're now talking about now.  You can add any number of application domains at runtime, but I don't believe you can change the system domain.

To change the system domain, you'd need to use Bosh.  I don't know how to do this with nise-bosh & cf.  Sorry.  You might try doing a search and replace in your manifest, I'm not really sure though.

Dan

[yaco...@gmail.com]

Exactly- I have a system domain and I need to be able to CF target the CF env via another address because the system domain's address is reachable only internally.

בתאריך יום שלישי, 30 בספטמבר 2014 22:26:14 UTC+3, מאת Daniel Mikusa:

[Daniel Mikusa]

I don't believe that you can have two system domains.  Maybe someone else can confirm.

You need to make sure that your system domain resolves and is routable from wherever you'd like to use it.

Dan

[yaco...@gmail.com]

That's what I was trying to pull using the haproxy host header changing, but it won't work for the loggregator for some reason.

Thanks anyway Dan.

בתאריך יום שלישי, 30 בספטמבר 2014 22:38:06 UTC+3, מאת Daniel Mikusa:

[Daniel Mikusa]

Ah, I see now.  Sorry I couldn't be of more assistance.

Dan