External/Outside Domain - The application is currently not serving requests at this endpoint

2,569 views
Skip to first unread message

Eduardo Lúcio Amorim Costa

unread,
Jun 28, 2021, 10:08:10 PM6/28/21
to okd-wg
Can anyone help me with the issue below?


We performed the command below to allow access from external/outside domain to the Web Console on the OKD cluster...

oc patch consoles.operator.openshift.io cluster --type merge --patch '{"spec":{"route":{"hostname":"console.okdsubdom.another.domain"}}}'

... and we can access the Web Console application in the domain "okdsubdom.another.domain" as in this example...

But now... A new problem has arisen "The application is currently not serving requests at this endpoint." ...

[root@okd4-services ~]# curl -k https://console-openshift-console.okdsubdom.another.domain

[...]
  <body>
    <div>
      <h1>Application is not available</h1>
      <p>The application is currently not serving requests at this endpoint. It may not have been started or is still starting.</p>

      <div class="alert alert-info">
        <p class="info">
          Possible reasons you are seeing this page:
        </p>
        <ul>
          <li>
            <strong>The host doesn't exist.</strong>
            Make sure the hostname was typed correctly and that a route matching this hostname exists.
          </li>
          <li>
            <strong>The host exists, but doesn't have a matching path.</strong>
            Check if the URL path was typed correctly and that the route was created using the desired path.
          </li>
          <li>
            <strong>Route and path matches, but all pods are down.</strong>
            Make sure that the resources exposed by this route (pods, services, deployment configs, etc) have at least one pod running.
          </li>
        </ul>
      </div>
    </div>
  </body>
</html>

NOTE: The "okdsubdom.certain.domain" is the domain in which the cluster was created and "okdsubdom.another.domain" is the new domain for external access to the Web Console in the OKD cluster.

NOTE: This problem only occurs if I try to access the application in the domain "okdsubdom.another.domain". Accessing by the domain "okdsubdom.certain.domain" everything works normally.

This issue is related to this thread: OpenShift/OKD cluster - Use with external/outside domain .

Thanks! =D

mohammad reza ostadi

unread,
Jun 28, 2021, 11:12:15 PM6/28/21
to okd-wg

Hi
It seems that you add replace a route and didn't add new domain (okdsubdom.another.domain) to your ingress controllers.
you can simply add new ingressController for the new domain (okdsubdom.another.domain) and add another route for the console and it's going to work.
For better understanding of what you need to create you can see the existing ingressController for initial domain (okdsubdom.certain.domain).

Eduardo Lúcio Amorim Costa

unread,
Jun 29, 2021, 12:17:55 PM6/29/21
to okd-wg

I transposed your comment to the project's discussion group...

OpenShift/OKD cluster - Use with external/outside domain

Thanks! =D

Eduardo Lúcio Amorim Costa

unread,
Jun 29, 2021, 12:29:21 PM6/29/21
to okd-wg
mr.o...@gmail.com

"It seems that you add replace a route" -> I don't know if I got it wrong, but I actually added one more route.

"add new ingressController for the new domain" -> Could you give us some more details about this?

On Tuesday, June 29, 2021 at 12:12:15 AM UTC-3 mr.o...@gmail.com wrote:

Eduardo Lúcio Amorim Costa

unread,
Jun 29, 2021, 8:28:52 PM6/29/21
to okd-wg
Guys! Come On!

No one accesses the web console from an external domain? Only I have this problem?

I really don't know where to find help for this...

Some help? Please! =D

Thanks! =D

mohammad reza ostadi

unread,
Jul 4, 2021, 1:16:12 PM7/4/21
to okd-wg
Hi Again
Sorry for delay. It has been a busy week.
this link in the Documentation site can help you with your issue.

Eduardo Lúcio Amorim Costa

unread,
Jul 4, 2021, 4:08:35 PM7/4/21
to okd-wg
Hi "mr.o...@gmail.com" and other friends! =D

For days I have had and observed a series of problems and difficulties in allowing external access (WAN access, custom domain) to OpenShift (OKD) =| . I think this is of interest to our community as it may be an important limitation for OpenShift (OKD).

At issue A new and (almost) effective way to update routes and allow external access I propose a relatively simple solution that seems quite viable, but I need your help to address some points. So I highly recommend everyone to take a look. =D

The main current problems are:

 I - New routes are not consumed by the Web Console...

"[...] Once I create these new routes, will the "Web Console" start using them? Let me explain [...] So if I create these new routes in the domain "mydomain.net" the Web Console will use them? [...]";


 II - The other routes used in the cluster are not updated and additionally we have problems in the login system (Web Console)...

"[...]
A - We've already tried this procedure Customizing the web console URL Customizing the web console URL, but it doesn't update all routes and does not update URLs in the Web Console and also has the same login issues above;
B - I also tried this procedure Creating a route through an Ingress object Creating a route through an Ingress object, but equally it doesn't seem to me something that will solve the problems pointed out.
[...]"


CONCLUSION: I invite everyone to try to solve this problem, as it is a basic need of almost everyone who builds an infrastructure using UPI/"bare metal". Other than that, the situation can become dramatic if the user needs to change the domain after the cluster is installed (this is also a common situation).

Thanks to everyone in our community! =D

Brian Innes

unread,
Jul 5, 2021, 9:49:14 AM7/5/21
to okd-wg
Hi Eduardo,

If I understand your issues then I think the system is behaving as expected, so maybe there is some misunderstanding on your part?
This is my understanding of what is happening :-

The documentation explains how to add additional ingress instances to allow a cluster to respond to different domains and how to customise the web console to use the ingress (https://docs.okd.io/latest/web_console/customizing-the-web-console.html#customizing-the-web-console-url_customizing-web-console).

However, I think there is some confusion about application configuration.  Your expectation is that existing application configuration will be updated to pick up the new ingress configuration automatically.

The "single source of truth" for application configuration does not necessarily reside within cluster object configurations, such as deployments, services, routes, ....  The configuration for an application may be controlled by a Helm chart deployment, an operator instance or within a git repository (using GitOps with ArgoCD), so the cluster should not arbitrarily modify a deployed application and any manual configuration modifications may be reverted back to their original values.  The host value for an application route (and possible TLS termination) may be part of the application configuration, so the application configuration needs to be updated then applied to the cluster.

Each application that should be exposed using the new or amended Ingress instance will need to have a route configured to use the ingress.  How this is achieved will depend on how the application was deployed and what is controlling/monitoring the application configuration.  Then any links that you want to use the new domains, such as Console Links, will need to be manually updated to use the new routes.

If there are a large number of applications then a scripted approach using Ansible or a shell script may be appropriate, but again the content of the script will depend on how the configuration for each application is being managed.
 
Does this addresses your issues?  If there are sections of the documentation that need improving or additional content, then please add a comment below.

There is a lot going on in this thread and I'm not sure what the login issues are?

Regards,

Eduardo Lúcio Amorim Costa

unread,
Jul 5, 2021, 6:43:58 PM7/5/21
to okd-wg
Hi Brian and other friends!

I'm analyzing all the guidelines that were sent to me, but basically from what I understand I need first a "new or amended Ingress instance"  for the new domain and then adjust the routes in the "Web Console".

About "If there are sections of the documentation that need improving or additional content, then please add a comment below" one thing I didn't understand is how I can create/edit this configuration Customizing the console route ... Let me explain better... What kind of parameter should I pass to "oc"? Something like this?

EXAMPLE

```
oc edit service/downloads -n openshift-console
```

MODEL

```
oc edit <KIND>/<RSRC_NAME> -n <NAME_SPACE> 
```

In other words, the instruction "Set the custom hostname and, if needed, set TLS on the ingress config's spec" doesn't make it clear to me whether I should edit or create a certain configuration and where I should do it.

Could you give me an example? That would help a lot!

NOTE: The process I used to deploy the cluster was basically this OpenShift 4.3 OKD Bare Metal Install on VMWare Home Lab .

Thanks! =D

Eduardo Lúcio Amorim Costa

unread,
Jul 5, 2021, 8:44:37 PM7/5/21
to okd-wg
Brian and other friends!

"Then any links that you want to use the new domains, such as Console Links, will need to be manually updated to use the new routes." -> Where and how do we do this manual update?

Thanks!

[]'s

Eduardo Lúcio Amorim Costa

unread,
Jul 5, 2021, 9:28:18 PM7/5/21
to okd-wg
People,

I read all the documentation on these links...


... and I confess that I still can't understand how to make OpenShift (OKD) serve different domains than what was used (okd.local) in its installation.

Remembering what my need is... Make these routes (okd.local)...

oauth-openshift.apps.mbr.okd.local
console-openshift-console.apps.mbr.okd.local
downloads-openshift-console.apps.mbr.okd.local
canary-openshift-ingress-canary.apps.mbr.okd.local
alertmanager-main-openshift-monitoring.apps.mbr.okd.local
grafana-openshift-monitoring.apps.mbr.okd.local
prometheus-k8s-openshift-monitoring.apps.mbr.okd.local
thanos-querier-openshift-monitoring.apps.mbr.okd.local

... start serving on a new domain (mydomain.net)...


PLUS: I don't know if this is my unfair assessment (this is not a criticism) of OpenShift (OKD). But, I think this should be a much simpler thing... Could there be nothing more practical and direct for us to resolve this?

IMPORTANT: Please note that in this scheme the login should still work (see here for more details A new and (almost) effective way to update routes and allow external access ) and the existing URLs in the Web Console should be updated for the new domain. I am saying this because in everything I have been guided and tried so far these problems have occurred. I don't know anymore how to explain what I need here. But, nothing I've tried for days works to solve this need. I think this shouldn't be something that needed a deep understanding of the API to resolve.

NOTE: Again, the process I used to deploy the cluster was basically this OpenShift 4.3 OKD Bare Metal Install on VMWare Home Lab .

Thanks! =D

Brian Innes

unread,
Jul 6, 2021, 6:30:11 AM7/6/21
to okd-wg
Hi Eduardo,

Now I understand what you are doing,  Changing a domain is not a trivial matter for any IT operations team.  If you are doing a fresh install then you should set the domain at install time, not install with the domain set as old.local then try to change it.

In the instructions you are following you download a file bundle from https://github.com/cragr/okd4_files, in this file bundle there is a file install-config.yaml which is where the domain is specified in the second line baseDomain: okd.local.
 Change the baseDomain in the config file to mydomain.net before installing then your cluster will be correctly configured to use your domain during the install.


Note that there are other configuration files that will also need updating to use your new domain.  

Regards,

Eduardo Lúcio Amorim Costa

unread,
Jul 6, 2021, 12:00:53 PM7/6/21
to okd-wg
Thanks Brian and all friends,

About "is specified in the second line baseDomain: okd.local"  this was already known to me. Thanks for the tip!

The infrastructure is already in use, so there's no way I can reinstall everything. As I said in a previous post "the situation can become dramatic if the user needs to change the domain after the cluster is installed (this is also a common situation)". So I understand, for good, that we should have this process documented somewhere on the internet in a systematic way. I can do this! But, I need your help.

I think a good start would be to find an answer to that (previous post)...

"
Hi Brian and other friends!

I'm analyzing all the guidelines that were sent to me, but basically from what I understand I need first a "new or amended Ingress instance"  for the new domain and then adjust the routes in the "Web Console".

About "If there are sections of the documentation that need improving or additional content, then please add a comment below" one thing I didn't understand is how I can create/edit this configuration Customizing the console route ... Let me explain better... What kind of parameter should I pass to "oc"? Something like this?

EXAMPLE

```
oc edit service/downloads -n openshift-console
```

MODEL

```
oc edit <KIND>/<RSRC_NAME> -n <NAME_SPACE> 
```

In other words, the instruction "Set the custom hostname and, if needed, set TLS on the ingress config's spec" doesn't make it clear to me whether I should edit or create a certain configuration and where I should do it.

Could you give me an example? That would help a lot!

NOTE: The process I used to deploy the cluster was basically this OpenShift 4.3 OKD Bare Metal Install on VMWare Home Lab .

Thanks! =D
"

IMPORTANT: All other aspects concerning DNS, load balancing etc... are already known to me. I just need to handle the OpenShift (OKD) cluster configuration issues properly.

Eduardo Lúcio Amorim Costa

unread,
Jul 6, 2021, 1:08:29 PM7/6/21
to okd-wg
Apparently the process described here Customizing the console route involves editing this feature...

```

So I change the content of this...

```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: config.openshift.io/v1
kind: Ingress
metadata:
 creationTimestamp: "2021-06-25T18:15:30Z"
 generation: 22
 name: cluster
 resourceVersion: "4608007"
 selfLink: /apis/config.openshift.io/v1/ingresses/cluster
 uid: a199db4b-9a4b-4729-8335-97cd9ec9c50b
spec:
 domain: apps.mbr.okd.local
status: {}
```

... to this...

```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: config.openshift.io/v1
kind: Ingress
metadata:
 creationTimestamp: "2021-06-25T18:15:30Z"
 generation: 22
 name: cluster
 resourceVersion: "4608007"
 selfLink: /apis/config.openshift.io/v1/ingresses/cluster
 uid: a199db4b-9a4b-4729-8335-97cd9ec9c50b
spec:
  componentRoutes:
    - name: console
      namespace: openshift-console
      servingCertKeyPairSecret:
        name: console-brlight-net
status: {}
```

...but when I open the resource for editing again, we have this...

```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: config.openshift.io/v1
kind: Ingress
metadata:
 creationTimestamp: "2021-06-25T18:15:30Z"
 generation: 22
 name: cluster
 resourceVersion: "4608007"
 selfLink: /apis/config.openshift.io/v1/ingresses/cluster
 uid: a199db4b-9a4b-4729-8335-97cd9ec9c50b
spec:
status: {}
```

NOTE: The secret "console-brlight-net" was created following the process described "complete script to customizing the 'Web Console" URL" .

Brian Innes

unread,
Jul 6, 2021, 2:29:47 PM7/6/21
to okd-wg
Hi Eduardo,

What you are doing is not a common use case.  Changing the domain of a running system is not a common IT operation.

It was confirmed today in the OKD working group meeting that what you are trying to do is not supported in OKD.  

It is possible to add support for additional domains in OKD and have the OKD web console, OKD  downloads and application endpoints be served on the new domains.  However, the base domain configured at install time will always be used for parts of the system and they cannot be modified, such as authentication.

To get a cluster working on a different base domain you will need to install a new cluster, then migrate the workloads from your existing cluster to the new cluster.

Eduardo Lúcio Amorim Costa

unread,
Jul 6, 2021, 4:23:05 PM7/6/21
to okd-wg
Brian and all friends,

I think maybe I expressed myself wrong... What I need is that these routes (okd.local)...

```
oauth-openshift.apps.mbr.okd.local
console-openshift-console.apps.mbr.okd.local
downloads-openshift-console.apps.mbr.okd.local
canary-openshift-ingress-canary.apps.mbr.okd.local
alertmanager-main-openshift-monitoring.apps.mbr.okd.local
grafana-openshift-monitoring.apps.mbr.okd.local
prometheus-k8s-openshift-monitoring.apps.mbr.okd.local
thanos-querier-openshift-monitoring.apps.mbr.okd.local
```

... start responding in this new domain (mydomain.net)...

```
```

... and not change the cluster's base domain.

However (so that there is no doubt), you told me that "[...] the base domain configured at install time will always be used for parts of the system and they cannot be modified, such as authentication". So it means that even if I change routes like I successful do here A new and (almost) effective way to update routes and allow external access I won't be able to allow authentication to they ( "oauth-openshift.apps.mbr.mydomain.net" ) ? That's it?

Thanks a lot! =D
Reply all
Reply to author
Forward
0 new messages