Can anyone help troubleshoot FluentD in an OpenShift (3.4) environment?

[Joel Sheppard]

Hi.  I've got an OpenShift test lab in AWS, all the Aggregated Logging PODs are deployed and running, and I believe I've also configured the OpenShift Master Node master-config.yaml to display the FluentD aggregated logs in the OpenShift Web Interface, however I don't see the aggregated logs for any of our running images.  I don't think the logs are throwing any kind of severe error that I can tell is causing this to not work.  What I think might be a problem is in the KIBANA 'ROUTES' in OpenShift.  I've attached a .png of the routes, one thing I can say is the logging-kibana route looks close to right but when I try to open that URL there's nothing there at that URL. In this setup port 8443 is open, not 443 (if that is what Kibana is trying to use), and the OpenShift console is at URL/console, yet that and the port seems to have been ignored?    Also the second ROUTE kibna-ops.example.com - that is obvisouly some sort of default URL, what would this need to be set to?

In the I've copied out and zipped each PODs log file. and the image of the ROUTES for Kibana. I don't know what else to check.

Any help would be greatly appreciated.

Thanks and best wishes,

Joel

[Rich Megginson]

On 04/17/2017 09:27 AM, Joel Sheppard wrote:
> Hi. I've got an OpenShift test lab in AWS, all the Aggregated Logging
> PODs are deployed and running, and I believe I've also configured the
> OpenShift Master Node master-config.yaml to display the FluentD
> aggregated logs in the OpenShift Web Interface,

Are you using https://github.com/openshift/origin-aggregated-logging ?
If so, I suggest using us...@lists.openshift.redhat.com, and adding
[logging] to your message, since this issue seems specific to that
implementation and not fluentd in general.

> however I don't see the aggregated logs for any of our running images.
> I don't think the logs are throwing any kind of severe error that I
> can tell is causing this to not work.

Have you looked at the pod logs for fluentd, elasticsearch, and kibana?

> What I think might be a problem is in the KIBANA 'ROUTES' in
> OpenShift. I've attached a .png of the routes, one thing I can say is
> the logging-kibana route looks close to right but when I try to open
> that URL there's nothing there at that URL. In this setup port 8443 is
> open, not 443 (if that is what Kibana is trying to use), and the
> OpenShift console is at URL/console, yet that and the port seems to
> have been ignored?

The way Routes work in OpenShift is that the hostname is used as the
discriminator for the route. I'm not sure if you can use the same
hostname for both the OpenShift console and Kibana even though they use
different ports. Usually, Kibana is deployed at kibana.{{ openshift
deployment subdomain }} and you will have to set up either DNS to have
the correct IP address for kibana.{{ openshift deployment subdomain }}
or just hack your /etc/hosts file.

> Also the second ROUTE kibna-ops.example.com - that is obvisouly some
> sort of default URL, what would this need to be set to?

You can ignore it if you deployed without the separate ops cluster.

>
> In the I've copied out and zipped each PODs log file. and the image of
> the ROUTES for Kibana. I don't know what else to check.

I don't see anything obvious in the logs. Please go to
us...@lists.openshift.redhat.com for follow ups.

>
> Any help would be greatly appreciated.
>
> Thanks and best wishes,
>
> Joel

> --
> You received this message because you are subscribed to the Google
> Groups "Fluentd Google Group" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to fluentd+u...@googlegroups.com
> <mailto:fluentd+u...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Joel Sheppard]

Thank you, I'll post my questions to the mail list you suggested.