Kiali graphs shows "unknown" instead of "istio-ingressgateway" as source of traffic in the mess

64 views
Skip to first unread message

niscovea...@gmail.com

unread,
Nov 3, 2020, 1:26:43 PM11/3/20
to kiali-users
Hello,

I have applications deployed on Kubernetes in the "default" namespace.
The "istio-ingressgateway" is deployed in the "istio-system" namespace.

I have created a Gateway and a VirtualService in order to configure the ingress traffic.

The problem is that I expect that in Kiali graphs to see (as below) the "istio-ingressgateway" as source of the traffic in the mesh.

What I get is "unknown" node in the graphs:


Can anyone help with this issue please ? Is it configuration issue ?


Gateway.yaml:
```
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: default-gateway
  namespace: default
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
```

VirtualService.yaml
```
kind: VirtualService
metadata:
  name: vs-ingress
  namespace: default
spec:
  hosts:
  - "*"
  gateways:
  - default-gateway
  http:
  - match:
    - headers:
        Host:
          exact: mail.test.comp.com
    route:
    - destination:
        host: mailhog
        port:
          number: 8025
  - match:
    - headers:
        Host:
          exact: cart.test.comp.com
    route:
    - destination:
        host: shopping
        port:
          number: 80
```

My Setup:
Kiali UI
1.25.0 (6646d8b248dad2f8aea0fcd915085065f25f2776)
Kiali Server
v1.25.0 (275a5cba051ed25677a513706cd9eff36dc58d78)
Kiali Container
v1.25.0

Components

Istio
1.7.3
Prometheus
2.19.0
Kubernetes
v1.19.2


John Mazzitelli

unread,
Nov 3, 2020, 1:39:16 PM11/3/20
to kiali-users


----- Original Message -----
> Hello,
>
> I have applications deployed on Kubernetes in the "default" namespace.
> The "istio-ingressgateway" is deployed in the "istio-system" namespace.
>
> I have created a Gateway and a VirtualService in order to configure the
> ingress traffic.
>
> The problem is that I expect that in Kiali graphs to see (as below) the
> "istio-ingressgateway" as source of the traffic in the mesh.
>
> What I get is "unknown" node in the graphs:
>


Does the FAQ help? "Why are there many unknown nodes in the graph?"

https://kiali.io/documentation/latest/faq/#many-unknown
> --
> You received this message because you are subscribed to the Google Groups
> "kiali-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kiali-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kiali-users/7835a574-f110-432f-a392-28118a2f5657o%40googlegroups.com.
>

Mircea Marius

unread,
Nov 3, 2020, 1:52:21 PM11/3/20
to kiali-users
@jmaz...@redhat.com  Thank you but it does not help me.
I want to see the "istio-ingressgateway" in my graph on the "default" namespace in Kiali.

If I switch to namespace "istio-system" in Kiali, I get the object  "istio-ingressgateway" as source of my traffic.
So I want to see the same but for the "default" namespace ( and having the istio-ingress installed in "istio-system" namespace ).

John Mazzitelli

unread,
Nov 3, 2020, 2:01:30 PM11/3/20
to kiali-users


----- Original Message -----
> @jmaz...@redhat.com Thank you but it does not help me.
> I want to see the "istio-ingressgateway" in my graph on the "default"
> namespace in Kiali.
>
> If I switch to namespace "istio-system" in Kiali, I get the object
> "istio-ingressgateway" as source of my traffic.
> So I want to see the same but for the "default" namespace ( and having the
> istio-ingress installed in "istio-system" namespace ).
>

What happens if you select to view the "istio-system" namespace AND the "default" namespace in the Kiali namespace drop down menu? I suspect you will still see that "unknown" node.

Keep in mind the Kiali graph is merely visualizing the underlying telemetry that Istio stores in Prometheus. So the "unknown" traffic might actually just be what is in telemetry and Kiali is simply showing it. So, I would also look at this FAQ "How do I inspect the underlying metrics used to generate the Kiali Graph?" and see if it helps: https://kiali.io/documentation/latest/faq/#prometheus

If you are seeing "unknown" telemetry in Prometheus itself, then this is not a Kiali misconfiguration, it is simply the way your Istio mesh is configured and working. In that case, it might be worth while to ask this on the Istio Slack or mailing list to see if anyone would know in the core Istio community. See: https://istio.io/latest/about/community/join/
> https://groups.google.com/d/msgid/kiali-users/e0533e93-304f-487c-9661-13785e1b75f7n%40googlegroups.com.
>

Jay Shaughnessy

unread,
Nov 3, 2020, 2:21:48 PM11/3/20
to kiali...@googlegroups.com


You see unknown when the namespace is "default", and not "istio-system", because the traffic is coming from unknown and into "default", but "unknown" is not sending requests into "istio-system".   When using "default" are you also seeing the expected traffic from ingressgateway?  I'm sort of thinking you are, because when setting the namespace to "istio-system" you say you do see the expected traffic, and that would indicate that traffic is in flact flowing from ingressgateway into "default".

As for the traffic from unknown, it is either real traffic coming from "the wild" and not from a source-proxy, or it is some sort of bad telemetry generated by Istio.  I see that almost all of thos edges are gray, indicating no recent traffic, so maybe it was some sort of bad telemetry. I'm not sure.

To clean up the graph, you could type "!traffic" into graph hide.  That should eliminate all of those edges with no traffic.

Mircea Marius

unread,
Nov 3, 2020, 4:10:37 PM11/3/20
to Jay Shaughnessy, jmaz...@redhat.com, kiali...@googlegroups.com
Hi,

First of all , I cleaned the graph with hide: "!traffic". It's much cleaner now.

I selected both namespaces : "istio-system" and "default" and it si clearly that something is wrong.

image.png

The "tree" in the right containing the "ingress" is from "istio-system" and the "unknown" tree is from "default" -> where the mailhog service and pod is located.

So, from the perspective of the ingressgateway , "mailhog.default.svc.cluster.local" is considered to be UNKNOWN destination (and it should be "default" namespace where kiali has access). Another problem is that the service "kiali.istio-system.svc.cluster.local" is located in the same ns as the ingressgateway ( istio-system ) and it is still seen as "unknown" destination.

So in this configuration that I have, the virtual-service and the gateway are installed in the "default" namespace. Is that the best-practice ? Or they should have been installed in the istio-system namespace ?

I do not know what to do more...

Regards,
Mircea


You received this message because you are subscribed to a topic in the Google Groups "kiali-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kiali-users/GK4HsgWqA-Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kiali-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kiali-users/68a6d226-4cd7-0c92-9654-b02df18a46c3%40redhat.com.

Jay Shaughnessy

unread,
Nov 3, 2020, 4:58:41 PM11/3/20
to kiali-users
I think your next step is to ask in Istio Discuss, or istio Slack about why you would see what you are seeing.  From looking at your graph it seems the request telemetry is failing in "metadata exchange".  That means the requests from ingressgateway to, say, mailhog, are probably happening just fine but the telemetry is being broken into two pieces, represented in one part as gateway to the bad service node and the second part as unknown to the mailhog service.  If you were to change from a Service graph to, say, a workload graph, you'd likely see edges from unknown to the actual mailhog workload.   I don't think ths is a Kiali issue but rather a problem in Istio telemetry.  You can also use Kiali to check for validation issues in your Istio Config.
Reply all
Reply to author
Forward
0 new messages