Kiali graphs shows "unknown" instead of "istio-ingressgateway" as source of traffic in the mess

[niscovea...@gmail.com]

Hello,

I have applications deployed on Kubernetes in the "default" namespace.

The "istio-ingressgateway" is deployed in the "istio-system" namespace.

I have created a Gateway and a VirtualService in order to configure the ingress traffic.

The problem is that I expect that in Kiali graphs to see (as below) the "istio-ingressgateway" as source of the traffic in the mesh.

What I get is "unknown" node in the graphs:

Can anyone help with this issue please ? Is it configuration issue ?

Gateway.yaml:

```

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: default-gateway

  namespace: default

spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"

```

VirtualService.yaml

```

kind: VirtualService
metadata:
  name: vs-ingress

  namespace: default

spec:
  hosts:
  - "*"
  gateways:
  - default-gateway
  http:
  - match:
    - headers:
        Host:
          exact: mail.test.comp.com
    route:
    - destination:
        host: mailhog
        port:
          number: 8025
  - match:
    - headers:
        Host:
          exact: cart.test.comp.com
    route:
    - destination:
        host: shopping
        port:
          number: 80

```

My Setup:

Kiali UI

1.25.0 (6646d8b248dad2f8aea0fcd915085065f25f2776)

Kiali Server

v1.25.0 (275a5cba051ed25677a513706cd9eff36dc58d78)

Kiali Container

v1.25.0

Components

Istio

1.7.3

Prometheus

2.19.0

Kubernetes

v1.19.2

[John Mazzitelli]

----- Original Message -----
> Hello,
>
> I have applications deployed on Kubernetes in the "default" namespace.
> The "istio-ingressgateway" is deployed in the "istio-system" namespace.
>
> I have created a Gateway and a VirtualService in order to configure the
> ingress traffic.
>
> The problem is that I expect that in Kiali graphs to see (as below) the
> "istio-ingressgateway" as source of the traffic in the mesh.
>

> What I get is "unknown" node in the graphs:
>

Does the FAQ help? "Why are there many unknown nodes in the graph?"

https://kiali.io/documentation/latest/faq/#many-unknown

> --
> You received this message because you are subscribed to the Google Groups
> "kiali-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kiali-users...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/kiali-users/7835a574-f110-432f-a392-28118a2f5657o%40googlegroups.com.
>

[Mircea Marius]

@jmaz...@redhat.com  Thank you but it does not help me.

I want to see the "istio-ingressgateway" in my graph on the "default" namespace in Kiali.

If I switch to namespace "istio-system" in Kiali, I get the object  "istio-ingressgateway" as source of my traffic.

So I want to see the same but for the "default" namespace ( and having the istio-ingress installed in "istio-system" namespace ).

[John Mazzitelli]

----- Original Message -----
> @jmaz...@redhat.com Thank you but it does not help me.
> I want to see the "istio-ingressgateway" in my graph on the "default"
> namespace in Kiali.
>
> If I switch to namespace "istio-system" in Kiali, I get the object
> "istio-ingressgateway" as source of my traffic.

> So I want to see the same but for the "default" namespace ( and having the
> istio-ingress installed in "istio-system" namespace ).
>

What happens if you select to view the "istio-system" namespace AND the "default" namespace in the Kiali namespace drop down menu? I suspect you will still see that "unknown" node.

Keep in mind the Kiali graph is merely visualizing the underlying telemetry that Istio stores in Prometheus. So the "unknown" traffic might actually just be what is in telemetry and Kiali is simply showing it. So, I would also look at this FAQ "How do I inspect the underlying metrics used to generate the Kiali Graph?" and see if it helps: https://kiali.io/documentation/latest/faq/#prometheus

If you are seeing "unknown" telemetry in Prometheus itself, then this is not a Kiali misconfiguration, it is simply the way your Istio mesh is configured and working. In that case, it might be worth while to ask this on the Istio Slack or mailing list to see if anyone would know in the core Istio community. See: https://istio.io/latest/about/community/join/

> https://groups.google.com/d/msgid/kiali-users/e0533e93-304f-487c-9661-13785e1b75f7n%40googlegroups.com.
>

[Jay Shaughnessy]

You see unknown when the namespace is "default", and not "istio-system", because the traffic is coming from unknown and into "default", but "unknown" is not sending requests into "istio-system".   When using "default" are you also seeing the expected traffic from ingressgateway?  I'm sort of thinking you are, because when setting the namespace to "istio-system" you say you do see the expected traffic, and that would indicate that traffic is in flact flowing from ingressgateway into "default".

As for the traffic from unknown, it is either real traffic coming from "the wild" and not from a source-proxy, or it is some sort of bad telemetry generated by Istio.  I see that almost all of thos edges are gray, indicating no recent traffic, so maybe it was some sort of bad telemetry. I'm not sure.

To clean up the graph, you could type "!traffic" into graph hide.  That should eliminate all of those edges with no traffic.

[Mircea Marius]

Hi,

First of all , I cleaned the graph with hide: "!traffic". It's much cleaner now.

I selected both namespaces : "istio-system" and "default" and it si clearly that something is wrong.

The "tree" in the right containing the "ingress" is from "istio-system" and the "unknown" tree is from "default" -> where the mailhog service and pod is located.

So, from the perspective of the ingressgateway , "mailhog.default.svc.cluster.local" is considered to be UNKNOWN destination (and it should be "default" namespace where kiali has access). Another problem is that the service "kiali.istio-system.svc.cluster.local" is located in the same ns as the ingressgateway ( istio-system ) and it is still seen as "unknown" destination.

So in this configuration that I have, the virtual-service and the gateway are installed in the "default" namespace. Is that the best-practice ? Or they should have been installed in the istio-system namespace ?

I do not know what to do more...

Regards,

Mircea

You received this message because you are subscribed to a topic in the Google Groups "kiali-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/kiali-users/GK4HsgWqA-Q/unsubscribe.
To unsubscribe from this group and all its topics, send an email to kiali-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kiali-users/68a6d226-4cd7-0c92-9654-b02df18a46c3%40redhat.com.

[Jay Shaughnessy]

I think your next step is to ask in Istio Discuss, or istio Slack about why you would see what you are seeing.  From looking at your graph it seems the request telemetry is failing in "metadata exchange".  That means the requests from ingressgateway to, say, mailhog, are probably happening just fine but the telemetry is being broken into two pieces, represented in one part as gateway to the bad service node and the second part as unknown to the mailhog service.  If you were to change from a Service graph to, say, a workload graph, you'd likely see edges from unknown to the actual mailhog workload.   I don't think ths is a Kiali issue but rather a problem in Istio telemetry.  You can also use Kiali to check for validation issues in your Istio Config.

[Janey Alam]

I know it's been a couple of years since this discussion started but I just wanted to check if we found the root cause and any solutions. I am also getting the same problem now. 

[Jay Shaughnessy]

Hi Janey,

If you are using new versions of Istio and Kiali, and you are seeing this sort of issue, the root cause is likely the same.  Kiali is probably just visualizing the telemetry being reported by Istio.  And so the root cause would likely be something to pursue with the Istio folks.  Note that sometimes an unexpected visualization is in fact correct, and could indicate the true behavior.

As an aside, this mailing list is deprecated in favor of either the #kiali channel on Istio Slack, or Kiali github discussions.

To view this discussion on the web visit https://groups.google.com/d/msgid/kiali-users/8986fcb6-9827-4a71-8225-7aba31e7b213n%40googlegroups.com.