RE: Cross cluster communication is not working in submariner

[Vaishnavi Rajulu]

Hi Everyone,

I have two clusters named A and B. In cluster-A I have deployed submariner broker and joined both the clusters. But when I try cross cluster pod accessing its not working

Cluster-A

subctl show all
Cluster "default"
 ✓ Detecting broker(s)
NAMESPACE               NAME                COMPONENTS                        GLOBALNET   GLOBALNET CIDR   DEFAULT GLOBALNET SIZE   DEFAULT DOMAINS
submariner-k8s-broker   submariner-broker   service-discovery, connectivity   yes         242.0.0.0/8      65536      

 ✓ Showing Connections
GATEWAY   CLUSTER    REMOTE IP       NAT   CABLE DRIVER   SUBNETS        STATUS      RTT avg.
worker2   cluster2   2.x.x.x   no    libreswan      242.1.0.0/16   connected

 ✓ Showing Endpoints
CLUSTER    ENDPOINT IP       PUBLIC IP         CABLE DRIVER   TYPE
cluster1   1.x.x.x   1.x.x.x   libreswan      local
cluster2   2.x.x.x     2.x.x.x     libreswan      remote

 ✓ Showing Gateways
NODE   HA STATUS   SUMMARY
k8-2   active      All connections (1) are established

 ✓ Showing Network details
    Discovered network details via Submariner:
        Network plugin:  generic
        Service CIDRs:   [10.43.0.0/16]
        Cluster CIDRs:   [10.42.0.0/16]
        Global CIDR:     242.0.0.0/16

 ✓ Showing versions
COMPONENT                       REPOSITORY           CONFIGURED   RUNNING                     ARCH
submariner-gateway              quay.io/submariner   0.18.0       release-0.18-e3f3e56b57fe   amd64
submariner-routeagent           quay.io/submariner   0.18.0       release-0.18-e3f3e56b57fe   amd64
submariner-globalnet            quay.io/submariner   0.18.0       release-0.18-e3f3e56b57fe   amd64
submariner-metrics-proxy        quay.io/submariner   0.18.0       release-0.18-011349c6f17e   amd64
submariner-operator             quay.io/submariner   0.18.0       release-0.18-68fefdd74105   amd64
submariner-lighthouse-agent     quay.io/submariner   0.18.0       release-0.18-02b6a5b37266   amd64
submariner-lighthouse-coredns   quay.io/submariner   0.18.0       release-0.18-02b6a5b37266   amd64

subctl version: v0.18.0

Cluster-B

subctl show all
Cluster "default"
 ✓ Detecting broker(s)
 ✓ No brokers found

 ✓ Showing Connections
GATEWAY   CLUSTER    REMOTE IP         NAT   CABLE DRIVER   SUBNETS        STATUS   RTT avg.
k8-2      cluster1   1.x.x.x   no    libreswan      242.0.0.0/16   error    0s

 ✓ Showing Endpoints
CLUSTER    ENDPOINT IP       PUBLIC IP         CABLE DRIVER   TYPE
cluster2   2.x.x.x                       2.x.x.x              libreswan      local
cluster1   1.x.x.x                       1.x.x.x                libreswan      remote

 ✓ Showing Gateways
NODE      HA STATUS   SUMMARY
worker2   active      0 connections out of 1 are established

 ✓ Showing Network details
    Discovered network details via Submariner:
        Network plugin:  calico
        Service CIDRs:   [10.43.0.0/16]
        Cluster CIDRs:   [10.42.0.0/16]
        Global CIDR:     242.1.0.0/16

 ✓ Showing versions
COMPONENT                       REPOSITORY           CONFIGURED   RUNNING                     ARCH
submariner-gateway              quay.io/submariner   0.18.0       release-0.18-e3f3e56b57fe   amd64
submariner-routeagent           quay.io/submariner   0.18.0       release-0.18-e3f3e56b57fe   amd64
submariner-globalnet            quay.io/submariner   0.18.0       release-0.18-e3f3e56b57fe   amd64
submariner-metrics-proxy        quay.io/submariner   0.18.0       release-0.18-011349c6f17e   amd64
submariner-operator             quay.io/submariner   0.18.0       release-0.18-68fefdd74105   amd64
submariner-lighthouse-agent     quay.io/submariner   0.18.0       release-0.18-02b6a5b37266   amd64
submariner-lighthouse-coredns   quay.io/submariner   0.18.0       release-0.18-02b6a5b37266   amd64

[Aswin Suryanarayanan]

Hi Vaishnavi,

If globalnet is enabled you need to create the GlobalEgressIP CR  as mentioned in the link above, for pod to pod connectivity.

Thanks

Aswin

On Wed, Sep 18, 2024 at 10:28 PM Vaishnavi Rajulu <vaishna...@gmail.com> wrote:

Hi Aswin,

Last time I did the same deployment without enabling globalnet but back there also I was having overlapping cidrs and it automatically configured a globalcidr. 

Validating globalnet configuration

Assigning Globalnet IPs

Using Pre-Configured global cidr 240.2.0.0/16

Currently I am trying to do the same but it doesn't allocate a cidr that is why i have enabled globalnet

On Wed, Sep 18, 2024 at 10:53 PM Aswin Suryanarayanan <asur...@redhat.com> wrote:

Hi Vaishnavi,

When globalnet is enabled , pod to pod connectivity is not available by default. If you create a service and export it, Submariner will allocate a global IP and resolve it. SImilarly if you export a headless service, Submariner will allocate IP for each pods backing the service.

If you want to access pod a using the global IP, you can refer to the section "Global IPs for a set of pods?" here [1]

Also if you don't want a global net (the clusters do not have overlapping CIDRs) you can disable globalnet while deploying. In non-globalnet deployments remote pods Ips are reachable by default.

Thanks,

Aswin

[1]https://submariner.io/getting-started/architecture/globalnet/ 

--
You received this message because you are subscribed to the Google Groups "submariner-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to submariner-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/submariner-users/df220925-25e7-4f8d-9b4a-51beee214e66n%40googlegroups.com.