Is it possible to use the clustered eventbus in the stop method?

38 views
Skip to first unread message

Jan Fengler

unread,
Apr 20, 2022, 6:37:13 AMApr 20
to vert.x

Hello all,

we have stumbled upon a problem during the shutdown of one node of our cluster:
We want to use the eventbus during stop() to send some data (e.g. running timers) over the net to another cluster member and that simply does not work.
Locally the bus is usable but the clustered communication is no longer possible in stop(), because of the following:
(VertxImpl.java, line 559ff, vertx 4.2.4):
  @Override
  public synchronized void close(Handler<AsyncResult<Void>> completionHandler) {
    if (closed || eventBus == null) {
      // Just call the handler directly since pools shutdown
      if (completionHandler != null) {
        completionHandler.handle(Future.succeededFuture());
      }
      return;
    }
    closed = true;
    closeFuture.close().onComplete(ar -> {
      deploymentManager.undeployAll().onComplete(ar1 -> {
        HAManager haManager = haManager();
        Promise<Void> haPromise = Promise.promise();
        if (haManager != null) {
          this.executeBlocking(fut -> {
            haManager.stop();
            fut.complete();
          }, false, haPromise);
        } else {
          haPromise.complete();
        }
        haPromise.future().onComplete(ar2 -> {
          addressResolver.close(ar3 -> {
            Promise<Void> ebClose = getOrCreateContext().promise();
            eventBus.close(ebClose);
            ebClose.future().onComplete(ar4 -> {
              closeClusterManager(ar5 -> {
                // Copy set to prevent ConcurrentModificationException
                deleteCacheDirAndShutdown(completionHandler);
              });
            });
          });
        });
      });
    });
  }

during closeFuture.close() the NetClient of the Eventbus is closed and therefore during undeployAll() the cluster communication is no longer usable.

The ClusteredEventBus creates the NetClient as follows (ClusteredEventBus.java, line 67ff):
  public ClusteredEventBus(VertxInternal vertx, VertxOptions options, ClusterManager clusterManager, NodeSelector nodeSelector) {
    super(vertx);
    this.options = options.getEventBusOptions();
    this.clusterManager = clusterManager;
    this.nodeSelector = nodeSelector;
    this.client = vertx.createNetClient(new NetClientOptions(this.options.toJson()));
  }

And that creates it (in VertxImpl.java) this way:
  public NetClient createNetClient(NetClientOptions options) {
    CloseFuture closeFuture = new CloseFuture(log);
    NetClient client = createNetClient(options, closeFuture);
    CloseFuture fut = resolveCloseFuture();
    fut.add(closeFuture);
    return client;
  }


Is it intentionally impossible to send something over a ClusteredEventbus during stop(), is this an error that will be fixed or is there another way/hook that can use the eventbus during and before shutdown? What did we miss?

Thx
  Jan Fengler

Jan Fengler

unread,
Apr 21, 2022, 6:50:29 AMApr 21
to vert.x
Hi all,

as I was informed, this is a known bug: https://github.com/eclipse-vertx/vert.x/pull/4327
Signed-off-by: Markus Spika markus...@gmail.com Motivation: was looking for this over a year now! finally found it.... Conformance: You should have signed the Eclipse Contributor Agreement as ex...
But I am unsure about the discussed solutions.
My proposal to fix this is to avoid registering the NetClient for the ClusteredEventBus with the closeFuture in VertxImpl on the instantiation of ClusteredEventBus.
Only downside is, that it has to be removed "manually" during the shutdown of the ClusteredEventBus later during the shutdown, but is'nt that the easiest solution for that problem?

During our investigation we created a workaround - a new hook that is executed before the entire shutdown-process, but that is only that - a workaround, but maybe this is also a possible way out of this problem?

I only wanted to share these ideas, because I cannot determine, if they are useful as I am not a vertx-developer.

Thx
  Jan Fengler



Von: ve...@googlegroups.com <ve...@googlegroups.com> im Auftrag von Jan Fengler <jan.f...@adviqo.com>
Gesendet: Mittwoch, 20. April 2022 12:37
An: vert.x <ve...@googlegroups.com>
Betreff: [vertx:54534] Is it possible to use the clustered eventbus in the stop method?
 
--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/vertx/000aa60f-3300-1c26-eaab-c7452c79d223%40adviqo.com.

Thomas SEGISMONT

unread,
Apr 26, 2022, 9:18:13 AMApr 26
to ve...@googlegroups.com
Hi Jan,

The problem you describe is not the same as in issue #4326

Would you mind sharing a small reproducer on GH? I have an idea about what happens but I'd like to be sure about the problem.

Thank you,
Thomas



Message has been deleted
Message has been deleted
Message has been deleted

Jan Fengler

unread,
Jun 1, 2022, 7:42:11 AMJun 1
to vert.x
Hi Thomas,

sorry for the delay. I do not think Github is necessary for this small amount of static code, so I have pasted the code here: https://drive.google.com/file/d/1HFWRd9RsZiwpJpO09ohrfSNIJC0qxvLj/view?usp=sharing
I hope this link works and it will help, if not, please let me know.

Thx
  Jan


Thomas SEGISMONT

unread,
Jun 1, 2022, 3:42:10 PMJun 1
to ve...@googlegroups.com
Thank you. Please subscribe to issue 4327. You were right it seems related. I will follow-up on GH.

--
You received this message because you are subscribed to the Google Groups "vert.x" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vertx+un...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages