in our team, we recently encountered a behavior of the JanusGraph management system, that we didn't expect.
We have some (currently 24) JanusGraph instances running in a Docker swarm. Every now and then, some of these instances are restarted, e.g. for maintenance or when healthchecks fail.
For us, it was expected that
should return a list of those 24 instances, that are currently running. This is not the case. Instead, over the course of months or even years, JanusGraph had accumulated tens of thousands of instances, that are seemingly still unclosed.
Why is this relevant?
In our case, we needed to stop all open instances in order to apply a change to a GLOBAL_OFFLINE config. Even though it was possible to close all instances via the management API, this process took about half an hour.
We thought it would be much cleaner if we found a way to automatically remove killed instances from the internal list of open instances. Does anybody know a way to accomplish this? Or does something like this require additional features in JanusGraph to be implemented?