Logging under the same mlflow experiment name, runs that has been done on independent GPUs
Rosana El Jurdi
Hello has anyone worked before on mlflow, particularly has tried to log to the same mlflow server from different gpu machines same experiment name but different runs ?
ex: here the same experiment is ran on two machine but via two different runs. despite the fact that it has the exact name, mlflow logs them as seperate experiments. This doesnt happen when I run on the same machine but with different run names. In fact , when that happens mlflow runs the two experiments in the same experiment directory
example: I am running a segmentation model given two different models. I specify the experiment name as segmentation and the run id for the first experiment as model A and the second experiment as model B. If these two models are ran from the same GPU server, mlflow logs them both under:
experiment name : segmentation -> run name : model A -> run name: model B
If model A is run on vidar while model B is run on freyja, despite the fact that the have same experiment names mlflow logs them as follows:
experiment name : segmentation -> run name : model A experiment name : segmentation -> run name: model B
Thank you for your time,