This seems to be fixed by jenkinsci/swarm-plugin#190.
Released in 3.19. Can you please confirm that this release addresses the problem? Thank you!