Having a look at the code it seems that the plugin is implemented as a build Listener and not a build Notifier that's why it is executed for all jobs on the server where it is installed.
A build Notifier should be the pattern to use to be sure to activate/configure it only on jobs where it is useful.