One idea is that instead of puppet on an agent doing something, it might make sense to have something watch puppetDB contents for the success of applying the class/package/file resources in question. As long as they are idempotent then it would only trigger once. I don't delve into puppetDB directly much, so sorry for the vagueness in this suggestion.
I assume this is something assigned during provisioning or when a feature flag is flipped. If so, you could also put the notification element there - notify the end user if success, notify the provisioning team if failure.
Finally, if you do resort to an exec, you can set refreshonly to true and notify it from the dependent resources. There's still no guarantee this only happens once, for instance if someone modifies the package resource by hand and puppet reverts it, but that's kind of outside puppet's scope anyway. The exec itself would still only fire based on the business rules you codify in your puppet code.
Lots of ways to tackle this one.
--
Rob Nelson