I'd like to second this sentiment.
I personally don't see the harm in allowing certain jobs to be run as admin in order to gain the ability to gain workspace auth - people say that it's a glaring security weakness because if people get access to the git access, then they can do anything on the Jenkins server. The pipeline shared libraries have the same problem though, so that doesn't hold water with me (if we're talking about restricting it to admins).
Even if it doesn't satisfy full security needs, I'd love to have the middle ground of "only admins can run jobs that import local files" as a mode. It'd be more secure than we have to do right now.
I'd even try to take a whack at implementing it myself if I was pointed in the right direction.