So when you add a private key for use to connect to agents, that private key will be in the System store. The System store maintains two scopes: SYSTEM (which is only available for "system" tasks and not available to build jobs) and GLOBAL (which is available for both "system" tasks and build jobs)
A secured Jenkins server will not permit build jobs to run on the master's JVM, only on agents. Thus no build job will ever have access to the master's filesystem.
Next, if the key is important to you, you should be storing the key protected with a passphrase. The
passphrase in Jenkins is encrypted so that should add an extra layer of security (as somebody need to grab the master key for Secret as well as your SSH private key)...
There are some issues that people have found in plugins when using SSH private keys that are protected by a passphrase (these are in general due to the plugins using those credentials on agents incorrectly... doing bold things like bringing classes that are intended to be instantiated only on the master JVM over to the agent JVM and trying to instantiate those classes there... unsurprisingly such things do not work... but the poor innocent credentials plugin gets the blame... I am pushing to make the root cause more visible in JENKINS-38370 to JENKINS-38373) some people suggest that one should not use passphrase protected private keys in Jenkins because of the bad coding issues in plugins... I disagree...
at least for the use case of a private key being used to *launch* a build agent... such private keys... if you value their security... should be encrypted with a passphrase and stored with SYSTEM scope in the System store, if you ensure that the master has 0 executors then that will make the key as secure as can be. Rotate the key on a semi-regular basis for extra security.