When using the "From a file on the Kubernetes master node" option for retrieving the Kubernetes cluster config file, the "File" field is ignored and the plugin always attempts to access ~/.kube/config.
I have prepared a pull request which appears to fix the issue, although I'm not that familiar with the Jenkins code base or with Java for that matter.
See https://github.com/jenkinsci/kubernetes-cd-plugin/pull/86
Thanks Rik Harris, I will help you to merge the fix.