For displaying in Google Earth, just use normal webserver password protection - eg Basic Auth, via .htaccess files.
But if you are displaying the KML in Google Maps there is no way. As you note their server has to be able to read the file. And the identity of the user is not passed to you, so you have to open it everyone.
The best can do is give it a really hard to guess URL, then the likelyhood of someone getting the URL is small.
Make sure robots.txt etc prevent indexing the file, to help prevent web-crawlers accidently indexing it.