First, deleting the files in the cache directory is not the way to go if you want to clear the cache. See:
As for your question: If you don't restrict the hostnames that a pagespeed-enabled vhost responds to, mod_pagespeed may write entries for request to any hostname, if in-place resource optimization is enabled. So maybe you are seeing artifacts of automated processes crawling the server. These cache entries don't cause any harm, but you could get rid of them by one of the following:
- Restricting the hostnames that the pagespeed enabled vhost responds to in Apache's configuration
- Turn off in-place resource optimization (which is the feature that is writing the entries you see into the cache)
Otto