Yes, believe me that I feel your pain :)
This is, unfortunately, incredibly difficult with the way that latex and the makefile work together. If it was just latex, there are command line options to do that. But, add in all of the automatic dependency generation logic in the makefile and you suddenly have a huge mess on your hands. I'm afraid that, though I wish it weren't so, the answer to this question is "no".
I'm very sorry. It does leave a lot of crap around. At least "make clean" works pretty well. :-)
I wonder if it would help if there were a "make ignorelist" or something similar that told you which files you could safely ignore in your version control? It wouldn't be perfect, but it might help at least to get you a good ignore file set up.