To expound:
I actually use routeProvider only for templates and specify my controllers in the root node of the template. Instead of ng-view, I use ng-include and watch for $routeChangeSuccess to update the template reference as necessary. What this does is give me control over whether or not I load a new template as I have some routes that point to the same template and don't want them to "reload", but simply have the same controller handle the route parameters differently. (this saves the user a "blink" on the page and prevents it from making the same ajax call twice for the same data that only has a slightly different view).
IMHO, ng-view is just a convenience wrapper for using ng-include with ng-controller mapped to specific routes.
I could be wrong, but I do not believe there's any special functionality happening when controllers are specified in routeProvider versus in the template.