Event handlers triggered by asynchronous browser events (i.e. user interactions, such as button clicks, XHR callbacks, and setTimeout callbacks) are always outside of AngularJS and so always need to be wrapped in a $apply.
The only time you may have a problem is if you are using a 3rd party library that provides callbacks (Facebook API, for example). In this case there is a chance that they sometime call their callbacks asynchronously and sometimes synchronously.
In the case of FB, there is an authentication method, I think, that calls the callback synchronously if there is a clientside error, otherwise passes the callback onto XHR which will call it asynch otherwise. Only in these cases do you need to worry. The easiest solution is to ensure the callbacks are always asynch by wrapping them in a $timeout(callback, 0) call.
Pete