If you're changing JS, CSS or CoffeeScript files and running in production mode, then yes you need to recompile assets to see the change. If you're changing ruby files then you don't need to recompile assets, but you do need to restart Canvas.
For development work, you should probably be running in development mode (RAILS_ENV=development). It will use uncompiled static assets and ruby classes will be reloaded on each page view so it will be very slow. If you need to run in production but don't want to have to recompile assets, you can tweak config/environments/production.rb (I think, not at a computer right now) to not used compiled JS. You don't want to run in actual production like that, though.