The term "single page app" is a misnomer in my opinion, because as you use "single page app" you see many pages. "Single page" just means it loads once in a single web server call, and then all subsequent interactions manipulate parts of the DOM and talk to an API on the back -end without reloading the page.
You don't have to "hack your way" to build an app that you can click from page-to-page, you just have to know how to use the framework correctly.
Angular and Ember are both relatively new (< 2 years old), and AFAIK there's a lot of active development going on. If you want to be on the bleeding edge and don't mind a steep learning curve, they are both worth learning.
If you perhaps don't want to be on the bleeding edge and want to stick with Rails, perhaps now isn't the right time to learn Ember or Angular.
Backbone, while very powerful, requires more coding and without Marionette you have to wire up a lot of UI updates manually (Ember and Angular take a more declarative approach and you have to do less manual UI updating). Marionette eliminates some of that, but you still have a lot of code to write and test.
I would classify all three as for experienced Javascript developers and wouldn't recommend any of them unless you had a solid foundation in the nuances and inner working of advanced Javascript (like scoping, event bubbling, and promises).