Hi Michel,
no, expect BRouter to be significantly slower on a phone compared to a Desktop- or Server-System. I cannot really tell the reason, because the clock-rates are not so much different, but there's also file-system buffer cache vs. plain sd-card access, HotSpot VM versus Dalvik etc.
You can check Harry's tests on osmands forum for some house numbers:
https://groups.google.com/d/msg/osmand/Scdy74-CixA/c8-G5rQJnooJHowever, please consider that processing time scales quadratic with distance, so what's getting a problem for long-distance application is no problem at all for "normal" distances. Please check also the explanation on "timeout-free recalculations" in that osmand-forum-thread, this is a working solution for handling long-distance trips, so I don't really see so much problems from the performance side.
In short: buy any (cheap) phone, you don't need a high-end device.