Hi,
You're exactly right-- the backend transcodes on the fly. This means you need a rather hefty system (or, if you have a middling system, you need to opt for low bitrates and/or resolutions) to accomplish the transcodes. There are some tweaks/undocumented multithreading features that may help, but they won't be a night and day difference. At best, they will provide a marginal improvement. Fundamentally your backend needs to be able to transcode to your resolution and bitrate of choice at real time or better (that is, the playback framerate or faster).
Here's a thread that can help you insert the setting to increase the number of transcode threads:
Hope this helps,
Robert