This is problem that seems to affect also other installed versions for some people, not only portable version.
I have played more with it and noticed that PNG files are loaded without problems as they don't require plugin.
You are right, in App/Lib/site-packages/PyQt4 there is qt.conf file that should by defaults pick up plugins as default value for Plugins (if you don't specify it) is "plugins" folder.
Workaround for portable version is to put qt.conf file in the App folder (next to python.exe) with following content
[Paths]
Prefix = .
Binaries = .
Plugins = ./Lib/site-packages/PyQt4/plugins
with this I was able to load jpeg's and gif's without problem.
please let me know does this help, I will include this fix in the next release.
keep pythoning !