I've managed to do it.
All you need to do is download spf13-vim using Git following the windows install notes. Then manually copy the contents of \.spf13-vim-3 to gVimPortable\Data\settings
You will need to make a symlink to .vimrc from a file called _vimrc in that same folder, or a line in the original _vimrc file like this:
This is so that your portable vim uses the spf13-vim configuration file.
Open gVim Portable via it's exe, and type ":BundleInstall"
Wait for Bundles to be loaded.
If all went well you will have a fully portable spf13-vim, although if you want to update in the usual way, via vundle using git and curl, you will need to set them up in every computer you use.