In case anyone has the same issue, I managed to struggle my way through this:
You can use the current version of rpyc, in combination with plumbum. Plumbum provides a 'PuttyMachine', which can be used to connect to the remote server and create a tunnel. However, for some reason the PuttyMachine.tunnel causes errors which I was not able to solve. I started out by setting up my own tunnel from command-line, and retrofitted this to plumbum.
My solution now looks like this:
keyfile = "key"
local_port = 1111
remote_port = 18861
remote_ip = "x.x.x.x"
remote_user = "user"
ssh_command = plumbum.local["C:\Program Files (x86)\PuTTY\plink.exe"][('-L','{}:localhost:{}'.format(local_port,remote_port))]
scp_command = plumbum.local["C:\Program Files (x86)\PuTTY\pscp.exe"]
remote = plumbum.PuttyMachine(remote_ip, user=remote_user, port=22, keyfile=keyfile,ssh_command=ssh_command, scp_command=scp_command)
with remote:
conn = rpyc.connect("localhost", local_port)
# Do rpyc stuff here