ptadapter: a Python Pluggable Transport interface library and standalone tunnel tool

[twisteroid ambassador]

ptadapter https://github.com/twisteroidambassador/ptadapter is a Python 3 package implementing the Pluggable Transport (v1) IPC interface, written by me. It serves as the controlling side ("acting as Tor"), allowing Python programs to use Pluggable Transports, either as a client to make obfuscated outgoing connections, or as a server to receive obfuscated connections. It also includes a console script to setup standalone tunnels that can be used without coding.

I actually started this a long time ago since I needed to run an obfs4 tunnel. The code has been rewritten from scratch a couple of times, and with the latest rewrite it's finally good enough for general consumption. It can be installed using pip:

pip install ptadapter

Visit the project home page for documentation, examples and other details: https://github.com/twisteroidambassador/ptadapter

Technical details:

ptadapter requires Python 3.7 or above. It is implemented in asyncio, and supports the usual StreamReader and StreamWriter semantics. It has no 3rd-party dependencies, and includes its own implementation of SOCKS4, SOCKS5 and ExtOrPort. Support for ExtOrPort means servers can get the client's original address and transport name.

The console script uses the "console_script" entry point, so after pip install-ing, you can often run "ptadapter" directly from the command line. It uses config files in INI (configparser) syntax, and supports multiple client tunnels per transport per process, and multiple server transports per process.

I'm thinking about implementing Version 2 of the PT IPC interface next.

[David Fifield]

On Sun, Jan 06, 2019 at 07:15:25AM -0800, twisteroid ambassador wrote:
> The console script uses the "console_script" entry point, so after pip
> install-ing, you can often run "ptadapter" directly from the command line. It
> uses config files in INI (configparser) syntax, and supports multiple client
> tunnels per transport per process, and multiple server transports per process.

This is a cool project. I haven't tried the Python library adapter, but
the console script adapter allows you to transform a program that only
supports the managed-transport IPC interface into one that just exposes
a socket at both ends, making it usable by other programs that don't
know about pluggable transports or even SOCKS.

Here's an example of setting up an obfs4 tunnel for a netcat channel
(connecting a local stdin with a remote stdout).

On the server, create a configuration file server.cfg:
[server]
exec = /usr/bin/obfs4proxy
forward = 127.0.0.1:10000 # this is where the server is running
tunnels = my_obfs4_server
[my_obfs4_server]
transport = obfs4
listen = 0.0.0.0:9000 # this is the external obfs4 port
Then on the server, run
ncat -l -k -v 10000
ptadapter server.cfg

On the client, create a configuration file client.cfg:
[client]
exec = /usr/bin/obfs4proxy
tunnels = my_obfs4_client
[my_obfs4_client]
transport = obfs4
listen = 127.0.0.1:8000 # this is the local listening port
upstream = server.example:9000 # matches "listen" in server.cfg
# Copy from state/obfs4_bridgeline.txt
options-cert = Eutc5OY9bk14y+HscnhsoFZTtf8py3R21PFHQQbyWiY1RYrpLQ6TQfAOIA9kIXu5eXj0Yw
options-iat-mode = 0
Then on the client, run
ptadapter -C client.cfg
ncat -v 127.0.0.1 8000 # matches "listen" in client.cfg

Now you can type into the client-side netcat and see the text appear on
the server-side netcat, with an obfs4 tunnel between them.


For comparison, here is how an equivalent configuration would look in
torrc. Server side:
ServerTransportPlugin obfs4 exec /usr/bin/obfs4proxy
ORPort 127.0.0.1:10000
ServerTransportListenAddr obfs4 0.0.0.0:9000
Client side:
ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy
SOCKSPort 127.0.0.1:8000 # not equivalent; ptadapter doesn't use SOCKS
Bridge obfs4 server.example:9000 cert=Eutc5OY9bk14y+HscnhsoFZTtf8py3R21PFHQQbyWiY1RYrpLQ6TQfAOIA9kIXu5eXj0Yw iat-mode=0

ptadapter's configuration syntax is better. For example, in ptadapter
you can run multiple instances of obfs4 on separate ports (you would
just add an additional section like [my_obfs4_server_2]). You can't do
that with Tor, even though the PT protocol allows it, because the
configuration file syntax can't express it. One reason for that is that
ServerTransportListenAddr is keyed by transport name--it assumes you
will never run more than one instance with the same transport name.

[David Fifield]

On the topic of making PTs usable by PT-naive programs, Hamy has a blog
post on doing it in a lightweight manner with shell scripts:
https://hamy.io/post/000d/how-to-hide-obfuscate-any-traffic-using-obfs4/
With examples of applying it to OpenSSH and OpenVPN:
https://hamy.io/post/000e/how-to-hide-obfuscate-ssh-traffic-using-obfs4/
https://github.com/HRomie/obfs4proxy-openvpn

And of course shapeshifter-dispatcher caters to a related use case.
While ptadapter and Hamy's scripts work with transports that use the IPC
interface, shapeshifter-dispatcher works with transports that are
implemented in terms of the PT API.
https://github.com/OperatorFoundation/shapeshifter-dispatcher
https://www.pluggabletransports.info/spec/pt2draft3#41-api-to-ipc-adapter

[vm...@riseup.net]

Thanks twisteroid ambassador and David for the info.

While we are at the subject:

I was wondering if there is any implementation of IPC Protocol
specified in PT Spec 2.0 in c/c++ (IPC 2.0). I guess the C++ Obfs4
implement the IPC used by Tor. But is there a implementation which works
with shapeshifter-dispatcher?

I would like to add it to Stegotorus.

Thanks a lot,
vmon

[David Fifield]

On Wed, Jan 16, 2019 at 12:22:38PM -0500, vm...@riseup.net wrote:
> Thanks twisteroid ambassador and David for the info.
>
> While we are at the subject:
>
> I was wondering if there is any implementation of IPC Protocol
> specified in PT Spec 2.0 in c/c++ (IPC 2.0). I guess the C++ Obfs4
> implement the IPC used by Tor. But is there a implementation which works
> with shapeshifter-dispatcher?

In order to work with shapeshifter-dispatcher, Stegotorus would have to
be written in Go and use the PT API, because shapeshifter-dispatcher
links its transports as executable code and doesn't execute them as
separate processes. (Correct me if I'm wrong.) I don't think that's what
you want.

obfsclient uses a separate library, https://github.com/Yawning/liballium,
for PT support. You would have to adapt it, because it's for PT 1.0.

It sounds like what you are looking for is the opposite of ptadapter:
something to wrap an obfuscation program that doesn't use the PT IPC
interface in one that does. I don't know of an existing program that
does that. A while back we had a prototype of such a wrapper, but it (by
design) only worked with stdin–stdout filters:
https://bugs.torproject.org/7822. I suppose that wouldn't work for
Stegotorus because Stegotorus wants to make its own network connections.

[vm...@riseup.net]

Thanks David for clarifying.

> In order to work with shapeshifter-dispatcher, Stegotorus would have to
> be written in Go and use the PT API, because shapeshifter-dispatcher
> links its transports as executable code and doesn't execute them as
> separate processes. (Correct me if I'm wrong.) I don't think that's what
> you want.

You are right. I guess I was confused with multiple use of word
"dispatcher" in this space (They use "dispatcher" as an application
which uses PT (like Tor) on the pt website :-/ )

So for now, I think I'll use

https://github.com/OperatorFoundation/shapeshifter-ipc

, write a go wrapper for Stegotorus, and run Stegotorus in the
background and communictate with it from the go code. Maybe later I'll
translate shapeshifter-ipc to c++.

Thanks a lot,
vmon

David Fifield <da...@bamsoftware.com> writes: