Pluggable Transports 2.0 Specification, Draft 2

[Brandon Wiley]

Attached is the second draft of the Pluggable Transport 2.0 Specification. If you have feedback on this draft, please send me your comments by July 20.

[Brandon Wiley]

Changelog from Draft 1

● Renamed version flag to ptversion to avoid naming conflict with goptlib
● Modified Go examples to use correct Go syntax
● Renamed pt module in Go examples to base to avoid naming conflict with goptlib
● Reworded introduction
● Clarified Go examples with more details on how to implement a transport in Go
● Removed unused Javascript and Python APIs
● Removed SSH transport example
● Standardized use of Transports API and Dispatcher IPC language throughout
● Added length to per-connection parameter encoding

[Sergey Frolov]

Many thanks! I am very excited to see that effort going!

I have a few questions about Golang interface, though, as I am not sure I understand `TransportConn`. It has a single field and a single method, which returns aforementioned field:

type TransportConn interface {
    net.Conn
    NetworkConn() net.Conn
}

So, we can call functions directly on struct, that implements TransportConn, or call NetworkConn() first, e.g.:
MyTransportConn.Write([]byte{"hello, world!"})

 or
MyTransportConn.NetworkConn().Write([]byte{"hello, world!"})
I am not sure I see the benefit and why not to use net.Conn interface directly without any additional interfaces?
(It seems that the same question applies to some extent to `TransportListener`)

Another note: `type Transport interface` implements Client and Server factories, which is great and elegant, if both sides are implemented in Golang and in the same package, but, unfortunately, that might not always be the case. Might be a good idea not to rely on such interface to be able to implement only the Client/Server part in Golang.

[David Fifield]

On Tue, Jun 20, 2017 at 01:31:56PM -0700, Sergey Frolov wrote:
> Another note: `type Transport interface` implements Client and Server
> factories, which is great and elegant, if both sides are implemented in Golang
> and in the same package, but, unfortunately, that might not always be the case.
> Might be a good idea not to rely on such interface to be able to implement only
> the Client/Server part in Golang.

That's a good point. obfs4proxy (which inspired the PT 2.0 Go API)
itself has only a client implementation of ScrambleSuit, no server
implementation. They work around it by having ServerFactory be a dummy
function that just returns an error.

https://gitweb.torproject.org/pluggable-transports/obfs4.git/tree/transports/scramblesuit/base.go?id=obfs4proxy-0.0.7#n60
// ServerFactory will one day return a new ssServerFactory instance.
func (t *Transport) ServerFactory(stateDir string, args *pt.Args) (base.ServerFactory, error) {
// TODO: Fill this in eventually, though obfs4 is better.
return nil, fmt.Errorf("server not supported")
}

[Brandon Wiley]

On Tue, Jun 20, 2017 at 4:31 PM, Sergey Frolov <sefr...@colorado.edu> wrote:
>
> Many thanks! I am very excited to see that effort going!
>
> I have a few questions about Golang interface, though, as I am not sure I understand `TransportConn`. It has a single field and a single method, which returns aforementioned field:
>
> type TransportConn interface {
>     net.Conn
>     NetworkConn() net.Conn
> }

The version of the interface in the specification has explanatory comments:

type TransportConn interface {
     // The TransportConn interface extends net.Conn, the standard Go          
     // interface for sockets.
     // This line means that a TransportConn has all of the same methods as a
     // normal Go socket.

     // The transport-specific logic for obfuscating network traffic is
     // implemented inside the methods inherited from net.Conn.
    net.Conn

    // Conn for the underlying network connection

    NetworkConn() net.Conn
}

> So, we can call functions directly on struct, that implements TransportConn, or call NetworkConn() first, e.g.:
> MyTransportConn.Write([]byte{"hello, world!"})
>  or
> MyTransportConn.NetworkConn().Write([]byte{"hello, world!"})
> I am not sure I see the benefit and why not to use net.Conn interface directly without any additional interfaces?
> (It seems that the same question applies to some extent to `TransportListener`)

As explained in the comments, you use the net.Conn interface directly. So in your example above, you would use MyTransportConn.Write([]byte{"hello, world!"}) to write to the transport.

The NetworkConn() method returns the underlying network connection, which in most cases will be the actual TCP socket used to communicate to the server.  As stated in the specification, "The networkConn() function returns this underlying network connection so that the application can manipulate the network connection, for instance by setting TCP socket options. ... For instance, the application may need to set the TCP_NODELAY socket option on the network connection in order to disable the Nagle algorithm on the underlying TCP socket."

Exposing the TCP socket used by the transport was a requested feature from application developers. The NetworkConn() function returns a net.Conn rather than a more specific type such as a net.TCPConn because some application developers stack multiple transports together. Since the transport implements net.Conn, you can wrap multiple transports around each other. By repeated calls to NetworkConn(), you can unwrap these stack transports and eventually access the TCP socket.

While many application developers will not need to call NetworkConn(), it is a required feature for some application developers. 

 
> Another note: `type Transport interface` implements Client and Server factories, which is great and elegant, if both sides are

> implemented in Golang and in the same package, but, unfortunately, that might not always be the case. Might be a good idea 

> not to rely on such interface to be able to implement only the Client/Server part in Golang.

The majority of transports in Go provide both a client and server implementations. In the case where there is no server implementation, the Listen() method can return nil.

Some application developers have requested splitting the client and server implementations so that they can reduce the size of their applications by importing only the client implementation. Therefore, this may be something that we revisit in the future.

[Ox Cart]

Let me start by saying thanks for putting this together! It's really exciting to see where this is headed.

Regarding the Go interface, I have a few suggestions:

1. I think the NetworkDialer() method should be removed from the Transport interface. If the intent is to customize the dialer used by the concrete Transport implementation, the net.Dialer should just be passed in when constructing it. Modifying the existing dialer is not a good idea because I don't think it's thread-safe to do so.

2. I think having the NetworkConn() method on the TransportConn interface makes sense, and not just for the purpose of modifying TCP parameters. By the time that you have the TransportConn, the TCP connection will already have been established and it will be too late to set certain parameters. If the Transport wants to allow callers to control the underlying TCP connection, it should allow them to pass in a Dial function when constructing the Transport, and that Dial function can do whatever it needs.

If I implemented a transport (say lampshade), I might provide something like this

The Dial() function would do whatever customized dialing/TCP setup/etc. that's needed. If the caller doesn't want to do anything special, they can either just pass net.Dial as the Dial function, or they can create a net.Dialer and pass its Dial function.

Cheers,

Ox

PGP Key - https://pgp.mit.edu/pks/lookup?op=get&search=o...@getlantern.org

-----------------------------------------------------------------------------------------

"I love people who harness themselves, an ox to a heavy cart,

who pull like water buffalo, with massive patience,

who strain in the mud and the muck to move things forward,

who do what has to be done, again and again."

- Marge Piercy

--
You received this message because you are subscribed to the Google Groups "Network Traffic Obfuscation" group.
To unsubscribe from this group and stop receiving emails from it, send an email to traffic-obf+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brandon Wiley]

Thanks Ox, this is great feedback and I think all very sensible.

On Tue, Jun 20, 2017 at 8:11 PM, Ox Cart <o...@getlantern.org> wrote:

Let me start by saying thanks for putting this together! It's really exciting to see where this is headed.

Regarding the Go interface, I have a few suggestions:

1. I think the NetworkDialer() method should be removed from the Transport interface. If the intent is to customize the dialer used by the concrete Transport implementation, the net.Dialer should just be passed in when constructing it. Modifying the existing dialer is not a good idea because I don't think it's thread-safe to do so.

2. I think having the NetworkConn() method on the TransportConn interface makes sense, and not just for the purpose of modifying TCP parameters. By the time that you have the TransportConn, the TCP connection will already have been established and it will be too late to set certain parameters. If the Transport wants to allow callers to control the underlying TCP connection, it should allow them to pass in a Dial function when constructing the Transport, and that Dial function can do whatever it needs.

If I implemented a transport (say lampshade), I might provide something like this

The Dial() function would do whatever customized dialing/TCP setup/etc. that's needed. If the caller doesn't want to do anything special, they can either just pass net.Dial as the Dial function, or they can create a net.Dialer and pass its Dial function.

Cheers,

Ox

PGP Key - https://pgp.mit.edu/pks/lookup?op=get&search=ox@getlantern.org

- Marge Piercy

[Sergey Frolov]

Brandon,
Thanks for your explanation: it makes sense and I see that having such a function is a good idea.

That was my first glance at PT, so I got very confused about which layer does "Conn for the underlying network connection" mean, but, before asking silly questions, I certainly should have further carefully read the spec to see that TCP is used as an example for that function.

Since the transport implements net.Conn, you can wrap multiple transports around each other.By repeated calls to NetworkConn(), you can unwrap these stack transports and eventually access the TCP socket

That sounds nifty: I was thinking to use similar interface to shape the traffic independently from the protocol stuff.

To unsubscribe from this group and stop receiving emails from it, send an email to traffic-obf...@googlegroups.com.

[David Fifield]

On Tue, Jun 20, 2017 at 02:08:45PM -0400, Brandon Wiley wrote:
> Attached is the second draft of the Pluggable Transport 2.0 Specification. If
> you have feedback on this draft, please send me your comments by July 20.

Thanks, Brandon. Here are my comments from a detailed readthrough.

A summary of main important points:
* command-line flags with a single dash are likely to cause problems;
apps need a way to separate their own flags from PT flags.
* error messages going to stderr instead of stdout seems to create
difficulties for no reason.
* we can remove TOR_PT_SERVER_TRANSPORTS.
* there is a need for a generic error-reporting mechanism in the IPC
Interface.


== Section 1

"Most PTs implement both a PT Client and a PT Server, but in
some cases only one of the two is unnecessary."

How can it be the case that either the client or the server is
unnecessary? They might reside in separate codebases, sure--is that what
it means? Possibly you can strike this sentence.


== Section 2

Figure 2 shows a dispatcher implemented atop a client library, and the
server implemented atop a server library, à la Section 4.1 "API to IPC
Adapter." But that's not necessary, correct? The client or server may be
implemented using code that doesn't conform to the interfaces of Section
3.2.

If you implement the API Interface, you get the IPC Interface for free
(using the dispatcher), but if you implement the IPC Interface, it
doesn't necessarily mean there's an API Interface underlying it.


== Section 3.2.1

"Transports require an explicit destination address to be
specified."

I think we don't need this point. A transport could conceivably have a
single address hardcoded or "curried" in, so to speak. Or a PT could
just be an interface to a radio broadcast, or something like that.

"* Transports may or may not generate, send, receive, store,
and/or update persistent or ephemeral state.
* Transports that do not need persistence or negotiation can
interact with the application through the simplest possible
interface
* Transports that do need persistence or negotiation can rely
on the application to provide it through the specified
interface, so the transport does not need to implement
persistence or negotiation internally."

I was confused by this. Is it about pt_state or something else? Or is it
about session persistence and reliability, TCP-in-TLS and things like
that? What's the "simplest possible interface"?

"Applications should be able to use a PT Client implementation
to establish several independent transport connections with
different parameters, with a minimum of complexity and latency."

There's no corresponding goal for the server; i.e., to be able to open
multiple listeners with the same or different transports and different
parameters. The PT 1.0 spec made that impossible and PT 2.0 seems to be
the same. The main reason is because of the keying by transport name in
TOR_PT_SERVER_BINDADDR and TOR_PT_SERVER_TRANSPORT_OPTIONS. If I want run
one "trebuchet" transport with password="foo" and another one with
password="bar", the closest I can come is this:
TOR_PT_SERVER_BINDADDR=trebuchet-0.0.0.0:1234,trebuchet-0.0.0.0:5678
TOR_PT_SERVER_TRANSPORT_OPTIONS=trebuchet:password=foo;trebuchet:password=bar
and then it's ambiguous which option corresponds to which bindaddr. (I
know the spec prohibits duplicate transport names in
TOR_PT_SERVER_BINDADDR, but there's nothing that prevents it from
working.) Here's an example of where I have encountered this limitation
before: https://bugs.torproject.org/11211#comment:22

In a glorious future, it would be good to remove some of these
ambiguities and limitations and just specify transports completely, not
having to bind multiple structured data fields together:
{"transport": "trebuchet", "addr": "0.0.0.0:1234", "options": {"password": "foo"}}
{"transport": "trebuchet", "addr": "0.0.0.0:5678", "options": {"password": "bar"}}


== Section 3.2.2.1

"Client Factory takes the address of a PT server..."

So the Client Factory gets its configuration settings from the
Transport. Does this contradict the design goal "Applications should be
able to use a PT Client implementation to establish several independent
transport connections with different parameters..."? It also seems to
contradict the paragraph following: "a read-only opaque argument (the
connection settings)".


== Section 3.2.2.2

"The connection object is extended to provide access to the
underlying actual network socket..."

The phrasing "actual network socket" doesn't work for me, because there
may be zero, one, or many actual sockets corresponding to a Connection.
For example, section 3.3.5.3 describes timeouts of TCP connections
underlying UDP associations--the TCP sockets will close and change over
time. Or imagine a transport that sends an email for every
packet--there's no persistent actual network socket there. I see how
NetworkConn really helps in some important special cases. What should
applications that don't fit into one of those models return, though?


== Section 3.2.4.1

The section is called "Modules" but there's only one module and
subsection. I would collapse the hierarchy up a level here.


== Section 3.3.1.1


"TOR_PT_EXIT_ON_STDIN_CLOSE or -exit-on-stdin-close"

If I were to redo PT 1.0, I would make TOR_PT_EXIT_ON_STDIN_CLOSE=1 be
implicit behavior. It is kind of a hack, but it is better than the
situation before, when on Windows a PT couldn't clean up its own
subprocesses, because Tor would TerminateProcess the PT without giving
it a chance to react; and Tor could die without its child processes
noticing and then fail to start back up because the ports were already
in use.

These two tickets explain the rational behind the existence of
TOR_PT_EXIT_ON_STDIN_CLOSE.
https://bugs.torproject.org/15434
https://bugs.torproject.org/15435

If you think you can get away with it, you could just specify that PTs
must always take their stdin closing as a signal to exit. You could also
require applications to always set TOR_PT_EXIT_ON_STDIN_CLOSE=1, for
compatibility with PT 1.0 transports.


== Section 3.3.1.2

"...the client-specific configuration parameters specified in section
3.3.1.2 MUST also be set..." contradicts "The 'TOR_PT_PROXY' environment
variable is OPTIONAL..."


== Section 3.3.1.3

I wonder if we can completely get rid of TOR_PT_SERVER_TRANSPORTS? As it
is, it is redundant with TOR_PT_SERVER_BINDADDR because the latter has
all the transport names anyway. The spec doesn't say what to do if a
transport name is present in TOR_PT_SERVER_TRANSPORTS but missing in
TOR_PT_SERVER_BINDADDR, or vice versa. What goptlib does, is it takes
TOR_PT_SERVER_BINDADDR as its primary source of information, and at the
end filters to remove transports that don't appear to
TOR_PT_SERVER_TRANSPORTS (which never happens in practice).
https://gitweb.torproject.org/pluggable-transports/goptlib.git/tree/pt.go?id=0.7#n541
I think that we can remove the requirement for PT servers to pay
attention to TOR_PT_SERVER_TRANSPORTS, and suggest that applications set
TOR_PT_SERVER_TRANSPORTS for compatibility.

"Specifies per-PT protocol configuration directives, as a
semicolon-separated list of <key>:<value> pairs, where <key> is
a PT name and <value> is a k=v string value with options that
are to be passed to the transport. Colons, semicolons, equal
signs and backslashes MUST be escaped with a backslash."

It would be really nice to have a grammar here. It's underspecified in
PT 1.0 as well. For example, is backslash escaping general (is it
permitted to escape other characters, or only the ones mentioned)?

pt-spec.txt recently had equals signs removed from the escaping list,
because apparently Tor has never escaped them. This unfortunately
creates an ambiguity in strings like "a=b=c": is it key "a" with value
"b=c", or key "a=b" with value "c"? https://bugs.torproject.org/12931

"Applications MUST NOT set more than one <address>:<port> pair
per PT name."

It would be nice if this requirement could be removed, but per my
comments on Section 3.2.1, I don't think it can be :(


== Section 3.3.1.4

It seems like this section, which states that configuration parameters
can be set by environment variables or by command-line flags, should
appear at the beginning of Section 3.3.1, not the end.

I think that starting long command-line flags with a single dash is
going to be a problem. The Go flag package works that way but I suspect
most other getopt packages do not. For example, -orport is
conventionally interpreted as -o -r -p -o -r -t. The Go flag package
also allows double dashes, so I would specify the flags that way:
--orport.

How are programs supposed to separate their own options, like --help and
--log, from those of the IPC Interface, like -ptversion and -orport? I'm
thinking of how one would implement a function like pt.ClientSetup,
which will need access to both the environment and to argv. It cannot
parse argv independently, because it doesn't know the semantics of the
other flags the program might have defined (and whether they take
arguments, for example). We could ask programmers to parse all the IPC
Interface–related flags manually, and pass their values in somehow, but
that's not very nice. In Go, you could do a two-step initialization,
where one function injects some arguments into the global
flag.CommandLine FlagSet, and then allows you to add your own flags
before calling flag.Parse:
pt.AddFlags()
log := flag.String("log", "", "log filename")
flag.Parse()
where pt.AddFlags() would do like:
flag.StringVar(&global.ptVersion, "ptversion", "", "")
flag.StringVar(&global.ptStateDir, "state", "", "")
flag.StringVar(&global.ptTransports, "transports", "", "")


== Section 3.3.2.1

I don't think I like the fact that error messages like VERSION-ERROR and
ENV-ERROR now go to stderr instead of stdout. It seems like a gratuitous
incompatibility with PT 1.0 and it's going to interact badly with, for
example, the Go log package by default logging to stderr.


== Section 3.3.2.1

"At any point, if there is an error encountered related to
configuration supplied via the environment variables, it MAY
respond with an error message and terminate."

How should it express this error message? As a VERSION-ERROR? As an
ENV-ERROR? The spec is missing a general-purpose error reporting
mechanism:
https://groups.google.com/d/msg/traffic-obf/LWT_3sOrBJk/zQFvkzDyAQAJ

Does "at any point" include after CMETHODS DONE/SMETHODS DONE? That
obliges the parent process to keep paying attention to the child's
output stream forever (might be a good idea, actually). Currently I
think Tor just consumes and throws away all output after the PT
negotiation.


== Section 3.3.2.2

"PROXY DONE"

I would add a MUST here, that the parent MUST terminate the PT if the
parent set TOR_PT_PROXY and the PT didn't output a PROXY DONE before
CMETHODS DONE. Because in that case, the PT ignore the parent's request
to use a proxy. Is this what is meant by "TOR_PT_PROXY's 'fail closed'
behavior" in Section 5? I didn't find what that "fail closed" is
referring to elsewhere in the spec.

"Upon sending the 'CMETHODS DONE' message, the PT proxy
initialization is complete."

What does the parent do if it receives later lines on stdout/stderr? Do
they get ignored or interpreted?


== Section 3.3.3

"PT proxies exiting after a graceful shutdown should use exit
code EX_OK (0)."

Here there is EX_OK and elsewhere there is EX_CONFIG, EX_USAGE, and
EX_UNAVAILABLE. Are these supposed to be well-known constants or are
they invented for the spec? At first I thought it meant EXIT_SUCCESS and
EXIT_FAILURE (from stdlib.h), but that's not it.


== Section 3.3.4

What's allowed for values? Strings only, or can it be any kind of JSON
object?

The text says big-endian but \x39\x00\x00\x00 in the example is
little-endian.

What is supposed to happen in case of a parse error in the
per-connection arguments?


== Section 3.3.5

This ways that PT proxies can operate in UDP mode by opening a UDP
listener rather than a TCP listener. But there seems to be no in-spec
way to negotiate that--it has to be set up out of band.
shapeshifter-dispatcher has a --udp option that does it, but that's
external to the spec. Suppose I have an API Interface implementation;
how do I instruct it to open a UDP listener rather than a TCP listener?
Is UDP-ness a quality of the transport itself, and the caller simply has
to know whether it will be TCP or UDP when it is instantiated?

"All transports that are currently implemented use TCP" is ambiguous.
It's not referring to the actual obfuscated transport channel, but to
the Client App and Server App's interface to it.


== Section 4.1

I like this section a lot. It shows that the IPC Interface can be
implemented in terms of the API Interface, which I think is a powerful
idea.


== Section 4.2

I would like it if this section had a comprehensive list of
incompatibilities with PT 1.0. The ones I can think of:
* errors go to stderr, not stdout
* JSON SOCKS args
* apps need to consider argv as well and environment
* what else?

"...a list of supported versions, for instance versions, for
instance “1.0,2.0”."

This is a bad example, because the known versions (as understood by
TOR_PT_MANAGED_TRANSPORT_VER) will be exactly "1" and "2", not "1.0" and
"2.0".


== Appendix C

"Renamed version flag to ptversion to avoid naming conflict with
goptlib"

I don't know what you're referring to in goptlib. Do you mean the
-version flag in obfs4proxy and meek-client?