NAT-Traversal

[Tanguille Grootaert]

Hey all, 

I wanna build a p2p network in Rust. After some research I discovered I wanna work with libp2p, Cap'n Proto, ... In the docs of libp2p I read that WebRTC has NAT-traversal built in which I need. Since I want to use Cap'n Proto I don't wanna use this but use the RPC-protocol of Cap'n Proto. Does that have support for NAT-traversal? I can't find anything about it. 

If you have a better idea on how to approach this, that's ofcourse always welcome. Thanks!

[Tanguille Grootaert]

I discovered that WebRTC is actually some kind of stream multiplexer. If I search for that in the context of Cap'n proto the only reference I can find is in the release notes of the 0.8 release (https://capnproto.org/news/). There it is stated that it can't be considered stable yet, this is now 3 years ago but I can't find more information about this topic. Can anyone point me to a good source or an example implementation?

Op woensdag 30 augustus 2023 om 12:27:54 UTC+2 schreef Tanguille Grootaert:

[Kenton Varda]

Hi Tanguille,

Cap'n Proto is agnostic to the underlying byte transport. It can layer on top of any byte stream or datagram stream. In terms of the C++ implementation, you can write a custom implementation of the capnp::MessageStream interface in terms of any transport you'd like. For example, you could use Cap'n Proto over WebRTC. It's up to you to establish the byte-oriented or datagram-oriented connection first, then Cap'n Proto helps you interact with the peer across that connection.

NAT traversal is the responsibility of the underlying transport, so is outside the scope of Cap'n Proto itself.

-Kenton

--
You received this message because you are subscribed to the Google Groups "Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/88ce497f-d49a-4b97-a130-b787e6d9d0cen%40googlegroups.com.

[Tanguille Grootaert]

Thanks for the explanation Kenton. Cool to see you're still so involved, it's an honor to get an answer from you.

I am now trying to implement it in a little testproject with rust. However I can't seem to figure out how to install `capnpc-rust`. I installed both `capnproto` and `libcapnp-dev` via apt. 

If I do the command `capnp compile -orust src/schema/user.capnp` it outputs 

`rust: no such plugin (executable should be 'capnpc-rust')

rust: plugin failed: exit code 1`

This confuses me because other then a repo that has now been integrated in the main rust repo I can't find anything about this. What am I missing?

Op woensdag 30 augustus 2023 om 16:43:09 UTC+2 schreef ken...@cloudflare.com:

Hi Tanguille,

Cap'n Proto is agnostic to the underlying byte transport. It can layer on top of any byte stream or datagram stream. In terms of the C++ implementation, you can write a custom implementation of the capnp::MessageStream interface in terms of any transport you'd like. For example, you could use Cap'n Proto over WebRTC. It's up to you to establish the byte-oriented or datagram-oriented connection first, then Cap'n Proto helps you interact with the peer across that connection.

NAT traversal is the responsibility of the underlying transport, so is outside the scope of Cap'n Proto itself.

-Kenton

On Wed, Aug 30, 2023 at 8:36 AM Tanguille Grootaert <tanguille...@gmail.com> wrote:

I discovered that WebRTC is actually some kind of stream multiplexer. If I search for that in the context of Cap'n proto the only reference I can find is in the release notes of the 0.8 release (https://capnproto.org/news/). There it is stated that it can't be considered stable yet, this is now 3 years ago but I can't find more information about this topic. Can anyone point me to a good source or an example implementation?

Op woensdag 30 augustus 2023 om 12:27:54 UTC+2 schreef Tanguille Grootaert:

Hey all,ﾠ

I wanna build a p2p network in Rust. After some research I discovered I wanna work with libp2p, Cap'n Proto, ... In the docs of libp2p I read that WebRTC has NAT-traversal built in which I need. Since I want to use Cap'n Proto I don't wanna use this but use the RPC-protocol of Cap'n Proto. Does that have support for NAT-traversal? I can't find anything about it.ﾠ

If you have a better idea on how to approach this, that's ofcourse always welcome. Thanks!

[Kenton Varda]

You need to install the Rust capnp plugin. I'm not very familiar with Rust but it looks like this is provided by the crate called `capnpc`.

https://github.com/capnproto/capnproto-rust

https://docs.rs/capnpc/latest/capnpc/

-Kenton

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/cd024e30-eb7d-4ca6-9c4d-9f7bf5a01da0n%40googlegroups.com.

[Tanguille Grootaert]

That can't be the issue since my depedencies are containing the same ones as the examples in that repo, where `capnp` and `capnpc` are a part of. The code compiles so the dependencies are valid. Only thing I can think about is that the apt repo has outdated binaries for the Cap'n Proto compiler causing it to use the old naming. 

Is there a another place I could pose this question, maybe rust specific by any chance?

Op donderdag 31 augustus 2023 om 14:30:42 UTC+2 schreef ken...@cloudflare.com:

[Kenton Varda]

When you run `capnp compile -orust`, the `capnp` tool will attempt to execute a binary `capnpc-rust` from your $PATH. This binary seems to be provided by the capnpc crate, but declaring a dependency on the crate probably does not automatically install it into your $PATH. You can either install the binary into $PATH, or you can provide the binary location explicitly like:

capnp compile -o/path/to/capnpc-rust src/schema/user.capnp

I don't think there's a mailing list specifically for the Rust implementation but you could try filing issues on the github repo.

-Kenton

To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/9ebe0b3f-acec-4765-8cbd-b4cbb57a3a25n%40googlegroups.com.

[Troy Farrell]

It may help to read some of the examples for capnproto-rust:

https://github.com/capnproto/capnproto-rust/tree/master/example

Using "cargo build" with a build.rs file is the normal way to compile your .capnp files for Rust.

[Tanguille Grootaert]

I finally got it working. The main problem was that I didn't understand that the build file automated the capnpc process and that I needed to understand that capnp disables the use of the std lib. There are still some problems. 

The build seems to have worked when I use the absolute path, while the examples can use just the filename or schema/filename. For me it searches in the root dir when doing that, any idea why that could be? 

This is my build.rs: 

fn main() {

    capnpc::CompilerCommand::new()

        .src_prefix("src/schema")

        .file("src/schema/user.capnp")

        .run()

        .expect("schema compiler command");

}

The other problem I'm facing is that I can't see the actual files the compiler created. Where should I expect those files? This is my lib.rs:

mod user_schema {

    include!(concat!(env!("OUT_DIR"), "/user_capnp.rs"));

}

Where can I specify the value of OUT_DIR and what would you recommend as a location for this?

Op vrijdag 1 september 2023 om 07:51:01 UTC+2 schreef Troy Farrell:

[Jason Heeris]

I (a random Rust/C Cap'n Proto user) started helping Tanguille on the
Discord and then saw their emails here, so I'll reply here in case
others find it in a search.

> I needed to understand that capnp disables the use of the std lib

Even though we accounted for what was happening here, just for future
readers — it's more than that. There's some weird interaction with
transitive dependencies and features (ie. conditional compilation
flags) in build scripts that I don't understand. I don't think it's
capnp or capnpc specific. I am going to see if I can repro it and post
an issue to the appropriate project.

> The build seems to have worked when I use the absolute path, while the examples can use just the filename or schema/filename. For me it searches in the root dir when doing that, any idea why that could be?

It's hard to tell without more information, so you should provide some
example compiler output (with any obvious noise removed). Strip out
anything not Cap'n Proto related from a project and try with that. All
the paths in the builder are relative to your project directory ie.
the one containing Cargo.toml.

> The other problem I'm facing is that I can't see the actual files the compiler created. Where should I expect those files?

These will appear in Cargo's target directory, which may be different
for different runs. The full path includes a hash that's designed to
allow certain kinds of caching but without causing invalid stale
builds to ruin your process. You almost certainly shouldn't specify
your OUT_DIR yourself. It's documented here:

https://doc.rust-lang.org/cargo/reference/environment-variables.html#environment-variables-cargo-sets-for-build-scripts

For example, my generated files (for your sandbox project) appear at
this path relative to the project root:

target/debug/build/sandbox-81257fcdc875cd8e/out/user_capnp.rs

It works fine for me if I have:

capnpc::CompilerCommand::new()
.src_prefix("src/schema")
.file("src/schema/user.capnp")

...in build.rs and:

pub mod user_schema {

include!(concat!(env!("OUT_DIR"), "/user_capnp.rs"));
}

...in lib.rs. If yours doesn't, it could be environment-related.

Cheers,
Jason