Usenet peering over Tor

[Jason Evans]

Hi all,

First of all, I know that there have been some usenet servers that offer read-
only access to groups via a Tor onion service and that's not what I'm writing
about today.

I would like to know if anyone here has been experimenting with server to
server peering over Tor.

From my experience with running Tor onion services, it's quite easy to set up a
server that can receive traffic over Tor. However sending traffic over Tor is
another issue. It requires either setting up system proxies for outgoing
traffic or wrapping the INN binaries in the torsocks application.

If you've been working on this or are interested in working on this, please
reply or shoot me an email.

Jason

[Miner]

Jason Evans wrote:

> I would like to know if anyone here has been experimenting with
> server to server peering over Tor.
>

> If you've been working on this or are interested in working on
> this, please reply or shoot me an email.

Peering over Tor require additional tricks. Peering over I2P
works great (without tricks).

--
Miner

[Retro Guy]

Jason Evans wrote:

> I would like to know if anyone here has been experimenting with server to
> server peering over Tor.

> From my experience with running Tor onion services, it's quite easy to set up a
> server that can receive traffic over Tor. However sending traffic over Tor is
> another issue. It requires either setting up system proxies for outgoing
> traffic or wrapping the INN binaries in the torsocks application.

> If you've been working on this or are interested in working on this, please
> reply or shoot me an email.

I have done this and it works fine. My previous partner in this project (rocksolid), trw, wrote a script to handle peering over tor and it works well. I'll post the script below. If you want to test, I'll be happy to test with you. Again, credit to trw for the script.

#!/bin/bash

# wrapper script for socat
# if socat crashes, which it likes to do, we wait for 120 s, and then restart the command
# waiting is neccessary because the port used by socat is not immediatly freed after the crash
# edit the variables below to control the behaviour of the socat command

localport="50119"
# the local port you want socat to listen on
proxyport="9050"
# the port of the proxy used (tor socks proxy)
proxyip="127.0.0.1"
# the ip on which the proxy is running
onion="localonionaddress.onion"
# the onion address of the hidden service you want to connect to
onionport="119"
# the port of the hidden service you want to connect to

while true
# enter eternal loop here
do
socat TCP4-LISTEN:"$localport",reuseaddr,fork SOCKS4A:"$proxyip":"$onion":"$onionport",socksport="$proxyport"
sleep 120
done
exit 0
# this line should never be reached, but just in case we close properly

--
Retro Guy

[Jason Evans]

On Wed, 18 May 2022 09:12:42 +0000, Retro Guy wrote:

> I have done this and it works fine. My previous partner in this project
> (rocksolid), trw, wrote a script to handle peering over tor and it works
> well. I'll post the script below. If you want to test, I'll be happy to
> test with you. Again, credit to trw for the script.

Hi Retro Guy,

Thanks for the script, I will look into this. I'll first try to establish
a connection between two VMs and then I may come back to you about peering
if everything works.

I wonder if creating a system service (e.g. systemd) would be more
beneficial than just having a bash script running in the background.

Jason

[Retro Guy]

Jason Evans wrote:

> On Wed, 18 May 2022 09:12:42 +0000, Retro Guy wrote:

>> I have done this and it works fine. My previous partner in this project
>> (rocksolid), trw, wrote a script to handle peering over tor and it works
>> well. I'll post the script below. If you want to test, I'll be happy to
>> test with you. Again, credit to trw for the script.

> Hi Retro Guy,

> Thanks for the script, I will look into this. I'll first try to establish
> a connection between two VMs and then I may come back to you about peering
> if everything works.

Anytime. While I do peer with several peers over i2p, I only peer between two
of my own servers with tor, but it works fine.

Feel free to just try running the script and pointing it to:
zkcvkb5xprurx5dvpanhyivneuzah6k6xayxgxd4h2ekklxgoi2x5aad.onion port 119

Then just telnet to the local port you set and you should hit one of my
inn2 servers (rocksolid2) via tor.

> I wonder if creating a system service (e.g. systemd) would be more
> beneficial than just having a bash script running in the background.

I don't see why that wouldn't work. You are just trying to proxy your connection
through a socks proxy (tor), so there are probably a few ways to do that.

trw, who wrote the script, is a bash master so that's how he chose to get the
job done. I've seen him slap out chans using just bash, so quite impressive to me.

--
Retro Guy

[Grant Taylor]

On 5/18/22 3:12 AM, Retro Guy wrote:
> I have done this and it works fine. My previous partner in this
> project (rocksolid), trw, wrote a script to handle peering over tor
> and it works well. I'll post the script below. If you want to test,
> I'll be happy to test with you. Again, credit to trw for the script.

My concern about peering over Tor is the absence of sending system
identification. It is typical for peering privileges to be associated
with the source IP. So, based on the very nature of Tor -- as I
understand it -- anyone that knows the Tor hidden service address will
be viewed as the same source and thus have the same privileges.

Does this mean that people will be using something else at the NNTP
level to manage peer identification and permissions therein?

Or is there some sort of Tor restriction on clients that can connect to
a Tor hidden service?

>                socat TCP4-LISTEN:"$localport",reuseaddr,fork
> SOCKS4A:"$proxyip":"$onion":"$onionport",socksport="$proxyport"

I don't see anything in the socat command to belay my concern.

It seems as if the command simply stands up a local listening socket
that passes through the Tor network to a news server that's behind a Tor
hidden service.

What am I missing?



--
Grant. . . .
unix || die

[Retro Guy]

Grant Taylor wrote:

> On 5/18/22 3:12 AM, Retro Guy wrote:
>> I have done this and it works fine. My previous partner in this
>> project (rocksolid), trw, wrote a script to handle peering over tor
>> and it works well. I'll post the script below. If you want to test,
>> I'll be happy to test with you. Again, credit to trw for the script.

> My concern about peering over Tor is the absence of sending system
> identification. It is typical for peering privileges to be associated
> with the source IP. So, based on the very nature of Tor -- as I
> understand it -- anyone that knows the Tor hidden service address will
> be viewed as the same source and thus have the same privileges.

Yes, that's true. I2P makes it much easier.

> Does this mean that people will be using something else at the NNTP
> level to manage peer identification and permissions therein?

While I'm not sure if using or understanding it correctly, I use a 'password'
in innfeed and incoming.conf. While it seems to work ok, I'm not sure why
you can set a username in innfeed, but not incoming.conf. It's been a while
since I've spent much time on this.

> Or is there some sort of Tor restriction on clients that can connect to
> a Tor hidden service?

The newer v3 tor addresses are a bit better, as they shouldn't be known to
anyone unless you provide them the address. I do use a different address
for each peer.

>>                socat TCP4-LISTEN:"$localport",reuseaddr,fork
>> SOCKS4A:"$proxyip":"$onion":"$onionport",socksport="$proxyport"

> I don't see anything in the socat command to belay my concern.

> It seems as if the command simply stands up a local listening socket
> that passes through the Tor network to a news server that's behind a Tor
> hidden service.

Yes, that's exactly what it's doing, nothing more.

> What am I missing?

All your concerns are valid, so I don't think you're missing anything.

--
Retro Guy

[Grant Taylor]

On 5/20/22 9:50 PM, Retro Guy wrote:
> Yes, that's true. I2P makes it much easier.

Would you please elaborate on what I2P does that's different?

I know /of/ I2P, but very little about it. Most of that is that the
client is written in Java, which is a bad thing IMHO.

> While I'm not sure if using or understanding it correctly, I use a
> 'password' in innfeed and incoming.conf. While it seems to work
> ok, I'm not sure why you can set a username in innfeed, but not
> incoming.conf. It's been a while since I've spent much time on this.

I don't know. That's probably a question for those that know INN better
than I do.

> The newer v3 tor addresses are a bit better, as they shouldn't be
> known to anyone unless you provide them the address. I do use a
> different address for each peer.

If I understand correctly, Tor v3 is effectively a larger address /
search space. Read: Bigger hey stack to hide in. ;-)

> Yes, that's exactly what it's doing, nothing more.

:-)

> All your concerns are valid, so I don't think you're missing anything.

As I see it, there should be something that provides client
authentication at the Tor layer or the INN / NNTP layer (preferably
something more than /just/ a password).

I suppose that you could add an additional VPN layer between Tor and
INN. Though now things are getting really complicated, and one needs to
ponder what the complication provides.

So I'll back up and ask what is the motivation for running / peering
with INN via a Tor hidden service?

[Retro Guy]

Grant Taylor wrote:

> On 5/20/22 9:50 PM, Retro Guy wrote:
>> Yes, that's true. I2P makes it much easier.

> Would you please elaborate on what I2P does that's different?

There are two features in I2P that help with this. One is that you
can tie a client key (generated by the client) to a server tunnel.
You can whitelist key(s) for this tunnel, and only allow specific
clients to connect.

Also, if you run I2P on the same server as inn2 (or whatever), I2P
can provide a specific ip address for each client, so you can allow
connections from one IP address, but not others.

None of this is available with Tor.

>> The newer v3 tor addresses are a bit better, as they shouldn't be
>> known to anyone unless you provide them the address. I do use a
>> different address for each peer.

> If I understand correctly, Tor v3 is effectively a larger address /
> search space. Read: Bigger hey stack to hide in. ;-)

I have read that it is also more difficult to just guess addresses, so
harder to just try to connect to servers. The chance that your new address
that you have not advertised is tried, is much lower.

>> All your concerns are valid, so I don't think you're missing anything.

> As I see it, there should be something that provides client
> authentication at the Tor layer or the INN / NNTP layer (preferably
> something more than /just/ a password).

> I suppose that you could add an additional VPN layer between Tor and
> INN. Though now things are getting really complicated, and one needs to
> ponder what the complication provides.

This is something I've considered. It shouldn't be too difficult, but I
haven't felt the need to put time into it. Maybe in the future.

> So I'll back up and ask what is the motivation for running / peering
> with INN via a Tor hidden service?

The motivation is the same as why I run my own news servers instead of use
someone else's. Why I run my own mail server. Why I develop a web interface
for usenet newsgroups, and my own php news server. Because it's fun, a
challenge, and keeps my brain busy.

None if this will change the world, it's just a hobby that I enjoy.

--
Retro Guy

[Matija Nalis]

On Fri, 20 May 2022 23:37:04 -0600, Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 5/20/22 9:50 PM, Retro Guy wrote:
>> While I'm not sure if using or understanding it correctly, I use a
>> 'password' in innfeed and incoming.conf. While it seems to work
>> ok, I'm not sure why you can set a username in innfeed, but not
>> incoming.conf. It's been a while since I've spent much time on this.
>
> I don't know. That's probably a question for those that know INN better
> than I do.

Hopefully someone who uses that should try changing password on only one side
and see if the authentication fails (or there might be a bug - e.g. option is
ignored and all access is always enabled)

>> The newer v3 tor addresses are a bit better, as they shouldn't be
>> known to anyone unless you provide them the address. I do use a
>> different address for each peer.
>
> If I understand correctly, Tor v3 is effectively a larger address /
> search space. Read: Bigger hey stack to hide in. ;-)

Sure - as are *all* username/password schemes (and even RSA keys, TLS etc)...

As long as attacker can try a "key", and see if worked, the system
is suspectable to brute force attacks. It is all about bigger haystack,
to make bruteforcing non-viable.

> As I see it, there should be something that provides client
> authentication at the Tor layer or the INN / NNTP layer (preferably
> something more than /just/ a password).

TLS itself (which hopefully one who cares about privacy uses on their NNTP servers!)
allows authentication via client-side certificates too.

I do not think inn implements that possibility directly (at least it
did not when I last looked at it in more detail - which was admittedly long ago).

Although if inn is in dedicated container (or other services do not mind),
one could probably choose to trust only their local CA (i.e. remove all
other CAs from /etc/ssl/certs), and sign only trusted peer certs with that
local CA.

Or one could use external TLS wrapper for inn (like stunnel) to accomplish that
client certificate verificaton (i.e. stunnel "verifyPeer=yes").

Although AUTHINFO password is probably just fine for most use cases (if it is
of reasonable complexity).
That is assuming that incoming.conf AUTHINFO feature actually works correctly :-)

> So I'll back up and ask what is the motivation for running / peering
> with INN via a Tor hidden service?

Few random ideas, I'm sure there are a lot more:

- you might want to protect the identity of peers posting information that high-level
adversaries do not like (e.g. China dissidents, wikileaks-like stuff etc).

- the more traffic initiates over your TOR node, the better you are protected against
side-channel attacks when you're being actively monitored, as more noise is introduced.
Usenet introduces lots of noise, so Usenet good :)

- non-public privacy oriented hierarchies

- sites whose operators prefer not divulging their IP address space / county of origin / company affiliations.

--
Opinions above are GNU-copylefted.
<

[Russ Allbery]

Matija Nalis <mnali...@voyager.hr> writes:

> TLS itself (which hopefully one who cares about privacy uses on their
> NNTP servers!) allows authentication via client-side certificates too.

> I do not think inn implements that possibility directly (at least it did
> not when I last looked at it in more detail - which was admittedly long
> ago).

INN does not. Client certificate authentication is in general kind of a
pain in the ass and is very rarely implemented by clients (and when it is,
the UI is normally horrible, such as in web browsers), so there was never
enough demand for this feature.

The code is probably straightforward enough (although I haven't looked at
the OpenSSL API in detail). The protocol would presumably just use SASL
EXTERNAL, which I think is also reasonably straightforward. The hard part
is the configuration and attempting to explain in even vaguely coherent
English how to set up the validation chain, CA, and so forth, without
generating a flurry of confused people and support questions.

I've done this multiple times for other services and, honestly, automating
the whole thing end-to-end was always easier than trying to explain to
someone else what to do. (But alas INN can't do that.)

> Although AUTHINFO password is probably just fine for most use cases (if
> it is of reasonable complexity).

Particularly if you're only doing it over TLS. Then you can just use a
long random password and while there are some remaining drawbacks compared
to mutual TLS, they're relatively unimportant most of the time.

--
Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

[Miner]

Grant Taylor wrote:

> I know /of/ I2P, but very little about it. Most of that is
> that the client is written in Java, which is a bad thing IMHO.

i2pd - a full-featured C++ implementation of the I2P router

--
Miner

[Julien ÉLIE]

Hi Matija,

>>> While I'm not sure if using or understanding it correctly, I use a
>>> 'password' in innfeed and incoming.conf. While it seems to work
>>> ok, I'm not sure why you can set a username in innfeed, but not
>>> incoming.conf. It's been a while since I've spent much time on this.

[...]

> Although AUTHINFO password is probably just fine for most use cases (if it is
> of reasonable complexity).
> That is assuming that incoming.conf AUTHINFO feature actually works correctly :-)

Yes, I confirm AUTHINFO works fine in both innfeed and innd.
I thoroughly tested it when implementing NNTP v2.

The reason behind that behaviour is historical. Prior to INN 2.6.0,
released in 2015, a remote peer could just send "AUTHINFO PASS" (without
a previous "AUTHINFO USER" command), which is contrary to RFC 4643, to
be authenticated.
innd does not historically take into account a username, but only a
password.

(Note that a mere "AUTHINFO USER" is allowed by RFC 4643, but not a mere
"AUTHINFO PASS".)


From ChangeLog in 2.6.0:

%%%
The NNTP protocol requires a username to be sent before a password when
authentication is used. innd was wrongly allowing only a password to be
sent by authenticated peers.

Owing to the implementation of RFC 4643 (AUTHINFO USER/PASS) in innd, if
remote peers have to authenticate in order to feed articles, they now
have to send a username (which was previously wrongly optional), before
sending their password. The mandatory username, though currently unused
by innd, can be whatever the remote peer wishes. In previous versions
of INN, inncheck was already complaining when passwd.nntp contained an
empty username associated with a password.

A manual review of authenticated feeds should then be done so as to
ensure that they are properly working.
%%%



When doing that change (committed in 2009), I normally made sure other
major open source NNTP servers were sending AUTHINFO USER and therefore
this would not break badly existing interoperability.
Seems like it did not (nobody using other NNTP servers I did not look at
ever complained) :-)

--
Julien ÉLIE

« Tout homme devrait un jour se marier ; après tout, le bonheur n'est
pas la seule chose dans la vie. »

[Retro Guy]

Julien_ÉLIE wrote:

> Hi Matija,

>>>> While I'm not sure if using or understanding it correctly, I use a
>>>> 'password' in innfeed and incoming.conf. While it seems to work
>>>> ok, I'm not sure why you can set a username in innfeed, but not
>>>> incoming.conf. It's been a while since I've spent much time on this.
> [...]
>> Although AUTHINFO password is probably just fine for most use cases (if it is
>> of reasonable complexity).
>> That is assuming that incoming.conf AUTHINFO feature actually works correctly :-)

> Yes, I confirm AUTHINFO works fine in both innfeed and innd.
> I thoroughly tested it when implementing NNTP v2.

> The reason behind that behaviour is historical. Prior to INN 2.6.0,
> released in 2015, a remote peer could just send "AUTHINFO PASS" (without
> a previous "AUTHINFO USER" command), which is contrary to RFC 4643, to
> be authenticated.
> innd does not historically take into account a username, but only a
> password.

Thank you for the info and confirmation. It works for me also, but I just never
understood if I was using it correctly due to USER only being available on one
end.

--
Retro Guy

[Julien ÉLIE]

Hi Retro Guy,

>> innd does not historically take into account a username, but only a
>> password.
>
> Thank you for the info and confirmation. It works for me also, but I
> just never understood if I was using it correctly due to USER only
> being available on one end.

This behaviour is worth documenting in incoming.conf; I'll add a
sentence for the "password" keyword saying that innd will accept any
username provided by the remote peer (mandatory in the authentication
protocol) as long as the password corresponds.

Thanks for recalling that point!

--
Julien ÉLIE

« – Tu parles ?
– Tu parles ! » (Astérix)

[Grant Taylor]

On 5/21/22 12:00 AM, Retro Guy wrote:
> There are two features in I2P that help with this.

Both of the listed options; client key, and client IP, sound like they
would help considerably to make connections over I2P behave more like
connections over native IP.

> This is something I've considered. It shouldn't be too difficult,
> but I haven't felt the need to put time into it. Maybe in the future.

My concern with VPN is the make it work with SOCKS(Tor) thing. There
are ways to SOCKSify things; LD_PRELOAD tricks, routing tricks, but they
are -- let's go with -- non-clean / non-obvious and feel like they might
be making things harder than strictly necessary.

> The motivation is the same as why I run my own news servers instead of
> use someone else's. Why I run my own mail server. Why I develop a web
> interface for usenet newsgroups, and my own php news server. Because
> it's fun, a challenge, and keeps my brain busy.

Okay. I can get behind that. But where does the academic exercise end
and practicality take over? E.g. is establishing peers via I2P and / or
Tor something that people are interested in? Or is this truly an
academic exercise.

Aside: I can get behind purely academic exercises. I just want to know
if that's all this will be or if there is more to this.

> None if this will change the world, it's just a hobby that I enjoy.

ACK

I can get behind and support that.

[Grant Taylor]

On 5/21/22 6:08 AM, Matija Nalis wrote:
> Sure - as are *all* username/password schemes (and even RSA keys,
> TLS etc)...

Eh ... now you're getting into why is fire hot territory.

> As long as attacker can try a "key", and see if worked, the system is
> suspectable to brute force attacks. It is all about bigger haystack,
> to make bruteforcing non-viable.

ACK

> TLS itself ... allows authentication via client-side certificates too.

True! I use this very thing between email servers.

> I do not think inn implements that possibility directly (at least it
> did not when I last looked at it in more detail - which was admittedly
> long ago).

I don't recall reading about support for it either. But my ignorance
doesn't preclude such support from existing.

> Although if inn is in dedicated container (or other services do
> not mind), one could probably choose to trust only their local CA
> (i.e. remove all other CAs from /etc/ssl/certs), and sign only trusted
> peer certs with that local CA.

I don't think you want to /only/ rely on if a client certificate
validates or not. I believe you want some sort of sub-selection of
which certificate is allowed, assuming that it does validate.

E.g. Sendmail will happily conditionally look for the CN of a client
certificate /if/ the signer is trusted. Meaning that I can specify my
server's CNs from their Let's Encrypt certificates and only my servers
are allowed to relay, no other servers using Let's Encrypt certificates.

> Or one could use external TLS wrapper for inn (like stunnel)
> to accomplish that client certificate verificaton (i.e. stunnel
> "verifyPeer=yes").

Ya.... But then you have the issue of getting client information from
the external sub-system into INN for it to make decisions.

[Grant Taylor]

On 5/21/22 12:45 PM, Miner wrote:
> i2pd - a full-featured C++ implementation of the I2P router

Thank you for that information Miner.

I'll keep that in mind for the next time the need for I2P comes up.

[Retro Guy]

Grant Taylor wrote:

> On 5/21/22 12:00 AM, Retro Guy wrote:

>> The motivation is the same as why I run my own news servers instead of
>> use someone else's. Why I run my own mail server. Why I develop a web
>> interface for usenet newsgroups, and my own php news server. Because
>> it's fun, a challenge, and keeps my brain busy.

> Okay. I can get behind that. But where does the academic exercise end
> and practicality take over? E.g. is establishing peers via I2P and / or
> Tor something that people are interested in? Or is this truly an
> academic exercise.

I would assume that my peers in I2P, I have three, probably do so for the same
reasons that I do, because we can. You will find, for example, a poster in
this group that runs their own server and peers through mine via I2P. You
can't trace them, and they are making valid and useful posts, so there are
people in these networks that are valuable to the community, and enjoy the
exercise of trying something different.

> Aside: I can get behind purely academic exercises. I just want to know
> if that's all this will be or if there is more to this.

One reason for me is to stay in touch with alternative networks. It keeps
me up to date on I2P and Tor. These networks may get a bad reputation from
some bad players, but they are functioning networks, and there are plenty
of people using them for very valid purposes.

I'm old enough to remember (and I would guess many of us in this group
are), when using a BBS or the internet made you suspect.

In the early 90s, my wife at the time found me talking (voice) to someone
online using powwow and accused me of all sorts of stuff. It was new to me,
voice over internet, and I just had to mess with it. I still have that same
interest and excitement doing things online.

--
Retro Guy

[Grant Taylor]

On 5/25/22 1:37 AM, Retro Guy wrote:
> I would assume that my peers in I2P, I have three, probably do so for
> the same reasons that I do, because we can. You will find, for example,
> a poster in this group that runs their own server and peers through
> mine via I2P.

That's interesting enough that I'll add the C++ implementation of i2pd
that Miner mentioned to my list of things to check out so that I could
consider being an additional I2P news peer.

> You can't trace them, and they are making valid and useful posts, so
> there are people in these networks that are valuable to the community,
> and enjoy the exercise of trying something different.

I learned a long time ago that the communications medium is not a
reliable indicator of the validity / usefulness of the communications.

> One reason for me is to stay in touch with alternative networks. It
> keeps me up to date on I2P and Tor.

I get that. That's one of the reasons that I got my amateur radio
license. Knowledge of how to use something and being able to do so
completely above board.

> These networks may get a bad reputation from some bad players, but
> they are functioning networks, and there are plenty of people using
> them for very valid purposes.

Yep. See previous statement.

> I'm old enough to remember (and I would guess many of us in this
> group are), when using a BBS or the internet made you suspect.

I'm guessing that you probably started using computers 5-10 years before me.

> In the early 90s, my wife at the time found me talking (voice) to
> someone online using powwow and accused me of all sorts of stuff. It
> was new to me, voice over internet, and I just had to mess with it. I
> still have that same interest and excitement doing things online.

Chuckle.

[John Goerzen]

On 2022-05-17, Jason Evans <jse...@mailfence.com> wrote:
> From my experience with running Tor onion services, it's quite easy to set up a
> server that can receive traffic over Tor. However sending traffic over Tor is
> another issue. It requires either setting up system proxies for outgoing
> traffic or wrapping the INN binaries in the torsocks application.

Late to the party I know, but I thought I might mention Yggdrasil:

https://yggdrasil-network.github.io/

Yggdrasil is an IPv6 global mesh network. It is always end-to-end encrypted,
and each node's IPv6 address is derived from its public key. You can therefore
do IP-based auth in traditional ways (in INN's config or with firewalls).
Yggdrasil shows up as a tun-type interface on your system.

Yggdrasil's goals are somewhat different than Tor's; it doesn't aim to provide
the kind of strong anonymity that Tor does, but rather it aims to be a
performant first-class network with modern security and routing.

I run NNCP over it and therefore already offer Usenet via NNCP over Yggdrasil
for those interested in that kind of peering.

There is no reason that you couldn't run INN over Yggdrasil because it's just
another interface to the system. Yggdrasil's design nicely avoids a lot of
problems with traditional IP networks (giving you a "direct" unfiltered
connection to the network, the IP address follows the device wherever it goes,
etc.)

John

[Miner]

John Goerzen wrote:

> On 2022-05-17, Jason Evans <jse...@mailfence.com> wrote:
> > From my experience with running Tor onion services, it's
> > quite easy to set up a server that can receive traffic over
> > Tor. However sending traffic over Tor is another issue. It
> > requires either setting up system proxies for outgoing
> > traffic or wrapping the INN binaries in the torsocks
> > application.
>
> Late to the party I know, but I thought I might mention Yggdrasil:

Good solution.

Someone from South Asia or from South America wanna peer with me
over Yggdrasil?

--
Miner

[Grant Taylor]

On 6/14/22 6:34 PM, John Goerzen wrote:
> Late to the party I know, but ...

ON topic and less than a month late, seems /okay/ to me.

> ... I thought I might mention Yggdrasil:
>
> https://yggdrasil-network.github.io/

Aside: Interesting (partial) name collision between "Yggdrasil Network"
and "Yggdrasil Linux" fro a VERY long time ago.

> Yggdrasil is an IPv6 global mesh network. It is always end-to-end
> encrypted, and each node's IPv6 address is derived from its public key.
> You can therefore do IP-based auth in traditional ways (in INN's config
> or with firewalls). Yggdrasil shows up as a tun-type interface on
> your system.

Very interesting!

I have a lot of questions, many of which I suspect will be answered when
i read about Yggdrasil Network at the link you provided.

> Yggdrasil's goals are somewhat different than Tor's; it doesn't aim
> to provide the kind of strong anonymity that Tor does, but rather
> it aims to be a performant first-class network with modern security
> and routing.

My initial assessment is that it's trying to be an overlay network.
Fair enough.

> I run NNCP over it and therefore already offer Usenet via NNCP over
> Yggdrasil for those interested in that kind of peering.

I'll keep that in mind.

> There is no reason that you couldn't run INN over Yggdrasil because
> it's just another interface to the system. Yggdrasil's design nicely
> avoids a lot of problems with traditional IP networks (giving you a
> "direct" unfiltered connection to the network, the IP address follows
> the device wherever it goes, etc.)

ACK

Presuming that my assumption is close and that I don't find anything
that disturbs me, I'll probably look at implementing Yggdrasil Network
support.

Thanks for sharing John.

[John Goerzen]

On 2022-06-15, Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 6/14/22 6:34 PM, John Goerzen wrote:
>> Late to the party I know, but ...
>
> ON topic and less than a month late, seems /okay/ to me.

<grin> I'm glad :-)

> Aside: Interesting (partial) name collision between "Yggdrasil Network"
> and "Yggdrasil Linux" fro a VERY long time ago.

Ah, I see I'm not the only one that had that thought!

> I have a lot of questions, many of which I suspect will be answered when
> i read about Yggdrasil Network at the link you provided.

Feel free to ask me also, by email or whatever newsgroup would be appropriate
(let me know if you go to a different group so I can make sure to subscribe)

> My initial assessment is that it's trying to be an overlay network.
> Fair enough.

I think they would put it slightly different; that the overlay network is one of
the key features, but is a bit limiting in terms of how to think about it.

Say, for instance, that you have a bunch of nodes using ad-hoc wifi, and they're
running Yggdrasil. Some of the laptops at, say, the northern end of the
location aren't within RF range of the ones at the southern end. But since
nodes within a broadcast domain auto-discover (via broadcast beacons) each other
and form a mesh, every node can communicate with every other node, using
go-betweens when necessary.

All it takes is one node to have access to other nodes and suddenly all of them
do. If one node has access to the Internet, then all of them can reach the
global Yggdrasil network. It will overlay across the public Internet when it's
the best path, but you don't even have to have a DHCP server for them to talk on
a LAN (it uses IPv6 link-local addresses in that case).

There's nothing that says that when you peer over the Internet, you must peer
with one of the public peers. If you only peer with private peers, then you can
effectively build an auto-meshing VPN.

As an aside, I love it for laptops. Yggdrasil+mosh is the perfect remote
terminal; close the laptop at home, open it up at a coffee shop or whatever, and
the laptop still has the same Yggdrasil IP, has discovered that it needs to
relay via the Internet instead of the LAN to its mosh/ssh destination at my home
network, and the session just keeps going; all that's different is higher ping
times. Go back home and it goes, "oh hey, now I can talk to you directly again,
let's do that" and ping times go back down.

> Presuming that my assumption is close and that I don't find anything
> that disturbs me, I'll probably look at implementing Yggdrasil Network
> support.

It's also on my list. I run INN inside Docker and would like to run Yggdrasil
inside Docker as well, and that is a little more complicated due to the kernel
permission structure around tun devices. But it's on my list for sure.

It is one of those "wow, this makes security so much easier" kinds of things.
Like, all those techniques from the 80s and 90s of IP-based security suddenly
work again :-)

John

[Grant Taylor]

On 6/15/22 3:14 PM, John Goerzen wrote:
> <grin> I'm glad :-)

:-)

> Ah, I see I'm not the only one that had that thought!

:-)

> Feel free to ask me also, by email or whatever newsgroup would be
> appropriate (let me know if you go to a different group so I can make
> sure to subscribe)

It's more that I feel like I should do some reading before asking a lot
of questions.

My gut reaction is that this is creating what I presume to be
point-to-point tunnels and using source & destination IP addresses
therein. Maybe it's lightweight tunnels that don't have dedicated
interfaces and choose encryption based on source & destination IP addresses.

I wonder if this interfaces with Linux very similarly to how IPsec does
via transformations. Wherein the Yggdrasil Network daemon is
functionally dynamically re-configuring the kernel transformation
mappings as necessary when an endpoint changes underlay connectivity.

> I think they would put it slightly different; that the overlay network
> is one of the key features, but is a bit limiting in terms of how to
> think about it.

Fair enough. I was using a common term for a network on top of a network.

I didn't mean to imply that the overlay network was only a network on
top of another network.

> Say, for instance, that you have a bunch of nodes using ad-hoc wifi,
> and they're running Yggdrasil. Some of the laptops at, say, the
> northern end of the location aren't within RF range of the ones at the
> southern end. But since nodes within a broadcast domain auto-discover
> (via broadcast beacons) each other and form a mesh, every node can
> communicate with every other node, using go-betweens when necessary.

Interesting.

> All it takes is one node to have access to other nodes and suddenly
> all of them do. If one node has access to the Internet, then all
> of them can reach the global Yggdrasil network. It will overlay
> across the public Internet when it's the best path, but you don't
> even have to have a DHCP server for them to talk on a LAN (it uses
> IPv6 link-local addresses in that case).

:-)

> There's nothing that says that when you peer over the Internet,
> you must peer with one of the public peers. If you only peer with
> private peers, then you can effectively build an auto-meshing VPN.

Good to know.

> As an aside, I love it for laptops. Yggdrasil+mosh is the perfect
> remote terminal; close the laptop at home, open it up at a coffee
> shop or whatever, and the laptop still has the same Yggdrasil IP,
> has discovered that it needs to relay via the Internet instead of the
> LAN to its mosh/ssh destination at my home network, and the session
> just keeps going; all that's different is higher ping times. Go back
> home and it goes, "oh hey, now I can talk to you directly again,
> let's do that" and ping times go back down.

Nice.

> It's also on my list. I run INN inside Docker and would like to run
> Yggdrasil inside Docker as well, and that is a little more complicated
> due to the kernel permission structure around tun devices. But it's
> on my list for sure.

Ya. The routing to interconnect between containers and the rest of the
LAN could be ... interesting ... if not trying.

> It is one of those "wow, this makes security so much easier" kinds
> of things. Like, all those techniques from the 80s and 90s of IP-based
> security suddenly work again :-)

Yep.

[John Goerzen]

On 2022-06-15, Grant Taylor <gta...@tnetconsulting.net> wrote:

> My gut reaction is that this is creating what I presume to be
> point-to-point tunnels and using source & destination IP addresses
> therein. Maybe it's lightweight tunnels that don't have dedicated
> interfaces and choose encryption based on source & destination IP addresses.

So there is only one Yggdrasil network interface on a computer: tun0 or ygg0 or
whatever you want to call it.

https://yggdrasil-network.github.io/about.html discusses this a bit.

Each Yggdrasil node has "peers" with which it has established direct
connectivity. This activity can be established opportunistically (as with
broadcasts over a LAN) or with defined peers (as with using it as an overlay
over the Internet). You can optionally exert control over who can peer with you
based on the peer's public key.

So once you have some route to the global Yggdrasil network, the Yggdrasil
daemon knows how to reach any of the thousands of addresses on it. This is
where the magic happens, but it's all within the daemon; to the kernel, it just
sends packets down tun0 and that's it.

Other than cjdns, which was sort of a predecessor technology, the closest
similar program I am aware of is tinc https://tinc-vpn.org/ . But there, you
have to explicitly set up each node to know about the others.

John

[meff]

On 2022-06-15, John Goerzen <jgoe...@complete.org> wrote:
> As an aside, I love it for laptops. Yggdrasil+mosh is the perfect remote
> terminal; close the laptop at home, open it up at a coffee shop or whatever, and
> the laptop still has the same Yggdrasil IP, has discovered that it needs to
> relay via the Internet instead of the LAN to its mosh/ssh destination at my home
> network, and the session just keeps going; all that's different is higher ping
> times. Go back home and it goes, "oh hey, now I can talk to you directly again,
> let's do that" and ping times go back down.

I actually use ZeroTier myself for that right now. Only trouble I've
had with ZeroTier is behind really restrictive firewalls (only
happened in a historic hotel I stayed in, but I had more going on
around me at that point than would encourage me to get onto the
computer much 😄 .) In these cases you could probably have your Ygg
node listen on 443 which should break through even most restrictive
firewalls, which I don't think ZT can do. ZT assigns private IPs on a
private network, both v4 and v6, and these IPs stay stable per
instance of the daemon running. So ZT on a home server has a stable ZT
IP for that network and unless you are on the network, you cannot
connect to that IP.

Ygg gives you the global network which is certainy more fun than just
a private network, but I've started using ZT for NNCP and even for
streaming security cameras using RDP. I also run a ZT network for my
family in their houses so I can remote troubleshoot things on their
networks without having to go over. I can also run ZT on my Android
phone and I know it has iOS support though I don't have any iOS
devices myself.

If anyone is looking for a private network solution I do recommend
ZeroTier. Sorry for a bit of a non-sequitur.

[Grant Taylor]

On 6/15/22 11:08 AM, Grant Taylor wrote:
> I have a lot of questions, many of which I suspect will be answered when
> i read about Yggdrasil Network at the link you provided.

Sadly, none of my technical questions were answered by reading pages on
the Yggdrasil Network website [1].

I never did find what I would consider to be a description of how things
work from a networking / application point of view. I was also very
surprised to not find any documentation on what settings to put into the
configuration file.

I actually got better technical information in this thread than I did
from the Yggdrasil Network website.

[1] https://yggdrasil-network.github.io/

[Grant Taylor]

On 6/15/22 3:14 PM, John Goerzen wrote:

> Feel free to ask me also, by email or whatever newsgroup would be
> appropriate

What addresses does Yggdrasil Network use? It looked to me like they
were using IPv6 addresses that aren't currently routed globally. I
think I saw a few different /16 xx:xx::/xx networks listed in
documentation. But I didn't see anything that actually clearly sated
what Yggdrasil Network uses.

Nor did I find anything that actually talked about how the addresses are
used.

I naively assume that standard destination based routing is used such
that the kernel sends traffic for various destinations into the tun0 /
ygg0 / etc. network interface and the Yggdrasil Network daemon handles
the rest.

But I didn't find anything to support this assumption, much less any
description of how such would be done.

I'm used to things like Tor that state -- effectively -- use a SOCKS
server at $CONFIGURED_ADDRESS on $CONFIGURED_PORT -- with more details
about how to do said configuration.

I wonder about the ability to have standard DNS names resolve to
Yggdrasil Network addresses. But, I didn't find enough information to
either confirm nor refute this.

My opinion is that the Yggdrasil Network's website is lacking a
significant amount of technical documentation.

[Andreas Kempe]

Den 2022-05-21 skrev Retro Guy <retr...@rocksolidbbs.com>:
> Grant Taylor wrote:
>
>> On 5/20/22 9:50 PM, Retro Guy wrote:
>>> Yes, that's true. I2P makes it much easier.
>
>> Would you please elaborate on what I2P does that's different?
>
> There are two features in I2P that help with this. One is that you
> can tie a client key (generated by the client) to a server tunnel.
> You can whitelist key(s) for this tunnel, and only allow specific
> clients to connect.
>

Sorry for coming in so late, but I want to point out that Tor does
support client keys with for onions. You can generate keys and
configure Tor to only allow connections from clients with specific
keys.

It is a useful feature that can definitely increase security should
your onion address leak for some reason so I thought it worth
mentioning.

There are instruction available from the Tor project at
https://community.torproject.org/onion-services/advanced/client-auth/.

Best regards,
Andreas Kempe

[Retro Guy]

Thank you for this info and link. I was not aware this is possible with tor.

--
Retro Guy

[Thomas Hochstein]

Grant Taylor wrote:

> What addresses does Yggdrasil Network use? It looked to me like they
> were using IPv6 addresses that aren't currently routed globally. I
> think I saw a few different /16 xx:xx::/xx networks listed in
> documentation. But I didn't see anything that actually clearly sated
> what Yggdrasil Network uses.

| Will Yggdrasil conflict with my network routing?
|
| Yggdrasil uses the 0200::/7 range, which is a range deprecated by the
| IETF. It has been deprecated since 2004, pending changes to an RFC which
| simply never materialised all these years later. It was decided to use
| this range instead of fc00::/7 (which is more typically allocated to
| private networks) in order to prevent conflicts with existing ULA ranges.
<https://yggdrasil-network.github.io/faq.html>

> I naively assume that standard destination based routing is used such
> that the kernel sends traffic for various destinations into the tun0 /
> ygg0 / etc. network interface and the Yggdrasil Network daemon handles
> the rest.

Looks like that.

> My opinion is that the Yggdrasil Network's website is lacking a
> significant amount of technical documentation.

Most probably that's "in the code", currently, and nobody wrote it down
yet. :)

-thh