info
Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Modern Uses of UUCP & NNCP
222 views
Skip to first unread message
John Goerzen
Jan 3, 2021, 8:06:52 PM1/3/21
to
Hello folks,
Well I just discovered this group still exists (sweet!) so I thought I would share a little bit. I have been writing a blog series about modern uses of UUCP (and more on NNCP), particularly focused around ZFS backups.
Before I dig into that, I searched and it looks like NNCP hasn't been mentioned here yet. I should do that now. NNCP is essentially an effort to modernize UUCP. It maintains equivolents of the basic uux, uucp, uucico, uuxqt, etc. functionality - in fact, with a few exceptions I'll note here, it has broadly achieved feature parity with UUCP.
There are some things that set it apart:
First of all is modern security. Every message packet is encrypted and authenticated. Routing via other hosts is done with Tor-style onion routing (so intermediate hosts can see neither the content nor the ultimate destination of the packet unless it is their neighbor). Communication is done securely or not at all. (And NNCP supports a pervasive "nice" concept, similar to the UUCP grade, that can be used in both ordering of packets in a call and in deciding which packets to transmit during a call.)
NNCP has uucico-style daemon and caller, but also some other modes: nncp-xfer (designed to facilitate easy airgapped transfer on a mountable device such as USB stick) and nncp-bundle (emits a tar-style stream that can be, eg, broadcasted on radio, burned to CD, even sent via uucp).
Some things it doesn't do that UUCP does:
NNCP requires a clean link and doesn't have any special logic for serial port handling. However, it can run over a pipe so a program like socat, getty, etc. could invoke it over a clean link. Authentication is done using public keypairs so no specific usernames are needed.
By the nature of its encryption, you cannot (for instance) send email to a node that you don't know the key for. You also need to know the public key of every intermediate hop along the way. But, the NNCP -via option is the exact same concept as UUCP bang paths.
And finally, some of the more esoteric uux invocations, for instance uux 'diff -c sys1!~user1/file1 sys2!~user2/file2 >!file.diff' as in the Taylor UUCP manpage, aren't supported. However, by the time I started using UUCP in the 90s, I had never seen uux used for anything but rmail and rnews. My copy of Managing and Using UUCP by O'Reilly claims that indeed those more esoteric uux invocations were almost never used by the time it was published.
The project's website is http://www.nncpgo.org/
My own blog series is at https://changelog.complete.org/archives/tag/asynchronous
It begins with an exploration of why airgapped backups could be valuable, then talks about why we tend to reach for synchronous tools like ssh even when asynchronous like UUCP or NNCP would be better, demonstrates sending mail across NNCP using bsmtp rather than rmail mode (the same could be done over UUCP, of course), then develops the ZFS backups over NNCP. I have several more installments planned as well - tuning the ZFS installation and then progressing into non-ZFS backups, which are rather more tricky to get right (both sync and async, but especially async).
On a personal note... I just checked my archives, and it looks like I first got a UUCP feed at the age of 16, for which I paid $12/mo to an ISP, plus long distance charges. I later worked as a sysadmin at that ISP, and by that point part of my job was maintaining what by that time were the "legacy" services like UUCP on BSD/OS.
Well I just discovered this group still exists (sweet!) so I thought I would share a little bit. I have been writing a blog series about modern uses of UUCP (and more on NNCP), particularly focused around ZFS backups.
Before I dig into that, I searched and it looks like NNCP hasn't been mentioned here yet. I should do that now. NNCP is essentially an effort to modernize UUCP. It maintains equivolents of the basic uux, uucp, uucico, uuxqt, etc. functionality - in fact, with a few exceptions I'll note here, it has broadly achieved feature parity with UUCP.
There are some things that set it apart:
First of all is modern security. Every message packet is encrypted and authenticated. Routing via other hosts is done with Tor-style onion routing (so intermediate hosts can see neither the content nor the ultimate destination of the packet unless it is their neighbor). Communication is done securely or not at all. (And NNCP supports a pervasive "nice" concept, similar to the UUCP grade, that can be used in both ordering of packets in a call and in deciding which packets to transmit during a call.)
NNCP has uucico-style daemon and caller, but also some other modes: nncp-xfer (designed to facilitate easy airgapped transfer on a mountable device such as USB stick) and nncp-bundle (emits a tar-style stream that can be, eg, broadcasted on radio, burned to CD, even sent via uucp).
Some things it doesn't do that UUCP does:
NNCP requires a clean link and doesn't have any special logic for serial port handling. However, it can run over a pipe so a program like socat, getty, etc. could invoke it over a clean link. Authentication is done using public keypairs so no specific usernames are needed.
By the nature of its encryption, you cannot (for instance) send email to a node that you don't know the key for. You also need to know the public key of every intermediate hop along the way. But, the NNCP -via option is the exact same concept as UUCP bang paths.
And finally, some of the more esoteric uux invocations, for instance uux 'diff -c sys1!~user1/file1 sys2!~user2/file2 >!file.diff' as in the Taylor UUCP manpage, aren't supported. However, by the time I started using UUCP in the 90s, I had never seen uux used for anything but rmail and rnews. My copy of Managing and Using UUCP by O'Reilly claims that indeed those more esoteric uux invocations were almost never used by the time it was published.
The project's website is http://www.nncpgo.org/
My own blog series is at https://changelog.complete.org/archives/tag/asynchronous
It begins with an exploration of why airgapped backups could be valuable, then talks about why we tend to reach for synchronous tools like ssh even when asynchronous like UUCP or NNCP would be better, demonstrates sending mail across NNCP using bsmtp rather than rmail mode (the same could be done over UUCP, of course), then develops the ZFS backups over NNCP. I have several more installments planned as well - tuning the ZFS installation and then progressing into non-ZFS backups, which are rather more tricky to get right (both sync and async, but especially async).
On a personal note... I just checked my archives, and it looks like I first got a UUCP feed at the age of 16, for which I paid $12/mo to an ISP, plus long distance charges. I later worked as a sysadmin at that ISP, and by that point part of my job was maintaining what by that time were the "legacy" services like UUCP on BSD/OS.
Grant Taylor
Jan 3, 2021, 9:30:56 PM1/3/21
to
On 1/3/21 6:06 PM, John Goerzen wrote:
> Hello folks,
Hi John,
> Well I just discovered this group still exists (sweet!) so I thought
> I would share a little bit.
The group existing is one thing. Active discussion in the group is
another. I'm showing 57 messages in the last ~2 years. So, just enough
to keep the lights on. ;-)
> I have been writing a blog series about modern uses of UUCP (and more
> on NNCP), particularly focused around ZFS backups.
$ReadingList++
Aside: I've found *MANY* things on your blog to be well worth the time
to read and understand them. Thank you for your content.
> Before I dig into that, I searched and it looks like NNCP hasn't
> been mentioned here yet.
I think that partially has to do with 1) this groups minimal activity,
and more so to do with 2) the newness of NNCP. -- The oldest thing
that I've found was Jan-Piet Mens' tweet about it mid December.
I actually went looking for (on topic) NNCP content last night and
couldn't find much at all on Google. JP's tweet, another tweet, and a
couple of blog articles, one of which was yours.
> I should do that now.
;-)
> NNCP is essentially an effort to modernize UUCP.
That's the impression that I'm getting.
> It maintains equivolents of the basic uux, uucp, uucico, uuxqt,
> etc. functionality - in fact, with a few exceptions I'll note here,
> it has broadly achieved feature parity with UUCP.
:-)
> There are some things that set it apart:
>
> First of all is modern security. Every message packet is encrypted
> and authenticated.
Am I correct in assuming that the authentication that you speak of is
based on the purported source needing to have the private key that is
associated with the configured public key for said source?
> Routing via other hosts is done with Tor-style onion routing (so
> intermediate hosts can see neither the content nor the ultimate
> destination of the packet unless it is their neighbor). Communication
> is done securely or not at all. (And NNCP supports a pervasive "nice"
> concept, similar to the UUCP grade, that can be used in both ordering
> of packets in a call and in deciding which packets to transmit during
> a call.)
>
> NNCP has uucico-style daemon and caller,
What is the role of the daemon?
I may not completely grok my UUCP configuration, but there isn't a
daemon running. (UUCP-over-SSH) My understanding is that things are
mainly spawned by cron and / or reactively to what the cron jobs kick
off when connecting to other systems.
> but also some other modes: nncp-xfer (designed to facilitate easy
> airgapped transfer on a mountable device such as USB stick) and
> nncp-bundle (emits a tar-style stream that can be, eg, broadcasted
> on radio, burned to CD, even sent via uucp).
Perhaps I'm mistaken, but I thought that it was possible to transplant
files from one UUCP queue to another UUCP queue, possibly via tape,
flash drive, floppy, etc.
I wonder if nncp-{xfer,bundle} are meant to simplify this transplanting
or make it more of a first class operation.
> Some things it doesn't do that UUCP does:
>
> NNCP requires a clean link and doesn't have any special logic
> for serial port handling.
I personally feel like requiring an 8-bit clean connection is probably
fairly safe these days.
> However, it can run over a pipe so a program like socat, getty,
> etc. could invoke it over a clean link.
If you're speaking to (near) feature parity, I think that there should
probably be some documentation of how to get NNCP to work over a serial
/ dial-up connection. Even if it's piping into something like cu.
Having the getty on the far side invoke what's necessary to answer the
call is probably sufficient. Though it should also be documented.
> Authentication is done using public keypairs so no specific usernames
> are needed.
The credentials aren't needed for NNCP, but they probably are still
needed to placate the getty on the remote side. ;-)
> By the nature of its encryption, you cannot (for instance) send
> email to a node that you don't know the key for.
How is this surfaced / exposed to end user? Would the MTA detect this
as a failure in the mailer (Sendmail parlance) and return a DSN to the
(purported) sender?
> You also need to know the public key of every intermediate hop along
> the way. But, the NNCP -via option is the exact same concept as UUCP
> bang paths.
Is there a way to have NNCP query nodes for the public keys of the other
nodes that it knows about?
Is there any plan to have any sort of discovery / coordination /
learning / ??? of public keys? Perhaps DNS record(s) of some sort
(TLSA, TXT, PTR, SRV, etc.); _nncp._tcp.$NODE...
Aside: Are NNCP node names unqualified or fully qualified?
Further aside: Has there been any thought to federating a la. MX records?
> And finally, some of the more esoteric uux invocations, for instance
> uux 'diff -c sys1!~user1/file1 sys2!~user2/file2 >!file.diff' as in
> the Taylor UUCP manpage, aren't supported. However, by the time I
> started using UUCP in the 90s, I had never seen uux used for anything
> but rmail and rnews.
I've personally run multiple other commands via uux within the last year.
> My copy of Managing and Using UUCP by O'Reilly claims that indeed
> those more esoteric uux invocations were almost never used by the
> time it was published.
The only time that I relied on different remote source files was when
experimenting with things.
I've read / skimmed a number of the English pages. I've not bothered to
try translating any of the Russian pages.
:-)
> It begins with an exploration of why airgapped backups could be
> valuable, then talks about why we tend to reach for synchronous tools
> like ssh even when asynchronous like UUCP or NNCP would be better,
Nitpick: I think you're alluding to store-and-forward networking vs.
end-to-end networking. I've found SSH between nodes to be quite
effective communications mechanism for store-and-forward UUCP-over-SSH
even when end-to-end UUCP-over-SSH is not possible.
1) UUCP-over-SSH between source and an intermediate notebook
2) The intermediate notebook travels in the back of a station wagon
3) UUCP-over-SSH between the intermediate notebook and destination.
> demonstrates sending mail across NNCP using bsmtp rather than rmail
> mode (the same could be done over UUCP, of course),
I'm not overly familiar with Batch SMTP [1], but my skim of the RFC
causes me some indigestion. Though, I'm not sure what similar
assumptions that UUCP / rmail make. In particular, how does UUCP deal
with the receiving system rejecting an incoming message? Does this
require a more complex BSMTP client / gateway that has the ability to
generate a bounce that goes back through UUCP (NNCP)?
> then develops the ZFS backups over NNCP.
I too *REALLY* like the idea of sending ZFS backups over a
store-and-forward network to a remote system. Though I see a lot of
unknowns there. Do you /blindly/ "receive" the "send" directly into a
target pool or do you queue it as a file somewhere first? What if the
pool is not in the proper state to receive what is sent, possibly
because of an outstanding send that hasn't been received yet? Or are
there implicit assumptions. -- I'd love to chat more (perhaps elsewhere).
Anyway ... back to NNCP.
> I have several more installments planned as well - tuning the ZFS
> installation and then progressing into non-ZFS backups, which are
> rather more tricky to get right (both sync and async, but especially
> async).
I suspect that NNCP has quite similar use cases as UUCP. One just needs
to be creative in how it's done.
~chuckle~ Broadcasting NNCP xfer / batch into a Usenet newsgroup for
the recipient to collect elsewhere.
> On a personal note... I just checked my archives, and it looks like
> I first got a UUCP feed at the age of 16, for which I paid $12/mo to
> an ISP, plus long distance charges. I later worked as a sysadmin at
> that ISP, and by that point part of my job was maintaining what by
> that time were the "legacy" services like UUCP on BSD/OS.
Fun.
I think that some thought, and comments / documentation should probably
be given to why NNCP /needs/ it's own transport. What if NNCP was the
next thing to use uux via rnncp. ;-) Meaning add the authentication on
top of the existing UUCP instead of outright replacing it.
I do think that NNCP is a very interesting idea. I feel like there are
a LOT of unknowns. Almost as if it's a solution in search of a problem.
But that may just be observation bias.
[1] RFC 2442 - The Batch SMTP Media Type
--
Grant. . . .
unix || die
> Hello folks,
Hi John,
> Well I just discovered this group still exists (sweet!) so I thought
> I would share a little bit.
another. I'm showing 57 messages in the last ~2 years. So, just enough
to keep the lights on. ;-)
> I have been writing a blog series about modern uses of UUCP (and more
> on NNCP), particularly focused around ZFS backups.
Aside: I've found *MANY* things on your blog to be well worth the time
to read and understand them. Thank you for your content.
> Before I dig into that, I searched and it looks like NNCP hasn't
> been mentioned here yet.
and more so to do with 2) the newness of NNCP. -- The oldest thing
that I've found was Jan-Piet Mens' tweet about it mid December.
I actually went looking for (on topic) NNCP content last night and
couldn't find much at all on Google. JP's tweet, another tweet, and a
couple of blog articles, one of which was yours.
> I should do that now.
> NNCP is essentially an effort to modernize UUCP.
> It maintains equivolents of the basic uux, uucp, uucico, uuxqt,
> etc. functionality - in fact, with a few exceptions I'll note here,
> it has broadly achieved feature parity with UUCP.
> There are some things that set it apart:
>
> First of all is modern security. Every message packet is encrypted
> and authenticated.
based on the purported source needing to have the private key that is
associated with the configured public key for said source?
> Routing via other hosts is done with Tor-style onion routing (so
> intermediate hosts can see neither the content nor the ultimate
> destination of the packet unless it is their neighbor). Communication
> is done securely or not at all. (And NNCP supports a pervasive "nice"
> concept, similar to the UUCP grade, that can be used in both ordering
> of packets in a call and in deciding which packets to transmit during
> a call.)
>
> NNCP has uucico-style daemon and caller,
I may not completely grok my UUCP configuration, but there isn't a
daemon running. (UUCP-over-SSH) My understanding is that things are
mainly spawned by cron and / or reactively to what the cron jobs kick
off when connecting to other systems.
> but also some other modes: nncp-xfer (designed to facilitate easy
> airgapped transfer on a mountable device such as USB stick) and
> nncp-bundle (emits a tar-style stream that can be, eg, broadcasted
> on radio, burned to CD, even sent via uucp).
files from one UUCP queue to another UUCP queue, possibly via tape,
flash drive, floppy, etc.
I wonder if nncp-{xfer,bundle} are meant to simplify this transplanting
or make it more of a first class operation.
> Some things it doesn't do that UUCP does:
>
> NNCP requires a clean link and doesn't have any special logic
> for serial port handling.
fairly safe these days.
> However, it can run over a pipe so a program like socat, getty,
> etc. could invoke it over a clean link.
probably be some documentation of how to get NNCP to work over a serial
/ dial-up connection. Even if it's piping into something like cu.
Having the getty on the far side invoke what's necessary to answer the
call is probably sufficient. Though it should also be documented.
> Authentication is done using public keypairs so no specific usernames
> are needed.
needed to placate the getty on the remote side. ;-)
> By the nature of its encryption, you cannot (for instance) send
> email to a node that you don't know the key for.
as a failure in the mailer (Sendmail parlance) and return a DSN to the
(purported) sender?
> You also need to know the public key of every intermediate hop along
> the way. But, the NNCP -via option is the exact same concept as UUCP
> bang paths.
nodes that it knows about?
Is there any plan to have any sort of discovery / coordination /
learning / ??? of public keys? Perhaps DNS record(s) of some sort
(TLSA, TXT, PTR, SRV, etc.); _nncp._tcp.$NODE...
Aside: Are NNCP node names unqualified or fully qualified?
Further aside: Has there been any thought to federating a la. MX records?
> And finally, some of the more esoteric uux invocations, for instance
> uux 'diff -c sys1!~user1/file1 sys2!~user2/file2 >!file.diff' as in
> the Taylor UUCP manpage, aren't supported. However, by the time I
> started using UUCP in the 90s, I had never seen uux used for anything
> but rmail and rnews.
> My copy of Managing and Using UUCP by O'Reilly claims that indeed
> those more esoteric uux invocations were almost never used by the
> time it was published.
experimenting with things.
I've read / skimmed a number of the English pages. I've not bothered to
try translating any of the Russian pages.
:-)
> It begins with an exploration of why airgapped backups could be
> valuable, then talks about why we tend to reach for synchronous tools
> like ssh even when asynchronous like UUCP or NNCP would be better,
end-to-end networking. I've found SSH between nodes to be quite
effective communications mechanism for store-and-forward UUCP-over-SSH
even when end-to-end UUCP-over-SSH is not possible.
1) UUCP-over-SSH between source and an intermediate notebook
2) The intermediate notebook travels in the back of a station wagon
3) UUCP-over-SSH between the intermediate notebook and destination.
> demonstrates sending mail across NNCP using bsmtp rather than rmail
> mode (the same could be done over UUCP, of course),
causes me some indigestion. Though, I'm not sure what similar
assumptions that UUCP / rmail make. In particular, how does UUCP deal
with the receiving system rejecting an incoming message? Does this
require a more complex BSMTP client / gateway that has the ability to
generate a bounce that goes back through UUCP (NNCP)?
> then develops the ZFS backups over NNCP.
store-and-forward network to a remote system. Though I see a lot of
unknowns there. Do you /blindly/ "receive" the "send" directly into a
target pool or do you queue it as a file somewhere first? What if the
pool is not in the proper state to receive what is sent, possibly
because of an outstanding send that hasn't been received yet? Or are
there implicit assumptions. -- I'd love to chat more (perhaps elsewhere).
Anyway ... back to NNCP.
> I have several more installments planned as well - tuning the ZFS
> installation and then progressing into non-ZFS backups, which are
> rather more tricky to get right (both sync and async, but especially
> async).
to be creative in how it's done.
~chuckle~ Broadcasting NNCP xfer / batch into a Usenet newsgroup for
the recipient to collect elsewhere.
> On a personal note... I just checked my archives, and it looks like
> I first got a UUCP feed at the age of 16, for which I paid $12/mo to
> an ISP, plus long distance charges. I later worked as a sysadmin at
> that ISP, and by that point part of my job was maintaining what by
> that time were the "legacy" services like UUCP on BSD/OS.
I think that some thought, and comments / documentation should probably
be given to why NNCP /needs/ it's own transport. What if NNCP was the
next thing to use uux via rnncp. ;-) Meaning add the authentication on
top of the existing UUCP instead of outright replacing it.
I do think that NNCP is a very interesting idea. I feel like there are
a LOT of unknowns. Almost as if it's a solution in search of a problem.
But that may just be observation bias.
[1] RFC 2442 - The Batch SMTP Media Type
--
Grant. . . .
unix || die
John Goerzen
Jan 4, 2021, 12:51:42 AM1/4/21
to
On Sun, Jan 03 2021, Grant Taylor wrote:
> On 1/3/21 6:06 PM, John Goerzen wrote:
> Hi John,
>
>> Well I just discovered this group still exists (sweet!) so I thought I would
>> share a little bit.
>
> The group existing is one thing. Active discussion in the group is another.
> I'm showing 57 messages in the last ~2 years. So, just enough to keep the
> lights on. ;-)
Hey, the posts are outnumbering the spam (at least on Google Groups [yeah, I
> On 1/3/21 6:06 PM, John Goerzen wrote:
> Hi John,
>
>> Well I just discovered this group still exists (sweet!) so I thought I would
>> share a little bit.
>
> The group existing is one thing. Active discussion in the group is another.
> I'm showing 57 messages in the last ~2 years. So, just enough to keep the
> lights on. ;-)
know, don't shoot me]) so I took that as a positive sign for Usenet :-)
> Aside: I've found *MANY* things on your blog to be well worth the time to
> read
> and understand them. Thank you for your content.
>
>> There are some things that set it apart:
>>
>> First of all is modern security. Every message packet is encrypted and
>> authenticated.
>
> Am I correct in assuming that the authentication that you speak of is based on
> the purported source needing to have the private key that is associated with
> the
> configured public key for said source?
of NNCP, so I can speak to my understanding of it, but not necessarily with
authority. There is a mailing list at
https://lists.cypherpunks.ru/pipermail/nncp-devel/ and the author is active
there.
With that out of the way, yes, that is indeed my understanding.
>> Routing via other hosts is done with Tor-style onion routing (so intermediate
>> hosts can see neither the content nor the ultimate destination of the packet
>> unless it is their neighbor). Communication is done securely or not at all.
>> (And NNCP supports a pervasive "nice" concept, similar to the UUCP grade,
>> that
>> can be used in both ordering of packets in a call and in deciding which
>> packets to transmit during a call.)
>>
>> NNCP has uucico-style daemon and caller,
>
> What is the role of the daemon?
>
> I may not completely grok my UUCP configuration, but there isn't a daemon
> running. (UUCP-over-SSH) My understanding is that things are mainly spawned
> by
> cron and / or reactively to what the cron jobs kick off when connecting to
> other
> systems.
uucico, which is both the outbound caller and the inbound listener/daemon. In
NNCP, these roles are split into three separate programs:
nncp-daemon - the listener (roughly uucico -l)
nncp-call - an on-demand caller (uucico -S)
nncp-caller - the caller with scheduling rules (uucico -s if memory serves?)
The "daemon" can be run as a daemon (I run it under systemd [again, don't shoot
me!] quite easily). But it also has an -inetd option that, of course, can run
under inetd... but is really just a "speak the protocol on stdin/stdout" so it
could just as easily be run under ssh as with uucico.
>> but also some other modes: nncp-xfer (designed to facilitate easy airgapped
>> transfer on a mountable device such as USB stick) and nncp-bundle (emits a
>> tar-style stream that can be, eg, broadcasted on radio, burned to CD, even
>> sent via uucp).
>
> Perhaps I'm mistaken, but I thought that it was possible to transplant files
> from one UUCP queue to another UUCP queue, possibly via tape, flash drive,
> floppy, etc.
>
> I wonder if nncp-{xfer,bundle} are meant to simplify this transplanting or
> make
> it more of a first class operation.
fundamentally store packets in regular files, so as long as you are careful not
to trip over race conditions in either direction, it should be reasonably
simple. (I wonder if sequence numbers make it at all more complicated on the
UUCP side?
This is one article about doing something similar with UUCP:
https://www.dumain.com/posts/Forward_to_the_1970s_with_UUCP.../ and it wasn't
super simple. TBH I've never tried it. The NNCP tools for this also have
built-in support for grades/nice, preventing duplicate packets (I can't remember
if UUCP does this anymore; I seem to recall not?).
>> Some things it doesn't do that UUCP does:
>>
>> NNCP requires a clean link and doesn't have any special logic for serial port
>> handling.
>
> I personally feel like requiring an 8-bit clean connection is probably fairly
> safe these days.
with 500mW or less) so it isn't ALWAYS that way even now!
>> However, it can run over a pipe so a program like socat, getty, etc. could
>> invoke it over a clean link.
>
> If you're speaking to (near) feature parity, I think that there should
> probably
> be some documentation of how to get NNCP to work over a serial / dial-up
> connection. Even if it's piping into something like cu. Having the getty on
> the
> far side invoke what's necessary to answer the call is probably sufficient.
> Though it should also be documented.
Must put that on my list...
>
>> Authentication is done using public keypairs so no specific usernames are
>> needed.
>
> The credentials aren't needed for NNCP, but they probably are still needed to
> placate the getty on the remote side. ;-)
>> By the nature of its encryption, you cannot (for instance) send email to a
>> node that you don't know the key for.
>
> How is this surfaced / exposed to end user? Would the MTA detect this as a
> failure in the mailer (Sendmail parlance) and return a DSN to the (purported)
> sender?
One could, in both UUCP and NNCP worlds, handle multi-hop mail routing by
either:
1) Having the MTA do it (presumably via a smarthost for leaf nodes) - in a
manner similar to how things happen over the Internet now
2) Doing this at the UUCP/NNCP protocol level
So basically if you are going to have the MTA on node1 execute something like
"nncp-exec -via node2,node3,node4 node5 rmail janedoe" (aka uux
node2!node3!node4!node5!rmail janedoe) then the sending node needs to have the
keys for all four of those nodes since it needs to prepare all of the onion
encryption layers.
If, on the other hand, node1 knows that "meh, I just send all my mail to node2
and it figures it out", then it would be doing "nncp-exec node2 rmail
janedoe@node5" or something similar; node2 presumably would know to route that
via node3 and so forth.
I have never set up an MTA with full bang-path routing (my UUCP uses were all
late enough to be essentially Internet leaf sites with RFC822-style addressing
routed over UUCP) so TBH I don't quite know how the MTAs handled that, but my
understanding is that is was pretty much on the user to know the full bang path
from the local system to the recipient and it just handed this over to uux.
NNCP's config file actually lets you specify a default route to a known node, so
there would be no need for explicit routing at the MTA level if you just do it
all at the NNCP level. Doing it at the NNCP level would have a security benefit
as well, since it would be neither necessary nor possible to decrypt it at MTAs
along the way.
Exim has an example config where you could send things to, eg,
jan...@node5.UUCP and it could have a routing lookup table -- or just pass this
to the lower level. With NNCP's default "via" settings available in the config,
the routing can be invisible to both the MTA and the user (but of course, the
keys are needed).
If the MTA were to call nncp-exec to a node that nncp-exec doesn't know about,
it would get an immediate command failure from the pipe to nncp-exec and that
would presumably turn into an immediate DSN. The difference is that with uux,
the local system would only need to know about node2 and if, say, node3 doesn't
know about node4, then that's an issue that surfaces later. (Technically this
could still happen in NNCP; you could know about all those nodes but then node3
might not know about node4, but I'd think it would be more rare)
>> You also need to know the public key of every intermediate hop along the way.
>> But, the NNCP -via option is the exact same concept as UUCP bang paths.
>
> Is there a way to have NNCP query nodes for the public keys of the other nodes
> that it knows about?
> Is there any plan to have any sort of discovery / coordination / learning /
> ???
> of public keys? Perhaps DNS record(s) of some sort (TLSA, TXT, PTR, SRV,
> etc.);
> _nncp._tcp.$NODE...
> Aside: Are NNCP node names unqualified or fully qualified?
node name is essentially a convenience alias for the node's public key (akin to
a hostname being a convenience alias for an IP address). A node doesn't have to
be known by the same name on every host, either, since the spool directories and
all internal structures are based off the node's public key rather than its
name.
> Further aside: Has there been any thought to federating a la. MX records?
nodes - which is what I've most commonly seen UUCP evolved into (as there is no
longer any global UUCP network) - but there is no particular reason it couldn't
evolve in that direction.
>> And finally, some of the more esoteric uux invocations, for instance uux
>> 'diff
>> -c sys1!~user1/file1 sys2!~user2/file2 >!file.diff' as in the Taylor UUCP
>> manpage, aren't supported. However, by the time I started using UUCP in the
>> 90s, I had never seen uux used for anything but rmail and rnews.
>
> I've personally run multiple other commands via uux within the last year.
>> It begins with an exploration of why airgapped backups could be valuable,
>> then
>> talks about why we tend to reach for synchronous tools like ssh even when
>> asynchronous like UUCP or NNCP would be better,
>
> Nitpick: I think you're alluding to store-and-forward networking vs.
> end-to-end
> networking. I've found SSH between nodes to be quite effective communications
> mechanism for store-and-forward UUCP-over-SSH even when end-to-end
> UUCP-over-SSH
> is not possible.
>
> 1) UUCP-over-SSH between source and an intermediate notebook
> 2) The intermediate notebook travels in the back of a station wagon
> 3) UUCP-over-SSH between the intermediate notebook and destination.
>> demonstrates sending mail across NNCP using bsmtp rather than rmail mode (the
>> same could be done over UUCP, of course),
>
> I'm not overly familiar with Batch SMTP [1], but my skim of the RFC causes me
> some indigestion. Though, I'm not sure what similar assumptions that UUCP /
> rmail make. In particular, how does UUCP deal with the receiving system
> rejecting an incoming message? Does this require a more complex BSMTP client
> /
> gateway that has the ability to generate a bounce that goes back through UUCP
> (NNCP)?
it with UUCP already out there :-) rmail examples, otoh, are readily available.
I am not a total expert on either, but let's just say that email has security
issues and both of those methods do also, particularly relating to spoofing.
>> then develops the ZFS backups over NNCP.
>
> I too *REALLY* like the idea of sending ZFS backups over a store-and-forward
> network to a remote system. Though I see a lot of unknowns there. Do you
> /blindly/ "receive" the "send" directly into a target pool or do you queue it
> as
> a file somewhere first? What if the pool is not in the proper state to
> receive
> what is sent, possibly because of an outstanding send that hasn't been
> received
> yet? Or are there implicit assumptions. -- I'd love to chat more (perhaps
> elsewhere).
I've done both nncp-exec (uux) and nncp-file (uucp), actually. NNCP doesn't
guarantee order of execution of packets. This happens to be fine for zfs
receive, because it will detect when the incremental source makes a packet be
un-receivable and exit with an error. nncp-toss (uuxqt) handles the error exit
code as "try this again on the next run" so it will eventually converge on the
appropriate solution even with a large backlog of packets to process.
I switched to using nncp-file (uucp) with a cron job to process it because I
discovered nncp-exec stores the stdin in RAM on the sending system (Sergey is
planning to fix that shortly). I encode the dataset to receive as part of the
filename (with s,/,@,g basically) and use the date as the leading part of the
filename. So I can minimize, but not eliminate, instances of out-of-order
execution there.
A non-ZFS backup is more complicated and will probably need to use something
like a sequence number to ensure incrementals are applied in the correct order.
An exception may be for setups that just send a full backup every day and dump
them into a deduplicating store (borgbackup, for instance) but that is probably
not super practical for most.
>> I have several more installments planned as well - tuning the ZFS
>> installation
>> and then progressing into non-ZFS backups, which are rather more tricky to
>> get
>> right (both sync and async, but especially async).
>
> I suspect that NNCP has quite similar use cases as UUCP. One just needs to be
> creative in how it's done.
>
> ~chuckle~ Broadcasting NNCP xfer / batch into a Usenet newsgroup for the
> recipient to collect elsewhere.
> I think that some thought, and comments / documentation should probably be
> given
> to why NNCP /needs/ it's own transport. What if NNCP was the next thing to
> use
> uux via rnncp. ;-) Meaning add the authentication on top of the existing
> UUCP
> instead of outright replacing it.
for those that need robust error handling (protocol g, for instance) to run NNCP
over uucp.
nncp-bundle -delete -tx node2 | uux node2!nncp-bundle -rx
would pretty much do it :-)
The NNCP protocol is documented at http://www.nncpgo.org/Sync.html#Sync but I'm
not immediately familiar enough with the low-level details of both it and UUCP's
to have an effective conversation on their relative merits.
I guess I could say that the NNCP protocol is designed to be self-securing and
reasonably delay-tolerant, even on half-duplex links. It also does integrity
checks of the whole of received packets. Of course you can run uucico over SSH,
but SSH is rather difficult to get running over some high-latency links; it
strongly wants a full-duplex link and has handshake timeouts that don't work
well with some of them. NNCP also has a mode where the connection is held open
reasonably indefinitely, and new packets queued for the remote would then be
transmitted essentially immediately.
Some messing with window sizes in UUCP protocols can get some decent performance
out of half-duplex links, but I haven't made a comparison. Of course UUCP has
far better support for noisy lines and modems and things.
> I do think that NNCP is a very interesting idea. I feel like there are a LOT
> of
> unknowns. Almost as if it's a solution in search of a problem. But that may
> just be observation bias.
1) Pervasive encryption
2) First-class support for alternative transports
I started using it, and my feeling was pretty much, "hey! This is UUCP 2.0! I
like this!"
In fact, 19 years ago I wrote a long-forgotten tool called grunt that was
designed to be layered atop uux. grunt permitted both file sending and remote
execution. It used gpg for both encryption and authentication. My last commit
on it, though, was in 2006. I mean, it worked, but it was a bit of a pain and
completely ruled out most of uux's intelligence.
I guess I would put it this way, in a very rough sense: NNCP is to UUCP what
ssh is to rsh.
I feel like neighbor-level transport security in UUCP is solved reasonably
easily (uucico over ssh, etc.) But in the example above, node5 can't really
tell that the packet originated on node1 and arrived at node5 unmodified with
any degree of certainty. NNCP exposes the origin node's public key via an
environment variable so it's all right there. Basically you get end-to-end
encryption with NNCP, as a fundamental part of the protocol. Intermediate hops
can't even tell what the ultimate destination of a packet is, let alone what
command it's going to run.
It also makes some things easier (nncp-xfer/nncp-bundle). But it's not a
drop-in replacement like ssh was for rsh.
So it's not my intent to drop into a UUCP group and go "hey y'all should be
using something else". That would not be cool, and would not be accurate
anyhow. UUCP's still got its place too. I'm just trying to say "hey, there's
another UUCP-ish thing out there now, you might be interested."
- John
Andy Valencia
Jan 4, 2021, 11:01:35 AM1/4/21
to
Grant Taylor <gta...@tnetconsulting.net> writes:
> > NNCP is essentially an effort to modernize UUCP.
> > NNCP is essentially an effort to modernize UUCP.
> That's the impression that I'm getting.
> > First of all is modern security. Every message packet is encrypted
> > and authenticated.
I would like to put on the table the goal of addressing the spam
> > and authenticated.
problem. We didn't deal with it in Usenet, and now we have FB.
We didn't address it in email, and now you can only send/receive
something if gmail says you can.
I've solved the spam problem personally:
http://www.vsta.org/spam/Traveler.html
But for purposes here, tl;dr is add a "X-Authorized-Sender: <token>".
Senders have a token to address you, your side remembers who got what
token. When one is abused, you can turn it off. When you get a new
correspondent, you give them a new token (and remember who got it).
A merchant can hand theirs to, e.g., UPS so you can hear about tracking
of the merchant's shipping.
Andy Valencia
Home page: https://www.vsta.org/andy/
To contact me: https://www.vsta.org/contact/andy.html
John Goerzen
Jan 4, 2021, 12:41:14 PM1/4/21
to
A brief addendum on a couple of your items...
On Sunday, January 3, 2021 at 8:30:56 PM UTC-6, Grant Taylor wrote:
> On 1/3/21 6:06 PM, John Goerzen wrote:
> > Hello folks,
Like several modern radio systems, LoRA is a packetized serial protocol. It offers slightly more promises than UDP: it does not guarantee that every packet will arrive, but it does guarantee that the ones that arrive won't be out-of-order or corrupted.
Another radio system is XBee, and they can offer TCP-like guarantees in hardware. In fact, they have a "serial emulation" mode where they present what looks like a regular serial line with hardware flow control to the system, but internally handle the packetization, collision detection, retransmission, etc. So for XBee, no additional software would be necessary at all to run uucico (or NNCP) over it. For XBee, I wrote xbnet and these instructions for UUCP https://github.com/jgoerzen/xbnet/blob/master/doc/xbnet.1.md#uucp . xbnet allows tunneling IPv4, IPv6, or Ethernet frames across XBee, among other things (also supporting use as a pipe for things like uucico).
> I do think that NNCP is a very interesting idea. I feel like there are
> a LOT of unknowns. Almost as if it's a solution in search of a problem.
> But that may just be observation bias.
I mentioned this thread on the NNCP mailing list. Its author, Sergey Matveev, chimed in with this comment:
"The main difference between UUCP and NNCP, in my opinion, except for
(current) lack of explicit ability to use serial lines, is that NNCP is
a friend-to-friend/darknet network, where each (well, with minor
exceptions of course) participant (on the packet's path) has to be
explicitly known and added to the list of known neighbours. While UUCP
is a greynet, where even "anonymous" (unauthenticated, unidentified)
peers can work.
And no, of course there are no plans for making NNCP opennet, by
automatic fetching of peer's public key via DNS, because... well, it
will simply destroy and ruin the whole network and its resources because
of no trust and control of communicating peers. F2F (darknet)
self-governed networks (FidoNet as an example of global scale world-wide
completely decentralized one) are more complicated to administer and
support, but they are immune to Sybil attacks."
In my opinion, this difference -- while relevant architecturally and for security -- is somewhat less of a practical consideration these days, since the only UUCP installations I am aware of anymore tend to be these small friend-to-friend networks anyhow (due to the demise of the global UUCPNET).
Also there is a new post in my exploration of store-and-forward backups: https://changelog.complete.org/archives/10186-more-topics-on-store-and-forward-possibly-airgapped-zfs-and-non-zfs-backups-with-nncp
- John
On Sunday, January 3, 2021 at 8:30:56 PM UTC-6, Grant Taylor wrote:
> On 1/3/21 6:06 PM, John Goerzen wrote:
> > Hello folks,
> > NNCP requires a clean link and doesn't have any special logic
> > for serial port handling.
> I personally feel like requiring an 8-bit clean connection is probably
> fairly safe these days.
Over at https://changelog.complete.org/archives/10042-long-range-radios-a-perfect-match-for-unix-protocols-from-the-70s I wrote about running UUCP over LoRA radios with success. I wrote lorapipe, which can be used as a transport for uucico. I documented some recommendations for it at https://github.com/jgoerzen/lorapipe/blob/master/doc/lorapipe.1.md#uucp
> > for serial port handling.
> I personally feel like requiring an 8-bit clean connection is probably
> fairly safe these days.
Like several modern radio systems, LoRA is a packetized serial protocol. It offers slightly more promises than UDP: it does not guarantee that every packet will arrive, but it does guarantee that the ones that arrive won't be out-of-order or corrupted.
Another radio system is XBee, and they can offer TCP-like guarantees in hardware. In fact, they have a "serial emulation" mode where they present what looks like a regular serial line with hardware flow control to the system, but internally handle the packetization, collision detection, retransmission, etc. So for XBee, no additional software would be necessary at all to run uucico (or NNCP) over it. For XBee, I wrote xbnet and these instructions for UUCP https://github.com/jgoerzen/xbnet/blob/master/doc/xbnet.1.md#uucp . xbnet allows tunneling IPv4, IPv6, or Ethernet frames across XBee, among other things (also supporting use as a pipe for things like uucico).
> I do think that NNCP is a very interesting idea. I feel like there are
> a LOT of unknowns. Almost as if it's a solution in search of a problem.
> But that may just be observation bias.
"The main difference between UUCP and NNCP, in my opinion, except for
(current) lack of explicit ability to use serial lines, is that NNCP is
a friend-to-friend/darknet network, where each (well, with minor
exceptions of course) participant (on the packet's path) has to be
explicitly known and added to the list of known neighbours. While UUCP
is a greynet, where even "anonymous" (unauthenticated, unidentified)
peers can work.
And no, of course there are no plans for making NNCP opennet, by
automatic fetching of peer's public key via DNS, because... well, it
will simply destroy and ruin the whole network and its resources because
of no trust and control of communicating peers. F2F (darknet)
self-governed networks (FidoNet as an example of global scale world-wide
completely decentralized one) are more complicated to administer and
support, but they are immune to Sybil attacks."
In my opinion, this difference -- while relevant architecturally and for security -- is somewhat less of a practical consideration these days, since the only UUCP installations I am aware of anymore tend to be these small friend-to-friend networks anyhow (due to the demise of the global UUCPNET).
Also there is a new post in my exploration of store-and-forward backups: https://changelog.complete.org/archives/10186-more-topics-on-store-and-forward-possibly-airgapped-zfs-and-non-zfs-backups-with-nncp
- John
Grant Taylor
Jan 4, 2021, 2:42:52 PM1/4/21
to
On 1/3/21 10:51 PM, John Goerzen wrote:
> Hey, the posts are outnumbering the spam (at least on Google Groups
> [yeah, I know, don't shoot me]) so I took that as a positive sign
> for Usenet :-)
;-)
> Hey, the posts are outnumbering the spam (at least on Google Groups
> [yeah, I know, don't shoot me]) so I took that as a positive sign
> for Usenet :-)
> Thank you!
You're welcome.
> So a quick disclaimer that applies to all of my answers here: I'm
> not the author of NNCP, so I can speak to my understanding of it,
> but not necessarily with authority.
> There is a mailing list at
> https://lists.cypherpunks.ru/pipermail/nncp-devel/ and the author is
> active there.
> With that out of the way, yes, that is indeed my understanding.
>
>
> The NNCP usage of the word "daemon" is a little loose.
Okay.... I'm using "daemon" to mean a process that is running all the
time, independent of what it does.
> So in UUCP, there is uucico, which is both the outbound caller and
> the inbound listener/daemon.
ports 540 / 4031.
All of the UUCP that I've done has been initiated via a cron job (thus
the cron daemon) and / or login shells and / or specifically executed
commands. Meaning that in an idle state, there was no daemon (running
process) specifically for UUCP.
> In NNCP, these roles are split into three separate programs:
>
> nncp-daemon - the listener (roughly uucico -l)
incoming TCP connections.
> nncp-call - an on-demand caller (uucico -S)
>
> nncp-caller - the caller with scheduling rules (uucico -s if memory
> serves?)
>
> The "daemon" can be run as a daemon (I run it under systemd [again,
> don't shoot me!] quite easily). But it also has an -inetd option
> that, of course, can run under inetd... but is really just a "speak
> the protocol on stdin/stdout" so it could just as easily be run under
> ssh as with uucico.
The daemon / (x)inetd aspect is just used to convert incoming TCP
connections to STDIN / STDOUT to / from the NNCP processes on the local
system.
> I think that would be an accurate characterization, yes. I mean,
> both programs fundamentally store packets in regular files, so as
> long as you are careful not to trip over race conditions in either
> direction, it should be reasonably simple.
> (I wonder if sequence numbers make it at all more complicated on the
> UUCP side?
more for the call. -- Seeing as how we're talking about moving files
between queues, which is independent of and after the call, I doubt that
they would be much of an issue. But it wouldn't be the first time that
I'm wrong.
> This is one article about doing something similar with UUCP:
> https://www.dumain.com/posts/Forward_to_the_1970s_with_UUCP.../
> and it wasn't super simple. TBH I've never tried it.
> The NNCP tools for this also have built-in support for grades/nice,
> preventing duplicate packets (I can't remember if UUCP does this
> anymore; I seem to recall not?).
recovery, was done as part of the line protocol. As such, I think it's
somewhat outside of UUCP's queued store-and-forward mechanism.
> Generally yes, but I play with long-distance low-power radios (think
> 20 miles with 500mW or less) so it isn't ALWAYS that way even now!
Aside: I'd like to learn more about what you're doing. Please reply
with where I can learn more. Do you have a blog, Twitter, etc?
(Perhaps direct email is better.)
> True, though TBH I don't think anybody has tried NNCP over raw serial
> lines yet. Must put that on my list...
> So I made an inartful simplification there.
>
> One could, in both UUCP and NNCP worlds, handle multi-hop mail routing
> by either:
>
> 1) Having the MTA do it (presumably via a smarthost for leaf nodes)
> - in a manner similar to how things happen over the Internet now
>
> 2) Doing this at the UUCP/NNCP protocol level
>
> So basically if you are going to have the MTA on node1 execute
> something like "nncp-exec -via node2,node3,node4 node5 rmail janedoe"
> (aka uux node2!node3!node4!node5!rmail janedoe) then the sending node
> needs to have the keys for all four of those nodes since it needs to
> prepare all of the onion encryption layers.
>
> If, on the other hand, node1 knows that "meh, I just send all my mail
> to node2 and it figures it out", then it would be doing "nncp-exec
> node2 rmail janedoe@node5" or something similar; node2 presumably
> would know to route that via node3 and so forth.
You can either do more of the work at the email layer, with direct
node-to-node email routing. Or you can do more of the work at the
transport layer, with node-to-node(-to-node) transport routing.
Method A [email] [email] [email] [email] [email]
[node1]---[node2]---[node3]---[node4]---[node5]
vs
Method B [email] [email]
[node1]---[node2]---[node3]---[node4]---[node5]
I'm eliding the smart host aspect because that would likely be the same
conceptual configuration of originating systems and first hop in either
system.
> I have never set up an MTA with full bang-path routing (my UUCP
> uses were all late enough to be essentially Internet leaf sites with
> RFC822-style addressing routed over UUCP) so TBH I don't quite know
> how the MTAs handled that, but my understanding is that is was pretty
> much on the user to know the full bang path from the local system to
> the recipient and it just handed this over to uux.
The experience I have with multi-node (bang) paths has more to do with
file copy (uucp / uuto) and command execution (uux).
From a purely efficiency point of view, it seems like it would be
better to have fewer MTA interactions with longer bang paths between them.
But none of that speaks to what a downstream MTA does with a message
that it won't accept.
> NNCP's config file actually lets you specify a default route to a
> known node, so there would be no need for explicit routing at the
> MTA level if you just do it all at the NNCP level.
such a default configuration on an MTA. Especially in idea of not being
an open relay. -- I guess that the MTA on the NNCP default could list
source nodes that it will allow relaying for.
As I think about it, by the time that rmail is invoked, it's too late to
actually reject a message and force the sending MTA to deal with it.
Instead, the receiving MTA will likely /need/ to /receive/ and then /
reject/ (bounce) the message. Lest messages make it to the receiving
MTA to only be dropped. Such drop behavior is antithetical to proper
operation of email.
> Doing it at the NNCP level would have a security benefit as well,
> since it would be neither necessary nor possible to decrypt it at
> MTAs along the way.
> Exim has an example config where you could send things to, eg,
> jan...@node5.UUCP and it could have a routing lookup table --
> or just pass this to the lower level.
to get to the next MTA in line.
> With NNCP's default "via" settings available in the config, the
> routing can be invisible to both the MTA and the user (but of course,
> the keys are needed).
any knowledge of the necessary public keys to make up the onion? Or is
that all handled by the NNCP transport layer? -- Much like the user
does not need to know the particulars about IPsec transport between two
systems as those details are handled at a lower layer and hidden from
the user.
> If the MTA were to call nncp-exec to a node that nncp-exec doesn't
> know about, it would get an immediate command failure from the pipe
> to nncp-exec and that would presumably turn into an immediate DSN.
> The difference is that with uux, the local system would only need
> to know about node2 and if, say, node3 doesn't know about node4,
> then that's an issue that surfaces later. (Technically this could
> still happen in NNCP; you could know about all those nodes but then
> node3 might not know about node4, but I'd think it would be more rare)
Method B above. The failures of each is different.
The MTAs of Method A would have the hard and fast error that the
{UUCP,NNCP} transport can't do it's job because of an unknown route.
The MTAs of Method B would not have a hard and fast error. They would
be dependent on some sort of {UUCP,NNCP} error message coming back and
being interpreted correctly.
Direct, single, and multiple intermediate systems (direct!user /
direct!single!user / direct!single!multiple!user) would probably all
have different types of errors.
> No. It would probably take an NNCP mapping project for this.
> Also no.
Something to facilitate this might be interesting.
> They are essentially whatever you want them to be. Unlike with UUCP,
> an NNCP node name is essentially a convenience alias for the node's
> public key (akin to a hostname being a convenience alias for an IP
> address).
> A node doesn't have to be known by the same name on every host,
> either, since the spool directories and all internal structures are
> based off the node's public key rather than its name.
cause routing loops.
Though, they may be able to be used to avoid a situation that I have yet
to find a solution for with UUCP. Node 2 going offline and never
returning in a node1!node2!node3!user situation. How does node1 manage
to re-route traffic to node3 which is online? This lack of redundancy /
re-routability has long bothered me.
> I don't think so. NNCP was designed primarily as a sort of private
> network of nodes - which is what I've most commonly seen UUCP evolved
> into (as there is no longer any global UUCP network) - but there is
> no particular reason it couldn't evolve in that direction.
> This makes me happy somehow :-)
> Fair nitpick, yes.
:-)
> To be sure, I did bsmtp in that example because there weren't examples
> of doing it with UUCP already out there :-) rmail examples, otoh,
> are readily available.
> I am not a total expert on either, but let's just say that email
> has security issues and both of those methods do also, particularly
> relating to spoofing.
doesn't have any knowledge of the upstream source, save for the previous
hop that sent the current packet / message / datagram / nomenclature?
So we are still reliant on things like DKIM to impose any form of
message authentication.
> Do I blindly receive? Pretty much yes :-)
loose mail.
> I've done both nncp-exec (uux) and nncp-file (uucp), actually. NNCP
> doesn't guarantee order of execution of packets. This happens to be
> fine for zfs receive, because it will detect when the incremental
> source makes a packet be un-receivable and exit with an error.
> nncp-toss (uuxqt) handles the error exit code as "try this again
> on the next run" so it will eventually converge on the appropriate
> solution even with a large backlog of packets to process.
snapshot that the subsequent snapshot was based on.
Though I guess this is a higher ZFS application layer problem and not
something that needs to be handled by {UUCP,NNCP} directly.
Layering can be tricky.
> I switched to using nncp-file (uucp) with a cron job to process it
> because I discovered nncp-exec stores the stdin in RAM on the sending
> system (Sergey is planning to fix that shortly).
> I encode the dataset to receive as part of the filename (with s,/,@,g
> basically) and use the date as the leading part of the filename. So I
> can minimize, but not eliminate, instances of out-of-order execution
> there.
Encode ZFS application layer details in the file name that's copied from
source to destination by the underlying transport, be it UUCP or NNCP.
> A non-ZFS backup is more complicated and will probably need to use
> something like a sequence number to ensure incrementals are applied
> in the correct order. An exception may be for setups that just send
> a full backup every day and dump them into a deduplicating store
> (borgbackup, for instance) but that is probably not super practical
> for most.
> Hey, there've been worse uses of Usenet :-)
> Oh I actually pointed out on the mailing list that it would be quite
> possible for those that need robust error handling (protocol g,
> for instance) to run NNCP over uucp.
> nncp-bundle -delete -tx node2 | uux node2!nncp-bundle -rx
>
> would pretty much do it :-)
> The NNCP protocol is documented at http://www.nncpgo.org/Sync.html#Sync
> but I'm not immediately familiar enough with the low-level details
> of both it and UUCP's to have an effective conversation on their
> relative merits.
> I guess I could say that the NNCP protocol is designed to be
> self-securing and reasonably delay-tolerant, even on half-duplex links.
> It also does integrity checks of the whole of received packets. Of
> course you can run uucico over SSH, but SSH is rather difficult to get
> running over some high-latency links; it strongly wants a full-duplex
> link and has handshake timeouts that don't work well with some of them.
with.
Is NNCP truly half duplex capable when it uses nncp-{bundle,batch}? Or
is that more a gateway of sorts to an intermediate format?
> NNCP also has a mode where the connection is held open reasonably
> indefinitely, and new packets queued for the remote would then be
> transmitted essentially immediately.
> Some messing with window sizes in UUCP protocols can get some decent
> performance out of half-duplex links, but I haven't made a comparison.
> Of course UUCP has far better support for noisy lines and modems
> and things.
> So the things that drew me to NNCP are:
>
> 1) Pervasive encryption
pervasive means opportunistic to me. IMHO such would require something
like the federation and auto-discovery like I was talking about.
> 2) First-class support for alternative transports
> I started using it, and my feeling was pretty much, "hey! This is
> UUCP 2.0! I like this!"
> In fact, 19 years ago I wrote a long-forgotten tool called grunt
> that was designed to be layered atop uux. grunt permitted both file
> sending and remote execution. It used gpg for both encryption and
> authentication. My last commit on it, though, was in 2006. I mean,
> it worked, but it was a bit of a pain and completely ruled out most
> of uux's intelligence.
>
> I guess I would put it this way, in a very rough sense: NNCP is to
> UUCP what ssh is to rsh.
> I feel like neighbor-level transport security in UUCP is solved
> reasonably easily (uucico over ssh, etc.) But in the example above,
> node5 can't really tell that the packet originated on node1 and arrived
> at node5 unmodified with any degree of certainty.
how this is any different with NNCP vs UUCP.
Note: I'm assuming that:
node5 only sees correct packets as being from node4.
node4 only sees correct packets as being from node3 and to node5.
node3 only sees correct packets as being from node2 and to node4.
node2 only sees correct packets as being from node1 and to node3.
node1 generates the node2(node3(node4(node5))) onion and sends it node2.
IMHO node5 has no idea where the packet / onion originated from, just a
path back to the unknown source.
> NNCP exposes the origin node's public key via an environment variable
> so it's all right there.
Or do I have an incorrect understanding of onion routing?
> Basically you get end-to-end encryption with NNCP, as a fundamental
> part of the protocol. Intermediate hops can't even tell what the
> ultimate destination of a packet is, let alone what command it's
> going to run.
> It also makes some things easier (nncp-xfer/nncp-bundle). But it's
> not a drop-in replacement like ssh was for rsh.
>
> So it's not my intent to drop into a UUCP group and go "hey y'all
> should be using something else". That would not be cool, and would
> not be accurate anyhow. UUCP's still got its place too. I'm just
> trying to say "hey, there's another UUCP-ish thing out there now,
> you might be interested."
That's how I took your message.
John Levine
Jan 4, 2021, 2:52:53 PM1/4/21
to
In article <160977571380.16164....@media.vsta.org>,
a year ago. See patent 6,643,686.
It's also a Well Known Bad Idea since it just replaces the intractable
spam problem with the equally intractable introduction problem.
--
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly
Andy Valencia <van...@vsta.org> wrote:
> http://www.vsta.org/spam/Traveler.html
>
>But for purposes here, tl;dr is add a "X-Authorized-Sender: <token>".
>Senders have a token to address you, your side remembers who got what
>token. When one is abused, you can turn it off. When you get a new
>correspondent, you give them a new token (and remember who got it).
>A merchant can hand theirs to, e.g., UPS so you can hear about tracking
>of the merchant's shipping.
That's called zoemail. It was patented so long ago that the patent expired
> http://www.vsta.org/spam/Traveler.html
>
>But for purposes here, tl;dr is add a "X-Authorized-Sender: <token>".
>Senders have a token to address you, your side remembers who got what
>token. When one is abused, you can turn it off. When you get a new
>correspondent, you give them a new token (and remember who got it).
>A merchant can hand theirs to, e.g., UPS so you can hear about tracking
>of the merchant's shipping.
a year ago. See patent 6,643,686.
It's also a Well Known Bad Idea since it just replaces the intractable
spam problem with the equally intractable introduction problem.
--
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly
Grant Taylor
Jan 4, 2021, 2:59:55 PM1/4/21
to
On 1/4/21 10:41 AM, John Goerzen wrote:
> A brief addendum on a couple of your items...
*nod*
> A brief addendum on a couple of your items...
> Over at
> https://changelog.complete.org/archives/10042-long-range-radios-a-perfect-match-for-unix-protocols-from-the-70s
> I wrote about running UUCP over LoRA radios with success.
> I wrote lorapipe, which can be used as a transport for
> uucico. I documented some recommendations for it at
> https://github.com/jgoerzen/lorapipe/blob/master/doc/lorapipe.1.md#uucp
> Like several modern radio systems, LoRA is a packetized serial
> protocol. It offers slightly more promises than UDP: it does not
> guarantee that every packet will arrive, but it does guarantee that
> the ones that arrive won't be out-of-order or corrupted.
I'm guessing that there is some sort of counter and only newer packets
are relayed from the radio to the other interface.
> Another radio system is XBee, and they can offer TCP-like guarantees
> in hardware. In fact, they have a "serial emulation" mode where they
> present what looks like a regular serial line with hardware flow
> control to the system, but internally handle the packetization,
> collision detection, retransmission, etc. So for XBee, no
> additional software would be necessary at all to run uucico (or NNCP)
> over it. For XBee, I wrote xbnet and these instructions for UUCP
> https://github.com/jgoerzen/xbnet/blob/master/doc/xbnet.1.md#uucp .
> xbnet allows tunneling IPv4, IPv6, or Ethernet frames across XBee,
> among other things (also supporting use as a pipe for things like
> uucico).
> I mentioned this thread on the NNCP mailing list. Its author, Sergey
> Matveev, chimed in with this comment:
>
> "The main difference between UUCP and NNCP, in my opinion, except for
> (current) lack of explicit ability to use serial lines, is that NNCP
> is a friend-to-friend/darknet network, where each (well, with minor
> exceptions of course) participant (on the packet's path) has to be
> explicitly known and added to the list of known neighbours. While UUCP
> is a greynet, where even "anonymous" (unauthenticated, unidentified)
> peers can work.
I've configured my UUCP-over-SSH network(s) (a few different ones over
the years) are both closed /and/ only relay packets (?) between
explicitly approved pairs of systems. So, even if someone did manage to
get into the UUCP network (independent of SSH) their packets wouldn't
pass without spoofing the source and / or destination. I think the
sequence number does make this more difficult. Save for injecting files
into UUCP queue directly.
So, I don't think that Sergey's statement is wrong per se, but I don't
think it's 100% accurate either.
> And no, of course there are no plans for making NNCP opennet, by
> automatic fetching of peer's public key via DNS, because... well,
> it will simply destroy and ruin the whole network and its resources
> because of no trust and control of communicating peers.
configure UUCP nodes to only allow packets from specified sources. As
such, I would expect that NNCP would have such a restriction too. Plus,
seeing as how things are cryptographically verifiable, the source can be
validated. Anything that doesn't pass can be rejected.
Thus, I don't think that having NNCP be open net capable would detract
from it's operation in any way, shape, or form.
> F2F (darknet) self-governed networks (FidoNet as an example of global
> scale world-wide completely decentralized one) are more complicated
> to administer and support, but they are immune to Sybil attacks."
incoming connection / packet is completely within the receiving
administrator's control. I don't see how any external information can
influence that decision.
> In my opinion, this difference -- while relevant architecturally and
> for security -- is somewhat less of a practical consideration these
> days, since the only UUCP installations I am aware of anymore tend
> to be these small friend-to-friend networks anyhow (due to the demise
> of the global UUCPNET).
>
> Also there is a new post in my
> exploration of store-and-forward backups:
> https://changelog.complete.org/archives/10186-more-topics-on-store-and-forward-possibly-airgapped-zfs-and-non-zfs-backups-with-nncp
Andy Valencia
Jan 4, 2021, 5:05:25 PM1/4/21
to
John Levine <jo...@taugh.com> writes:
> That's called zoemail. It was patented so long ago that the patent expired
> a year ago. See patent 6,643,686.
>
> It's also a Well Known Bad Idea since it just replaces the intractable
> spam problem with the equally intractable introduction problem.
This isn't alt.spam.argue.argue.argue, so I'll just say that I've
> That's called zoemail. It was patented so long ago that the patent expired
> a year ago. See patent 6,643,686.
>
> It's also a Well Known Bad Idea since it just replaces the intractable
> spam problem with the equally intractable introduction problem.
lived with the technique by overloading the email address with the
token. And it is much more effective than you might expect.
John Levine
Jan 4, 2021, 5:53:44 PM1/4/21
to
In article <160979781021.23695....@media.vsta.org>,
handy for telling who leaks their addresses, but they are far from a
FUSSP.
Andy Valencia <van...@vsta.org> wrote:
>John Levine <jo...@taugh.com> writes:
>> That's called zoemail. It was patented so long ago that the patent expired
>> a year ago. See patent 6,643,686.
>>
>> It's also a Well Known Bad Idea since it just replaces the intractable
>> spam problem with the equally intractable introduction problem.
>
>This isn't alt.spam.argue.argue.argue, so I'll just say that I've
>lived with the technique by overloading the email address with the
>token. And it is much more effective than you might expect.
I've probably been using tagged addresses as long as anyone. They are
>John Levine <jo...@taugh.com> writes:
>> That's called zoemail. It was patented so long ago that the patent expired
>> a year ago. See patent 6,643,686.
>>
>> It's also a Well Known Bad Idea since it just replaces the intractable
>> spam problem with the equally intractable introduction problem.
>
>This isn't alt.spam.argue.argue.argue, so I'll just say that I've
>lived with the technique by overloading the email address with the
>token. And it is much more effective than you might expect.
handy for telling who leaks their addresses, but they are far from a
FUSSP.
John Goerzen
Jan 4, 2021, 5:55:21 PM1/4/21
to
On Mon, Jan 04 2021, Grant Taylor wrote:
> On 1/4/21 10:41 AM, John Goerzen wrote:
>> Like several modern radio systems, LoRA is a packetized serial protocol. It
>> offers slightly more promises than UDP: it does not guarantee that every
>> packet will arrive, but it does guarantee that the ones that arrive won't be
>> out-of-order or corrupted.
>
> Intriguing.
>
> I'm guessing that there is some sort of counter and only newer packets are
> relayed from the radio to the other interface.
Oh it is more mundane than that :-) If a packet fails to decode correctly, it
> On 1/4/21 10:41 AM, John Goerzen wrote:
>> Like several modern radio systems, LoRA is a packetized serial protocol. It
>> offers slightly more promises than UDP: it does not guarantee that every
>> packet will arrive, but it does guarantee that the ones that arrive won't be
>> out-of-order or corrupted.
>
> Intriguing.
>
> I'm guessing that there is some sort of counter and only newer packets are
> relayed from the radio to the other interface.
is simply dropped on the floor. There is no retransmit in LoRA, there are no
routers, it is just basically a bunch of radios in a single broadcast domain
with iffy collision detection.
That said, LoRA radios can achieve absolutely incredible range with tiny amounts
of power, especially when you get down into the <1000bps range. I mean, 10+
miles with like 70mW! Tradeoffs abound but they are used a lot for remote
sensing and control applications where the batteries last for years.
There is LoRAWAN, a layer atop LoRA that uses synchronized timeslots to minimize
collisions, has limited ARQ, etc. It is not really suitable for many uses
because proper LoRAWAN "gateway" nodes tend to be quite expensive, and they use
widely asymmetric uplink vs. downlink speeds, and would tend to require one to
tunnel UUCP over JSON. I'm willing to tunnel UUCP over a lot of things but that
one is a bridge too far :-)
>> Another radio system is XBee, and they can offer TCP-like guarantees in
>> hardware. In fact, they have a "serial emulation" mode where they present
>> what looks like a regular serial line with hardware flow control to the
>> system, but internally handle the packetization, collision detection,
>> retransmission, etc. So for XBee, no additional software would be necessary
supports encryption over the air. It also runs at higher speeds than LoRA,
maxing out at roughly 50-100Kbps effective bitrate. The range is somewhat less
than LoRA, but I have had success with a 10-mile XBee link at about 50Kbps.
There are a bunch of different XBee modules that use different wire protocols
and frequencies; the longest range are the SX modules. They can also operate
over Zigbee, Wifi, custom 2.4GHz mesh (non-Wifi), which have varying speeds and
such - but with a single serial interface to the controller.
I've not specifically replied to your responses to Sergey, on the grounds that I
didn't write it :-) You make good points but thinking about both perspectives
requires a quantity of thought I haven't had the time for today. Not trying to
ignore it, just being honest here :-)
- John
John Goerzen
Jan 4, 2021, 6:00:50 PM1/4/21
to
On Mon, Jan 04 2021, Grant Taylor wrote:
> On 1/3/21 10:51 PM, John Goerzen wrote:
> All of the UUCP that I've done has been initiated via a cron job (thus the
> cron
> daemon) and / or login shells and / or specifically executed commands.
> Meaning
> that in an idle state, there was no daemon (running process) specifically for
> UUCP.
ACK. The same is possible with the "nncp-daemon".
> All of the UUCP that I've done has been initiated via a cron job (thus the
> cron
> daemon) and / or login shells and / or specifically executed commands.
> Meaning
> that in an idle state, there was no daemon (running process) specifically for
> UUCP.
> Aside: I'd like to learn more about what you're doing. Please reply with
> where
> I can learn more. Do you have a blog, Twitter, etc? (Perhaps direct email is
> better.)
https://changelog.complete.org/ and has my radio experiments. I linked you one
post but if you search for LoRA and Xbee there, you'll turn up others. I also
have been meaning to set up a Meshtastic device soon (Meshtastic is a
low-powered long-distance IM mesh that uses LoRA under the hood; ideal for
off-grid communication or communication in censored areas). I'm also
@jgoe...@floss.social on Mastodon (I am trying to ease out of the centralized
social media companies and into the decentralized ones). Also you're welcome
to email.
> This is effectively the difference of where you put the bulk of the logic.
> You can either do more of the work at the email layer, with direct
> node-to-node
> email routing. Or you can do more of the work at the transport layer, with
> node-to-node(-to-node) transport routing.
>> NNCP's config file actually lets you specify a default route to a known node,
>> so there would be no need for explicit routing at the MTA level if you just
>> do
>> it all at the NNCP level.
>
> Hum. I'd have to spend some time thinking about the implications of such a
> default configuration on an MTA. Especially in idea of not being an open
> relay.
> -- I guess that the MTA on the NNCP default could list source nodes that it
> will allow relaying for.
is allowed to execute. If you don't want to accept mail from a given node with
either system, the only really safe thing is to not give it access to rmail in
the first place.
> As I think about it, by the time that rmail is invoked, it's too late to
> actually reject a message and force the sending MTA to deal with it. Instead,
> the receiving MTA will likely /need/ to /receive/ and then / reject/ (bounce)
> the message. Lest messages make it to the receiving MTA to only be dropped.
> Such drop behavior is antithetical to proper operation of email.
three hops down the line and then your destination has denied permission to run
rmail, for instance). ISTR uuxqt can generate an error report at this point. I
would need to look into NNCP's behavior here; I believe it holds the packet in
the incoming queue thinking that the failure may eventually get resolved.
Probably not ideal in this situation.
>> With NNCP's default "via" settings available in the config, the routing can
>> be
>> invisible to both the MTA and the user (but of course, the keys are needed).
>
> Now I have to stop and ask: Does the user (human or MTA or other) have any
> knowledge of the necessary public keys to make up the onion? Or is that all
> handled by the NNCP transport layer? -- Much like the user does not need to
> know the particulars about IPsec transport between two systems as those
> details
> are handled at a lower layer and hidden from the user.
nodename aliases, both for input from users and for display purposes -- though I
believe you can also hand it raw public keys if you like.
That said, if you have read access to the spool directory, everything there is
by public key so you can certainly access things at that level if you feel the
need. But nncp-stat and friends will show the human-friendly aliases instead.
>> A node doesn't have to be known by the same name on every host, either, since
>> the spool directories and all internal structures are based off the node's
>> public key rather than its name.
>
> IMHO aliases / a.k.a.s / nicknames can complicate things and possibly cause
> routing loops.
lower-level protocol uses the aliases at all.
> Though, they may be able to be used to avoid a situation that I have yet to
> find
> a solution for with UUCP. Node 2 going offline and never returning in a
> node1!node2!node3!user situation. How does node1 manage to re-route traffic
> to
> node3 which is online? This lack of redundancy / re-routability has long
> bothered me.
NNCP is going to be worse than UUCP at this, because the packets in that
situation will have been encrypted to node2 along their way to node3, and unless
you have node2's private keys, will become useless if node2 goes away.
With UUCP, you could presumably do some ugly munging and get what you want in
the end.
> I guess do to the nature of onion routing, the down stream system doesn't have
> any knowledge of the upstream source, save for the previous hop that sent the
> current packet / message / datagram / nomenclature?
source but not the hops it took to get there. More on that below.
>> Do I blindly receive? Pretty much yes :-)
>
> See above. I now realize that it's blindly receive and then bounce -or-
> loose mail.
conversation here, you're not wrong.
>> un-receivable and exit with an error. nncp-toss (uuxqt) handles the error
>> exit
>> code as "try this again on the next run" so it will eventually converge on
>> the
>> appropriate solution even with a large backlog of packets to process.
>
> I was thinking where an intermediate snapshot was lost. You know, the
> snapshot
> that the subsequent snapshot was based on.
zfs receive an incremental based on it, ever. However, if the source snapshot
just hasn't been received yet, then the zfs receive based on the source will
fail initially, but when NNCP retries it later after the source has been
received, it will succeed.
>> I guess I could say that the NNCP protocol is designed to be self-securing
>> and
>> reasonably delay-tolerant, even on half-duplex links. It also does integrity
>> checks of the whole of received packets. Of course you can run uucico over
>> SSH, but SSH is rather difficult to get running over some high-latency links;
>> it strongly wants a full-duplex link and has handshake timeouts that don't
>> work well with some of them.
>
> I wonder how well TCP will do over the links that SSH is less than happy with.
So almost all radios are half-duplex at heart. Many modern digital ones
simulate full duplex operation by very rapid switching, but the lower-speed ones
won't.
So I had to add a whole algorithm to lorapipe to deal with ssh, because part of
ssh's handshake has both the client and the server send data simultaneously. A
sensible performance optimization in general but terrible for an RF link with
virtually nonexistent collision detection. By the time packets would get
retransmitted, ssh would have timed out and aborted. I had to add a very simple
feature to lorapipe: a "more data is coming after this packet" bit that
facilitated turn-taking, allowing ssh handshake to proceed.
That's not to say that TCP with its ACKs was great over LoRA with my initial
algorithm, just that SSH was particularly pathological.
But of course, with either UUCP or NNCP, one doesn't actually have to run TCP
(in NNCP's case, you do need a reliable link, but it need not be a TCP one).
> Is NNCP truly half duplex capable when it uses nncp-{bundle,batch}? Or is
> that
> more a gateway of sorts to an intermediate format?
in the outgoing spool directory. nncp-bundle -rx does the reverse, processing
packets in the tarball destined for the local node (and ignoring everything
else). That's why you can pipe nncp-bundle to uux. The NNCP wire protocol
itself is, I think, designed to be half-duplex but obviously there is going to
be some back-and-forth on any wire protocol.
nncp-xfer moves packets to a directory structure at an arbitrary point (usually
a USB drive or something).
>> I feel like neighbor-level transport security in UUCP is solved reasonably
>> easily (uucico over ssh, etc.) But in the example above, node5 can't really
>> tell that the packet originated on node1 and arrived at node5 unmodified with
>> any degree of certainty.
>
> Maybe it's my lack of understanding of onion routing, but I don't see how this
> is any different with NNCP vs UUCP.
>
> Note: I'm assuming that:
>
> node5 only sees correct packets as being from node4.
> node4 only sees correct packets as being from node3 and to node5.
> node3 only sees correct packets as being from node2 and to node4.
> node2 only sees correct packets as being from node1 and to node3.
> node1 generates the node2(node3(node4(node5))) onion and sends it node2.
>
> IMHO node5 has no idea where the packet / onion originated from, just a path
> back to the unknown source.
using node5's public key, and that packet is signed by node1. Next, it takes
that whole thing, encryption and signature and all, and uses it as input for an
encrypted packet to node4 -- that packet itself also signed by node1. It
repeats this process until it generates the packet to node2.
Think of it like this:
node1$ gpg --sign -e -r node5 < datafile > node5.gpg
node1$ gpg --sign -e -r node4 < node5.gpg > node4.gpg
node1$ gpg --sign -e -r node3 < node4.gpg > node3.gpg
node1$ gpg --sign -e -r node2 < node3.gpg > outbound-to-node2.gpg
node1$ rm node[345.jpg]
Node2 receives the packet - encrypted only to it, and signed by node1 - and
decrypts the payload. The payload is a command (send this on to node2) and an
encrypted payload for THAT command -
encrypted to node2 and signed by node1. This process also continues until you
get to node5.
So, at node4, node4 receives its packet from node3 -- signed by node1, encrypted
to node4. Node4 decrypts it and finds a payload for node5 -- again, signed by
node1, encrypted to node5. It places this in the queue for node5. (Just like
with node2)
Node5 receives the packet. It verifies the signature from node1, and it is the
only node that can decrypt it. nncp-toss finds the packet in the rx queue from
node4, but it doesn't really pass this information on anywhere; it knows from
the signature that the origin was node1 (doesn't much care how it arrived) and
that single verified signature is what is used for all processing (looking up
whether node1 is authorized to run a command, exposed as an environment variable
to nncp-exec runners, used to determine the incoming directory for nncp-file,
etc).
Node5 could sort of know that the packet arrived from node4, but there is no
cryptographic guarantee of that (the ONLY signatures in this whole chain come
from node1) so that information is basically discarded.
This is different from the UUCP model, in which node5 only really knows about
its neighbor node4 and may have no idea whatever about node1's existence.
Permissions in UUCP are based on what the neighbor can do, not what the source
can do, if I remember correctly.
The other thing that follows from this is that if node3 were compromised, the
worst it can do is fail to relay packets. It cannot modify packets (signature
validation from node1 would fail), it cannot spoof node1 (lacking its private
key), it can't inject fake packets claiming to have followed the
node1!node2!node3 path (again, lacking node1's private key), it can't even see
what the ultimate destination is (it just knows the next hop is node4 but since
it doesn't have node4's private key, it doesn't even know if node4 is the
ultimate hop or not).
I really like this.
>> NNCP exposes the origin node's public key via an environment variable so it's
>> all right there.
>
> Does the original node add additional identifying information to the packet?
>
> Or do I have an incorrect understanding of onion routing?
Yes, all packets in the chain are signed by the origin and only the origin.
- John
Grant Taylor
Jan 4, 2021, 9:05:56 PM1/4/21
to
On 1/4/21 4:00 PM, John Goerzen wrote:
> ACK. The same is possible with the "nncp-daemon".
>
> ACK. The same is possible with the "nncp-daemon".
>
> I seem to have anticipated this :-) The blog is at
> https://changelog.complete.org/ and has my radio experiments. I linked
> you one post but if you search for LoRA and Xbee there, you'll turn up
> others.
Thank you for the link. I will be consuming more of your articles.
> https://changelog.complete.org/ and has my radio experiments. I linked
> you one post but if you search for LoRA and Xbee there, you'll turn up
> others.
> I also have been meaning to set up a Meshtastic device soon (Meshtastic
> is a low-powered long-distance IM mesh that uses LoRA under the hood;
> ideal for off-grid communication or communication in censored areas).
> I'm also @jgoe...@floss.social on Mastodon (I am trying to ease out
> of the centralized social media companies and into the decentralized
> ones). Also you're welcome to email.
> NNCP, as with UUCP, will let you specify what (if any) commands a
> given remote is allowed to execute. If you don't want to accept mail
> from a given node with either system, the only really safe thing is
> to not give it access to rmail in the first place.
I would also hope that there is a way to cause NNCP (UUCP) to simply
drop anything that wasn't from an explicitly allowed host, independent
of how it got into the system.
Preferably using anything not explicitly allowed as an Indicator of
Compromise.
> Yes, though in reality I could see it happening with both UUCP and NNCP
> (you get three hops down the line and then your destination has denied
> permission to run rmail, for instance).
Though, hopefully such a path would be tested before put into
production. Thus meaning that this type of failure is less likely to
occur. I guess it depends if the path is user provided or admin provided.
> ISTR uuxqt can generate an error report at this point. I would need to
> look into NNCP's behavior here; I believe it holds the packet in the
> incoming queue thinking that the failure may eventually get resolved.
> Probably not ideal in this situation.
where rmail was prohibited.
> It's all handled at the transport layer. All the NNCP commands work
> with the nodename aliases, both for input from users and for display
> purposes -- though I believe you can also hand it raw public keys if
> you like.
> That said, if you have read access to the spool directory, everything
> there is by public key so you can certainly access things at that
> level if you feel the need. But nncp-stat and friends will show the
> human-friendly aliases instead.
This brings to mind another short coming I've seen with UUCP.
Inter-user access on the same system.
Does NNCP even extend the encryption to the user level on the system?
E.g. destination!user1 vs destination!user2. As in if files were sent
something like uuto /path/to/local-file destination!user3.
The UUCP installations that I've been exposed to were largely if you can
access the UUCP spool, you could access just about anything in it.
Perhaps this was a misconfiguration on those systems.
> I don't *think* so, since it is source-routed just like UUCP, and
> none of the lower-level protocol uses the aliases at all.
> So I have actually brought a situation like this up on the NNCP list.
> Actually NNCP is going to be worse than UUCP at this, because the
> packets in that situation will have been encrypted to node2 along
> their way to node3, and unless you have node2's private keys, will
> become useless if node2 goes away.
> With UUCP, you could presumably do some ugly munging and get what
> you want in the end.
> Not exactly; the downstream system actually has conclusive knowledge of
> the source but not the hops it took to get there. More on that below.
I feel like I should go back and re-read about ToR.
I wonder if I'm incorrect about it too or of NNCP's behavior diverges at
this point.
> I meant that in the context of ZFS receive actually. But if I'm
> following the conversation here, you're not wrong.
> If the origin snapshot is just lost to thin air, then it will not be
> possible to zfs receive an incremental based on it, ever. However,
> if the source snapshot just hasn't been received yet, then the zfs
> receive based on the source will fail initially, but when NNCP retries
> it later after the source has been received, it will succeed.
the previous snapshot and loosing one snapshot along the line. I'd
think that all subsequent snapshots would be unusable.
> Better. I actually have experience with this.
> So almost all radios are half-duplex at heart. Many modern digital
> ones simulate full duplex operation by very rapid switching, but the
> lower-speed ones won't.
>
> So I had to add a whole algorithm to lorapipe to deal with ssh, because
> part of ssh's handshake has both the client and the server send data
> simultaneously. A sensible performance optimization in general but
> terrible for an RF link with virtually nonexistent collision detection.
> By the time packets would get retransmitted, ssh would have timed out
> and aborted. I had to add a very simple feature to lorapipe: a "more
> data is coming after this packet" bit that facilitated turn-taking,
> allowing ssh handshake to proceed.
> That's not to say that TCP with its ACKs was great over LoRA with my
> initial algorithm, just that SSH was particularly pathological.
It also brings to mind some of the optimization methods that have been
tried over satellite networks. E.g. ""terminate the TCP at a proxy that
spoofs remote IPs, send the data as a non-TCP stream, and then originate
a new TCP connection from the remote proxy spoofing local IPs to the
remote end point.
> But of course, with either UUCP or NNCP, one doesn't actually have
> to run TCP (in NNCP's case, you do need a reliable link, but it need
> not be a TCP one).
> Yes. So nncp-bundle -tx is basically a program to create a tarball
> out of files in the outgoing spool directory. nncp-bundle -rx does the
> reverse, processing packets in the tarball destined for the local node
> (and ignoring everything else). That's why you can pipe nncp-bundle
> to uux. The NNCP wire protocol itself is, I think, designed to be
> half-duplex but obviously there is going to be some back-and-forth
> on any wire protocol.
> nncp-xfer moves packets to a directory structure at an arbitrary point
> (usually a USB drive or something).
refresh myself on how ToR operates.
> Node5 could sort of know that the packet arrived from node4, but
> there is no cryptographic guarantee of that (the ONLY signatures in
> this whole chain come from node1) so that information is basically
> discarded.
flight. Thus the flight path really doesn't make any difference to NNCP.
A non-trivial portion of UUCP is about the flight path, accounts that
are used, and other associated metadata. In addition to the data at
rest (what queue does it go into).
> This is different from the UUCP model, in which node5 only really
> knows about its neighbor node4 and may have no idea whatever about
> node1's existence. Permissions in UUCP are based on what the neighbor
> can do, not what the source can do, if I remember correctly.
Though I think there is some security around what remote node(s) the
local node will forward from and / or to. I can't tell you at the
moment if that's the remote source node or the direct neighbor node.
> The other thing that follows from this is that if node3 were
> compromised, the worst it can do is fail to relay packets. It cannot
> modify packets (signature validation from node1 would fail), it
> cannot spoof node1 (lacking its private key), it can't inject fake
> packets claiming to have followed the node1!node2!node3 path (again,
> lacking node1's private key),
bogus packets that ultimately consume resources.
> it can't even see what the ultimate destination is (it just knows
> the next hop is node4 but since it doesn't have node4's private key,
> it doesn't even know if node4 is the ultimate hop or not).
analysis and speculate about anything.
> I really like this.
*nod*
> I think you did, but hopefully it's cleared up above :-)
> Yes, all packets in the chain are signed by the origin and only
> the origin.
Grant Taylor
Jan 4, 2021, 9:11:02 PM1/4/21
to
On 1/4/21 3:55 PM, John Goerzen wrote:
> Oh it is more mundane than that :-) If a packet fails to decode
> correctly, it is simply dropped on the floor. There is no retransmit
> in LoRA, there are no routers, it is just basically a bunch of radios
> in a single broadcast domain with iffy collision detection.
It seems to me like each LoRa radio is in a slightly different broadcast
> Oh it is more mundane than that :-) If a packet fails to decode
> correctly, it is simply dropped on the floor. There is no retransmit
> in LoRA, there are no routers, it is just basically a bunch of radios
> in a single broadcast domain with iffy collision detection.
domain than other LoRa radios simply based off of RF propagation and
what other LoRa radios it is in reception distance of.
> That said, LoRA radios can achieve absolutely incredible range with
> tiny amounts of power, especially when you get down into the <1000bps
> range. I mean, 10+ miles with like 70mW! Tradeoffs abound but they
> are used a lot for remote sensing and control applications where the
> batteries last for years.
> There is LoRAWAN, a layer atop LoRA that uses synchronized timeslots to
> minimize collisions, has limited ARQ, etc. It is not really suitable
> for many uses because proper LoRAWAN "gateway" nodes tend to be quite
> expensive, and they use widely asymmetric uplink vs. downlink speeds,
> and would tend to require one to tunnel UUCP over JSON.
> I'm willing to tunnel UUCP over a lot of things but that one is a
> bridge too far :-)
But I'm also the person that would try to create his own LoRaWAN gateway.
> I should also add that the XBee firmware, by default, forms an ad-hoc
> mesh and supports encryption over the air. It also runs at higher
> speeds than LoRA, maxing out at roughly 50-100Kbps effective bitrate.
> The range is somewhat less than LoRA, but I have had success with a
> 10-mile XBee link at about 50Kbps. There are a bunch of different
> XBee modules that use different wire protocols and frequencies; the
> longest range are the SX modules. They can also operate over Zigbee,
> Wifi, custom 2.4GHz mesh (non-Wifi), which have varying speeds and
> such - but with a single serial interface to the controller.
> I've not specifically replied to your responses to Sergey, on the
> grounds that I didn't write it :-) You make good points but thinking
> about both perspectives requires a quantity of thought I haven't
> had the time for today. Not trying to ignore it, just being honest
> here :-)
John Goerzen
Jan 4, 2021, 10:33:54 PM1/4/21
to
On Monday, January 4, 2021 at 8:05:56 PM UTC-6, Grant Taylor wrote:
>
> I would also hope that there is a way to cause NNCP (UUCP) to simply
> drop anything that wasn't from an explicitly allowed host, independent
> of how it got into the system.
Yes, NNCP would fail to look up the node in its configuration, and therefore would drop it. (I haven't explicitly tested this but this seems to be an explicit design goal)
>
> I would also hope that there is a way to cause NNCP (UUCP) to simply
> drop anything that wasn't from an explicitly allowed host, independent
> of how it got into the system.
> > ISTR uuxqt can generate an error report at this point. I would need to
> > look into NNCP's behavior here; I believe it holds the packet in the
> > incoming queue thinking that the failure may eventually get resolved.
> > Probably not ideal in this situation.
> I presume that you're referring to NNCP on the receiving end system
> where rmail was prohibited.
> This brings to mind another short coming I've seen with UUCP.
> Inter-user access on the same system.
>
> Does NNCP even extend the encryption to the user level on the system?
> E.g. destination!user1 vs destination!user2. As in if files were sent
> something like uuto /path/to/local-file destination!user3.
>
> The UUCP installations that I've been exposed to were largely if you can
> access the UUCP spool, you could access just about anything in it.
> Perhaps this was a misconfiguration on those systems.
My own setup has been to run NNCP as the nncp user, and create sudo rules to allow passwordless sudo from my user account to the nncp-* commands as nncp. That segments off my private files from any kind of NNCP bug for the most part. I add myself to the nncp group so I can pull things out of the incoming directory.
Another approach would be to create a separate installation for each user. This would also be possible in UUCP, but is simpler in NNCP. For instance, I could have a master NNCP on my system, that all inboud and outbound traffic is routed via. Then I would set up another NNCP installation as jgoerzen, with all the files in my own home directory. Then they could talk to each other; for instance, the master would "call" the jgoerzen one like this:
"|sudo -H -u jgoerzen nncp-daemon -inetd"
and vica-versa for calls in the other direction.
This could be extended arbitrarily to any number of users.
With NNCP only needing a single config file, it is somewhat faster to set up than UUCP, but conceptually I don't think there is anything ruling this out from a UUCP perspective either.
> I feel like I should go back and re-read about ToR.
>
> I wonder if I'm incorrect about it too or of NNCP's behavior diverges at
> this point.
> > Node5 could sort of know that the packet arrived from node4, but
> > there is no cryptographic guarantee of that (the ONLY signatures in
> > this whole chain come from node1) so that information is basically
> > discarded.
> So NNCP really is about the data at rest as opposed to the data in
> flight. Thus the flight path really doesn't make any difference to NNCP.
But I think this is a key point and puts a finer point on what was a bit hand-wavy on my part in my initial post. There is no username/password authentication in NNCP because there is no point; a node will prove who it is by its keypairs on the wire protocol (and by what it can decrypt on the offline formats) and the path a packet takes to its destination doesn't matter to the destination.
> A non-trivial portion of UUCP is about the flight path, accounts that
> are used, and other associated metadata. In addition to the data at
> rest (what queue does it go into).
> > This is different from the UUCP model, in which node5 only really
> > knows about its neighbor node4 and may have no idea whatever about
> > node1's existence. Permissions in UUCP are based on what the neighbor
> > can do, not what the source can do, if I remember correctly.
> I largely agree.
>
> Though I think there is some security around what remote node(s) the
> local node will forward from and / or to. I can't tell you at the
> moment if that's the remote source node or the direct neighbor node.
> > The other thing that follows from this is that if node3 were
> > compromised, the worst it can do is fail to relay packets. It cannot
> > modify packets (signature validation from node1 would fail), it
> > cannot spoof node1 (lacking its private key), it can't inject fake
> > packets claiming to have followed the node1!node2!node3 path (again,
> > lacking node1's private key),
> Eh ... I suspect that an upstream node could perform a DoS and send
> bogus packets that ultimately consume resources.
> > it can't even see what the ultimate destination is (it just knows
> > the next hop is node4 but since it doesn't have node4's private key,
> > it doesn't even know if node4 is the ultimate hop or not).
> I wonder if it would be possible to perform some metadata (packet size)
> analysis and speculate about anything.
- John
Grant Taylor
Jan 4, 2021, 11:36:07 PM1/4/21
to
On 1/4/21 8:33 PM, John Goerzen wrote:
> Yes, NNCP would fail to look up the node in its configuration, and
> therefore would drop it. (I haven't explicitly tested this but this
> seems to be an explicit design goal)
ACK
> Yes, NNCP would fail to look up the node in its configuration, and
> therefore would drop it. (I haven't explicitly tested this but this
> seems to be an explicit design goal)
> The story is only marginally better with NNCP. You can setuid/setgid
> some stuff as is often done with UUCP, or mess with sticky bits
> on spool directories and such. Outbound packets will be safe
> from inspection, since they are encrypted using a different node's
> public key. Inbound packets could be decoded by any user that can
> read the NNCP config file (though you could probably restrict that,
> resulting in some protection). There is no default support for a
> multiuser setup, and I haven't messed with it either.
how they get messages into files that the user and root can access.
Granted, that relies on an MTA and / or LDA to do that final delivery
that crosses the UID / GID barrier.
> My own setup has been to run NNCP as the nncp user, and create sudo
> rules to allow passwordless sudo from my user account to the nncp-*
> commands as nncp.
Though I prefer to use authentication (other than password) to sudo via
SSH keys.
> That segments off my private files from any kind of NNCP bug for the
> most part. I add myself to the nncp group so I can pull things out
> of the incoming directory.
in my experience is that the nncp user probably can't access files from
your home directory to send. So you need to arrange access be it
permissions or file location.
> Another approach would be to create a separate installation for each
> user. This would also be possible in UUCP, but is simpler in NNCP.
Though I prefer the idea of leaning on ownership like MTAs / LDAs do.
> For instance, I could have a master NNCP on my system, that all inboud
> and outbound traffic is routed via. Then I would set up another NNCP
> installation as jgoerzen, with all the files in my own home directory.
> Then they could talk to each other; for instance, the master would
> "call" the jgoerzen one like this:
>
> "|sudo -H -u jgoerzen nncp-daemon -inetd"
>
> and vica-versa for calls in the other direction.
>
> This could be extended arbitrarily to any number of users.
> With NNCP only needing a single config file, it is somewhat faster
> to set up than UUCP, but conceptually I don't think there is anything
> ruling this out from a UUCP perspective either.
>
> Don't discount the possibility that I am incorrect also :-)
Sure.
> From a configuration and authorization perspective, yes. Both data
> at rest and data in flight are secured.
>
> But I think this is a key point and puts a finer point on what
> was a bit hand-wavy on my part in my initial post. There is no
> username/password authentication in NNCP because there is no point;
> a node will prove who it is by its keypairs on the wire protocol
> (and by what it can decrypt on the offline formats) and the path a
> packet takes to its destination doesn't matter to the destination.
>
> Right. NNCP does maintain an outbound queue in subdirectories per
> node (named after the node's public key). But there is no way for
> the receiver to see what the via-path was for a packet to reach it;
> it is expected that for an answer to be returned, the via-path should
> be defined in the config file on the receiver.
Please elaborate. How would node3 use a via-path and be able to send an
> node (named after the node's public key). But there is no way for
> the receiver to see what the via-path was for a packet to reach it;
> it is expected that for an answer to be returned, the via-path should
> be defined in the config file on the receiver.
error to node1?
Something like how ICMP errors are dealt with in regard to various
tunneling techniques come to mind. In short, push the error to the end
(presuming it can get there) and have the end generate the error back
tot he sender. -- There is the obvious problem of what to do if the
problem can't be pushed to the end because the end is unreachable. I
still wonder if there might not be some prior art here.
> Basically the routing is supposed to be transparent to the user,
> though it can be explicitly given at any time with a -via parameter.
> Control over forwarding is a bit of a missing link in NNCP right now,
> and probably wouldn't hurt to have. Though I can see why Sergey left
> it out, if he envisioned an NNCP network to be one of curated peers
> and the forwarded packets are opaque to the nodes anyway, and the
> path is irrelevant to permissions.
/ or between a small group of trusted friends. Likely in a mostly full
mesh between each other, thus no real need for forwarding, save for
gatewaying like below.
Though I have had a ""public UUCP network between my public VPSs and my
home router, which acted as a gateway to a ""private UUCP network for my
internal systems. So, friends wouldn't be able to UUCP to my private
workstation, but my VPSs could.
> True. A compromised node3 could send packets that claim to be from
> node3, as of course they would be, and could execute whatever attacks
> node3 would be granted permissions to do. I didn't mean to imply
> that compromise of node3 would be a "meh, I don't care" event, but
> rather that it couldn't be used to spoof another node.
> Possibly, and that's why there is a minsize configuration option and
> parameter: http://www.nncpgo.org/Commands.html#OptMinSize
bytes per wrapper to guestimate how many wrappers there are.
I wonder if you could mess with things by doing something like this:
node1!node2!node3!node2!node3!node4!node5 }:-)
John Goerzen
Jan 4, 2021, 11:48:28 PM1/4/21
to
On Monday, January 4, 2021 at 10:36:07 PM UTC-6, Grant Taylor wrote:
> I largely agree. Though the thing that fails this type of configuration
> in my experience is that the nncp user probably can't access files from
> your home directory to send. So you need to arrange access be it
> permissions or file location.
Ahh, but nncp-file can read from stdin (without buffering in RAM!) So you can do:
> I largely agree. Though the thing that fails this type of configuration
> in my experience is that the nncp user probably can't access files from
> your home directory to send. So you need to arrange access be it
> permissions or file location.
sudo -Hu nncp nncp-file - node2:blah.tar.gz < bigfile.tar.gz
Now if you want to use NNCP's support for sending a directory (in which it internally makes a tar file anyway), then you'd have to move it to a location NNCP could acces, yes. Or else make the tar file yourself and pipe it to nncp-file.
> > Right. NNCP does maintain an outbound queue in subdirectories per
> > node (named after the node's public key). But there is no way for
> > the receiver to see what the via-path was for a packet to reach it;
> > it is expected that for an answer to be returned, the via-path should
> > be defined in the config file on the receiver.
> Please elaborate. How would node3 use a via-path and be able to send an
> error to node1?
I think the basic idea with NNCP is that you generally know the routes to take in advance, so specify them as defaults to take that burden away from every command invocation. You still CAN do that, but usually don't need to.
> I wonder if you could mess with things by doing something like this:
> node1!node2!node3!node2!node3!node4!node5 }:-)
- John
Grant Taylor
Jan 5, 2021, 10:51:26 PM1/5/21
to
On 1/4/21 9:48 PM, John Goerzen wrote:
> Ahh, but nncp-file can read from stdin (without buffering in RAM!)
*nod*nod*
> Ahh, but nncp-file can read from stdin (without buffering in RAM!)
> It would have to use the default via-path defined in its config file.
> In the absence of that, it would queue it for node1 directly.
> I think the basic idea with NNCP is that you generally know the
> routes to take in advance, so specify them as defaults to take that
> burden away from every command invocation. You still CAN do that,
> but usually don't need to.
> Sooner or later you're going to have to install this thing to test
> your evil machinations yourself :-)
investigation. I will likely spin up a couple of VMs and play with
things as time permits in the future.
0 new messages