Question on how bang-paths worked from the old days

[John Goerzen]

Hi folks,

I've been trying to sort out some mysteries about rmail from the old pre-domain-addressing days.

The questions are two:

1) Exactly how did mail routing work, and what/why added extra From/>From lines to messages?

2) Was there any sane "reply" function in mail software, and if so, how would it compute a reply path?

So, tackling #1, the sendmail package provides an rmail(8) manpage, which state:

"Rmail interprets incoming mail received via uucp(1), collapsing ``From'' lines in the form generated by mail.local(8) into a single line of the form ``return-path!sender'', and passing the processed mail on to sendmail(8)."

So that made me go "hmm? Multiple From lines?" And looking at the rmail source code, it does look that way. Interestingly the rmail from exim does no such processing (but is probably aimed squarely at a more modern world)

Then I got to looking at the sendmail configs for UUCP. I see this:

define(`UUCP_MAILER_ARGS',`uux - -r -z -a$g -gC $h!rmail ($u)')

It has been a LONG while since I have hacked on sendmail configs, and even then I only barely scratched the non-m4 world... But from what I can tell, if you were sending a message to host1!host2!host3!user, it would run

uux ... host1!rmail host2!host3!user

(I think?) Is that correct?

So why would it not have done:

uux .... host1!host2!host3!rmail user

instead, using UUCP routing to get all the way to the destination? Perhaps these added From lines have part of the answer (being needed to compute a Return-Path?)

And, an any case, what program along the way was adding the extra From or >From lines?

2) So was there a "reply" command in mail software at the time? If so, how could it compute the destination address? (Even more complicated if there was a "reply all"!) Moreover, what went into the From: and To: headers (if, indeed, those even existed?) It seems those would almost have to be rewritten along the way, but I don't see any mention of that.

I have tried to buy every old book on Usenet I can find -- several O'Reilly editions on "Managing UUCP and Usenet" or "Using UUCP" but none of them pay much attention to email over it. It's all about files, Usenet, and a bit of remote execution.

Thanks!

- John

[John Levine]

It appears that John Goerzen <jgoe...@complete.org> said:
>The questions are two:
>
>1) Exactly how did mail routing work, and what/why added extra From/>From lines to messages?

The bang path really was a path, and if you sent mail to a!b!c!bob it would send
the message to host a with address b!c!bob, it'd then send it to host b with address c!bob, etc.

Later on we had the uucp mapping project so you'd send mail to c!bob and your mail server would
look up c in the map, prepend the path, and send it on its way.

>2) Was there any sane "reply" function in mail software, and if so, how would it compute a reply path?

Once we had the maps, yes, before that, not without hand editing the address.

>sending a message to host1!host2!host3!user, it would run
>
>uux ... host1!rmail host2!host3!user
>
>(I think?) Is that correct?

Yes.

>So why would it not have done:
>
>uux .... host1!host2!host3!rmail user
>
>instead, using UUCP routing to get all the way to the destination? Perhaps these added From lines have part of the answer (being needed to compute a Return-Path?)

UUCP routing? There was and is no such thing. The uux command only knows how to send
a command and an input file one hop.

That's why we needed the mapping project to put source routes in the rmail commands.

>And, an any case, what program along the way was adding the extra From or >From lines?

The Unix mail program prepended the From line when it stored the
message in the recipient mailbox. (Still does, in fact.) It is the
delimiter between messages.

If that sounds like a really lame idea that someone might have
invented in two seconds in the 1970s, yup.

By the way, FreeBSD and probably other unix-ish systems still come with uucp and uux.
If you want to experiment, you can try them out.

R's,
John, ima!johnl
--
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

[John Goerzen]

On Monday, August 2, 2021 at 3:23:23 PM UTC-5, John Levine wrote:
> It appears that John Goerzen <jgoe...@complete.org> said:
>
> Later on we had the uucp mapping project so you'd send mail to c!bob and your mail server would
> look up c in the map, prepend the path, and send it on its way.

Interesting. sendmail would call pathalias or something, I guess? I had heard of the maps but hadn't heard of this kind of integration!

> >2) Was there any sane "reply" function in mail software, and if so, how would it compute a reply path?
> Once we had the maps, yes, before that, not without hand editing the address.

Cool. That makes sense.

> >sending a message to host1!host2!host3!user, it would run
> >
> >uux ... host1!rmail host2!host3!user
> >
> >(I think?) Is that correct?
> Yes.
> >So why would it not have done:
> >
> >uux .... host1!host2!host3!rmail user
> >
> >instead, using UUCP routing to get all the way to the destination? Perhaps these added From lines have part of the answer (being needed to compute a Return-Path?)
> UUCP routing? There was and is no such thing. The uux command only knows how to send
> a command and an input file one hop.

Wow. That is a detail I had totally missed. I guess it does make sense given what I remember of the UUCP queue format; the destination node was encoded only in the directory name (at least in Taylor), IIRC.

> That's why we needed the mapping project to put source routes in the rmail commands.

That totally makes sense now, thank you.

> >And, an any case, what program along the way was adding the extra From or >From lines?
> The Unix mail program prepended the From line when it stored the
> message in the recipient mailbox. (Still does, in fact.) It is the
> delimiter between messages.
>
> If that sounds like a really lame idea that someone might have
> invented in two seconds in the 1970s, yup.

There's something deeper going on here than just the mbox format. So if you're host2 in a host1!host2!host3!user message, you're not going to be writing the message into any kind of mailbox - you'll be passing it on to rmail host3!user. From what I can gather, it was expected that every hop along the way would add a "From " or ">From " line identifying itself. Even today, the rmail packaged with sendmail collapses these into a bang path that is basically the return path, and uses them as the -f parameter to the sendmail invocation when it eventually pipes the message on to sendmail. I found an oblique reference to this as the "UUCP From_ header lines" in an early version of a sendmail book. The strong implication I am getting is that rmail/MTA in some fashion was adding just just a Received: header but also a "From_" header at each hop, and the From_ headers (NOT the RFC822 From: line) were processed and coalesced by rmail to form the envelope from (same as SMTP MAIL FROM:).

That much I can deduce by reading the source code to rmail (which, though not terribly long, is terribly old C and not the easiest read). What I haven't figured out is what the mechanism for adding those extra From_ headers was. Was it delivermail (sendmail predecessor) doing it? Or surely something in the Sendmail configuration may have been? I grabbed the earliest sendmail I could easily find (from 2000) but it didn't have easy answers to this question either.

> By the way, FreeBSD and probably other unix-ish systems still come with uucp and uux.
> If you want to experiment, you can try them out.

Oh yes, I've got a couple UUCP nodes running Debian on Docker. I used UUCP "back in the day", but not far enough back in the day for this. In the 90s, I lived in a rural area and used UUCP on FreeBSD and then Debian as a way to get email. Later I worked for the ISP that had provided me the UUCP feed, and was (by then) the guy designated to administer our by then barely-used UUCP service, which ran on BSD/i. As far as I can recall, all of these were Taylor UUCP. They were all used for email and Usenet, but solely for leaf sites hanging off the Internet, so domain addressing was used everywhere, file copying wasn't used at all, and uux was used only as a means to invoke rmail and rnews. So it was a pretty limited view of UUCP, and I didn't really understand the breadth of what had been UUCPNET until a number of years later when things like Usenet archives and search engines became more widely available.

So while I'm familiar with UUCP itself, UUCPNET (as more than a node that dials into exactly one place (an ISP) because of lack of dedicated link) was just before my time.

I did participate in other store-and-forward networks in the 90s, from the BBS world: primarily FidoNet and VirtualNet.

Thanks John!

- John

[Grant Taylor]

On 8/2/21 2:23 PM, John Levine wrote:
> Later on we had the uucp mapping project so you'd send mail to c!bob
> and your mail server would look up c in the map, prepend the path,
> and send it on its way.

I would like to learn more about this. But my (limited) efforts to do
so have not been fruitful. It seems like pathalias is the thread that
is still lingering. Unfortunately I've not found much beyond references
to maps that are now gone.

I have seen some noise about old copies of maps, but things didn't seem
to plug together well when I tried.

> Once we had the maps, yes, before that, not without hand editing
> the address.

I'm mildly surprised by this.

But, I guess the from and to addresses were probably not modified in
transit. So they likely couldn't simply be munged / reversed (take part
of one, swap ends, add it to the other) to work.

I guess that there would be some information in the form of other
Received: headers that might be able to be used to build a reply path.
Or manual email reversing (see above)?

Only /mild/ surprise that this wasn't somewhat automated.

> UUCP routing? There was and is no such thing. The uux command only
> knows how to send a command and an input file one hop.

I question the veracity of that statement. I've used uux from
contemporary version of Taylor UUCP (no known relation) to send commands
to remote systems via intermediary systems. I've got commands like the
following in shell history:

$ uux 'host1!host2!uname -a > /tmp/host2-uname-uux.out'

I seem to recall these commands working as desired. My client / sending
system needed to send the command through host1 to be able to reach host2.

> That's why we needed the mapping project to put source routes in the
> rmail commands.

From what I remember of my testing (within the last 18 months) there
was very little difference between "uux 'host1!rmail host2!host3!user'"
and "uux 'host1!host2!rmail host3!user'". In fact, the biggest
difference was in which layer handled the multiple hops; UUCP or mail.
The former showed multiple Received: headers in the message. The later
only showed the final one. (Both showed the very first MSA injection.)

Maybe this is a newer capability in contemporary Taylor UUCP that didn't
exist in UUCP implementations of old.

> The Unix mail program prepended the From line when it stored the
> message in the recipient mailbox. (Still does, in fact.) It is the
> delimiter between messages.

I think this is highly dependent on which version of mail you are using
and if it is being used as the local delivery agent or not. E.g. a mail
command that is a front end for a different version of mail / nail /
mailx reading from a Maildir.

> If that sounds like a really lame idea that someone might have invented
> in two seconds in the 1970s, yup.

:-)

> By the way, FreeBSD and probably other unix-ish systems still come
> with uucp and uux. If you want to experiment, you can try them out.

I've been using Taylor UUCP over SSH on and off for the last 10 years or so.



--
Grant. . . .
unix || die

[Grant Taylor]

On 8/2/21 3:11 PM, John Goerzen wrote:
> Wow. That is a detail I had totally missed. I guess it does make
> sense given what I remember of the UUCP queue format; the destination
> node was encoded only in the directory name (at least in Taylor), IIRC.

I think it's more accurate to think of the directories as the next hop
(to borrow a descriptive phrase from more contemporary networking). As
in the local host would put messages in the queue for host1 when sending
a message via host1; host1!host2!host3!user. When host1 received the
UUCP file(s) from host0, it would put them in the outgoing queue for
host2. When host2 received the UUCP file(s) from host1, it would put
them in the outgoing queue for host3. When host3 received the UUCP
file(s) from host2, it would then interface with rmail.

Aside: I fear that my examples may be not syntactically correct. I
hope that you can over look that and understand the underlying meaning
that I'm trying to convey.

> That much I can deduce by reading the source code to rmail (which,
> though not terribly long, is terribly old C and not the easiest read).
> What I haven't figured out is what the mechanism for adding those
> extra From_ headers was. Was it delivermail (sendmail predecessor)
> doing it? Or surely something in the Sendmail configuration may
> have been? I grabbed the earliest sendmail I could easily find
> (from 2000) but it didn't have easy answers to this question either.

I don't know. I've done more with file transfer and remote commands
than I have with email via UUCP.

My experience with email via UUCP and Sendmail is that Sendmail does as
little as it can realistically get away with and then hands the message
off to the system's UUCP subsystem rather than getting deeper into the
minutia of UUCP itself.

> Oh yes, I've got a couple UUCP nodes running Debian on Docker.
> I used UUCP "back in the day", but not far enough back in the day
> for this. In the 90s, I lived in a rural area and used UUCP on
> FreeBSD and then Debian as a way to get email. Later I worked for
> the ISP that had provided me the UUCP feed, and was (by then) the guy
> designated to administer our by then barely-used UUCP service, which
> ran on BSD/i. As far as I can recall, all of these were Taylor UUCP.
> They were all used for email and Usenet, but solely for leaf sites
> hanging off the Internet, so domain addressing was used everywhere,
> file copying wasn't used at all, and uux was used only as a means
> to invoke rmail and rnews. So it was a pretty limited view of UUCP,
> and I didn't really understand the breadth of what had been UUCPNET
> until a number of years later when things like Usenet archives and
> search engines became more widely available.

My ah ha moment with UUCP was when I first built my UUCP network between
multiple sites which needed bastions at each side. It was really my
home and office, both of which had UUCP on the edge with systems behind
them. So office-workstation talked to office-router which talked to
home-router which talked to home-workstation. Configuring that was an
eye opening experience. I've since maintained some of that type of
functionality. Though I rarely use it.

I even had it configured on a macOS X 10.<something fairly new> before
macOS 11 Big Sur. -- Apple's updates have since stomped on the config.
It was a pita to tilt at the Apple macOS X windmill to configure.
Something I'm not eager to do if I'm not actually /using/ it.

The process was something like the following:

1) Reboot into maintenance mode.
2) Disable system protection. (I've forgotten what it's called.)
3) Reboot into normal mode and make all desired UUCP config changes.
4) Reboot into maintenance mode.
5) Re-enable system protection.
6) Reboot into normal mode and use UUCP.

> So while I'm familiar with UUCP itself, UUCPNET (as more than a
> node that dials into exactly one place (an ISP) because of lack of
> dedicated link) was just before my time.

I've had a full functional network of 2-4 hosts behind bastions before.
Multiple office computers on one side and multiple home computers on the
other side. So I got to play with UUCP routing.

> I did participate in other store-and-forward networks in the 90s,
> from the BBS world: primarily FidoNet and VirtualNet.

I suspect that even the venerable UUNET et al. mostly did point to point
connections and didn't do much with actual routing. In such as two
adjacent nodes exchanged mail and / or news directly between each other
and expected the higher layer application to do any routing. E.g. the
news server would receive articles from a direct peer, and the news
server itself would send them to other direct attached peers.

Even email, which would need the bang path would still likely do things
between adjacently connected mail servers rather than sending down to
UUCP to send through multiple hops to the far end before coming back up
to the receiving mail server. See the "uux 'host1!rmail
host2!host3!user'" vs "uux 'host1!host2!rmail host3!user'" comment in my
other reply.

[John Levine]

According to Grant Taylor <gta...@tnetconsulting.net>:

>> UUCP routing? There was and is no such thing. The uux command only
>> knows how to send a command and an input file one hop.
>
>I question the veracity of that statement. I've used uux from
>contemporary version of Taylor UUCP (no known relation) to send commands
>to remote systems via intermediary systems. I've got commands like the
>following in shell history:
>
> $ uux 'host1!host2!uname -a > /tmp/host2-uname-uux.out'
>
>I seem to recall these commands working as desired. My client / sending
>system needed to send the command through host1 to be able to reach host2.

I honestly don't remember, but I am pretty sure that other than the
mail routes we got from the mapping project, you had to give the full
bang path for all of the hops. When I was using uucp, it was point to
point over dialup using Telebit modems. If anyone wants a full sized
ISA card Telebit modem, I may still have it in the junk pile.

>Maybe this is a newer capability in contemporary Taylor UUCP that didn't
>exist in UUCP implementations of old.

Maybe. I ditched UUCP in the mid 1980s when I got my a rather funky Internet
connection using Wavelan proto-wifi cards from an antenna in my attic to a
friend's house a block away to share his blazing fast 56Kb DDS connection.
It was still better than dialup uucp.

[Grant Taylor]

On 8/2/21 8:24 PM, John Levine wrote:
> I honestly don't remember, but I am pretty sure that other than the
> mail routes we got from the mapping project, you had to give the full
> bang path for all of the hops. When I was using uucp, it was point
> to point over dialup using Telebit modems.

Oh, I definitely used the full bang path through my test network.

I started to glance at (path)aliases but never got anywhere with it
before life took over.

> If anyone wants a full sized ISA card Telebit modem, I may still have
> it in the junk pile.

Um ... I might be interested. Email me directly if you are serious
about this.

Admittedly, it will be used across a PBX when I get around to setting it up.

> Maybe. I ditched UUCP in the mid 1980s when I got my a rather funky
> Internet connection using Wavelan proto-wifi cards from an antenna
> in my attic to a friend's house a block away to share his blazing
> fast 56Kb DDS connection.

*nod*

56 kB - bit robed signaling. }:-)

> It was still better than dialup uucp.

I feel like I would have appreciated UUCP, especially with how I run my
email and news server.

Don't get me wrong, full access to the Internet would be better. But I
believe that UUCP (or FidoNet point) would be better than /just/ BBS.

[Crypto God]

Grant Taylor wrote:

> On 8/2/21 8:24 PM, John Levine wrote:

>> If anyone wants a full sized ISA card Telebit modem, I may still have
>> it in the junk pile.

I used to have a Trailblazer modem. I think I lost it in Hurricane Ivan
:(

> I feel like I would have appreciated UUCP, especially with how I run my
> email and news server.
>
> Don't get me wrong, full access to the Internet would be better. But I
> believe that UUCP (or FidoNet point) would be better than /just/ BBS.

Despite people throwing rocks at UUCP and similar technologies, not
everyone lives in an always-connected world - in fact, a majority of the
world does not. There's still a need for store-and-forward technology,
even if it's accessed via tethered 1xRTT cell phones, HF radio, or
1200-baud half-duplex packet radio equipment, and UUCP was (and still
is) very suitable for this sort of connectivity (or lack thereof).

[Avon Kerr]

Crypto God <Cryp...@china.com> wrote:

> Despite people throwing rocks at UUCP and similar technologies, not
> everyone lives in an always-connected world - in fact, a majority of the
> world does not. There's still a need for store-and-forward technology,
> even if it's accessed via tethered 1xRTT cell phones, HF radio, or
> 1200-baud half-duplex packet radio equipment, and UUCP was (and still
> is) very suitable for this sort of connectivity (or lack thereof).
>

on this we agree ;-)

--
Agency News | news.bbs.nz

[j...@example.invalid]

Crypto God <Cryp...@china.com> wrote:
>> I feel like I would have appreciated UUCP, especially with how I run my
>> email and news server.
>>
>> Don't get me wrong, full access to the Internet would be better. But I
>> believe that UUCP (or FidoNet point) would be better than /just/ BBS.
>
> Despite people throwing rocks at UUCP and similar technologies, not
> everyone lives in an always-connected world - in fact, a majority of the
> world does not. There's still a need for store-and-forward technology,
> even if it's accessed via tethered 1xRTT cell phones, HF radio, or
> 1200-baud half-duplex packet radio equipment, and UUCP was (and still
> is) very suitable for this sort of connectivity (or lack thereof).

I got on the net just as UUCP was dying off, but I used to download
"SOUP packets" and read news offline. I did use UUCP over compuserve
once or twice, a long, long time ago. :-)

Going forward, I think people are getting sick of always being connected.

I make it a point not to carry my cel phone with me anymore because I
get creeped out knowing some computer-AI could have my mic on and is
transcribing everything the people around me are saying, to be used as
evidence against me or anyone I was talking to at a later date.

There was a time when that was an incredibly self centered idea, but
these days computers do the listening so it's more affordable for them
to do stuff like that. The "human listening" doesn't come into play
until HAL alerts the authorities about your questionable conversation.