Peering implementations

[immibis]

What format do NNTP peers usually take?

I see the "suck" tool can download import articles from any
public-access server without special negotiation. Does it put excess
load on the upstream server?

Most people are using innfeed, right? innfeed peering is either IHAVE
(chatty) or CHECK/TAKETHIS (streaming), is that right? Is there any
reason to use IHAVE except for compatibility with older servers?

How's the connection managed? In other streaming protocols I'm more
familiar with (Redis, Apache Kafka) a receiving client would connect to
a server that has messages, state its last synchronization point and
then downloads messages from that point on. If the connection is
interrupted, the messages still arrive upstream and get stored until
they're pulled by the receiver.
NNTP streaming doesn't have that feature, and I think innfeed is
designed so upstream servers try to push articles downstream, rather
than downstream ones pulling them, right? And failures to a separate
spool for re-processing?

Is there really no better way to authorize a connection than checking
the other side's IP address?

Besides UUCP, is anyone using anything exotic?

(I should set up a server with a Kafka database and convince people to
peer directly to Kafka. Sounds !!fun!!)

[Marco Moock]

Am 09.01.2024 um 06:06:54 Uhr schrieb immibis:

> What format do NNTP peers usually take?

> I see the "suck" tool can download import articles from any
> public-access server without special negotiation.

They use peering and not the reader mode that suck will use.

> Does it put excess load on the upstream server?

Depends on the amount of article you load.

> How's the connection managed?

Both sites need to configure their NNTP server to have outgoing
articles and incoming.

> In other streaming protocols I'm more
> familiar with (Redis, Apache Kafka) a receiving client would connect
> to a server that has messages, state its last synchronization point
> and then downloads messages from that point on. If the connection is
> interrupted, the messages still arrive upstream and get stored until
> they're pulled by the receiver.

IIRC INN can spool messages if a server is unreachable (the statistics
sometimes show that). But for servers that are not always online (most
time), UUCP is a better protocol.

> Is there really no better way to authorize a connection than checking
> the other side's IP address?

Using a TLS certificate should be technically possible, but I don't
know which servers support that.

[Grant Taylor]

On 1/8/24 23:06, immibis wrote:
> What format do NNTP peers usually take?

Peers usually push articles to each other.

Readers usually poll / pull articles from the server(s).

> I see the "suck" tool can download import articles from any
> public-access server without special negotiation. Does it put excess
> load on the upstream server?

It depends.

I would never want to use suck for a full Usenet feed for anything other
than academic purposes.

> Most people are using innfeed, right? innfeed peering is either IHAVE
> (chatty) or CHECK/TAKETHIS (streaming), is that right? Is there any
> reason to use IHAVE except for compatibility with older servers?

I would have to refer to RFCs to know for sure, but these seem to be
ways to push articles to peers. Which is used when requires brushing up
on some RFCs.

I believe that innfeed is just one of the ways that INN supports to feed
articles to a configured peer.

> How's the connection managed? In other streaming protocols I'm more
> familiar with (Redis, Apache Kafka) a receiving client would connect to
> a server that has messages, state its last synchronization point and
> then downloads messages from that point on.

My understanding is that INN (which uses innfeed) has knowledge of
articles that are queued to be sent to a peer and which queued articles
have been sent.

With this in mind, INN simply sends articles for desired newsgroups /
distributions to wanting peers henceforth.

Peers don't have a way to say I want you to send me article <such and
such>. That's the domain of the reader to pull articles.

> If the connection is interrupted, the messages still arrive upstream
> and get stored until they're pulled by the receiver.
> NNTP streaming doesn't have that feature, and I think innfeed is
> designed so upstream servers try to push articles downstream, rather
> than downstream ones pulling them, right?

My understanding of the mechanics may be flawed.

> And failures to a separate spool for re-processing?

In a manner of speaking, sort of.

INN will periodically try to establish communications and will send (or
at least offer) spooled articles once communications is established.

> Is there really no better way to authorize a connection than checking
> the other side's IP address?

What is "better"? IP is quite convenient and works quite well between
static (or rarely changing) IPs.

You could use something like IPsec transport mode, or a full fledged VPN
to have much stronger cryptographic validation about the remote system.
But is such worth it for a news server?

> Besides UUCP, is anyone using anything exotic?

I believe that some have TLS encryption in play.

I believe a very small number are using NNCP (?)

I would not be shocked to learn about IPsec / VPN being used.

> (I should set up a server with a Kafka database and convince people to
> peer directly to Kafka. Sounds !!fun!!)

Why?

IMHO that wouldn't be worth it.

What reason do you have to upset the apple cart? -- This is one of the
reasons that I'm not interested in NNCP.

I'd be more inclined to use standard NNTP on a standard news server and
rely on the kernel to add IPsec transport mode encryption +
authentication to the traffic. News software doesn't have to be
involved and can rely on the kernel.



--
Grant. . . .

[Julien ÉLIE]

Hi immibis,

> Is there really no better way to authorize a connection than checking
> the other side's IP address?

FWIW, innd also supports identifying a peer with the Ident protocol (RFC
1413) or with a password.
I doubt they are really used by people nowadays. Besides, these data
are not encrypted.

See the identd and password parameters in incoming.conf:
https://www.eyrie.org/~eagle/software/inn/docs/incoming.conf.html

peer bob {
hostname: "1.2.3.4"
password: "therealbob"
}


You may use that as an additional "proof" (because that's still
insecure) that the peer at 1.2.3.4 is the expected one.

Note that you cannot use "*" as hostname, and then parameter several
peers matching any IP but with several different passwords because the
first matching entry for hostname in incoming.conf, read from the last
one to the first one in the file, will be used. If the conditions like
max-connections, identd and password do not correspond, then access is
denied.

--
Julien ÉLIE

« You say that love is nonsense… I tell you it is no such thing. For
weeks and months it is a steady physical pain, an ache about the
heart, never leaving one, by night or by day; a long strain on one's
nerves like toothache or rheumatism, not intolerable at any one
instant, but exhausting by its steady drain on the strength. » (Henry
Adams)

[Julien ÉLIE]

Hi immibis,

> Is there really no better way to authorize a connection than
> checking the other side's IP address?

FWIW, innd also supports identifying a peer with the Ident protocol (RFC
1413) or with a password.
I doubt they are really used by people nowadays. Besides, these data
are not encrypted.

See the identd and password parameters in incoming.conf:
https://www.eyrie.org/~eagle/software/inn/docs/incoming.conf.html

peer bob {
hostname: "1.2.3.4"
password: "therealbob"
}


You may use that as an additional "proof" (because that's still
insecure) that the peer at 1.2.3.4 is the expected one.

Note that you cannot use "*" as hostname, and then parameter several

peers matching any IP ("*") but with several different passwords because
the first matching entry for hostname in incoming.conf will be used for
the upcoming connection. If the conditions like max-connections, identd