Conference about Usenet, federation and moderation

[Julien ÉLIE]

Hi all,

To share a video I've just came across about "Usenet as the original
decentralized social network".
By Rayner Lucas and Tristan Miller at a LibrePlanet conference in March
2023. Thanks to both of you for this video!

https://framatube.org/w/97acbef0-dd05-45d4-a1df-c8ac9cbe36f0


Indeed, fighting spam and abuse is a daily challenge.

And it's not easy to improve the "poor support for multimedia" and "hard
for new users to find it" problems. We would have to guess the first
steps to do about that...

--
Julien ÉLIE

« – I see the world didn't end yesterday.
– Are you sure? » (Alan Moore, _Watchmen_)

[Michael Uplawski]

Julien ÉLIE wrote in news.software.nntp,news.misc:

> https://framatube.org/w/97acbef0-dd05-45d4-a1df-c8ac9cbe36f0

(...)

> And it's not easy to improve the "poor support for multimedia" and "hard
> for new users to find it" problems. We would have to guess the first
> steps to do about that...

I admit that I do not like the format of this video. Can you tell me what makes
you want to improve about multimedia in Usenet and in which way you think this
should be done?

If I had to guess steps to improve the multimedia capabilities of Usenet, I'd
say: *None*.

The World-Wide-Web exists. Maybe the protocol is rotten beyond repair, maybe we
left it to the industries to destroy it and maybe the abyss of antisocial
networks and video platforms is so unavoidable that you feel forced to sell
your soul to them.

Technically, nothing impedes your regaining pocession of a technology that is
(only) dominated by unnecessary transnational companies.

Do not pass on the problem to Usenet.

Cheerio

f'up news.misc

[Marco Moock]

Am 08.07.2023 um 09:42:23 Uhr schrieb Julien ÉLIE:

> And it's not easy to improve the "poor support for multimedia" and
> "hard for new users to find it" problems.

Getting new users means it must be findable via normal web searches.

Something like narkive.com, but with posting possible.

[Adam H. Kerman]

Marco Moock <mo...@posteo.de> wrote:

>Am 08.07.2023 um 09:42:23 Uhr schrieb Julien:

>>And it's not easy to improve the "poor support for multimedia" and
>>"hard for new users to find it" problems.

>Getting new users means it must be findable via normal web searches.

>Something like narkive.com, but with posting possible.

I don't agree. Google has been doing that ever since they bought out
Deja News. Usenet Article Format and Usenet conventions are different
enough from Web pages that the search fails to parse the article in a
useful manner and the person performing the search doesn't get useful
results. Conventional Usenet articles have extensive quoting that
confuse pattern matching, and plenty of users quote just differently
enough from other users that it's a mess.

Gatewaying to and from Usenet from another medium of communication is
UNSUCCESSFUL. The other medium has its own quirks and conventions that
work poorly on Usenet. Usenet followups require followups. Generally, a
Web forum shouldn't have quoting unless the reply is to a comment
further back. A web forum is flat; Usenet is threaded.

Even gatewaying to and from mailing lists is unsuccessful. I do
Usenet-style quoting in reply to an email message. Nearly no one else
does.

You're kind of arguing that gatewaying will increase the poast count and
that will save Usenet. That's just traffic for the sake of traffic but
that's not more Usenet discussion.

What we need are users willing to learn to use a threading newsreader and
subscribe to a News server who want to hold discussions with other
people using newsreaders and not Web gateways. A Web interface to Usenet
WITH POSTING will not result in a conventional Usenet article. A gateway
from a Web forum will not result in Usenet-style discussion.

There's nothing wrong with having different media of discussion, with
different conventions, different formats, and different methods of
access. Usenet's benefit is that our communication method works quite
well for what we do. Making it as Web-forum-like as possible will ruin
Usenet's inherent advantages.

[Marco Moock]

Am 09.07.2023 um 14:40:20 Uhr schrieb Adam H. Kerman:

> I don't agree. Google has been doing that ever since they bought out
> Deja News. Usenet Article Format and Usenet conventions are different
> enough from Web pages that the search fails to parse the article in a
> useful manner and the person performing the search doesn't get useful
> results. Conventional Usenet articles have extensive quoting that
> confuse pattern matching, and plenty of users quote just differently
> enough from other users that it's a mess.

A web gateway could have threading, Google just didn't implement it.

If I look for content, I regularly get results from Google Groups and
narkive.com.

rocksolid is another software that has a news2web gateway.

Most internet users only know a web browser.
To find the Usenet, they need to find interesting information.
The next step is to make them using an NNTP software.

[Marco Moock]

Am 09.07.2023 um 14:29:59 Uhr schrieb Doc O'Leary ,:

> On the contrary, it is almost trivially easy. The only real problem
> is that binary groups got some people upset right around the time
> that broadband was exploding. While it can certainly be argued that
> encoded multipart binaries are not the best way to deal with large
> data files, the RFCs haven’t been updated in something like 25 years.
> Consequently, neither have many of the clients, and that’s where 99%
> of the work has to be done to support non-text messages.

Why do people feel disturbed by them?
They don't need to subscribe to them and news servers need to make sure
that binary attachments cannot be posted.
That's all.

[Adam H. Kerman]

Marco Moock <mo...@posteo.de> wrote:
>Am 09.07.2023 um 14:40:20 Uhr schrieb Adam H. Kerman:

>>I don't agree. Google has been doing that ever since they bought out
>>Deja News. Usenet Article Format and Usenet conventions are different
>>enough from Web pages that the search fails to parse the article in a
>>useful manner and the person performing the search doesn't get useful
>>results. Conventional Usenet articles have extensive quoting that
>>confuse pattern matching, and plenty of users quote just differently
>>enough from other users that it's a mess.

>A web gateway could have threading, Google just didn't implement it.

We've had decades of experience with Web gateways. Yes, in theory, they
could possibly produce a conventional article. Nevertheless, in the real
world, there are no examples of gateways that produce a conventional article.

>If I look for content, I regularly get results from Google Groups and
>narkive.com.

There's no convenient way to limit the search to just Usenet. Deja's
search method was adequate. Google's has been dreadful. To search for a
Usenet article, you need the search to be set up to look for headers and
to be able to distinguish between header and body.

Conflating a Web and Usenet search will give the user lousy results,
particularly as the parser cannot identify the quote and NOT provide the
quote as a result. And then you'll be led to an interface that won't
thread.

I'm sorry but this will not attract a new user because it's so ugly.

>rocksolid is another software that has a news2web gateway.

>Most internet users only know a web browser.
>To find the Usenet, they need to find interesting information.
>The next step is to make them using an NNTP software.

When I first used Usenet, I tried pine, but its Usenet presentation was
lousy. I then learned to use a threading newsreader. I'm sorry but
it wasn't a barrier. If my first experience with Usenet was via a Web
interface, I would have been entirely turned off.

A threading newsreader is key to a decent experience. The Web interface
doesn't provide that.

As you point out long-standing gateways, I'm not aware that any
significant number of their users learned to post via newsreaders.
Instead, they appear to be people who were using Usenet who use the Web
interface because of technology limitations with cell phone and their
personal dislike of laptop or desktop computers.

We already know what hasn't been attracting new users to Usenet.

[Marco Moock]

Am 09.07.2023 um 15:37:50 Uhr schrieb Adam H. Kerman:

> We already know what hasn't been attracting new users to Usenet.

I was one of them.
Without finding articles via narkive.com, I would never post here.

[Adam H. Kerman]

I'll note you didn't state that you had used a non-site-limited Google search.

[John]

Doc O'Leary , <drolear...@2023.impossiblystupid.com> writes:

> For your reference, records indicate that

> =?UTF-8?Q?Julien_=c3=89LIE?= <iul...@nom-de-mon-site.com.invalid> wrote:
>
>> Indeed, fighting spam and abuse is a daily challenge.
>

> Only inasmuch as people don’t *actually* want to take the steps needed to
> solve the problem. The UDP was a rare thing, but cutting off hostile
> networks should be one of the first steps in eliminating abuse.
>

If anything deserves a UDP it's Google Groups.


john

[John Levine]

According to Doc O'Leary , <drolear...@2023.impossiblystupid.com>:
>In general, actual *people* aren’t disturbed. It’s the *powers* that are
>disturbed. Usenet is a content distribution network that isn’t beholden
>to capitalistic principles. Rather than figuring out a way to use it to
>their advantage, they rolled out the legal threats and destroyed it.

Hi, I've managed news servers since the 1980s. I never carried many of
the binary newsgoups for entirely practical reasons. They were (and
are) enormous which in the era of slow, often dialup, transfers tied
up phone lines for hours on end and filled up the small disks we had.
The contents were mostly pictures of nude women and hacked computer
games, neither of which were of much interest to my users.

While there is plenty not to like about the way copyright law works,
that doesn't mean you get to ignore it. Before the DMCA was passed in
1998 it was entirely unclear what liability a server operator had for
pirate content other people put on our servers. Since we generally
were running our servers in our spare time with a legal budget of $0,
taking risks for stuff our users didn't care about would have been
idiotic. So we didn't. Once people started to get dialup or direct
connections to the Internet, they had a lot better ways to download
binary stuff than multipart uuencoded news messages.

I realize that conspiracy theories are more fun, but sometimes
life is boring and practical.

--
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

[Russ Allbery]

Doc O'Leary , <drolear...@2023.impossiblystupid.com> writes:

> In general, actual *people* aren’t disturbed. It’s the *powers* that
> are disturbed. Usenet is a content distribution network that isn’t
> beholden to capitalistic principles. Rather than figuring out a way to
> use it to their advantage, they rolled out the legal threats and
> destroyed it.

A caveat to this analysis is CSAM (child sexual abuse material). (And a
few related nonconsentual things like revenge porn and torture videos, but
CSAM is the most obvious and straightforward to analyze.) This problem is
unfortunately used by the most authoritarian assholes in government as an
excuse to do all sorts of nasty surveillance state shit, so it's very
tempting to decide the whole problem is imaginary and was invented by
wannabe tin-pot dictators to try to get vast police powers. But
unfortunately if you work for any length of time in or around abuse
prevention, you very quickly realize that it's also a real problem. There
are indeed organized groups of peoople who are trying to trade CSAM and
will use your service to do it if you let them, and there's real abuse of
real people underneath it.

Every single service that lets people trade file formats that can be used
for CSAM eventually has to come to terms with this and figure out how
they're going to handle it. And regardless of what you personally may
feel about how the law *should* handle this problem, the rest of society
has really strong opinions about it for rather understandable reasons and
in most countries has set a bunch of rules for how you WILL handle it
whether you like it or not, often backed up by the threat of actual
criminal prosecution, not just the copyright cartel nonsense.

I mostly stay out of these discussions because I ended up saying pretty
much everything I had to say back in 2005 or so and burned out on them,
but people who haven't worked in platform abuse regularly drastically
underestimate the amount of really dark and disturbing shit that people
try to use services to distribute *and make* and assume that the usual
suspects are just lying about it to scare people. And don't get me wrong,
they do routinely lie about things to scare people. But this one is worth
thinking about independently and figuring out how you are going to deal
with, because getting in the middle of that is really bad. Legally, yes,
but also seeing stuff that you will really want to unsee.

> Kind of off topic, but it has been amusing to watch the saga of how the
> alternative have played out in a way that still did not benefit the
> content creators. Musicians complain about the pennies they make from
> streaming. Writers and actors are both on strike now because Hollywood
> I s being Hollywood. And the places providing porn are still under
> attack from the morality police, as the woes of PornHub and others in
> Utah and beyond are many. It almost makes it seem like the problem
> never *was* Usenet at all! :-/

The problem certainly was never Usenet. Usenet has always been a bit of a
backwater, and a lot of this stuff is indeed capitalist bullshit or
prudish nonsense. But Usenet is also not somehow immune from the more
serious problems that do exist.

The absolute libertarian position is very tempting here. For a long time
I too was persuaded by it, and I think it does work up to a certain scale.
But the world is very large and full of people and a tiny fraction of
those people want to do some seriously evil shit and will use your servers
to do it if you let them, and you do need to have some sort of plan to
stop them unless you are willing to support a type of social free-for-all
that 99.99% of humanity is not going to be willing to tolerate.

--
Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

[Russ Allbery]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:

> Russ Allbery <ea...@eyrie.org> wrote:

>> There are indeed organized groups of peoople who are trying to trade
>> CSAM and will use your service to do it if you let them, and there's
>> real abuse of real people underneath it.

> So you put in the effort to *solve* that problem, not sweep it under the
> rug, which is all you accomplish when you start deleting data.

The specific thing you are required to do by US law (in my personal
understanding, I am not a lawyer, this is not legal advice for your
specific situation, read it for yourself at [1]) is report the data to
NCMEC and then make it inaccessible, not delete it (which would be
destroying evidence), so in that narrow sense I do actually agree with
you. I suspect this is similar in most other jurisdictions, although
obviously NCMEC is a US thing so the details will vary.

[1] https://www.law.cornell.edu/uscode/text/18/2258A

However, I personally would rather juggle raw plutonium than spend any
time handling that kind of legal evidence and therefore opt out of the
entire problem by not carrying binaries, since otherwise I am legally
obligated to spend whatever time it takes me to handle that data properly
should any problem arise. Since personally I don't care about any of the
binaries anyway (there are numerous better sources for any non-textual
information I want than Usenet), this seems like it would be a very bad
use of my time, even apart from the fact that if I ever got a report I
would have to go look at the reported data to see if the report was
correct and that does not sound fun. Likewise for DMCA, which is less
legally fraught but still highly annoying.

Other people's mileage may vary, and that's fine! Just know what you're
getting into and realize that no one involved in enforcing laws cares in
the slightest what your opinion is of those laws, so it's worth carefully
picking what types of civil disobedience you want to engage in.
Personally, I vote against copyright bullshit but don't engage in civil
disobedience around it. That's where my personal risk tradeoff is.

> My point is always going to come back to the fact that few people are
> actually interested in solving abuse on *any* platform.

Well, I have spent a bunch of time working professionally with people who
are trying to reduce platform abuse and I have a huge amount of respect
for the work that they do. I don't think anyone who works professionally
in the field thinks this problem is *solvable*. It's a classic
adversarial security problem, and those are almost never solvable. You
would have to make all of your opponents permanently disappear, and, well,
good luck with that. Governments have been trying for millennia. The
best you can manage is harm reduction.

You can do various things to make it easier and various things to make it
harder. One of the most effective things you can do to make dealing with
abuse easier is to ban all non-textual media, because that takes a lot of
the most annoying, dangerous, or horrific types of abuse off the table.
Obviously, that's a tradeoff, and if everyone made that tradeoff that
would be sad for society since there are a lot of non-textual things that
are worth sharing. But I leave hosting the non-textual stuff to people
with platform abuse teams and lawyers on retainer. Again, that's where my
risk tradeoff is.

I think people who think fully decentralized annoymous (or even mostly
anonymous) non-textual file sharing is something they want to get involved
in are completely out of their gourd because you're just hanging up a
giant sign saying "trade all of your illegal shit here, viruses welcome,"
but I'm not the government and I'm not going to stop you. My only goal
here is to make sure people at least go into it with their eyes open.

> But it’s worse than that! It’s gone the other way: cloud platforms have
> business models where the profit motive is in *supporting* that evil
> shit.

I wanted to say that this is definitely not true, but I think I can see
how one might see that this is true from a particular angle. It is true
that in pursuit of profits, a bunch of companies have built network
platforms that make abuse much easier, and are now desperately trying to
play catch-up to filter out the shit that they don't want to carry. There
is an interesting argument to be made that social media itself is the
problem and we should not have any social media at all because it enables
abuse. I don't think I *agree* with that argument, but one can certainly
make that argument coherently, and it's gaining some social popularity.
(This is what repealing section 230 in the US would mean: make social
media effectively illegal by making every service provider liable for
everything they carry. And there are a fair number of advocates for that,
although not all of them understand what they're advocating.)

But if you mean the cloud providers are happy about or actively encourage
people doing evil shit like CSAM on their platforms, this is absolutely
100% not true and I know it's not true from direct personal experience.
Cloud platforms spend large quantities of money, hire whole teams of very
expensive people, and write whole new algorithms and scanning methods to
try to get rid of shit like CSAM. It's a significant expense; it is
absolutely not a profit center. They do that in part because people who
work for cloud platforms are human and have normal human feelings about
CSAM, in part because it's a public relations nightmare, and in part
because not doing some parts of that work is illegal.

[Russ Allbery]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:

> John Levine <jo...@taugh.com> wrote:

>> While there is plenty not to like about the way copyright law works,
>> that doesn't mean you get to ignore it.

> Neither do lawyers get to ignore the realities of math. And math always
> wins, because it isn’t fabricated the way laws are.

Standing in court yelling MATH WILL ALWAYS WIN is very emotionally
satisfying, but weirdly it doesn't make the court judgment go away. Maybe
the lawyers won't be able to ignore the realities of math forever, but
they do in fact get to ignore the realities of math long enough to tell
the men with guns to go take your money.

> What killed Usenet is that it wasn’t practical any longer. No grand
> conspiracies from me. People simply wanted binaries, even if only to
> share family photos and pet videos. When they got a “no”, they didn’t
> even care that you were hiding behind your lawyers and suggesting they
> might be thieves, they just heard the “no” and left.

This is a bizarrely confused history of Usenet. The binary groups were
going strong for years after the text groups were dying. What killed
Usenet was that it had no solution for spam that actually worked for the
average person, only complicated and weird filtering experiments that
never quite worked right.

Spoiler: Still has exactly the same problems. Usenet is just too dead for
spammers to care about it (mostly). If it were ever revived, the same
problem would immediately come back.

[Richard Kettlewell]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:
> John Levine <jo...@taugh.com> wrote:

>> According to Doc O'Leary , <drolear...@2023.impossiblystupid.com>:
>> >In general, actual *people* aren’t disturbed. It’s the *powers* that are
>> >disturbed. Usenet is a content distribution network that isn’t beholden
>> >to capitalistic principles. Rather than figuring out a way to use it to
>> >their advantage, they rolled out the legal threats and destroyed it.
>>
>> Hi, I've managed news servers since the 1980s. I never carried many of
>> the binary newsgoups for entirely practical reasons. They were (and
>> are) enormous which in the era of slow, often dialup, transfers tied
>> up phone lines for hours on end and filled up the small disks we had.

Like John, when I was operating NNTP service for anyone other than
myself, we excluded binaries because they were too expensive to
carry. The CSAM risk was mitigated by subscribing to a service which
notified us of illegal material so we could delete it.

Text groups carried legal risks too, at the time, but that didn’t stop
us carrying text Usenet.

Meanwhile AFAIK binary groups still exist today, there’s just a
relatively limited set of providers who carry them. I don’t know how
they escape being sued into obvlivion by copyright holders.

> That simply speaks to the problem if scaling the network, which has
> nothing to do with any particular data format or content. If
> store-and-forward is the wrong solution, what is the right solution?
> What about a caching proxy for just-in-time delivery of messages?
> It’s kinda moot to have this discussion now, though, because we’ve had
> 20 years of alternative protocols establishing themselves.

Those alternative protocols suggest that the solution is for end users
to download from centralized services (sometimes through a caching
proxy, indeed).

>> I realize that conspiracy theories are more fun, but sometimes
>> life is boring and practical.
>

> What killed Usenet is that it wasn’t practical any longer. No grand
> conspiracies from me. People simply wanted binaries, even if only to
> share family photos and pet videos. When they got a “no”, they didn’t
> even care that you were hiding behind your lawyers and suggesting they

> might be thieves, they just heard the “no” and left. If you want them
> to return, you have to formulate a more practical answer than “still
> no”.

People left Usenet because of the spam and trolling, not because they
couldn’t use it to get hold of pirated video games and movies.

--
https://www.greenend.org.uk/rjk/

[Jesse Rehmer]

On Jul 20, 2023 at 1:21:06 AM CDT, "Russ Allbery" <ea...@eyrie.org> wrote:

> But if you mean the cloud providers are happy about or actively encourage
> people doing evil shit like CSAM on their platforms, this is absolutely
> 100% not true and I know it's not true from direct personal experience.
> Cloud platforms spend large quantities of money, hire whole teams of very
> expensive people, and write whole new algorithms and scanning methods to
> try to get rid of shit like CSAM. It's a significant expense; it is
> absolutely not a profit center. They do that in part because people who
> work for cloud platforms are human and have normal human feelings about
> CSAM, in part because it's a public relations nightmare, and in part
> because not doing some parts of that work is illegal.

Maybe you aren't aware but numerous claims have been made over the years about
service providers profiting from CSAM specifically, including a Usenet Service
Provider whom straddles both sides of profit and cooperation with law
enforcement.

https://cryptome.org/2014/09/giganews-fbi.htm

Having worked in the Service Provider industry my entire career, whether or
not you believe the particular account cited above, it is not far-fetched
based on my experiences.

[Jesse Rehmer]

On Jul 20, 2023 at 2:37:07 AM CDT, "Richard Kettlewell"

<inv...@invalid.invalid> wrote:

> Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:
>> John Levine <jo...@taugh.com> wrote:
>>> According to Doc O'Leary , <drolear...@2023.impossiblystupid.com>:
>>>> In general, actual *people* aren’t disturbed. It’s the *powers* that are
>>>> disturbed. Usenet is a content distribution network that isn’t beholden
>>>> to capitalistic principles. Rather than figuring out a way to use it to
>>>> their advantage, they rolled out the legal threats and destroyed it.
>>>
>>> Hi, I've managed news servers since the 1980s. I never carried many of
>>> the binary newsgoups for entirely practical reasons. They were (and
>>> are) enormous which in the era of slow, often dialup, transfers tied
>>> up phone lines for hours on end and filled up the small disks we had.
>
> Like John, when I was operating NNTP service for anyone other than
> myself, we excluded binaries because they were too expensive to
> carry. The CSAM risk was mitigated by subscribing to a service which
> notified us of illegal material so we could delete it.
>
> Text groups carried legal risks too, at the time, but that didn’t stop
> us carrying text Usenet.
>
> Meanwhile AFAIK binary groups still exist today, there’s just a
> relatively limited set of providers who carry them. I don’t know how
> they escape being sued into obvlivion by copyright holders.

I can tell you one VERY interesting FACT about all commercial Usenet providers
in the United States - their servers and infrastructure handling Usenet feeds
are located in Ashburn Virginia, in facilities near the NSA.

[Marco Moock]

Am 20.07.2023 um 13:24:19 Uhr schrieb Jesse Rehmer:

> I can tell you one VERY interesting FACT about all commercial Usenet
> providers in the United States - their servers and infrastructure
> handling Usenet feeds are located in Ashburn Virginia, in facilities
> near the NSA.

Is there a specific (legal) reason for that?

[Adam H. Kerman]

Lots of communications infrastructure and power to serve Fort Meade?
Seems like the right place to site a server farm.

[Richard]

[Please do not mail me a copy of your followup]

=?UTF-8?Q?Julien_=c3=89LIE?= <iul...@nom-de-mon-site.com.invalid> spake the secret code
<u8b40v$30nta$1...@news.trigofacile.com> thusly:

>Indeed, fighting spam and abuse is a daily challenge.

The only way to combat this is to sign articles and establish trust
networks and deny spammers membership in the trust networks. Note
that this does not preclude anonymity.

>And it's not easy to improve the "poor support for multimedia"

This is a solved problem for decades (MIME). What's always been
missing is good newsreader support and a cultural willingness to
embrace content beyond text/plain. IMO, the latter is the big
stumbling point; it's a cultural issue not a technological one.

>and "hard for new users to find it" problems.

This is a problem? :)

Being hard to find keeps out the rugrats.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> Maybe you aren't aware but numerous claims have been made over the years
> about service providers profiting from CSAM specifically, including a
> Usenet Service Provider whom straddles both sides of profit and
> cooperation with law enforcement.

> https://cryptome.org/2014/09/giganews-fbi.htm

I should be clear: I am not saying that absolutely no service provider
anywhere has ever decided to try to profit from CSAM. Obviously I can't
make a claim like that, particularly given that 8chan and company exists.
I personally know nothing about Giganews and would be a fool to comment.

The world is large and full of people. I'm sure just about every awful
thing anyone can think of has been attempted by someone at one point or
another.

My point is only that the idea that Amazon, Microsoft, Google, Meta,
Apple, etc. (the companies that most people think of when someone says
"cloud provider" which was the term used in the message I was replying to)
have a business model built on CSAM is directly contradicted by the very
real and large expenditures by those same companies to try to get that
stuff off their platforms as much as they possibly can given other
constraints they have around promises of user privacy, etc.

The shape of their problem is instead that they have been wildly
successful beyond any of their reasonable expectations and with that level
of scale comes a level of platform abuse that they were wholly unprepared
originally to deal with. They're still desperately digging themselves out
of that hole and occasionally falling into it again. But if they could
somehow make the problem go away, they would *love* to do so.

One tension they have is that a lot of people also (quite understandably)
feel pretty weird, at best, about their cloud provider saying "oh, we're
going to scan everything you do to see if any of it would be of interest
to the cops," which means they would strongly prefer to be reactive rather
than proactive, but some of these groups are quite sophisticated and find
new ways of hiding from reactive scanning techniques. Another tension
they have is that in a lot of cases they would like to promise end-to-end
user privacy because it's a selling point, but end-to-end user privacy
from the cloud provider means they by definition cannot scan or do
anything else about whatever is crossing that channel, so they end up in a
three way fight between privacy advocates, the government, and the news
media. (And, to be clear, usually manage to say the stupidest possible
things and step on five rakes in the process.)

It's an incredibly tricky area and I certainly don't have all of the right
answers. All I know is that (a) Usenet is a hobby for me and I'm not
going near this mess with a ten foot pole and would strongly recommend
against anyone else doing so either unless you have the money for lawyers
and a real anti-abuse team, and (b) the idea that cloud providers in
general benefit from CSAM seems like bullshit to me given the amount of
money they spend on trying to get rid of it and trying to control the
public relations fallout from the bits they didn't manage to get rid of.
Even if the balance of money is slightly towards profit (which I highly
doubt), it's UTTERLY dwarfed by their legitimate business and directly
threatens it, so it is certainly not a motivating force for the typical
cloud provider.

[Jesse Rehmer]

I shouldn't have painted with such a broad stroke, as I didn't mean to imply
that the large cloud providers mentioned had such business models or
interests, but the service provider realm is broad and complicated.

In the early 2000s I worked for SAVVIS (now Lumen) and at the time they were
harboring a lot of very bad actors within their hosting platform as well as
providing IP transit services to such entities, mostly stemming from the
acquisitions of Cable & Wireless and Exodus/MCI, but it was a bigger spectrum
than the "spammers" that were represented in the media at the time.

I was not directly involved, but worked on the Hosting help desk while this
was ongoing, and know that we had the knowledge, capabilities, and man power
to shut them all off within a few hours, but the higher ups let it drag out
for a year or more before finally terminating contracts. It wasn't until it
started to hit the news and they began receiving external pressure that they
changed their tune, and very slowly terminated the bad actors. I realize this
is not exactly the same scenario as CSAM, but I've witnessed decisions made at
executive levels that chose profits over doing what is legal or right.

Service Providers in the USA have immense protections, perhaps a lot more than
they should, and we're going through another phase where that is being
re-examined and up for litigation. Having been somewhat of a man-in-the-middle
of all things related to Internet and Society - I'm not exactly sure where I
stand on most of it either.

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> I was not directly involved, but worked on the Hosting help desk while
> this was ongoing, and know that we had the knowledge, capabilities, and
> man power to shut them all off within a few hours, but the higher ups
> let it drag out for a year or more before finally terminating
> contracts. It wasn't until it started to hit the news and they began
> receiving external pressure that they changed their tune, and very
> slowly terminated the bad actors. I realize this is not exactly the same
> scenario as CSAM, but I've witnessed decisions made at executive levels
> that chose profits over doing what is legal or right.

> Service Providers in the USA have immense protections, perhaps a lot
> more than they should, and we're going through another phase where that
> is being re-examined and up for litigation. Having been somewhat of a
> man-in-the-middle of all things related to Internet and Society - I'm
> not exactly sure where I stand on most of it either.

Yeah, this is all very fair and I agree with everything you say here. The
smaller the provider, the more likely that the response to platlform abuse
will be... shall we say, random. Some companies will take immediate
action; some companies will avoid doing anything for as long as possible.

Perhaps I am too optimistic, but I think in *most* cases the lack of
action is some combination of the fact that dealing with this stuff
requires spending money that doesn't make any profits and small companies
are more likely to feel like this is unfair and be very grudging about
doing it, plus the general human tendency to believe that any problem that
one doesn't want to deal with can't possibly be as bad as people claim it
is. I think actual malice is rather rare, but cutting corners and not
spending money where it seems avoidable is very common.

There are also, of course, folks with strong libertarian principles,
probably more common among the sort of folks who often start small
businesses, and as long as the argument feels like it's on the level of
abstract principles, it's easy to decide that the libertarian principles
should win. (Again, maybe I'm too optimistic, but I don't think this
tends to survive direct contact with the worst types of platform abuse.
But thankfully that sort of abuse is relatively rare, so the debate
sometimes stays abstract for a long time.)

The larger companies do not have the luxury of ignoring this sort of
problem and already have frequent contact with law enforcement due to the
simple fact that people live large amounts of their lives on cloud
providers these days, so legal investigations constantly go there.
They're more likely to have thought somewhat seriously about this and have
employees whose job it is to figure out what to do, although even in those
cases they tend to be underresourced compared to the size of the problem.

[Jesse Rehmer]

In my personal experience I find that companies which are controlled by a
Board of Directors generally lean towards doing and spending as little as
possible in order to retain valuable customers or prevent legal penalties, and
have no issue directing those below to ignore their conscience.

The smaller or privately held providers whom are controlled by founders or
'techies' tend to take such matters more seriously and with a lot more
conscience (unless their only business model is the trouble makers).

It all depends on who is at the top and what sort of conscience they have, or
don't, I suppose.

[Jesse Rehmer]

I have no substantial evidence to support what I'm about to throw out here,
and it is my own wild speculation.

There are far cheaper geographical locations in the USA to obtain reliable
power, IP/transit, and secure raised floor space than Ashburn VA. US-based
commercial Usenet providers' corporations are not based out of Virginia (last
I checked most were based out of Florida and one in Texas). In the USA, before
full-feed Usenet turned into a commercial venture, providers were spread out
and their infrastructure lived close to their headquarters. To my knowledge
(in this area it is more limited), there are no longer any non-commercial
entities exchanging a full Usenet feed in the USA, perhaps apart from a few
research facilities.

Don't you think it is highly curious that service providers whose profits
center around the exchange of copyrighted material and pornography are all
located in the one geographical location in which the US government
semi-openly admits is their largest spy point?

(Let's not kid ourselves, I know the full-feed Usenet operators are swimming
in such content, and I don't believe anyone is paying for Usenet access to
text-based discussion or share family photos.)

I'm not convinced the US government doesn't want Usenet and the commercial
Usenet providers to exist, contrary to what some lawmakers have stated in the
past, but instead could care less about the profits made as long as the
providers are cooperating with the surveillance and whatever is required for
prosecution of end-users/bad actors/criminals/etc.

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> There are far cheaper geographical locations in the USA to obtain
> reliable power, IP/transit, and secure raised floor space than Ashburn
> VA.

There are a lot of good reasons to put your stuff there that were even
stronger a decade or more ago when a lot of those hosting decisions were
made. There are really good reasons why Amazon's us-east-1 region and
their original S3 storage was there, entirely technical reasons.

Some of them:

1. It's no longer the cheapest region for power, etc., but it is still
very cheap, particularly compared to, say, New York or New Jersey.
2. It's very close to where most of the people in the United States live,
which matters a lot for latency.
3. Huge amounts of legacy infrastructure is based there, going all the way
back to ARPANET, so it's very easy to get peering (this is changing
over time).
4. It's physically close to Wall Street, which is a huge central point for
network infrastructure, without actually being in Wall Street and
having to pay the costs designed for algorithmic trading.

It was only very recently that Amazon started pushing us-east-2 in Ohio,
and basically everyone in the first round of businesses that hosted on AWS
have significant infrastructure in us-east-1.

I think people reach a little too far for conspiracy theories about stuff
like this. The reason why the national government, Wall Street, the CIA
and NSA, the FBI, a bunch of network infrastructure, tons of ISPs, and
tons of servers are all in the same place is because that's where all the
people lived in the early United States and therefore that's where all the
cities were and it's still where the major population centers of the
United States cluster. Obviously, California, Texas, and Florida are
changing that, but there's a ton of momentum behind those patterns and I
believe the imbalance of US residents towards the eastern time zone is
still quite large.

Virginia actively tried to be the cheap state close enough to all the
expensive states but far enough away that you're not paying the expensive
state premium, and that worked really well for them.

[Jesse Rehmer]

I absolutely accept all of that, you're right, many of us reach pretty far.

However, with everything you said, wouldn't it seem to make more sense that
commercial Usenet operators would have first started in Virginia and migrated
elsewhere, versus the other way around, which is what happened with the full
Usenet feed? Today you've got to be on the Equinix IX in Ashburn to convince
anyone to give you a full feed. Perhaps the consolidation of Usenet
services/feeds exchanged there is based on common financial benefit of them
all living/exchanging the feed there than dispersed over transit providers.

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> However, with everything you said, wouldn't it seem to make more sense
> that commercial Usenet operators would have first started in Virginia
> and migrated elsewhere, versus the other way around, which is what
> happened with the full Usenet feed? Today you've got to be on the
> Equinix IX in Ashburn to convince anyone to give you a full
> feed. Perhaps the consolidation of Usenet services/feeds exchanged there
> is based on common financial benefit of them all living/exchanging the
> feed there than dispersed over transit providers.

So, there's this really common phenomenon in economics where certain
places become the center of a particular type of industry for reasons that
aren't really obvious. Usually there's *some* initial impetus, but it's
often minor and doesn't explain the level of concentration. (For
instance, for Virginia, doubtless it got a head start because of ARPANET.)
For example, why are all the movie studios in Los Angeles, or all the tech
companies in San Francisco and San Jose (and now Seattle and Austin, but
still not, say, San Antonio or Phoenix), or all the banks in New York?

Standard economic theory says that a lot of this is due to network
effects. Once there is an industry concentration in an area, it tends to
become more concentrated because it's just so convenient. In northern
Virginia, there are a bunch of well-established data facility companies
that know how to run data facilities and are competing with each other.
There is a huge trained local population of data center workers so it's
easy to hire techs. All the local power equipment and installation
companies do data center work so they're good at it. There are a bunch of
local HVAC companies that know how to cool data centers. There's a big
Dell service center right there to repair your servers and they have all
the parts. All the servers are already there so the transatlantic cables
terminate there because why not. Etc.

If you start out as some small business back in the day when that meant
physical servers (these days, everyone just starts on AWS or GCS or
Azure), you probably want those servers close by because you're going
there yourself to fix them. So you start with some local hosting provider
wherever you are physically located.

But then if you succeed and grow, suppose that you want to provide a
high-bandwidth service, and you want it to be low latency for the majority
of the US population while still having reasonable latency to Europe and
the rest of the US. Where do you rent a data center? You're a US company
and don't want to deal with international corporate law, so Toronto is
out. What are the other options?

Virginia looks really good, and ten years ago looked even better. That's
where all the expertise is, it's centrally located for both the US and
Europe, there's a lot of market competition so you can shop around, it's
close to all the peering... looks great. Sure, you can probably get
cheaper buildings in, oh, Charleston, South Carolina or Louisville,
Kentucky or whatever, but that supporting network of specialization isn't
there. You could go to Miami, but that's both probably not cheap and also
you have to worry about hurricane-related disruptions. You could do
Pittsburgh, or Ohio, or whatever, but Virginia is really appealing.

[Adam H. Kerman]

Russ Allbery <ea...@eyrie.org> wrote:
>Jesse Rehmer <jesse....@blueworldhosting.com> writes:

>>However, with everything you said, wouldn't it seem to make more sense
>>that commercial Usenet operators would have first started in Virginia
>>and migrated elsewhere, versus the other way around, which is what
>>happened with the full Usenet feed? Today you've got to be on the
>>Equinix IX in Ashburn to convince anyone to give you a full
>>feed. Perhaps the consolidation of Usenet services/feeds exchanged there
>>is based on common financial benefit of them all living/exchanging the
>>feed there than dispersed over transit providers.

>So, there's this really common phenomenon in economics where certain
>places become the center of a particular type of industry for reasons that
>aren't really obvious. Usually there's *some* initial impetus, but it's
>often minor and doesn't explain the level of concentration. (For
>instance, for Virginia, doubtless it got a head start because of ARPANET.)
>For example, why are all the movie studios in Los Angeles, or all the tech
>companies in San Francisco and San Jose (and now Seattle and Austin, but
>still not, say, San Antonio or Phoenix), or all the banks in New York?

Movie production is in Los Angeles because that's about as far from New
York as you could get a century ago, without sending the black sheep
brothers in the family to China, the ones exiled from New York to run
several of the studios. The other brothers stayed in New York to raise
money.

There's also reliable sunshine.

For a few years, in early days, there was a number of movie studios in
Chicago, but that would not last.

>. . .

[Jesse Rehmer]

Excellent points.

I don't know why, but the one thing I've completely overlooked until you
pointed out, having the transatlantic peering points there may be a big draw
since most of the commercial Usenet providers are now also operating in parts
of Europe, or exchanging feeds directly with the European providers, so that
makes a lot of sense.

[John Levine]

It appears that Jesse Rehmer <jesse....@blueworldhosting.com> said:
>> Meanwhile AFAIK binary groups still exist today, there’s just a
>> relatively limited set of providers who carry them. I don’t know how
>> they escape being sued into obvlivion by copyright holders.

There's no secret. They act on the DMCA notices they receive and
delete the offending articles. I've been a technical expert in some
court cases on this very topic.

>I can tell you one VERY interesting FACT about all commercial Usenet providers
>in the United States - their servers and infrastructure handling Usenet feeds
>are located in Ashburn Virginia, in facilities near the NSA.

The guy who runs Giganews, which is located in Texas, will be
surprised to hear that someone moved his data cernter while he wasn't
looking. Who knew the NSA was so crafty?

[John Levine]

Since it's not true, probably not.

I expect some are in Ashburn since that's where vast numbers of cloud
providers are. The NSA is in Ft Meade, 50 miles away, but why let
facts get in the way of a good conspiracy theory?

[John Levine]

According to Adam H. Kerman <a...@chinet.com>:

>Movie production is in Los Angeles because that's about as far from New
>York as you could get a century ago, without sending the black sheep
>brothers in the family to China, the ones exiled from New York to run
>several of the studios. The other brothers stayed in New York to raise
>money.

I have heard it was as far as possible from Thomas Edison who had a whole
bunch of patents they hadn't licensed.

>There's also reliable sunshine.

That too, and fairly close to Mexico in case of (legal) emergencies.

>For a few years, in early days, there was a number of movie studios in
>Chicago, but that would not last.

In the really early days they were in Ithaca NY, but if you've ever spent
a winter in Ithaca, you know why they moved.

[John Levine]

According to Jesse Rehmer <jesse....@blueworldhosting.com>:

>However, with everything you said, wouldn't it seem to make more sense that
>commercial Usenet operators would have first started in Virginia and migrated

>elsewhere, versus the other way around, which ...

which, as previously noted, never happened.

[Adam H. Kerman]

John Levine <jo...@taugh.com> wrote:
>According to Adam H. Kerman <a...@chinet.com>:

>>Movie production is in Los Angeles because that's about as far from New
>>York as you could get a century ago, without sending the black sheep
>>brothers in the family to China, the ones exiled from New York to run
>>several of the studios. The other brothers stayed in New York to raise
>>money.

>I have heard it was as far as possible from Thomas Edison who had a whole
>bunch of patents they hadn't licensed.

That is an excellent point.

[Jesse Rehmer]

On Jul 20, 2023 at 4:39:29 PM CDT, "John Levine" <jo...@taugh.com> wrote:

> It appears that Jesse Rehmer <jesse....@blueworldhosting.com> said:
>>> Meanwhile AFAIK binary groups still exist today, there’s just a
>>> relatively limited set of providers who carry them. I don’t know how
>>> they escape being sued into obvlivion by copyright holders.
>
> There's no secret. They act on the DMCA notices they receive and
> delete the offending articles. I've been a technical expert in some
> court cases on this very topic.
>
>> I can tell you one VERY interesting FACT about all commercial Usenet providers
>> in the United States - their servers and infrastructure handling Usenet feeds
>> are located in Ashburn Virginia, in facilities near the NSA.
>
> The guy who runs Giganews, which is located in Texas, will be
> surprised to hear that someone moved his data cernter while he wasn't
> looking. Who knew the NSA was so crafty?

From https://giganews.com/peering:

We will only exchange full binary feeds with peers who enter into settlement
free network peering with us. We are currently able to peer in the following
facilities:
Equinix, Ashburn, VA (ASN 30094) – direct cross-connect required
AMS-IX, Amsterdam, NL (ASN 30094)

[John Levine]

For reasons everyeone else has explained, it's not surprising they
have a point of presence in Ashburn, since that's where everyone else
is and it is a whole lot cheaper to peer with a bunch of people at an
IX than one at a time with separate links.

But the company is mostly in Texas and always has been.

Re that decaode old web page claiming they're an FBI front, if you
read it with your brain turned on it was clear that they were doing
something with the FBI to try and catch people distributing CSAM,
which is not the same thing as being an FBI front. Again, considering
how big they are, it's not very surprising the FBI would ask them to
do that.

[Russ Allbery]

John Levine <jo...@taugh.com> writes:

> Re that decaode old web page claiming they're an FBI front, if you read
> it with your brain turned on it was clear that they were doing something
> with the FBI to try and catch people distributing CSAM, which is not the
> same thing as being an FBI front.

I suspect this is not uncommon because it makes a lot of sense given how
CSAM trading rings work. From the perspective of the provider, their
service is being abused by an organized criminal gang, and they've
probably only sure about a small fraction of the activity because that's
how anti-abuse measures usually work. But they probably have enough data
to connect a series of accounts that *might* be part of the same activity,
which they can't prove that without police powers to subpoena other ISP
records, etc.

So they turn that all over to the FBI and the FBI maybe says "don't do
anything for a moment, let us get some more data and see if we can roll up
the entire gang rather than just pick off some low-hanging fruit." If it
works, the FBI gets to arrest a whole bunch of people at once and the
provider gets an entire organized *network* off their service in a way
that makes it less likely to crop up again, *and* they get a reputation as
a dangerous place to do CSAM trading. From their perspective, it's a
nearly ideal outcome.

[Jesse Rehmer]

On Jul 20, 2023 at 9:19:19 PM CDT, "John Levine" <jo...@taugh.com> wrote:

> It appears that Jesse Rehmer <jesse....@blueworldhosting.com> said:
>>> The guy who runs Giganews, which is located in Texas, will be
>>> surprised to hear that someone moved his data cernter while he wasn't
>>> looking. Who knew the NSA was so crafty?
>>
>> From https://giganews.com/peering:
>>
>> We will only exchange full binary feeds with peers who enter into settlement
>> free network peering with us. We are currently able to peer in the following
>> facilities:
>> Equinix, Ashburn, VA (ASN 30094) – direct cross-connect required
>> AMS-IX, Amsterdam, NL (ASN 30094)
>
> For reasons everyeone else has explained, it's not surprising they
> have a point of presence in Ashburn, since that's where everyone else
> is and it is a whole lot cheaper to peer with a bunch of people at an
> IX than one at a time with separate links.
>
> But the company is mostly in Texas and always has been.
>
> Re that decaode old web page claiming they're an FBI front, if you
> read it with your brain turned on it was clear that they were doing
> something with the FBI to try and catch people distributing CSAM,
> which is not the same thing as being an FBI front. Again, considering
> how big they are, it's not very surprising the FBI would ask them to
> do that.

I agree, the "FBI front" claim as a whole is a stretch; however, from what I
know about where and how full binary feeds are being exchanged today, it is
not far-fetched to believe the operators may be allowed the privilege of
operating with minimal interference from law enforcement for the privilege of
"eavesdropping" or other high-level cooperative efforts.

Commercial Usenet operators adhere to DMCA in the USA, but that doesn't cover
CSAM or other illegal content outside of copyright. If Usenet providers are
not monitoring and detecting CSAM or other illegal content, it makes sense to
me that the only other invested and capable party would be law enforcement at
the national level. Encryption and obfuscation are the most common way binary
data is posted to Usenet today, so there likely has to be a high level of
cooperation between the providers and law enforcement on access to the content
as well as ease of tracing back to the origin. Within the realm of monitoring
and tracing, again, I find it not far-fetched to believe the government would
provide their own "employees" to perform this work for the providers whom may
or may not be financial/technically able to do so on their own.

I'm not making any proclamations about GigaNews or other Usenet providers and
their relationship with law enforcement, but pointing out interesting
coincidences that did not exist when larger numbers of Usenet operators were
exchanging full feeds and within the USA specifically, the Usenet network was
far more decentralized. Full feed exchange has become completely centralized
within the USA to the Ashburn EQ-IX (as best I can tell since Altopia shut
down, their infrastructure is still sort of in play, assume someone purchased
it and may be reselling, but the alt.net Path element goes in/out Ashburn
based on my inflow data). The consolidation to the Ashburn area did not happen
until ISPs and other entities dropped full feeds when Usenet became a dirty
word, also around the same time law enforcement became highly interested in
monitoring it and other Internet traffic on a large scale.

I like to speculate, and am distrustful of the government and service
providers (having been on the inside for enough time), so why would GigaNews
operate servers in Texas or anywhere else for that matter, when their feeds
are exchanged exclusively in Ashburn? Backhauling 200-400TB of traffic per day
over IX or similar cheaper transit links, is not usually efficient or cost
effective, especially if the majority of your IP traffic originates in/out a
location hundreds of miles away. They may have some redundancy where they are
storing data outside of Ashburn, but replicating the entire feed and keeping
any kind of retention in multiple locations seems very cost prohibitive for
businesses claiming to operate on thin profit margins. HighWinds Network
Group, in my opinion, has been the leader in Usenet from an infrastructure and
network architecture standpoint. They have a vast global network that they
primarily own and operate themselves, but again, their Usenet feeds are
exchanged in Ashburn within the USA and their other Usenet server farm is in
Amsterdam. If the largest global player isn't replicating beyond their two
primary points of presence, I doubt GigaNews would be capable from a cost
standpoint alone.

GigaNews's AS and associated prefixes are currently only announced from Deft's
network (out of Chicago from my perspective) and the two IXs in Ashburn VA and
Frankfurt Germany. It seems very recently they lost many IP peers and prefix
announcements, but maybe they sold off old parts of the business unrelated to
GigaNews. From a network/service provider perspective, they are the most
opaque of the bunch operating in the USA in terms of exactly how they operate,
and they haven't answered an e-mail sent to their peering address in over 4
years from myself and other operators I am in contact with attempting to peer
or adjust feeds with them.

[Richard Kettlewell]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:
> Don't you think it is highly curious that service providers whose
> profits center around the exchange of copyrighted material and
> pornography are all located in the one geographical location in which
> the US government semi-openly admits is their largest spy point?

The NSA and their peers are quite capable of taking an NNTP feed like
anyone else, why would they care about the location of other Usenet
providers?

--
https://www.greenend.org.uk/rjk/

[Jesse Rehmer]

On Jul 21, 2023 at 2:42:00 AM CDT, "Richard Kettlewell"

Why would they trust anyone to hand them a potentially filtered feed for
surveillance?

[Richard Kettlewell]

Easy risk to mitigate: take multiple feeds, use a front organization,
etc.

If a peer does start filtering then that represents an interesting line
of investigation l-)

--
https://www.greenend.org.uk/rjk/

[Richard Kettlewell]

John Levine <jo...@taugh.com> writes:
>>> Meanwhile AFAIK binary groups still exist today, there’s just a
>>> relatively limited set of providers who carry them. I don’t know how

>>> they escape being sued into obvlivion by copyright holders.
>
> There's no secret. They act on the DMCA notices they receive and
> delete the offending articles. I've been a technical expert in some
> court cases on this very topic.

I’m not very familiar with US law. The reason I’m puzzled is that the
point of carrying binary groups is, rather obviously, to facilitate
large-scale copyright violation, not some more innocuous pursuit that
happens to be troubled by the occasional pirated movie.

Can you really get away with that (at least in the USA) as long as you
respond to notices from the rights holders who actually bother to check?
If so then how is it that Napster was destroyed?

(Has it not occurred to anyone to automate ‘NNTP to DMCA notice’?)

--
https://www.greenend.org.uk/rjk/

[Russ Allbery]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:
> Russ Allbery <ea...@eyrie.org> wrote:

>> Standing in court yelling MATH WILL ALWAYS WIN is very emotionally
>> satisfying, but weirdly it doesn't make the court judgment go away.
>> Maybe the lawyers won't be able to ignore the realities of math
>> forever, but they do in fact get to ignore the realities of math long
>> enough to tell the men with guns to go take your money.

> That’s a nice straw man scenario, but it bears no relation to the argument
> I was making. Laws are but one means to an end, and bad laws do *not* do
> what they’re supposed to be doing (and fuel the conspiracy theories that
> their “unintended consequences” were actually intended the whole time).

You don't seem to understand my point. This whole thread started with
talking about news administration and with you being upset that people
aren't carrying binaries. The point that I and several other people are
making is that carrying binaries creates legal hassles. Now you're saying
the laws are bad.

The laws may or may not be bad, but my point is that it doesn't matter.
The people enforcing the laws do not give a single shit what your opinion,
or my opinion, of the laws are. All of these arguments about whether the
laws work or not are therefore entirely beside the point. I think use of
at least some currently illegal drugs should be legalized and a lot of
drug laws are very bad and counterproductive; that doesn't mean I'm going
to start selling drugs while drugs are illegal.

The laws are a constraint on how people run their servers (this is sort of
the definition of laws). If you think the laws should change so that
carrying binaries wouldn't pose legal risk, go get them changed and then
let us know. Until then, we have to start with the laws the way that they
are right now, whether we like them or not, which makes carrying binaries
rather risky unless you have a lot of expensive infrastructure in place to
deal with the highly predictable consequences.

You can instead that Usenet server operators should engage in coordinated
civil disobedience because the laws are bad, but, well, good luck with
that. I'm not interested, at least.

> Regardless, my point remains that people wanted a “one stop shop” for
> their group chat messages, which were increasingly becoming non-text.
> Binaries being segregated like they are is both inconvenient and made
> them easy to drop completely.

Sure. I agree with that. What I'm pointing out is *why* they were
segregated and why to this day you're going to have an extremely hard time
convincing anyone who doesn't have a lot of resources and a legal and
anti-abuse team to unsegregate them, or carry them at all.

(There are a bunch of other problems with Usenet being that one stop shop,
too, but we'll stick with that one for now.)

I also disagree that this is what killed Usenet in large part because the
phenomenon you describe about wanting non-text messages is newer than when
Usenet started running into trouble. I've not been around Usenet as long
as some of the folks here, but I've been using Usenet since 1993, and I
can tell you from personal experience that although copyright and legality
issues around binaries did show up early, Usenet had a mostly working way
of dealing with that and was still going very strong with widely-used text
discussion groups until the spamming (and off-topic trolling and other
types of unwanted messages that are even harder to moderate) took off.
That was the challenge that was happening about the time that other social
media platforms started taking off, and that's when usage of Usenet
started dropping and the signal to noise ratio started dropping even
faster.

That spam was directly the cause of Usenet's decline is just my opinion
and there are a few other viable theories. But most of them are some
version of Usenet being outcompeted by other protocols that people found
easier to use and less annoying for whatever reason, whether that be
better moderation (my theory), better *text* message formatting, better
client software, better topic organization, shorter messages, more
convenient access, cost, etc. Or of course maybe a combination of all of
those things.

But I'm dubious that it was *primarily* about non-text content because of
the timing. Usenet's problems started back in the late 1990s and very
early 2000s, long before the iPhone, which is when non-text chat started
taking off for the average person. There certainly was some demand for
photographs around the time that Facebook started, and Usenet has always
been bad at that (even apart from the separate binary groups, Usenet
*software* has always been bad), so maybe it was a factor, but I think it
was too early for it to be the main factor.

*Now*, *today*, I agree with you that this is a huge missing feature if
one wanted Usenet to compete with, say, Tiktok, although there is also a
huge list of other features that Usenet is missing, one of which is (for
all that people love to complain about the algorithms) adaptive moderation
so that people can very quickly filter out shit they don't want to see.

>> What killed Usenet was that it had no solution for spam that actually
>> worked for the average person, only complicated and weird filtering
>> experiments that never quite worked right.

> There isn’t a single platform without spam problems, so it is ludicrous to
> suggest that people abandoned Usenet for some spam-free social network.

The large commercial social media services have whole teams of people,
often thousands of people, who are actually paid (admittedly often very
poorly) to get rid of spam and abuse. Usenet had a handful of volunteers
and a janky cancel system and therefore had a spam problem that became
orders of magnitude worse than the user experience on other social media
services. People will tolerate a small amount of visible spam. Usenet
did not have a small amount of spam problem.

> Even so, there were tools that could have been brought to bear to
> greatly reduce the problem (Hello, UDP!), and I can only speculate on
> why the abuse wasn’t policed (insert your favorite conspiracy theory
> here).

Well, I was actually here then. UDP was used, and it didn't work. It
didn't greatly reduce the problem. The Usenet protocol makes that type of
enforcement nearly impossible, the amount of work required is
considerable, some sites that were sources of significant percentages of
the wanted articles were also sources of significant amounts of spam and
there was no consensus to cut them off, and what was able to be done was
completely unsustainable with a group of volunteers with no legal
protections and constant ongoing harassment. Due to the way Usenet works,
there was no governance system *capable* of making decisions; it was
essentially a free-for-all, which didn't work.

The same mechanism that was used for UDPs was also then used for denial of
service attacks, making it hard to run a service with those mechanisms
enabled, and no one managed to get an authenticated protocol really
working, in part due to the constant disagreement over who should have the
right to moderate Usenet. There were experiments with NoCeM (and indeed
there are ongoing experiments with NoCeM), but they never took off.

> Yep, and that’s why it was a mistake to not actually solve those
> problems decades ago.

I would also like to have cold fusion and perfectly efficient solar
panels. Doing spam control in a highly distributed system with no central
authority in the face of active harassment is very hard! Doing it with
almost no income stream to pay people is nearly impossible. Putting aside
the individual message moderation and focusing only on the necessary
adversarial software work, good platform abuse people are not cheap, and
it's not the sort of job that you do for free. It's not something you
implement once; it's something that you come in and do each day, every
day, against human adversaries who are adapting to your defenses. It's
not a *hobby*, which is part of why all the spam cancellers burned out.

It's quite hard even *with* the advantages that other social media
services have, such as central authentication and complete IP logging and
paid moderators. One could argue that a lot of them are still failing at
controlling spam, and Usenet is playing on a much harder mode.

[Russ Allbery]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:
> Russ Allbery <ea...@eyrie.org> wrote:

>> You would have to make all of your opponents permanently disappear,
>> and, well, good luck with that.

> We must be on different Internets. The one I’m on doesn’t have
> “opponents”. It has connections between myself and other people I want
> to engage with. What the hell kind of network have you set up for
> yourself that maintains connections to people who are out to do you
> harm?

Uh... Usenet? Usenet is that network? This is the entire way that it
works? People post articles that are flood-filled through a network?
They are therefore present on servers with no direct connection with the
person who posted it?

The whole point of Usenet is to talk about things with strangers.
Protocols for talking to strangers are inherently vulnerable to abusive
strangers and thus tend to have serious problems unless they're moderated
in some fashion. This is, like, social media 101. If I were only
connecting to people I want to engage with, I wouldn't have any of your
articles and we wouldn't be having this discussion. :)

https://xkcd.com/386/

More generally, this is inherently the challenge of *any* form of
community forum or commons. If something is intended to be open to
everyone by default unless they have done something wrong, some number of
those people will try to use it to do harm, and then one has to build some
sort of system that prevents or mitigates that harm by filtering it out or
removing those people. This is *inherently unavoidable*, and it's also
*inherently adversarial*. Whatever moderation system you build will be
tested by people who are trying to bypass it. And it's far more
challenging to write those systems on the internet where there is no
single authentication system (which would be bad to have for other
reasons, anyway) and it is trivially easy to create new identities.

>> But if you mean the cloud providers are happy about or actively
>> encourage people doing evil shit like CSAM on their platforms, this is
>> absolutely 100% not true and I know it's not true from direct personal
>> experience.

> My personal experience says otherwise. Nobody is kicking their paying
> customers to the curb unless they’re forced to.

Okay, well, this makes it clear to me that you have absolutely no idea
what you're talking about and are just making things up based on your own
prejudices. So probably much to everyone else's relief, I'll try to stop
responding here, and you can have the last words.

[Todd M. McComb]

In article <87cz0i8...@hope.eyrie.org>,
Russ Allbery <ea...@eyrie.org> wrote:
>... Usenet had a mostly working way of dealing with that and was

>still going very strong with widely-used text discussion groups
>until the spamming (and off-topic trolling and other types of
>unwanted messages that are even harder to moderate) took off.

I never found spam per se to be that big of an issue on Usenet --
although admittedly I had/have other people doing a lot of the work
on my behalf.

And you focus on "spam" in your further comments in the quoted post,
while I think these "off-topic trolling and other types of unwanted
messages that are even harder to moderate" is actually more on the
mark. The general anti-social behavior.

[Russ Allbery]

mcc...@medieval.org (Todd M. McComb) writes:

> And you focus on "spam" in your further comments in the quoted post,
> while I think these "off-topic trolling and other types of unwanted
> messages that are even harder to moderate" is actually more on the mark.
> The general anti-social behavior.

Yeah, I think you're right. Particularly by the classic Usenet definition
of spam, that was to some degree the easy bit.

I don't think it was really under control (people tend to have a much
lower tolerance than I do for spam, judging from people's reaction to
email spam amounts that I would find trivial), but even if it was, the
moderation problem was much harder to solve.

Other protocols went with either a reputation system, human moderators who
could delete stuff (as opposed to Usenet's awkward everything-or-nothing,
no take-backs, easy-to-bypass system), or using the social graph as a
moderation mechanism to filter what you see (or all three). There were
attempts at the first two on Usenet, but the Usenet protocol and the lack
of any central point of control makes it rather challenging.

[Todd M. McComb]

In article <874jlu8...@hope.eyrie.org>,
Russ Allbery <ea...@eyrie.org> wrote:
>... the Usenet protocol and the lack of any central point of control
>makes it rather challenging.

Before I go back to lurking, if only for others reading this, perhaps
it's worth pointing out that what brought someone like me to Usenet
(in the mid-80s) is exactly what amplifies these social issues:
Aggregation. So it's great when it's possible to draw from specialists
around the world in order to discuss a topic.... But we're "drawing
from" all the sociopaths out there too, not that they're any more
numerous than before, but they can all come here to disrupt others'
conversations for whatever reasons they may have (which aren't only
monetary...). In other words, it's a hobby & passion for many of
the troublemakers, with all the energy that implies, as I'm sure
you're aware.

(Still, I also don't agree with any implication that the public
made a good choice by abandoning this sort of system for for-profit,
top-controlled platforms. It was an expedient choice, and predictably,
isn't going well.)

[Richard]

[Please do not mail me a copy of your followup]

mcc...@medieval.org (Todd M. McComb) spake the secret code
<u9khe5$2ep$1...@hope.eyrie.org> thusly:

>I never found spam per se to be that big of an issue on Usenet --

It used to be, when lots of eyeballs were looking at usenet. Spam
follows the eyeballs. Because usenet is largley ignored now, the spam
problem has solved itself. Should usenet ever see a resurgence in
usage by the average person, the spam will resurge at that time.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Richard]

[Please do not mail me a copy of your followup]

mcc...@medieval.org (Todd M. McComb) spake the secret code

<u9kli3$4rs$1...@hope.eyrie.org> thusly:

>Before I go back to lurking, if only for others reading this, perhaps
>it's worth pointing out that what brought someone like me to Usenet
>(in the mid-80s) is exactly what amplifies these social issues:
>Aggregation. So it's great when it's possible to draw from specialists
>around the world in order to discuss a topic.... But we're "drawing

>from" all the sociopaths out there too [...]

Agreed. The traditional solution was a moderated group, e.g.
comp.lang.c++.moderated, to keep out the trolls. The C++ programming
newsgroup remains somewhat active, but has the problem of trolls and
nuiscance posters. We've discussed resurrecting the moderated group
from time to time, but the whole moderation infrastructure on usenet
appears to have fallen into disarray from lack of use.

[Todd M. McComb]

In article <u9m77h$2lkb9$4...@news.xmission.com>, Richard <> wrote:
>... the whole moderation infrastructure on usenet appears to have

>fallen into disarray from lack of use.

I'm one of the people who maintains the Usenet moderation system,
and I assure you that it does work as before. Prospective moderators
seem to have difficulty using the old (mostly shell-based) tools,
so it can be an issue to find someone, but it's not an infrastructure
problem.

[John]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:

> Usenet’s answer to that was, at first, “No, if you want to talk about cars
> you go to this one group, but if you want to post a picture of a cool car
> you saw at a show, you have to go to this other group (never mind the whole
> process of uploading and downloading) that has none of the conversation
> context. Oh, you have a *video* of the car, well now you need to go to an
> even *different* group and discover how much more of a pain large files
> are!” Then even that option went away.
>

I think it's a damn fool idea to flood the same 200MB video to 1,000
different Usenet servers on the off chance that somebody might be
reading alt.cars.ford.edsel and wants to watch it. Makes a lot more
sense to post a link, be it to Youtube or some other hosting option
(self-hosted if you're bold). If one wishes to be a grumpy protocol
contrarian there's no reason it has to be an HTTP link either, by all
means go wild and use FTP or gopher or 9p or *something* that makes more
sense for distributing large files than the Net News Transport
Protocol. Are links ephemeral? Yeah sure, but so are the expiration
rules on Usenet servers, especially once you start bringing in binary
posts.

Your vaunted web forums usually don't host their own images or videos
either, or if they do host images they are much restricted (1 image per
post, max 200KB, only viewable by signed-in members etc.) Reddit famously
outsourced image hosting to imgur and videos to sites like
gfycat. Something Awful has image uploads but you can only do one per
post, which led people to use outside hosts like Waffleimages (which
went down and broke tons of threads) or photobucket (which changed its
policies and broke tons of threads) or imgur (which deleted a bunch of
images and broke tons of threads).


john

[Julien ÉLIE]

Hi Jesse,

> GigaNews's AS and associated prefixes are currently only announced from Deft's
> network (out of Chicago from my perspective) and the two IXs in Ashburn VA and
> Frankfurt Germany. It seems very recently they lost many IP peers and prefix
> announcements, but maybe they sold off old parts of the business unrelated to
> GigaNews. From a network/service provider perspective, they are the most
> opaque of the bunch operating in the USA in terms of exactly how they operate,
> and they haven't answered an e-mail sent to their peering address in over 4
> years from myself and other operators I am in contact with attempting to peer
> or adjust feeds with them.

Strange that the Giganews folks don't even bother responding to their
peers who ask them some feed adjustement. They claim to have a 24/7
support team... but not for their peers! Maybe they should be contacted
via their support team instead of their peering address?

FWIW, in relation with this thread, they are using a special 451 NNTP
response code when an article cannot be found because it was removed as
part of a DMCA request:

https://support.giganews.com/hc/en-us/articles/9952660678285-What-are-errors-430-and-451-


This discussion is pretty interesting, and helped in improving my
knowledge of the states and cities of the USA :)

--
Julien ÉLIE

« – Connaissez-vous la différence entre l'ignorance et l'apathie ?
– J'en sais rien et je m'en fous. »

[The Doctor]

In article <ua5iv6$ctmb$1...@news.trigofacile.com>,

How sad!

>--
>Julien ÉLIE
>
>« – Connaissez-vous la différence entre l'ignorance et l'apathie ?
> – J'en sais rien et je m'en fous. »

--
Member - Liberal International This is doc...@nk.ca Ici doc...@nk.ca
Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising!
Look at Psalms 14 and 53 on Atheism https://www.empire.kred/ROOTNK?t=94a1f39b
If they have not admitted their failures, they cannot have learned from them. -unknown Beware https://mindspring.com

[Syber Shock]

On Sun, 30 Jul 2023 01:26:46 -0000 (UTC)
Doc O'Leary , <drol...@2017usenet1.subsume.com> wrote:

> I’m not trying to convince anyone of anything, excepting to have a
> technical discussion on what it will take to turn Usenet into a
> modern service.

Something like this is actually happening in the newsgroup,
rocksolid.nodes.help. They are devising encrypted messaging via NNTP
that obfuscates sender from public disclosure but with less complexity
than a mixnet.

Have a look see at the last two weeks of messages in
rocksolid.nodes.help. Two hackers are turning the Rocksolid Light engine
into a quasi-social platform with some social features and encrypted
messaging built-in, on a NNTP backend. I'm sure they would appreciate
some more PHP coders and crypto-chango artists helping it along.

Think of the end goal as 'fediverse' without the 'fed' and without the
PC censorship so prevalent on the fediverse.

SugarBug | https://sybershock.com

[John Levine]

According to Doc O'Leary , <drol...@2017usenet1.subsume.com>:
>For your reference, records indicate that

>John <jo...@building-m.simplistic-anti-spam-measure.net> wrote:
>
>> I think it's a damn fool idea to flood the same 200MB video to 1,000
>> different Usenet servers on the off chance that somebody might be
>> reading alt.cars.ford.edsel and wants to watch it.
>

>I agree. As I have said in other posts, it points to the fact that
>store-and-forward is no longer a practical way to build a distributed
>network. It may have made sense in the early days of Usenet when
>reliable connectivity wasn’t a given. We’re at least a decade past when
>that was the norm.

The people who run CDNs would likely disagree with you. Sometimes it
makes sense to store copies of the content near the users, sometimes
more sense to have everyone fetch it from one place.

E-mail has the same problem, when someone sends a message with a large
attachment to a distribution list. It would make a lot more sense to
send a link that can be fetched as needed. Back in 1996 RFC 2017 defined
the MIME message/external-body type intended to do that, but it never
caught one. People I know in the IETF are thinking about updating it, with
more modern URLs, and also a hash so you can be sure the file you fetch is
the one you expect, and a decoding key so the file can be stored encrypted.

If they do that, which seems likely, it wouldn't be hard to retrofit to
usenet since we use the same MIME types that mail does.

[Jesse Rehmer]

On Aug 5, 2023 at 12:36:38 PM CDT, "Doc O'Leary ,"
<drol...@2017usenet1.subsume.com> wrote:

> I still don’t understand why people act like NNTP cannot be updated to
> something that actually supports “links” internally. I mean, really, all
> we’re talking about here is the idea that a server can refer to a message
> ID that it may not be able to provide. If it simply tracked a “source”
> server, it could either then later store-and-forward on request, or
> instruct the client to directly fetch it from that server itself.
>
> I’m all ears for better solutions than that, too. What I’m tired of
> hearing is “Oh, Usenet didn’t do that in 1998, so it can’t be done.”

This is essentially what the commercial providers do with resellers. Some
resellers have their own frontends, but they simply point to the source
provider's backends. What type of agreements and how they "bill" for this kind
of access I have no idea. If you take a look at Diablo's dreaderd code/configs
it makes it easy to setup frontends that fetch articles by Message-ID from
distributed backends.