Conference about Usenet, federation and moderation

[Julien ÉLIE]

Hi all,

To share a video I've just came across about "Usenet as the original
decentralized social network".
By Rayner Lucas and Tristan Miller at a LibrePlanet conference in March
2023. Thanks to both of you for this video!

https://framatube.org/w/97acbef0-dd05-45d4-a1df-c8ac9cbe36f0


Indeed, fighting spam and abuse is a daily challenge.

And it's not easy to improve the "poor support for multimedia" and "hard
for new users to find it" problems. We would have to guess the first
steps to do about that...

--
Julien ÉLIE

« – I see the world didn't end yesterday.
– Are you sure? » (Alan Moore, _Watchmen_)

[Michael Uplawski]

Julien ÉLIE wrote in news.software.nntp,news.misc:

> https://framatube.org/w/97acbef0-dd05-45d4-a1df-c8ac9cbe36f0

(...)

> And it's not easy to improve the "poor support for multimedia" and "hard
> for new users to find it" problems. We would have to guess the first
> steps to do about that...

I admit that I do not like the format of this video. Can you tell me what makes
you want to improve about multimedia in Usenet and in which way you think this
should be done?

If I had to guess steps to improve the multimedia capabilities of Usenet, I'd
say: *None*.

The World-Wide-Web exists. Maybe the protocol is rotten beyond repair, maybe we
left it to the industries to destroy it and maybe the abyss of antisocial
networks and video platforms is so unavoidable that you feel forced to sell
your soul to them.

Technically, nothing impedes your regaining pocession of a technology that is
(only) dominated by unnecessary transnational companies.

Do not pass on the problem to Usenet.

Cheerio

f'up news.misc

[Marco Moock]

Am 08.07.2023 um 09:42:23 Uhr schrieb Julien ÉLIE:

> And it's not easy to improve the "poor support for multimedia" and
> "hard for new users to find it" problems.

Getting new users means it must be findable via normal web searches.

Something like narkive.com, but with posting possible.

[Adam H. Kerman]

Marco Moock <mo...@posteo.de> wrote:

>Am 08.07.2023 um 09:42:23 Uhr schrieb Julien:

>>And it's not easy to improve the "poor support for multimedia" and
>>"hard for new users to find it" problems.

>Getting new users means it must be findable via normal web searches.

>Something like narkive.com, but with posting possible.

I don't agree. Google has been doing that ever since they bought out
Deja News. Usenet Article Format and Usenet conventions are different
enough from Web pages that the search fails to parse the article in a
useful manner and the person performing the search doesn't get useful
results. Conventional Usenet articles have extensive quoting that
confuse pattern matching, and plenty of users quote just differently
enough from other users that it's a mess.

Gatewaying to and from Usenet from another medium of communication is
UNSUCCESSFUL. The other medium has its own quirks and conventions that
work poorly on Usenet. Usenet followups require followups. Generally, a
Web forum shouldn't have quoting unless the reply is to a comment
further back. A web forum is flat; Usenet is threaded.

Even gatewaying to and from mailing lists is unsuccessful. I do
Usenet-style quoting in reply to an email message. Nearly no one else
does.

You're kind of arguing that gatewaying will increase the poast count and
that will save Usenet. That's just traffic for the sake of traffic but
that's not more Usenet discussion.

What we need are users willing to learn to use a threading newsreader and
subscribe to a News server who want to hold discussions with other
people using newsreaders and not Web gateways. A Web interface to Usenet
WITH POSTING will not result in a conventional Usenet article. A gateway
from a Web forum will not result in Usenet-style discussion.

There's nothing wrong with having different media of discussion, with
different conventions, different formats, and different methods of
access. Usenet's benefit is that our communication method works quite
well for what we do. Making it as Web-forum-like as possible will ruin
Usenet's inherent advantages.

[Marco Moock]

Am 09.07.2023 um 14:40:20 Uhr schrieb Adam H. Kerman:

> I don't agree. Google has been doing that ever since they bought out
> Deja News. Usenet Article Format and Usenet conventions are different
> enough from Web pages that the search fails to parse the article in a
> useful manner and the person performing the search doesn't get useful
> results. Conventional Usenet articles have extensive quoting that
> confuse pattern matching, and plenty of users quote just differently
> enough from other users that it's a mess.

A web gateway could have threading, Google just didn't implement it.

If I look for content, I regularly get results from Google Groups and
narkive.com.

rocksolid is another software that has a news2web gateway.

Most internet users only know a web browser.
To find the Usenet, they need to find interesting information.
The next step is to make them using an NNTP software.

[Marco Moock]

Am 09.07.2023 um 14:29:59 Uhr schrieb Doc O'Leary ,:

> On the contrary, it is almost trivially easy. The only real problem
> is that binary groups got some people upset right around the time
> that broadband was exploding. While it can certainly be argued that
> encoded multipart binaries are not the best way to deal with large
> data files, the RFCs haven’t been updated in something like 25 years.
> Consequently, neither have many of the clients, and that’s where 99%
> of the work has to be done to support non-text messages.

Why do people feel disturbed by them?
They don't need to subscribe to them and news servers need to make sure
that binary attachments cannot be posted.
That's all.

[Adam H. Kerman]

Marco Moock <mo...@posteo.de> wrote:
>Am 09.07.2023 um 14:40:20 Uhr schrieb Adam H. Kerman:

>>I don't agree. Google has been doing that ever since they bought out
>>Deja News. Usenet Article Format and Usenet conventions are different
>>enough from Web pages that the search fails to parse the article in a
>>useful manner and the person performing the search doesn't get useful
>>results. Conventional Usenet articles have extensive quoting that
>>confuse pattern matching, and plenty of users quote just differently
>>enough from other users that it's a mess.

>A web gateway could have threading, Google just didn't implement it.

We've had decades of experience with Web gateways. Yes, in theory, they
could possibly produce a conventional article. Nevertheless, in the real
world, there are no examples of gateways that produce a conventional article.

>If I look for content, I regularly get results from Google Groups and
>narkive.com.

There's no convenient way to limit the search to just Usenet. Deja's
search method was adequate. Google's has been dreadful. To search for a
Usenet article, you need the search to be set up to look for headers and
to be able to distinguish between header and body.

Conflating a Web and Usenet search will give the user lousy results,
particularly as the parser cannot identify the quote and NOT provide the
quote as a result. And then you'll be led to an interface that won't
thread.

I'm sorry but this will not attract a new user because it's so ugly.

>rocksolid is another software that has a news2web gateway.

>Most internet users only know a web browser.
>To find the Usenet, they need to find interesting information.
>The next step is to make them using an NNTP software.

When I first used Usenet, I tried pine, but its Usenet presentation was
lousy. I then learned to use a threading newsreader. I'm sorry but
it wasn't a barrier. If my first experience with Usenet was via a Web
interface, I would have been entirely turned off.

A threading newsreader is key to a decent experience. The Web interface
doesn't provide that.

As you point out long-standing gateways, I'm not aware that any
significant number of their users learned to post via newsreaders.
Instead, they appear to be people who were using Usenet who use the Web
interface because of technology limitations with cell phone and their
personal dislike of laptop or desktop computers.

We already know what hasn't been attracting new users to Usenet.

[Marco Moock]

Am 09.07.2023 um 15:37:50 Uhr schrieb Adam H. Kerman:

> We already know what hasn't been attracting new users to Usenet.

I was one of them.
Without finding articles via narkive.com, I would never post here.

[Adam H. Kerman]

I'll note you didn't state that you had used a non-site-limited Google search.

[John]

Doc O'Leary , <drolear...@2023.impossiblystupid.com> writes:

> For your reference, records indicate that

> =?UTF-8?Q?Julien_=c3=89LIE?= <iul...@nom-de-mon-site.com.invalid> wrote:
>
>> Indeed, fighting spam and abuse is a daily challenge.
>

> Only inasmuch as people don’t *actually* want to take the steps needed to
> solve the problem. The UDP was a rare thing, but cutting off hostile
> networks should be one of the first steps in eliminating abuse.
>

If anything deserves a UDP it's Google Groups.


john

[John Levine]

According to Doc O'Leary , <drolear...@2023.impossiblystupid.com>:
>In general, actual *people* aren’t disturbed. It’s the *powers* that are
>disturbed. Usenet is a content distribution network that isn’t beholden
>to capitalistic principles. Rather than figuring out a way to use it to
>their advantage, they rolled out the legal threats and destroyed it.

Hi, I've managed news servers since the 1980s. I never carried many of
the binary newsgoups for entirely practical reasons. They were (and
are) enormous which in the era of slow, often dialup, transfers tied
up phone lines for hours on end and filled up the small disks we had.
The contents were mostly pictures of nude women and hacked computer
games, neither of which were of much interest to my users.

While there is plenty not to like about the way copyright law works,
that doesn't mean you get to ignore it. Before the DMCA was passed in
1998 it was entirely unclear what liability a server operator had for
pirate content other people put on our servers. Since we generally
were running our servers in our spare time with a legal budget of $0,
taking risks for stuff our users didn't care about would have been
idiotic. So we didn't. Once people started to get dialup or direct
connections to the Internet, they had a lot better ways to download
binary stuff than multipart uuencoded news messages.

I realize that conspiracy theories are more fun, but sometimes
life is boring and practical.

--
Regards,
John Levine, jo...@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

[Russ Allbery]

Doc O'Leary , <drolear...@2023.impossiblystupid.com> writes:

> In general, actual *people* aren’t disturbed. It’s the *powers* that
> are disturbed. Usenet is a content distribution network that isn’t
> beholden to capitalistic principles. Rather than figuring out a way to
> use it to their advantage, they rolled out the legal threats and
> destroyed it.

A caveat to this analysis is CSAM (child sexual abuse material). (And a
few related nonconsentual things like revenge porn and torture videos, but
CSAM is the most obvious and straightforward to analyze.) This problem is
unfortunately used by the most authoritarian assholes in government as an
excuse to do all sorts of nasty surveillance state shit, so it's very
tempting to decide the whole problem is imaginary and was invented by
wannabe tin-pot dictators to try to get vast police powers. But
unfortunately if you work for any length of time in or around abuse
prevention, you very quickly realize that it's also a real problem. There
are indeed organized groups of peoople who are trying to trade CSAM and
will use your service to do it if you let them, and there's real abuse of
real people underneath it.

Every single service that lets people trade file formats that can be used
for CSAM eventually has to come to terms with this and figure out how
they're going to handle it. And regardless of what you personally may
feel about how the law *should* handle this problem, the rest of society
has really strong opinions about it for rather understandable reasons and
in most countries has set a bunch of rules for how you WILL handle it
whether you like it or not, often backed up by the threat of actual
criminal prosecution, not just the copyright cartel nonsense.

I mostly stay out of these discussions because I ended up saying pretty
much everything I had to say back in 2005 or so and burned out on them,
but people who haven't worked in platform abuse regularly drastically
underestimate the amount of really dark and disturbing shit that people
try to use services to distribute *and make* and assume that the usual
suspects are just lying about it to scare people. And don't get me wrong,
they do routinely lie about things to scare people. But this one is worth
thinking about independently and figuring out how you are going to deal
with, because getting in the middle of that is really bad. Legally, yes,
but also seeing stuff that you will really want to unsee.

> Kind of off topic, but it has been amusing to watch the saga of how the
> alternative have played out in a way that still did not benefit the
> content creators. Musicians complain about the pennies they make from
> streaming. Writers and actors are both on strike now because Hollywood
> I s being Hollywood. And the places providing porn are still under
> attack from the morality police, as the woes of PornHub and others in
> Utah and beyond are many. It almost makes it seem like the problem
> never *was* Usenet at all! :-/

The problem certainly was never Usenet. Usenet has always been a bit of a
backwater, and a lot of this stuff is indeed capitalist bullshit or
prudish nonsense. But Usenet is also not somehow immune from the more
serious problems that do exist.

The absolute libertarian position is very tempting here. For a long time
I too was persuaded by it, and I think it does work up to a certain scale.
But the world is very large and full of people and a tiny fraction of
those people want to do some seriously evil shit and will use your servers
to do it if you let them, and you do need to have some sort of plan to
stop them unless you are willing to support a type of social free-for-all
that 99.99% of humanity is not going to be willing to tolerate.

--
Russ Allbery (ea...@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

[Russ Allbery]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:

> Russ Allbery <ea...@eyrie.org> wrote:

>> There are indeed organized groups of peoople who are trying to trade
>> CSAM and will use your service to do it if you let them, and there's
>> real abuse of real people underneath it.

> So you put in the effort to *solve* that problem, not sweep it under the
> rug, which is all you accomplish when you start deleting data.

The specific thing you are required to do by US law (in my personal
understanding, I am not a lawyer, this is not legal advice for your
specific situation, read it for yourself at [1]) is report the data to
NCMEC and then make it inaccessible, not delete it (which would be
destroying evidence), so in that narrow sense I do actually agree with
you. I suspect this is similar in most other jurisdictions, although
obviously NCMEC is a US thing so the details will vary.

[1] https://www.law.cornell.edu/uscode/text/18/2258A

However, I personally would rather juggle raw plutonium than spend any
time handling that kind of legal evidence and therefore opt out of the
entire problem by not carrying binaries, since otherwise I am legally
obligated to spend whatever time it takes me to handle that data properly
should any problem arise. Since personally I don't care about any of the
binaries anyway (there are numerous better sources for any non-textual
information I want than Usenet), this seems like it would be a very bad
use of my time, even apart from the fact that if I ever got a report I
would have to go look at the reported data to see if the report was
correct and that does not sound fun. Likewise for DMCA, which is less
legally fraught but still highly annoying.

Other people's mileage may vary, and that's fine! Just know what you're
getting into and realize that no one involved in enforcing laws cares in
the slightest what your opinion is of those laws, so it's worth carefully
picking what types of civil disobedience you want to engage in.
Personally, I vote against copyright bullshit but don't engage in civil
disobedience around it. That's where my personal risk tradeoff is.

> My point is always going to come back to the fact that few people are
> actually interested in solving abuse on *any* platform.

Well, I have spent a bunch of time working professionally with people who
are trying to reduce platform abuse and I have a huge amount of respect
for the work that they do. I don't think anyone who works professionally
in the field thinks this problem is *solvable*. It's a classic
adversarial security problem, and those are almost never solvable. You
would have to make all of your opponents permanently disappear, and, well,
good luck with that. Governments have been trying for millennia. The
best you can manage is harm reduction.

You can do various things to make it easier and various things to make it
harder. One of the most effective things you can do to make dealing with
abuse easier is to ban all non-textual media, because that takes a lot of
the most annoying, dangerous, or horrific types of abuse off the table.
Obviously, that's a tradeoff, and if everyone made that tradeoff that
would be sad for society since there are a lot of non-textual things that
are worth sharing. But I leave hosting the non-textual stuff to people
with platform abuse teams and lawyers on retainer. Again, that's where my
risk tradeoff is.

I think people who think fully decentralized annoymous (or even mostly
anonymous) non-textual file sharing is something they want to get involved
in are completely out of their gourd because you're just hanging up a
giant sign saying "trade all of your illegal shit here, viruses welcome,"
but I'm not the government and I'm not going to stop you. My only goal
here is to make sure people at least go into it with their eyes open.

> But it’s worse than that! It’s gone the other way: cloud platforms have
> business models where the profit motive is in *supporting* that evil
> shit.

I wanted to say that this is definitely not true, but I think I can see
how one might see that this is true from a particular angle. It is true
that in pursuit of profits, a bunch of companies have built network
platforms that make abuse much easier, and are now desperately trying to
play catch-up to filter out the shit that they don't want to carry. There
is an interesting argument to be made that social media itself is the
problem and we should not have any social media at all because it enables
abuse. I don't think I *agree* with that argument, but one can certainly
make that argument coherently, and it's gaining some social popularity.
(This is what repealing section 230 in the US would mean: make social
media effectively illegal by making every service provider liable for
everything they carry. And there are a fair number of advocates for that,
although not all of them understand what they're advocating.)

But if you mean the cloud providers are happy about or actively encourage
people doing evil shit like CSAM on their platforms, this is absolutely
100% not true and I know it's not true from direct personal experience.
Cloud platforms spend large quantities of money, hire whole teams of very
expensive people, and write whole new algorithms and scanning methods to
try to get rid of shit like CSAM. It's a significant expense; it is
absolutely not a profit center. They do that in part because people who
work for cloud platforms are human and have normal human feelings about
CSAM, in part because it's a public relations nightmare, and in part
because not doing some parts of that work is illegal.

[Russ Allbery]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:

> John Levine <jo...@taugh.com> wrote:

>> While there is plenty not to like about the way copyright law works,
>> that doesn't mean you get to ignore it.

> Neither do lawyers get to ignore the realities of math. And math always
> wins, because it isn’t fabricated the way laws are.

Standing in court yelling MATH WILL ALWAYS WIN is very emotionally
satisfying, but weirdly it doesn't make the court judgment go away. Maybe
the lawyers won't be able to ignore the realities of math forever, but
they do in fact get to ignore the realities of math long enough to tell
the men with guns to go take your money.

> What killed Usenet is that it wasn’t practical any longer. No grand
> conspiracies from me. People simply wanted binaries, even if only to
> share family photos and pet videos. When they got a “no”, they didn’t
> even care that you were hiding behind your lawyers and suggesting they
> might be thieves, they just heard the “no” and left.

This is a bizarrely confused history of Usenet. The binary groups were
going strong for years after the text groups were dying. What killed
Usenet was that it had no solution for spam that actually worked for the
average person, only complicated and weird filtering experiments that
never quite worked right.

Spoiler: Still has exactly the same problems. Usenet is just too dead for
spammers to care about it (mostly). If it were ever revived, the same
problem would immediately come back.

[Richard Kettlewell]

Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:
> John Levine <jo...@taugh.com> wrote:

>> According to Doc O'Leary , <drolear...@2023.impossiblystupid.com>:
>> >In general, actual *people* aren’t disturbed. It’s the *powers* that are
>> >disturbed. Usenet is a content distribution network that isn’t beholden
>> >to capitalistic principles. Rather than figuring out a way to use it to
>> >their advantage, they rolled out the legal threats and destroyed it.
>>
>> Hi, I've managed news servers since the 1980s. I never carried many of
>> the binary newsgoups for entirely practical reasons. They were (and
>> are) enormous which in the era of slow, often dialup, transfers tied
>> up phone lines for hours on end and filled up the small disks we had.

Like John, when I was operating NNTP service for anyone other than
myself, we excluded binaries because they were too expensive to
carry. The CSAM risk was mitigated by subscribing to a service which
notified us of illegal material so we could delete it.

Text groups carried legal risks too, at the time, but that didn’t stop
us carrying text Usenet.

Meanwhile AFAIK binary groups still exist today, there’s just a
relatively limited set of providers who carry them. I don’t know how
they escape being sued into obvlivion by copyright holders.

> That simply speaks to the problem if scaling the network, which has
> nothing to do with any particular data format or content. If
> store-and-forward is the wrong solution, what is the right solution?
> What about a caching proxy for just-in-time delivery of messages?
> It’s kinda moot to have this discussion now, though, because we’ve had
> 20 years of alternative protocols establishing themselves.

Those alternative protocols suggest that the solution is for end users
to download from centralized services (sometimes through a caching
proxy, indeed).

>> I realize that conspiracy theories are more fun, but sometimes
>> life is boring and practical.
>

> What killed Usenet is that it wasn’t practical any longer. No grand
> conspiracies from me. People simply wanted binaries, even if only to
> share family photos and pet videos. When they got a “no”, they didn’t
> even care that you were hiding behind your lawyers and suggesting they

> might be thieves, they just heard the “no” and left. If you want them
> to return, you have to formulate a more practical answer than “still
> no”.

People left Usenet because of the spam and trolling, not because they
couldn’t use it to get hold of pirated video games and movies.

--
https://www.greenend.org.uk/rjk/

[Jesse Rehmer]

On Jul 20, 2023 at 1:21:06 AM CDT, "Russ Allbery" <ea...@eyrie.org> wrote:

> But if you mean the cloud providers are happy about or actively encourage
> people doing evil shit like CSAM on their platforms, this is absolutely
> 100% not true and I know it's not true from direct personal experience.
> Cloud platforms spend large quantities of money, hire whole teams of very
> expensive people, and write whole new algorithms and scanning methods to
> try to get rid of shit like CSAM. It's a significant expense; it is
> absolutely not a profit center. They do that in part because people who
> work for cloud platforms are human and have normal human feelings about
> CSAM, in part because it's a public relations nightmare, and in part
> because not doing some parts of that work is illegal.

Maybe you aren't aware but numerous claims have been made over the years about
service providers profiting from CSAM specifically, including a Usenet Service
Provider whom straddles both sides of profit and cooperation with law
enforcement.

https://cryptome.org/2014/09/giganews-fbi.htm

Having worked in the Service Provider industry my entire career, whether or
not you believe the particular account cited above, it is not far-fetched
based on my experiences.

[Jesse Rehmer]

On Jul 20, 2023 at 2:37:07 AM CDT, "Richard Kettlewell"

<inv...@invalid.invalid> wrote:

> Doc O'Leary , <drol...@2017usenet1.subsume.com> writes:
>> John Levine <jo...@taugh.com> wrote:
>>> According to Doc O'Leary , <drolear...@2023.impossiblystupid.com>:
>>>> In general, actual *people* aren’t disturbed. It’s the *powers* that are
>>>> disturbed. Usenet is a content distribution network that isn’t beholden
>>>> to capitalistic principles. Rather than figuring out a way to use it to
>>>> their advantage, they rolled out the legal threats and destroyed it.
>>>
>>> Hi, I've managed news servers since the 1980s. I never carried many of
>>> the binary newsgoups for entirely practical reasons. They were (and
>>> are) enormous which in the era of slow, often dialup, transfers tied
>>> up phone lines for hours on end and filled up the small disks we had.
>
> Like John, when I was operating NNTP service for anyone other than
> myself, we excluded binaries because they were too expensive to
> carry. The CSAM risk was mitigated by subscribing to a service which
> notified us of illegal material so we could delete it.
>
> Text groups carried legal risks too, at the time, but that didn’t stop
> us carrying text Usenet.
>
> Meanwhile AFAIK binary groups still exist today, there’s just a
> relatively limited set of providers who carry them. I don’t know how
> they escape being sued into obvlivion by copyright holders.

I can tell you one VERY interesting FACT about all commercial Usenet providers
in the United States - their servers and infrastructure handling Usenet feeds
are located in Ashburn Virginia, in facilities near the NSA.

[Marco Moock]

Am 20.07.2023 um 13:24:19 Uhr schrieb Jesse Rehmer:

> I can tell you one VERY interesting FACT about all commercial Usenet
> providers in the United States - their servers and infrastructure
> handling Usenet feeds are located in Ashburn Virginia, in facilities
> near the NSA.

Is there a specific (legal) reason for that?

[Adam H. Kerman]

Lots of communications infrastructure and power to serve Fort Meade?
Seems like the right place to site a server farm.

[Richard]

[Please do not mail me a copy of your followup]

=?UTF-8?Q?Julien_=c3=89LIE?= <iul...@nom-de-mon-site.com.invalid> spake the secret code
<u8b40v$30nta$1...@news.trigofacile.com> thusly:

>Indeed, fighting spam and abuse is a daily challenge.

The only way to combat this is to sign articles and establish trust
networks and deny spammers membership in the trust networks. Note
that this does not preclude anonymity.

>And it's not easy to improve the "poor support for multimedia"

This is a solved problem for decades (MIME). What's always been
missing is good newsreader support and a cultural willingness to
embrace content beyond text/plain. IMO, the latter is the big
stumbling point; it's a cultural issue not a technological one.

>and "hard for new users to find it" problems.

This is a problem? :)

Being hard to find keeps out the rugrats.
--
"The Direct3D Graphics Pipeline" free book <http://tinyurl.com/d3d-pipeline>
The Terminals Wiki <http://terminals-wiki.org>
The Computer Graphics Museum <http://computergraphicsmuseum.org>
Legalize Adulthood! (my blog) <http://legalizeadulthood.wordpress.com>

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> Maybe you aren't aware but numerous claims have been made over the years
> about service providers profiting from CSAM specifically, including a
> Usenet Service Provider whom straddles both sides of profit and
> cooperation with law enforcement.

> https://cryptome.org/2014/09/giganews-fbi.htm

I should be clear: I am not saying that absolutely no service provider
anywhere has ever decided to try to profit from CSAM. Obviously I can't
make a claim like that, particularly given that 8chan and company exists.
I personally know nothing about Giganews and would be a fool to comment.

The world is large and full of people. I'm sure just about every awful
thing anyone can think of has been attempted by someone at one point or
another.

My point is only that the idea that Amazon, Microsoft, Google, Meta,
Apple, etc. (the companies that most people think of when someone says
"cloud provider" which was the term used in the message I was replying to)
have a business model built on CSAM is directly contradicted by the very
real and large expenditures by those same companies to try to get that
stuff off their platforms as much as they possibly can given other
constraints they have around promises of user privacy, etc.

The shape of their problem is instead that they have been wildly
successful beyond any of their reasonable expectations and with that level
of scale comes a level of platform abuse that they were wholly unprepared
originally to deal with. They're still desperately digging themselves out
of that hole and occasionally falling into it again. But if they could
somehow make the problem go away, they would *love* to do so.

One tension they have is that a lot of people also (quite understandably)
feel pretty weird, at best, about their cloud provider saying "oh, we're
going to scan everything you do to see if any of it would be of interest
to the cops," which means they would strongly prefer to be reactive rather
than proactive, but some of these groups are quite sophisticated and find
new ways of hiding from reactive scanning techniques. Another tension
they have is that in a lot of cases they would like to promise end-to-end
user privacy because it's a selling point, but end-to-end user privacy
from the cloud provider means they by definition cannot scan or do
anything else about whatever is crossing that channel, so they end up in a
three way fight between privacy advocates, the government, and the news
media. (And, to be clear, usually manage to say the stupidest possible
things and step on five rakes in the process.)

It's an incredibly tricky area and I certainly don't have all of the right
answers. All I know is that (a) Usenet is a hobby for me and I'm not
going near this mess with a ten foot pole and would strongly recommend
against anyone else doing so either unless you have the money for lawyers
and a real anti-abuse team, and (b) the idea that cloud providers in
general benefit from CSAM seems like bullshit to me given the amount of
money they spend on trying to get rid of it and trying to control the
public relations fallout from the bits they didn't manage to get rid of.
Even if the balance of money is slightly towards profit (which I highly
doubt), it's UTTERLY dwarfed by their legitimate business and directly
threatens it, so it is certainly not a motivating force for the typical
cloud provider.

[Jesse Rehmer]

I shouldn't have painted with such a broad stroke, as I didn't mean to imply
that the large cloud providers mentioned had such business models or
interests, but the service provider realm is broad and complicated.

In the early 2000s I worked for SAVVIS (now Lumen) and at the time they were
harboring a lot of very bad actors within their hosting platform as well as
providing IP transit services to such entities, mostly stemming from the
acquisitions of Cable & Wireless and Exodus/MCI, but it was a bigger spectrum
than the "spammers" that were represented in the media at the time.

I was not directly involved, but worked on the Hosting help desk while this
was ongoing, and know that we had the knowledge, capabilities, and man power
to shut them all off within a few hours, but the higher ups let it drag out
for a year or more before finally terminating contracts. It wasn't until it
started to hit the news and they began receiving external pressure that they
changed their tune, and very slowly terminated the bad actors. I realize this
is not exactly the same scenario as CSAM, but I've witnessed decisions made at
executive levels that chose profits over doing what is legal or right.

Service Providers in the USA have immense protections, perhaps a lot more than
they should, and we're going through another phase where that is being
re-examined and up for litigation. Having been somewhat of a man-in-the-middle
of all things related to Internet and Society - I'm not exactly sure where I
stand on most of it either.

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> I was not directly involved, but worked on the Hosting help desk while
> this was ongoing, and know that we had the knowledge, capabilities, and
> man power to shut them all off within a few hours, but the higher ups
> let it drag out for a year or more before finally terminating
> contracts. It wasn't until it started to hit the news and they began
> receiving external pressure that they changed their tune, and very
> slowly terminated the bad actors. I realize this is not exactly the same
> scenario as CSAM, but I've witnessed decisions made at executive levels
> that chose profits over doing what is legal or right.

> Service Providers in the USA have immense protections, perhaps a lot
> more than they should, and we're going through another phase where that
> is being re-examined and up for litigation. Having been somewhat of a
> man-in-the-middle of all things related to Internet and Society - I'm
> not exactly sure where I stand on most of it either.

Yeah, this is all very fair and I agree with everything you say here. The
smaller the provider, the more likely that the response to platlform abuse
will be... shall we say, random. Some companies will take immediate
action; some companies will avoid doing anything for as long as possible.

Perhaps I am too optimistic, but I think in *most* cases the lack of
action is some combination of the fact that dealing with this stuff
requires spending money that doesn't make any profits and small companies
are more likely to feel like this is unfair and be very grudging about
doing it, plus the general human tendency to believe that any problem that
one doesn't want to deal with can't possibly be as bad as people claim it
is. I think actual malice is rather rare, but cutting corners and not
spending money where it seems avoidable is very common.

There are also, of course, folks with strong libertarian principles,
probably more common among the sort of folks who often start small
businesses, and as long as the argument feels like it's on the level of
abstract principles, it's easy to decide that the libertarian principles
should win. (Again, maybe I'm too optimistic, but I don't think this
tends to survive direct contact with the worst types of platform abuse.
But thankfully that sort of abuse is relatively rare, so the debate
sometimes stays abstract for a long time.)

The larger companies do not have the luxury of ignoring this sort of
problem and already have frequent contact with law enforcement due to the
simple fact that people live large amounts of their lives on cloud
providers these days, so legal investigations constantly go there.
They're more likely to have thought somewhat seriously about this and have
employees whose job it is to figure out what to do, although even in those
cases they tend to be underresourced compared to the size of the problem.

[Jesse Rehmer]

In my personal experience I find that companies which are controlled by a
Board of Directors generally lean towards doing and spending as little as
possible in order to retain valuable customers or prevent legal penalties, and
have no issue directing those below to ignore their conscience.

The smaller or privately held providers whom are controlled by founders or
'techies' tend to take such matters more seriously and with a lot more
conscience (unless their only business model is the trouble makers).

It all depends on who is at the top and what sort of conscience they have, or
don't, I suppose.

[Jesse Rehmer]

I have no substantial evidence to support what I'm about to throw out here,
and it is my own wild speculation.

There are far cheaper geographical locations in the USA to obtain reliable
power, IP/transit, and secure raised floor space than Ashburn VA. US-based
commercial Usenet providers' corporations are not based out of Virginia (last
I checked most were based out of Florida and one in Texas). In the USA, before
full-feed Usenet turned into a commercial venture, providers were spread out
and their infrastructure lived close to their headquarters. To my knowledge
(in this area it is more limited), there are no longer any non-commercial
entities exchanging a full Usenet feed in the USA, perhaps apart from a few
research facilities.

Don't you think it is highly curious that service providers whose profits
center around the exchange of copyrighted material and pornography are all
located in the one geographical location in which the US government
semi-openly admits is their largest spy point?

(Let's not kid ourselves, I know the full-feed Usenet operators are swimming
in such content, and I don't believe anyone is paying for Usenet access to
text-based discussion or share family photos.)

I'm not convinced the US government doesn't want Usenet and the commercial
Usenet providers to exist, contrary to what some lawmakers have stated in the
past, but instead could care less about the profits made as long as the
providers are cooperating with the surveillance and whatever is required for
prosecution of end-users/bad actors/criminals/etc.

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> There are far cheaper geographical locations in the USA to obtain
> reliable power, IP/transit, and secure raised floor space than Ashburn
> VA.

There are a lot of good reasons to put your stuff there that were even
stronger a decade or more ago when a lot of those hosting decisions were
made. There are really good reasons why Amazon's us-east-1 region and
their original S3 storage was there, entirely technical reasons.

Some of them:

1. It's no longer the cheapest region for power, etc., but it is still
very cheap, particularly compared to, say, New York or New Jersey.
2. It's very close to where most of the people in the United States live,
which matters a lot for latency.
3. Huge amounts of legacy infrastructure is based there, going all the way
back to ARPANET, so it's very easy to get peering (this is changing
over time).
4. It's physically close to Wall Street, which is a huge central point for
network infrastructure, without actually being in Wall Street and
having to pay the costs designed for algorithmic trading.

It was only very recently that Amazon started pushing us-east-2 in Ohio,
and basically everyone in the first round of businesses that hosted on AWS
have significant infrastructure in us-east-1.

I think people reach a little too far for conspiracy theories about stuff
like this. The reason why the national government, Wall Street, the CIA
and NSA, the FBI, a bunch of network infrastructure, tons of ISPs, and
tons of servers are all in the same place is because that's where all the
people lived in the early United States and therefore that's where all the
cities were and it's still where the major population centers of the
United States cluster. Obviously, California, Texas, and Florida are
changing that, but there's a ton of momentum behind those patterns and I
believe the imbalance of US residents towards the eastern time zone is
still quite large.

Virginia actively tried to be the cheap state close enough to all the
expensive states but far enough away that you're not paying the expensive
state premium, and that worked really well for them.

[Jesse Rehmer]

I absolutely accept all of that, you're right, many of us reach pretty far.

However, with everything you said, wouldn't it seem to make more sense that
commercial Usenet operators would have first started in Virginia and migrated
elsewhere, versus the other way around, which is what happened with the full
Usenet feed? Today you've got to be on the Equinix IX in Ashburn to convince
anyone to give you a full feed. Perhaps the consolidation of Usenet
services/feeds exchanged there is based on common financial benefit of them
all living/exchanging the feed there than dispersed over transit providers.

[Russ Allbery]

Jesse Rehmer <jesse....@blueworldhosting.com> writes:

> However, with everything you said, wouldn't it seem to make more sense
> that commercial Usenet operators would have first started in Virginia
> and migrated elsewhere, versus the other way around, which is what
> happened with the full Usenet feed? Today you've got to be on the
> Equinix IX in Ashburn to convince anyone to give you a full
> feed. Perhaps the consolidation of Usenet services/feeds exchanged there
> is based on common financial benefit of them all living/exchanging the
> feed there than dispersed over transit providers.

So, there's this really common phenomenon in economics where certain
places become the center of a particular type of industry for reasons that
aren't really obvious. Usually there's *some* initial impetus, but it's
often minor and doesn't explain the level of concentration. (For
instance, for Virginia, doubtless it got a head start because of ARPANET.)
For example, why are all the movie studios in Los Angeles, or all the tech
companies in San Francisco and San Jose (and now Seattle and Austin, but
still not, say, San Antonio or Phoenix), or all the banks in New York?

Standard economic theory says that a lot of this is due to network
effects. Once there is an industry concentration in an area, it tends to
become more concentrated because it's just so convenient. In northern
Virginia, there are a bunch of well-established data facility companies
that know how to run data facilities and are competing with each other.
There is a huge trained local population of data center workers so it's
easy to hire techs. All the local power equipment and installation
companies do data center work so they're good at it. There are a bunch of
local HVAC companies that know how to cool data centers. There's a big
Dell service center right there to repair your servers and they have all
the parts. All the servers are already there so the transatlantic cables
terminate there because why not. Etc.

If you start out as some small business back in the day when that meant
physical servers (these days, everyone just starts on AWS or GCS or
Azure), you probably want those servers close by because you're going
there yourself to fix them. So you start with some local hosting provider
wherever you are physically located.

But then if you succeed and grow, suppose that you want to provide a
high-bandwidth service, and you want it to be low latency for the majority
of the US population while still having reasonable latency to Europe and
the rest of the US. Where do you rent a data center? You're a US company
and don't want to deal with international corporate law, so Toronto is
out. What are the other options?

Virginia looks really good, and ten years ago looked even better. That's
where all the expertise is, it's centrally located for both the US and
Europe, there's a lot of market competition so you can shop around, it's
close to all the peering... looks great. Sure, you can probably get
cheaper buildings in, oh, Charleston, South Carolina or Louisville,
Kentucky or whatever, but that supporting network of specialization isn't
there. You could go to Miami, but that's both probably not cheap and also
you have to worry about hurricane-related disruptions. You could do
Pittsburgh, or Ohio, or whatever, but Virginia is really appealing.

[Adam H. Kerman]

Russ Allbery <ea...@eyrie.org> wrote:
>Jesse Rehmer <jesse....@blueworldhosting.com> writes:

>>However, with everything you said, wouldn't it seem to make more sense
>>that commercial Usenet operators would have first started in Virginia
>>and migrated elsewhere, versus the other way around, which is what
>>happened with the full Usenet feed? Today you've got to be on the
>>Equinix IX in Ashburn to convince anyone to give you a full
>>feed. Perhaps the consolidation of Usenet services/feeds exchanged there
>>is based on common financial benefit of them all living/exchanging the
>>feed there than dispersed over transit providers.

>So, there's this really common phenomenon in economics where certain
>places become the center of a particular type of industry for reasons that
>aren't really obvious. Usually there's *some* initial impetus, but it's
>often minor and doesn't explain the level of concentration. (For
>instance, for Virginia, doubtless it got a head start because of ARPANET.)
>For example, why are all the movie studios in Los Angeles, or all the tech
>companies in San Francisco and San Jose (and now Seattle and Austin, but
>still not, say, San Antonio or Phoenix), or all the banks in New York?

Movie production is in Los Angeles because that's about as far from New
York as you could get a century ago, without sending the black sheep
brothers in the family to China, the ones exiled from New York to run
several of the studios. The other brothers stayed in New York to raise
money.

There's also reliable sunshine.

For a few years, in early days, there was a number of movie studios in
Chicago, but that would not last.

>. . .

[Jesse Rehmer]

Excellent points.

I don't know why, but the one thing I've completely overlooked until you
pointed out, having the transatlantic peering points there may be a big draw
since most of the commercial Usenet providers are now also operating in parts
of Europe, or exchanging feeds directly with the European providers, so that
makes a lot of sense.

[John Levine]

It appears that Jesse Rehmer <jesse....@blueworldhosting.com> said:
>> Meanwhile AFAIK binary groups still exist today, there’s just a
>> relatively limited set of providers who carry them. I don’t know how
>> they escape being sued into obvlivion by copyright holders.

There's no secret. They act on the DMCA notices they receive and
delete the offending articles. I've been a technical expert in some
court cases on this very topic.

>I can tell you one VERY interesting FACT about all commercial Usenet providers
>in the United States - their servers and infrastructure handling Usenet feeds
>are located in Ashburn Virginia, in facilities near the NSA.

The guy who runs Giganews, which is located in Texas, will be
surprised to hear that someone moved his data cernter while he wasn't
looking. Who knew the NSA was so crafty?

[John Levine]

Since it's not true, probably not.

I expect some are in Ashburn since that's where vast numbers of cloud
providers are. The NSA is in Ft Meade, 50 miles away, but why let
facts get in the way of a good conspiracy theory?

[John Levine]

According to Adam H. Kerman <a...@chinet.com>:

>Movie production is in Los Angeles because that's about as far from New
>York as you could get a century ago, without sending the black sheep
>brothers in the family to China, the ones exiled from New York to run
>several of the studios. The other brothers stayed in New York to raise
>money.

I have heard it was as far as possible from Thomas Edison who had a whole
bunch of patents they hadn't licensed.

>There's also reliable sunshine.

That too, and fairly close to Mexico in case of (legal) emergencies.

>For a few years, in early days, there was a number of movie studios in
>Chicago, but that would not last.

In the really early days they were in Ithaca NY, but if you've ever spent
a winter in Ithaca, you know why they mo