Please complain to Google about their spamming of Usenet

[Wally J]

Given a call to Mountainview is never picked up by a human being...
<https://i.postimg.cc/d388rqkj/google02.jpg> +1 (650) 253-0000

If you care about either Usenet or the DejaNews DejaGoogle search
engine (which provides links to specific Usenet posts), then...

*Please Do This!*
<https://groups.google.com/g/google-usenet/about>
Which will look like this:
<https://i.postimg.cc/3JzWxG3f/please-do-this.jpg>

The DejaGoogle search engine is useful to everyone (not just us) because:
a. DejaGoogle doesn't require an account or paying for retention
b. DejaGoogle links work for everyone (even your 99 year old mother)
c. DejaGoogle only needs a web browser (which everyone has)

The problem with all this spam from Google servers is that even finding the
URI to an article posted _today_ is a mess of wading through that garbage.
<https://i.postimg.cc/yxpSLVrr/Google-Groups-Usenet-Portal-spam-20231206-730am.jpg>

Details follow...

That is the easiest way (I know of) to complain to Google about
their Google-Groups servers allowing Google Usenet portal spam
(which ruins their own Usenet DejaGoogle search engine output)
<https://groups.google.com/g/comp.mobile.android>
You're welcome to upload this screenshot showing the problem:
<https://i.postimg.cc/fyCXPjpR/Google-Groups-Usenet-Portal-spam-20231206-730am.jpg>

Together, maybe we can get Google to at least look at the problem we face.

*Please do this today:*
1. Go to <https://groups.google.com/g/google-usenet/about>
2. Click the "Gear" icon at the top right of that web page
3. Select the option to "Send feedback to Google"
Box 1: "Tell us what prompted this feedback."
Box 2: "A screenshot will help us better understand your feedback."

Optionally, you can do the deluxe version of sending feedback to Google.
A. In tab 1, go to <https://groups.google.com/g/comp.mobile.android>
B. Take a screenshot & save it to a date-related name you can easily find.
C. In tab 2, go to <https://groups.google.com/g/google-usenet/about>
D. Click the "Gear" icon at the top right of that "about" web page
E. In the first box "Tell us what prompted this feedback."
tell Google the problem in a way that Google 'may' care about.

For example, tell them something like "Your Google Groups servers
are allowing obvious off-topic rampant spamming by few individuals
<https://groups.google.com>
such that your own Google Groups Server Search Engine
<http://groups.google.com/g/comp.mobile.android>
is now useless because a few users are abusing your Google servers."

F. In the second box upload that screenshot of the first tab.
G. Press the "Send" button on the bottom right of that second tab.

Here's what it looks like, with every step above documented below.
<https://i.postimg.cc/25gytfM9/googlebug1.jpg>
<https://i.postimg.cc/sX0KBm6Z/googlebug2.jpg>
<https://i.postimg.cc/mgt9kRxV/googlebug3.jpg>
<https://i.postimg.cc/Mp2wMbN4/googlebug4.jpg>
<https://i.postimg.cc/CLVYdsW-2/googlebug5.jpg>
<https://i.postimg.cc/4ysLRySW-/googlebug6.jpg>

In summary, I tried contacting Google to find a better way, to no avail.
<https://i.postimg.cc/kgFknPX0/google01.jpg>
So this online complaint form is the only one that I know about.
<https://groups.google.com/g/google-usenet/about>

If you know of a better way to complain about this, please let us know.
--
Together we can get Google to stop spamming their own Usenet search engine.

[The Doctor]

In article <ukqa3n$1g7rj$1...@paganini.bofh.team>,

Better yet , Depeer Google Groups!

>--
>Together we can get Google to stop spamming their own Usenet search engine.
>

--
Member - Liberal International This is doc...@nk.ca Ici doc...@nk.ca
Yahweh, King & country!Never Satan President Republic!Beware AntiChrist rising!
Look at Psalms 14 and 53 on Atheism ; unsubscribe from Google Groups to be seen
Merry Christmas 2023 and Happy New year 2024 Beware https://mindspring.com

[Alan]

On 2023-12-06 09:13, Wally J wrote:
> Given a call to Mountainview is never picked up by a human being...
> <https://i.postimg.cc/d388rqkj/google02.jpg> +1 (650) 253-0000

I'm sorry, Arlen...

...just what do you think a picture of a smartphone dialing screen proves...

...other that the fact that for all your protestations, it turns out you
actually use an iPhone?

[david]

Using <news:ukqefp$sbs1$1...@dont-email.me>, Alan wrote:

> ...just what do you think a picture of a smartphone dialing screen proves...

What were your results when you called Google to help complain?

> ...other that the fact that for all your protestations, it turns out you
> actually use an iPhone?

The image status bar & aspect ratio seems to not be that of an iphone.
But of an iPad.

[david]

Using <news:ukqeea$1mnv$2...@gallifrey.nk.ca>, The Doctor wrote:

>>If you know of a better way to complain about this, please let us know.
>
> Better yet , Depeer Google Groups!

Agree that it's highwinds who needs to depeer its google feed.
What's the easiest way to complain directly to highwinds so they get it?

[Alan]

Hardly relevant.

The POINT was an iOS device.

[D]

midnight seance, ouija board, full moon, psychic medium, hold hands

[Brian Gregory]

Which News server are you using?

I'm using news.individual.net and I don't see anything from Google
Groups here.

--
Brian Gregory (in England).

[Wally J]

Brian Gregory <void-invalid...@email.invalid> wrote

> I'm using news.individual.net and I don't see anything from Google
> Groups here.

Even so, you still have the problem described in the original post.

[Jörg Lorenz]

On 06.12.23 18:13, Wally J wrote:
> Given a call to Mountainview is never picked up by a human being...
> <https://i.postimg.cc/d388rqkj/google02.jpg> +1 (650) 253-0000
>
> If you care about either Usenet or the DejaNews DejaGoogle search
> engine (which provides links to specific Usenet posts), then...

You are a brain dead idiot.
The smart girls and guys do not use Google in the first place. They use
a decent nntp-server with login and filtering. That would be a place
where they kick you out inevitably after a week max.

And it is none of your business anyway.

--
"Gutta cavat lapidem." (Ovid)

[Jörg Lorenz]

Bullshit.

[Jörg Lorenz]

On 07.12.23 01:05, Brian Gregory wrote:
> Which News server are you using?
>
> I'm using news.individual.net and I don't see anything from Google
> Groups here.
>

+1

[Jörg Lorenz]

On 06.12.23 21:21, david wrote:
> Using <news:ukqefp$sbs1$1...@dont-email.me>, Alan wrote:
>
>> ...just what do you think a picture of a smartphone dialing screen proves...
>
> What were your results when you called Google to help complain?

A waste of time and none of your or anybody else's business in this group.

[Wally J]

Ralph Fox <-rf-nz-@-.invalid> wrote

>> Together, maybe we can get Google to at least look at the problem we face.
>

> In the past, Google's response to getting many spam complaints for a group
> has been to block the group from being *read* on Google Groups.
> This would mean we will all lose access to DejaGoogle search in this group.

Thanks for that input, because you seem to a rare respondent who understood
the problem set, as most self-centered people think it's only about them.

Just filtering out the spam only helps one person; what I'm seeking (as
always) is a general-purpose solution that improves life for everyone.

That's why I posted this thread - to find people who cared about others.
(Where it's clear that most of the respondents only care about themselves.)

If Google wipes out the dejanews/dejagoogle search engine, that's bad.
Everyone loses if the dejanews/dejagoogle searcb engine is wiped out.

There is great utility in the dejanews archives found nowhere else.
a. Dejagoogle allows easy cites to individual threads & articles
b. Dejagoogle allows anyone to see those cites (no newsreader needed)
c. Dejagoogle allows searches without news server retention, etc.

What you understood is _everyone_ is affected when dejagoogle is.

It would mean that every discussion on this group would be unsearchable.
<http://groups.google.com/g/news.admin.peering>
<http://groups.google.com/g/news.admin.net-abuse.usenet>
<http://groups.google.com/g/comp.mobile.android>

Except by HowardKnight (or narkives) neither of which has the same utility.
<http://news.admin.peering.narkive.com>
<http://news.admin.net-abuse.usenet.narkive.com>
<http://comp.mobile.android.narkive.com>

Note: I realize people who only post for their amusement won't have a need
for a general purpose search engine that the whole world can easily use.

> A couple of examples where this has already happened:
>
> * mozilla.support.seamonkey <https://groups.google.com/g/mozilla.support.seamonkey>
> * misc.test <https://groups.google.com/g/misc.test>
>
>      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Banned content warning
> mozilla.support.seamonkey has been identified as containing spam,
> malware, or other malicious content.
> For more information about content policies on Google Groups see our
> Help Center article on abuse and our Terms of Service.
> [ Back to safety ]
>      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Thanks for that example, although when Google took over DejaNews' search
engine, Google was never faithful to any product which competed with it.

An example is that the Windows XP newsgroup was already archived
<http://groups.google.com/g/microsoft.public.windowsxp.general>
But, Google refused to add the newer windows versions after that
<http://alt.comp.os.windows-8.narkive.com>
<http://alt.windows7.general.narkive.com>
<http://alt.comp.os.windows-10.narkive.com>
<http://alt.comp.os.windows-11.narkive.com>
Although, to Google's credit, there is one overarching Windows archive
<http://groups.google.com/g/alt.comp.microsoft.windows>
But, it's full of Google spam too (plus nobody usually posts to it).

I'm still shocked that Google Groups' Usenet portal allows all this obvious
spam, but if they're going to wipe out the dejanews/dejagoogle search
engine, even the self-centered people who will never understand this
problem set (which is 9,999 out of 1,000) will be adversely affected.
--
On Usenet we can discuss problems with people who have more information.

[Wally J]

Jörg Lorenz <hugy...@gmx.net> wrote

>> What were your results when you called Google to help complain?
>
> A waste of time and none of your or anybody else's business in this group.

If you've never run a search before posting, then it won't affect you.

But it's everyone's business if they care about the utility of
search-before-posting and if they care to begin their next question at the
stage that others already fleshed it out so as to not repeat mistakes.

*Anyone can filter out spam*

Anyone can easily filter out all spam emanating from Google Groups'
accounts, but that spam isn't the problem that we're trying to solve.

*We're trying to save DejaNews/DejaGoogle web-searchable archives*

For those who care about the population at large, losing the utility
of the DejaNews archives is a big deal as it easily can affect everyone.

Yes. Everyone.
Well, anyone who is thoughtful enough to run a search before posting.

But - unfortunately - losing DejaNews also affects those _outside_ Usenet.

Because it only takes a web browser to run a search on anything.
<http://groups.google.com/g/<putnameofusenetgrouphere>

This helps everyone both inside the Usenet community & outside it.
a. Searchers don't need servers & readers; they only need a web browser
b. Recipients don't either; they click on a link to an article or thread
c. Retention is almost "permanent" (with regard to Dejanews' originals)

Therefore, this spam, if it kills DejaNews, affects everyone if not solved.
The only to solve the Dejanews issue is to get Google to stop the spam.

I saw Ralph Fox' post saying that the way Google solves that is to wipe out
the utility of the DejaNews search engine - so I hope that doesn't happen.
--
There are two kinds of people who post to Usenet, one of which is a class
of self-centered people who post only for their amusement - but the other
is a class of people who are kind hearted and helpful by their very nature.

[D]

On Thu, 7 Dec 2023 11:38:58 -0400, Wally J <walte...@invalid.nospam> wrote:
snip

>I'm still shocked that Google Groups' Usenet portal allows all this obvious
>spam, but if they're going to wipe out the dejanews/dejagoogle search
>engine, even the self-centered people who will never understand this
>problem set (which is 9,999 out of 1,000) will be adversely affected.

almost 10 out of every 1 will be . . . none of this makes any sense

[met...@newjersey.metaed.com]

In news.admin.net-abuse.usenet Wally J <walte...@invalid.nospam> wrote:
> If you know of a better way to complain about this, please let us know.

For projects such as Android that fall under Google Developers, maybe there is:
the Google Bughunters tracking system.

On Google Groups, there's a "public-ntp-discuss" group. It was created for
public discussion and support of the Google Public NTP Project under Google
Developers.

Recently, their group was overrun by spam in the same way as these Usenet
groups, and the group got replaced by a banned content warning. I reported this
as a security vulnerability (vulnerable to DOS attack) at bughunters.google.com
on Monday. I asked that it be brought specifically to the attention of the
Google Public NTP Project. Three days later their group is back online and the
spam is gone.

So it's not unthinkable that a bug report about comp.mobile.android being
vulnerable to DOS attack might also get some traction, if specifically reported
to the attention of the Android project under Google Developers, and if the
Android project team is invested at all in comp.mobile.android.

Cheers! Edward

[The Doctor]

In article <uksoui$1oqca$1...@paganini.bofh.team>,

Google Groups has now made the Dejanews Archive worth less than 1 cent US.

>--
>On Usenet we can discuss problems with people who have more information.

[Wally J]

The Doctor <doc...@doctor.nl2k.ab.ca> wrote

> Google Groups has now made the Dejanews Archive worth less than 1 cent US.

To that point, it's not yet impossible to find something in Dejanews
web-searchable archives, but, with the spam-to-post ratio being 100:1,
it's certainly a _lot_ harder than it should be to use the search
productively.

Of course, a lot of people never search before posting, so, those are the
people who have already written saying there's no problem for them.

The problem, as you seem to understand, is for everyone who runs a search.

[Scott Dorsey]

Wally J <walte...@invalid.nospam> wrote:
>That's why I posted this thread - to find people who cared about others.
>(Where it's clear that most of the respondents only care about themselves.)
>
>If Google wipes out the dejanews/dejagoogle search engine, that's bad.
>Everyone loses if the dejanews/dejagoogle searcb engine is wiped out.

They broke the indices years ago and old posts can no longer be found. They
are in there, but you can't get to them except by message-id.

>There is great utility in the dejanews archives found nowhere else.
>a. Dejagoogle allows easy cites to individual threads & articles
>b. Dejagoogle allows anyone to see those cites (no newsreader needed)
>c. Dejagoogle allows searches without news server retention, etc.

Yes, but it's broken. If it actually worked properly, the way it did
originally, it would be a huge benefit to Usenet as a whole. But mostly
it's not useful anymore.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

[Wally J]

Scott Dorsey <klu...@panix.com> wrote

>>If Google wipes out the dejanews/dejagoogle search engine, that's bad.
>>Everyone loses if the dejanews/dejagoogle searcb engine is wiped out.
>
> They broke the indices years ago and old posts can no longer be found. They
> are in there, but you can't get to them except by message-id.

Ruefully, I must agree that I can't find even my own articles sometimes,
when I know the keywords used, where, interestingly, sometimes my articles
are in the top of a "normal" Google search but not found in dejagoogle.

There was a WIRED article from a while back which said as much, that the
search engine isn't anywhere near the quality of the main Google Search.

Worse, Google removed the headers (which is probably good for privacy), so
the message-ID is only useful at Howard Knight (as far as I'm still aware).

In summary, it sucks - but it's still better than nothing, especially when
we want to search a newsgroup to see if the topic has been covered already.

An example is searching the alt.usage.english newsgroup before asking about
a word or searching the Android newsgroup before asking about ACR apps.
<http://groups.google.com/g/alt.usage.english>

Then the post we make _after_ that search won't start from the beginning.
It will start where the ball hit the ground & then we can push it forward.
--
(We need a name for what Google Groups' search engine is, as we're looking
at it from the Usenet perspective and not from the Google Groups angle.)

[John McCue]

Followups trimmed to news.admin.net-abuse.usenet

In news.admin.net-abuse.usenet The Doctor <doc...@doctor.nl2k.ab.ca> wrote:
<snip>

> Google Groups has now made the Dejanews Archive worth less
> than 1 cent US.

There is this, they are trying to create a useful archive
of usenet:

https://yarchive.net/

--
[t]csh(1) - "An elegant shell, for a more... civilized age."
- Paraphrasing Star Wars

[D]

On Fri, 8 Dec 2023 00:13:19 -0000 (UTC), doc...@doctor.nl2k.ab.ca (The Doctor) wrote:
>In article <uksoui$1oqca$1...@paganini.bofh.team>,
>Wally J <walte...@invalid.nospam> wrote:

snip

>>I'm still shocked that Google Groups' Usenet portal allows all this obvious
>>spam, but if they're going to wipe out the dejanews/dejagoogle search
>>engine, even the self-centered people who will never understand this
>>problem set (which is 9,999 out of 1,000) will be adversely affected.
>
>Google Groups has now made the Dejanews Archive worth less than 1 cent US.

__________________
/ \
| |
| Deja-News D.M. |
| Diis Manibus |
| 12 February 2001 |
| R.I.P. |
| |
/\/\/\/\/\/\/\/\/\/\/\/\

http://www.dejanews.com/help/help_index.shtml
>General Information
>*Deja News Homepage
> Learn the ins and outs of the Deja News Homepage.
>*New Users
> The ideal starting place if this is your first time to Deja News.
>*Contacting Us
> How to get help via email if you can't find an answer to your question!
>Frequently Asked Questions
>*Deja News FAQ
> A general Frequently Asked Questions about Deja News specific issues.
>*Usenet FAQ
> Got a question about Usenet in general? Get the FAQs!!
>*Posting FAQ
> All the FAQs about posting to Deja News.
>And More...
>*Bookmark Us
> How to add Deja News to your hotlist or bookmark file.
>*Link to Deja News
> Link to Deja News from your Web site!
>*Deja News Glossary
> Find out more about the terminology most commonly used on the Deja News
> site.
[end quote]

http://dejanews.com/help/faq.shtml
>Searching
> How do I search for something?
> Deja News is very easy to use (and it's free, of course), so all you have
> to do is go to our Quick Search or Power Search page, enter some keywords
> and have fun! Please be sure to read our help documents for more advanced
> search features.
>What's the difference between Quick Search and Power Search?
> Our Quick Search page will perform a search on our current database
> (which is usually the last several weeks of newsgroup articles) matching,
> if possible, ALL of your keywords. On our Power Search page, you can
> customize your search with a variety of sorting options, view older Usenet
> articles, select a set of articles before searching on keywords, and much
> more!
[end quote]

[Julieta Shem]

klu...@panix.com (Scott Dorsey) writes:

> Wally J <walte...@invalid.nospam> wrote:
>>That's why I posted this thread - to find people who cared about others.
>>(Where it's clear that most of the respondents only care about themselves.)
>>
>>If Google wipes out the dejanews/dejagoogle search engine, that's bad.
>>Everyone loses if the dejanews/dejagoogle searcb engine is wiped out.
>
> They broke the indices years ago and old posts can no longer be found. They
> are in there, but you can't get to them except by message-id.

That's already something of value, but I don't even know how to do that
anymore. I used to. By seeing how the interface changes from time to
time, I gave up on Google Groups even as an archive. I still think we
need a all-time USENET serious archive. I would like to locate any
message by message-id, no matter in which group it was posted. Is that
a dream?

[Grant Taylor]

On 12/8/23 21:47, Julieta Shem wrote:
> That's already something of value, but I don't even know how to do that
> anymore. I used to. By seeing how the interface changes from time to
> time, I gave up on Google Groups even as an archive. I still think we
> need a all-time USENET serious archive. I would like to locate any
> message by message-id, no matter in which group it was posted. Is that
> a dream?

I think that finding a collection of all the messages is going to be the
biggest hurtle.

I had a collection going back to late 2018 and it was 27 million
articles. Had because I've since removed articles from Google Groups.

So if it's 27 million articles ~5 years, I don't want to fathom what it
would be going back to the '80s.

I strongly suspect much of that data is gone.



--
Grant. . . .

[Wally J]

Julieta Shem <js...@yaxenu.org> wrote

> I would like to locate any message by message-id, no matter in
> which group it was posted. Is that a dream?

You didn't mention whether or not you already know the message id.
If you already know it, what's wrong with the Howard Knight site?
<http://al.howardknight.net/>

If you don't already know the message id, I'm not sure of any lookup that
will give it to you - as I don't use the narkives & I think dejagoogle
stopped archiving the headers a few years ago (as I recall).
<https://news.admin.net-abuse.usenet.narkive.com>
<https://groups.google.com/g/news.admin.net-abuse.usenet>

How are you going to find the message id if you don't already have the
article's headers in your possession (e.g., from your news server)?

[Jörg Lorenz]

On 07.12.23 19:15, met...@newjersey.metaed.com wrote:
> On Google Groups, there's a "public-ntp-discuss" group.

Time servers are different ball game.
Probably you mean NNTP.

[Jesse Rehmer]

On Dec 8, 2023 at 11:43:08 PM CST, "Grant Taylor" <gta...@tnetconsulting.net>
wrote:

I have a little over 20 years of many Usenet hierarchies, 336,216,091
messages, about 1.5TB uncompressed. Using ZFS file system with compression and
CNFS buffers gets it down to ~450GB. It is far from "complete" but I'm working
on it.

Stalled on getting the rest imported from archive.org due storage, but it is
on the roadmap (waiting on some some high endurance SSDs).

Between the Uztoo archive, others on archive.org, and what is still available
on commercial providers, I believe we have the vast majority of Usenet's
history available, but there are bound to be gaps or missing articles
throughout any archive.

[Ivo Gandolfo]

On 09/12/2023 14:09, Jesse Rehmer wrote:
>
> I have a little over 20 years of many Usenet hierarchies, 336,216,091
> messages, about 1.5TB uncompressed. Using ZFS file system with compression and
> CNFS buffers gets it down to ~450GB. It is far from "complete" but I'm working
> on it.
>
> Stalled on getting the rest imported from archive.org due storage, but it is
> on the roadmap (waiting on some some high endurance SSDs).
>
> Between the Uztoo archive, others on archive.org, and what is still available
> on commercial providers, I believe we have the vast majority of Usenet's
> history available, but there are bound to be gaps or missing articles
> throughout any archive.

I'm trying to do the same thing, I'll contact you privately to
coordinate efforts, I currently have 24TB of storage waiting to be
filled (I'm currently at 2TB, and the script is continuing to import
from various sources, all the way back to 2000 )

If you have stuff, we can discuss what the best route is.
The hierarchy that is currently giving me the most problems is alt.* as
it is the most "sewer" of all the others.


Sincerely

--
Ivo Gandolfo

[Grant Taylor]

On 12/9/23 09:17, Ivo Gandolfo wrote:
> I'm trying to do the same thing,

I should have a snapshot or an older copy of my new spool from earlier
this year that will have all articles my server received from late '18
through early '23 if someone wants a copy of it.

[Ted Heise]

On Fri, 8 Dec 2023 23:43:08 -0600,
Grant Taylor <gta...@tnetconsulting.net> wrote:
> On 12/8/23 21:47, Julieta Shem wrote:
> > That's already something of value, but I don't even know how
> > to do that anymore. I used to. By seeing how the interface
> > changes from time to time, I gave up on Google Groups even as
> > an archive. I still think we need a all-time USENET serious
> > archive. I would like to locate any message by message-id, no
> > matter in which group it was posted. Is that a dream?
>
> I think that finding a collection of all the messages is going
> to be the biggest hurtle.

Ouch! Hope you can clear the hurdle without injury. ;)

--
Ted Heise <the...@panix.com> West Lafayette, IN, USA

[Jesse Rehmer]

Yep, hit me up. Most of what I have I sucked from other sources, and yes,
alt.* is ridiculous. I leave pyClean on for most operations but effectively
disable a lot of the filters so it is basically just the EMP body filter I
rely on. The commercial sources are filled with misplaced binaries, so if you
don't filter at least some it's going to a useless sewer.

[Adam H. Kerman]

This shouldn't be available on line where Google can index it. It makes
it painful to search for Usenet articles as so much other stuff from the
Web comes up. Similarly, if I'm searching the Web, normally I don't want
Usenet articles coming up.

Plenty of Usenet shouldn't be archived. Too many newsgroups are nothing
but reposts of articles from the Web whose true author had not posted
them to Usenet. That's plagarism.

Also I think some of the longest-running flame wars -- going back three
decades as someone pointed out the other day -- are cron jobs.

There never has been a lot on Usenet that shouldn't be ephemeral.

[Jesse Rehmer]

If Google wanted to index NNTP servers, they could, but they don't, and I
believe Ivo and I are talking about NNTP.

I don't share the same views regarding archival. Minus binaries, pretty much
everything on Usenet is accessible via the web, whether anyone likes it or
not. Besides Google Groups there are several, mostly not well known, web front
ends to NNTP servers that are indexable. Usenet's content isn't that
interesting anymore and I've never had a Google search pull up site containing
a Usenet article unless I was specifically looking for one.

[Ralph Fox]

On Sat, 09 Dec 2023 00:47:37 -0300, Julieta Shem wrote:

> I would like to locate any
> message by message-id, no matter in which group it was posted. Is that
> a dream?

In addition to the Howard Knight site...

Put the following two together and you can do this for messages posted
within the last 20 years (text posts) or 15 years (binary posts).

1) Some news servers have over 20 years of text retention and over
15 years of binary retention.
2) Some NNTP newsreader client will let you download a message by

message-id, no matter in which group it was posted.

--
Kind regards
Ralph Fox
🦊

The self-edge makes shew of the cloth.

[Olivier Miakinen]

Le 09/12/2023 19:00, Adam H. Kerman a écrit :
>
> [...]
>
> Plenty of Usenet shouldn't be archived. [...]

>
> There never has been a lot on Usenet that shouldn't be ephemeral.

Agreed.

--
Olivier Miakinen

[Olivier Miakinen]

Le 09/12/2023 19:57, Ralph Fox a écrit :
>
> 1) Some news servers have over 20 years of text retention and over
> 15 years of binary retention.

That is way too much! The nntp server that I use has one month of text
retention, which is consistent with the notion of « news » (not « olds »)


--
Olivier Miakinen

[Frank Slootweg]

The context - Julieta Shem's questions - was archives. For archives, a
month of retention is useless.

FYI, because there are no good - searchable - Usenet archives, I keep
my own archive, for my selected/subscribed groups and largely filtered
on trolls, spam, kooks, etc.. My retention is nearly 20 years.

[Olivier Miakinen]

Le 09/12/2023 20:28, Frank Slootweg a écrit :
> Olivier Miakinen <om+...@miakinen.net> wrote:
>> Le 09/12/2023 19:57, Ralph Fox a écrit :
>> >
>> > 1) Some news servers have over 20 years of text retention and over
>> > 15 years of binary retention.
>>
>> That is way too much! The nntp server that I use has one month of text
>> retention, which is consistent with the notion of « news » (not « olds »)
>
> The context - Julieta Shem's questions - was archives. For archives, a
> month of retention is useless.

Yes, it is ok for archives, accessible via the web. But some news servers
have up to one year accessible by NNTP, or even more. And IMHO *that* is
way too much (though I know that my opinion is not widely shared).

> FYI, because there are no good - searchable - Usenet archives, I keep
> my own archive, for my selected/subscribed groups and largely filtered
> on trolls, spam, kooks, etc.. My retention is nearly 20 years.

Ok for that.


--
Olivier Miakinen

[Grant Taylor]

On 12/9/23 13:40, Olivier Miakinen wrote:
> Yes, it is ok for archives, accessible via the web. But some news servers
> have up to one year accessible by NNTP, or even more. And IMHO *that* is
> way too much (though I know that my opinion is not widely shared).

If it's okay for an archive to have ~20 years of retention and it's okay
for an archive to be accessible via the web (HTTP(S)), why is it not
also okay for the archive to be accessible via NNTP(S).

Remember, NNTP(S) is the (current) native news protocol where as HTTP(S)
and others are not nearly as native.

>> FYI, because there are no good - searchable - Usenet archives, I keep
>> my own archive, for my selected/subscribed groups and largely filtered
>> on trolls, spam, kooks, etc.. My retention is nearly 20 years.
>
> Ok for that.

Why does it matter if there are any subscribers or not?

News operators can choose to have as much or as little disk space
backing their spool / archive as they want to have. It's their choice.

[Olivier Miakinen]

Le 09/12/2023 20:49, Grant Taylor a écrit :
>
>> Yes, it is ok for archives, accessible via the web. But some news servers
>> have up to one year accessible by NNTP, or even more. And IMHO *that* is
>> way too much (though I know that my opinion is not widely shared).
>
> If it's okay for an archive to have ~20 years of retention and it's okay
> for an archive to be accessible via the web (HTTP(S)), why is it not
> also okay for the archive to be accessible via NNTP(S).

Maybe I have a bad newsreader or I don't know how to use it ?

The fact is, when I have to look for a given topic, and I register to
the corresponding newsgroup, it takes me a lot of time to read the
whole newsgroup when there are several months (or years) of retention
on the server.

--
Olivier Miakinen

[Grant Taylor]

On 12/9/23 14:09, Olivier Miakinen wrote:
> Maybe I have a bad newsreader or I don't know how to use it ?

Eh ... I'm not convinced that there are any -- what I would consider to
be -- good news readers left. All that I've tried have left something
to be desired.

> The fact is, when I have to look for a given topic, and I register to
> the corresponding newsgroup, it takes me a lot of time to read the
> whole newsgroup when there are several months (or years) of retention
> on the server.

Oh, definitely.

When I have Thunderbird (re)download a newsgroup for the first time it
will often pop up a window asking how many messages to download and what
to do with the rest of them. I usually have it download the last 500
and mark all older messages as read.

But I consider this to be a client side implementation problem and has
very little influence on server side retention.

After all, multiple years of a newsgroup that gets fewer than 1 message
a week can be less of a load on a news reader than a month of a
newsgroup that gets more than one message an hour. It's a numbers game
and how well news readers work with massive numbers of messages.

Or the news server can retain all the messages it wants to devote disk
space to and allow the client to decide how many to download / display. ;-)



Grant. . . .

[Ralph Fox]

On Sat, 9 Dec 2023 21:09:36 +0100, Olivier Miakinen wrote:

> Maybe I have a bad newsreader or I don't know how to use it ?
>
> The fact is, when I have to look for a given topic, and I register to
> the corresponding newsgroup, it takes me a lot of time to read the
> whole newsgroup when there are several months (or years) of retention
> on the server.

> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
> Firefox/52.0 SeaMonkey/2.49.4


Set this in SeaMonkey:

Edit >> Mail & Newsgroup Account Settings >> (select newsgroup account) >> Server Settings
[x] Ask me before downloading more than [ 100 ] messages

Pick a number to fit how far back you want messages.

When there are many headers to download, SeaMonkey will ask you like this:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Download Headers

news.admin.rhubarb.yadda
There are 185969 new message headers to download for
this newsgroup

( ) Download all headers
(*) Download [ 100 ] headers
[ ] Mark remaining headers as read

[ Download ] [ Cancel ]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ QUOTE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Choose the option to download only a limited number of headers.
Pick a number which fits how far back you want messages.

Put a check-mark in "Mark remaining headers as read". This is
so that the group's count of unread messages will not include
the older messages not downloaded.

--
Kind regards
Ralph Fox
🦊

A broken sack will hold no corn.

[Julieta Shem]

You two have my respect. That's a pretty serious project. The USENET
carries principles such as decentralization and censorship resistance.
And likely way more important than that is the principles of
self-organization that might be the only real promise of any social
organization.

[Julieta Shem]

Trimming the destinataries to news.admin.net-abuse.usenet only.

Wally J <walte...@invalid.nospam> writes:

> Julieta Shem <js...@yaxenu.org> wrote
>
>> I would like to locate any message by message-id, no matter in
>> which group it was posted. Is that a dream?
>
> You didn't mention whether or not you already know the message id.
> If you already know it, what's wrong with the Howard Knight site?
> <http://al.howardknight.net/>

I had never heard of this website. It's asking me for a username and
password, which I don't have. It also presenting a certificate issued
to a different hostname.

> If you don't already know the message id, I'm not sure of any lookup that
> will give it to you - as I don't use the narkives & I think dejagoogle
> stopped archiving the headers a few years ago (as I recall).
> <https://news.admin.net-abuse.usenet.narkive.com>
> <https://groups.google.com/g/news.admin.net-abuse.usenet>

I do know the message-id. I think it's nice if posts and papers could
reference a USENET article by the message-id and with a click of the
mouse people could look it up on the web or gopher, gemini or something.
Having to mention the group (along) seems undesired.

> How are you going to find the message id if you don't already have the
> article's headers in your possession (e.g., from your news server)?

Someone sends me an e-mail saying --- ``see <message-id> for how X can
be accomplished'' --- and some archive-service could display the message
given the <message-if>. The USENET could be what electronic archives
for scientific papers still can't do. (You read a paper, find a
reference and must still manually go to questionably legal services to
try to find a copy.)

Thanks for the pointers.

[Spiros Bousbouras]

On Sun, 10 Dec 2023 02:29:07 -0300
Julieta Shem <js...@yaxenu.org> wrote:
>
> Trimming the destinataries to news.admin.net-abuse.usenet only.
>
> Wally J <walte...@invalid.nospam> writes:
>
> > Julieta Shem <js...@yaxenu.org> wrote
> >
> >> I would like to locate any message by message-id, no matter in
> >> which group it was posted. Is that a dream?
> >
> > You didn't mention whether or not you already know the message id.
> > If you already know it, what's wrong with the Howard Knight site?
> > <http://al.howardknight.net/>
>
> I had never heard of this website. It's asking me for a username and
> password, which I don't have. It also presenting a certificate issued
> to a different hostname.

The username and password stuff was news to me and seems to be the case if
you go to https://al.howardknight.net .But if you go to
http://al.howardknight.net/ , there's none of that stuff.

[Spiros Bousbouras]

On Sat, 9 Dec 2023 20:40:28 +0100

Olivier Miakinen <om+...@miakinen.net> wrote:
> Le 09/12/2023 20:28, Frank Slootweg a écrit :
> > Olivier Miakinen <om+...@miakinen.net> wrote:
> >> Le 09/12/2023 19:57, Ralph Fox a écrit :
> >> >
> >> > 1) Some news servers have over 20 years of text retention and over
> >> > 15 years of binary retention.
> >>
> >> That is way too much! The nntp server that I use has one month of text
> >> retention, which is consistent with the notion of « news » (not « olds »)

"news" is just a name. The groups I mostly read have invaluable (mostly
technical) information which should be retained for as long as possible ,
certainly decades.

> > The context - Julieta Shem's questions - was archives. For archives, a
> > month of retention is useless.
>
> Yes, it is ok for archives, accessible via the web. But some news servers
> have up to one year accessible by NNTP, or even more. And IMHO *that* is
> way too much (though I know that my opinion is not widely shared).

Count me among those who do not share your opinion. As far as I'm concerned ,
the longer retention there is , the better.

[D]

plus ones (one plus its own shadow) . . . unmoderated is free speech

[Julieta Shem]

Thanks! That's a pretty useful service. Special thanks to Howard
Knight. (It would be nice if we could get a raw copy of the article.)

[Scott Dorsey]

Frank Slootweg <th...@ddress.is.invalid> wrote:
> FYI, because there are no good - searchable - Usenet archives, I keep
>my own archive, for my selected/subscribed groups and largely filtered
>on trolls, spam, kooks, etc.. My retention is nearly 20 years.

And from my perspective, 20 years only goes back to 2003 when Usenet was
already on the way down. I'd like to see maybe 1983 to 2010 and I don't
much care about anything after then.

Guys like Larry Lippman and Bill Vermillion left a lot of really useful
posts. They are now deceased and all that information is lost.
--scott
--
"C'est un Nagra. C'est suisse, et tres, tres precis."

[Julieta Shem]

Spiros Bousbouras <spi...@gmail.com> writes:

> On Sat, 9 Dec 2023 20:40:28 +0100

[...]

>> Le 09/12/2023 20:28, Frank Slootweg a écrit :

[...]

>> Yes, it is ok for archives, accessible via the web. But some news servers
>> have up to one year accessible by NNTP, or even more. And IMHO *that* is
>> way too much (though I know that my opinion is not widely shared).
>
> Count me among those who do not share your opinion. As far as I'm concerned ,
> the longer retention there is , the better.

Sure, but, assuming we have a ``complete'' archive somewhere, a sysadmin
with few resources could run a thin server and these thin users could
have a smart client that whenever we would like to fetch a nonexistent
article we would ask the archive.

Say I'm reading a post and I'd like to take a look at its parent, which
has been expired. My client asks Howard Knight, fetches a copy and I'm
good. So an archive with an API for fetching raw articles is a pretty
nice service.

[Julieta Shem]

klu...@panix.com (Scott Dorsey) writes:

> Frank Slootweg <th...@ddress.is.invalid> wrote:
>> FYI, because there are no good - searchable - Usenet archives, I keep
>>my own archive, for my selected/subscribed groups and largely filtered
>>on trolls, spam, kooks, etc.. My retention is nearly 20 years.
>
> And from my perspective, 20 years only goes back to 2003 when Usenet was
> already on the way down. I'd like to see maybe 1983 to 2010 and I don't
> much care about anything after then.
>
> Guys like Larry Lippman and Bill Vermillion left a lot of really useful
> posts. They are now deceased and all that information is lost.

I care about that too. There's history, too. Say we're talking about
the birth of the GNU system. Why can't we cite the message-id of the of
the post by Richard Stallman? What's sad is that keeping a service
online on the Internet is never an easy job --- there's attacks and
whatnot. I wish I could put a well set-up service online and turn my
back on it forever. But that's a dream I could never realize.

[Grant Taylor]

On 12/10/23 10:37, Julieta Shem wrote:
> Sure, but, assuming we have a ``complete'' archive somewhere, a sysadmin
> with few resources could run a thin server and these thin users could
> have a smart client that whenever we would like to fetch a nonexistent
> article we would ask the archive.

I question the veracity of "few resources".

I'm seeing 5+ million articles per year in my short (5+ year) archive.
-- Those numbers might not stay the same, but they are a starting point
for this discussion.

If we say ~5 million articles per year going back 30+ years, that's 150
million messages. That's not just a "few resources" to make accessible
in relatively short order.

> Say I'm reading a post and I'd like to take a look at its parent, which
> has been expired. My client asks Howard Knight, fetches a copy and I'm
> good. So an archive with an API for fetching raw articles is a pretty
> nice service.

We already have an API for fetching articles from a server. Network
News Transfer Protocol does perfectly adequate job at it. NNTP is also
already very well supported in most news clients. So why not use it?

I think what you are after is an NNTP server that has an extremely deep
(30+ year) history.

There may be some room for added complexity in the servers so that they
can look up an old article somewhere else outside of their corpus. But
that's a different problem for a different day.

I believe that an NNTP server meant for archive access and doesn't allow
posting could provide what you're after /if/ such a corpus could be put
together and such a system could be hosted somewhere (and likely
supported by it's user base).

[Grant Taylor]

On 12/10/23 10:51, Julieta Shem wrote:
> I care about that too. There's history, too. Say we're talking about
> the birth of the GNU system. Why can't we cite the message-id of the of
> the post by Richard Stallman?

I believe you can.

What is different about citing an article from yesterday vs an article
from 25 years ago /other/ /than/ lack of the latter on the news server
you're using?

I believe that current methodologies can provide what is being asked for
if the data was available and people were willing to commit the
resources for it.

> What's sad is that keeping a service
> online on the Internet is never an easy job --- there's attacks and
> whatnot. I wish I could put a well set-up service online and turn my
> back on it forever. But that's a dream I could never realize.

It's not trivial. But it's not impossible either.

[Scott Dorsey]

Grant Taylor <gta...@tnetconsulting.net> wrote:
>On 12/10/23 10:51, Julieta Shem wrote:
>> I care about that too. There's history, too. Say we're talking about
>> the birth of the GNU system. Why can't we cite the message-id of the of
>> the post by Richard Stallman?
>
>I believe you can.
>
>What is different about citing an article from yesterday vs an article
>from 25 years ago /other/ /than/ lack of the latter on the news server
>you're using?

The lack of access to the latter from any source at all. I mean, I can
cite the article (if I could find it), but there is no point in citing
something that nobody can read.

>I believe that current methodologies can provide what is being asked for
>if the data was available and people were willing to commit the
>resources for it.

Dejanews provided exactly what was asked for, and so did Google for a
while, until they "improved" things to the point where it was completely
broken.

Committing the resources is an issue, but not the big one. The real problem
is availability of the data. When dejanews was created an enormous amount of
effort went into finding pre-dejanews material and passing it off to dejanews.
At this point, most of that likely no longer exists except in google's
database. And there's no point in it being in there if people can't find it.

[Grant Taylor]

On 12/10/23 11:23, Scott Dorsey wrote:
> The lack of access to the latter from any source at all.

ACK

> I mean, I can
> cite the article (if I could find it), but there is no point in citing
> something that nobody can read.

I think there is some value in saying something and saying where it came
from. It offers a modicum of veracity and enables the reader to fact
check. What the reader does and does not have access to is not your
primary problem. Though it does behoove you to cite sources that you
know the reader has access to.

> Dejanews provided exactly what was asked for, and so did Google for a
> while, until they "improved" things to the point where it was completely
> broken.

ACK

> Committing the resources is an issue, but not the big one.

I think the necessary resources are going to be a bigger issue than many
other people.

INN's traditional spool directories are going to tax just about any file
system when you try to put hundreds of thousands of files in a single
directory. Some newsgroups will have significantly more articles than
others.

Sure, there are ways to store articles so that they aren't all in one
directory. But that's now a news server change. It can happen, but
it's not as simple as a configuration change. It will likely be a code
and a configuration change.

There are other message stores, but the ones that I'm aware of tend to
by cyclical in nature and of a fixed size which is antithetical to the
archive forever goal.

This is probably a case for a custom NNTP server that is really a
gateway (of sorts) to some sort of object store that is distributed and
designed to scale to millions of objects in a container (newsgroup).

Whatever is done needs to be flexible and have the ability to be
reconfigured as things grow. It should also have a little bit of
redundancy as the more systems that are added to it, the more fragile it
will become.

> The real problem is availability of the data.

I agree that's probably the /primary/ problem and supersedes the storage
in such as I believe that computer science / systems people can overcome
the problem mentioned above. -- I'm not as confident that we can
recover a full archive of Usenet any more. After all, according to
Wikipedia, we're talking about 43+ years of data. Much of that data was
deemed ephemeral by most people. Much of that data was difficult to
collect 20+ years ago. The intervening 20 years won't have helped the
matter.

I maintain that storage and accessibility to the data is the /secondary/
largest problem.

> When dejanews was created an enormous amount of
> effort went into finding pre-dejanews material and passing it off to dejanews.
> At this point, most of that likely no longer exists except in google's
> database. And there's no point in it being in there if people can't find it.

Eh ... given Google's propensity to not get rid of things, I sort of
suspect that the old DeJa News archive is probably still exists
somewhere. It may actually be directly behind Google Groups Usenet
gateway and the gateway itself may be munging articles as they are
presented. Google really is antithetical to destroying data. Getting
to that data, that's an entirely different issue.

[Frank Slootweg]

Scott Dorsey <klu...@panix.com> wrote:
> Frank Slootweg <th...@ddress.is.invalid> wrote:
> > FYI, because there are no good - searchable - Usenet archives, I keep
> >my own archive, for my selected/subscribed groups and largely filtered
> >on trolls, spam, kooks, etc.. My retention is nearly 20 years.
>
> And from my perspective, 20 years only goes back to 2003 when Usenet was
> already on the way down. I'd like to see maybe 1983 to 2010 and I don't
> much care about anything after then.

Yes, that would be nice. I started on Usenet at about that time, or
perhaps a year later or earlier. 'We' tought that Deja/Google would keep
it for us. How wrong/gullible we were!

At the time I tried, my oldest article I could find in Google Groups
was of February 24, 1989 (of course just '89' at that time! :-)). It's
still there. (To preserve my privacy and those of others, I won't give
the URL.)

So maybe you can still find something of what you're looking for.

> Guys like Larry Lippman and Bill Vermillion left a lot of really useful
> posts. They are now deceased and all that information is lost.

Perhaps someone on the admin or/and newsreader groups has some idea
about how to get (acces to) this kind of material. IIRC, there have been
recent discussions about old archives and how to get/preserve them.

[Wally J]

Brian Gregory <void-invalid...@email.invalid> wrote
> I'm using news.individual.net and I don't see anything from Google
> Groups here.

Hi Brian,
Your admin appears to be working on the problem as shown here.
<https://groups.google.com/g/news.admin.peering/c/AgrNUeZuAkw/m/f9PSZkv4AAAJ>

Here's what he said, verbatim.
My apologies to the Individual.net news server admins.

==< cut here >==
We follow the discussion here and are aware of the Google spam problem. We
also have some anti-spam measures for our reader servers. But it is really
easy to find our contact address (ne...@individual.net) on
https://news.individual.net/

In the past, issuing a UDPš has often not been particularly successful.

Heiko (for Newsmaster-Team of individual.net)
https://en.wikipedia.org/wiki/Usenet_Death_Penalty

[Wally J]

Julieta Shem <js...@yaxenu.org> wrote

>> You didn't mention whether or not you already know the message id.
>> If you already know it, what's wrong with the Howard Knight site?
>> <http://al.howardknight.net/>
>
> I had never heard of this website. It's asking me for a username and
> password, which I don't have. It also presenting a certificate issued
> to a different hostname.

I'm sorry about that.
I care that you get your answer that you need.
But I didn't respond at first as I have nothing more to help you with.

Nobody else has responded to this, unfortunately.
So let's home someone else can help you explain what is happening.
Especially as it purports to do exactly what it is you asked for.

To be clear, I can't explain your results as when I go to howardknight.net,
it only asks me for the message id. But I'm just a long-time user.

I'm not an admin in any way.

All we can do to help is ask the others here why Howardknight.net would ask
for a password from you - when it has never asked for a password from me.

[Andy Burns]

Wally J wrote:

> Julieta Shem wrote

To expand the URLs slightly as an explanation ...

<http://al.howardknight.net:80/>
is different from

<https://al.howardknight.net:443/>
which redirects to
<https://al.howardknight.net:443/login_up.php>

The former is where the lookup service is running (no need for
encryption) the latter is where howard administers the
shared?/virtual?/colo? host that it runs on, that does need to be
encrypted to keep the server safe.

[Julieta Shem]

Grant Taylor <gta...@tnetconsulting.net> writes:

> On 12/10/23 10:37, Julieta Shem wrote:
>> Sure, but, assuming we have a ``complete'' archive somewhere, a sysadmin
>> with few resources could run a thin server and these thin users could
>> have a smart client that whenever we would like to fetch a nonexistent
>> article we would ask the archive.
>
> I question the veracity of "few resources".
>
> I'm seeing 5+ million articles per year in my short (5+ year)
> archive. -- Those numbers might not stay the same, but they are a
> starting point for this discussion.
>
> If we say ~5 million articles per year going back 30+ years, that's
> 150 million messages. That's not just a "few resources" to make
> accessible in relatively short order.

I meant a sysadmin with little money.

>> Say I'm reading a post and I'd like to take a look at its parent, which
>> has been expired. My client asks Howard Knight, fetches a copy and I'm
>> good. So an archive with an API for fetching raw articles is a pretty
>> nice service.
>
> We already have an API for fetching articles from a server. Network
> News Transfer Protocol does perfectly adequate job at it. NNTP is
> also already very well supported in most news clients. So why not use
> it?

I totally agree. (Whatever works.)

> I think what you are after is an NNTP server that has an extremely
> deep (30+ year) history.

Yes, I'd like that. I think it's something we can all use.

> There may be some room for added complexity in the servers so that
> they can look up an old article somewhere else outside of their
> corpus. But that's a different problem for a different day.
>
> I believe that an NNTP server meant for archive access and doesn't
> allow posting could provide what you're after /if/ such a corpus could
> be put together and such a system could be hosted somewhere (and
> likely supported by it's user base).

That makes sense --- an NNTP made to be used as as reference, optimized
only to provide a message per message-id.

We should also have a solution for keeping addresses for a very long
time. In the same way we have organized ourselves to keep the USENET
rolling, we should also organize ourselves to make things last. But
that's seems less easy. When there is money involved, people set up
institutions that keep things going. We should have things like that.
Could a sum of money be allocated, providing interest with which we can
keep basic things like a hostname so that

nntp://arxiv.use.net/<message@id>

always fetches the article with <message@id>? I think that's the chance
we have of running things properly in the world. The representative
idea is likely at the end of its time now, finally. Electronic
communication and the electronic printing press might give us the means
to run stuff ourselves now. (An interesting keyword --- sortition.)

[Julieta Shem]

Grant Taylor <gta...@tnetconsulting.net> writes:

> On 12/10/23 11:23, Scott Dorsey wrote:
>> The lack of access to the latter from any source at all.
>
> ACK
>
>> I mean, I can
>> cite the article (if I could find it), but there is no point in citing
>> something that nobody can read.
>
> I think there is some value in saying something and saying where it
> came from. It offers a modicum of veracity and enables the reader to
> fact check. What the reader does and does not have access to is not
> your primary problem. Though it does behoove you to cite sources that
> you know the reader has access to.

Sure. I cite these messages when I need to. But people often wonder
--- how can I get it? Sometimes there is an answer such as some
website, but such websites scramble over the years. The USENET deserves
the highest standard in archiving.

> INN's traditional spool directories are going to tax just about any
> file system when you try to put hundreds of thousands of files in a
> single directory. Some newsgroups will have significantly more
> articles than others.
>
> Sure, there are ways to store articles so that they aren't all in one
> directory. But that's now a news server change. It can happen, but
> it's not as simple as a configuration change. It will likely be a
> code and a configuration change.
>
> There are other message stores, but the ones that I'm aware of tend to
> by cyclical in nature and of a fixed size which is antithetical to the
> archive forever goal.
>
> This is probably a case for a custom NNTP server that is really a
> gateway (of sorts) to some sort of object store that is distributed
> and designed to scale to millions of objects in a container
> (newsgroup).
>
> Whatever is done needs to be flexible and have the ability to be
> reconfigured as things grow. It should also have a little bit of
> redundancy as the more systems that are added to it, the more fragile
> it will become.

ACK. When you said NNTP, I said --- okay ---, but I would think that a
whole new system should be set-up. I would think that archive.org would
be interested in getting a proposal for something like that. They seem
pretty serious about archiving the Internet.

>> The real problem is availability of the data.
>
> I agree that's probably the /primary/ problem and supersedes the
> storage in such as I believe that computer science / systems people
> can overcome the problem mentioned above. -- I'm not as confident
> that we can recover a full archive of Usenet any more. After all,
> according to Wikipedia, we're talking about 43+ years of data. Much
> of that data was deemed ephemeral by most people. Much of that data
> was difficult to collect 20+ years ago. The intervening 20 years
> won't have helped the matter.

Sadly, this could be true, but I feel optimistic about it. If we could
get people's attention, I believe some people would show up saying ---
oh, hey, I got a lot of data here and there and the whole thing would
appear.

>> When dejanews was created an enormous amount of effort went into
>> finding pre-dejanews material and passing it off to dejanews. At
>> this point, most of that likely no longer exists except in google's
>> database. And there's no point in it being in there if people can't
>> find it.
>
> Eh ... given Google's propensity to not get rid of things, I sort of
> suspect that the old DeJa News archive is probably still exists
> somewhere. It may actually be directly behind Google Groups Usenet
> gateway and the gateway itself may be munging articles as they are
> presented. Google really is antithetical to destroying data. Getting
> to that data, that's an entirely different issue.

I totally bet on the same.

[Julieta Shem]

Someone helped me with it --- it was an address difference. And, by the
way, I have no current need of anything, but it's nice to know what to
do when the need arise. Thank you so much for your attention!

[Scott Dorsey]

Grant Taylor <gta...@tnetconsulting.net> wrote:
>Eh ... given Google's propensity to not get rid of things, I sort of
>suspect that the old DeJa News archive is probably still exists
>somewhere. It may actually be directly behind Google Groups Usenet
>gateway and the gateway itself may be munging articles as they are
>presented. Google really is antithetical to destroying data. Getting
>to that data, that's an entirely different issue.

I agree. It's in there, but we can't get it out, and that is the
most frustrating thing possible. We gave it to them, but we can't
it it back.

[Jesse Rehmer]

On Dec 10, 2023 at 12:13:06 PM CST, "Grant Taylor"

<gta...@tnetconsulting.net> wrote:

> I think the necessary resources are going to be a bigger issue than many
> other people.
>
> INN's traditional spool directories are going to tax just about any file
> system when you try to put hundreds of thousands of files in a single
> directory. Some newsgroups will have significantly more articles than
> others.

I have 1.6TB of tradspool on NVMe and no performance issues. Some groups have
over 4 million articles.

> Sure, there are ways to store articles so that they aren't all in one
> directory. But that's now a news server change. It can happen, but
> it's not as simple as a configuration change. It will likely be a code
> and a configuration change.
>
> There are other message stores, but the ones that I'm aware of tend to
> by cyclical in nature and of a fixed size which is antithetical to the
> archive forever goal.

You can set CNFS buffers to be written to sequentially, and as long as you're
paying attention you can add new buffers without ever losing anything and
wrapping to the beginning of the metabuffer. I've experimented with 100GB and
1TB buffer files and performance doesn't seem to matter the CNFS buffer size.

On a ZFS filesystem with CNFS buffers I get roughly 2.86X compression so that
1.6TB of tradspool shrinks down to ~450GB of CNFS buffers.

> This is probably a case for a custom NNTP server that is really a
> gateway (of sorts) to some sort of object store that is distributed and
> designed to scale to millions of objects in a container (newsgroup).
>
> Whatever is done needs to be flexible and have the ability to be
> reconfigured as things grow. It should also have a little bit of
> redundancy as the more systems that are added to it, the more fragile it
> will become.

If one wants to dabble in distributed NNTP architecture, Diablo and its
accompanying dreaderd are the way to go. Or NNTPSwitch if you can get it to
compile on anything.

>
>> The real problem is availability of the data.
>
> I agree that's probably the /primary/ problem and supersedes the storage
> in such as I believe that computer science / systems people can overcome
> the problem mentioned above. -- I'm not as confident that we can
> recover a full archive of Usenet any more. After all, according to
> Wikipedia, we're talking about 43+ years of data. Much of that data was
> deemed ephemeral by most people. Much of that data was difficult to
> collect 20+ years ago. The intervening 20 years won't have helped the
> matter.

I believe we have every bit of Usenet (may not *every single article* but
close) with all of the archives on archive.org, and what is still available on
commercial Usenet spools (most go back ~20 years for text retention).

It's a matter of putting it all back together on a usable NNTP spool that's
not as easy, but not undoable. There are a few of us working on projects of
our own with the same goal.

[Grant Taylor]

On 12/10/23 15:01, Julieta Shem wrote:
> I meant a sysadmin with little money.

I think it's actually going to take a fair bit more than a little money.

> That makes sense --- an NNTP made to be used as as reference, optimized
> only to provide a message per message-id.

NNTP itself wouldn't be changed at all. Or at least it shouldn't need
to be. Changing NNTP would fundamentally alter how clients interact and
what clients would be able to interact.

Or did you mean a server daemon / software that speaks standard NNTP?

> We should also have a solution for keeping addresses for a very long
> time.

Please elaborate on what you mean by "addresses" in this context.

> In the same way we have organized ourselves to keep the USENET
> rolling, we should also organize ourselves to make things last.

Making things last is quite a bit more difficult than many like.
Hardware fails. Software gets corrupted. Disks fill up. Systems get
compromised.

Things are going to require care, feeding, burping, and changing.

> But that's seems less easy. When there is money involved, people set
> up institutions that keep things going. We should have things like that.

That's done for significant sums of money. Trust funds probably work
for five or six digits. Institutions are formed for more digits.

> Could a sum of money be allocated, providing interest with which we can
> keep basic things like a hostname so that
>
> nntp://arxiv.use.net/<message@id>
>
> always fetches the article with <message@id>?

Sadly, I don't think that's going to work as well as you might hope.

That's a singular host name, which implies a singular point of
communications. Which in and of itself sort of implies a single point
of failure.

Sure, there are ways to make some of that non-singular, but the
complexity goes way up and things start being duplicated.

> I think that's the chance
> we have of running things properly in the world.

"properly" starts to get problematic to define the more people you add
to the mix and the longer you want things to last.

> The representative
> idea is likely at the end of its time now, finally. Electronic
> communication and the electronic printing press might give us the means
> to run stuff ourselves now. (An interesting keyword --- sortition.)

I don't know.

There is also the problem that such an archive isn't static. It is
currently growing at thousands, if not tens of thousands of messages a day.

[Grant Taylor]

On 12/10/23 18:28, Julieta Shem wrote:
> Sure. I cite these messages when I need to. But people often wonder
> --- how can I get it? Sometimes there is an answer such as some
> website, but such websites scramble over the years.

Agreed.

> The USENET deserves the highest standard in archiving.

Unfortunately most people don't agree.

I reluctantly don't agree by the very fact that I'm purging messages
from Google Groups from my private server.

I think that there is a LOT of Usenet that isn't worth archiving for
very long. In fact, many articles become increasingly less valuable
with each successively larger unit of time.

> ACK. When you said NNTP, I said --- okay ---, but I would think that a
> whole new system should be set-up.

Please clarify what you mean by "whole new system". Are you referring
to a new server that speaks NNTP to make articles available?

Or are you referring to something more radical?

> I would think that archive.org would
> be interested in getting a proposal for something like that. They seem
> pretty serious about archiving the Internet.

Maybe.

There are static Usenet archives on archive.org already. But they are
both incomplete and /static/. They also aren't readily usable the way
that a news server that speaks NNTP is.

> Sadly, this could be true, but I feel optimistic about it. If we could
> get people's attention, I believe some people would show up saying ---
> oh, hey, I got a lot of data here and there and the whole thing would
> appear.

I am not that optimistic.

I suspect each digit in the percentage is going to be an order of
magnitude more difficult to get than the previous digit.

I suspect that we can easily get 9%.

I don't know if we could get to 90% without support from people like
Google providing the DeJa News archive (unmodified).

Getting the next digit, 99.9% is going to be like pulling hens teeth.

Then what about 99.99%?

> I totally bet on the same.

ACK

[Grant Taylor]

On 12/10/23 19:04, Scott Dorsey wrote:
> I agree. It's in there, but we can't get it out, and that is the
> most frustrating thing possible. We gave it to them, but we can't
> it it back.

Having a better idea than many how Google lawyercats behave, I suspect
they would say "you didn't give it to us" and / or "we bought the
company and the data they had". They likely feel zero obligation to
provide it.

Then there's the "you gave X number of articles to the free global
Usenet out of the Y total articles therein" where X is significantly
smaller than Y and as such you get an infinitesimally small part back,
if that.

[Grant Taylor]

On 12/10/23 19:08, Jesse Rehmer wrote:
> I have 1.6TB of tradspool on NVMe and no performance issues. Some groups have
> over 4 million articles.

That sounds like a very nice setup.

I suspect that most people wouldn't want to spend the money /today/
especially for Usenet, and 40 year old articles therein.

But there are a number of people who would.

But then it does require some money to acquire the equipment, host it,
keep it online, etc. etc. etc.

Can it be done? Yes.

Is it static? Absolutely not.

> You can set CNFS buffers to be written to sequentially, and as long as you're
> paying attention you can add new buffers without ever losing anything and
> wrapping to the beginning of the metabuffer. I've experimented with 100GB and
> 1TB buffer files and performance doesn't seem to matter the CNFS buffer size.

Good to know.

But it sounds like you need to monitor it and make sure that your CNFS
buffers don't fill / wrap.

I'm sure there are ways to streamline this. But it's antithetical to
static / simple which is the domain most archives operate in.

> On a ZFS filesystem with CNFS buffers I get roughly 2.86X compression so that
> 1.6TB of tradspool shrinks down to ~450GB of CNFS buffers.

ACK

I'm glad to know that others are using ZFS too.

I'm not at all surprised to learn about the 2.86x compression.

> If one wants to dabble in distributed NNTP architecture, Diablo and its
> accompanying dreaderd are the way to go. Or NNTPSwitch if you can get it to
> compile on anything.

Does Diablo + dreader use it's own protocol? Or is it standard NNTP?

If it is it's own protocol, could someone write a gateway to be an NNTP
server that is a dreader client on the back end?

There is also the fact that with distributed systems, there need to be
the multiple systems to distribute the load over. -- Something that's
definitely possible, but it strongly speaks to something that needs
constant care and feeding.

> I believe we have every bit of Usenet (may not *every single article* but
> close) with all of the archives on archive.org, and what is still available on
> commercial Usenet spools (most go back ~20 years for text retention).

I would be very pleasantly surprised to learn if that is indeed the case.

I would not bet a fancy coffee on it. Maybe I'm more cynical than most.

> It's a matter of putting it all back together on a usable NNTP spool that's
> not as easy, but not undoable.

Agreed.

As I indicated in another post, I think the biggest issue will be
acquiring the corpus of articles.

> There are a few of us working on projects of our own with the same goal.

Cool!

Let me know if I can help in any way.

[Scott Dorsey]

In article <ul64k8$s6g$3...@tncsrv09.home.tnetconsulting.net>,

Yes, precisely.

[met...@newjersey.metaed.com]

In news.admin.net-abuse.usenet Joerg Lorenz <hugy...@gmx.net> wrote:
> Probably you mean NNTP.

The point was to share a successful outcome, using Bughunters to report a spam
attack on a Google Group having ties to a Google development team. What worked
for that Group might not work for another, but a person could try it. My
suggestion was that spam overrunning comp.mobile.android could be posted on
Bughunters to the attention of the Android development team. The Android team
might be invested enough and in a position to escalate it internally and get it
fixed, just as the Time team did.

(Nothing to do with NTP and NNTP having similar spelling.)

Cheers! Edward

[Wally J]

Scott Dorsey <klu...@panix.com> wrote

>>Then there's the "you gave X number of articles to the free global
>>Usenet out of the Y total articles therein" where X is significantly
>>smaller than Y and as such you get an infinitesimally small part back,
>>if that.
>
> Yes, precisely.

BTW, on the Android newsgroup some of us are discussing WHY they're doing
all this spam, where not every newsgroup is being spammed, it seems.

These are:
<https://groups.google.com/g/alt.internet.wireless>
<https://groups.google.com/g/alt.comp.microsoft.windows>
<https://groups.google.com/g/uk.telecom.mobile>
<https://groups.google.com/g/rec.photo.digital>
<https://groups.google.com/g/alt.home.repair>
<https://groups.google.com/g/comp.mobile.android>
etc.

These are not:
<http://alt.comp.os.windows-10.narkive.com>
<http://alt.comp.os.windows-11.narkive.com>
<https://groups.google.com/g/comp.mobile.ipad>
<https://groups.google.com/g/misc.phone.mobile.iphone>
etc.

I haven't looked extensively but they don't seem to be spamming groups
(such as the Windows 10 and 11 newsgroups most people post to) which aren't
auto-archived - but that could also be because the Google-to-Usenet portal
might not work for groups that aren't part of the DejaNews archives.

Dunno what they're doing for real, but it's only some newsgroups.
Not all.

[Grant Taylor]

On 12/12/23 19:38, Wally J wrote:
> BTW, on the Android newsgroup some of us are discussing WHY they're doing
> all this spam, where not every newsgroup is being spammed, it seems.

I'm of the opinion that it's a bunch of different spam campaigns, likely
by almost as many spammers.

> I haven't looked extensively but they don't seem to be spamming groups
> (such as the Windows 10 and 11 newsgroups most people post to) which aren't
> auto-archived - but that could also be because the Google-to-Usenet portal
> might not work for groups that aren't part of the DejaNews archives.

^DejaNews^Usenet

I know that the newsgroups for Thunderbird / Firefox support and some of
the newer versions of Windows don't have Usenet newsgroups inside of
Google Groups.

In traditional news parlance, Google doesn't carry said newsgroups /
they aren't in Google's active newsgroups file. As such there is
nothing inside of Google Groups that the spammers can post to using the
Google Groups Usenet gateway.

> Dunno what they're doing for real, but it's only some newsgroups.
> Not all.

I don't think I've ever seen a single spam campaign hit all of the
newsgroups that I subscribe to, much less all of the thousands in my
server's active file. All of them are one or few groups and there is no
rime or reason that I'm aware of in their selection of groups.



Grant. . . .

[Wally J]

Grant Taylor <gta...@tnetconsulting.net> wrote

>> BTW, on the Android newsgroup some of us are discussing WHY they're doing
>> all this spam, where not every newsgroup is being spammed, it seems.
>
> I'm of the opinion that it's a bunch of different spam campaigns, likely
> by almost as many spammers.
>
>> I haven't looked extensively but they don't seem to be spamming groups
>> (such as the Windows 10 and 11 newsgroups most people post to) which aren't
>> auto-archived - but that could also be because the Google-to-Usenet portal
>> might not work for groups that aren't part of the DejaNews archives.
>
> ^DejaNews^Usenet

I wish we had a name for it but it's like when we talk about unwanted spam.
No matter what name we use, everyone knows what we're talking about.

I call it dejagoogle sometimes.
a. It's a search engine to me
b. And it's a cite reference (better'n a message-ID is anyway)

> I know that the newsgroups for Thunderbird / Firefox support and some of
> the newer versions of Windows don't have Usenet newsgroups inside of
> Google Groups.

Yup. I tried valiantly, as did you, I believe, to get Google to add them to
the auto-archives so that others can run searches to find answers before
they post a question. I failed.
<https://groups.google.com/g/alt.comp.software.thunderbird> 404 not found
<https://groups.google.com/g/alt.comp.software.firefox> Content unavailable
NOTE: Interesting I got two different errors for those two URIs just now.

We might check for the Google spam on the narkives though.
<http://alt.comp.software.thunderbird.narkive.com> (unsafe)
<http://alt.comp.software.firefox.narkive.com> (unsafe)
But my browser setup won't let me.

> In traditional news parlance, Google doesn't carry said newsgroups /
> they aren't in Google's active newsgroups file. As such there is
> nothing inside of Google Groups that the spammers can post to using the
> Google Groups Usenet gateway.

Yeah. That must be why the Gspammers aren't spamming these Windows ngs.
<http://alt.comp.os.windows-10.narkive.com>
<http://alt.comp.os.windows-11.narkive.com>

>> Dunno what they're doing for real, but it's only some newsgroups.
>> Not all.
>
> I don't think I've ever seen a single spam campaign hit all of the
> newsgroups that I subscribe to, much less all of the thousands in my
> server's active file. All of them are one or few groups and there is no
> rime or reason that I'm aware of in their selection of groups.

I've noticed a lot of the spam is in funky characters, which I find odd,
but many also have URLs so they could be phishing attacks for all I know.

The M0VIE spam is particularly repetitive - I suspect they want to get into
search engine results - but I don't know that for a fact. I'm just guessing
as a lot of my tutorials posted to Usenet end up in the first page of hits
from a "normal" www.google.com search - so Usenet _does_ show up there too.

BTW, regarding the message from Individual.net, I was heartened they care.
Anybody have any new datapoints from Giganews & Highwinds admins yet?

[Marc Haber]

Wally J <walte...@invalid.nospam> wrote:
>Scott Dorsey <klu...@panix.com> wrote

>I haven't looked extensively but they don't seem to be spamming groups
>(such as the Windows 10 and 11 newsgroups most people post to) which aren't
>auto-archived - but that could also be because the Google-to-Usenet portal
>might not work for groups that aren't part of the DejaNews archives.
>
>Dunno what they're doing for real, but it's only some newsgroups.
>Not all.

And it is not only Usenet. I saw the spam happening on pandoc-users, a
googlegroup about the pandoc software which is not gated to Usenet. A
user complained on-list, I tried to reply, got a bounce ("You are not
allowed to write to this list"). I finally shrugged it off and tried
ot unsubscribe via the Mail address that is in the headers, got a
bounce ("You are not allowed to write to this list"). Tried to
unsubscribe via the URL from the header, got redirected to an error
page saying that the group is being blocked due to malicious content.

I eneded up putting the mail address (thankfully, an exclusive mail
address just for this group) to ":fail: Depeer Google Groups Now".

Google doesn't even care about its own groups any more.

Greetings
Marc
--
-------------------------------------- !! No courtesy copies, please !! -----
Marc Haber | " Questions are the | Mailadresse im Header
Mannheim, Germany | Beginning of Wisdom " |
Nordisch by Nature | Lt. Worf, TNG "Rightful Heir" | Fon: *49 621 72739834

[D]

On Tue, 12 Dec 2023 21:38:27 -0400, Wally J <walte...@invalid.nospam> wrote:
snip

>BTW, on the Android newsgroup some of us are discussing WHY they're doing
>all this spam, where not every newsgroup is being spammed, it seems.
>These are:
> <https://groups.google.com/g/alt.internet.wireless>
> <https://groups.google.com/g/alt.comp.microsoft.windows>
> <https://groups.google.com/g/uk.telecom.mobile>
> <https://groups.google.com/g/rec.photo.digital>
> <https://groups.google.com/g/alt.home.repair>
> <https://groups.google.com/g/comp.mobile.android>
> etc.
>These are not:
> <http://alt.comp.os.windows-10.narkive.com>
> <http://alt.comp.os.windows-11.narkive.com>
> <https://groups.google.com/g/comp.mobile.ipad>
> <https://groups.google.com/g/misc.phone.mobile.iphone>
> etc.

rather than feed the google monster or other "sanitized" web links, this
format is more user-friendly because it invites the reader to browse any
of these aforecited newsgroups using their favorite default newsreaders
and downloading all or part of what article headers may be available on
their preferred news servers; plus any downloaded messages will show the
complete raw message with full headers which can be filtered accordingly:

<news:alt.internet.wireless>
<news:alt.comp.microsoft.windows>
<news:uk.telecom.mobile>
<news:rec.photo.digital>
<news:alt.home.repair>
<news:comp.mobile.android>
<news:alt.comp.os.windows-10>
<news:alt.comp.os.windows-11>
<news:comp.mobile.ipad>
<news:misc.phone.mobile.iphone>

[Grant Taylor]

On 12/13/23 00:22, Wally J wrote:
> I wish we had a name for it but it's like when we talk about unwanted spam.
> No matter what name we use, everyone knows what we're talking about.

There are (at least) three distinct things in the discussion:

1) The copy of Usenet articles that Deja News had which Google acquired.
2) Usenet as it exists today as a set of newsgroups.
3) The groups in the Google Groups system

#1 is a static and has not changed since acquisition.

#2 is what the people outside of Google use.

#3 is what the people inside of Google use.

#2 and #3 are close and related but are not the same thing.

> Yup. I tried valiantly, as did you, I believe, to get Google to add them to
> the auto-archives so that others can run searches to find answers before
> they post a question. I failed.

I got an authoritative refusal to add the newsgroups from the people
that would do it or authorize me to do it.

> NOTE: Interesting I got two different errors for those two URIs just now.

There could be a number of reasons for that and I wouldn't trust Google
1 mm for each bit making up the NOTE: message.

> Yeah. That must be why the Gspammers aren't spamming these Windows ngs.

They aren't because they can't because the group doesn't exist where the
spammers are originating the spam.

> I've noticed a lot of the spam is in funky characters, which I find odd,
> but many also have URLs so they could be phishing attacks for all I know.

The spam is using Unicode, some are quite literally bells and whistles,
to draw attention or to be funny or cute.

> The M0VIE spam is particularly repetitive - I suspect they want to get into
> search engine results - but I don't know that for a fact. I'm just guessing
> as a lot of my tutorials posted to Usenet end up in the first page of hits
> from a "normal" www.google.com search - so Usenet _does_ show up there too.

I don't think that they are trying to get into web search engines.
After all, most of the operations that I saw were quite questionable in
nature. I think they are simply trying to be the most recent "we have
movie" poster so that they appear most recently in the newsgroup when
someone looks for "what's the most recent movie post?". Quite literally
climbing on top of each other and not caring about the trash they leave
behind them.

> BTW, regarding the message from Individual.net, I was heartened they care.
> Anybody have any new datapoints from Giganews & Highwinds admins yet?

IMHO Individual.net is run by an individual, much like most of the text
only Usenet servers. Both GigaNews and HighWinds are commercial
entities and likely don't care unless one of their paying users complains.

[Grant Taylor]

On 12/13/23 07:12, D wrote:
> rather than feed the google monster or other "sanitized" web links, this
> format is more user-friendly because it invites the reader to browse any
> of these aforecited newsgroups using their favorite default newsreaders
>

> <news:comp.mobile.android>

I like your goal D.

But sadly contemporary Thunderbird (115) has forgotten how to honor that
syntax. :-(

Let's try it without the enclosing angle brackets.

news:alt.internet.wireless
news:alt.comp.microsoft.windows
news:uk.telecom.mobile
news:rec.photo.digital
news:alt.home.repair
news:comp.mobile.android
news:alt.comp.os.windows-10
news:alt.comp.os.windows-11
news:comp.mobile.ipad
news:misc.phone.mobile.iphone

[Grant Taylor]

On 12/13/23 09:22, Grant Taylor wrote:
> Let's try it without the enclosing angle brackets.

Nope. :-(

That used to work in Thunderbird.

But, Thunderbird is becoming worse at what it does year after year.

[Andrew]

Grant Taylor wrote on Wed, 13 Dec 2023 09:18:37 -0600 :

>> BTW, regarding the message from Individual.net, I was heartened they care.
>> Anybody have any new datapoints from Giganews & Highwinds admins yet?
>
> IMHO Individual.net is run by an individual, much like most of the text
> only Usenet servers. Both GigaNews and HighWinds are commercial
> entities and likely don't care unless one of their paying users complains.

The 'paying customers' of GigaNews & HighWinds seem to be all spammers. ;->

[Ray Banana]

Thus spake Grant Taylor <gta...@tnetconsulting.net>

> IMHO Individual.net is run by an individual, much like most of the
> text only Usenet servers.

News.Individual.net is provided by FU (Free University) Berlin, one of three State
Universities in Berlin, Germany. It is run by the IT department (ZEDAT)
of FU University.

HTH

--
Пу́тін — хуйло́
http://www.eternal-september.org

[Grant Taylor]

On 12/13/23 09:41, Andrew wrote:
> The 'paying customers' of GigaNews & HighWinds seem to be all spammers. ;->

I question the veracity of that.

I've paid both for service for a short time in the past. Before I stood
up my own news server.

[D]

in the kitchen i've always preferred using antique cast iron skillets
on open-flame burners . . . if it ain't broke don't fix it; similarly
40tude Dialog still runs great on Windows 11 23h2 and older, familiar
hamster scoring, it's freeware, two decades old and works good as new

[Sn!pe]

I would expect that a highly experienced and multiply-qualified Internet
expert such as your interlocutor should know that; is he being playfully
disingenuous?

Would a real Usenet user access News by HTTP? I rather think not.

--
^Ï^. Sn!pe, PA, FIBS - Professional Crastinator
<snip...@gmail.com>
Google Groups articles seen only if poster whitelisted.
My pet rock Gordon just is.

[D]

by content, likely a significant percentage; but by quasi-randomized
crossposting en masse, why any server allows more than three (3) per
article (xpost %>3) seems unreasonable to say the least; it's simple
to forward all or part of an article to another newsgroup (fw:) with
reference headers intact where such content may be topical elsewhere;
if every server would limit crossposting that alone might help a lot

[Scott Dorsey]

In article <ulbhfc$sbm$1...@tncsrv09.home.tnetconsulting.net>,

Grant Taylor <gta...@tnetconsulting.net> wrote:
>On 12/12/23 19:38, Wally J wrote:
>> BTW, on the Android newsgroup some of us are discussing WHY they're doing
>> all this spam, where not every newsgroup is being spammed, it seems.
>
>I'm of the opinion that it's a bunch of different spam campaigns, likely
>by almost as many spammers.

If this is the case, and it's possible, first thing is that they are using
the same script to do it. And secondly, they all are dumping stuff with
the intention of being disruptive rather than the intention of gimmicking
search engines. I suspect initially they were trying to get search engine
results up, but at this point they are just intending to be destructive.

This is why I suspect it's more likely to be one spammer, but I am not
positive.

[Scott Dorsey]

Grant Taylor <gta...@tnetconsulting.net> wrote:
>There are (at least) three distinct things in the discussion:
>
>1) The copy of Usenet articles that Deja News had which Google acquired.
>2) Usenet as it exists today as a set of newsgroups.
>3) The groups in the Google Groups system
>
>#1 is a static and has not changed since acquisition.

This is not true. It has degraded since acquisition which is where the
resentment of older Usenet users comes from.

>#2 is what the people outside of Google use.

Yes.

>#3 is what the people inside of Google use.

Yes.

>#2 and #3 are close and related but are not the same thing.

They are quite remarkably different, especially in that #3 is far more full
of spam than #2.

[Olivier Miakinen]

Hi all,

Le 09/12/2023 21:09, I wrote :
>
> Maybe I have a bad newsreader or I don't know how to use it ?
>
> The fact is, when I have to look for a given topic, and I register to
> the corresponding newsgroup, it takes me a lot of time to read the
> whole newsgroup when there are several months (or years) of retention
> on the server.

... and I am sorry that I didn't respond yet to those who gave me an answer.

Sorry to be seen as rude, it is rather that I do not have the time that I
would like, and that reading and writing in English is more difficult for
me than in French.

But I promise that I will do it.

Regards,
--
Olivier Miakinen

[Grant Taylor]

On 12/13/23 15:57, Scott Dorsey wrote:
> This is not true. It has degraded since acquisition which is where the
> resentment of older Usenet users comes from.

I maintain that the corpus of articles that Google acquired from Deja
News has not changed since the acquisition.

Yes, the interface thereto has gotten worse and yes more has been added
to the newsgroups.

But the articles as they existed in late 2000 still exist and have not
changed.

> They are quite remarkably different, especially in that #3 is far more full
> of spam than #2.

Google is spewing the spam out to the world.

It is by the grace of server administrators retroactively cleaning up
Google's mess that you aren't seeing nearly as much.

But if the outside news admins don't clean things up / block Google
Groups, you will see very close to the same thing as inside of Google.

I'll say it this way: An unfiltered news feed outside of Google will
show very similar things as a news feed inside of Google.

[Grant Taylor]

On 12/13/23 15:54, Scott Dorsey wrote:
> If this is the case, and it's possible, first thing is that they are using
> the same script to do it.

I suspect that some ... creative miscreants have produced code /
programs to spam Google Groups and distributed it -- likely for profit
-- to even lower miscreants who are spewing massive amounts of spam.

Before I started filtering I was seeing very similarly formatted things
in multiple languages advertising very different things.

I suspect it's a common tool and possibly common template therein that
causes many of the similarities.

> And secondly, they all are dumping stuff with the intention of being
> disruptive rather than the intention of gimmicking search engines.

I'm not convinced of that.

> I suspect initially they were trying to get search engine results up,
> but at this point they are just intending to be destructive.

I don't think that it ever did much for a /web/ search engine. Maybe it
did something for a /Usenet/ search engine. But I'm defaulting to /web/
as that's what 90% of people will think of when you ask them what a
search engine searches. ;-)

> This is why I suspect it's more likely to be one spammer, but I am not
> positive.

Let's agree to disagree. :-D

[Grant Taylor]

On 12/13/23 15:02, D wrote:
> by content, likely a significant percentage; but by quasi-randomized
> crossposting en masse, why any server allows more than three (3) per
> article (xpost %>3) seems unreasonable to say the least; it's simple
> to forward all or part of an article to another newsgroup (fw:) with
> reference headers intact where such content may be topical elsewhere;
> if every server would limit crossposting that alone might help a lot

I believe that many news servers have a default for a maximum number of
cross posts.

I believe that Cleanfeed has similar.

But the defaults are likely set to five or more newsgroups.

[Scott Dorsey]

Grant Taylor <gta...@tnetconsulting.net> wrote:
>On 12/13/23 15:57, Scott Dorsey wrote:
>> This is not true. It has degraded since acquisition which is where the
>> resentment of older Usenet users comes from.
>
>I maintain that the corpus of articles that Google acquired from Deja
>News has not changed since the acquisition.
>
>Yes, the interface thereto has gotten worse and yes more has been added
>to the newsgroups.
>
>But the articles as they existed in late 2000 still exist and have not
>changed.

This is possible, but we can't really know with the indices having been
broken so badly. There are messages that I know were in the database
in 2000 which I have copies of, but which I cannot find on google with any
search. They might be there but I have to assume they aren't.

>I'll say it this way: An unfiltered news feed outside of Google will
>show very similar things as a news feed inside of Google.

Unfortunately true, which is why everyone uses some degree of filtering.

[Scott Dorsey]

Let's send a legal disclosure request to Google for user information.

[D]

out of many . . . one

[Wally J]

Grant Taylor <gta...@tnetconsulting.net> wrote

>> I suspect initially they were trying to get search engine results up,
>> but at this point they are just intending to be destructive.
>
> I don't think that it ever did much for a /web/ search engine. Maybe it
> did something for a /Usenet/ search engine. But I'm defaulting to /web/
> as that's what 90% of people will think of when you ask them what a
> search engine searches. ;-)

We know the difference between what http://groups.google.com/g/<nameofng>
searches versus what http://google.com searches where the groups.google.com
search is _only_ inside that singular newsgroup...

But...

I'll just make a point that when I search on the normal google.com search
engine, I sometimes find my own tutorials showing up, especially when the
topic is esoteric...

Which means...

Usenet results can spill into the regular google.com search engine output;
but (to my knowledge), web sites outside of Usenet/GoogleGroups do NOT
spill into the DejaGoogle search engine results.

If the spam wasn't there, DejaGoogle is rather useful for two purposes:
a. Looking things up BEFORE you post to a newsgroup, and,
b. Referencing an article by URI instead of by Message-ID

The beauty is that no newsreader is needed - and no account is needed.
And article retention is, for our purposes, forever (so to speak).
--
Usenet is a way to express what you think is the situation only to find
others who know it better than you do who correct your prior assessment.

[Julieta Shem]

klu...@panix.com (Scott Dorsey) writes:

> Grant Taylor <gta...@tnetconsulting.net> wrote:
>>On 12/13/23 15:54, Scott Dorsey wrote:
>>
>>> This is why I suspect it's more likely to be one spammer, but I am not
>>> positive.
>>
>>Let's agree to disagree. :-D
>
> Let's send a legal disclosure request to Google for user information.

That's an interesting idea.

[Grant Taylor]

On 12/13/23 18:06, Scott Dorsey wrote:
> This is possible, but we can't really know with the indices having been
> broken so badly. There are messages that I know were in the database
> in 2000 which I have copies of, but which I cannot find on google with any
> search. They might be there but I have to assume they aren't.

I wonder if it would be possible, all be it extremely annoying ~>
infuriating, to go backwards message by message from current to find the
message that you're looking for.

> Unfortunately true, which is why everyone uses some degree of filtering.

I believe there are still people who aren't filtering in any way.

[Grant Taylor]

On 12/13/23 18:07, Scott Dorsey wrote:
> Let's send a legal disclosure request to Google for user information.

Good luck.

Google will fight a user disclosure request with a LOT of money.

Then there is the question of what constitutes a user. Is a singular
person using two completely independent and unrelated accounts one or
two users?

[Grant Taylor]

On 12/13/23 19:41, Wally J wrote:
> We know the difference between what http://groups.google.com/g/<nameofng>
> searches versus what http://google.com searches where the groups.google.com
> search is _only_ inside that singular newsgroup...

We do. But does the average user using (unqualified) Google searching
for movies going to find web sites or news articles?

What if they use a different (unqualified) search engine?

My point being that even searching for the exact phrase in one of the
spam is going to be fairly unlikely to pop up on the first page of
(unqualified) Google search results. And it's even less likely to pop
up on the first page of some other search engine.

> I'll just make a point that when I search on the normal google.com search
> engine, I sometimes find my own tutorials showing up, especially when the
> topic is esoteric...

Yes, it is possible.

I suspect that the plethora of much things for streaming movies will
drown out such a Google Groups post so far past the first page that it's
not even funny.

> Which means...
>
> Usenet results can spill into the regular google.com search engine output;

Yes, it is possible. I just think that it will be extremely unlikely
given the context that the spam I've seen is advertising.

> but (to my knowledge), web sites outside of Usenet/GoogleGroups do NOT
> spill into the DejaGoogle search engine results.

Agreed.

But how many people will search inside of Google Groups Usenet gateway
search for movies to download? The people that will actually do so will
probably use a different search which is associated with their
commercial Usenet provider.

Sure, there will be a few people that try it and run into problems. But
the people actually using Usenet to transfer binary content almost
certainly aren't in the same group.

> If the spam wasn't there, DejaGoogle is rather useful for two purposes:
> a. Looking things up BEFORE you post to a newsgroup, and,
> b. Referencing an article by URI instead of by Message-ID

I absolutely agree.

> The beauty is that no newsreader is needed - and no account is needed.

I'd like to think that some other news provider also provides a web
interface.

Though I don't know about the account part.

> And article retention is, for our purposes, forever (so to speak).

True enough.

[Wally J]

Grant Taylor <gta...@tnetconsulting.net> wrote

>> This is possible, but we can't really know with the indices having been
>> broken so badly. There are messages that I know were in the database
>> in 2000 which I have copies of, but which I cannot find on google with any
>> search. They might be there but I have to assume they aren't.
>
> I wonder if it would be possible, all be it extremely annoying ~>
> infuriating, to go backwards message by message from current to find the
> message that you're looking for.

I've been on Usenet for as long as anyone, and certainly prior to the
takeover of DejaNews by Google (which I'll call groups.google.com below).

Every once in a while I look for one of my old Usenet tutorials, where my
observation is that it's amazing how _different_ the results are when using
the www.google.com versus groups.google.com search engines (i.e.,
www.google.com is better at finding things by keyword, and specifically for
requiring and/or excluding items with the "+" and/or "-" directives.

Having observed that, I will also note that, to my recollection, in the
end, I _always_ find what I am seeking using the groups.google.com search
engine - so my plain assessment is that it's all there.

Unfortunately, as of only recently, the inordinate google-based spamming
is, in some groups, currently 99.5% of the spam (e.g., on c.m.a).
<https://groups.google.com/g/comp.mobile.android>

>> Unfortunately true, which is why everyone uses some degree of filtering.
>
> I believe there are still people who aren't filtering in any way.

In teh olden days, I implemented procmail filters, for the occasional spam
of one or two a week, and then I had to rely on the POP3/IMAP4 servers
handling that task, as it had suddenly become immense a decade or so ago.

For Usenet...

Up until I opened the thread asking where the google-based spam was coming
from, the only filters I had were for the likes of Snit & Sn!pe & % (aka
Rod Speed). Probably a half dozen nyms were plonked over the years.

Now I'm _forced_ to implement filters, where, shockingly, they're so simple
as to simply delete anything and everything that comes from google groups.

My telnet scripts will grab upwards of five hundred to a thousand articles
a day on c.m.a., only to result in an actual count of merely a handful.

At this point, for some Usenet newsgroups, much as we used procmail in the
olden days of SMTP, for Usenet, user filtering is currently de-rigueur.

Let's hope the "de-peering" that was initially recommended occurs, or, even
better, that enough people complain to Google such that they fix the issue.

*Please Do This!*
<https://groups.google.com/g/google-usenet/about>
--
The whole point of Usenet is to find team mates who know more than you do.
And to contribute to the overall tribal knowledge value of the newsgroup.
It's a domino effect where each of us helps the next person in the lineup.