Usenet in 2022, some thoughts and impressions

[Alterego]

I found this group due to reading some reddit posts about usenet. I'm
somewhat fascinated with usenet but I didn't start with it back in its
heyday so I only know it since around 2013. I find the talk of a usenet
revival kind of interesting but most of what I've seen is centered around
usenet as a discussion forum. I'm all for that but usenet is somewhat of
a stem cell technology that has been used for multiple purposes and many
modern internet technologies have evolved from it. I've just noted down
some of my thoughts and impressions on the current usenet and this group
looks like a good place to post them. I have no desire to join reddit to
post about usenet.





1. The Usenet Jungle.

When I fist started playing around with usenet, the first thing I noticed
was that a there was a lot of content that was similar to what's found on
the dark web. It's something like hiding in plain sight. There are so
many groups and so much content that it's very easy to post something and
have it be completely invisible to the web and most of the outside world.
Even though usenet is an open standard, there is so much content on it so
poorly indexed that it's very easy to hide something in the huge mass of
data that makes up usenet. There are 111,082 groups listed in my
newsreader. Some are filled with posts, others haven't been posted on for
years. Only some of it is indexed by usenet indexes on the web. Google
doesn't index binary groups and the binary indexers only index a very
small percentage of the binary groups so there are many groups that can
only be viewed with a newsreader. Usenet is like a huge urban space
filled with abandoned real estate with some areas still active and
thriving. There's plenty of territory for a discreet post or two. As in
any big urban area, some neighborhoods are a lot nicer than others.


2. The Usenet Library

Usenet is one of the biggest online libraries in existence. The ever
increasing retention of usenet means that there is a huge amount of
content posted since 2008 that is accessible to anyone with the patience
to look for it. I got into usenet long after its heyday and I was mainly
interested in old comics and magazines and this is the section of usenet
I know the best but what I say also applies to music, radio drama, old
cinema and just about any other form of media. And it's not all piracy
either. There is a lot of obscure material that wouldn't be hosted on
commercial web sites due to lack of interest whether legal or pirate. In
one magazine group, I can find things like copies of the Saturday Evening
Post going all the way back to 1865, for example. In many cases, even if
there still is a valid copyright on the material, it is so obscure that
the copyright holders really don't care or haven't noticed. The comic and
magazine binary groups are very open and civil for the most part and the
uploaders tend to be gentlemen archivers who have a real love for the
content they upload and preserve, not the typical pirates and warez
groups who also use usenet. It's pretty common to see both text posts
about the content and binary posts of the content in the same group.
Usenet has a nice flexibility that way. Using a newsreader to find this
sort of material does remind me of going through a card catalog at a
library to find a book that interested me. Using usenet for file sharing
goes way back. Usenet was where file sharing started and file sharing is
one of its main uses today and what keeps it economically viable. The
sharing culture that started on usenet moved on to napster, torrents and
cyberlockers and became less friendly and more about money. On usenet,
it's still pretty friendly in places.


3. Usenet as cloud storage.

Usenet is not only the ancestor of forums and social media, it is also
the first form of cloud storage and it can be used for both communication
and online data storage. Usenet is actually a very good system for
storing data in the cloud due to its decentralized nature with data being
duplicated on multiple independent servers around the world not under the
control of a single corporation or institution. The same encryption and
obfuscation technologies, ie ngPost, that are used to post pirated
content these days can also be used for personal data backups. NgPost can
automatically compress and encrypt data into multipart rar files with par
files and split post all the parts across multiple groups with random
subjects and authors. It will create an nzb file for the post and without
the nzb file, it's virtually impossible to reconstruct the original
material and even if it's done, there's still the encryption to deal with
which, of course, can be much more than the rar password used in ngPost.
The data could have several layers of encryption put on it before it goes
to ngPost for the final processing and uploading.

4. Usenet as a communication medium.

This is what usenet was originally designed for and it still works as
well today as it ever did. It has some real advantages over social media
and forums, once again due to its decentralized nature. Free speech
reigns on usenet. Moderation is practically non existant these days and
you can say and post whatever you want, nobody's going to delete it,
nobody's going to like or dislike it, up or down vote it, all they can do
is reply if they feel so inclined. There is one caveat to free speech on
usenet and it's actually a good one. There's no such thing as deletion,
once you post something, it's up forever and there's no taking it back so
it's best to think about what you're saying before you post it. Another
thing I like about posting on usenet, is that you are making your post on
your own computer and what you post isn't public until you hit the send
button. It's occurred to me as I was writing something on gmail or
facebook, what I typed was going to directly to the server and the server
could keep drafts of what I typed that I later revised or deleted without
informing me. I much prefer to post my thoughts in final form and not let
google or facebook in on the thought process that led me to the final
post. Another great advantage of usenet is that there's no account,
registration, activation or anything required to read and post other than
having a usenet account and newsereader and finding the groups you want
to post in. You can't be denied entrance or blocked from a group. Other
users who don't like your posts may block them as individuals from their
newsreader with a scoring rule but there's no authority from above that
can globally block or ban you. And finally, you can post anonymously on
usenet if you so desire or post using a real name and email. It's up to
you. You can make up any kind of posting identity you want, you can have
multiple identities for different groups.




To sum up my thoughts after 9 years or so of playing around with usenet,
the key word would be flexible. I started casually with usenet for just
one purpose and have found it to be a multi faceted multi use technology
that in many ways is better than what it was replaced with. It's from a
truly decentralized internet that was much smaller in terms of size and
bandwidth but also an internet that wasn't dominated by a few large and
wealthy corporations that we refer to as big tech.

[meff]

On 2022-02-21, Alterego <A...@alt.org> wrote:
> 1. The Usenet Jungle.
>
> When I fist started playing around with usenet, the first thing I noticed
> was that a there was a lot of content that was similar to what's found on
> the dark web. It's something like hiding in plain sight. There are so
> many groups and so much content that it's very easy to post something and
> have it be completely invisible to the web and most of the outside world.

Lots of the Web isn't indexed, and any part of the Net that isn't
found on the Web is like this too. You can call it the "dark web", but
I don't know if that term has ever had a particular rigorous
definition. You can still find unindexed websites where you can post
whatever you want. You can do so on your own website! Hop onto a
Matrix or IRC room/channel that's not indexed and likewise you'll be
posting content that "nobody" knows about.

> Usenet is not only the ancestor of forums and social media, it is also
> the first form of cloud storage and it can be used for both communication
> and online data storage. Usenet is actually a very good system for
> storing data in the cloud due to its decentralized nature with data being
> duplicated on multiple independent servers around the world not under the
> control of a single corporation or institution. The same encryption
> and

The issue here is retention. You would need to keep posting your
backups to keep them from being dropped due to retention
reasons. Moreover, if you ever lose your NZB files, you're essentially
SOL. But yeah this is a pretty interesting idea. It's one I've thought
of myself.

> that in many ways is better than what it was replaced with. It's from a
> truly decentralized internet that was much smaller in terms of size and
> bandwidth but also an internet that wasn't dominated by a few large and
> wealthy corporations that we refer to as big tech.

There's nothing inherent about Usenet that stops it from being run by
big tech. In fact, Email is a close cousin of Usenet and is dominantly
run by big tech firms (Google, Microsoft, et al.) The only reason that
hasn't happened here is that the folks that still use Usenet to derive
value either a) are downloading binaries through NZB and don't
actually care about the spam on Usenet or b) the kinds of folks who
are motivated enough to judiciously lean on their killfiles/scorefiles
in order to curate their Usenet experience and fight spam. If people
en masse still used Usenet, you'd see the same issues arise as with
Email. Moderated groups used to exist to try to provide islands of
calm, but most moderated groups these days are dead and the moderators
themselves unresponsive.

[Alterego]

By dark web I was referring to TOR .onion domains, not unindexed web
material.

The retention on the Omicron backbone goes back to August 2008 and it's
not moving. I don't know if that will last but you do get a lot of
retention these days from a good provider. One thing is that data
capacity and bandwidth has grown enormously in the last 10 years and it
probably doesn't use up a lot of capacity to store older binary files.
Who was uploading 10-20gb obfuscated movie files in 2008? It's the newer
binaries that would eat up hard drive space on a usenet server.

True that there's nothing in Usenet that keeps it from being run by big
tech but the inverse applies as well to usenet and email. I've set up
email servers in the past on VPSes and the option is there. Right now I'm
migrating some email accounts away from Google due to the end of free
legacy Google Apps accounts. You lose a bit of data redundancy by not
having your email on corporate servers but you gain a lot of privacy.
Most web hosting accounts come with lots of email. Mine allows up to 1000
email accounts and does have web email but I prefer to use Thunderbird
which is standard with Linux. One thing I like about using this older
internet technolgy is that the server connection isn't constant, it's on
demand. That's an enormous privacy benefit. No web interface running java
script, with cookies and constant tracking, you're connecting to a server
at sporadic intervals with client software on your own machine.

[meff]

On 2022-02-25, Alterego <A...@alt.org> wrote:
> The retention on the Omicron backbone goes back to August 2008 and it's
> not moving. I don't know if that will last but you do get a lot of
> retention these days from a good provider. One thing is that data
> capacity and bandwidth has grown enormously in the last 10 years and it
> probably doesn't use up a lot of capacity to store older binary files.
> Who was uploading 10-20gb obfuscated movie files in 2008? It's the newer
> binaries that would eat up hard drive space on a usenet server.

Yeah while this is a tangent, I wonder what the distribution of file
sizes is. As you say, it's probably weighed toward newer files as
newer files are larger.

> True that there's nothing in Usenet that keeps it from being run by big
> tech but the inverse applies as well to usenet and email. I've set up
> email servers in the past on VPSes and the option is there. Right now I'm
> migrating some email accounts away from Google due to the end of free
> legacy Google Apps accounts. You lose a bit of data redundancy by not
> having your email on corporate servers but you gain a lot of privacy.
> Most web hosting accounts come with lots of email. Mine allows up to 1000
> email accounts and does have web email but I prefer to use Thunderbird
> which is standard with Linux. One thing I like about using this older
> internet technolgy is that the server connection isn't constant, it's on
> demand. That's an enormous privacy benefit. No web interface running java
> script, with cookies and constant tracking, you're connecting to a server
> at sporadic intervals with client software on your own machine.

Definitely, no disagreements here. It's true that despite email being,
de facto owned by Big Tech, small players _can_ still run their own
email infrastructure and opt out of their conditions as
necessary or desired. I'm just trying to say, there's a world where
big corporations or big entities in general _can_ simply offer
services that interoperate with services offered by smaller
organizations/firms/individuals.

In this world, competition can still carve out niches for competitors
who exploit gaps in the coverage of big firms. ProtonMail and Fastmail
are great examples of this. Alongside bespoke infrastructure run by
individuals and small netops, these companies offer a service that
offers value that the big tech companies just will not or cannot
deliver. I certainly think the state of social media would be much
more healthy if this sort of protocol-level interoperation were
mandated, but alas we live in a world where Email and Usenet are
recreated in proprietary silos every 5 years or so.

https://app.media.ccc.de/v/36c3-11086-the_ecosystem_is_moving is a
good talk by Moxie Marlinspike, the creator of Signal, on some of the
issues that these approaches face. Ossification is a well-known
failure mode of standardized protocols.

[5GyYap52yQ1UGMWD]

Hi Alterego,

Alterego <A...@alt.org> writes:

>
> 1. The Usenet Jungle.
>
> When I fist started playing around with usenet, the first thing I noticed
> was that a there was a lot of content that was similar to what's found on
> the dark web. It's something like hiding in plain sight. There are so
> many groups and so much content that it's very easy to post something and
> have it be completely invisible to the web and most of the outside world.
> Even though usenet is an open standard, there is so much content on it so
> poorly indexed that it's very easy to hide something in the huge mass of
> data that makes up usenet. There are 111,082 groups listed in my
> newsreader. Some are filled with posts, others haven't been posted on for
> years. Only some of it is indexed by usenet indexes on the web. Google
> doesn't index binary groups and the binary indexers only index a very
> small percentage of the binary groups so there are many groups that can
> only be viewed with a newsreader. Usenet is like a huge urban space
> filled with abandoned real estate with some areas still active and
> thriving. There's plenty of territory for a discreet post or two. As in
> any big urban area, some neighborhoods are a lot nicer than others.
>

For me, this is one of USENET's strength and weakness. It's hard to find
groups that are active but if you do find one your efforts will be
gratified by being able to discuss with likeminded
individuals. Whenever I'm bored, I just go through the list of groups
and open one up. Most of the time, I just get a massive wall of spam
that's was built over years of neglect but sometimes I find an active
community that still fairly active.

100,000 groups are no joke. Even if assuming that 0.5% of that are only
active, that's still 500 different groups scattered in this forgotten
internet frontier.

I agree. Though I haven't been in a position, yet, where I find the need
to look things up in USENET. I am trying to pull all of the posts that I
can get though from the groups that I'm interested in. You'll just never
know when you'll need to look for answers that even Google can't find
for you.

>
> 3. Usenet as cloud storage.
>
> Usenet is not only the ancestor of forums and social media, it is also
> the first form of cloud storage and it can be used for both communication
> and online data storage. Usenet is actually a very good system for
> storing data in the cloud due to its decentralized nature with data being
> duplicated on multiple independent servers around the world not under the
> control of a single corporation or institution. The same encryption and
> obfuscation technologies, ie ngPost, that are used to post pirated
> content these days can also be used for personal data backups. NgPost can
> automatically compress and encrypt data into multipart rar files with par
> files and split post all the parts across multiple groups with random
> subjects and authors. It will create an nzb file for the post and without
> the nzb file, it's virtually impossible to reconstruct the original
> material and even if it's done, there's still the encryption to deal with
> which, of course, can be much more than the rar password used in ngPost.
> The data could have several layers of encryption put on it before it goes
> to ngPost for the final processing and uploading.
>

This is interesting. I have been a frequent binary user in the past as
well, but I've never really understood how to make those nzb files. I
might consider taking a look just to backup some of my files. One
question, how secure are they? I'm considering bundling them into .tar
files and encrypting it though GPG. Would that be enough to make sure
that my files will not be snooped on by someone?

I agree. A lot of people always make to a point that moderation is a
necessary evil to combat spam. While I can certainly see where they're
coming from, I think there's merit in having a purely unmoderated
discussion where the "moderation" is at the user's hands.

Over the years of being a web denizen, I've had my fair share of
moderation power-tripping on the various forums and groups that I've
been a part of. While there are moderated communities that can exist for
years without melting down in a sea of fire and passion I find those to
be the exception, not the rule. You can even see this with Discord
"servers" today, where some power trippy bloke grinds the gears of his
other mates so much that drama tears the "server" apart.

With unmoderated USENET, that's not the case. Sure there will always be
pricks and spammers but they're always a single scorefile and killfile
away.

--
Pointless meanderings in a bleak and lonely world.