Old USENET / Netnews (pre-Dejanews) archives: discussion

T. Gryn

unread,

Aug 31, 1999, 3:00:00 AM8/31/99

to

For those interested in searching past USENET posts, IMHO
DejaNews is an invaluable resource. Considering the large
number of posts made per day, that any one company can
archive so much material, then make it searchable, is a rather amazing
accomplishment. It would be a serious loss to the online
community if DejaNews should ever disappear, or their
database be lost.

Having said that...

Up until their recent reorganization, DejaNews appeared to
also be interested in recovering the USENET posts made
prior to DN's inception (mid-'95). Unfortunately, based on their
current online FAQs, it appears that this effort has been
discontinued.

In my personal research, I've come across only two WWW sites
which really begin to address this need:

---
http://communication.ucsd.edu/A-News/

(USENET OldNews Archive; has all USENET posts from May '81
to May '82)

http://starbase.neosoft.com/~claird/news.lists/newsgroup_archives.html

(document written by Cameron Laird in '94, lists the USENET archives
which existed at that time.)
----

And that's it. From searching in DejaNews for past discussions
of this topic, it appears that prior to DejaNews, SIMTEL was selling
CDROMs of USENET post collections, though it does not appear that
any attempt to collect these CDs has been made. How many
of these CDROMs are in private hands is anyone's guess.
A number of the archives which Mr. Laird lists have disappeared.

My concern is that, the longer this material goes uncollected,
it will be harder and harder to reclaim it in any usable form.
In addition to it's value to future digital archeologists, there's
the collected thoughts of thousands, if not tens of thousands,
of individuals which are encompassed in these posts.

It just seems a terrible waste for them to be lost. Yes, many
of them weren't particularly memorable, but there's a lot there
that was. It's also a history of the development of the Internet,
through the online discussions of USENET posters.

In addition to soliciting other's thoughts on this issue,
I have these specific questions. Any help with them
would be appreciated!

(1) Has DejaNews abandoned their efforts to recover pre-1995
USENET posts? If so, did they succeed in recovering any
archival material prior to this discontinuation, and if yes, what
is the extent of what they have?

For example, the webmaster of the A-News archive
emailed me that he has the USENET posts from the end of
May '81 to some time in '85, and he has passed them along
to DejaNews several years ago, but hasn't heard anything
back from them. He doesn't have the resources to organize
or post them on the WWW or in FTP; the size of his archive
is around 12-14 GB.

(2) Has there been any effort beyond DejaNews' to collect
pre-1995 USENET material, by anyone? I'm not sure if the
Internet Archive is conducting anything like this or not...

Mainly, though, I'm curious to see if anyone else is concerned
that the olde posts are basically being lost to time, and if anything
has (or can) be done about it.

Thanks for your time.

Tom Gryn..............gryn.1@osu.edu

P.S. Please excuse the cross-posting, but I couldn't find a newsgroup
which seemed to fit the topic precisely.

Ian Stirling

unread,

Sep 1, 1999, 3:00:00 AM9/1/99

to

T. Gryn <gry...@osu.edu> wrote:
>For those interested in searching past USENET posts, IMHO
>DejaNews is an invaluable resource. Considering the large
>number of posts made per day, that any one company can
>archive so much material, then make it searchable, is a rather amazing
>accomplishment. It would be a serious loss to the online
>community if DejaNews should ever disappear, or their
>database be lost.

I wonder if FOIA'ng certain TLAs might yeild results.

There is of course the problem of the size of the database.
0-85 might be 20G, which is under $100, even if held online.

85-95 I don't really have a good handle on, I'd guess around at least
3Tb, which would be substantially more expensive.

I'd be willing to setup http://www.oldnews.com/ (probably exists)
and do a dejanews like 0-95 service, but the finance needed is way beyond
my means.

One of the large news vendors might be able to do this, with substantial
investment, though I don't think it's likely.

I'd be very willing to pay for such a service, though not that much, maybe
10 pounds/year, or 5p/search.
(pound = 1.66 US dollars)

It would be a great shame if all that history has been lost.

Brad Knowles

unread,

Sep 1, 1999, 3:00:00 AM9/1/99

to

In article <936208595.17360.0...@news.demon.co.uk>, Ian
Stirling <ro...@mauve.demon.co.uk> wrote:

> It would be a great shame if all that history has been lost.

I don't believe that it has been lost. Back when they were first
setting up DejaNews (and before it was publicly available), they came to
AOL to give a demo of their system.

I spoke with the primary technical person who was there (I think he
had brought with him the one server they had set up so far), and I was
told that they were buying their backfill archive from Uunet, who had an
archive of all USENET news posts that they had ever seen.

At that point, they were buying one month's worth of backfill archive
each month, and archiving their own month as they saw articles come in.
So, after three months of operation, they had three months of their own
news articles archived online, plus three months further back that they
had bought from Uunet.

I don't know how far they carried this operation, and it's entirely
possible that they've since thrown away everything they bought from Uunet
(no need for it, with the years that have gone by).

Of course, I don't think anyone anywhere is archiving all the binary
posts, so even if it was 1GB per day for text-only posts going all the way
back to 1969 (30 years), that'd only be 10.693TB of storage.

Since we know that growth has been exponential over this time
(reaching 1GB/day now), I would expect that it would actually be far less
than this. Even 10% compound annual growth rate over 30 years would
result in only 3.428TB of storage.

--
Brad Knowles <br...@shub-internet.org> <http://www.shub-internet.org/brad/>
<http://wwwkeys.pgp.net:11371/pks/lookup?op=get&search=0xE38CCEF1>

Jeremy

unread,

Sep 1, 1999, 3:00:00 AM9/1/99

to

Brad Knowles <br...@shub-internet.org> wrote:

>> It would be a great shame if all that history has been lost.
>
> I don't believe that it has been lost. Back when they were first
> setting up DejaNews (and before it was publicly available), they came to
> AOL to give a demo of their system.
>
> I spoke with the primary technical person who was there (I think he
> had brought with him the one server they had set up so far), and I was
> told that they were buying their backfill archive from Uunet, who had an
> archive of all USENET news posts that they had ever seen.

Since you last mentioned that alleged archive to me, I have been unable
to verify its existence. If it existed, I would think that someone would
know about it, so at this point I'm largely convinced that it does not
exist.

At RemarQ, my understanding is that we have back to 1995-ish pretty well
covered (though not yet online). I have also found a potential source of
archived articles from the 1980s, most of which are apparently on ancient
magtapes (which is not necessarily a show-stopper, just a cost-raiser).
I'm trying to get the right people in touch with the right people on that
front. I have not found anyone who even claims to have the early 1990s
(though I admit I'm not conducting an exhaustive search). If anyone
*does* have such archives, I want to hear from them.

--
Jeremy | jer...@exit109.com
"How extravagant you are, throwing away women like that. Someday they
may be scarce." (Casablanca)

Uwe Brockmann

unread,

Sep 1, 1999, 3:00:00 AM9/1/99

to

In article <rsqvp6...@news.supernews.com>,

Jeremy <jer...@exit109.com> wrote:
>. I have not found anyone who even claims to have the early 1990s
>(though I admit I'm not conducting an exhaustive search). If anyone
>*does* have such archives, I want to hear from them.

I seem to remember that Young Minds, Inc., once offered or planned to
offer archived newsgroups on CD-ROM for sale. I may be wrong about
this. I also do not know which period might be covered.

They have a web site at http://www.ymi.com which has no information on
any such archival effort. However, you could contact them and ask.

Uwe Brockmann, u...@netcom.com

Scott Hazen Mueller

unread,

Sep 2, 1999, 3:00:00 AM9/2/99

to

In the quoted message, 'Jeremy <jer...@exit109.com>' wrote:

>At RemarQ, my understanding is that we have back to 1995-ish pretty well
>covered (though not yet online). I have also found a potential source of
>archived articles from the 1980s, most of which are apparently on ancient
>magtapes (which is not necessarily a show-stopper, just a cost-raiser).
>I'm trying to get the right people in touch with the right people on that

>front. I have not found anyone who even claims to have the early 1990s

>(though I admit I'm not conducting an exhaustive search). If anyone
>*does* have such archives, I want to hear from them.

I don't recall what period Henry Spencer had archived. I believe from the
start of Usenet around 1980 forward to some indeterminate point. He had
loaned his tapes to David Wiseman at UWO, who said in private e-mail to me
last year that he'd gotten the data off the tapes, and work was needed on
organizing the archives. Contact me if you'd like to get in touch with David
to offer help.

\scott

Cameron Laird

unread,

Sep 2, 1999, 3:00:00 AM9/2/99

to

In article <37cc36e5...@nntp.service.ohio-state.edu>,

T. Gryn <gry...@osu.edu> wrote:
>For those interested in searching past USENET posts, IMHO
>DejaNews is an invaluable resource. Considering the large

.
.
.

>http://starbase.neosoft.com/~claird/news.lists/newsgroup_archives.html
>
>(document written by Cameron Laird in '94, lists the USENET archives
>which existed at that time.)

.
.
.
A few words of clarification: while I started this
document in '94, I wrote most of it in '95. It's
presented not as archives of USENET, but rather as
archives of particular newsgroups. I still occasi-
onally add new sites, as I locate them. I'd estimate
old sites go stale at a considerably higher rate than
new ones are created. Most of my time now goes to
other projects, so I'm not even purging broken links
more than spasmodically.

It'll surprise people how little space is taken up by
very interesting slices of Usenet. All of soc.culture.*,
say, or comp.protocols*, can reasonably by maintained on
modest hardware.

I'll cheerfully volunteer advice and even code to anyone
who wants to set up archives. It's not hard to do.
--

Cameron Laird http://starbase.neosoft.com/~claird/home.html
cla...@NeoSoft.com +1 281 996 8546 FAX

T. Gryn

unread,

Sep 2, 1999, 3:00:00 AM9/2/99

to

On Thu, 2 Sep 1999 00:10:02 GMT, sc...@zorch.SF-Bay.ORG (Scott Hazen
Mueller) wrote:
>I don't recall what period Henry Spencer had archived. I believe from the
>start of Usenet around 1980 forward to some indeterminate point. He had
>loaned his tapes to David Wiseman at UWO, who said in private e-mail to me
>last year that he'd gotten the data off the tapes, and work was needed on
>organizing the archives.

I think this is the same resource which the A-News archive was taken
from. According email from Bruce Jones, one of the webmasters of that
site, "After about mid-1985 Henry stopped archiving the entire news
spool and only saved stuff like rec.birds (UT Zoology) and
comp.sources." So, that's about the extent of it, apparently.

Tom Gryn...................gryn.1@osu.edu

Bernie Cosell

unread,

Sep 3, 1999, 3:00:00 AM9/3/99

to

br...@shub-internet.org (Brad Knowles) wrote:

} Of course, I don't think anyone anywhere is archiving all the binary
} posts, so even if it was 1GB per day for text-only posts going all the way
} back to 1969 (30 years), that'd only be 10.693TB of storage.

A few posts in this thread have mentioned 1969, and I'm not sure why. 1969
is when the ARPAnet first went into operation, but that's most orthogonal.
Usenet didn't start up until 19>7<9. so you're only looking at 20 yrs, not
30.

/Bernie\
--
Bernie Cosell Fantasy Farm Fibers
ber...@fantasyfarm.com Pearisburg, VA
--> Too many people, too few sheep <--

Brad Knowles

unread,

Sep 3, 1999, 3:00:00 AM9/3/99

to

In article <37d1238f....@news.remarq.com>, ber...@fantasyfarm.com
(Bernie Cosell) wrote:

> A few posts in this thread have mentioned 1969, and I'm not sure why. 1969
> is when the ARPAnet first went into operation, but that's most orthogonal.
> Usenet didn't start up until 19>7<9. so you're only looking at 20 yrs, not
> 30.

I was trying to set an upper limit on the amount of disk space that
might be required. Since I didn't know exactly when USENET came into
existance, I figured I'd use a conservative figure based on the overall
age of ARPAnet. If it's only twenty years and not thirty, the storage
requirements for storing all non-binary articles that have ever been
posted should be even further reduced.

Drew Thomas

unread,

Sep 3, 1999, 3:00:00 AM9/3/99

to

So, someone must have those on archive, by the look of it, you could
store it all on a new PC.

In article <brad-03099...@brad.techos.skynet.be>, Brad Knowles
<br...@shub-internet.org> writes

>In article <37d1238f....@news.remarq.com>, ber...@fantasyfarm.com
>(Bernie Cosell) wrote:
>
>> A few posts in this thread have mentioned 1969, and I'm not sure why. 1969
>> is when the ARPAnet first went into operation, but that's most orthogonal.
>> Usenet didn't start up until 19>7<9. so you're only looking at 20 yrs, not
>> 30.
>
> I was trying to set an upper limit on the amount of disk space that
>might be required. Since I didn't know exactly when USENET came into
>existance, I figured I'd use a conservative figure based on the overall
>age of ARPAnet. If it's only twenty years and not thirty, the storage
>requirements for storing all non-binary articles that have ever been
>posted should be even further reduced.
>

--
Drew Thomas, subscriber to:
alt.tv.southpark
alt.tv.simpsons
microsoft.public.ageofempires
rec.puzzle
AND PROUD OF IT!!!

Joe Bernstein

unread,

Sep 3, 1999, 3:00:00 AM9/3/99

to

This is partly a response to various posts in the thread but mainly
to the post I'm actually following up to. Apologies if this is off-topic
anywhere else - like the original poster, I can't think of a better set
of groups. The reason I don't think it's off-topic to news.groups is
actually the reason I'm interested in the thread. Our newsgroup line
reads "Discussion and lists of newsgroups". For some time now, I've
wanted to recover lists of newsgroups as of old dates, in order to
construct a chronology of newsgroup creation and destruction; and the
period of primary interest to the original poster (roughly 1985-1995)
remained mostly* equally blank to me after an extended effort in this
direction in 1997. I have a couple of documents related to the Great
Renaming, which do not purport to list all newsgroups; books on Usenet
as early as the mid-80s apparently had stopped attempting to do so,
quite aside from the impossibility of dating lists in books.

So. Um. This in *turn*, believe it or not, is partly in service to
the charter archive project which I endlessly hope someday to be able
to do. And the Great Renaming FAQ (whose acknowledgements gave me
crucial pointers in 1997), the charter archive project, and lists of
newsgroups (before the creation of news.lists), have all been topics
of news.groups. So.

In article <37cc36e5...@nntp.service.ohio-state.edu>,
T. Gryn <gry...@osu.edu> wrote:

> For those interested in searching past USENET posts, IMHO
> DejaNews is an invaluable resource. Considering the large

> number of posts made per day, that any one company can
> archive so much material, then make it searchable, is a rather amazing
> accomplishment. It would be a serious loss to the online
> community if DejaNews should ever disappear, or their
> database be lost.

Since I still think that DejaNews *will* die the first time someone with
enough money sues them over copyright, I actively fear that day. Even
with their new interface (which among other things seems inferior for
tracing threads to the most recent version of their old), they remain
invaluable. I'm delighted to hear that RemarQ (is *that* how you
capitalise it?) is archiving too, particularly given that Jeremy is
there, but since I've seen two attempts to compete with DejaNews
fail (Altavista and Reference.com), I still think the day of that
lawsuit against DejaNews will be a bad, bad day.

> Having said that...
>
> Up until their recent reorganization, DejaNews appeared to
> also be interested in recovering the USENET posts made
> prior to DN's inception (mid-'95). Unfortunately, based on their
> current online FAQs, it appears that this effort has been
> discontinued.

Is this positively stated, or is it just that they don't ask?

They may very well figure that if nobody's written by now in response
to that ad, nobody is going to. I think it's highly unlikely that
people sorting out their elderly relatives' estates are going to come
across Usenet archives on diskettes or whatever, and think immediately
"Oh! I should give this to DejaNews!" It's perfectly possible that,
somewhere, there's someone who, like me, was offline for an extended
period, and will find DejaNews on coming back online (as I did, very
quickly); but I have a hard time imagining that someone who had been
archiving a significant portion of Usenet would fit that scenario. So
I suspect that DejaNews probably has not seen any useful replies to
that ad in a couple of years, and in preference to continuing to have
to thank people every couple of weeks for pointers to Bruce Jones's
OldNews, they took the ad down.

> In my personal research, I've come across only two WWW sites
> which really begin to address this need:
>
> ---
> http://communication.ucsd.edu/A-News/
>
> (USENET OldNews Archive; has all USENET posts from May '81
> to May '82)

No, it certainly doesn't. I know this because I once combed it for
information on the creation of groups listed there. (See above.)
There are many cases where there's a post thanking someone for creating
a group, but the post that actually created the group is absent. (In
those days you didn't have control messages. You created a group by
posting to it. Hence all group-creation posts would be archived in
a 'complete' archive, even if as a matter of some sort of policy it
excluded control messages.)

Please remember that "propagation" used to *mean* something, even
before there was an alt.*.

I was on Usenet basically only from about February to June of 1985,
and at that time propagation was still pretty erratic. My server,
the University of Chicago's, was exceptionally well-served, but still.
Maps of Usenet were still being posted at that time, basically to
enable people to get better feeds (whether more full, or simply cheaper,
I couldn't say!). I think I remember the lists of active groups of that
time as fairly routine affairs; it's worth noting though that the lists
posted in the OldNews archive are much more slapdash and have a variety
of authors. (I seem to recall that the second list I found there did
not know of the existence of the first, and omitted groups the first
had listed; but the second appeared to be the True Ancestor of the
document now posted as a sort of checkgroups by David Lawrence.)

The OldNews archive is based on the University of Toronto's incoming
feed, which is certainly an excellent feed for it to *be* based on.
But I'm not sure how news worked over the period of the archive (both
the parts webbed and the parts through 1985 which are still inaccessible).
Seems to me that it's at least *possible* that during some or all of that
period, Toronto might have had a system which accepted (or at least stored)
posts only for groups on its active list; certainly by 1985 the U of
Chicago had a functioning active file - you could not create newsgroups by
typo at our site. In that case, whatever groups-by-typo or controversial-
groups or whatever, that Toronto didn't carry, would be out in the cold.

Certainly there are groups represented in the OldNews archive which
were visibly in existence long before OldNews contains postings for
them, for that matter. There's some discussion visible somewhere
on Bruce Jones's site (I forget whether in the archive, around it,
or in the other side of the site which is a mailing list archive on
the history of Usenet) - somewhere, anyway, there's talk about how
Toronto picked up Usenet gradually. It's not very clear but if you
comb the archive with dates of postings in mind you get the picture:
There are several groups which OldNews only carries from a date in
1982, but which quite obviously have ongoing discussions and
traditions as of the first post present in OldNews.

Given the structure of the early Usenet community, I would be
*seriously* surprised to hear that anyone at all had archived the
period before Toronto started. This is because 1) I'd have thought
Henry Spencer would've obtained copies of such archives to add to
his, or at least would've told Bruce Jones about such; it's a given
that he'd have *known* about them - and 2) it seems unlikely that
Spencer would have undertaken doing something unless nobody else
was known to be doing it. I suppose it remains possible that at one
of the major corporations that was then doing Usenet, e.g. what is
now known as Lucent, they *might* still have actual backup tapes from
the relevant computers, though I'd have thought given the way early
Usenet was paid for (usually by hiding the costs!), the news-admins
would consistently have found ways to take the news spools off of
the backup schedule. (I eventually had to do this while archiving
a single mailing list at a work computer.)

> My concern is that, the longer this material goes uncollected,
> it will be harder and harder to reclaim it in any usable form.
> In addition to it's value to future digital archeologists, there's
> the collected thoughts of thousands, if not tens of thousands,
> of individuals which are encompassed in these posts.

> It just seems a terrible waste for them to be lost. Yes, many
> of them weren't particularly memorable, but there's a lot there
> that was. It's also a history of the development of the Internet,
> through the online discussions of USENET posters.

It's worth remembering that what's lost is what is now seen today as
the golden age of Usenet. This is the time in which the vast majority
of FAQs were written, which tells you something about the volunteer
spirit of that day; it's the time of the Great Renaming and also the
time in which alt.sex.* worked; it's the majority of the time, and
the vast majority of the posts, before the long September and before
spam. This is the time when, to pick just one thing that happened
before I was completely offline and ignoring the topic, Usenet enabled
people to organise a mass campaign against the introduction of "New
Coke" in the then-stunning speed of a few days. (I do not know of
a prior equivalent action done by computer communications, though I
wouldn't be shocked if there had *been* one; at any rate, for Usenet,
it was long a claim to fame.)

I've been reading a lot of Greek and Roman history and art history lately.
Seems clear that in both places, the early, "classic", period, in which
the accomplishments happened that later Greeks and Romans took as
defining their peoples' greatness, is largely lost, because in both
places, there wasn't much care taken to protect the heritage of those
times from technological change and simple attrition, until it was
too late. The nearly complete loss of Usenet from 1986 to 1994 is an
analogous, if much smaller, loss.

Joe Bernstein

* Oops, snipped the original poster's question about other archives
than the OldNews one and the ones listed by Cameron Laird. Amazingly,
it looks like I get to be the first person in a thread *on news.groups
about archives* to mention the impressive archive David Lawrence has
maintained of news.announce.newgroups since something like December 1992.
This provides lists of new newsgroups from April 1991, and from January
1992 on. Cameron Laird's list does not appear to include it. It also
includes other resources for tracing the history of groups, noted below.

You can also find a copy of the List of Active Newsgroups posted by Gene
Spafford January 22, 1991 at
<http://sasun4.epfl.ch/News/Document/List_of_Active_Newsgroups.html>.
I have no idea why this Swiss site keeps it there, but they do. (From
a cursory exploration of the site, I can find no live pages that link
to it.) Using this and the lists of new and bogus groups and current
doings, and actual newsgroup-creation postings, archived by David Lawrence
at <ftp://ftp.isc.org/pub/usenet/news.announce.newgroups/>,
it is possible to figure out reasonably easily whether a given newsgroup
older than Dejanews is from the archive period, or existed in January
1991, or was created in the narrow interval in between. In addition,
note that Lawrence has archived UUnet's feed of newgroup and rmgroup
messages since around 1992 or earlier, and such messages often but
not always include information about the group's creation. These are
at <ftp://ftp.isc.org/pub/usenet/control/>, and are generally compressed.

--
Joe Bernstein, writer
jos...@tezcat.com, j...@sfbooks.com, and other recently unreliable addresses

Ian Stirling

unread,

Sep 4, 1999, 3:00:00 AM9/4/99

to

In alt.folklore.computers Drew Thomas <dr...@michthom.demon.co.uk> wrote:
>So, someone must have those on archive, by the look of it, you could
>store it all on a new PC.

Probably.
0-90 or so, isn't a big problem.
90-95 would take quite a bit of funding, and it's not so certain that
a full archive would be nearly as easy to assemble.

I think the best source for 90-95 would be an intelligence agency that's
stored it all.
Getting them to release it may be hard though.

Rodger Whitlock

unread,

Sep 4, 1999, 3:00:00 AM9/4/99

to

On 3 Sep 1999 20:46:47 -0500, jos...@tezcat.com (Joe Bernstein)
wrote:

>...since I've seen two attempts to compete with DejaNews
>fail (Altavista and Reference.com)...

AltaVista still has a newsgroup archive and search engine (or, it did
within the last two weeks). However, it differs fundamentally from
Deja.com (and always has, AFAIK) in that it does not keep messages
indefinitely. I do not know what determines message retention at
AltaVista. Generally, the AltaVista archive seems to go back somewhere
between six months and a year. I certainly wouldn't say it has failed.
Have I missed something?

Quite a long while ago, Reference.com (according to their web page at
the time) suffered some kind of catastrophic hardware failure. After
initial optimistic "we are rebuilding" announcements, it seems to be
permanently dead.

--
Rodger Whitlock
Victoria, British Columbia, Canada

T. Gryn

unread,

Sep 4, 1999, 3:00:00 AM9/4/99

to

On 3 Sep 1999 20:46:47 -0500, jos...@tezcat.com (Joe Bernstein)
wrote:

[...]

>> Up until their recent reorganization, DejaNews appeared to
>> also be interested in recovering the USENET posts made
>> prior to DN's inception (mid-'95). Unfortunately, based on their
>> current online FAQs, it appears that this effort has been
>> discontinued.
>Is this positively stated, or is it just that they don't ask?
>They may very well figure that if nobody's written by now in response
>to that ad, nobody is going to. I think it's highly unlikely that
>people sorting out their elderly relatives' estates are going to come
>across Usenet archives on diskettes or whatever, and think immediately
>"Oh! I should give this to DejaNews!" It's perfectly possible that,
>somewhere, there's someone who, like me, was offline for an extended
>period, and will find DejaNews on coming back online (as I did, very
>quickly); but I have a hard time imagining that someone who had been
>archiving a significant portion of Usenet would fit that scenario.
>So I suspect that DejaNews probably has not seen any useful replies to
>that ad in a couple of years, and in preference to continuing to have
>to thank people every couple of weeks for pointers to Bruce Jones's
>OldNews, they took the ad down.

OK, to clairify: in the pre-Deja.com period (before the takeover), one
of the questions in their FAQ was "Do you plan to expand your
database to include older years?" and the answer was (paraphrased)
"Yes! We are continually amassing archives from previous years, and
hope to include these in our archive in the future." In their recent
revisions of their online FAQs, this question has been removed, hence
my assumption that this has moved to the back burner at Deja, if
not abandoned.

I've tried multiple times to get confirmation of the status of this
from DejaNews, but have never recieved a reply.

[...]

>I've been reading a lot of Greek and Roman history and art history lately.
>Seems clear that in both places, the early, "classic", period, in which
>the accomplishments happened that later Greeks and Romans took as
>defining their peoples' greatness, is largely lost, because in both
>places, there wasn't much care taken to protect the heritage of those
>times from technological change and simple attrition, until it was
>too late. The nearly complete loss of Usenet from 1986 to 1994 is an
>analogous, if much smaller, loss.

The analogy which the Internet Archive site invokes is the burning of
the library at Alexandria, not specifically to this issue, but more in
terms of loss of digital information on the Internet as sites come and
go. I don't know if I'd go that far, but the analogy is a powerful
one.

Thanks for the information!

Tom Gryn..........................gryn.1@osu.edu

Denis McKeon

unread,

Sep 4, 1999, 3:00:00 AM9/4/99

to

In news.groups, in <37d14c24....@news.newsguy.com> Rodger Whitlock wrote:
>On 3 Sep 1999 20:46:47 -0500, jos...@tezcat.com (Joe Bernstein)
>wrote:
>

>>...since I've seen two attempts to compete with DejaNews
>>fail (Altavista and Reference.com)...
>
>AltaVista still has a newsgroup archive and search engine (or, it did
>within the last two weeks). However, it differs fundamentally from
>Deja.com (and always has, AFAIK) in that it does not keep messages
>indefinitely. I do not know what determines message retention at
>AltaVista. Generally, the AltaVista archive seems to go back somewhere
>between six months and a year. I certainly wouldn't say it has failed.
>Have I missed something?
>
>Quite a long while ago, Reference.com (according to their web page at
>the time) suffered some kind of catastrophic hardware failure. After

>initial optimistic "we are rebuilding" announcements, it seems to be
>permanently dead.

It might be worth reasearching this:

http://www.archive.org/
Internet Archive: Building a Digital Library for the Future

--
Denis McKeon

Joe Bernstein

unread,

Sep 4, 1999, 3:00:00 AM9/4/99

to

In article <7qs304$o2s$1...@sloth.swcp.com>,
Denis McKeon <Dmc...@swcp.com> wrote:

> In news.groups, in <37d14c24....@news.newsguy.com> Rodger
> Whitlock wrote:

> >On 3 Sep 1999 20:46:47 -0500, jos...@tezcat.com (Joe Bernstein)
> >wrote:

> >>...since I've seen two attempts to compete with DejaNews
> >>fail (Altavista and Reference.com)...

> >AltaVista still has a newsgroup archive and search engine (or, it did
> >within the last two weeks). However, it differs fundamentally from
> >Deja.com (and always has, AFAIK) in that it does not keep messages
> >indefinitely. I do not know what determines message retention at
> >AltaVista. Generally, the AltaVista archive seems to go back somewhere
> >between six months and a year. I certainly wouldn't say it has failed.
> >Have I missed something?

Yes. You've missed the early days of AltaVista's Usenet archive, in
which they claimed that they wanted to store Usenet indefinitely.
This claim changed, not much later than three months after they started,
to wanting to store Usenet for three months or so.

Now, this is arguably not "failure" but "sensible retrenchment of
overly ambitious plans", but as far as *our* goal of simply archiving
Usenet is concerned, it does constitute failure.

> >Quite a long while ago, Reference.com (according to their web page at
> >the time) suffered some kind of catastrophic hardware failure. After

> >initial optimistic "we are rebuilding" announcements, it seems to be
> >permanently dead.

Thanks for this info, which I missed as you missed AltaVista's start
apparently. I was greatly impressed by Reference.com in late 1997,
when it was invaluable for researching the status of apparently-dead
groups because it preserved full headers (which were then unavailable
from DejaNews; I'm not sure whether that's still the case, let alone
whether if it isn't, DejaNews has also restored headers stripped in
earlier days).

But when I visited it again recently, I was astonished at the state of
their newsgroup list, and belatedly realised that it had entered stasis
sometime in *1996*. Oops. Early 1997 at latest - no later than March
or so. Very embarrassing. At that time they had nothing that I saw
about a hardware failure.

I wrote them and got an auto-reply but nothing else.

> It might be worth reasearching this:
>
> http://www.archive.org/
> Internet Archive: Building a Digital Library for the Future

Actually, I looked at that site while writing my previous post. It is
not really a very helpful site. It gives no hint of what they
currently offer, let alone how you can go about getting access to it,
other than by writing to them. (They do provide full contact info -
postal and phone not just e-mail - as well as biographies of their
leaders to enable you to structure the most flattering possible way
to approach them. ;-)

They link to a site, http://www.alexa.com/, which is significantly
more promising. Apparently what they do is basically store Alexa's
database, and Alexa is the commercial arm of storing the Web. You
download Alexa's software for free and it installs a toolbar on your
screen as you web-surf, which allegedly offers a variety of useful things;
the most useful of these for the present purpose is enabling you to look
up broken links such as 404s, *presumably* (but this is not stated)
by searching Alexa's database. Since the archive.org site includes
a paper which talks about how expensive it is both to maintain and
to search a complete archive of the Web, I can't help thinking there's
got to be *something* Alexa makes money off of in this process, but
they don't tell you on their website what that something is.

For people archiving the Web, and thus presumably unusually familiar
with what it can offer, these two outfits do not present all that
impressive a pair of websites themselves. Grumble.

I know of no archive comparable to DejaNews which was begun before
DejaNews; AltaVista and Reference.com are both later. So at best,
what archive.org/alexa.com could offer is access to the lost archives
of individual newsgroups listed by Cameron Laird. (I have no idea
whatever how you could find archives not listed by Cameron Laird,
an example of which I posted earlier in this thread. Nor have I
any idea how many other such archives there might be, although that
list does obviously represent a lot of hard work.)

Joe Bernstein

--
Joe Bernstein, writer
j...@sfbooks.com, jos...@tezcat.com, and other recently unreliable addresses

Ian Stirling

unread,

Sep 5, 1999, 3:00:00 AM9/5/99

to

Joe Bernstein <jos...@tezcat.com> wrote:
<snip>

>Since I still think that DejaNews *will* die the first time someone with
>enough money sues them over copyright, I actively fear that day. Even
>with their new interface (which among other things seems inferior for
>tracing threads to the most recent version of their old), they remain
>invaluable. I'm delighted to hear that RemarQ (is *that* how you
>capitalise it?) is archiving too, particularly given that Jeremy is
>there, but since I've seen two attempts to compete with DejaNews
>fail (Altavista and Reference.com), I still think the day of that
>lawsuit against DejaNews will be a bad, bad day.

If dejanews also offered nntp access, then they could argue, that they
are simply a newsserver, albeit one with rather unusual expiry policies.

Going by the RFC's that define news, they don't actually specify nntp as the
only transport.

For an example of the sillyness, look at http://www.demon.net/ find
the court case info bit.

Basically someone apparently forged a post from Dr Lawrence Godfrey,
to soc.culture.thai, claiming that he'd had sex with little thai girls.

Instead of posting a denial, just ignoring it, or issuing cancels,
he took the bizzare (imo) step of faxing several news admins in the UK,
demanding they remove the article from their servers.

Some complied, demon diddn't, and so, he started legal proceedings against
them for defamation.

Rodger Whitlock

unread,

Sep 5, 1999, 3:00:00 AM9/5/99

to

On 4 Sep 1999 18:03:41 -0500, j...@sfbooks.com (Joe Bernstein) wrote:

>I know of no archive comparable to DejaNews which was begun before
>DejaNews; AltaVista and Reference.com are both later.

One more thing that might be news to you: about the time I got
involved in newsgroups (Feb-Mar 1995), I saw somewhere, perhaps in a
trade paper, mention of somebody (don't you love these detailed
recollections?) who was publishing all of usenet on CD-rom. I recall
that it was a CD per week.

Now for the life of me I cannot tell you more. I cannot tell you if
this was a report of something already up and running, or something
proposed. I cannot tell you if it was Murrican in focus or Canajun.
(At this rate we'lll be lucky if I can find the send button to click!)

But perhaps there is a lead there somewhere...

Postscript: as I'm in Canada and I only see Canajun computer trade
papers, I'd guess the focus was Canajun, almost certainly some outfit
in Ontario.

HTH

Tim Shoppa

unread,

Sep 5, 1999, 3:00:00 AM9/5/99

to

Rodger Whitlock wrote:
> One more thing that might be news to you: about the time I got
> involved in newsgroups (Feb-Mar 1995), I saw somewhere, perhaps in a
> trade paper, mention of somebody (don't you love these detailed
> recollections?) who was publishing all of usenet on CD-rom. I recall
> that it was a CD per week.

I recall the CD being discussed as well. In particular, the controversy
was that *someone* was going to be making money by selling everyone's
valuable Usenet posts.

Tim. (sho...@trailing-edge.com)

Jorn Barger

unread,

Sep 5, 1999, 3:00:00 AM9/5/99

to

Rodger Whitlock <toto...@mail.pacificcoast.net> wrote:
> One more thing that might be news to you: about the time I got
> involved in newsgroups (Feb-Mar 1995), I saw somewhere, perhaps in a
> trade paper, mention of somebody (don't you love these detailed
> recollections?) who was publishing all of usenet on CD-rom. I recall
> that it was a CD per week.

They didn't last long, I think for copyright reasons:

===
From: <in...@CDPublishing.com>
Date: Sat, 8 Jan 94 22:41:33 PST

NetNews on CD's
---------------
A USENET News on CD-ROM product from CD Publishing Corporation

"NetNews on CD's" is the USENET News on CD-ROM's product from
CD Publishing Corporation. This product provides an archiving
and distribution service for USENET news, along with efficient
software search tools for quick and easy access to this huge
base of valuable information.

The "NetNews on CD's" product line consists of several
CD-ROM subscription offerings (check with CD Publishing for
commencement dates) designed to meet your needs in the most
appropriate manner. Since the inception of the service,
USENET volume has been increasing steadily, resulting in
the need to constantly tune the breakdown of subscription
offerings. Over time, with volume considerations, and
suggestions from subscribers, we hope to continue to provide
you with an useful and usable product.

"NetNews on CD's" is an archival and distribution service,
making the wealth of information on USENET available whenever
you need it. "NetNews on CD's" alleviates the requirement
for excessive disk storage, tape archival/retrieval facilities,
limitations of communication link speeds, and prohibitively
high news feed/telephone charges. It provides a permanent
archive for access any time.

Each issue of "NetNews on CD's" contains approximately 600
megabytes of news. All issues are produced as soon as the
sufficient amount of news is collected, and shipped within
14 days of the last day covered by the disc. The precise
cut-off dates for each issue may vary with news volume, thereby
altering the issue production time cycle. CD Publishing
reserves the right to continue to further divide and define
the various series, depending on news volume constraints and
user requirements.

There are no restrictions on the subscribers' use of the news
articles on the CD-ROM, except those that were placed by the
original authors of the news articles. However, some of the
proprietary software packages and databases placed on the disc
to increase the utility and usefulness of "NetNews on CD's"
are subject to commercial software licensing terms.

"NetNews on CD's" is currently being published in the international
standard ISO 9660 format, and support for the industry standard
Rock Ridge extensions is planned as required. Indexing and
retrieval resources are being enhanced on an ongoing basis.

News reading software for the DOS/Windows, UNIX and Macintosh
platforms is currently included on the disc. Other platforms
which support the ISO 9660 CD-ROM format (such as DOS with
MSCDEX) are also capable of retrieving the data contained on
the disc.

The current "NetNews on CD's" product series are described below:

-- Technical Series

The Technical Series contains the "comp" and "gnu" hierarchies,
as well as related computer-technical groups from such as the
"alt", "bit" and "news" newsgroup hierarchies. The production
frequency (based on USENET volumes as at late 1993) is once every
four weeks. As space permits, appropriate computer-technical
groups from various regional and international heirarchies are
used to fill out issues.

-- Educational Series

The Educational Series contains a selection of various groups
which are deemed relevant to the educational community, and for
use in an educational/library resource setting. The production
frequency (based on USENET volumes as at late 1993) is once
every four to six weeks.

-- Specialized/Regional Series

The Specialized/Regional Series contains the specialized newsgroup
hierarchies and the various regional newsgroup hierarchies. The
production frequency (based on USENET volumes as at late 1993) is
once every four weeks.

The Specialized newsgroup hierarchies included in this series are:

bionet, bit, biz, ddn, hiv, ieee, k12, pubnet, sura, u3b,
uunet, vmsnet, among others

The Regional newsgroup hierarchies included in this series
encompass the various North American and International regional
newsgroups, including:

can, de, fj, maus, relcom, sfnet, uk, zer, among others

-- General Series

The General Series has contents from the non-computer-technical
mainstream newsgroup hierarchies. The production frequency
(based on USENET volumes as at late 1993) is once every two weeks.

The major newsgroup hierarchies drawn from for this series are:

alt, misc, news, rec, sci, soc, talk

Note:

- The Educational, Specialized/Regional and General Series
will begin production when there is sufficient support for
the service. In the case of archive offerings, production
re-occurs at each point when sufficient committed orders
have been received.

- The following sub-heirarchies are among those not supported
due to legal reasons: alt.binaries, alt.sex, alt.toon-pics,
alt.tasteless, de.alt.binaries, fj.binaries, rec.arts.erotica

Product Pricing
---------------

"NetNews on CD's" subscription rates are based on the quantity
of CD's ordered. The CD's for each Series may be ordered in
single quantities or as multiple-issue subscription sets. A
"set" is considered to be 6, 12 or 24 consecutive issues of a
Series. Depending on the selected subscription start date,
the term of the subscription will depend on the frequency of
publication of the selected Series.

Subscription can begin with any past, present or future issue,
subject to stock availability. Back issues dating as far back
as January 1992 are available.

Subscription Rates

Number of Issues 1 6 12 24
Unit Price US$ 34 US$ 28 US$ 24 US$ 20
Sub-Total US$ 34 US$ 168 US$ 288 US$ 480

Shipping and Handling

Number of Issues 1 6 12 24
U.S. / Canada US$ 5 US$ 30 US$ 60 US$ 120
International US$ 7 US$ 42 US$ 84 US$ 168

All prices are quoted in U.S. currency and are subject to change
without notice. As subscriber volume increases and new economies
of scale are achieved, any lowering of rates will reflect in the
appropriate lengthening of a subscriber's remaining subscription
terms.

Quantity discounts are available, please contact us for further
details.

More Information
----------------

For orders, inquiries, and technical support, please contact:

NetNews on CD's
CD Publishing Corporation
4824 Fraser Street
Vancouver, British Columbia
Canada V5V 4H4

604-874-1430
800-333-7565
604-874-1431 (FAX)

Information on CD Publishing Corporation products and services
can also be obtained via:

Gopher: gopher.CDPublishing.com
WWW: www.CDPublishing.com
FTP: ftp.CDPublishing.com
E-mail: <in...@CDPublishing.com>

------------------------------------------------------------------------
--

NetNews on CD's Order Form
==========================

Ordering Information
--------------------

Subscription and/or selected issue orders can be selected below.
Enter quantities at left, specify desired starting dates, or
selected issue dates (leave blank for current issue), and fill in
totals at right. If there is insufficient space (e.g. if ordering
multiple single issues or subscriptions), please copy this form,
attach a separate sheet, or add directly to this form.

Subscription: may begin on any year/month/issue (subject to
availability)

1 issue @ US$ 34/issue = US$ 34
6 issues set @ US$ 28/issue = US$ 168
12 issues set @ US$ 24/issue = US$ 288
24 issues set @ US$ 20/issue = US$ 480

Issues Series Start (year/month/issue#)
______ Technical _________________________ =
_________
______ Educational _________________________ =
_________
______ Specialized/Regional _________________________ =
_________
______ General _________________________ =
_________

Archives: Volume 1 - for period 92/01/01 to 93/03/31 (43 issues)

Qty. Each Qty. Each Qty. Each
1-4 US$ 34 10-19 US$ 24 40+ please call
5-9 US$ 28 20-39 US$ 20

______ issue(s) starting ___________________ @ US$ _____/issue =
_________

Archives: Volume 2 - for period 93/04/01 to 93/12/31

___ Technical 10 issues x US$ 24/issue = US$ 240 =
_________
___ Educational 10 issues x US$ 24/issue = US$ 240 =
_________
___ Specialized/Regional 10 issues x US$ 24/issue = US$ 240 =
_________
___ General 20 issues x US$ 20/issue = US$ 400 =
_________

Shipping and Handling:

USA/Canada shipping & handling @ US$ 5 x _____ issue(s) =
_________
International shipping & handling @ US$ 7 x _____ issue(s) =
_________

Sub-total =
_________
Taxes:

Canadian residents add 7% GST =
_________
British Columiba residents add 7% PST =
_________

-----------
TOTAL Cost TOTAL (US$) =

=========

Payment Information
-------------------

Payment is accepted in US funds via VISA, MasterCard, American Express,
check/cheque, money order and bank draft/transfer. Please make
check/cheque or money order payable to "CD Publishing Corporation".

Payment Type: [ ] VISA [ ] MasterCard [ ] American Express
[ ] Check/Cheque [ ] Money Order [ ] Bank
Draft/Transfer

Credit Card Information

Card No.: _______________________________ Expiry Date (MM/YY):
_____/_____

Cardholder's Name:
_______________________________________________________

Cardholder's Signature:
__________________________________________________

Address Information
-------------------

Billing Address

Name:
____________________________________________________________________

Title:
___________________________________________________________________

Department:
______________________________________________________________

Organization:
____________________________________________________________

Address:
_________________________________________________________________

_________________________________________________________________

City:
____________________________________________________________________

State/Province:
_________________________________________________________

Country:
_________________________________________________________________

Postal/ZIP Code:
_________________________________________________________

Phone: _______________________________ FAX:
______________________________

Internet E-Mail:
_________________________________________________________

Shipping Address (if different from above)