email list archive and non-google-able messages

[Neil Van Dyke]

I think some past Racket email list discussions currently are no longer
findable via Google. Anyone know anything more about this?

Details...

Yesterday, I was trying to use Google to find particular messages from a
couple years ago, and was getting 0 hits. I was only able to find the
messages archives on pages that *are* being served by
"lists.racket-lang.org" *after* I found other references to the dates of
the messages, and navigated through "lists.racket-lang.org" manually. I
confirmed that the Google keyword searches should've matched the message
pages, and that the message pages were available from Google text cache.

Just now, I quickly confirmed that the current Google coverage of email
list archives on "lists.racket-lang.org" is spotty (see below), from
current date back through January 2014. I did not check other dates. I
don't know whether that has anything to do with the slightly odd
"robots.txt" (see below), which the validator I just tried claimed would
permit all crawling, but I'm not sure.

Google Text Cache *Does* Exist
==============================
http://lists.racket-lang.org/users/archive/
http://lists.racket-lang.org/users/archive/2015-May/thread.html
http://lists.racket-lang.org/users/archive/2015-March/thread.html
http://lists.racket-lang.org/users/archive/2015-February/thread.html
http://lists.racket-lang.org/users/archive/2014-December/thread.html
http://lists.racket-lang.org/users/archive/2014-November/thread.html
http://lists.racket-lang.org/users/archive/2014-July/thread.html
http://lists.racket-lang.org/users/archive/2014-June/thread.html
http://lists.racket-lang.org/users/archive/2014-April/thread.html
http://lists.racket-lang.org/users/archive/2014-March/thread.html
http://lists.racket-lang.org/users/archive/2014-January/thread.html

Google Text Cache Does *Not* Exist
==================================
http://lists.racket-lang.org/users/archive/2015-January/thread.html
http://lists.racket-lang.org/users/archive/2014-October/thread.html
http://lists.racket-lang.org/users/archive/2014-September/thread.html
http://lists.racket-lang.org/users/archive/2014-August/thread.html
http://lists.racket-lang.org/users/archive/2014-May/thread.html
http://lists.racket-lang.org/users/archive/2014-February/thread.html

http://lists.racket-lang.org/robots.txt
=======================================
User-agent: *
Disallow:

Neil V.

[Neil Van Dyke]

Bad typo correction: the message pages that Google could not find were
*not* available from Google text cache.