An elisp package to browse the comp.os.linux archives
Raymond Chen
back-issues of the Linux-Activists list (also known as comp.os.linux)
that I figured they deserved to have a cool browser. So I wrote one.
I've found it quite useful. Helped my figure out how to get the arrow
keys to work in my emacs session without having to post the question
to the newsgroup. Saved me lots of embarrassment. :-)
The following patch kit contains two files, `lxbrowse.el' and `mklxindex'.
The latter is a perl script which builds the indexes; the former is
an elisp script that uses the index to browse the archives.
To install...
0. Install emacs and perl first. (Arguably the hard part. :-) )
If you didn't put perl into /bin/perl, change the first line of
mklxindex to match.
1. Unpack the patch in any convenient directory.
2. Type `mklxindex' with no arguments to see the instructions for
the perl half of the installation.
The instructions are different depending on whether you are the
administrator of an archive site or are a mere human being.
3. Read the comments at the top of lxbrowse.el to see what sort of
customization you may need to do on the emacs end. Pay particular
attention to the stuff marked YOU WILL PROBABLY NEED TO CHANGE
THIS and YOU MAY ALSO WANT TO CHANGE THIS.
4. Load lxbrowse.el, then type `M-x lxbrowse', tell it which volume
you want to browse, and have fun. The only command is `f' to
(f)ind a thread.
5. Don't get too attached to the format of the index files.
I'm changing them. See ``FUTURE PLANS''.
6. Send feedback to raym...@microsoft.com. Feedback of the form ``you
idiot, you didn't need to write this; look at package BLAH'' is
particularly helpful, since it saves me a lot of work!
Have fun.
--
Raymond (just another linux hacker) Chen
*** /dev/null Sat Aug 29 21:48:28 1992
--- lxbrowse.el Mon Jun 7 08:50:56 1993
***************
*** 0 ****
--- 1,198 ----
+ ;;; Linux mail archives browser for GNU Emacs
+ ;;; version -1.0. (That's *negative* one.)
+ ;;;
+ ;;; You are granted permission to do with this file whatever you wish.
+ ;;; If you can con somebody into paying for it, more power to you.
+ ;;;
+ ;;; Please send comments / complaints / bug reports to raym...@microsoft.com.
+ ;;;
+ ;;; Presents a title-summarized table of articles in the tsx-11 Linux
+ ;;; archives of the Linux-Activists list (bidirectionally linked to the
+ ;;; comp.os.linux newsgroup).
+ ;;;
+ ;;; Very handy if you have a linux problem that you know somebody else
+ ;;; has encountered somewhere before. Just search the database for
+ ;;; key words or phrases.
+ ;;;
+
+ ;;;
+ ;;; KNOWN BUGS
+ ;;;
+ ;;; Subjects consisting entirely of digits are oft screwed up.
+ ;;;
+ ;;; Error handling is phenomenally crude. (So don't make a mistake :-) )
+ ;;;
+
+ ;;;
+ ;;; FUTURE PLANS
+ ;;;
+ ;;; Change index file format to reduce size by 33%.
+ ;;; Change index file format to remove need for `parsing' pass on load.
+ ;;; Ability to view indexes from multiple volumes simultaneously.
+ ;;; Ability to merge volume indexes into a "master index".
+ ;;;
+
+ ;;;
+ ;;; The format of the index file is as a sequence of THREADs.
+ ;;;
+ ;;; The first line of a THREAD is the thread subject.
+ ;;; The second line is the number of articles in the thread (N).
+ ;;; The next N lines describe the location of an article in the thread.
+ ;;; The last line is a ^L on a line by itself.
+ ;;;
+ ;;; The line describing the location of an article in a thread is
+ ;;; of the form
+ ;;;
+ ;;; filename,begin end
+ ;;;
+ ;;; The filename is relative to lxbrowse-root.
+ ;;; The begin and end points define the region.
+ ;;;
+ ;;; THREADs in the index file are kept in chronological order.
+ ;;;
+ ;;; Thread files are created by the companion perl script mklxindex.
+
+ (provide 'lxbrowse)
+
+ ;;;
+ ;;; YOU WILL PROBABLY NEED TO CHANGE THIS VARIABLE.
+ ;;;
+ (defvar lxbrowse-root "/cd/linux/mail-archive/"
+ "The location at which the Linux-Activists archives can be found.
+ It is assumed that subdirectories of this directory are named
+ Volume1 through VolumeN, and that each such subdirectory has
+ a CONTENTS file.")
+
+ ;;;
+ ;;; YOU MAY ALSO WANT TO CHANGE THIS ONE.
+ ;;;
+ (defvar lxbrowse-should-split-window nil
+ "Set to non-NIL if lxbrowse-find-thread should show the new thread
+ in a new window, leaving the table of contents also visible. Use
+ \\[other-window] to switch back to the table of contents.
+
+ Set to NIL if lxbrowse-find-thread should switch to the thread window,
+ replacing the table of contents. Use \\[switch-to-buffer] to switch
+ back to the table of contents.")
+
+ (defvar lxbrowse-volume ""
+ "The name of the Volume currently being studied.")
+
+ (defvar lxbrowse-index-buffer nil
+ "The buffer holding the raw index under study.")
+
+ (defvar lxbrowse-toc "*lxbrowse-toc*"
+ "The name of the buffer used to display the table of contents.")
+
+ (defvar lxbrowse-thread "*lxbrowse-thread*"
+ "The name of the buffer used to display a thread.")
+
+ (defvar lxbrowse-uncompress "*Lxbrowse-uncompress*"
+ "The name of the buffer used to contain the uncompressed digest
+ during thread extraction.")
+
+ (defvar lxbrowse-uncompress-file ""
+ "The name of the digest currently in the lxbrowse-uncompress buffer.")
+
+ (defvar lxbrowse-mode-map nil)
+
+ (if lxbrowse-mode-map
+ nil
+ (setq lxbrowse-mode-map (make-keymap))
+ (suppress-keymap lxbrowse-mode-map)
+ (define-key lxbrowse-mode-map "f" 'lxbrowse-find-thread))
+
+ (defun lxbrowse-index-location ()
+ "Given an archive directory name, return the location of its index file.
+ If we can't find it on the archive, look in the user's home directory."
+ (let ((s (concat lxbrowse-root lxbrowse-volume ".lxindex")))
+ (if (file-exists-p s) s))
+ (concat (getenv "HOME") "/.lx" lxbrowse-volume))
+
+ (defun lxbrowse-erase-buffer (b)
+ "Wipe out the buffer, creating it if necessary, returning the empty buffer."
+ (save-excursion
+ (set-buffer (get-buffer-create b))
+ (kill-region (point-min) (point-max))
+ (current-buffer)))
+
+ (defun lxbrowse-append-toc (s)
+ "Append string S to lxbrowse-toc"
+ (save-excursion
+ (set-buffer (get-buffer-create lxbrowse-toc))
+ (goto-char (point-max)) (insert s)))
+
+ (defun lxbrowse-parse-index (src &optional s p i)
+ "Convert the index in SRC into a user-readable table of contents
+ in lxbrowse-toc."
+ (save-excursion
+ (message "Parsing index...")
+ (setq i 0)
+ (set-buffer src)
+ (goto-char (point-min))
+ (while (not (eobp))
+ (setq i (1+ i)) (if (zerop (% i 500)) (message "Parsing index...%d" i))
+ (setq p (point)) (forward-line) (setq s (buffer-substring p (point)))
+ (lxbrowse-append-toc (format "%2d %s" (read src) s))
+ (search-forward "\^L\n"))))
+
+ (defun lxbrowse-ensure-buffer (f)
+ "Load the file f into the lxbrowse-uncompress buffer. If the file is
+ already in lxbrowse-uncompress, then nothing is done."
+ (or (string-equal f lxbrowse-uncompress-file)
+ (save-excursion
+ (lxbrowse-erase-buffer lxbrowse-uncompress)
+ (message "Uncompressing %s..." f)
+ (call-process "uncompress" (concat lxbrowse-root f)
+ lxbrowse-uncompress nil)
+ (setq lxbrowse-uncompress-file f)
+ (message ""))))
+
+ (defun lxbrowse-build-thread (&optional p b e)
+ "Build the thread viewer buffer based on the THREAD structure at point."
+ (forward-line 2) ; Skip the subject line and count
+ (lxbrowse-erase-buffer lxbrowse-thread)
+ (while (not (looking-at "\^L"))
+ (setq p (point)) ; p = start of line
+ (search-forward ",") ; find the first separator
+ (lxbrowse-ensure-buffer (buffer-substring p (1- (point)))) ; filename
+ (setq b (read (current-buffer))) ; Read the starting point
+ (setq e (read (current-buffer))) ; Read the ending point
+ (save-excursion
+ (set-buffer (get-buffer lxbrowse-uncompress))
+ (append-to-buffer (get-buffer lxbrowse-thread) b e))))
+
+ (defun lxbrowse-find-thread (&optional s) (interactive)
+ "Build the thread viewer buffer based on the subject line at point."
+ (save-excursion
+ (beginning-of-line)
+ (forward-char 3) ; Skip article count
+ (setq s (buffer-substring (point) (progn (end-of-line) (point))))
+ (set-buffer (get-buffer lxbrowse-index-buffer))
+ (goto-char (point-min))
+ (re-search-forward (concat "^" (regexp-quote s) "$"))
+ (lxbrowse-build-thread))
+ (if lxbrowse-should-split-window
+ (switch-to-buffer-other-window lxbrowse-thread)
+ (switch-to-buffer lxbrowse-thread))
+ (goto-char (point-min)))
+
+ (defun lxbrowse (n &optional s) (interactive "NVolume to browse: ")
+ (setq s (format "Volume%d" n))
+ (if (not (string-equal lxbrowse-volume s))
+ (progn
+ (setq lxbrowse-volume s)
+ (setq lxbrowse-index-buffer
+ (find-file-noselect (lxbrowse-index-location)))
+ (lxbrowse-erase-buffer lxbrowse-toc)
+ (switch-to-buffer lxbrowse-toc)
+ (lxbrowse-parse-index lxbrowse-index-buffer)))
+ (switch-to-buffer lxbrowse-toc)
+ (if (file-newer-than-file-p (concat lxbrowse-root lxbrowse-volume)
+ (lxbrowse-index-location))
+ (progn
+ (message "Warning: Index file appears to be out of date")
+ (sit-for 1)))
+ (use-local-map lxbrowse-mode-map)
+ (message "Move to the subject you want to view and press `f'"))
+
*** /dev/null Sat Aug 29 21:48:28 1992
--- mklxindex Sun Jun 6 22:23:20 1993
***************
*** 0 ****
--- 1,72 ----
+ #!/bin/perl
+
+ sub line {
+ local($_); $_ = <I>; $point += length($_);
+ $_;
+ }
+
+ $vol = shift;
+
+ die <<EUSAGE unless -d $vol;
+ $0 -- Builds the index tables for the lxbrowse.el elisp package.
+
+ The following directory structure is expected:
+
+ ./Volume1/digest*.Z -- 18 Jan 92 - 31 Mar 92
+ ./Volume2/digest*.Z -- 31 Mar 92 - 17 Aug 92
+ ./Volume3/digest*.Z -- 17 Aug 92 - 11 Jan 93
+ ./Volume4/digest*.Z -- 11 Jan 93 - present
+
+ The current directory should be the one containing the Volume subdirectories.
+
+ If you have write permission into the Volume? directories, type
+
+ $0 VolumeN >VolumeN/.lxindex
+
+ where N varies from 1 to 4.
+
+ If you do not have write permission in the Volume? directories (e.g., they
+ are mounted via CD-ROM), type
+
+ $0 VolumeN >~/.lxVolumeN
+
+ again where N varies from 1 to 4.
+ EUSAGE
+
+ print STDERR "Inspecting digests $vol/digest*.Z";
+
+ for ($i = 1; -r "$vol/digest$i.Z"; $i++) {
+ open(I, "zcat $vol/digest$i.Z|");
+ print STDERR ".";
+ print STDERR $i if $i % 10 == 0;
+ $point = 0; $subject = '';
+
+ # Skip until we hit the first article.
+ 0 while ($_ = &line) && $_ ne ("-" x 76) . "\n";
+
+ ($_ = &line) eq "\n" || (warn "Damaged digest $i", next);
+ while ($begin = $point, ($_ = &line) ne "\n") {
+ $subject = '';
+ $subject = $2 if /^Subject:\s+(Re:\s*)?(.*)/i;
+ while ($_ = &line) {
+ last if $_ eq ("-" x 30) . "\n";
+ $subject = $subject || $2 if /^Subject:\s+(Re:\s*)?(.*)/i;
+ }
+ $subject = $subject || "(none)";
+ push(@seen, $subject) unless $arts{$subject};
+ $arts{$subject} .= "$vol/digest$i.Z,$begin $point\n";
+ ($_ = &line) eq "\n" || (warn "Damaged digest $i", next);
+ }
+ warn "Damaged digest $i" unless &line eq "** FOR YOUR REFERENCE **\n";
+ }
+
+ print STDERR "dumping index...";
+
+ for (@seen) {
+ print $_, "\n";
+ print $arts{$_} =~ y/\n/\n/, "\n";
+ print $arts{$_};
+ print "\014\n";
+ }
+
+ print STDERR "done\n";
# end of patch