[hypermail] Incremental updates (was: Problems with 2.1.8)
Bill Moseley
> mos...@hank.org (Bill Moseley) writes:
> >So, this creates an initial archive
> >
> > $ rm -rf archive
> > $ cat lots-of-message | hypermail -i
> >
> >So at this point what's the difference between these (not lack of -u):
> >
> > $ cat new_message | hypermail -i
> > $ cat new_message | hypermail -i -x
>
> If you are doing an incremental update (i.e. have an .hmrc with increment = 1
> or increment = -1 and new_message has no old messages), the -x option makes no
> difference (I guess I ought to add that to the documentation).
And increment = 1 is the same as -u on the command line, correct?
> If you are not doing an incremental update (i.e. have an .hmrc with increment
> = 0 and new_message contains lots-of-message plus something new), then
> if you changed your config file from showhtml = 1 to showhtml = 2 between
> the lots-of-message command and the new_message command, the -x will cause
> the entire archive to have the showhtml = 2 format, and avoiding -x will
> produce and archive with mixed format.
In other words, if you feed hypermail a bunch of messages it normally
doesn't process messages it already has seen and just works on new
messages in the input stream. The -x option says to treat every message
as new -- rewriting existing \d+.html files.
How does hypermail keep track of messages it has already seen?
Why would you send hypermail messages it's already processed? To add to
the archive I would think that you would just pipe the new message to
hypermail as each one comes in.
Another question. Say I've got an existing archive and some old message
needs to be removed or modified. What is the procedure?
--
Bill Moseley
mos...@hank.org
Bill Moseley
I found that I could not feed documents to hypermail out of date order
-- at least in my 2.1.8 tests I had to sort the messages before feeding
them to hypermail. Is this what's expected?
I have a situation where an archive was being incrementally added to,
but for for a while messages were not being delivered correctly -- so I
have a gap in the archive.
I now have those messages. Is there a way to add those in or do I need
to recreate the entire archive?
--
Bill Moseley
mos...@hank.org
Peter C. McCluskey
>Oh, I left one question out.
>
>I found that I could not feed documents to hypermail out of date order
>-- at least in my 2.1.8 tests I had to sort the messages before feeding
>them to hypermail. Is this what's expected?
My recollection is that feeding them out of order should produce an
archive that looks mostly right but that the Next and Previous message
links will reflect the order in which hypermail got them. Maybe there
are some other problems I don't recall, but those are the only ones
that seem hard to avoid.
>I have a situation where an archive was being incrementally added to,
>but for for a while messages were not being delivered correctly -- so I
>have a gap in the archive.
>
>I now have those messages. Is there a way to add those in or do I need
>to recreate the entire archive?
If you're willing to accept the minor problems mentioned above, just
feed them in with -u or increment = 1. Otherwise recreate the archive
from scratch.
--
------------------------------------------------------------------------------
Peter McCluskey | "To announce that there must be no criticism of
http://www.rahul.net/pcm | the President, or that we are to stand by the
| President right or wrong, is not only unpatriotic
| and servile, but morally treasonable to the
| American public." - Theodore Roosevelt
Bill Moseley
> mos...@hank.org (Bill Moseley) writes:
> >Oh, I left one question out.
> >
> >I found that I could not feed documents to hypermail out of date order
> >-- at least in my 2.1.8 tests I had to sort the messages before feeding
> >them to hypermail. Is this what's expected?
>
> My recollection is that feeding them out of order should produce an
> archive that looks mostly right but that the Next and Previous message
> links will reflect the order in which hypermail got them. Maybe there
> are some other problems I don't recall, but those are the only ones
> that seem hard to avoid.
The big problem I had was with the folders list. The dates were quite
mixed up.
--
Bill Moseley
mos...@hank.org
Peter C.McCluskey
>And increment = 1 is the same as -u on the command line, correct?
Yes.
>In other words, if you feed hypermail a bunch of messages it normally
>doesn't process messages it already has seen and just works on new
>messages in the input stream. The -x option says to treat every message
>as new -- rewriting existing \d+.html files.
Sort of, but it does most of the processing the same in either case,
and it's only when it goes to write the message that -x controls whether
to do the write or skip it.
>How does hypermail keep track of messages it has already seen?
It either uses the gdbm file or loks for existing html files in the archive.
>Why would you send hypermail messages it's already processed? To add to
>the archive I would think that you would just pipe the new message to
>hypermail as each one comes in.
Updating the archive once a day can be a lot less cpu intensive than
updating it for every message.
Seperating the hypermail process from the mail delivery process may also
improve error handling.
>Another question. Say I've got an existing archive and some old message
>needs to be removed or modified. What is the procedure?
See the delete_msgnum option.