MSPOOL = 3,368,146 KB -- Am I doomed Mr Mainwaring?

[Ian Wade G3NRW]

I have just completed my weekly Turnpike backup, and noticed that the
MSPOOL size is now 3,368,146 KB. Back in the dim recesses of my mind I
seem to remember there is a 4GB limit on this file (and on NSPOOL, but
that one is only 1GB at the moment), and I have a horrible feeling the
world is about to end.

My mail messages go back 15 years or more, and I want to keep them.
Ideally, I would like to hive off most of them to an off-line archive,
leaving just the last 12 months or so on line, but I don't know how to
do this.

As I understand it, I can export the messages to text files,
compress/archive the old ones, delete MSPOOL, then import the latest
back into MSPOOL. But this will be a *very* long job (which maybe I
should have thought about 15 years ago).

Are there any shortcuts?

--
Ian

[Martin Liddle]

On 11/11/2012 08:16, Ian Wade G3NRW wrote:
>
> As I understand it, I can export the messages to text files,
> compress/archive the old ones, delete MSPOOL, then import the latest
> back into MSPOOL. But this will be a *very* long job (which maybe I
> should have thought about 15 years ago).
>

You are making it more complicated than necessary. After
archiving/deleting the old messages you need to rebuild the database
(possibly twice) to shrink it.


--
Martin Liddle, Tynemouth Computer Services, Chesterfield, Derbyshire,UK
http://www.tynecomp.co.uk

[Andy]

In message <sIkXtwEQ...@ntlworld.com>, Ian Wade G3NRW
<g3nrw...@ntlworld.com> wrote

>I have just completed my weekly Turnpike backup, and noticed that the
>MSPOOL size is now 3,368,146 KB. Back in the dim recesses of my mind I
>seem to remember there is a 4GB limit on this file (and on NSPOOL, but
>that one is only 1GB at the moment), and I have a horrible feeling the
>world is about to end.

It will when you reach 4Gb

>
>My mail messages go back 15 years or more, and I want to keep them.
>Ideally, I would like to hive off most of them to an off-line archive,
>leaving just the last 12 months or so on line, but I don't know how to
>do this.
>
>As I understand it, I can export the messages to text files,
>compress/archive the old ones, delete MSPOOL, then import the latest
>back into MSPOOL. But this will be a *very* long job (which maybe I
>should have thought about 15 years ago).
>
>Are there any shortcuts?
>

It is most likely that you have a relatively small number of large
attachments rather than a monstrous number of short emails.

Print this out then do it...

Make a backup justincase

Follow the standard procedure for compressing/rebuilding the mailbase

If all is well, proceed...

Close news-view etc so you have the Turnpike Tree displayed. CHeck
Connect is not running.

Click once on "Turnpike" at the top of the tree.

From top menu choose Tools then Searchmessages

Check that "all email messages" is in bottom box, select it if not

Search Now

{wait}

a list appears.

if one column isn't 'size' click View Columns and add it

click the word 'size' at the top of its column to sort by size

if you prefer them the other way round click it once more

check right hand slidy bar to verify you are at the end of the list

starting with the biggest, open it and decide if you really really need
to keep the attachment. If not, export it to somewhere you'll be able to
find it in later (consider renaming it) then delete it. The
with-attachment email is moved to the wastebin and its place taken by an
attachment-free copy.

after say 25, go to the wastebin and empty it

after say 100, empty bin, close TP and rebuild the database to recover
the space.

Repeat ad nauseam.
--
Andy Taylor [Editor, Austrian Philatelic Society].
Visit <URL:http://www.austrianphilately.com>

[Jim Crowther]

In demon.ip.support.turnpike, on Sun, 11 Nov 2012 08:16:16, Ian Wade

G3NRW wrote:

>My mail messages go back 15 years or more, and I want to keep them.
>Ideally, I would like to hive off most of them to an off-line archive,
>leaving just the last 12 months or so on line, but I don't know how to
>do this.

Make a backup of your turnpike folder, naming this folder
'2012-11-11-turnpike' or somesuch.

This will be your archive, which you can access in the future by
changing the 'Paths' in the Connect log-in screen.

Now delete all messages you wish from your current working folder and
empty the wastebasket. Re-build the Mailbase.


--
Jim Crowther

[Tony]

Ian Wade G3NRW <g3nrw...@ntlworld.com> wrote on Sun, 11 Nov 2012 at
08:16:16:

>As I understand it, I can export the messages to text files,
>compress/archive the old ones, delete MSPOOL, then import the latest
>back into MSPOOL. But this will be a *very* long job (which maybe I
>should have thought about 15 years ago).
>
>Are there any shortcuts?

There's no need to archive by export/import. Instead, just copy the
Turnpike folder and subfolders to an archive folder using Windows
Explorer, with Turnpike and Connect closed. You don't need the bin
folder (hmm, you're still using 6.03, that might not have had a bin
folder; if so, you don't need to copy the program EXE or DLL files). It
would be wise to check access to the archive at this point, as below.

I suggest you empty the wastebasket and rebuild the databases to compact
them before archiving.

Start Turnpike again and delete all messages older than 12 months from
your working mailbase. Create a search for "all my email" dated
("Received") between a date before the earliest message on the system
(say 1/1/1996) and 31 December 2011, say. NB a Turnpike bug seems to
prevent setting a date earlier than 10 years ago, so you may have to use
the Advanced IMAP search instead (e.g.
UNDELETED SINCE "1-Jan-1996" BEFORE "31-Dec-2011"). Untick "Include
deleted messages in search" (speeds up delete process). That will
collect together all the mail that you want to delete (e.g. more than 1
year old), regardless of the folders it is in. Finally, empty the
wastebasket. Do this step for each user. Then rebuild the mailbase. Your
working mailbase will now have emails received after 31 Dec 2011 only.

To access your archive, start TP Explorer and at the login dialog box
change the paths to point to the archive copy. DO NOT run Connect. When
you have finished, repeat this step but change the paths to point to the
working copy, and then you can run Connect again.

When you subsequently need to archive again, create a different archive
folder. Including the date or even just year of archiving in the folder
name would help.

Regards,
--
Tony

[Andy]

In message <z3tE$6c+T3...@hotair.localhost.invalid>, Tony
<tonyh1...@hotair.demon.co.uk> wrote
[]

>
>Start Turnpike again and delete all messages older than 12 months from
>your working mailbase.

[snip details]

I find that if I have a large (eg 1000) number of things in my wastebin
then telling TP to empty it doesn't work. I have to go into the
wastebin, select a few screenfuls, and manually delete them.

[Ian Wade G3NRW]

___Original Message_________________________________________

From: Ian Wade G3NRW <g3nrw...@ntlworld.com>
Date: Sun, 11 Nov 2012 Time: 08:16:16

>I have a horrible feeling the world is about to end.
>

Thanks everyone for the very helpful messages. I hadn't realised that
you could reduce the size of the spool files by deleting messages then
rebuilding the databases twice. I had thought that the files just grew
and grew, and the only way to reduce their size was to delete them and
start again.

That gives me hope. I estimate I have about a month to go before
Armageddon.

--
Ian

[Tony]

Andy <an...@kitzbuhel.demon.co.uk> wrote on Sun, 11 Nov 2012 at 11:02:05:
>I find that if I have a large (eg 1000) number of things in my wastebin
>then telling TP to empty it doesn't work. I have to go into the
>wastebin, select a few screenfuls, and manually delete them.

I'll test whether that problem happens here next time I archive
(January). I do find that the empty wastebasket command is incredibly
slow if you have the wastebasket folder open at the time with several
screenfulls waiting to be deleted; I make sure another folder is open
instead and right-click wastebasket/Empty in the Folders pane. This is
with TP running in XP Mode under Win7/64-bit.
--
Tony

[J. P. Gilliver (John)]

In article <VlxJ25AG...@ntlworld.com>, on Sun, 11 Nov 2012, Ian
Wade G3NRW <g3nrw...@ntlworld.com> wrote

I'd add my encouragement to sorting by size and then exporting
attachments; IMO keeping (any, but certainly large) attachments _inside_
emails not only slows down TP (as well as making the 'base files big),
but is also slightly more risky for the attachments: you're relying on
TP _as well as_ the OS to get at them, whereas if you export them,
you're only relying on the OS. (This is assuming you _do_ consider the
attachment important; if not, delete it altogether!) You _don't_ have to
delete the email: you can export the attachment, then remove it from the
email. (TP will replace it in the email by a text line saying where you
exported it.)
--
J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

"I'd give my right arm to be ambidextrous"

I already am largely ambisinistral.

[J. P. Gilliver (John)]

In article <mnw9hUBt...@kitzbuhel.demon.co.uk>, on Sun, 11 Nov
2012, Andy <an...@kitzbuhel.demon.co.uk> wrote

>In message <z3tE$6c+T3...@hotair.localhost.invalid>, Tony
><tonyh1...@hotair.demon.co.uk> wrote
>[]
>>
>>Start Turnpike again and delete all messages older than 12 months from
>>your working mailbase.
>[snip details]
>
>I find that if I have a large (eg 1000) number of things in my wastebin
>then telling TP to empty it doesn't work. I have to go into the
>wastebin, select a few screenfuls, and manually delete them.

I also find there are two sorts of things that can be in the WB - one
that (I think) _is_ deleted when I quit Turnpike, another that isn't:

Some while ago, I asked here if there was any way I could stop TP from
keeping a copy of everything I email or post. (If I _want_ to keep a
copy, I'd Cc: myself, for emails anyway.) Someone suggested (the general
consensus was that there's no way to stop TP making copies as such)
changing the settings so that TP stores the copies in the WB rather than
the default "Filed". This worked inasmuch as the copies do indeed go
into the WB, but they're _not_ deleted when I close TP; if I look into
the WB, there is a distinction - one sort (I think it's the ones that
_are_ going to be deleted) are shown in grey, the other in black.

From time to time I have a WB-emptying session.

[Roy Brown]

In message <VlxJ25AG...@ntlworld.com>, Ian Wade G3NRW
<g3nrw...@ntlworld.com> writing at 11:46:14 in his/her local time
opines:-

?

I estimate 3 years and 8 months, based on the numbers you quote.

Years Current Size Growth per Year
15 3,368,146 224,543
Max Size (Max - Curr)/Growth per Year
4,194,303 3.679

Or is this too simplistic?

Also, you don't say if you do regularly rebuild your TP databases or
not.
If you simply do a rebuild, what does MSPOOL come down to?
--
Roy Brown 'Have nothing in your houses that you do not know to be
Kelmscott Ltd useful, or believe to be beautiful' William Morris

[J. P. Gilliver (John)]

In article <3H7FR8IX76nQFwFZ@x.x>, on Sun, 11 Nov 2012, Roy Brown
<Roy_now_fre...@acanthus.demon.co.uk> wrote

>In message <VlxJ25AG...@ntlworld.com>, Ian Wade G3NRW
><g3nrw...@ntlworld.com> writing at 11:46:14 in his/her local time
>opines:-

[]

>>That gives me hope. I estimate I have about a month to go before
>>Armageddon.
>
>?
>
>I estimate 3 years and 8 months, based on the numbers you quote.
>
>Years Current Size Growth per Year
>15 3,368,146 224,543
> Max Size (Max - Curr)/Growth per Year
> 4,194,303 3.679
>
>Or is this too simplistic?

It doesn't take account of the increase in the average size of
attachments (I'd guess 15 years ago most were on dialup), and the
reduction in snippage (linked to the increase in top-posting - most
top-posters don't snip, which is its real disadvantage, rather than the
top-posting itself).

>
>Also, you don't say if you do regularly rebuild your TP databases or
>not.
>If you simply do a rebuild, what does MSPOOL come down to?

(I was wondering that too. Might be worth doing twice too, especially if
the first time reports any problems.)

--
J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

"There are a great many people in the country today who, through no fault of
their own, are sane." - Monty Python's Flying Circus

[Mike Barnes]

Andy <an...@kitzbuhel.demon.co.uk>:

>I find that if I have a large (eg 1000) number of things in my wastebin
>then telling TP to empty it doesn't work. I have to go into the
>wastebin, select a few screenfuls, and manually delete them.

I wonder why that is.

You've just reminded me that it's a long time since I cleared old stuff
from the wastebasket, so I deleted everything over a couple of months
old. Just shy of 2000 items, deleted in one go, taking less than five
seconds.

--
Mike Barnes

[Ian]

In message <rAXatWv$N6nQ...@soft255.demon.co.uk>, "J. P. Gilliver
(John)" <ju...@nospam.demon.co.uk> writes

>In article <mnw9hUBt...@kitzbuhel.demon.co.uk>, on Sun, 11 Nov
>2012, Andy <an...@kitzbuhel.demon.co.uk> wrote
>>In message <z3tE$6c+T3...@hotair.localhost.invalid>, Tony
>><tonyh1...@hotair.demon.co.uk> wrote
>>[]
>>>
>>>Start Turnpike again and delete all messages older than 12 months from
>>>your working mailbase.
>>[snip details]
>>
>>I find that if I have a large (eg 1000) number of things in my
>>wastebin then telling TP to empty it doesn't work. I have to go into
>>the wastebin, select a few screenfuls, and manually delete them.
>
>I also find there are two sorts of things that can be in the WB - one
>that (I think) _is_ deleted when I quit Turnpike, another that isn't:
>
>Some while ago, I asked here if there was any way I could stop TP from
>keeping a copy of everything I email or post. (If I _want_ to keep a
>copy, I'd Cc: myself, for emails anyway.) Someone suggested (the
>general consensus was that there's no way to stop TP making copies as
>such) changing the settings so that TP stores the copies in the WB
>rather than the default "Filed". This worked inasmuch as the copies do
>indeed go into the WB, but they're _not_ deleted when I close TP; if I
>look into the WB, there is a distinction - one sort (I think it's the
>ones that _are_ going to be deleted) are shown in grey, the other in black.
>
>From time to time I have a WB-emptying session.

In Folder Properties/Filing, there is an option to turn off filing for
that folder, or to choose where to file.

Is that not what you want?
--
Ian

[J. P. Gilliver (John)]

In article <+OVvruBl...@nospam.demon.co.uk>, on Mon, 12 Nov 2012,
Ian <ne...@nospam.demon.co.uk> wrote

>In message <rAXatWv$N6nQ...@soft255.demon.co.uk>, "J. P. Gilliver
>(John)" <ju...@nospam.demon.co.uk> writes

[]

>>Some while ago, I asked here if there was any way I could stop TP from
>>keeping a copy of everything I email or post. (If I _want_ to keep a
>>copy, I'd Cc: myself, for emails anyway.) Someone suggested (the
>>general consensus was that there's no way to stop TP making copies as
>>such) changing the settings so that TP stores the copies in the WB
>>rather than the default "Filed". This worked inasmuch as the copies do
>>indeed go into the WB, but they're _not_ deleted when I close TP; if I
>>look into the WB, there is a distinction - one sort (I think it's the
>>ones that _are_ going to be deleted) are shown in grey, the other in black.
>>
>>From time to time I have a WB-emptying session.
>
>In Folder Properties/Filing, there is an option to turn off filing for
>that folder, or to choose where to file.
>
>Is that not what you want?

Could be! I've just set it to not file, and we'll see if this post gets
copied.

--
J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

Jurassic-period viewers like me for whom /The Flintstones/ was actually a
fly-on-the-wall documentary series. - Alison Graham in Radio Times 3-9 March
2012

[J. P. Gilliver (John)]

In article <$JQPf7Kc...@soft255.demon.co.uk>, on Mon, 12 Nov 2012,
"J. P. Gilliver (John)" <ju...@nospam.demon.co.uk> wrote

>In article <+OVvruBl...@nospam.demon.co.uk>, on Mon, 12 Nov 2012,
>Ian <ne...@nospam.demon.co.uk> wrote
>>In message <rAXatWv$N6nQ...@soft255.demon.co.uk>, "J. P. Gilliver
>>(John)" <ju...@nospam.demon.co.uk> writes
>[]
>>>Some while ago, I asked here if there was any way I could stop TP
>>>from keeping a copy of everything I email or post. (If I _want_ to
>>>keep a copy, I'd Cc: myself, for emails anyway.) Someone suggested
>>>(the general consensus was that there's no way to stop TP making
>>>copies as such) changing the settings so that TP stores the copies in
>>>the WB rather than the default "Filed". This worked inasmuch as the
>>>copies do indeed go into the WB, but they're _not_ deleted when I
>>>close TP; if I look into the WB, there is a distinction - one sort (I
>>>think it's the ones that _are_ going to be deleted) are shown in grey, the other in black.
>>>
>>>From time to time I have a WB-emptying session.
>>
>>In Folder Properties/Filing, there is an option to turn off filing for
>>that folder, or to choose where to file.
>>
>>Is that not what you want?
>
>Could be! I've just set it to not file, and we'll see if this post gets
>copied.

Nope - copy in Wastebasket as usual. (And I did make sure the post had
actually been posted.)

[Paul Hyett]

On Sun, 11 Nov 2012 at 18:22:32, Mike Barnes <mikeba...@gmail.com>
wrote in demon.ip.support.turnpike :

Shouldn't the wastebasket automatically clear when you exit the program?
--
Paul Hyett, Cheltenham

[Ian Jackson]

In message <iActtrLx...@blueyonder.co.uk>, Paul Hyett
<p...@invalid.invalid> writes

Only if you choose that option.
--
Ian

[Mike Barnes]

Paul Hyett <p...@invalid.invalid>:

It's optional. I prefer not.

--
Mike Barnes

[Andy]

In message <vCrh+Y5Y...@g52lk5g23lkgk3lk345g.invalid>, Mike Barnes
<mikeba...@gmail.com> wrote

There was some reason - maybe back in TP5 days - for not selecting
auto-empty. Maybe it made some problem-diagnosis difficult?

The more common reason is protection against deleting your master-work!

[Ian Wade G3NRW]

A further sign that Armageddon is nigh.

Until today, I had 102 mailing list folders set up (sad, I know), and
everything worked fine. This morning I set up the 103rd. The mailing
list folder appeared to be created as normal, but subsequent incoming
emails were not routed to that folder.

Solution: I deleted about 20 of the most unused folders. Routing to the
new folder seems to be working OK now.

Bottom line: it looks like 102 mailing list folders is the limit.

--
Ian

[Tony]

Ian Wade G3NRW <g3nrw...@ntlworld.com> wrote on Sat, 17 Nov 2012 at
13:07:50:

More likely a limit in v6.03 for the space available for mailing list
rules. It was increased significantly for v6.07.

--
Tony

[J. P. Gilliver (John)]

In article <Ymn6WMLm...@ntlworld.com>, on Sat, 17 Nov 2012, Ian
Wade G3NRW <g3nrw...@ntlworld.com> wrote

>A further sign that Armageddon is nigh.
>
>Until today, I had 102 mailing list folders set up (sad, I know), and
>everything worked fine. This morning I set up the 103rd. The mailing
>list folder appeared to be created as normal, but subsequent incoming
>emails were not routed to that folder.
>
>Solution: I deleted about 20 of the most unused folders. Routing to the

Did you also send 20 unsubscribe emails (or whatever is the relevant
procedure in each case)? [Or does Turnpike do that when you delete such
a folder? I suspect not.] Of course [unless TP does that] I suppose
you'll still get those emails in your ordinary inbox or whatever, but if
they're little-used lists, you won't necessarily have a list of them all
...

>new folder seems to be working OK now.
>
>Bottom line: it looks like 102 mailing list folders is the limit.
>
--

J. P. Gilliver. UMRA: 1960/<1985 MB++G()AL-IS-Ch++(p)Ar@T+H+Sh0!:`)DNAf

Diplomacy is the art of letting someone have your way.

[Ian Wade G3NRW]

___Original Message_________________________________________
From: "J. P. Gilliver (John)" <ju...@nospam.demon.co.uk>
Date: Sat, 17 Nov 2012 Time: 13:57:02

>
>Did you also send 20 unsubscribe emails (or whatever is the relevant
>procedure in each case)?

No, I didn't need to, because I had already unsubscribed. The folders
were just sitting there with old messages in them. I should have cleaned
up a long time ago, but it's so hard to let go!

--
Ian

[Ian Wade G3NRW]

___Original Message_________________________________________
From: Tony <tonyh1...@hotair.demon.co.uk>
Date: Sat, 17 Nov 2012 Time: 13:40:19

>
>More likely a limit in v6.03 for the space available for mailing list
>rules. It was increased significantly for v6.07.
>

Makes sense. But I'm quite happy with 6.03.

Having 100+ mailing lists was a bit OTT anyway, as I only skim very
briefly through most of them each day.

--
Ian