removing duplicate and triplicate emails

[Friedrich Vosberg]

Morning.

I have a local mailbox in Mail.app on OS X 10.8.3 with thousands of
duplicate and triplicate emails. This mailbox includes appr. 19'000
emails, of which appr. two thirds are duplicates, driplicates,
quadruplicates, ..., and the related attachment files as well.

The mailbox is about 17 GB of data volume.

How can I remove all of the dupli-, tripli-, quadruplicates?

Kind regards, Friedrich Vosberg

--
http://www.rafv.de
post (at) vatolin (punkt) de

[Steve Crow]

On Tue, 16 Apr 2013, Friedrich Vosberg wrote:

> Morning.
>
> I have a local mailbox in Mail.app on OS X 10.8.3 with thousands of
> duplicate and triplicate emails. This mailbox includes appr. 19'000
> emails, of which appr. two thirds are duplicates, driplicates,
> quadruplicates, ..., and the related attachment files as well.
>
> The mailbox is about 17 GB of data volume.
>
> How can I remove all of the dupli-, tripli-, quadruplicates?

Friedrich,

A quick Google Search turns up a de-duplicator script discussed here:

http://www.applecider.org/forums/viewtopic.php?id=1560

The download link is here:

http://jollyroger.kicks-ass.org/software/

I have not used it personally but it sounds like the solution to your
problem. What I don't know is how much has changed in Mail.app since 2011
when that forum discussion was posted, so your mileage may vary. As
always, I would back up that mail folder before running any script on it.

I had a similar issue with a Thunderbird mailbox a couple of years ago.
There are tons of scripts for Thunderbird, so it may come down to
migrating that mailbox over to Thunderbird.

Good luck!

Steve

[Friedrich Vosberg]

Steve Crow <otaku!scrow...@sdf.lonestar.org> wrote:

> A quick Google Search turns up a de-duplicator script discussed here:
>
> http://www.applecider.org/forums/viewtopic.php?id=1560
>
> The download link is here:
>
> http://jollyroger.kicks-ass.org/software/
>

> I have not used it personally ...

But I did it. And it did not work. The script freezes after some
seconds.

Kind regards, Friedrich

[Steve Crow]

On Thu, 18 Apr 2013, Friedrich Vosberg wrote:

> Steve Crow <otaku!scrow...@sdf.lonestar.org> wrote:
>
>> A quick Google Search turns up a de-duplicator script discussed here:
>>
>> http://www.applecider.org/forums/viewtopic.php?id=1560
>>
>> The download link is here:
>>
>> http://jollyroger.kicks-ass.org/software/
>>
>> I have not used it personally ...
>
> But I did it. And it did not work. The script freezes after some
> seconds.
>
> Kind regards, Friedrich

Oh well. At least there are other options.

[Jolly Roger]

In article <1l1innl.pie8ebvvl8omN%po...@vatolin.de>,

po...@vatolin.de (Friedrich Vosberg) wrote:

> Steve Crow <otaku!scrow...@sdf.lonestar.org> wrote:
>
> > A quick Google Search turns up a de-duplicator script discussed here:
> >
> > http://www.applecider.org/forums/viewtopic.php?id=1560
> >
> > The download link is here:
> >
> > http://jollyroger.kicks-ass.org/software/
> >
> > I have not used it personally ...
>
> But I did it. And it did not work. The script freezes after some
> seconds.
>
> Kind regards, Friedrich

Like many Applescripts, this script doesn't show progress while it waits
for the mail client to respond. Unfortunately, Applescript support in
mail clients tends to be very slow in terms of performance. I'm betting
you just didn't wait long enough for the script to complete.

--
Send responses to the relevant news group rather than email to me.
E-mail sent to this address may be devoured by my very hungry SPAM
filter. Due to Google's refusal to prevent spammers from posting
messages through their servers, I often ignore posts from Google
Groups. Use a real news client if you want me to see your posts.

JR

[Friedrich Vosberg]

Jolly Roger <jolly...@pobox.com> wrote:

> I'm betting you just didn't wait long enough for the script to complete.

What do you think how long it takes to check a mailbox of 11,22 GB and
containing 9'512 messages?

I could split the messages, create submailboxes for each month of the
respecting year so that I would have 12 folders each appr. 1 GB and less
than 1'000 messages in each submailbox. Would the script those 12
smaller mailboxes check significantly faster than one large mailbox as
descibed above?

Thanks for your support!

[Jolly Roger]

In article <1l1kmik.1r342ze15h4p5sN%po...@vatolin.de>,

po...@vatolin.de (Friedrich Vosberg) wrote:

> Jolly Roger <jolly...@pobox.com> wrote:
>
> > I'm betting you just didn't wait long enough for the script to complete.
>
> What do you think how long it takes to check a mailbox of 11,22 GB and
> containing 9'512 messages?

I've heard of people letting it run overnight and even a couple days,
so... It could take quite a long time, unfortunately.

> I could split the messages, create submailboxes for each month of the
> respecting year so that I would have 12 folders each appr. 1 GB and less
> than 1'000 messages in each submailbox. Would the script those 12
> smaller mailboxes check significantly faster than one large mailbox as
> descibed above?
>
> Thanks for your support!
>
> Kind regards, Friedrich

Absolutely. The more messages you ask the Mail application to process,
the longer the Applescript queries take. The overwhelming majority of
the script's execution time is actually spent idle, waiting for
responses from the Mail application. : ( Reducing the overall count of
messages is one of the most effective ways to control how long the
application takes to answer those queries.

[Chris Ridd]

On 2013-04-20 16:31:42 +0000, Jolly Roger said:

> In article <1l1kmik.1r342ze15h4p5sN%po...@vatolin.de>,
> po...@vatolin.de (Friedrich Vosberg) wrote:
>
>> Jolly Roger <jolly...@pobox.com> wrote:
>>
>>> I'm betting you just didn't wait long enough for the script to complete.
>>
>> What do you think how long it takes to check a mailbox of 11,22 GB and
>> containing 9'512 messages?
>
> I've heard of people letting it run overnight and even a couple days,
> so... It could take quite a long time, unfortunately.

If you're curious you could run Activity Monitor and find the process
that is (probably) using lots of CPU and which looks like the
conversion tool. Then click "Inspect" and look in Open Files and Ports.

You should see the files that it is processing at that time. Repeat
these steps a few times and see if the files change.

It doesn't beat having a proper progress bar, but it might reassure you
that the tool is progressing correctly.

--
Chris