Thunderbird - Slow search of local folders

[Twenty Ten]

Hi all,

I'm using TB 17.0.6 in portable mode. Search of is generally very slow. For example, if I search only a single local folder with about 7000 messages, it takes about 10 seconds to find emails with a two-word search string and TB freezes up during the search. Sometimes it takes longer.

It's very annoying because it's a business email account and I have to search for old emails frequently enough.

Is there any way to speed up search? I've tried reindexing and cleansing and compacting. I don't know if this is relevant, but I delete attachments from emails when they come in. No emails have any attachments.

Thanks.

[Twenty Ten]

I just installed the new Opera Mail client and imported all my emails to test its search function. It's instantaneous. So the problem must be with TB. No ideas anyone? I'd like to stick with my customized TB version, but I can't if search is so slow.

[Christian Riechers]

Make sure the index creation has been completed. Depending on the size
of your mail archive this may take a while. You can check the status via
Activity Manager.

--
Christian

[Twenty Ten]

Hi Christian,

Thanks for your help! I've indexed and even re-recreated the index.

How fast is TB search for you guys in large email folders?

[Twenty Ten]

OK, so I downloaded a new, clean TB and transferred my emails over. It's fast! So something must be messed up with my default installation.

[Christian Riechers]

On 06/03/2013 09:46 PM, Twenty Ten wrote:
> On Monday, June 3, 2013 9:03:09 PM UTC+2, Christian Riechers wrote:
>> On 06/03/2013 08:50 PM, Twenty Ten wrote:
>>> On Sunday, June 2, 2013 8:58:32 PM UTC+2, Twenty Ten wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I'm using TB 17.0.6 in portable mode. Search of is generally very slow. For example, if I search only a single local folder with about 7000 messages, it takes about 10 seconds to find emails with a two-word search string and TB freezes up during the search. Sometimes it takes longer.
>>>> It's very annoying because it's a business email account and I have to search for old emails frequently enough.
>>>> Is there any way to speed up search? I've tried reindexing and cleansing and compacting. I don't know if this is relevant, but I delete attachments from emails when they come in. No emails have any attachments.
>>>> Thanks.
>>>
>>> I just installed the new Opera Mail client and imported all my emails to test its search function. It's instantaneous. So the problem must be with TB. No ideas anyone? I'd like to stick with my customized TB version, but I can't if search is so slow.
>>
>> Make sure the index creation has been completed. Depending on the size
>> of your mail archive this may take a while. You can check the status via
>> Activity Manager.
>

> Hi Christian,
>
> Thanks for your help! I've indexed and even re-recreated the index.
> How fast is TB search for you guys in large email folders?

My search index (global-messages-db.sqlite) is about 60 MB. Search is
instantly (about a second).

--
Christian

[Twenty Ten]

Thanks! I'm not sure what the issue was, but things seem to be running smoothly again. My search indes is about double and search is also about a second. Not as fast as some of the other programs I was testing, but you can't really complain about a second.

[Twenty Ten]

> Thanks! I'm not sure what the issue was, but things seem to be running smoothly again. My search indes is about double and search is also about a second. Not as fast as some of the other programs I was testing, but you can't really complain about a second.

I've done some more testing and it seems as if search is only fast if there are few results. If there are many, it takes ages. TB is generally awesome...except for search. :(

[Wayne]

On 6/4/2013 2:14 PM, Twenty Ten wrote:
>
>> Thanks! I'm not sure what the issue was, but things seem to be running smoothly again. My search indes is about double and search is also about a second. Not as fast as some of the other programs I was testing, but you can't really complain about a second.
>
> I've done some more testing and it seems as if search is only fast if there are few results. If there are many, it takes ages. TB is generally awesome...except for search. :(

First, let's not assume which "search" you are talking about. Please
clarify which you are using:
a) Search (ctrl+K)
b) Search Messages (ctrl+shift+F)
c) Filter (ctrl+shift+K)

For a) there is no such thing as "single folder search" - it searches
all folders, all accounts. Only a) uses global-messages-db.sqlite and is
affected by the size of global-messages-db.sqlite. (And fwiw, the speed
of these methods is not particularly affected by compact, given that
compact should be happening automatically. And even then only speed of
body searches would be impacted by poorly compacted folders)

Second, if you are using b) or c) did you specify Search or Filter to
look in message body?

Third, if speed is your goal then using "portable" is a poor choice of
hardware. Notwithstanding the fact that you find opera to be faster.

Fourth, why have you stated "local folders" in your posting title?
Should we infer that non-local folders do not suffer the speed problem?

Fifth, not so related to search, you have your business data on a USB.
Why? And is the USB itself backed up to another device?

[Twenty Ten]

Hi Wayne,

> First, let's not assume which "search" you are talking about. Please
>
> clarify which you are using:
>
> a) Search (ctrl+K)
>
> b) Search Messages (ctrl+shift+F)
>
> c) Filter (ctrl+shift+K)

I'm using c). I just tested a) and it's much faster. Why is c) so slow? I'll just use a) from now on.

> Second, if you are using b) or c) did you specify Search or Filter to
>
> look in message body?

Yes, I'm looking in the message body.

> Third, if speed is your goal then using "portable" is a poor choice of
>
> hardware.

It's not my goal, but TB shouldn't be slow, either. But search is a) is perfect speed-wise.

> Notwithstanding the fact that you find opera to be faster.

It was.

> Fourth, why have you stated "local folders" in your posting title?

Because I'm only searching local folders.

> Should we infer that non-local folders do not suffer the speed problem?

No. My non-local folders only have very few emails in them. It's not a good comparison.

> Fifth, not so related to search, you have your business data on a USB.

Nope. TB is a portable installation, but it's on my HD.

> Why? And is the USB itself backed up to another device?

Portable suits my needs better. Switching computers frequently, easier backup, et cetera.

[Wayne]

On 6/7/2013 6:14 AM, Twenty Ten wrote:
> Hi Wayne,
>
>> First, let's not assume which "search" you are talking about. Please
>>
>> clarify which you are using:
>>
>> a) Search (ctrl+K)
>>
>> b) Search Messages (ctrl+shift+F)
>>
>> c) Filter (ctrl+shift+K)
>
> I'm using c). I just tested a) and it's much faster. Why is c) so slow? I'll just use a) from now on.

c) is slower because it reads actual message folder files - when not
searching body it reads the "smallish" summary foldername.msf file, and
when searching body it it reads the entire, not so small foldername file

a) (indexed search) can be faster for large folders because it is
preindexed.

>> Second, if you are using b) or c) did you specify Search or Filter to
>>
>> look in message body?
>
> Yes, I'm looking in the message body.
>
>> Third, if speed is your goal then using "portable" is a poor choice of
>>
>> hardware.
>
> It's not my goal, but TB shouldn't be slow, either. But search is a) is perfect speed-wise.

if your folder is large, say multiple GB (even smaller if on a laptop),
then there's no way to avoid the fact that a non-indexed filter being
slower than an indexed search. If Opera is faster, then it's because it
is using an indexed search or it's doing the search on the mail server.

>> Notwithstanding the fact that you find opera to be faster.
>
> It was.
>
>> Fourth, why have you stated "local folders" in your posting title?
>
> Because I'm only searching local folders.

a) searches ALL folders. c) (filters) scans only the selected folder,
regardless of whether it is local or not. Only with b) can you search
multiple/all folders of an account (local or not) at once.

>> Should we infer that non-local folders do not suffer the speed problem?
>
> No. My non-local folders only have very few emails in them. It's not a good comparison.
>
>> Fifth, not so related to search, you have your business data on a USB.
>
> Nope. TB is a portable installation, but it's on my HD.
>
>> Why? And is the USB itself backed up to another device?
>
> Portable suits my needs better. Switching computers frequently, easier backup, et cetera.

You are referring to
http://portableapps.com/support/thunderbird_portable correct?

I don't see your point. As far as I know, portable offers no advantage
when using multiple computers unless you are using portable *media* such
as USB drive, USB memory stick, etc.

Portable has a decent reputation, but there are specific downsides such
as it's not an official mozilla distribution, and (iirc) does not use
the mozilla automatic update process.

[Twenty Ten]

> You are referring to
>
> http://portableapps.com/support/thunderbird_portable correct?

Yes.

> I don't see your point. As far as I know, portable offers no advantage
>
> when using multiple computers unless you are using portable *media* such
>
> as USB drive, USB memory stick, etc.

It's easier to transfer your installation to other computers and easier to back up and restore. You just grab the directory.

> Portable has a decent reputation, but there are specific downsides such
>
> as it's not an official mozilla distribution, and (iirc) does not use
>
> the mozilla automatic update process.

It does use the automatic update process.