Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Re: TB2 -> TB3 in corporate environment

48 views
Skip to first unread message

Timo Pietilä

unread,
Jan 4, 2010, 10:08:24 AM1/4/10
to Annailís
Annail�s wrote:
> Timo Pietil� wrote:
>> Hello.
>>
>> I have tried to look some advices to how to upgrade thousands of
>> computers from TB2 to TB3 without changing their "feel" and keeping
>> initial settings as close to TB2 as possible.
>>
>> What I would require is at least:
>>
>> Keep "smart folders" out of the picture.
>> Keep toolbars as close to original as possible
>>
>> and because of roaming profiles and gigabyte-class mailboxes
>> - indexing off
>> - IMAP -mail folder settings offline off for all folders.
>>
>> Users could then, if they want to, change those any way they want, but
>> for migration process change is too big and would just block our
>> helpdesk completely if those cannot be set during installation.
>>
>> Is there any way to force these settings for initial installation?
>>
>> Timo Pietil�
>
> You perhaps should ask these questions in the
> mozilla.dev.apps.thunderbird group as well.

x-post added there.

Timo Pietil�

Mark Banner

unread,
Jan 6, 2010, 4:20:44 AM1/6/10
to
On 04/01/2010 15:08, Timo Pietil� wrote:
> Annail�s wrote:
>> Timo Pietil� wrote:
>>> Hello.
>>>
>>> I have tried to look some advices to how to upgrade thousands of
>>> computers from TB2 to TB3 without changing their "feel" and keeping
>>> initial settings as close to TB2 as possible.
>>>
>>> What I would require is at least:
>>>
>>> Keep "smart folders" out of the picture.
>>> Keep toolbars as close to original as possible
>>>
>>> and because of roaming profiles and gigabyte-class mailboxes
>>> - indexing off
>>> - IMAP -mail folder settings offline off for all folders.
>>>
>>> Users could then, if they want to, change those any way they want, but
>>> for migration process change is too big and would just block our
>>> helpdesk completely if those cannot be set during installation.
>>>
>>> Is there any way to force these settings for initial installation?

How do you deploy and set up Thunderbird currently?

I can think of a mixture of two possible ways: one via preferences in
Thunderbird, the second via an extension. Which method depends on how it
is currently coded, but I expect something could be done to improve this.

It would be interesting to hear from other organisations as well about
upgrade experiences as I know there were one or two working with us
before the release that didn't seem to be concerned about the issues
that you're raising.

Standard8

Timo Pietilä

unread,
Jan 7, 2010, 7:06:57 AM1/7/10
to
Mark Banner wrote:
> On 04/01/2010 15:08, Timo Pietil� wrote:
>> Annail�s wrote:
>>> Timo Pietil� wrote:
>>>> Hello.
>>>>
>>>> I have tried to look some advices to how to upgrade thousands of
>>>> computers from TB2 to TB3 without changing their "feel" and keeping
>>>> initial settings as close to TB2 as possible.
>>>>
>>>> What I would require is at least:
>>>>
>>>> Keep "smart folders" out of the picture.
>>>> Keep toolbars as close to original as possible
>>>>
>>>> and because of roaming profiles and gigabyte-class mailboxes
>>>> - indexing off
>>>> - IMAP -mail folder settings offline off for all folders.

This BTW almost classifies as a bug. IIRC upgrading to TB3 doesn't keep
older Thunderbird 2 folder settings. It instead marks all "offline"
downloads on on all folders.

>>>> Users could then, if they want to, change those any way they want, but
>>>> for migration process change is too big and would just block our
>>>> helpdesk completely if those cannot be set during installation.
>>>>
>>>> Is there any way to force these settings for initial installation?
>
> How do you deploy and set up Thunderbird currently?

Repackage to MSI then deploy. This is necessary because I need to turn
off autoupdate and stuff like that.

> I can think of a mixture of two possible ways: one via preferences in
> Thunderbird, the second via an extension. Which method depends on how it
> is currently coded, but I expect something could be done to improve this.

Some sort of default migration settings could be nice. Many things can
be set using all-Thunderbird.js and prefs.js under "defaults" in program
installation, but migration settings are not there.

Also some sort of "policy settings" that override users settings could
be nice to have (like what sites are allowed to show popups etc.)

> It would be interesting to hear from other organisations as well about
> upgrade experiences as I know there were one or two working with us
> before the release that didn't seem to be concerned about the issues
> that you're raising.

One other thing I miss in both Thunderbird and in Firefox are decent
certificate handling. It would be nice to be able to easily import and
export certificates directly to users certificate store.

Timo Pietil�

Timo Pietilä

unread,
Jan 11, 2010, 6:20:45 AM1/11/10
to
Mark Banner wrote:
> On 04/01/2010 15:08, Timo Pietil� wrote:
>> Annail�s wrote:
>>> Timo Pietil� wrote:
>>>> Hello.
>>>>
>>>> I have tried to look some advices to how to upgrade thousands of
>>>> computers from TB2 to TB3 without changing their "feel" and keeping
>>>> initial settings as close to TB2 as possible.
>>>>
>>>> What I would require is at least:
>>>>
>>>> Keep "smart folders" out of the picture.
>>>> Keep toolbars as close to original as possible
>>>>
>>>> and because of roaming profiles and gigabyte-class mailboxes
>>>> - indexing off
>>>> - IMAP -mail folder settings offline off for all folders.
>>>>
>>>> Users could then, if they want to, change those any way they want, but
>>>> for migration process change is too big and would just block our
>>>> helpdesk completely if those cannot be set during installation.
>>>>
>>>> Is there any way to force these settings for initial installation?
>
> How do you deploy and set up Thunderbird currently?
>
> I can think of a mixture of two possible ways: one via preferences in
> Thunderbird, the second via an extension. Which method depends on how it
> is currently coded, but I expect something could be done to improve this.

Any news on this? Developer think tank for corporate environment upgrade
processes? How to make it as invisible as possible and handling thru
some sort of policies or somesuch?

> It would be interesting to hear from other organisations as well about
> upgrade experiences as I know there were one or two working with us
> before the release that didn't seem to be concerned about the issues
> that you're raising.

That depends of corporate. If those are IT-industry organizations then
typical user knows how to read and write and possible even talk computer.

In university-environment even people with IQ nearing 200 don't always
know difference between monitor and computer and "internet is broken"
when their e-mail program icon changes color. And apparently they also
forget alphabets if program asks them to do something that is a bit
technical (like "installation might require reboot" or "this operation
wipes all of your harddisks, are you absolutely sure to continue?" or
something like that) and they hurry to press "OK" in order to get rid of
that annoying popup.

Some of them do understand computers a bit but majority only use them
for work and know only those few programs they need for their work and
don't care less about others or what's makes them tick, and if something
needs to be done for programs they don't care as long as it is done as
invisible as possible and interrupt their work as little as possible.

Timo Pietil�

Mark Banner

unread,
Jan 11, 2010, 6:48:08 AM1/11/10
to
On 11/01/2010 11:20, Timo Pietil� wrote:
> Any news on this? Developer think tank for corporate environment upgrade
> processes? How to make it as invisible as possible and handling thru
> some sort of policies or somesuch?

Sorry, I've been tied up preparing 3.0.1 for the last few days. I've
added it to my list of things to look at, hopefully later this week.

Standard8

Jason Oster

unread,
Jan 11, 2010, 2:46:39 PM1/11/10
to
On 01/04/2010 08:08 AM, Timo Pietilä wrote:
> Annailís wrote:

>> Timo Pietilä wrote:
>>> Hello.
>>>
>>> I have tried to look some advices to how to upgrade thousands of
>>> computers from TB2 to TB3 without changing their "feel" and keeping
>>> initial settings as close to TB2 as possible.
>>>
>>> What I would require is at least:
>>>
>>> Keep "smart folders" out of the picture.
>>> Keep toolbars as close to original as possible
>>>
>>> and because of roaming profiles and gigabyte-class mailboxes
>>> - indexing off
>>> - IMAP -mail folder settings offline off for all folders.
>>>
>>> Users could then, if they want to, change those any way they want, but
>>> for migration process change is too big and would just block our
>>> helpdesk completely if those cannot be set during installation.
>>>
>>> Is there any way to force these settings for initial installation?
>>>
>>> Timo Pietilä

>>
>> You perhaps should ask these questions in the
>> mozilla.dev.apps.thunderbird group as well.
>
> x-post added there.
>
> Timo Pietilä

Hello Timo,

At our organization, where I'm in charge of deploying updates, I've
found that "disabling" Smart Folders means copying or patching
localstore.rdf in the user's Thunderbird profile. I typically create a
default localstore.rdf by opening Thunderbird with a new profile, and
copying the file from there into our deployment server.

When installing/updating Thunderbird, that file just gets copied into
the Default User Thunderbird profile, and any other profiles on the
workstations.

Overwriting this file might not work for you, since it will change
toolbar positions, window size/position, etc. A Diff/Patch approach
might be a better option.

Timo Pietilä

unread,
Jan 12, 2010, 3:08:24 AM1/12/10
to

Or just deploy using a script that temporarily copies that file to safe
place and restores it after upgrading has happened. Or does the format
of the file change?

I need to check that. This alone is great help for upgrading, thanks.

There is still the problem that TB3 has bugs and is not very stable yet,
and TB2 has security hole that apparently gets never fixed. This is very
very bad for Thunderbird reputation.

Developers! note this:

In corporate environments new stuff gets never ever installed at X.0
stage, because experience has told that new stuff is never as stable as
old for few first versions of new product. Corporates use stuff that a)
works, b) is stable. New features are not concern of us. If it works
there is no point replacing it unless new version offers something that
users would need. This is not the case with TB3.

In fact I don't see any reason to upgrade to TB3 at all. I use TB2 at
work, rock solid, no bugs, works like a charm, and TB3 at home because I
needed to see if it is any worth, slow, uses huge amount of additional
disk space, I haven't found any features yet that makes me want to
upgrade. In fact, if there weren't that security issue I would revert
back to TB2 for home machine too.

Timo Pietilä

klint

unread,
Jan 15, 2010, 4:44:23 AM1/15/10
to
The new features I'm using HEAVILY in my company are:
- Open a full thread in a tab (using the Gmail Conversation View
addon)
- Global search
- Full-fledged calendar management with new Lighting addon (requires
TB3)
- Mails in tabs
- Less resource consumption

These alone were sufficient drivers for migration.

My 5cts!

Wayne Mery

unread,
Jan 15, 2010, 10:02:09 AM1/15/10
to
On 1/15/2010 4:44 AM, klint wrote:
> - Less resource consumption

klint, can you elaborate on that point?

--
http://wiki.mozilla.org/Thunderbird:Testing
http://www.spreadthunderbird.com/aff/165/

klint

unread,
Jan 15, 2010, 11:55:47 AM1/15/10
to
I'm sorry but I haven't noted figures, but from my experience at work
and at home (4 years old Celeron on XP), I've noted that less memory
is consumed than in TB2, and the overall response time is better. With
one exception : the folder panel is redrawn with some latency with the
Celeron, and the time to display a message is a little bit longer than
in TB2, but still acceptable (I have seen some posts complaining about
unbearable response times with TB3, but that is not my own
experience).

Of course, you should wait for and test the next version to be
released very soon (before the end of the month), TB3.01, correcting a
lot of youth issues, and hopefully all migration issues met so far
(they were some...)

Last, to be honest, I'm not using TB3 at work as the official
corporate tool (it is Outlook 2003), so I have no experience about
rolling out TB3 to a large number of users.

Hope this helps anyway.


Nicolas Cuissard

unread,
Jan 25, 2010, 5:00:56 AM1/25/10
to
Timo Pietil� a �crit :


Hi all,

I'm dealing with the same issues, this is what I did so far:

* Preventing indexation : set the preference
"mailnews.database.global.indexer.enabled" to "false"

* Preventing offline synchronization for IMAP account : for each
account, set the preference "mail.server.<server_id>.offline_download"
to "false".

Do do that, the easiest way is to use the "autoconfig" which allow you
to force setting on a system-level bassis and not on user-lever. It's
very simple, you need to deploy two files (works on linux too) :

* "%PROGRAMFILES%\Mozilla Thunderbird\defaults\pref\oldschool.js"
containing :

pref("general.config.obscure_value", 0);
pref("general.config.filename", "oldschool.cfg");

* "%PROGRAMFILES%\Mozilla Thunderbird\oldschool.cfg" containing :

//
/* This will disable indexation */
lockPref("mailnews.database.global.indexer.enabled", false);

/* This will parse the prefs.js and disable "offline_download" for each
IMAP account */

if (getPref("mail.accountmanager.accounts")) {

var listExistingAccounts = getPref("mail.accountmanager.accounts");
var arrayExistingAccounts = listExistingAccounts.split(',');

for (var i=0; i < arrayExistingAccounts.length; i++) {
var serverFromAccount = getPref("mail.account." +
arrayExistingAccounts[i] + ".server");
var configType = getPref("mail.server." + serverFromAccount +
".type");
if (configType == "imap") {
lockPref("mail.server." + serverFromAccount +
".offline_download", false);
}
}
}

I'm still investigating a way to disable smart folders automatically but
as far as it's not managed in the "prefs.js" (but "localstore.rdf") this
is far more tricky.

Regards,

Nicolas

Timo Pietilä

unread,
Mar 2, 2010, 7:01:36 AM3/2/10
to
Nicolas Cuissard wrote:

<snip>

> /* This will parse the prefs.js and disable "offline_download" for each
> IMAP account */
>
> if (getPref("mail.accountmanager.accounts")) {
>
> var listExistingAccounts = getPref("mail.accountmanager.accounts");
> var arrayExistingAccounts = listExistingAccounts.split(',');
>
> for (var i=0; i < arrayExistingAccounts.length; i++) {
> var serverFromAccount = getPref("mail.account." +
> arrayExistingAccounts[i] + ".server");
> var configType = getPref("mail.server." + serverFromAccount +
> ".type");
> if (configType == "imap") {
> lockPref("mail.server." + serverFromAccount +
> ".offline_download", false);
> }
> }
> }

I used that, and it did disable offline synchronization for account, but
it did keep them for folders (basically "no not synchronize for this
account, but download everything anyway). Which is weird. Also
"migration assistant" appears in upgrades.

localstore.rdf is also pain. Apparently default settings do not apply to
upgraded installations, only for new installations.

That still needs work to do.

Any smart ideas? If I came up something I put it here (now that I again
have some time to concentrate on TB3 -issues).

Timo Pietil�

Timo Pietilä

unread,
Mar 2, 2010, 9:43:18 AM3/2/10
to

This is what I have done this far:

File %programfiles%\mozilla thunderbird\settings.cfg

----------


//
/* This will disable indexation */
lockPref("mailnews.database.global.indexer.enabled", false);

/* prevent offline download */
lockPref("mail.server.default.offline_download", false);

/* no autosync of folders after migration */
lockPref("mail.server.default.autosync_offline_stores",false);

/* no copying mail files for migration */
lockPref("mail.migration.copyMailFiles", false);

/* disable autoupdate */
lockPref("app.update.enabled", false);

/* disable autoupdate */
lockPref("app.update.auto", false);

/* This will parse the prefs.js and disable "offline_download" for each
IMAP account */

if (getPref("mail.accountmanager.accounts")) {

var listExistingAccounts = getPref("mail.accountmanager.accounts");
var arrayExistingAccounts = listExistingAccounts.split(',');

for (var i=0; i < arrayExistingAccounts.length; i++) {
var serverFromAccount = getPref("mail.account." +
arrayExistingAccounts[i] + ".server");
var configType = getPref("mail.server." + serverFromAccount +
".type");
if (configType == "imap") {
lockPref("mail.server." + serverFromAccount +
".offline_download", false);
}
}
}

---------
File %programfiles%\mozilla thunderbird\defaults\pref\settings.js

-----
pref("general.config.obscure_value", 0);
pref("general.config.filename", "settings.cfg");
-----

Edit file messenger.xul *INSIDE*
%programfiles%\Mozilla Thunderbird\chrome\messenger.jar

Change this part:

<tree id="folderTree" class="plain" flex="1"
hidecolumnpicker="true" persist="mode" mode="smart"

to

<tree id="folderTree" class="plain" flex="1"
hidecolumnpicker="true" persist="mode" mode="all"

and close file.

This makes folder view "all" default after migration from 2.0.0.X to 3.X
to all *OLD* accounts too. It doesn't force that, so users can still
change it to "smart folders" if they want to (I don't see any reason to
use that, but who knows?).

Issues still not solved:

Looks like all IMAP -folders are still marked for "offline download"
after migration even that account itself is not (new account doesn't
have that problem), but it seems that at least it doesn't start
downloading them immediately after migration. Need to check if this is
permanent change, or if those files still get downloaded after some time.

Message header panel is unusable large with all those buttons. Maybe I
need to add some add-on to disable those and revert to old behavior
where buttons are all under toolbar.

Migration wizard appears after migration even that it is no longer
needed. No idea yet how to disable that.

Same thing for "know your rights" -bar.

New account setting wizard is too stupid and as such panic-causing
nightmare for new users. Need to disable that. No idea how to yet.

Timo Pietil�

Ben Bucksch

unread,
Mar 2, 2010, 10:43:03 AM3/2/10
to
On 02.03.2010 15:43, Timo Pietil� wrote:
> /* This will disable indexation */
> lockPref("mailnews.database.global.indexer.enabled", false);
>
> /* prevent offline download */
> lockPref("mail.server.default.offline_download", false);
>
> /* no autosync of folders after migration */
> lockPref("mail.server.default.autosync_offline_stores",false);
>
> /* no copying mail files for migration */
> lockPref("mail.migration.copyMailFiles", false);

Why do you *lock* these prefs instead of just changing the default? In
other words, if a user specifically wants to enable gloda (it's useful,
after all!), why would you *forbid* them?

If you just want to easy migration, you should just change the default.
You can do that by putting another foo.js (alphabethic order of files
matters!) where you have the same pref() lines as in mailnews.js, just
with different values, as you already do with settings.js.

Please note that gloda is extraordinarily useful, and will become ever
more useful. Forcing a disable and not even allowing users to enable it
is "unwise".

> /* disable autoupdate */
> lockPref("app.update.enabled", false);
>
> /* disable autoupdate */
> lockPref("app.update.auto", false);

And that's an outright security risk, unless you have a system to deploy
updates automatically throughout your organization.

> Edit file messenger.xul *INSIDE*
> %programfiles%\Mozilla Thunderbird\chrome\messenger.jar
> Change this part:
> <tree id="folderTree" class="plain" flex="1"
> hidecolumnpicker="true" persist="mode" mode="smart"

> _to
> _ <tree id="folderTree" class="plain" flex="1"


> hidecolumnpicker="true" persist="mode" mode="all"

I strongly recommend against messing with source code files. Any update
will overwrite it, and fast updates are critical for security. In fact,
you were yelling loudly for them and calling names, because we didn't
deliver security updates to old branches fast enough. With this, you
only make updates hard for yourself.

Instead of changing messenger.xul, just alter localstore.rdf in the
profile. That's what's it for.

In general, you shouldn't mess with Thunderbird on this low level. You
will break more than you fix, overall.

Ben

Mark Banner

unread,
Mar 2, 2010, 11:16:41 AM3/2/10
to

I'm actually just thinking, couldn't a simple extension override that value?

Standard8

Timo Pietilä

unread,
Mar 2, 2010, 11:25:34 AM3/2/10
to
Ben Bucksch wrote:
> On 02.03.2010 15:43, Timo Pietil� wrote:
>> /* This will disable indexation */
>> lockPref("mailnews.database.global.indexer.enabled", false);
>>
>> /* prevent offline download */
>> lockPref("mail.server.default.offline_download", false);
>>
>> /* no autosync of folders after migration */
>> lockPref("mail.server.default.autosync_offline_stores",false);
>>
>> /* no copying mail files for migration */
>> lockPref("mail.migration.copyMailFiles", false);
>
> Why do you *lock* these prefs instead of just changing the default? In
> other words, if a user specifically wants to enable gloda (it's useful,
> after all!), why would you *forbid* them?

Because that ruins roaming profiles. I don't like allowing users to
shoot at their own legs.

>> /* disable autoupdate */
>> lockPref("app.update.enabled", false);
>>
>> /* disable autoupdate */
>> lockPref("app.update.auto", false);
>
> And that's an outright security risk, unless you have a system to deploy
> updates automatically throughout your organization.

I do have such a system.

>> Edit file messenger.xul *INSIDE*
>> %programfiles%\Mozilla Thunderbird\chrome\messenger.jar
>> Change this part:
>> <tree id="folderTree" class="plain" flex="1"
>> hidecolumnpicker="true" persist="mode" mode="smart"
>> _to
>> _ <tree id="folderTree" class="plain" flex="1"
>> hidecolumnpicker="true" persist="mode" mode="all"
>
> I strongly recommend against messing with source code files. Any update
> will overwrite it, and fast updates are critical for security.

Fast? 2.0.0.23 is over six months old.

> In fact,
> you were yelling loudly for them and calling names, because we didn't
> deliver security updates to old branches fast enough. With this, you
> only make updates hard for yourself.

Nope. Repackaging Thunderbird with reasonable settings is about 15
minutes work (after I have figured out those settings).

> Instead of changing messenger.xul, just alter localstore.rdf in the
> profile. That's what's it for.

How do you do that for 12000+ computers with old 2.0.0.X profiles during
migration phase?

Give me a solution and I do that.

> In general, you shouldn't mess with Thunderbird on this low level. You
> will break more than you fix, overall.

That one doesn't break anything.

Timo Pietil�

Timo Pietilä

unread,
Mar 2, 2010, 11:29:33 AM3/2/10
to

I don't want to force that setting, only change the default so that
users do have usable UI after migration. Any recommendation I have this
far found don't affect migration phase. This is only solution I have
figured out that makes both: changes default and allows change later if
user wants that.

Timo Pietil�

JarrE

unread,
Mar 2, 2010, 1:55:13 PM3/2/10
to Timo Pietilä
On 02.03.2010 17:25, Timo Pietil� wrote:
> Ben Bucksch wrote:

>>> /* disable autoupdate */
>>> lockPref("app.update.enabled", false);
>>>
>>> /* disable autoupdate */
>>> lockPref("app.update.auto", false);
>>
>> And that's an outright security risk, unless you have a system to
>> deploy updates automatically throughout your organization.
>
> I do have such a system.

And besides: users do not have elevated privileges - so automatic updates
are only noise (and I don't like noise)

Mvh/
JarrE

JarrE

unread,
Mar 2, 2010, 1:55:29 PM3/2/10
to
On 02.03.2010 17:25, Timo Pietil� wrote:
> Ben Bucksch wrote:

>>> /* disable autoupdate */
>>> lockPref("app.update.enabled", false);
>>>
>>> /* disable autoupdate */
>>> lockPref("app.update.auto", false);
>>
>> And that's an outright security risk, unless you have a system to
>> deploy updates automatically throughout your organization.
>
> I do have such a system.

And besides: users do not have elevated privileges - so automatic updates

Ben Bucksch

unread,
Mar 2, 2010, 6:37:31 PM3/2/10
to
On 02.03.2010 17:25, Timo Pietil� wrote:
> Ben Bucksch wrote:
>> In other words, if a user specifically wants to enable gloda (it's
>> useful, after all!), why would you *forbid* them?
>
> Because that ruins roaming profiles. I don't like allowing users to
> shoot at their own legs.

asuth, shouldn't gloda be in the non-roaming part of Windows profiles?
At least as option?

Timo Pietilä

unread,
Mar 3, 2010, 2:31:44 AM3/3/10
to

It should, but as it is now all things in Thunderbird except temp-files
go to appdata and not in local settings\appdata.

Also I think that in order to gloda to work it requires that all
messages are being downloaded into computer, so it is not only that
quite big database it constructs that prevents using it it is also the
hugely increased message load in profile. Those also go to appdata, and
not in local settings\appdata. It breaks quota limits and causes huge
network load.

That one is the major blocking "feature" in TB3 that prevents migrating
to it in corporate environment. It is worse than program crash. It is
even worse than single computer crash. It equals that every computer you
use crashes in entire corporation after you _once_ read your posts in
computer with gloda enabled for that person.

I believe this is being discussed by developers and some future release
will correct this showstopping "feature".

Timo Pietil�

Andrew Sutherland

unread,
Mar 3, 2010, 3:17:43 AM3/3/10
to
On 03/02/2010 11:31 PM, Timo Pietil� wrote:
> Also I think that in order to gloda to work it requires that all
> messages are being downloaded into computer, so it is not only that

Gloda will actually work even without the message bodies available; it
just won't work as well. I've created the following page to try and
capture relevant information about the indexing process that is likely
not appropriate for the support site (support.mozillamessaging.com) but
which would also be unreasonable to expect people to read the source
comments to figure out:

https://developer.mozilla.org/en/Thunderbird/Gloda_indexing

Andrew

Timo Pietilä

unread,
Mar 3, 2010, 3:34:23 AM3/3/10
to

Thanks for the info.

There is one part in that that catches my eye:

-----
What Messages get Indexed?

A message is indexed if all of the following are true:

* Its folder is indexed.
* The folder is marked for offline storage and the message body is
already offline. (Otherwise we wait for the message body to get
downloaded by auto-sync or some other mechanism.)
* The folder is not marked for offline storage. (We just don't
index the contents of the body and the attachment names.)
-----

For some reason migrating from 2.0.0.X account to 3.0.x account you get
all of your IMAP-folders marked for offline storage even that account
itself can be prevented for that to happen. I think that qualifies as a
bug, because if I create new account using that same installation then
none of the folders get that mark.

What happens if I have offline synchronization off at account level but
on at folder level and I activate gloda? Does it trigger offline
download for indexing purposes? Does it wait indefinitely for that
download to happen before indexing that message? Does it index the
header only, and wait for the rest?

Timo Pietil�

Timo Pietilä

unread,
Mar 3, 2010, 5:21:35 AM3/3/10
to

One another question about gloda: how much impact does it have to disk
space if only message headers are being indexed? Some estimate would be
nice, so that I can estimate do I allow it or not.

Lets say person has 20000 messages at his mailbox, and entire mailbox
size is 500MB (with attachments taking 90% of space). What is gloda
database size with only headers included?

Timo Pietil�

Andrew Sutherland

unread,
Mar 3, 2010, 6:16:56 AM3/3/10
to
On 03/03/2010 02:21 AM, Timo Pietil� wrote:
> One another question about gloda: how much impact does it have to disk
> space if only message headers are being indexed? Some estimate would be
> nice, so that I can estimate do I allow it or not.
>
> Lets say person has 20000 messages at his mailbox, and entire mailbox
> size is 500MB (with attachments taking 90% of space). What is gloda
> database size with only headers included?

Couldn't say; we haven't tried to gather such numbers. If someone would
like to try and research such numbers and put them on wiki.mozilla.org,
that would be very useful and interesting.

I can say that running the sqlite3_analyzer against my own gloda
database (available here: http://www.sqlite.org/download.html ), it
tells me that:

69.4% of the pages in my database are used for the messagesText table.
23.7% are for messages/messageAttributes/other
5.8% are free pages

Now, messagesText would still have stuff in it without bodies (those
e-mail addresses and message subjects). As such, I would not be
surprised if the database ends up being 1/3 of its size or maybe even
smaller. Of course, this is also my e-mail repository which is made up
of a lot of technical mailing lists and bug reports, and that may not be
representative of your users.

Andrew

Nicolas Cuissard

unread,
Mar 9, 2010, 9:29:55 AM3/9/10
to
Timo Pietil� a �crit :

> Nicolas Cuissard wrote:
>
> <snip>
>
>> /* This will parse the prefs.js and disable "offline_download" for
>> each IMAP account */
>>
>> if (getPref("mail.accountmanager.accounts")) {
>>
>> var listExistingAccounts = getPref("mail.accountmanager.accounts");
>> var arrayExistingAccounts = listExistingAccounts.split(',');
>>
>> for (var i=0; i < arrayExistingAccounts.length; i++) { var
>> serverFromAccount = getPref("mail.account." +
>> arrayExistingAccounts[i] + ".server"); var configType =
>> getPref("mail.server." + serverFromAccount + ".type"); if
>> (configType == "imap") { lockPref("mail.server." +
>> serverFromAccount + ".offline_download", false); } } }
>
> I used that, and it did disable offline synchronization for account,
> but it did keep them for folders (basically "no not synchronize for
> this account, but download everything anyway). Which is weird.

Be careful, the script will set the settings on the first run of
thunderbird, but it will be effective on the second run only.

This means that if you run the autoconfig script the first time
Thunderbird 3.x is launched (just after upgrade), it won't work.

That's why I already applied this script on all my boxes running
Thunderbird 2.x : the preferences will be ready for Thunderbird 3.x.

Try to run the autoconfig script on Thunderbird 2.x, close and open
Thunderbird once, upgrade to 3.x and see if the IMAP accounts are
synchronized.

It didn't for me.

> Also "migration assistant" appears in upgrades.

Yes, I couldn't get ride of it, very annoying.

> localstore.rdf is also pain. Apparently default settings do not apply
> to upgraded installations, only for new installations.

Yes, existing "localstore.rdf" are not affected by
"defaults\profile\localstore.rdf", it is used for initialization of the
profile only.

--
Nicolas

Timo Pietilä

unread,
Mar 10, 2010, 2:12:19 AM3/10/10
to
Nicolas Cuissard wrote:
> Timo Pietil� a �crit :

>> localstore.rdf is also pain. Apparently default settings do not apply
>> to upgraded installations, only for new installations.
>
> Yes, existing "localstore.rdf" are not affected by
> "defaults\profile\localstore.rdf", it is used for initialization of the
> profile only.

That is kind of annoying that you can't set default setting for feature
that isn't there in old versions. I assumed that during installation of
TB3 in top of TB2 profile it would read defaults from that file. No such
luck.

Default for installation can be set by tweaking insides of jar -file,
but, as people here has been saying to me, it is not the preferred way
to do things. Unfortunately that is only easy way to do that during
migration.

Obviously you can write scripts to change user settings, but that is too
much work for too insignificant program.

Developers: I, and believe most, would really like that during migration
from one version to another default is that user settings are changed as
little as possible. In future versions when new features are introduced,
don't make those new features forced default when there *is* setting for
that in old version. Especially something like "smart folders" which
usefulness is more than questionable. You can make it default for new
installations, but changing user setting is really something you just
don't do.

Timo Pietil�

Jaakko Luoto

unread,
Mar 15, 2010, 5:39:15 AM3/15/10
to
Timo Pietilä wrote:
> Obviously you can write scripts to change user settings, but that is too
> much work for too insignificant program.
>
> Developers: I, and believe most, would really like that during migration
> from one version to another default is that user settings are changed as
> little as possible. In future versions when new features are introduced,
> don't make those new features forced default when there *is* setting for
> that in old version. Especially something like "smart folders" which
> usefulness is more than questionable. You can make it default for new
> installations, but changing user setting is really something you just
> don't do.

Props to Timo on reporting these problems. Thunderbird 2 is the primary
mail reader at our faculty. We repackage it to MSI for deployment with
TB's own update features disabled.

Some of the issues mentioned in this thread have become blockers on
migrating our users to TB 3.0. So we are waiting for 3.1 and hoping that
there will be system-wide configuration options which can be used to
disable these new features:

- IMAP-folders being marked automatically as offline folders
- All automatic indexing
- Automatic server name guessing during new account creation, without
the option to do manual configuration right from the start (it guesses
and uses exactly the server name that we DON'T want to direct our users
use by default)

Also we dislike, but might cope with:
- Migration wizard
- Smart folders view mode on by default

The new organization of UI elements is not a big problem.

--
Jaakko Luoto <jaakko.luot...@tut.fi>
Faculty of Computing and Electrical Engineering at TUT
IT Administration

Blake Winton

unread,
Mar 15, 2010, 11:11:24 AM3/15/10
to
On 10-03-15 5:39 , Jaakko Luoto wrote:
> - IMAP-folders being marked automatically as offline folders
> - All automatic indexing
> - Automatic server name guessing during new account creation, without
> the option to do manual configuration right from the start (it
> guesses and uses exactly the server name that we DON'T want to
> direct our users use by default)

There is a way for you to feed your users exactly the settings you would
like them to use. If you are interested, please email me, and I'll help
you through the process.

> Also we dislike, but might cope with: - Migration wizard

I'm doing some work on the Migration Wizard now. What sort of stuff
could we do with it so that you would like it more?

> - Smart folders view mode on by default

I believe this has been turned off for 3.1

Thanks,
Blake.

Timo Pietilä

unread,
Mar 15, 2010, 6:56:55 PM3/15/10
to
Blake Winton wrote:
> On 10-03-15 5:39 , Jaakko Luoto wrote:
>> - IMAP-folders being marked automatically as offline folders
>> - All automatic indexing

These two are the real blockers, because they make use of roaming
profiles impossibility.

>> - Automatic server name guessing during new account creation, without
>> the option to do manual configuration right from the start (it
>> guesses and uses exactly the server name that we DON'T want to
>> direct our users use by default)
>
> There is a way for you to feed your users exactly the settings you would
> like them to use. If you are interested, please email me, and I'll help
> you through the process.
>
>> Also we dislike, but might cope with: - Migration wizard
>
> I'm doing some work on the Migration Wizard now. What sort of stuff
> could we do with it so that you would like it more?

Possibility to disable it during update. In corporate environment where
you have set most of the user UI by autoconfig scripts that Migration
Wizard is unnecessary. However, if user wants to change settings and
wants to learn "whats new" then that Wizard could be useful, so complete
disable for entire feature is not what I would want.

Note that some of us (admins) are dealing with people that have
"internet broken" when their e-mail program icon changes shape or color,
and who do not understand difference between computer and monitor. They
use computers for work without slightest idea how they really work, and
if we can do things with as little interference as possible to what they
are used to the better. Migration wizard just alerts them that
"something catastrophic has happened" and helpdesk gets drowned by
phonecalls.

I have seen otherwise smart people losing their ability to read when
computer unexpectedly prompts them to do something: "Are you sure you
want to delete all the work you have done in last six months?" (Yes/No),
"IIIK! Need to get rid of that popup! (pressing OK)" "Where did all my
work go?".

>> - Smart folders view mode on by default
>
> I believe this has been turned off for 3.1

Beautiful. That and that Migration Wizard are the only two features I
haven't yet figured out how to set during migration from TB2 to TB3.
Especially this smart folders required so deep tweaking that I rather
not do it (and several people said to me that it is not how things are
supposed to be done).

If I could also get that "compress headers" or whatever its name was as
TB3 default feature I would be happy. To me ability to get rid of
unnecessary headers in message panel is a valuable feature, and current
TB3 design doesn't allow that because it moved some of the buttons away
of rest of the buttons. I *much* rather take ability to collapse headers
to single line with one mouse click than use those buttons in header panel.

BTW. Now that I have learned howto use autoconfig script (IE. one .js
file that tells that there is one .cfg file in root of the program
directory which contains system-wide configurations) I actually like it.
Makes setting things much easier. However there is no good FAQ of how to
set that thing anywhere in "the net". At least I haven't found one.
There are lots of bits and pieces here and there, but nothing "official".

If any of the Mozilla Wiki people read this good documentation about how
to set configuration script would be very valuable for corporate admins.
And also some oddities explained (like how Firefox default homepage
setting is localized string that needs to be entered in special way).

Timo Pietilä

Dan Mosedale

unread,
Mar 17, 2010, 8:08:44 PM3/17/10
to dev-apps-t...@lists.mozilla.org
On 3/15/10 3:56 PM, Timo Pietilä wrote:
> BTW. Now that I have learned howto use autoconfig script (IE. one .js
> file that tells that there is one .cfg file in root of the program
> directory which contains system-wide configurations) I actually like
> it. Makes setting things much easier. However there is no good FAQ of
> how to set that thing anywhere in "the net". At least I haven't found
> one. There are lots of bits and pieces here and there, but nothing
> "official".
If I understood jenzed correctly on this week's call, someone from the
community has stepped up to put such a page together. Jen, is that
correct? If so, does that page want to live on SuMo?

> If any of the Mozilla Wiki people read this good documentation about
> how to set configuration script would be very valuable for corporate
> admins. And also some oddities explained (like how Firefox default
> homepage setting is localized string that needs to be entered in
> special way).
I suspect a list of the possible prefs and other config tweaks that site
admins are likely to want to make deserves it's own page on SuMo,
probably one that cross-links with the autoconfig page mentioned above.
Jen, Roland, what are your thoughts here?

Dan

Jaakko Luoto

unread,
Mar 18, 2010, 9:39:51 AM3/18/10
to
Blake Winton wrote:
>> - IMAP-folders being marked automatically as offline folders
>> - All automatic indexing
>> - Automatic server name guessing during new account creation, without
>> the option to do manual configuration right from the start (it
>> guesses and uses exactly the server name that we DON'T want to
>> direct our users use by default)
>
> There is a way for you to feed your users exactly the settings you would
> like them to use. If you are interested, please email me, and I'll help
> you through the process.

Sounds interesting, I'll do that.

>> Also we dislike, but might cope with: - Migration wizard
> I'm doing some work on the Migration Wizard now. What sort of stuff
> could we do with it so that you would like it more?

I have to admit that I haven't had time to play a lot with it, so I
don't know exactly what can be already done. In a nutshell, I want an
option to prevent the wizard opening on users' screens when they run TB3
for the first time with a pre-existing TB2 profile.

I'd also like to pre-answer the questions that the wizard presents, in
behalf of the users. It's better that one admin spends a few hours
deciding the best settings that fit our environment, than all users
spending the same time with the wizard, straining the helpdesk, and then
some of them would still click on the wrong answers and we would have to
fix them afterwards.

>> - Smart folders view mode on by default
> I believe this has been turned off for 3.1

Nice!

Blake Winton

unread,
Mar 18, 2010, 11:47:49 AM3/18/10
to
On 10-03-18 9:39 , Jaakko Luoto wrote:

> Blake Winton wrote:
>>> Also we dislike, but might cope with: - Migration wizard
>> I'm doing some work on the Migration Wizard now. What sort of stuff
>> could we do with it so that you would like it more?
> I have to admit that I haven't had time to play a lot with it, so I
> don't know exactly what can be already done. In a nutshell, I want an
> option to prevent the wizard opening on users' screens when they run TB3
> for the first time with a pre-existing TB2 profile.

So, something like
http://hg.mozilla.org/users/bwinton_latte.ca/prefsui/rev/af0a6652cf09,
which I'm hoping to land as part of
https://bugzilla.mozilla.org/show_bug.cgi?id=545563 would work for you?

> I'd also like to pre-answer the questions that the wizard presents, in
> behalf of the users. It's better that one admin spends a few hours
> deciding the best settings that fit our environment, than all users
> spending the same time with the wizard, straining the helpdesk, and then
> some of them would still click on the wrong answers and we would have to
> fix them afterwards.

It should be possible to manually set the various prefs that the
autoconfig wizard sets.

Later,
Blake.

Timo Pietilä

unread,
Mar 19, 2010, 4:10:25 AM3/19/10
to
Blake Winton wrote:
> On 10-03-18 9:39 , Jaakko Luoto wrote:
>> Blake Winton wrote:
>>>> Also we dislike, but might cope with: - Migration wizard
>>> I'm doing some work on the Migration Wizard now. What sort of stuff
>>> could we do with it so that you would like it more?
>> I have to admit that I haven't had time to play a lot with it, so I
>> don't know exactly what can be already done. In a nutshell, I want an
>> option to prevent the wizard opening on users' screens when they run TB3
>> for the first time with a pre-existing TB2 profile.
>
> So, something like
> http://hg.mozilla.org/users/bwinton_latte.ca/prefsui/rev/af0a6652cf09,
> which I'm hoping to land as part of
> https://bugzilla.mozilla.org/show_bug.cgi?id=545563 would work for you?

If I read this correctly there is pref for migration wizard.

Wouldn't it then be enough to just simply set

lockPref("mailnews.ui.show.migration.on.upgrade", false);

in configuration script to prevent that migration wizard from opening?

Need to test that...

Timo Pietilä

Timo Pietilä

unread,
Mar 19, 2010, 4:49:27 AM3/19/10
to

No, didn't work.

Timo Pietilä

Blake Winton

unread,
Mar 19, 2010, 10:03:12 AM3/19/10
to

Well, bug 545563 hasn't landed yet, so I wouldn't expect it to work out
of the box.

Were you using a build from my prefsui branch for testing? Can you send
me the configuration script you used, so that I can also test it?

Thanks,
Blake.

Timo Pietilä

unread,
Mar 19, 2010, 10:23:57 AM3/19/10
to
Blake Winton wrote:

> Were you using a build from my prefsui branch for testing?

No. I'm not a developer so this is a bit new world to me. Where can I
get that build from prefsui branch? I would like to test it.

Timo Pietilä

Wayne Mery

unread,
Mar 19, 2010, 10:42:21 AM3/19/10
to

unless blake has a tryserver build, you'd need to build your own, and
integrate his patch from his branch into your build source.
https://developer.mozilla.org/En/Simple_Thunderbird_build

--
http://wiki.mozilla.org/Thunderbird:Testing
http://www.spreadthunderbird.com/aff/165/

Jaakko Luoto

unread,
Mar 19, 2010, 11:12:02 AM3/19/10
to
Blake Winton wrote:
> So, something like
> http://hg.mozilla.org/users/bwinton_latte.ca/prefsui/rev/af0a6652cf09,
> which I'm hoping to land as part of
> https://bugzilla.mozilla.org/show_bug.cgi?id=545563 would work for you?

That looks about right, yes.

BTW: are this patches being worked for the 3.1 branch, so that when TB
3.1 for Windows comes out, they would be included?

>> I'd also like to pre-answer the questions that the wizard presents, in
>> behalf of the users. It's better that one admin spends a few hours
>> deciding the best settings that fit our environment, than all users
>> spending the same time with the wizard, straining the helpdesk, and then
>> some of them would still click on the wrong answers and we would have to
>> fix them afterwards.
>
> It should be possible to manually set the various prefs that the
> autoconfig wizard sets.

OK. But is the somewhat ugly javascript-loop that was presented earlier
in this thread, a correct way to do it? Or do you suggest something else?

I would like to direct Thunderbird to *not* automatically set any new
folders for offline use during the migration. But on the other hand, I
also don't want to disable the caching of those folders, that the user
has manually enabled for offline use with TB2. Simply said: I don't want
any changes to the offline settings during migration.

Here's the javascript code that I mean. To me it looks like that this
would disable _all_ caching on every account, overwriting the offline
settings that a user may have made earlier:

/* This will parse the prefs.js and disable "offline_download" for each
IMAP account */

if (getPref("mail.accountmanager.accounts")) {

var listExistingAccounts = getPref("mail.accountmanager.accounts");
var arrayExistingAccounts = listExistingAccounts.split(',');

for (var i=0; i < arrayExistingAccounts.length; i++) {
var serverFromAccount = getPref("mail.account." +
arrayExistingAccounts[i] + ".server");
var configType = getPref("mail.server." + serverFromAccount +
".type");
if (configType == "imap") {
lockPref("mail.server." + serverFromAccount +
".offline_download", false);
}
}
}

Timo Pietilä

unread,
Mar 20, 2010, 6:57:32 AM3/20/10
to
Wayne Mery wrote:
> On 3/19/2010 10:23 AM, Timo Pietilä wrote:
>> Blake Winton wrote:
>>
>>> Were you using a build from my prefsui branch for testing?
>>
>> No. I'm not a developer so this is a bit new world to me. Where can I
>> get that build from prefsui branch? I would like to test it.

> unless blake has a tryserver build, you'd need to build your own, and

> integrate his patch from his branch into your build source.
> https://developer.mozilla.org/En/Simple_Thunderbird_build

Thanks. I have been contributing a bit in roguelike gaming, so these
tools are not entirely unknown to me, but I just lacked basic
information (like where to get the source). That link and sites that
link there seem to have everything I need.

Will do testing once I have a bit more time to set things up.

Timo Pietilä

0 new messages