Evolution email (problem?)

[Default User]

Strange . . .

I run Debian 11 (Bullseye) Stable, up to date, Gnome 3 desktop
environment.

I recently set up Evolution email. Works okay.

Two days ago, I realized that it absolutely refuses to delete email
messages directly from the [Gmail]/All Mail folder of any email
account. To delete a message in the All Mail folder, that is not also
in another folder, the message must be moved into another folder and
then deleted from there.

I sure think I remember being able to delete directly from the All Mail
folder when I first set up Evolution.

I re-started Evolution. No change.
I re-booted. No change.
I did sudo aptitude reinstall evolution. No Change.
I moved ~/.local/share/evolution/mail/local/folders.db to Trash, then
re-started Evolution to construct a new folders.db file. No change.
I did sudo aptitude purge evolution, then sudo aptitude install
Evolution. No change.
I moved ~/.local/share/evolution, ~/.cache/evolution, and
~/.config/evolution to Trash, then did sudo aptitude purge evolution,
then sudo aptitude install Evolution. No change.
I did some research online, seeking a solution, or at least a "good"
explanation. No solution or "good" answer was found.

But . . . then I booted into a Debian 11.6 Live/Install usb thumb
drive, and then installed Evolution into the Live session, upon which
Evolution DID allow message deletion directly from the [Gmail]/All Mail
folder!

Back to my regular Debian 11 Stable install. No change.

It makes no sense to me that it would work from a Debian 11
Live/Install session, but not (currently) from a regular Debian
session.

Even if it did not work from a Live Debian session, it makes no sense
to me that Evolution could be designed this way. I see no good reason
that messages can be directly deleted from any other folder, but not
[Gmail]/All Mail.

Does anyone have a solution, so that messages must be moved to another
folder (if they are not also there already), just to be deleted? Or at
least a "good" explanation as to why Evolution appears to be designed
this way?

[Patrick Wiseman]

Can't explain it, but it strikes me it's almost certainly a permissions problem.

Patrick

[Default User]

Hi, Patrick.

I don't see anything obviously wrong about the permissions on my
computer.  Not surprising, since I rarely change any permissions, and
can't recall doing so where it might cause the condition in question.

But I don't know what exactly to look for.
Perhaps a directory that has extra or inappropriate read-only
permission(s)?

And I don't think that any permissions on my computer are involved,
since the email folders in question are not on my local computer, they
are on one or more Google server(s) elsewhere. I am just accessing
them remotely, they are not downloaded to my computer.

Possibly interesting note: The [Gmail]/Important and [Gmail]/All Mail
folders both have listed in "folder properties"/"Labels" the Server Tag
"$Phishing". No other folders do.

But again, messages can be directly deleted from [Gmail]/Important
folders, but not from [Gmail]/All Mail folders. So it probably doesn't
mean anything in the current context.

Well, anyway, do other Debian Stable Evolution users experience this
problem?

Or is it really "just me"?

[Max Nikulin]

On 26/04/2023 22:05, Default User wrote:
> it absolutely refuses to delete email
> messages directly from the [Gmail]/All Mail folder of any email
> account.

Disclaimer: I do not use evolution.

Do you use IMAP for your gmail account? Notice that IMAP assumes
tree-like structure of folders and messages. Gmail uses concept of
labels, so the same message may have multiple labels. When mapped to
IMAP it means the same message (not a copy) may be accessed from
different folders. Moreover All Mail is a meta label/folder that
includes every message (besides trash?). I am unsure if it can be
reasonably handled using conventional IMAP.

So gmail IMAP accounts may be special to some degree for mail clients.
Perhaps some kind of cooperation between a mailer and the server is
necessary. I am unsure at which moment a message should completely
disappear from the server in response to removing it from particular
folders (actually labels).

[Default User]

Hi, Max.

Evolution does indeed use IMAP. Unfortunately, I don't know much about
email protocols.

I am aware that Gmail uses the concept of labels, and that "Inbox",
"Sent Mail", "All Mail", etc. are just labels assigned to email
messages, with "All Mail" apparently assigned to all messages (except
those with the "Trash" label).

Anyway, it would be interesting to hear from Evolution users, to see if
others experience the situation I have described. I would find it hard
to believe that no one else on debia...@lists.debian.org uses
Gnome/Evolution on Debian Stable!

[Michel Verdier]

Le 27 avril 2023 Default User a écrit :

> Anyway, it would be interesting to hear from Evolution users, to see if
> others experience the situation I have described. I would find it hard
> to believe that no one else on debia...@lists.debian.org uses
> Gnome/Evolution on Debian Stable!

I don't use evolution and I use pop3 to fetch mails from gmail. I have
problems deleting mails from lists : duplicate mails are not included in
fetch and have to be deleted directly on gmail. So I think your problem
is not on evolution or imap, it's only on gmail...

[Max Nikulin]

On 27/04/2023 21:38, Default User wrote:
> "All Mail" apparently assigned to all messages

Do you expect that deleting from "All Mail" removes message from another
folder to which you moved it earlier? I am in doubts it should be done
without a dialog to confirm such operation. However I just realized that
thunderbird does it quietly.

My impression is that deleting message from Inbox removes it from "All
mail" (thunderbird again), however moving message from Inbox to a folder
from another account removes it from Inbox, but not from "All Mail". I
admit that such peculiarities are unavoidable due to tree vs. labels
concepts difference.

[Default User]

Hi, Max.

If a message is in both of the folders "Inbox" and "All Mail", it CAN
be directly deleted from "Inbox", which removes it from both folders,
moving it to the "Trash" folder, from which it can be "expunged"
("Trash" is then "emptied").

If a message is in both of the folders "Inbox" and "All Mail", it CAN
NOT be directly deleted from "All Mail". It stays in both folders, with
only the one in "All Mail" having a line through it, showing that is
marked for deletion, but is not deleted, and is not even moved to
"Trash", which remains empty.

Moving the message from "Inbox" to another folder in the same email
account, for example "Spam", DOES remove the message from "All Mail".
That does not make sense to me!

I am reluctant to test moving a message from one email account to
another one, as I do not do not want to have an another unexpected
problem occur, and I do want to keep the email accounts strictly
isolated from each other.

I do wish that "deleting" a message from any folder would remove it
from all folders. But, of course, a confirmation dialog before doing
any deletion or expungement would seem to be virtually mandatory!

[Tixy]

On Wed, 2023-04-26 at 11:05 -0400, Default User wrote:
> Even if it did not work from a Live Debian session, it makes no sense
> to me that Evolution could be designed this way.  I see no good reason
> that messages can be directly deleted from any other folder, but not
> [Gmail]/All Mail. 
>
> Does anyone have a solution, so that messages must be moved to another
> folder (if they are not also there already), just to be deleted?  Or at
> least a "good" explanation as to why Evolution appears to be designed
> this way? 

I'd be inclined to blame Gmail's quirks.

Personally, I've used Evolution since Debain 5 without issue, though
not with Gmail though. Well, except on a work laptop for a few months
until I realised it didn't behave properly (like hiding your own mails
sent to lists) so I just set up a rule for gmail to forward all mail to
proper IMAP sever (Dovecot) run by me and used that for work email.

-- 
Tixy

[Tixy]

On Thu, 2023-04-27 at 12:15 -0400, Default User wrote:
> If a message is in both of the folders "Inbox" and "All Mail", it CAN
> NOT be directly deleted from "All Mail". It stays in both folders, with
> only the one in "All Mail" having a line through it, showing that is
> marked for deletion, but is not deleted, and is not even moved to
> "Trash", which remains empty. 

Well, one email can't be _in_ two folders. You can have two emails, one
in each folder, and deleting one shouldn't affect the other. Except
you're using Google's Gmail where 'folders' aren't really folders, just
tags applied in a database. Who knows how it's going to behave when
they Google maps that onto the IMAP protocol.

--
Tixy

[Default User]

Hi, Tixy.

Yes, I think there is never more than one actual message, there is just
one message with one or more "labels" of virtual folders attached to
it, until it is really deleted from Gmail's database.

[Default User]

Hi, Michel.

Yes, it could well be. Gmail is Google, after all.
As actress/comedian Lilly Tomlin used to say on the old American
television show "Laugh In":

"We don't care. We don't have to. We're the phone company!"

[Default User]

Hi, Tixy:

It is unfortunately a well-known "feature" of Gmail: if you send an
email message, it does not show up in your "Inbox". You have to look
in your "Sent Mail" folder to see it.

But hey, "It's not a bug - it's a feature!"
Right . . .

[Max Nikulin]

For those who missed start of the thread: it is dedicated to IMAP access
to Gmail and Evolution behavior.

On 27/04/2023 23:15, Default User wrote:
> It stays in both folders, with
> only the one in "All Mail" having a line through it, showing that is
> marked for deletion, but is not deleted, and is not even moved to
> "Trash", which remains empty.

What will happen if you try to "compact" the "All Mail" folder? I am
unsure concerning precise name of such action in evolution. In
thunderbird delete vs. mark for deletion setting is account-wide and can
not be chosen per folder. However even when "delete" is chosen, messages
may be retained in folders till they are automatically compacted or this
action is forced by user.

> Moving the message from "Inbox" to another folder in the same email
> account, for example "Spam", DOES remove the message from "All Mail".
> That does not make sense to me!

It is better to experiment with a custom label. Spam or Trash are rather
special cases (perhaps on both sides: client and server).

> I am reluctant to test moving a message from one email account to
> another one, as I do not do not want to have an another unexpected
> problem occur, and I do want to keep the email accounts strictly
> isolated from each other.

Almost certainly messages may be deleted from web UI. Anyway a couple of
test messages should not cause real mess.

[Default User]

Hi, Max.

I can not seem to find any information about compacting folders in
Evolution. Which is strange. In Thunderbird, you can manually compact
folders any time, and it will compact automatically by default, with
adjustable parameters.

Perhaps Evolution considers "expunging" (actual deletion) of messages
in the Trash folder, or otherwise marked for deletion (except in the
All Mail folder), as "compacting". I sure hope that Evolution is
actually compacting folders somehow, and just not bothering to explain
when or how.

Anyway, I think I have spent enough time on the Evolution All Mail
folder issue. For now, I will just add one or more "Saved" folders,
and pretty much just ignore the All Mail folder, unless for some reason
I need to move messages from there to other folders.

So, thanks to all who weighed in on the issue.

(BTW, if anyone does have information about compacting folders in
Evolution, I would love to hear about it!)

[Tixy]

On Sun, 2023-04-30 at 11:19 -0400, Default User wrote:
> (BTW, if anyone does have information about compacting folders in
> Evolution, I would love to hear about it!)

What is 'compacting', what is it meant to do? Sounds like 'compressing'
to me, don't know if protocols like IMAP has a way to ask the server to
compress it's folders. (I would have though how emails are actually
stored would be an internal implementation detail of the server).

If it just means really deleting email, then you can select the menu
option File > Empty Wastebasket which does that for all accounts. Or
for just a single account, right click on it's Wastebasket. There is
also Folder > Expunge which I believe does it for a single folder.

You can also configure Evolution to empty the wastebasket after a
period of time, or as I have it configured, whenever you close
Evolution. (I've always hated the idea of Wastebaskets, if I delete
something I want it deleted, not just hidden and taking up space).

--
Tixy

[to...@tuxteam.de]

On Sun, Apr 30, 2023 at 06:26:53PM +0100, Tixy wrote:
> On Sun, 2023-04-30 at 11:19 -0400, Default User wrote:
> > (BTW, if anyone does have information about compacting folders in
> > Evolution, I would love to hear about it!)
>

> What is 'compacting', what is it meant to do? [...]

> If it just means really deleting email, then you can select the menu
> option File > Empty Wastebasket which does that for all accounts. Or
> for just a single account, right click on it's Wastebasket. There is
> also Folder > Expunge which I believe does it for a single folder.

This is probably the intended meaning. For strange historical reasons
(mbox), reclaiming the space of a deleted mail used to be a non trivial
operation, so the user interface separated both: marking single mails
for deletion and actually doing it (which went by different names,
like "expunge", possibly "compact" -- the latter reminiscent of crushing
the holes left in a linear mbox).

That user interface proved useful for other reasons (time for remorse),
so it stuck, probably.

So yes, wastebasket emptying is probably it.

Cheers
--
t

[Default User]

Hi Tixy.

I don't know technically what "compacting" is supposed to mean, but
just as a total guess, I wonder if it is something like the 'vacuuming"
operation for databases.

According to this:
https://support.mozilla.org/en-US/kb/compacting-folders
compacting does not:
- delete messages
- remove messages
- compress folders

Note that this just says that compacting improves performance. But I
believe I have read elsewhere that not compacting regularly can have
potentially dire consequences, such as corrupted email storage files
and lost email messages.

Also consider this:
https://thunderbirdtweaks.blogspot.com/2011/07/compacting-what-is-it-and-why-must-i-do.html

So "expunging" emails may not be the same thing as compacting. FWIW,
in Evolution I had multiple email accounts with more than 1,000
messages each, including one which had over 24,000 messages! Spending
many days, deleting sometimes several hundred messages at a time, I
have now reduced that to less than 6,000. No lost messages or file
corruption yet . . .

It does occur to me that Evolution may use the maildir format rather
than the mbox format. I just ASSumed that it used mbox, since in Menu
> File, there is an option to save messages in mbox format. If
Evolution uses the maildir format, compacting apparently does not
apply, which would seem to explain it. I have not yet determined
whether Evolution uses mbox or maildir.

BTW, I also never did really like the wastebasket (recycle bin)
concept, in any application program. After all, delete should mean
delete, right? But I must confess that the wastebasket idea has
rescued me more than once from a deletion by mistake, or when I have
"changed my mind"!

[Stefan Monnier]

> According to this:
> https://support.mozilla.org/en-US/kb/compacting-folders

This basically explains that compacting is an operation that should
fundamentally be transparent to the user, and that Thunderbird makes the
user aware of it for ... no good reason.

Thunderbird will compact things for you as needed even if you don't
explicitly ask for it, so you don't need to worry about it.

Evolution doesn't bother the user with this, which is the better option.


Stefan

[Default User]

Hi, Tomas.

Okay, it looks like Evolution does indeed use the MAILDIR format. See:

https://help.gnome.org/users/evolution/stable/import-supported-file-formats.html.en

which says, 

"The format used by Evolution (for local folders since version 3.0).
There is no need to import Maildir files as you can configure a Maildir
account in Evolution and point to the folder where the Maildir files
are stored."

And according to:

https://support.mozilla.org/en-US/kb/compacting-folders


MBOX is the default format, where all of a folder's messages are stored
in a single file on disk. This is where the compact process is useful,
and the purpose of this article is to explain how and why.  

Maildir is a newer storage format, where every message of a folder is a
separate file. Maildir does not need compact, and so this article is
not applicable to Maildir folders.

So, apparently Evolution does not need "compacting"!

BTW, interesting that Thunderbird seems to still use the mbox format by
default, although you can apparently choose to store messages in either
mbox or maildir format. Which is why they explain and offer the choice
to users.

[Max Nikulin]

On 01/05/2023 05:36, Stefan Monnier wrote:
>> According to this:
>> https://support.mozilla.org/en-US/kb/compacting-folders
>
> This basically explains that compacting is an operation that should
> fundamentally be transparent to the user, and that Thunderbird makes the
> user aware of it for ... no good reason.

This support page discusses behavior for local folders. This thread is
dedicated to specific of Gmail and Evolution interaction in respect to
IMAP protocols. I mentioned compacting of folders in Thnderbird because
earlier I have seen that performing "Compact" in Thunderbird expunges
messages earlier marked for deletion. Strike through in message list may
be a sign that the message is marked for deletion.

I expect that the following applies to message state on server, not to
local folders:
https://datatracker.ietf.org/doc/html/rfc3501#section-2.3.2
RFC 3501 IMAPv4
2.3.2. Flags Message Attribute

> \Deleted
> Message is "deleted" for removal by later EXPUNGE

It is independent of maildir or mbox storage format of local folders or
local offline cache of remote folder. Gmail may use internally some
proprietary format that is neither mbox nor maildir, e.g. records in
some database.

I like that Thunderbird exposes the "compact" option to UI because I
earlier used it to forced removal of large messages from server account
with rather strict quota. Several times I used "undelete" to recover
messages I deleted by mistake, so I like that messages just marked for
deletion by default as well.

The following is related neither to Evolution (at least directly) nor to
Thunderbird, but still might be relevant:
https://support.microsoft.com/en-us/office/restore-deleted-messages-when-using-an-imap-account-4559a297-4d46-47e7-bfbf-71287b1935ed

> If you use Google Gmail as an IMAP account, messages marked for deletion
> are permanently deleted from their original folder but may still appear
> there in strikethrough text. These messages are still recoverable
> because a copy is kept in the Gmail Trash folder.

On 01/05/2023 05:24, Default User wrote:
> compacting does not:
> - delete messages
> - remove messages

These statements may be confusing in the context of this thread. It is
true that the action does not deletes messages if the user earlier has
not requested removal. However in some cases it possible to undelete
message before compacting, but not after. So it really deletes messages,
perhaps hidden from UI (or displayed as strike through and still available).

[Tixy]

On Sun, 2023-04-30 at 18:24 -0400, Default User wrote:
> It does occur to me that Evolution may use the maildir format rather
> than the mbox format.  I just ASSumed that it used mbox, since in Menu
> > File, there is an option to save messages in mbox format.  If
> Evolution uses the maildir format, compacting apparently does not
> apply, which would seem to explain it.  I have not yet determined
> whether Evolution uses mbox or maildir. 

I thought we were talking about IMAP protocol access to Google? In
which case Evolution isn't storing email in any format [1], it's
accessing it on a remote server. Evolution does have an 'On This
Computer' email account, don't know what format it uses for that. In
Evolution you can also create email accounts where email is stored
locally in maildir or mbox format. I used maildir some years ago to
have an email archive of some mailing lists.

[1] Well, Evolution does cache email in ~/.cache/evolution/mail
which seems to involve SQLite database and a file for each email. If
you want to free that space you can just 'rm -rf ~/.cache/evolution'
but I guess if you have 'download for offline use' configured or
actually look at the remote email folders again it will re-download
them.

--
Tixy

[to...@tuxteam.de]

On Mon, May 01, 2023 at 07:38:45AM +0100, Tixy wrote:
> On Sun, 2023-04-30 at 18:24 -0400, Default User wrote:
> > It does occur to me that Evolution may use the maildir format rather

> > than the mbox format [...]

> I thought we were talking about IMAP protocol access to Google? In
> which case Evolution isn't storing email in any format [1], it's

> accessing it on a remote server [...]

Oh, goody. I only brought mbox up as an attempt to explain how such
a kind of unser interface (mark for deletion, then bulk expunge/
compact/whatyoucallit) might have developed.

After that, users might expect that kind of interface regardless of
the actual storage backend, so many mailers offer it.

Cheers
--
t

[rhkr...@gmail.com]

On Sunday, April 30, 2023 06:24:14 PM Default User wrote:
> > What is 'compacting', what is it meant to do?

The definition of compacting as I "grew up" with it (not sure where I first
encountered it is the idea that in some applications, the act of "deleting"
something doesn't actually delete it from the file, instead it is marked for
deletion (and no longer visible to the user).

Compacting actually deletes that thing, and shrinks the file to avoid any
wasted (empty") space previously used by those deleted things.

I suspect I first encountered that back in the early days of DOS based
databases, even prior to dBase II.

--
rhk

(sig revised 20230312 -- modified first paragraph, some other irrelevant
wordsmithing)

| No entity has permission to use this email to train an AI.

If you reply: snip, snip, and snip again; leave attributions; avoid HTML;
avoid top posting; and keep it "on list". (Oxford comma (and semi-colon)
included at no charge.) If you revise the topic, change the Subject: line.
If you change the topic, start a new thread.

Writing is often meant for others to read and understand (legal documents
excepted?) -- make it easier for your reader by various means, including
liberal use of whitespace (short paragraphs, separated by whitespace / blank
lines) and minimal use of (obscure?) jargon, abbreviations, acronyms, and
references.

If someone has already responded to a question, decide whether any response
you add will be helpful or not ...

A picture is worth a thousand words. A video (or "audio"): not so much --
divide by 10 for each minute of video (or audio) or create a transcript and
edit it to 10% of the original.

A speaker who uses ahhs, ums, or such may have a real physical or mental
disability, or may be showing disrespect for his listeners by not properly
preparing in advance and thinking before speaking. (That speaker might have
been "trained" to do this by being interrupted often if he pauses.) (Remember
Cicero who did not have enough time to write a short missive.)

A radio (or TV) station which broadcasts speakers with high pitched voices (or
very low pitched / gravelly voices) (which older people might not be able to
hear properly) disrespects its listeners. Likewise if it broadcasts
extraneous or disturbing sounds (like gunfire or crying), or broadcasts
speakers using their native language (with or without an overdubbed
translation).

A person who writes a sig this long probably has issues and disrespects (and
offends) a large number of readers. ;-)
'

[rhkr...@gmail.com]

Oh, I meant to add that compacting is typically useful when record (email?)
storage is in something like an mbox file -- it saves the need to rewrite the
file each time a single file is deleted (for example).

On the other hand, with storage in something like mdirr files (right name --
one email / record per file) it is not really relevant (unless those records /
emails are indexed by some other indexing system that stores pointers to all
of those records).

In (older versions?) of kmail, with mbox storage, compactiing is necessary to
reclaim unused space after deletions, and there is an option in the context
menu for (most) kmail folders to initiate compaction.

On Monday, May 01, 2023 08:45:15 AM rhkr...@gmail.com wrote:
> On Sunday, April 30, 2023 06:24:14 PM Default User wrote:
> > > What is 'compacting', what is it meant to do?
>
> The definition of compacting as I "grew up" with it (not sure where I first
> encountered it is the idea that in some applications, the act of "deleting"
> something doesn't actually delete it from the file, instead it is marked
> for deletion (and no longer visible to the user).
>
> Compacting actually deletes that thing, and shrinks the file to avoid any
> wasted (empty") space previously used by those deleted things.
>
> I suspect I first encountered that back in the early days of DOS based
> databases, even prior to dBase II.

--
rhk