Filtering rules

[Desiree]

Thunderbird thinks "automatically" is the same thing as "auto" when it
comes to filtering rules. I was forced to set up filtering rules for Fx
Enterprise list serve since there are a lot of members who don't know
correct netiquette for listserves. "Auto" is necessary as a subject
line filter so that I don't see all the idiot auto generated messages to
the list about being out of the office, being on vacation,
unsubscribing, etc. However, just now, TB automatically sent two
messages to the list that contained the word "automatically" in the
subject field to Trash when they should have been retained in the Inbox.
Why would it think "auto" and "automatically" are the same?

[Christian Riechers]

Possibly because you defined your filter as 'contains' auto, and not
'is' auto.

--
Christian

[VanguardLH]

While the idea is that you search on a word instead of substring, I
suspect most users would believe "is" means match on the entire string,
not on a word within the string.

Does "is" actually match on just a word (delimited by whitespace or
begin/end line) within the string, or does it match on the entire
string?

If "is" matches on the entire string, that operand wouldn't work for the
OP since I suspect the Subject has more than just the work "auto", like
it says "Auto - Meeting request". If "is" matches on the entire string
then the rule wouldn't match on that Subject.

Does Thunderbird support regex? If so, something like \bauto\b might
work where you're looking for the word "auto". If "auto" had to be at
the left end of the string then ^auto\b might work. If Tbird doesn't
yet support regular expressions, would an add-on work that does, like
https://addons.mozilla.org/en-US/thunderbird/addon/gmailui/?

[Mike Easter]

VanguardLH wrote:

> Does Thunderbird support regex?

Not natively. Nor wildcards.

http://kb.mozillazine.org/Filters_(Thunderbird) Thunderbird doesn't
have built-in support for using wild cards or regular expressions in
filters.

Filtaquilla does http://mesquilla.com/extensions/filtaquilla/
FiltaQuilla provides a variety of additional message filter actions and
search terms.


--
Mike Easter

[Wolf Kirchmeir]

Because "auto" is included in "automatically". If you wrote the rule,
use "is", and remember that spaces are also characters. Eg "auto ", "
auto", "auto", "au to", etc, are different.

If you marked the messages as Junk, the junk filter learned that "auto"
is a no-no. Mark acceptable messages as non-junk, that will (eventually)
reduce the false positives.

But you cannot eliminate false positives. Check the junk folder before
deleting its contents.

HTH

--
Wolf K.
kirkwood40.blogspot.ca

[EE]

You can set up more than one condition for a filter, right? Would
'contains' "auto" plus 'doesn't contain' "automatic" work?

[VanguardLH]

Maybe. The "All" specifier is probably an AND'ing operand.

Subject contains "auto"
AND
Subject does not contain "automatic"

For a Subject of "Auto - meeting request" you'd get: 1 AND 1 = 1 (true)
For a Subject of "Automatic request" you'd get: 1 AND 0 = 0 (false)
For a Subject of "Autoselect a choice" you'd get: 1 AND 1 = 1 (true)
For a Subject of "Semiauto pistol 4 sale" you'd get: 1 AND 1 = 1 (true)
For a Subject of "Remove the tautologies" you'd get: 1 AND 1 = 1 (true)

I doubt the OP wants to false alert on just any Subject that has "auto"
but not "automatic". There are a LOT of matches on "not 'automatic'".
Any Subject that had "auto" as part of a mashed up word having "auto"
would fire this rule.

I don't think Thunderbird searches on word boundaries. It looks for
substrings. So "auto" being at the end of a word or somewhere inside
would also trigger the first condition. It isn't looking for words that
begin with "auto". It's looking for "auto" anywhere in the Subject
header. That's why something like regex and using \bauto\b would find
the *word* "auto", and ^auto\b would find the *word* "auto" at the
beginning of a line so something like "Lights are auto timed" won't
match.

[Ralph Fox]

Try the following in Thunderbird:
(*) Match any of the following
Subject begins with "auto "
Subject contains " auto "
Subject ends with " auto"
(Removing the quote marks but leaving the spaces)

What this won't handle is when the word "auto" ends (or begins) with
punctuation that is not a space.

You are right that Thunderbird looks for substrings, whether or not they
are whole words.


--
Kind regards
Ralph

[Desiree]

No. I don't use filter rules for junk mail. I teach Thunderbird what is
junk and what is not junk. I have never told TB to mark messages with
"auto" in the subject as "junk".

This was a filtering rule for "auto" in the subject then the message was
to be sent to Trash. But I did not want "automatically", "automatic" in
the subject to also mean TB should send it to Trash.

The only thing I could see to try was to add another rule so that if
"automatically" or "automatic" is in the subject to "suspend filtering"
which should be what I want but who knows since TB does not check
individual words in the subject.

I put two messages that TB put in the trash because they had
"automatically" in the subject back in the Inbox. I put the filter to
"suspend filtering" if "automatically" and "automatic" were in the
subject as the first filter. As a test, I ran that filter on the inbox
and TB did not move those two messages to trash. But then when the
second filter "auto" in the subject was run manually, TB moved those two
messages to trash.

I can't see a way to tell TB that if a Subject contains "auto" but NOT
"automatically" or "automatic" to send the message to trash. In Outlook
Express (which I used for 14 years until I got Windows 8 Pro computer a
year ago) I can filter via one word, several words, or a phrase in the
Subject. I had assumed TB was the same.

[VanguardLH]

Ralph Fox wrote:

> Try the following in Thunderbird:
> (*) Match any of the following
> Subject begins with "auto "
> Subject contains " auto "
> Subject ends with " auto"
> (Removing the quote marks but leaving the spaces)
>
> What this won't handle is when the word "auto" ends (or begins) with
> punctuation that is not a space.

Although not provided with an example of a Subject string that Desiree
would expect or requires for proper filtering, my guess is the 1st
example you gave would probably suffice. She is looking for a Subject
beginning with the word "Auto", a space, and then something more after
that. If the expected Subject were "Auto - <topic>" then test on
Subject begins with 'auto - '. If a prefix is required for filtering
then a prefix must be used that can be uniquely identified, so the
prefix really should be something that is unlikely to be used, like
"::[auto]:: ".

The problem appears is that the Subject cannot be enforced to contain a
particular string starting with an identifiable and rather unique
prefix. From Desiree's description, she is filtering out strings used
by auto-responders some of which may be canned bounceback messages but
others could be what the recipient choose for their message. To avoid
looping due to sending a message, bouncing it back, the other end
bouncing it again with an auto-responder, you bouncing it back, them
bouncing it again, and so forth to consume gigabytes of worthless
bounceback messages and choke bandwidth, it has long been customary to
add a header to an auto-responder that tells the recipient of an
automated message to NOT respond to it.

Rather than filter out on some substring you hope is within the Subject
header instead look for the following header in any e-mail you receive:

Reply-To:

The space delimiter character (headers are supposed to be <name>, colon,
and space delimiter) followed by a blank value is what tells you that
you should NOT reply to this message. That is the RFC standardized
method for handling auto-replies to prevent looping. I think MS
Exchange might also add:

x-auto-response-suppress:all

I don't operate a listserver but I suspect it sends out a message, the
recipient has an auto-responder enabled saying they're temporarily
unavailable which goes back to the listserver, and Desiree gets back
these worthless automated replies which should've been handled by a rule
in her listserver to simply drop (delete) any incoming messages with the
above header in them.

If the list member's auto-responder doesn't add an appropriate header to
identifier it is an automated reply then send a warning to that member
about their defective auto-responder and that you will no longer accept
any messages from them until they (or their e-mail provider) fix their
auto-responder. Inspecting the headers of the auto-replies that Desiree
already gets would reveal what headers the auto-responders are using to
identity the message is an auto-reply.

It isn't just "out of office" or vacation auto-replies that are a
problem. A lot of boobs that don't understand how SMTP works employ a
client (not a server) to bounceback a challenge. That is, they employ a
challenge-response scheme for filtering out unwanted e-mails (typically
spam). Since the client is issuing the challenge, it has no clue who is
the true sender of an e-mail. SMTP doesn't guarantee the client has any
correct info identifying the sender. The client doesn't get to see the
envelope during the mail session between SMTP servers (where the
receiving SMTP server would always know who was the sending SMTP
server). The client only gets to see what the sender put in their
message for headers they claim identify the sender. These headers get
sent during the DATA command. So recipients using client-side C-R
filtering end up sending challenges to innocents who the spammer claimed
to be. When I get such challenges, I correct report this willfully
(although ignorantly) misdirected backscatter as spam. The user of a
C-R scheme employed by their client is spamming me with their challenges
for e-mails that I never sent to them because the spammer claimed they
were me in the headers the *spammer* gets to insert in their message.
If that user's C-R system inserts the proper header to indicate it is an
automated reply then I won't report the spam because I can filter those.

Recommendations for Automatic Responses to Electronic Mail
http://www.rfc-editor.org/rfc/rfc3834.txt
3.1.2. Reply-To field
"Note that since (according to the previous section) the From field of
the response SHOULD contain the address of a human, if the Reply-To
field of the response is used to direct replies to a responder it will
not be thesame as the address in the From field."

So the Reply-To: header should be present and must not be the same value
as the From: header. Normal e-mails either have no Reply-To: header
because the sender wants replies sent to the From: address or they
contain a *non-blank* Reply-To: header of where replies are to be sent
elsewhere. A blank Reply-To: header should not be inserted by a client
because a blank value is not a valid target address but it should be
inserted by a server for an auto-reply to denote that the recipient of
the message should NOT REPLY to that message. It seems an easy way to
detect the message originated from an auto-responder instead of a human.

If present, a "Reply-To: " header (that string must start in column 1 of
a line) with a blank value tells the recipient (their server or the
recipient) that replies go ... wait for it ... nowhere. That means
Desiree's listserver program should not reply to those e-mails. Because
this is a listserver instead of an e-mail server also means Desiree's
listserver should simply drop those incoming automated replies. Could
be the member's auto-responder is broken (didn't add a Reply-To header
with a blank value). Could be Desiree's listserver program is broken
(doesn't ignore e-mails that are automated replies).

In the same RFC:
3.1.5. Subject field
"The prefix "Auto:" MAY be used as such an indication."

"MAY" means not required and, in practice, often ignored. The Subject
header from auto-responders is too variable to trust you can use it to
identify e-mails originated by auto-responders. Since Desiree mentioned
searching on "auto", those auto-responders should be prefixing the
Subject with "Auto: ", not with "Auto" or "Auto ". So if Desiree was
filtering on "Auto: " or even on "Auto:" then she would never be
matching on "Automatic".

In addition:
"However the "Auto:" indication is intended only as an aid to humans in
processing the message. Mail processing software SHOULD NOT assume that
the presence of "Auto:" at the beginning of a Subject field is an
indication that the message was automatically submitted."

It is incorrect to rely on the Subject header to identify a message was
sent by an auto-responder. My recollection was the auto-responder is
supposed to add a Reply-To header with a blank value. This tells the
recipient that there is nowhere the recipient should send a reply.
Well, if this identifies an e-mail is an automated reply then Desiree's
listserver should be discarding that e-mail.

[Mark Filipak]

On 2014/2/2 7:42 AM, Desiree wrote:
> On 2/1/2014 7:17 AM, Wolf Kirchmeir wrote:
>> On 2014-02-01 5:27 AM, Desiree wrote:

-snip-
-snip-
-snip-

> This was a filtering rule for "auto" in the subject then the message was
> to be sent to Trash. But I did not want "automatically", "automatic" in
> the subject to also mean TB should send it to Trash.

-snip-

(*)Match any of the following

[Subject] [contains] [ auto ]
[Subject] [begins with] [auto ]
[Subject] [ends with] [ auto]