Anti-spam/phish/junk feature for Thunderbird

[Patrick Klos]

Hello Thunderbird Developers:

I have a anti-spam/phish/junk feature I'd like to enlist someone's help
to add to Thunderbird. I can handle some of the "grunt work", but I
know nothing about GUIs or the structure of Thunderbird as a whole.

Here is my idea:

In the displayed message header, I would like to add the images of flags
(or some other designation) of all the countries through which the email
has passed based on the "Received:" lines in the header.



All the "Received:" lines will be parsed for their IP addresses, and the
location (country code) for that IP address will be determined by
mapping IP addresses to country codes (or some similar technology).

Return-Path: <fbidirec...@fbi.gov>
Received: from aprdist.goapr.asia (aprdist.goapr.asia [124.217.244.83])
by sage.klos.com (8.14.6/8.14.4) with ESMTP id r3UAkwT0020097
for <postm...@xyzzy.com>; Tue, 30 Apr 2013 10:46:59 GMT
Message-Id: <201304301046....@sage.klos.com>
Received: from User (unknown [83.137.98.93])
by aprdist.goapr.asia (Postfix) with ESMTPA id 6DD8F39D5A58;
Tue, 30 Apr 2013 17:20:19 +0800 (MYT)
Reply-To: <fbidirec...@superposta.com>
From: "F.B.I WASHINGTON D.C"<fbidirec...@fbi.gov>
Subject: F.B.I-SECOND REMINDER NOTICE
Date: Tue, 30 Apr 2013 02:20:57 -0700


The displayed header will include a country's flag image for each
different country that the message appears to have been sent through.
In the case of the above headers, the flags will indicate the countries
of Germany and Malaysia. (I'm pretty sure the F.B.I doesn't send emails
through this path!) This will give the user an immediate visual
indication of the number of countries a message has traversed through,
as well as their flags. At a glance, users will know if an email from
PayPal or Amazon actually came from outside the country. It won't
guarantee that emails are legitimate, but it will probably make it
easier for the average user to determine if an email is likely fake
(phish/spam).

Parsing the "Received:" headers could also be used to filter/flag new
messages if, for example, someone knows they will never receive an email
from North Korea.

I don't know if there's enough infrastructure within Thunderbird so this
could be an optional plug-in or if this would need to be built into
Thunderbird proper?

What do you think? Is there someone who would like to handle the GUI
portion of these feature(s) with my help?

Thanks,

Patrick

[Patrick Klos]

Since the list apparently doesn't accept images in the emails posted to
the list, I've created a PDF file of the complete message as originally
composed:


http://www.klos.com/~patrick/Anti-spam-phish-junkFeatureForThunderbird.pdf

I think the graphic helps get my point across?

Thanks,

Patrick

[Josiah]

Hey Patrick,

This is actually a very fascinating idea. I personally think it could be
accomplished and would be pleased if it was. As long as:

- The parsing does not slow down the receiving process much/at all. Now,
this doesn't sound like it would cause much slow down, but suppose
someone sets up to pull every hour, and on that hour they receive a
hundred emails, will trying to detect countries lag that process. I
doubt it, but it must be considered.

- You will be willing to take most of the back-end work. There may be a
few free developers available, but most of the time everyone is very
busy, so as long as you are willing to do most of the heavy-lifting, be
my guest.

I really like the idea of being able to filter countries, and allowing
users to have a visual cue to where the email has been sent from. Now,
is it possible. I would say yes it is, but someone else should really
verify this other than me.

As for the GUI. I would be glad to help with that if needed, and have
created a sketchup of how this could be used.

https://sites.google.com/site/josiahsbruner/files/FlagIdea.png
<https://sites.google.com/site/josiahsbruner/files/FlagIdea.png>

The idea would be to add another column to the message list, and allow
little, lightweight images to show up there. Also, we can have a little
flag button in the message viewer (here I designated it as an "F"), and
when that button it toggled, show up to three flags.

(This is just a really quick sketch, and the design could change later.
E.G. Having a bunch of flags in the mail list causes scrolling lag, then
the idea will need to be changed.)

Anyway, for now I would suggest filing a bug on Bugzilla, and asking
more about this idea on irc.mozilla.org -> #maildev.

You will need someone who knows more about what TB is capable of to
approve of this.

All the best,
--
Josiah Bruner
Volunteer,
Mozilla, Front-end

> _______________________________________________
> dev-apps-thunderbird mailing list
> dev-apps-t...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-apps-thunderbird

[Patrick Klos]

Josiah wrote:
> This is actually a very fascinating idea. I personally think it could
> be accomplished and would be pleased if it was. As long as:
>
> - The parsing does not slow down the receiving process much/at all.
> Now, this doesn't sound like it would cause much slow down, but
> suppose someone sets up to pull every hour, and on that hour they
> receive a hundred emails, will trying to detect countries lag that
> process. I doubt it, but it must be considered.

Considering everything else that goes on when email is loaded, including
the fact that the header has to be parsed anyway to get the
To:/From:/Date:/Subject:/etc, I'm sure the time to parse "Received:"
lines would be negligible.

> - You will be willing to take most of the back-end work. There may be
> a few free developers available, but most of the time everyone is very
> busy, so as long as you are willing to do most of the heavy-lifting,
> be my guest.

I don't know anything about the front-end/UI. When a message is
downloaded or displayed, I'll be happy to provide a routine that is fed
the email headers, and that routine will extract the information needed
and return a list of country codes. There are already public domain
images of flags (small to medium size) that can be used for display.

> I really like the idea of being able to filter countries, and allowing
> users to have a visual cue to where the email has been sent from. Now,
> is it possible. I would say yes it is, but someone else should really
> verify this other than me.

I know it's possible - I just can't do it (all) myself.

> As for the GUI. I would be glad to help with that if needed, and have
> created a sketchup of how this could be used.
>
> https://sites.google.com/site/josiahsbruner/files/FlagIdea.png

Looks good, but I'm envisioning the flag images being placed
side-by-side in the header (and wrapping if necessary) similar to how
the To: line is displayed.

> The idea would be to add another column to the message list, and allow
> little, lightweight images to show up there. Also, we can have a
> little flag button in the message viewer (here I designated it as an
> "F"), and when that button it toggled, show up to three flags.

I like adding the "flags" column to the summary pane.

> (This is just a really quick sketch, and the design could change
> later. E.G. Having a bunch of flags in the mail list causes scrolling
> lag, then the idea will need to be changed.)

If the headers have so many flags that it causes "scrolling lag", then
it's probably not a legitimate email in the first place, so performance
considerations should be moot.

> Anyway, for now I would suggest filing a bug on Bugzilla, and asking
> more about this idea on irc.mozilla.org -> #maildev.

Thanks, I'll look into that.

Patrick

[Josiah]

/Looks good, but I'm envisioning the flag images being placed

side-by-side in the header (and wrapping if necessary) similar to how

the To: line is displayed./

I would be against this for a few reasons.

A. If placed next to one of the existing displays (Probably the "From"),
then the line would be very clutter, and little flag icons aren't very
nice looking. Simply esthetically, it makes sense to be able to easily
hide the flags, especially if we show them in the message list.

B. If you meant to have another line (From, Subject, To, Locations),
then again, that would take up quite a bit of space, and could clutter
things up again.

Now, if you want more flags to show, I would not be against expanding
the width of the flag button. So that when toggled it brings a pulldown
view that could look like:

---------
[--] [--]
[--] [--]
[--] [--]
---------

So perhaps six flags could show. You see, I personally wouldn't want a
bunch of flags to display in that header every time.

Anyway, definitely file a bug, this seems like a swell idea. The UI can
be decided on later.

--
Josiah Bruner
Volunteer, Mozilla,
Front-end

[smo...@gmail.com]

Flagfox for Thunderbird? I love it! I want it! I also know nothing of Thunderbird internals. Part of this is, I have found basically nothing of Thunderbird add-on tutorials on the web. Perhaps I could implement it as a browser extension for web-based email and at least some of the code would transfer to what you're describing.

[Joshua Cranmer 🐧]

On 4/30/2013 7:28 AM, Patrick Klos wrote:
> Hello Thunderbird Developers:
>
> I have a anti-spam/phish/junk feature I'd like to enlist someone's
> help to add to Thunderbird. I can handle some of the "grunt work",
> but I know nothing about GUIs or the structure of Thunderbird as a whole.
>
> Here is my idea:
>
> In the displayed message header, I would like to add the images of
> flags (or some other designation) of all the countries through which
> the email has passed based on the "Received:" lines in the header.

I doubt many users will be able to positively identify countries based
on flags outside of a few well-known ones. For example, could you
(without looking it up) identify Nigeria's flag on site?

> All the "Received:" lines will be parsed for their IP addresses, and
> the location (country code) for that IP address will be determined by
> mapping IP addresses to country codes (or some similar technology).

A lot of Received headers will have private IPs (10.*, 192.168.*,
127.0.0.1, etc.). IP addresses are also a poor proxy for country: if I
were to send a message via the gmail UI, for example, it would be
recorded as coming from the US, even if I'm sitting in India (as
verified by Received stamps of messages). I've also tended to have low
faith in geographic IP address lookup services, but that may just be
unfounded fears.

>
> The displayed header will include a country's flag image for each
> different country that the message appears to have been sent through.
> In the case of the above headers, the flags will indicate the
> countries of Germany and Malaysia. (I'm pretty sure the F.B.I doesn't
> send emails through this path!) This will give the user an immediate
> visual indication of the number of countries a message has traversed
> through, as well as their flags. At a glance, users will know if an
> email from PayPal or Amazon actually came from outside the country.
> It won't guarantee that emails are legitimate, but it will probably
> make it easier for the average user to determine if an email is likely
> fake (phish/spam).

I'm also pretty sure the FBI does not send messages with a Reply-To
address different from the From address only in the domain name, both of
which are readily shown in the UI. I'm skeptical that users who would be
duped by these kinds of scams would be swayed by flags of countries.

>
> Parsing the "Received:" headers could also be used to filter/flag new
> messages if, for example, someone knows they will never receive an
> email from North Korea.
>
> I don't know if there's enough infrastructure within Thunderbird so
> this could be an optional plug-in or if this would need to be built
> into Thunderbird proper?
>

Personally, I think this is better suited to an add-on. There are minor
things that would need to be done before including in Mozilla (you need
a set of license-compatible flag images, and the ability to use
alternate geographic mapping services may be necessary).

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

[Joshua Cranmer 🐧]

On 4/30/2013 8:55 AM, Patrick Klos wrote:

>
>> - You will be willing to take most of the back-end work. There may be
>> a few free developers available, but most of the time everyone is
>> very busy, so as long as you are willing to do most of the
>> heavy-lifting, be my guest.
>
> I don't know anything about the front-end/UI. When a message is
> downloaded or displayed, I'll be happy to provide a routine that is
> fed the email headers, and that routine will extract the information
> needed and return a list of country codes. There are already public
> domain images of flags (small to medium size) that can be used for
> display.

Looking briefly at the display code, it looks like, when a message is
loaded, you get the received headers stuffed in the global variable
currentHeaderData at received, received1, received2, etc.

>
>> The idea would be to add another column to the message list, and
>> allow little, lightweight images to show up there. Also, we can have
>> a little flag button in the message viewer (here I designated it as
>> an "F"), and when that button it toggled, show up to three flags.
>
> I like adding the "flags" column to the summary pane.

The message list pane is a very performance-sensitive part of our
codebase. Descriptions on how to add a new column to the message list
are at
<https://developer.mozilla.org/en-US/docs/Extensions/Thunderbird/Creating_a_Custom_Column>,
but note that you do not get message sources easily (or performantly!)
when that code would be called.

[Ron K.]

Josiah on 4/30/2013 10:09 AM, keyboarded a reply:

> /Looks good, but I'm envisioning the flag images being placed
> side-by-side in the header (and wrapping if necessary) similar to how
> the To: line is displayed./
>
> I would be against this for a few reasons.
>
> A. If placed next to one of the existing displays (Probably the "From"),
> then the line would be very clutter, and little flag icons aren't very
> nice looking. Simply esthetically, it makes sense to be able to easily
> hide the flags, especially if we show them in the message list.
>
> B. If you meant to have another line (From, Subject, To, Locations),
> then again, that would take up quite a bit of space, and could clutter
> things up again.
>
> Now, if you want more flags to show, I would not be against expanding
> the width of the flag button. So that when toggled it brings a pulldown
> view that could look like:
>
> ---------
> [--] [--]
> [--] [--]
> [--] [--]
> ---------
>
> So perhaps six flags could show. You see, I personally wouldn't want a
> bunch of flags to display in that header every time.
>
> Anyway, definitely file a bug, this seems like a swell idea. The UI can
> be decided on later.
>

An interesting proposal that has value if users will understand what it
will do for them.

Reminder that TB has a very simple "Scam" feature which many of us
deactivated long ago in our profiles. Back during TB2 and earlier, the
notifier was part of a hidden stack that displayed between the thread
pane and the message pane in classic 3 pane view. Pref is set from
Options/Preferences > Security > E-mail Scams.

May want to look at how that feature displays now when it triggers. I am
not in favor of adding more Col to display in the thread pane, we have
18 options now. Nor do I consider using the message header bar a
suitable place to display more content. I think hooking into the
existing notifier when this proposal believes a mail is a scam is the
way to go. If it is like the older TB display there will be a wide space
for a localized text message as well as the flag icons in a 16x16 pix size.

--
Ron K.
Who is General Failure, and why is he searching my HDD?
Kernel Restore reported Major Error used BSOD to msg the enemy!

[David Lechner]

On 4/30/2013 8:55 AM, Patrick Klos wrote:

> I know it's possible - I just can't do it (all) myself.

In the GmailButtons addon that I wrote, I am adding Gmail Labels to the
message header area which is very similar to how you would like to add
the flags.

Source code is at <https://github.com/dlech/gmailbuttons>. You are
welcome to copy anything you like there.

> I don't know anything about the front-end/UI. When a message is
> downloaded or displayed, I'll be happy to provide a routine that is
> fed the email headers, and that routine will extract the information
> needed and return a list of country codes. There are already public
> domain images of flags (small to medium size) that can be used for
> display.

You can probably just cut out all the parts of GmailButtons that you
don't need (which is most of it) and add this routine and be pretty
close to a usable product.

I don't really have the interest to contribute new code to your project,
but I would be glad to answer any questions about the code I've already
written.

Have fun with it.

[Robert Miles]

On 4/30/2013 7:28 AM, Patrick Klos wrote:

> Hello Thunderbird Developers:
>
> I have a anti-spam/phish/junk feature I'd like to enlist someone's help
> to add to Thunderbird. I can handle some of the "grunt work", but I
> know nothing about GUIs or the structure of Thunderbird as a whole.
>
> Here is my idea:
>
> In the displayed message header, I would like to add the images of flags
> (or some other designation) of all the countries through which the email
> has passed based on the "Received:" lines in the header.

The Path: header also contains such information; any reason for not
using it as well?

I might be interested, but I've never worked with GUIs or interfacing
to Thunderbird before.

[Robert Miles]

On 4/30/2013 8:55 AM, Patrick Klos wrote:

> Josiah wrote:
[snip]

>> I really like the idea of being able to filter countries, and allowing
>> users to have a visual cue to where the email has been sent from. Now,
>> is it possible. I would say yes it is, but someone else should really
>> verify this other than me.
>
> I know it's possible - I just can't do it (all) myself.

Are you aware that many of the IP ranges have been assigned to
companies, not to countries? If the company is international, it can
assign IP addresses within its range or ranges to a computer in any
country where it has a computer.

[Joshua Cranmer 🐧]

On 4/30/2013 11:00 PM, Robert Miles wrote:
> On 4/30/2013 7:28 AM, Patrick Klos wrote:
>> Hello Thunderbird Developers:
>>
>> I have a anti-spam/phish/junk feature I'd like to enlist someone's help
>> to add to Thunderbird. I can handle some of the "grunt work", but I
>> know nothing about GUIs or the structure of Thunderbird as a whole.
>>
>> Here is my idea:
>>
>> In the displayed message header, I would like to add the images of flags
>> (or some other designation) of all the countries through which the email
>> has passed based on the "Received:" lines in the header.
>
> The Path: header also contains such information; any reason for not
> using it as well?

The Path header is not used for email messages, and the network
structure of NNTP means that the servers in the path header are not a
good indicator of the origination of the message. For example, my last
message would appear to have originated in Germany by this metric,
despite myself having never stepped in foot in that country.

[Patrick Klos]

Robert Miles wrote:
> Are you aware that many of the IP ranges have been assigned to
> companies, not to countries?

Yes. But most spam and phishing emails are not delivered via such
companies.

> If the company is international, it can assign IP addresses within its
> range or ranges to a computer in any country where it has a computer.

Let's start with the basics and see where it gets us?

Patrick