duplicates

[Robert E. Carneal]

Occasionally, I will receive a bunch of duplicates from someone who actually meant to send one email but for whatever reason, sends the exact same thing multiple times. I received the exact same email from someone today, same name, same time sent, same header, same content, same *everything* as far as I could tell.

Can I "teach" Gmail that if it detects exact same messages, it can delete all but *one* message?

Thank you.

Robert
Genealogy without documentation is mythology! Always SOURCE your work.

[Kenneth Ayers]

I tried just now sending an email from Yahoo! that was addressed to
two different Gmail email addresses, one of which is auto forwarded to
the other. Gmail only displays one received email in this case rather
than the two that I might have expected.

Seems like Gmail is already doing the duplicate deletion for me.

Kenneth

> --
> You received this message because you are subscribed to the Google Groups
> "Gmail-Users" group.
> To post to this group, send email to gmail...@googlegroups.com.
> To unsubscribe from this group, send email to
> gmail-users...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/gmail-users?hl=en.

[Andy]

> Can I "teach" Gmail that if it detects exact same messages, it can delete
> all but *one* message?

Gmail already is *supposed* to delete duplicate emails (which I
actually find annoying, because Gmail uses a liberal definition of
"duplicate" and many similar but far-from-identical messages also get
deleted ... and because the deleted message is gone forever, NOT sent
to Trash).

However, sometimes it works, sometimes it doesn't!

I don't know why. I'm guessing there must be something about those
emails, perhaps in the headers, that causes them to be seen as
distinct and not deleted.

I don't think we have any control over this. There is no setting, and
nothing to "teach".

Andy

[JohnW]

It's perhaps because the message may actually have been sent out twice (whether by design or accident): if the message has different IDs (remember, each e-mail does have a definite identity "number"!) then that's the parameter which Gmail sees as the "duplicate" rather than the actual content of the message to decide whether the message requires to be displayed/stored.
If the OP looks at the raw content of each of those 'identical' messages, then the e-mail identifier number will likely show up as being different (if the message was indeed sent twice).

[Kenneth Ayers]

I can confirm that when I send a single email that is addressed to
both my primary Gmail address and another Gmail which is auto
forwarded to my primary Gmail that I receive only one email at my
primary Gmail. However, if I send two emails, one to my primary Gmail
and the other to the other one, at almost the same time, I receive two
emails at my primary Gmail address. This is consistent with John's
explanation.

On Sun, Feb 5, 2012 at 4:17 AM, JohnW <john.w...@gmail.com> wrote:
> It's perhaps because the message may actually have been sent out twice (whether by design or accident): if the message has different IDs (remember, each e-mail does have a definite identity "number"!) then that's the parameter which Gmail sees as the "duplicate" rather than the actual content of the message to decide whether the message requires to be displayed/stored.
> If the OP looks at the raw content of each of those 'identical' messages, then the e-mail identifier number will likely show up as being different (if the message was indeed sent twice).
>

> --
> You received this message because you are subscribed to the Google Groups "Gmail-Users" group.

> To view this discussion on the web visit https://groups.google.com/d/msg/gmail-users/-/PKqWKGPqZy8J.

[Robert E. Carneal]

Ok, thanks.  Now, a quick clarifying question please-

Suppose someone send me the SAME message ten times, this identifying number will be different for all ten? In that case, I just delete nine of them.

If they send ten messages, and also manage to get the same identifying number on all ten (Is that even possible?), then, are you saying I can have Gmail auto-delete nine of those for me?  If so, can you tell me how please?

Thank you.

Robert
Genealogy without documentation is mythology! Always SOURCE your work.

[RebootAgain]

Wouldn't it be better to "teach" the sender of the duplicates how to send mail more accurately?  

I'd be wary of Gmail deleting messages automatically, even if they were seemingly identical.

On Sat, Feb 4, 2012 at 12:26 PM, Robert E. Carneal <kentuckyg...@gmail.com> wrote:

Occasionally, I will receive a bunch of duplicates from someone who actually meant to send one email but for whatever reason, sends the exact same thing multiple times....

[Kenneth Ayers]

I agree. I don't like Gmail deleting duplicates, even if it is the
same msg id. I know if I send an email addressed to both my primary
Yahoo! email and to an alias I have for it which goes to the same
account, I will get two emails in my Yahoo! inbox. Granted I don't
need the two emails but it is what I expect to see.

By deleting duplicates, Gmail inadvertently helps addressing problems
to go unnoticed. For instance, it may be that someone has an email
distribution list setup and my email is in there twice somehow. If I
could see that I'm getting duplicates, I could alert them to their
error.

Kenneth

> --
> You received this message because you are subscribed to the Google Groups
> "Gmail-Users" group.

[Koen Piller]

Kenneth,

And you would alert them with the purpose that they will not mail you in duplicate in future anymore i presume?
That is a good idea.
Even better idea:
instruct Google not to send duplicates. :)

Conclusion: it is a perfect good idea from Google programmers to avoid unness. mails resulting in lots of actions of people to tell other people the used database is not unique.

Regards,
Koen

2012/2/5 Kenneth Ayers <justk...@gmail.com>

[Robert E. Carneal]

I would normally agree. But occasionally what is happening is there is a new computer user, he or she will click send. Sometimes it does not go out "Star Trek" fast, they can still see it in their box.  So they try to send it again, and sometimes, again and again. Those are the ones I am trying to figure out how to reduce- I seem to get about 200 of those per day, seriously.

Thank you.

Robert
Genealogy without documentation is mythology! Always SOURCE your work.

[JohnW]

Each time the sender presses 'Send' then the message is issued with a new message ID (since it's a new "instruction" each time you press the 'Send' button, regardless of whether the content has changed or not) so those repeat transmissions will come in as different messages. However, after experimentation with attempting to recreate that situation, it does appear that Google's software is intelligent enough to "stack those messages" presumably as they have the same content, and Gmail will do the usual three-dot symbol indicating 'trimmed content'.
It is one of Google's "saving graces" (for me, at least!) that Gmail only stores one copy of a message: since Gmail is fundamentally a database, storing duplicates is truly a waste of space - and do remember that it's Google's space you'd be wasting holding those duplicates! It's much more efficient to store multiple references to the same message than to store multiple copies: besides there's nothing in it for Google by keeping those multiple copies - they would all simply point to the same content keywords which are used to steer their advertising content to your display, and that's where they raise their money (which pays for the storage quota they provide to you) after all.
If you are really seeing many, many multiple copies per day, I'm bound to ask what interface you're using to view the content of your mailbox. Are you using POP to recover the messages (this could explain the multiple deliveries of the same message) or is it a case of seeing these messages which are forwarded from many other accounts into the one 'main' Gmail account?
But if neither of those is the case, then there isn't anything you can do: it's another of Gmail's real blessings that it will deliver all messages to you (unlike others who won't deliver what they regard as Spam!) even if it does choose to label some of those as Spam (which you don't "have" to look at!). I really doubt that you can build a filter which would attempt to do exactly what you're attempting.

[Andy]

Under normal circumstances, the things I am aware of that cause duplicates are:

(1) I send a message to one (or more) email reflector lists or groups
like this one. I should receive one message back from each group that
I sent it to; and if I didn't use Gmail, I would. But because of the
way Gmail works, it deletes all of the incoming emails because I
already have my own copy in Sent Mail. Personally I find this
annoying because I have no easy way of telling if my messages actually
made it through all the groups I sent it to; and because the received
copies are NOT duplicates, despite having the same message ID.

(2) Occasionally there is an Internet "hiccup" that causes a duplicate
copy of an email to be generated. I don't know why, but it happens.
Perhaps if some node (computer) on the 'net relays your message, but
then crashes just when it is about to mark your message as relayed.
So some time later it still has messages in its queue that it thinks
it needs to send again.

If a user keeps pressing his "Send" button over and over, each message
ought to have a unique message ID, I think; so according to what I've
read, Google should see them as distinct and not as duplicates.
Unfortunately, you would need to deal with that yourself (with your
Delete key).

If you are really getting hundreds of copies of the same message, then
something is seriously wrong, probably closer to the sender's computer
than with your Gmail. Are you sure they aren't spam messages? Spam
is unfortunately supposed to work that way (inundate you with so many
messages you didn't want, so that you read at least a few of them).

Earlier you asked how to train or set Gmail to delete duplicates with
the same message ID. There is no training, and no setting. It does
that already. But ONLY if they have the same message ID.

This "feature" of Gmail that it deletes duplicates (and we have NO
control over it) is for me one of Gmail's biggest negatives; but I
live with it.

Andy

[Robert E. Carneal]

Emphasis:

If you are really getting hundreds of copies of the same message, then
something is seriously wrong, probably closer to the sender's computer
than with your Gmail.  Are you sure they aren't spam messages?  Spam
is unfortunately supposed to work that way (inundate you with so many
messages you didn't want, so that you read at least a few of them).

Earlier you asked how to train or set Gmail to delete duplicates with
the same message ID.  There is no training, and no setting.  It does
that already.  But ONLY if they have the same message ID.

They are not Spam messages. I *think* what may be happening is the person sending them is from the nursing home. (About my mother) Unfortunately, I discovered yesterday nearly all, if not indeed all, of the employees there do not know how email works-  One lady thought email would get to me immediately whether I was home or not and when I did not respond in two seconds, she just repeatedly re-sent and re-sent the message!!! You are right, they all have unique IDs. She was trying to get my attention here at home, but email does not work that way. She does not know that.

I have offered to show them how it works, but they declined. I will just delete the duplicates myself. Thank you, everyone.

Thank you.

Robert
Genealogy without documentation is mythology! Always SOURCE your work.

On Mon, Feb 6, 2012 at 8:30 AM, Andy <AI.e...@gmail.com> wrote:

If you are really getting hundreds of copies of the same message, then
something is seriously wrong, probably closer to the sender's computer
than with your Gmail.  Are you sure they aren't spam messages?  Spam
is unfortunately supposed to work that way (inundate you with so many
messages you didn't want, so that you read at least a few of them).

Earlier you asked how to train or set Gmail to delete duplicates with
the same message ID.  There is no training, and no setting.  It does
that already.  But ONLY if they have the same message ID.

--

[Andy]

> ... Gmail's real blessings that it will deliver all messages to you (unlike
> others who won't deliver what they regard as Spam!) ...

Getting a bit off-track here perhaps ... but I need to disagree with
that statement.

A) The fact is that Gmail DOES delete incoming messages to you that
it thinks are duplicates (based solely on the Message-ID, apparently).
Delete as in trashed, gone forever, bits dumped on the floor with no
chance of recovery. I really hate that; it is irresponsible and
stupid, and I've argued/complained to them about it, but it hasn't
changed.

B) Gmail also DOES delete (block) some messages that it considers
spam. I have seen evidence of this only recently. I have a ...
what's it called ... "Google Apps"? account, from a non-profit I
belong to. I have mine configured to forward to my personal Gmail
account. In the last few weeks, the Google Apps account has been
getting bounce messages from some emails, saying that the receiving
system (= Gmail) blocked the message that was forwarded from the
Google Apps account. I think it might have implied that the sender's
domain was blacklisted. Some of those emails were indeed spam, but
some were not. I find this very disturbing.

You are MOSTLY correct that Gmail delivers ALMOST everything, even if
it thinks it is spam, in contrast to some others. But Gmail does
indeed block SOME spam, and even some non-spam. (Which doesn't
explain why I still receive a lot of spam in my Gmail Spam 'folder'.)

Andy

[JohnW]

Andy, perhaps it is somewhat off-track, but hopefully there will be readers with slightly less knowledge who might just pick up some information to help broaden their own understanding!

Re. A) The fact is that Gmail DOES delete incoming messages to you that

it thinks are duplicates (based solely on the Message-ID, apparently).

If the Message-ID is the same, that can only be because it is "the same message"! And as such it'll be the same as you already have in your mailbox. Why would you necessarily want to see the two copies of the same content? And I can show that if you receive exactly the same content (but with different Message-IDs) then Gmail will show them as a single stack in Conversation mode, displaying the Clipped message symbol.

Re. B) Gmail also DOES delete (block) some messages <snipped> I think it might have implied that the sender's domain was blacklisted.
This type of "non-delivery" is very suspect, with that I would agree: but I personally have not experienced it (I wouldn't know some e-mails haven't been delivered if I was unaware they'd been sent in the first place!). But frankly, if the sender's domain is black-listed for some reason, frankly I think I'm better off if Google doesn't deliver mail from such origins. I get more than enough spam from domains that are correctly set up, and thankfully in my case Gmail appears quite able to distinguish 99.997% of such rubbish correctly. The point I was really trying to make was that Gmail as a service does 'predominantly' allow the recipient to decide what to discard/regard as Spam rather than the Service Provider (AOL, Yahoo!, etc.,) make such decisions.
The quantity of messages which other Service Providers simply don't process at all (and instead simply junk them!) sometimes beggars belief! - all predicated on the basis that the messages originate from gmail.com. This is why I've always recommended business users to have their own domain rather than use ordinary gmail addresses - that way they would stand a better chance of not falling into the junked gmail.com message bin!

[Andy]

On Wed, Feb 8, 2012 at 06:56, JohnW <john.w...@gmail.com> wrote:

> If the Message-ID is the same, that can only be because it is "the same
> message"! And as such it'll be the same as you already have in your mailbox.

No. When I send a message to this (the Gmail-Users) group, I get my
copy sent back to me (which Google deletes), with a few lines added at
the bottom about group management. My message has been modified by
Googlegroups, but apparently carries the same Message-ID. (I can't be
100% certain of that, because Gmail summarily deletes the copy it
receives, so I can't compare their Message-IDs.)

Now those few added lines are not much of a problem for a Googlegroup;
but for some of my other email groups, it certainly is. On many of
them, the Moderators are free to edit messages, and at least a couple
of them routinely answer your questions right within the body of your
own message. The consequence is that I never receive the answers to
my questions! But everyone else does, so I look stupid by asking the
question again and again.

> Why would you necessarily want to see the two copies of the same content?

Even if they were the same content, the reason why I might want that,
is to know that my message was indeed distributed.

Anyway, it should be **MY** choice whether I want to get rid of
duplicates, NOT Google's. Google is telling me I don't want to see
those messages. Yet you said Google is unlike other services because
it faithfully delivers everything to me uncensored. That is not true,
because they do indeed censor and decide what I am allowed to see.

If it's a matter of disk space, Google is always saying we don't need
to delete messages because there is so much disk space. So why must I
accept them deleting my emails for me?

> ... Gmail will show them as a single stack in Conversation mode,

> displaying the Clipped message symbol.

I'm not sure what the "Clipped message symbol" is.

I'm also not sure what you mean by "Gmail will show THEM" (emphasis
added). Gmail doesn't show THEM, it shows ONE of them and deletes the
rest. So what is it that you are seeing, that I'm not? Does my Gmail
work differently than yours?

Andy

[Andy]

> ... Gmail will show them as a single stack in Conversation mode, displaying
> the Clipped message symbol.

Can anybody show me an example of this "Clipped message symbol"?

[Marko Vukovic]

--
Marko

[Kenneth Ayers]

Gmail really loves truncating emails with that clipped message symbol.

I've recently been using Gmail with conversation view off rather than on.  I thought that with conversation view off that emails would be displayed in their entirety rather than being truncated since I'm displaying without the context of conversation.  While it is true that there is less truncation with conversation view off, Gmail still truncates the footers or signatures of received emails.

Kenneth

On Fri, Feb 10, 2012 at 1:00 AM, Marko Vukovic <marko....@gmail.com> wrote:

--
Marko

[Andy]

So that's not the same thing as the "- Show quoted text -" in Gmail's
"old look"??

Or is it the same?

"Show quoted text" has been here for a very long time. I thought you
were referring to some new feature that shows duplicate emails that we
thought Gmail had deleted. As far as I know, when Gmail deletes a
duplicate, (a) no portion of it is displayed or displayable by
clicking on something, (b) there is no symbol left behind to indicate
that there was a duplicate, and (c) it is really really gone, with no
chance of recovery.

Is that not still correct?

[Kenneth Ayers]

On Wed, Feb 8, 2012 at 5:44 AM, Andy <AI.e...@gmail.com> wrote:
> On Wed, Feb 8, 2012 at 06:56, JohnW <john.w...@gmail.com> wrote:
>
>> If the Message-ID is the same, that can only be because it is "the same
>> message"! And as such it'll be the same as you already have in your mailbox.
>
> No.  When I send a message to this (the Gmail-Users) group, I get my
> copy sent back to me (which Google deletes), with a few lines added at
> the bottom about group management.  My message has been modified by
> Googlegroups, but apparently carries the same Message-ID.  (I can't be
> 100% certain of that, because Gmail summarily deletes the copy it
> receives, so I can't compare their Message-IDs.)
>
> Now those few added lines are not much of a problem for a Googlegroup;
> but for some of my other email groups, it certainly is.  On many of
> them, the Moderators are free to edit messages, and at least a couple
> of them routinely answer your questions right within the body of your
> own message.  The consequence is that I never receive the answers to
> my questions!  But everyone else does, so I look stupid by asking the
> question again and again.
>

Andy,

So the email you send to your email group and the moderated/altered
email everyone (but you) receives back from the email group have the
same msg ID? If so, this is an excellent example of how two emails
with the same msg ID can be different. Sounds like Gmail may be
guilty of non-duplicate email deletion.

Kenneth

[Andy]

On Fri, Feb 10, 2012 at 15:18, Kenneth Ayers <justk...@gmail.com> wrote:

> So the email you send to your email group and the moderated/altered
> email everyone (but you) receives back from the email group have the
> same msg ID?  If so, this is an excellent example of how two emails
> with the same msg ID can be different.  Sounds like Gmail may be
> guilty of non-duplicate email deletion.

Have you ever received a copy of your own messages (like the one you
sent above) back from this group? Can you find any copy of one that
you've sent that has those four footer lines at the bottom that are
added by the group?

There's your proof.

The reason you don't have them, is because Gmail deleted them,
claiming they were "duplicates" of the one you sent.

We are told that Gmail uses the Message-ID as evidence of a duplicate,
but doesn't compare the content to see if they really are exact
duplicates. Yet when a message goes through an email reflector, its
Message-ID apparently remains the same, even when the message body is
modified.

Andy

[Sean Murphy]

    It is exactly the same thing. And sometimes, when someone deliberately or accidentally sends multiple copies of the same message, they display as multiple messages, with the extras just displaying as "-Show quoted text-" in the old look or the ellipse symbol in the new look. Two different types of "duplicate" messages are being talked about here.