Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
How to obtain the message.ID from postfix?
1,513 views
Skip to first unread message
Vincenzo Romano
Nov 25, 2011, 10:09:01 AM11/25/11
to
Hi all.
My application uses the sendmail interface of its local postfix
installation to send emails, a few thousand a day.
Of course, this has been working pretty fine so far.
Now I'd need to be able to track these emails and the best thing I
could do is to "somehow" obtain from sendmail/postfix
the message-ID it generated to each single submission so I can later
check the logs about delays and so on.
I've not been able to find a reliable way to do it as my issue is that
I can send several emails to the same destination address within a few
seconds and simply use the destination address and the (supposed)
submission timestamp won't help much.
Any idea whether this is doable and how to?
TIA.
--
Vincenzo Romano
My application uses the sendmail interface of its local postfix
installation to send emails, a few thousand a day.
Of course, this has been working pretty fine so far.
Now I'd need to be able to track these emails and the best thing I
could do is to "somehow" obtain from sendmail/postfix
the message-ID it generated to each single submission so I can later
check the logs about delays and so on.
I've not been able to find a reliable way to do it as my issue is that
I can send several emails to the same destination address within a few
seconds and simply use the destination address and the (supposed)
submission timestamp won't help much.
Any idea whether this is doable and how to?
TIA.
--
Vincenzo Romano
Ralf Hildebrandt
Nov 25, 2011, 10:11:51 AM11/25/11
to
* Vincenzo Romano <vincenz...@notorand.it>:
Rather use SMTP injection and then use the queueid:
MAIL FROM:<hil...@charite.de>
250 2.1.0 Ok
RCPT TO:<vincenz...@notorand.it>
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
foo
.
250 2.0.0 Ok: queued as 3Sqgph2ghTz2r0b <-------- !
QUIT
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hil...@charite.de | http://www.charite.de
MAIL FROM:<hil...@charite.de>
250 2.1.0 Ok
RCPT TO:<vincenz...@notorand.it>
250 2.1.5 Ok
DATA
354 End data with <CR><LF>.<CR><LF>
foo
.
250 2.0.0 Ok: queued as 3Sqgph2ghTz2r0b <-------- !
QUIT
--
Ralf Hildebrandt
Geschäftsbereich IT | Abteilung Netzwerk
Charité - Universitätsmedizin Berlin
Campus Benjamin Franklin
Hindenburgdamm 30 | D-12203 Berlin
Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962
ralf.hil...@charite.de | http://www.charite.de
Vincenzo Romano
Nov 25, 2011, 10:14:42 AM11/25/11
to
2011/11/25 Ralf Hildebrandt <Ralf.Hil...@charite.de>:
Yes, that's the SMTP dialogue.
But it seems to me that the sendmail (or any other postfix tool) can
provide it to be on a per-submission basis ...
But it seems to me that the sendmail (or any other postfix tool) can
provide it to be on a per-submission basis ...
Wietse Venema
Nov 25, 2011, 10:48:03 AM11/25/11
to
Vincenzo Romano:
The Postfix sendmail command creates a temporary queue file that
is deleted as soon as the local mail pickup daemon has read the
file. So, that queue file ID is useless.
You can specify your own envelope ID on the Postfix sendmail command
line. This will be returned in delivery status notifications (as
long as the remote MTA implements the DSN RFCs).
Postfix does not log the DSN envelope ID information, but code for
that could be added to the cleanup daemon.
There is also RFC 3885 which builds a tracking mechanism on top of
the aforementioned DSN envelope ID information. But this is of limited
interest, since qmail, exim and some other MTAs don't implement DSN.
Wietse
is deleted as soon as the local mail pickup daemon has read the
file. So, that queue file ID is useless.
You can specify your own envelope ID on the Postfix sendmail command
line. This will be returned in delivery status notifications (as
long as the remote MTA implements the DSN RFCs).
Postfix does not log the DSN envelope ID information, but code for
that could be added to the cleanup daemon.
There is also RFC 3885 which builds a tracking mechanism on top of
the aforementioned DSN envelope ID information. But this is of limited
interest, since qmail, exim and some other MTAs don't implement DSN.
Wietse
Vincenzo Romano
Nov 26, 2011, 4:00:58 AM11/26/11
to
2011/11/25 Wietse Venema <wie...@porcupine.org>:
Thanks Wietse (also for the honor).
Maybe I've been not clear enough and I apologyze for that.
I'll try to describe a real life case.
Once I do the submission with sendmail tool, I need to check on my
machine whether the message actually left the system for the delivery
(which can access the Internet) or if it got queued or trashed for
some reason.
As far as I remember I can link the various lines in the log only
thanks to the queue ID.
This is what I do, step by step, within the logs:
0. First restrict the search field to a "reasonable" time period in
the logs. Usually one day.
1. I look for "postfix/qmgr" and the destination email address to
extract che queue ID (something like 42DF66C96E).
2. I look then for both that queue ID and "postfix/smtp" in order to
extract the status (something like "status=sent") and the remote SMTP
response (something like "(250 2.0.0 OK 1322295987
b11si12544560fak.190)" ).
3. I look then for both that queue ID and "postfix/cleanup" in order
to extract the local message-ID (something like
"message-id=<201111260826...@system.domain>")
The step #2 is needed in order to know whether the message actually
left the system (status=sent) or bounced or got deferred. I can also
get the actual response from the remore server.
Steps #2 and #3 are needed in order to be able to look for the message
on either side.
The steps #0 and #1 are approximate, especially when I send several
distinct messages in the same second to the same destination. Which
can easily happen.
I understood that I cannot grab the local message ID straight from
sendmail, this is why I focused on the queue ID.
But, is it possible to know the queue ID from the sendmail tool (or
whatever other submission tool) to be reasonably precise on the
subsequent email tracking lookups?
Maybe I've been not clear enough and I apologyze for that.
I'll try to describe a real life case.
Once I do the submission with sendmail tool, I need to check on my
machine whether the message actually left the system for the delivery
(which can access the Internet) or if it got queued or trashed for
some reason.
As far as I remember I can link the various lines in the log only
thanks to the queue ID.
This is what I do, step by step, within the logs:
0. First restrict the search field to a "reasonable" time period in
the logs. Usually one day.
1. I look for "postfix/qmgr" and the destination email address to
extract che queue ID (something like 42DF66C96E).
2. I look then for both that queue ID and "postfix/smtp" in order to
extract the status (something like "status=sent") and the remote SMTP
response (something like "(250 2.0.0 OK 1322295987
b11si12544560fak.190)" ).
3. I look then for both that queue ID and "postfix/cleanup" in order
to extract the local message-ID (something like
"message-id=<201111260826...@system.domain>")
The step #2 is needed in order to know whether the message actually
left the system (status=sent) or bounced or got deferred. I can also
get the actual response from the remore server.
Steps #2 and #3 are needed in order to be able to look for the message
on either side.
The steps #0 and #1 are approximate, especially when I send several
distinct messages in the same second to the same destination. Which
can easily happen.
I understood that I cannot grab the local message ID straight from
sendmail, this is why I focused on the queue ID.
But, is it possible to know the queue ID from the sendmail tool (or
whatever other submission tool) to be reasonably precise on the
subsequent email tracking lookups?
Wietse Venema
Nov 26, 2011, 8:33:36 AM11/26/11
to
Vincenzo Romano:
> 2011/11/25 Wietse Venema <wie...@porcupine.org>:
I answered that in the FIRST paragraph of my reply.
Wietse
> 2011/11/25 Wietse Venema <wie...@porcupine.org>:
Wietse
Vincenzo Romano
Nov 26, 2011, 10:21:36 AM11/26/11
to
2011/11/26 Wietse Venema <wie...@porcupine.org>:
You say it's useless, I say I can use it to retrieve the message IDs I need.
Is there a way to get the queue ID back from sendmail?
Is there a way to get the queue ID back from sendmail?
Noel Jones
Nov 26, 2011, 10:43:24 AM11/26/11
to
On 11/26/2011 9:21 AM, Vincenzo Romano wrote:
> You say it's useless, I say I can use it to retrieve the message IDs I need.
> Is there a way to get the queue ID back from sendmail?
Not possible. The sendmail command has no knowledge of the queueID
> You say it's useless, I say I can use it to retrieve the message IDs I need.
> Is there a way to get the queue ID back from sendmail?
that will eventually be assigned. The queueID is not -- and cannot
be -- assigned until after sendmail(1) finishes processing the mail.
Your best bet it so set your own Message-ID header so you know what
to search for in the logs.
-- Noel Jones
Wietse Venema
Nov 26, 2011, 10:55:37 AM11/26/11
to
Vincenzo Romano:
it is not the queue ID that you see in Postfix logfiles.
Wietse
> 2011/11/26 Wietse Venema <wie...@porcupine.org>:
> > Vincenzo Romano:
> >> 2011/11/25 Wietse Venema <wie...@porcupine.org>:
> >> > The Postfix sendmail command creates a temporary queue file that
> >> > is deleted as soon as the local mail pickup daemon has read the
> >> > file. So, that queue file ID is useless.
...
> > Vincenzo Romano:
> >> 2011/11/25 Wietse Venema <wie...@porcupine.org>:
> >> > The Postfix sendmail command creates a temporary queue file that
> >> > is deleted as soon as the local mail pickup daemon has read the
> >> > file. So, that queue file ID is useless.
> >> But, is it possible to know the queue ID from the sendmail tool (or
> >> whatever other submission tool) to be reasonably precise on the
> >> subsequent email tracking lookups?
> >
> > I answered that in the FIRST paragraph of my reply.
>
> >> whatever other submission tool) to be reasonably precise on the
> >> subsequent email tracking lookups?
> >
> > I answered that in the FIRST paragraph of my reply.
>
> You say it's useless,
The Postfix sendmail command queue file name is useless, because
it is not the queue ID that you see in Postfix logfiles.
Wietse
Vincenzo Romano
Nov 28, 2011, 11:29:39 AM11/28/11
to
Now I could select the log rows I'm interested in.
The point is that postfix/smtp is not logging the Message-ID along
with the other details it logs.
Is there a way to ask postfix/smtp to log also the Message-ID?
Thanks again.
Wietse Venema
Nov 28, 2011, 11:49:35 AM11/28/11
to
Vincenzo Romano:
ID in the end-of-data reply.
When you search the maillog file for this queue ID, then you will
find the Message-ID which is logged by the cleanup server.
Wietse
> The point is that postfix/smtp is not logging the Message-ID along
> with the other details it logs.
> Is there a way to ask postfix/smtp to log also the Message-ID?
If you submit over port 25, then the SMTP server provides the queue
> with the other details it logs.
> Is there a way to ask postfix/smtp to log also the Message-ID?
ID in the end-of-data reply.
When you search the maillog file for this queue ID, then you will
find the Message-ID which is logged by the cleanup server.
Wietse
Vincenzo Romano
Nov 28, 2011, 12:17:30 PM11/28/11
to
Viktor Dukhovni
Nov 28, 2011, 12:31:52 PM11/28/11
to
globally unique application-assigned Message-Id. That message-id
is logged by cleanup(8) and the corresponding queue-id correlates
this log message with the other log entries for the same message.
It seems this question has been re-asked and re-answered many times,
I think it is time to stop.
--
Viktor.
Vincenzo Romano
Nov 28, 2011, 2:16:39 PM11/28/11
to
2011/11/28 Viktor Dukhovni <postfi...@dukhovni.org>:
Correct, Viktor.
Once I'm said the queue ID is useful, then useless, then useful again.
Now I know.
Once I'm said the queue ID is useful, then useless, then useful again.
Now I know.
Simone Caruso
Nov 30, 2011, 5:39:48 AM11/30/11
to
che sia univoco e che t renda tracciabile un messaggio.
Se vuoi che un ID VALIDO venga assegnato da Postfix devi inviare tramite la
porta 25, al contrario l'ID restituito da sendmail non puoi prenderlo in
considerazione perché è temporaneo.
In breve... usa smtp.
Spero ti averti chiarito la cosa.
--
Simone Caruso
IT Consultant
+39 349 65 90 805
Reindl Harald
Nov 30, 2011, 5:45:01 AM11/30/11
to
Am 30.11.2011 11:39, schrieb Simone Caruso:
>> Correct, Viktor.
>> Once I'm said the queue ID is useful, then useless, then useful again.
>> Now I know.
> Te lo spiego in italiano, Viktor dice di assegnare un ID a livello APPLICATIVO
> che sia univoco e che t renda tracciabile un messaggio.
>
> Se vuoi che un ID VALIDO venga assegnato da Postfix devi inviare tramite la
> porta 25, al contrario l'ID restituito da sendmail non puoi prenderlo in
> considerazione perché è temporaneo.
>
> In breve... usa smtp.
> Spero ti averti chiarito la cosa.
the language inside a running thread?
Simone Caruso
Nov 30, 2011, 6:05:27 AM11/30/11
to
>
> what about speaking english in a public mailing-list instead switch
> the language inside a running thread?
>
Sorry I forgot to remove the list from 'Cc'; anyway i translated only Wietse and
> what about speaking english in a public mailing-list instead switch
> the language inside a running thread?
>
Viktor emails without adding anything.
0 new messages