rewrite From: header with name

[Hans Ginzel]

Hello!

How can I, please, rewrite the "From:" header of the local user when
sending e-mail remotely? Not only the e-mail address but also the name.
E.g.

From: Technical user <na...@host.company.corp.local>
to
From: Department <Department.country@group_company.com>

I use postfix-2.6.6 on CentOS.

Regards,
Hans

[Wietse Venema]

Hans Ginzel:

If it is a few users, use header_checks. If it is many users,
use a Milter (available in Python, Perl, C).

Wietse

[Hans Ginzel]

Dne 24.08.2015 12:45, wie...@porcupine.org napsal:

> If it is a few users, use header_checks. If it is many users,
> use a Milter (available in Python, Perl, C).

Thank you.

Few users. I added
header_checks = pcre:/etc/postfix/header_checks
to /etc/postfix/main.cf.

I ran
postmap pcre:/etc/postfix/header_checks
postmap: fatal: unsupported map type: pcre

Based on http://comments.gmane.org/gmane.mail.postfix.user/184149 I
realised there is not need to run postmap. Feature requets: The error
message could be explicit:
"postmap: info: There is no need to run postmap on types: pcre,
regexp."

I added into /etc/postfix/header_checks
/^From(.*)/ WARN $1

I sent some e-mails, looked in /var/log/maillog, found messages like
“…(delivered to mailbox)”, “…removed”; but did not find any warning
messages. Where they were gone, please?

Regards,
Hans

[Viktor Dukhovni]

On Mon, Aug 24, 2015 at 03:53:24PM +0200, Hans Ginzel wrote:

> >If it is a few users, use header_checks. If it is many users,
> >use a Milter (available in Python, Perl, C).
>
> Thank you.
>
> Few users. I added
> header_checks = pcre:/etc/postfix/header_checks
> to /etc/postfix/main.cf.

And tested that it is set? (Hint: "postconf -n header_checks").
And made sure that header_checks has no overrides in the relevant
master.cf cleanup entry?

> I added into /etc/postfix/header_checks
> /^From(.*)/ WARN $1

And tested via

postmap -fq "From: Joe User <joe....@example.com>" pcre:/etc/postfix/header_checks

before sending email?

> I sent some e-mails, looked in /var/log/maillog, found messages like
> "(delivered to mailbox)", "removed"; but did not find any warning
> messages. Where they were gone, please?

Check your syslog configuration.

--
Viktor.

[Hans Ginzel]

Thank you.

Dne 24.08.2015 16:03, Viktor Dukhovni napsal:

> And tested that it is set? (Hint: "postconf -n header_checks").
> And made sure that header_checks has no overrides in the relevant
> master.cf cleanup entry?

I have reloaded the daemon
service postfix reload

The check is OK:
postconf -n header_checks
header_checks = pcre:/etc/postfix/header_checks

> And tested via
> postmap -fq "From: Joe User <joe....@example.com>"
> pcre:/etc/postfix/header_checks
> before sending email?

Wow! I am sorry, I didn't. It also works

postmap -fq "From: Joe User <joe....@example.com>"
pcre:/etc/postfix/header_checks

WARN From: Joe User <joe....@example.com>
(I rewrote parenthesis in the rule to include whole pattern – also
“From: ”)

I have default rsyslog configuration:
…
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none
/var/log/messages

# Log all the mail messages in one place.
mail.*
-/var/log/maillog
…

I have looked for newest log files:
ls -lt /var/log |head
and searched:
grep -lsi user_name@ /var/log/*
but found nothing appropriate.

How to debug header_checks, please? How should the postfix warning
message look like in log file?

Regards,
Hans

[Wietse Venema]

Hans Ginzel:

> How to debug header_checks, please?

Use Postfix logging. If you can't find it, then you need to fix
your logging infrastructure which is not part of Postfix.

Perhaps it helps to turn off chroot:
http://www.postfix.org/DEBUG_README.html#no_chroot. Header checks
are logged by the cleanup daemon.

> How should the postfix warning message look like in log file?

The same text that you get with the "postmap -q" example. Postfix
logs all mail activity. If you have no logs then you are blind.

Wietse

[Viktor Dukhovni]

On Mon, Aug 24, 2015 at 05:20:51PM +0200, Hans Ginzel wrote:

> How to debug header_checks, please?

First, make sure logging works:

$ postlog -p warn -t postfix/test "warning: this is a test".

find the log message in the log file. If that shows up, but still
cleanup warnings in the same file, then look for inadvertent or
unwanted settings of "receive_override_options" that might disable
header checks.

And of course do look into chroot issues as suggested by Wietse.

--
Viktor.

[Hans Ginzel]

Thank you.

Dne 24.08.2015 17:40, Viktor Dukhovni napsal:

> On Mon, Aug 24, 2015 at 05:20:51PM +0200, Hans Ginzel wrote:
> First, make sure logging works:
> $ postlog -p warn -t postfix/test "warning: this is a test".

It works. Message is in /var/log/maillog.

> find the log message in the log file. If that shows up, but still
> cleanup warnings in the same file, then look for inadvertent or
> unwanted settings of "receive_override_options" that might disable
> header checks.

postconf -n receive_override_options
shows nothing.

> And of course do look into chroot issues as suggested by Wietse.

I will realise that.

Regards,
HG

[Benny Pedersen]

Hans Ginzel skrev den 2015-08-24 17:47:

> postconf -n receive_override_options
> shows nothing.

it will be stupid in main.cf anyway

postconf -Mf | grep recieve_override_options

[Hans Ginzel]

Thank you.

Dne 24.08.2015 17:32, wie...@porcupine.org napsal:

> Use Postfix logging. If you can't find it, then you need to fix
> your logging infrastructure which is not part of Postfix.

As I wrote, logging messages goes to /var/log/maillog.

> Perhaps it helps to turn off chroot:
> http://www.postfix.org/DEBUG_README.html#no_chroot. Header checks
> are logged by the cleanup daemon.

All lines in master.cf has “n” in the chroot column.

There is a cleanup line in the master.cf:
grep cleanup /etc/postfix/master.cf
cleanup unix n - n - 0 cleanup

How can I check, please, if the daemon “is called”?

Regards,
Hans

[Hans Ginzel]

Thank you.

Dne 24.08.2015 17:52, Benny Pedersen napsal:

> it will be stupid in main.cf anyway
> postconf -Mf | grep recieve_override_options

postconf -Mf | grep recieve_override_options
postconf: invalid option -- 'M'
postconf: fatal: usage: postconf …

rpm -q -f `which postconf`
postfix-2.6.6-6.el6_5.x86_64

Regards,
Hans

[Benny Pedersen]

Hans Ginzel skrev den 2015-08-24 17:57:

> rpm -q -f `which postconf`
> postfix-2.6.6-6.el6_5.x86_64

possible to upgrade it ?

grep override /etc/postfix/master.cf

[Wietse Venema]

Hans Ginzel:

> cleanup unix n - n - 0 cleanup
>

> How can I check, please, if the daemon ?is called??

Send an email, and it will log something. For example this was
logged when your posting arrived at my machine:

Aug 24 11:55:55 spike postfix/cleanup[88653]: 3n0J2g49SHzJrQ2: message-id=<b3f2bb0fcd9a0bfa...@artax.karlin.mff.cuni.cz>

Wietse

[Hans Ginzel]

Thank you.

On Mon, Aug 24, 2015 at 01:43:08PM -0400, Wietse Venema wrote:
>Send an email, and it will log something. For example this was

I have removed timestamp and hostname:
postfix/pickup[20385]: 52D3E801BA: uid=494 from=<dw>
postfix/cleanup[23831]: 52D3E801BA: message-id=<201508242023...@CZ00TDW001.localdomain>
postfix/qmgr[9872]: 52D3E801BA: from=<d...@CZ00TDW001.localdomain>, size=489, nrcpt=1 (queue active)
postfix/local[23833]: 52D3E801BA: to=<T91...@localhost.localdomain>, orig_to=<T912205@localhost>, relay=local, delay=0.2, delays=0.17/0/0/0.02, dsn=2.0.0, status=sent (delivered to mailbox)
postfix/qmgr[9872]: 52D3E801BA: removed

It seems that cleanup “was called”. But how to debug header_checks, please? Where is the warning message?

Regards,
Hans

[Christian Kivalo]

When there are logs but no warning message then your header_checks are
not triggered.

Whats the content of your header_checks file?

> Regards,
> Hans

- c

[Hans Ginzel]

On Tue, Aug 25, 2015 at 09:37:37AM +0200, Christian Kivalo wrote:
>Whats the content of your header_checks file?

/^(From:.*)/i
WARN header_checks: $1

And test

postmap -fq "From: Joe User <joe....@example.com>" pcre:/etc/postfix/header_checks

works:
WARN header_checks: From: Joe User <joe....@example.com>

Howto configure postfix to log to file instaed of syslogd, please?

Regards,
Hans

[Koko Wijatmoko]

On Tue, 25 Aug 2015 10:30:07 +0200
Hans Ginzel <ha...@matfyz.cz> wrote:

> Howto configure postfix to log to file instaed of
> syslogd, please?
>

postfix default not log to file, but use syslogd service.
maybe your syslogd is filtered the output? re-check your
syslogd daemon configuration (rsyslog, syslog-ng etc).

[Christian Kivalo]

On 2015-08-25 10:30, Hans Ginzel wrote:
> On Tue, Aug 25, 2015 at 09:37:37AM +0200, Christian Kivalo wrote:
>> Whats the content of your header_checks file?
>
> /^(From:.*)/i
> WARN header_checks: $1

is this on one line in the file?

> And test
> postmap -fq "From: Joe User <joe....@example.com>"
> pcre:/etc/postfix/header_checks
> works:
> WARN header_checks: From: Joe User <joe....@example.com>

Are you sure your emails contain a line with a ^From:.* header? Maybe
try with something else first?

> Howto configure postfix to log to file instaed of syslogd, please?
>

> Regards,
> Hans

[Christian Kivalo]

On 2015-08-25 10:30, Hans Ginzel wrote:
> On Tue, Aug 25, 2015 at 09:37:37AM +0200, Christian Kivalo wrote:
>> Whats the content of your header_checks file?
>
> /^(From:.*)/i
> WARN header_checks: $1
>

> And test
> postmap -fq "From: Joe User <joe....@example.com>"
> pcre:/etc/postfix/header_checks
> works:
> WARN header_checks: From: Joe User <joe....@example.com>
>

> Howto configure postfix to log to file instaed of syslogd, please?

Maybe this applies to your problem, from
http://www.postfix.org/header_checks.5.html

Message headers added by the cleanup(8) daemon itself are
excluded from
inspection. Examples of such message headers are From:,
To:, Mes-
sage-ID:, Date:.



> Regards,
> Hans

regards
- c

[Hans Ginzel]

On Tue, Aug 25, 2015 at 11:21:12AM +0200, Christian Kivalo wrote:
>On 2015-08-25 10:30, Hans Ginzel wrote:
>>/^(From:.*)/i
>>WARN header_checks: $1
>

>is this on one line in the file?

It is on two lines, but the second is indented by tab.

>Are you sure your emails contain a line with a ^From:.* header?

No, I am not. I have supposed MUA creates this header.
Test results: mutt does but mailx not (mailx-12.4-8).

>Maybe try with something else first?

I have changed header_checks to
/^(.*)/ WARN header_checks: $1
and headers have been loged.
Thank you!

Best regards,
Hans

[Hans Ginzel]

On Tue, Aug 25, 2015 at 11:30:07AM +0200, Christian Kivalo wrote:
>Maybe this applies to your problem, from
>http://www.postfix.org/header_checks.5.html
>
> Message headers added by the cleanup(8) daemon itself are
>excluded from inspection. Examples of such message headers are From:,

>To:, Message-ID:, Date:.

Thank you, Christian!

But how to influence the way Cleanup is creating the From: header
for a particular user? Not to use the e-mail address
user...@hostname.domainname
but a specific one? And not to use the user full name from gecos field from /etc/passwd.

Regards,
Hans

[Wietse Venema]

Hans Ginzel:

Try using smtp_header_checks instead.

Wietse

[Viktor Dukhovni]

On Tue, Aug 25, 2015 at 10:30:07AM +0200, Hans Ginzel wrote:
> On Tue, Aug 25, 2015 at 09:37:37AM +0200, Christian Kivalo wrote:
> >Whats the content of your header_checks file?
>

> /^(From:.*)/i
> WARN header_checks: $1

Note that the "/i" turns off case-insensitive matching, which is
enabled by default.

--
Viktor.

[Hans Ginzel]

On Tue, Aug 25, 2015 at 06:24:12AM -0400, Wietse Venema wrote:
>Try using smtp_header_checks instead.

Thank you.

I have changed header_checks to smtp_header_checks in main.cf:

service postfix reload
postconf -n header_checks
postconf -n smtp_header_checks
smtp_header_checks = pcre:/etc/postfix/smtp_header_checks

cat /etc/postfix/smtp_header_checks
/^(.*)/ WARN SMTP_HC: $1

But in log there is only (begin of lines removed):

pickup[21474]: 5507980827: uid=494 from=<dw>
cleanup[22120]: 5507980827: message-id=<201508251246...@CZ00TDW001.localdomain>
qmgr[21473]: 5507980827: from=<d...@CZ00TDW001.localdomain>, size=489, nrcpt=1 (queue active)
local[22123]: 5507980827: to=<T91...@localhost.localdomain>, orig_to=<T912205@localhost>, relay=local, delay=0.15, delays=0.13/0/0/0.02, dsn=2.0.0, status=sent (delivered to mailbox)
qmgr[21473]: 5507980827: removed

I suppose that logging works well from the prev. posts.
Where is the problem that smtp_header_checks are not applied, please?
How to debug it?

Regards,
Hans

[Hans Ginzel]

On Tue, Aug 25, 2015 at 02:51:45PM +0200, Hans Ginzel wrote:
>Where is the problem that smtp_header_checks are not applied, please?

I think it is because of local delivery.
It seems smtp_header_checks are applied only for remote delivery.

How to “enable” them for local delivery, please?
Or how to influence header creation in Cleanup for a particular user?

Regards,
Hans

[Wietse Venema]

Hans Ginzel:

> local[22123]: 5507980827: to=<T91...@localhost.localdomain>, orig_to=<T912205@localhost>, relay=local, delay=0.15, delays=0.13/0/0/0.02, dsn=2.0.0, status=sent (delivered to mailbox)

This was delivered with the local(8) delivery agent.

> I suppose that logging works well from the prev. posts.

smtp_header_checks is an SMTP client feature.

Wietse

[Wietse Venema]

Hans Ginzel:

> On Tue, Aug 25, 2015 at 02:51:45PM +0200, Hans Ginzel wrote:
> >Where is the problem that smtp_header_checks are not applied, please?
>
> I think it is because of local delivery.
> It seems smtp_header_checks are applied only for remote delivery.
>

> How to ?enable? them for local delivery, please?

> Or how to influence header creation in Cleanup for a particular user?

Based on the tiny bits of information revealed sofar, I suggest
using non_smtpd_milters.

Had we known that the problem is in local submissions without From:
header, that could have saved a ton of time.

Wietse

[Hans Ginzel]

On Tue, Aug 25, 2015 at 09:51:50AM -0400, Wietse Venema wrote:
>Based on the tiny bits of information revealed sofar, I suggest
>using non_smtpd_milters.

Thank you.

>Had we known that the problem is in local submissions without From:
>header, that could have saved a ton of time.

I am sorry, but the issue was general – rewrite for both
remote and local delivery and for both MUAs
with and without From:.
Somthing we realised only “during the way”.

Thank you for what I could learn.

Best regards,
Hans

[Wietse Venema]

Hans Ginzel:

> On Tue, Aug 25, 2015 at 09:51:50AM -0400, Wietse Venema wrote:
> >Based on the tiny bits of information revealed sofar, I suggest
> >using non_smtpd_milters.
>
> Thank you.

If you need to rewrite headers in remote email, then you can achieve
that with smtpd_milters.

Milters receive the message content after Postfix has processed it,
including From: and other headers that were added by Postfix itself.

Milters can be implemented with Python, Perl, and (ugh) C.

Wietse

[Viktor Dukhovni]

On Tue, Aug 25, 2015 at 10:24:48AM -0400, Wietse Venema wrote:

> Hans Ginzel:
> > On Tue, Aug 25, 2015 at 09:51:50AM -0400, Wietse Venema wrote:
> > >Based on the tiny bits of information revealed sofar, I suggest
> > >using non_smtpd_milters.
> >
> > Thank you.
>
> If you need to rewrite headers in remote email, then you can achieve
> that with smtpd_milters.
>
> Milters receive the message content after Postfix has processed it,
> including From: and other headers that were added by Postfix itself.

It is also possible to introduce a "null" content-filter hop, and
then use either smtp_header_checks or header_checks via the
reinjection smtpd/cleanup service.

By a "null" content-filter I mean a configuration in which the
content_filter setting reinjects mail right back into Postfix (at
some loopback SMTP port othat than 25) without actually doing any
filtering (beyond smtp_header_checks and the like).

--
Viktor.

[Hans Ginzel]

On Tue, Aug 25, 2015 at 10:24:48AM -0400, Wietse Venema wrote:
>Milters receive the message content after Postfix has processed it,
>including From: and other headers that were added by Postfix itself.
>

>Milters can be implemented with Python, Perl, and (ugh) C.

Thank you!

I have decided to use the native smtp_header_checks for remote delivery only.

H.