Mails with responses don't appear on Alaveteli

[Andrei Cristian Petcu]

Hi,

Some user of nuvasuparati.info got the answer for a FOI request on their
personal email. The requester told me to mark the request as answered. I
wanted to manually send the response to the FOI email so I sent a new
mail from and...@nuvasuparati.info to
foi+request-...@nuvasuparati.info with the answer in the
attachment. It did not appear on the web site,

http://nuvasuparati.info/request/raportul_privind_execuia_bugetar

Here is a part of the log and a print screen with the admin page.
https://my.owndrive.com/index.php/s/MNJg7Du6uFxCJlb

How can I troubleshoot this?

Thank you,
Andrei Petcu

[Henare Degan]

On 17 January 2016 at 02:32, Andrei Cristian Petcu <and...@ceata.org> wrote:

How can I troubleshoot this?

That error 75 was raised by the mailin script: https://github.com/mysociety/alaveteli/blob/develop/script/mailin#L30

As you can see in that script, it should have emailed full details of the error to the address set in the FORWARD_NONBOUNCE_RESPONSES_TO option.

Cheers,

Henare

--
Henare Degan
hen...@oaf.org.au
Volunteer & worker - OpenAustralia Foundation - @OpenAustralia

 

[Andrei Cristian Petcu]

Hi Henare,

I changed FORWARD_NONBOUNCE_RESPONSES_TO to and...@nuvasuparati.info
which is an alias and I should be getting the mail.

I keep getting these messages

From: Mailer...@nuvasuparati.info
To: and...@nuvasuparati.info
Subject: Warning: message 1aKRc1-00078W-VL delayed 48 hours
Body:
This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 48 hours on the queue on nuvasuparati.info.
The message identifier is: 1aKRc1-00078W-VL
The subject of the message is: Raspuns Solicitare
The date of the message is: Sat, 16 Jan 2016 16:12:26 +0200
The address to which the message has not yet been delivered is:
pipe to |/var/www/alaveteli/script/mailin
generated by foi+request-...@nuvasuparati.info
Delay reason: local delivery failed
No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

How do I debug the mailin script? I want to manually run this script
with my params so I can understand what's going on but I don't
understand what are these params that I have to run the message with.

Thank you,
Andrei Petcu

On 01/18/2016 01:25 AM, Henare Degan wrote:
> On 17 January 2016 at 02:32, Andrei Cristian Petcu <and...@ceata.org

> <mailto:and...@ceata.org>> wrote:
>
> How can I troubleshoot this?
>
>
> That error 75 was raised by the mailin script:
> https://github.com/mysociety/alaveteli/blob/develop/script/mailin#L30
>
> As you can see in that script, it should have emailed full details of
> the error to the address set in the FORWARD_NONBOUNCE_RESPONSES_TO option.
>
> Cheers,
>
> Henare
>
> --
> Henare Degan

> hen...@oaf.org.au <mailto:hen...@oaf.org.au>
> Volunteer & worker - OpenAustralia Foundation <http://www.oaf.org.au/> -
> @OpenAustralia <https://twitter.com/OpenAustralia>
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "Alaveteli Dev" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to alaveteli-de...@googlegroups.com
> <mailto:alaveteli-de...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

[Henare Degan]

Hi Andrei,

On 20 January 2016 at 08:51, Andrei Cristian Petcu <and...@ceata.org> wrote:

Hi Henare,

I changed FORWARD_NONBOUNCE_RESPONSES_TO to and...@nuvasuparati.info
which is an alias and I should be getting the mail.

So you should now get detailed information on the problem with the mailin script sent to that address.
 

I keep getting these messages

From: Mailer...@nuvasuparati.info
To: and...@nuvasuparati.info
Subject: Warning: message 1aKRc1-00078W-VL delayed 48 hours
Body:
This message was created automatically by mail delivery software.
A message that you sent has not yet been delivered to one or more of its
recipients after more than 48 hours on the queue on nuvasuparati.info.
The message identifier is:     1aKRc1-00078W-VL
The subject of the message is: Raspuns Solicitare
The date of the message is:    Sat, 16 Jan 2016 16:12:26 +0200
The address to which the message has not yet been delivered is:
  pipe to |/var/www/alaveteli/script/mailin
    generated by foi+request-...@nuvasuparati.info
    Delay reason: local delivery failed
No action is required on your part. Delivery attempts will continue for
some time, and this warning may be repeated at intervals if the message
remains undelivered. Eventually the mail delivery software will give up,
and when that happens, the message will be returned to you.

How do I debug the mailin script? I want to manually run this script
with my params so I can understand what's going on but I don't
understand what are these params that I have to run the message with.

Yes, that's the mail server attempting to resend to Alaveteli. That should then result in the mailin script failing and sending the above address a detailed problem report to allow you to debug the problem.

Cheers,

Henare

[Andrei Cristian Petcu]

Hi all,

Sorry for being so persistent but this problem really bugs me.

So from what I understand the flow is:

1) foreign SMTP server -> 2) my exim4 -> 3)
/etc/exim4/conf.d/transport/04_alaveteli -> 4)
/var/www/alaveteli/script/mailin

So the problem is at 4). If I want to debug it, I need to figure out
what are the exact parameters mailin is invoked with.

I checked the email log with less /var/log/exim4/exim-mainlog-20160120
and this is the result: https://my.owndrive.com/index.php/s/DwM7qroY3C4fHJY

I want to fetch the email body and somehow pipe it trough the script and
see how it breaks but to my shame I don't know how to get a correct
email that I should pipe to mailin. Do I need the headers or not? The
email I got does not seem complete. I guess the full email is somewhere
in the log files.

I started getting the error email messages after I changed the
FORWARD_NONBOUNCE_RESPONSES_TO vairable. Here is one of them:
https://my.owndrive.com/index.php/s/Fg2c311cuFhK7sD

Thank you,
Andrei

On 01/20/2016 02:09 AM, Henare Degan wrote:
> Hi Andrei,
>
> On 20 January 2016 at 08:51, Andrei Cristian Petcu <and...@ceata.org

> <mailto:and...@ceata.org>> wrote:
>
> Hi Henare,
>
> I changed FORWARD_NONBOUNCE_RESPONSES_TO to and...@nuvasuparati.info

> <mailto:and...@nuvasuparati.info>

> which is an alias and I should be getting the mail.
>
>
> So you should now get detailed information on the problem with the
> mailin script sent to that address.
>
>
> I keep getting these messages
>
> From: Mailer...@nuvasuparati.info
> <mailto:Mailer...@nuvasuparati.info>
> To: and...@nuvasuparati.info <mailto:and...@nuvasuparati.info>
> Subject: Warning: message 1aKRc1-00078W-VL delayed 48 hours
> Body:
> This message was created automatically by mail delivery software.
> A message that you sent has not yet been delivered to one or more of its
> recipients after more than 48 hours on the queue on
> nuvasuparati.info <http://nuvasuparati.info>.
> The message identifier is: 1aKRc1-00078W-VL
> The subject of the message is: Raspuns Solicitare
> The date of the message is: Sat, 16 Jan 2016 16:12:26 +0200
> The address to which the message has not yet been delivered is:
> pipe to |/var/www/alaveteli/script/mailin
> generated by foi+request-...@nuvasuparati.info

> <mailto:foi%2Brequest-3...@nuvasuparati.info>

> Delay reason: local delivery failed
> No action is required on your part. Delivery attempts will continue for
> some time, and this warning may be repeated at intervals if the message
> remains undelivered. Eventually the mail delivery software will give up,
> and when that happens, the message will be returned to you.
>
>
> How do I debug the mailin script? I want to manually run this script
> with my params so I can understand what's going on but I don't
> understand what are these params that I have to run the message with.
>
>
> Yes, that's the mail server attempting to resend to Alaveteli. That
> should then result in the mailin script failing and sending the above
> address a detailed problem report to allow you to debug the problem.
>
> Cheers,
>
> Henare
>

[Henare Degan]

On 21 January 2016 at 06:58, Andrei Cristian Petcu <and...@ceata.org> wrote:

Hi all,

Sorry for being so persistent but this problem really bugs me.

It's good - you need to fix it :)
 

So from what I understand the flow is:

1) foreign SMTP server -> 2) my exim4 -> 3)
/etc/exim4/conf.d/transport/04_alaveteli -> 4)
/var/www/alaveteli/script/mailin

So the problem is at 4). If I want to debug it, I need to figure out
what are the exact parameters mailin is invoked with.

I checked the email log with less /var/log/exim4/exim-mainlog-20160120
and this is the result: https://my.owndrive.com/index.php/s/DwM7qroY3C4fHJY

I want to fetch the email body and somehow pipe it trough the script and
see how it breaks but to my shame I don't know how to get a correct
email that I should pipe to mailin. Do I need the headers or not? The
email I got does not seem complete. I guess the full email is somewhere
in the log files.

That's my understanding. Unfortunately I'm not sure how to fetch the email either - can someone else on the list help?

I started getting the error email messages after I changed the
FORWARD_NONBOUNCE_RESPONSES_TO vairable. Here is one of them:
https://my.owndrive.com/index.php/s/Fg2c311cuFhK7sD

That's a bounce message, not the debugging message that gets sent to FORWARD_NONBOUNCE_RESPONSES_TO that would help with this problem.

You can see the mailin script generates these debugging messages here: https://github.com/mysociety/alaveteli/blob/develop/script/mailin#L22-L26

Cheers,

Henare

[Louise Crow]

Hi,

So it looks to me like the mailin script is hitting an error and therefore returning the deferral code '75' to exim, but is not sending on the details of the error to you - I wonder if that's because you don't have mutt installed, or it's not working correctly. As you can see from the script, in the case of an error, it uses the mutt executable to send the error email - https://github.com/mysociety/alaveteli/blob/develop/script/mailin#L26. For viewing the raw email and piping it into the script, I tend to use the 'Show original' option in gmail. Maybe there's some equivalent option in your mail client (might just be for incoming mail, so you might need to send it to yourself first and then edit the 'To' header to put in the request address before piping it into the mailin script.  The mail does need to have at least some of the headers intact (the "Received" headers aren't needed). For debugging the script, I would comment out the line "rm -f "$INPUT" "$OUTPUT" from the path where there's an error, (https://github.com/mysociety/alaveteli/blob/develop/script/mailin#L29) and then take a look at the temporary files created. They should show you the content of the mail and the content of the error message from Alaveteli. Note that if mailin itself generates any output, incoming messages will bounce back to the sender.

Hope that's useful

Louise

[Andrei Cristian Petcu]

Hi,

I managed to see the full text that is piped to the mailin script.

I started debugging it but it will take time until I figure out what is
the problem.

Thank you both for your help,
Andrei

[Andrei Cristian Petcu]

Hi again,

So I started to debug this issue. But not much luck. I want to run it
from the command line, not by email.

So I ran this as www-data
$ cat /tmp/foi-mailin-mail-VHEbv7D5 | ./mailin

From what I could figure out the mailin script gets it's input from the
previous process trough stdin over a pipe. If I run the above command I
get the email in the holding pen. This should be ok since I'm sending it
from an unknown email instead of the authority or the FOI requester's
email. The strange thing is that I see no body or subject to this email.

Here are the raw content of the email and a print screen of the admin
interface with the holding pen issue.

https://my.owndrive.com/index.php/s/bmpg4dX4YOgQ4Lf

Any input on this issue is welcome.

Andrei

[Andrei Cristian Petcu]

I think the mailin script works now.

Apparently while troubleshooting I broke the mailin script.

Now I got 3 messages
http://nuvasuparati.info/request/raportul_privind_execuia_bugetar
unfortunately I cannot delete 2 of them since I get an error when I try.

It is still strange that the mail initially did not get picked up by
Alaveteli. I'll park it for now and work on the Alaveteli update and
hope it either does not happen on the new instance or it will be easier
to troubleshoot. We have a small number of requests/month and I hope
this will change after bulk update will land,

Thank you for helping me on this
Andrei

[Louise Crow]

Hi Andrei,

I think it's a good idea to see what issues remain after upgrade - you're currently on a quite old version of Alaveteli (0.6.5, I think).

Cheers

Louise

To unsubscribe from this group and stop receiving emails from it, send an email to alaveteli-de...@googlegroups.com.