Using net/mail and reconstituting mail headers in order

[Mark Fletcher]

I'm using the net/mail package to parse/manipulate email messages. I would like to be able to regenerate the emails with the headers in the original order, but because mail uses a map to store the header lines, this isn't possible. Short of writing my own email parsing package, does anyone have any ideas for how I could augment net/mail so that I can get the headers back in order? I've racked my brain (didn't take long), and couldn't think of any way to do so.

Thanks,

Mark

[Giulio Iotti]

On Thursday, October 29, 2015 at 6:52:44 PM UTC+2, Mark Fletcher wrote:

I'm using the net/mail package to parse/manipulate email messages. I would like to be able to regenerate the emails with the headers in the original order, but because mail uses a map to store the header lines, this isn't possible. Short of writing my own email parsing package, does anyone have any ideas for how I could augment net/mail so that I can get the headers back in order? I've racked my brain (didn't take long), and couldn't think of any way to do so.

If you really need to do this (because the order doesn't really matter, why do you want to keep the order?) I can suggest two options:

1. As you say, implement the parser. It's actually quite simple: look at the source of net/textproto.ReadMIMEHeader (it uses a couple of unexported methods that actually have exported counterparts. Change the code to use a map, and you're done.

2. Prepend all header lines (lines before an empty line) that do not start with a space to become like X-{number}-Header (like this: fmt.Sprintf("X-%d-%s", cnt, line); cnt++). Now parse normally and the headers are easy to sort. Note however that the parse does make the header keys canonical, and if you use this trick you need to make them canonical yourself after removing "X-{number}-".

Does this help?

-- 

Giulio Iotti

[James Bardin]

On Thursday, October 29, 2015 at 1:28:21 PM UTC-4, Giulio Iotti wrote:

On Thursday, October 29, 2015 at 6:52:44 PM UTC+2, Mark Fletcher wrote:

I'm using the net/mail package to parse/manipulate email messages. I would like to be able to regenerate the emails with the headers in the original order, but because mail uses a map to store the header lines, this isn't possible. Short of writing my own email parsing package, does anyone have any ideas for how I could augment net/mail so that I can get the headers back in order? I've racked my brain (didn't take long), and couldn't think of any way to do so.

If you really need to do this (because the order doesn't really matter, why do you want to keep the order?) I can suggest two options:

IIRC the RFC specifies that general headers SHOULD NOT be reordered, and some headers specifically MUST NOT be reordered. 

(well, I'm this far, might as well look it up ;) ...) 

RFC5322 3.6

 

> It is important to note that the header fields are not guaranteed to be in a particular order. They may appear in any order, and they have been known to be reordered occasionally when transported over the Internet. However, for the purposes of this specification, header fields SHOULD NOT be reordered when a message is transported or transformed. More importantly, the trace header fields and resent header fields MUST NOT be reordered, and SHOULD be kept in blocks prepended to the message.

[Giulio Iotti]

On Thursday, October 29, 2015 at 7:44:37 PM UTC+2, James Bardin wrote:

> It is important to note that the header fields are not guaranteed to be in a particular order. They may appear in any order, and they have been known to be reordered occasionally when transported over the Internet. However, for the purposes of this specification, header fields SHOULD NOT be reordered when a message is transported or transformed. More importantly, the trace header fields and resent header fields MUST NOT be reordered, and SHOULD be kept in blocks prepended to the message.

Thanks, nice hint. The part about the trace headers makes it quite explicit why reordering is bad (however, they will result in one key an multiple values in the values array, in order).

I am thinking that if you just want to modify a message (add a trace header and pass it on), I'd wrap the reader to read me header from when some position is reached on the underlying reader.

If I want to display a message, I display certain headers only and the original order doesn't matter.

-- 

Giulio Iotti

[Mark Fletcher]

On Thu, Oct 29, 2015 at 10:28 AM, Giulio Iotti <dullg...@gmail.com> wrote:

1. As you say, implement the parser. It's actually quite simple: look at the source of net/textproto.ReadMIMEHeader (it uses a couple of unexported methods that actually have exported counterparts. Change the code to use a map, and you're done.

2. Prepend all header lines (lines before an empty line) that do not start with a space to become like X-{number}-Header (like this: fmt.Sprintf("X-%d-%s", cnt, line); cnt++). Now parse normally and the headers are easy to sort. Note however that the parse does make the header keys canonical, and if you use this trick you need to make them canonical yourself after removing "X-{number}-".

Thanks for the suggestions, I appreciate it. It confirms that I wasn’t missing an obvious way to do this. I guess I’m off to implement the parser. 

Thanks,

Mark