using eww/shr to view rich text messages

[Dirk-Jan C. Binnema]

Hi all,

I've added a little snippet in `mu4e-contrib' to make using the shr html
renderer to view rich-text messages easy. It's based on the snippet by
Titus von der Malsburg posted on this very list.

With mu4e from git, all that's needed is adding the following to your
configuration.

--8<---------------cut here---------------start------------->8---
(require 'mu4e-contrib)
(setq mu4e-html2text-command 'mu4e-shr2text)
--8<---------------cut here---------------end--------------->8---

This works with emacs 24.4 (or older versions if you install shr
yourself) and with mu4e from git.

Cheers,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C
1

[Ondra Kudlík]

Hi Dirk,

finally found time to try it and its amazing! Alongside with already working GPG I think it is good time to release new version (so distributions can create new packages).

Do you have any good resources on how to change/handle colors with this mu4e-shr2text? About 50 % of mails I receive have gray background with gray (only little brighter) font.

Thanks

--
You received this message because you are subscribed to the Google Groups "mu-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mu-discuss+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dirk-Jan C. Binnema]

On Monday Nov 10 2014, Ondra Kudlík wrote:

> Hi Dirk,
>
> finally found time to try it and its amazing! Alongside with already
> working GPG I think it is good time to release new version (so
> distributions can create new packages).
>
> Do you have any good resources on how to change/handle colors with this
> mu4e-shr2text? About 50 % of mails I receive have gray background with gray
> (only little brighter) font.

You can customize eww/shr; there's variables like:
shr-color-visible-luminance-min
shr-color-visible-distance-min
(at least in the latest versions) which should be useful for this; check
the eww docs for details.

[Ondra Kudlík]

Thanks, works like a charm! I had problem identifying what documentation to check.

Kepi

[Alasdair Muckart]

On Sunday, 2 November 2014 00:01:51 UTC+13, djcb wrote:

Hi all,

I've added a little snippet in `mu4e-contrib' to make using the shr html
renderer to view rich-text messages easy. It's based on the snippet by
Titus von der Malsburg posted on this very list.

With mu4e from git, all that's needed is adding the following to your
configuration.

--8<---------------cut here---------------start------------->8---
 (require 'mu4e-contrib)
 (setq mu4e-html2text-command 'mu4e-shr2text)
--8<---------------cut here---------------end--------------->8---

This works with emacs 24.4 (or older versions if you install shr
yourself) and with mu4e from git.

I'm running 24.4 and mu4e from git on OS X 10.10 and this is reliably hanging my emacs session when I view certain emails, requiring it to be force killed. 

If I change to using OS X textutil it gives me a slightly useless rendering but doesn't cause emacs to hang.

It seems to be trying to open an https session and I'm getting an error in the minibuffer "eeror in process filter: Symbol's value as variable is void: url-http-response-status

If I open the messages buffer before running mu4e I can see that it is attempting to open a TLS connection to api.inktad.com but this is failing. I can kill the gnutls sessions from terminal but that doesn't un-hang emacs. 

It looks like it trys with gnutls-cli a few times and eventually falls back to openssl. 

I'm a little worried that viewing an HTML email without following any links in it or anything is causing mu4e to follow links or connections in that email without any user intervention. Is there a way to prevent that from happening? 

Unfortunately the email in question is a flight itinerary that has a load of my personal information in it or I'd attach a copy of it here. 

Thanks.

[Titus von der Malsburg]

On 2014-11-13 Thu 23:07, Alasdair Muckart wrote:
> I'm running 24.4 and mu4e from git on OS X 10.10 and this is reliably
> hanging my emacs session when I view certain emails, requiring it to be
> force killed.
>
> If I change to using OS X textutil it gives me a slightly useless rendering
> but doesn't cause emacs to hang.
>
> It seems to be trying to open an https session and I'm getting an error in
> the minibuffer "eeror in process filter: Symbol's value as variable is
> void: url-http-response-status
>
> If I open the messages buffer before running mu4e I can see that it is
> attempting to open a TLS connection to api.inktad.com but this is failing.
> I can kill the gnutls sessions from terminal but that doesn't un-hang
> emacs.
>
> It looks like it trys with gnutls-cli a few times and eventually falls back
> to openssl.
>
> I'm a little worried that viewing an HTML email without following any links
> in it or anything is causing mu4e to follow links or connections in that
> email without any user intervention. Is there a way to prevent that from
> happening?
>
> Unfortunately the email in question is a flight itinerary that has a load
> of my personal information in it or I'd attach a copy of it here.

I experienced hangs and overall poor performance in earlier versions of
shr. However, this seems to have improved a lot more recently since
shr and eww are under heavy development. With the current version of
Emacs, I didn't experience any hangs and render times are much faster
now. So perhaps try to update Emacs. If the problem persists, you
could file a bug with Emacs.

The second issue is that shr may retrieve data from untrusted URLs which
is bad for a number of reasons. I don't know whether it's possible to
bar shr from making connections but that would be a very useful feature,
also in other contexts. So that may be a good feature request.

Titus

[Tamas Papp]

On Fri, Nov 14 2014, Titus von der Malsburg <mals...@posteo.de> wrote:

> The second issue is that shr may retrieve data from untrusted URLs which
> is bad for a number of reasons. I don't know whether it's possible to
> bar shr from making connections but that would be a very useful feature,
> also in other contexts. So that may be a good feature request.

That's a good point that I haven't thought of --- I looked at the eww
manual and did not find anything, maybe someone could ask in the Emacs
newsgroup.

Best,

Tamas

[Ken Mankoff]

(setq shr-inhibit-images t) turns off eww images.

-k.

[Alasdair Muckart]

Ken Mankoff <man...@gmail.com> @ 2014-11-15 10:48 NZDT:

Yep, I've already set that, but this isn't downloading images. It
appears to be downloading tracker cookies and javascript.

Thanks.
--
Alasdair Muckart | William de Wyke | http://wherearetheelves.net
"There is hardly anything in the world that some man cannot make a little worse
and sell a little cheaper, and the people who consider price only are this man's
lawful prey." - John Ruskin, 1819-1900.

[Alasdair Muckart]

Titus von der Malsburg <mals...@posteo.de> @ 2014-11-15 07:17 NZDT:

> I experienced hangs and overall poor performance in earlier versions of
> shr. However, this seems to have improved a lot more recently since
> shr and eww are under heavy development. With the current version of
> Emacs, I didn't experience any hangs and render times are much faster
> now. So perhaps try to update Emacs. If the problem persists, you
> could file a bug with Emacs.

I'm aready on the latest stable Emacs.

> The second issue is that shr may retrieve data from untrusted URLs which
> is bad for a number of reasons. I don't know whether it's possible to
> bar shr from making connections but that would be a very useful feature,
> also in other contexts. So that may be a good feature request.

I think the issue is that shr doesn't know when it's being called to
render a webpage or render an HTML email. Those are very different use
cases, in one I want the images etc. but in the other I don't. Likewise
javascript and so on.

[Dirk-Jan C. Binnema]

On Saturday Nov 15 2014, Alasdair Muckart wrote:

> I think the issue is that shr doesn't know when it's being called to
> render a webpage or render an HTML email. Those are very different use
> cases, in one I want the images etc. but in the other I don't. Likewise
> javascript and so on.

You can advise (or replace) `mu4e-shr2text', and let-bind the settings
you want.

[Titus von der Malsburg]

On 2014-11-14 Fri 19:19, Alasdair Muckart wrote:
> I think the issue is that shr doesn't know when it's being called to
> render a webpage or render an HTML email. Those are very different use
> cases, in one I want the images etc. but in the other I don't. Likewise
> javascript and so on.

You can use the definition of mu4e-html2text given below. This inhibits
images when rendering HTML emails but does not affect eww. I submitted
a pull-request for that.

Titus

(defun mu4e-shr2text ()
"Html to text using the shr engine; this can be used in
`mu4e-html2text-command' in a new enough emacs. Based on code by
Titus von der Malsburg."
(interactive)
(let ((dom (libxml-parse-html-region (point-min) (point-max)))
(shr-inhibit-images t))
(erase-buffer)
(shr-insert-document dom)
(goto-char (point-min))))

[Titus von der Malsburg]

On 2014-11-14 Fri 19:17, Alasdair Muckart wrote:
>>> I'm a little worried that viewing an HTML email without following any
>>> links in it or anything is causing mu4e to follow links or connections
>>> in that email without any user intervention. Is there a way to prevent
>>> that from happening?
>>
>> (setq shr-inhibit-images t) turns off eww images.
>
> Yep, I've already set that, but this isn't downloading images. It
> appears to be downloading tracker cookies and javascript.

I doubt that. Cookies are not retrieved via separate connections and
eww/shr do not support JavaScript to my knowledge (I think RMS would not
allow this to happen). It would be really helpful if you could provide
a sample email. Just redact the parts that contain sensitive
information.

Titus

[Titus von der Malsburg]

On 2014-11-14 Fri 19:17, Alasdair Muckart wrote:

> Ken Mankoff <man...@gmail.com> @ 2014-11-15 10:48 NZDT:
>> * On 2014-11-14 at 02:07, Alasdair Muckart wrote:
>>> On Sunday, 2 November 2014 00:01:51 UTC+13, djcb wrote:
>>
>>> I'm a little worried that viewing an HTML email without following any
>>> links in it or anything is causing mu4e to follow links or connections
>>> in that email without any user intervention. Is there a way to prevent
>>> that from happening?
>>
>> (setq shr-inhibit-images t) turns off eww images.
>
> Yep, I've already set that, but this isn't downloading images. It
> appears to be downloading tracker cookies and javascript.

Another possibility would have been that shr retrieves style
sheets. However, I just checked and it doesn't do that. This is also
consistent with some (admittedly not very recent) discussions that I
found in the relevant mailing lists: shr seems to have very basic
support for style sheets but does not attempt to retrieve separate style
sheets files.

Titus

[Alasdair Muckart]

Titus von der Malsburg <mals...@posteo.de> @ 2014-11-16 07:05 NZDT:

> You can use the definition of mu4e-html2text given below. This inhibits
> images when rendering HTML emails but does not affect eww. I submitted
> a pull-request for that.
>
> Titus
>
> (defun mu4e-shr2text ()
> "Html to text using the shr engine; this can be used in
> `mu4e-html2text-command' in a new enough emacs. Based on code by
> Titus von der Malsburg."
> (interactive)
> (let ((dom (libxml-parse-html-region (point-min) (point-max)))
> (shr-inhibit-images t))
> (erase-buffer)
> (shr-insert-document dom)
> (goto-char (point-min))))

I only vaguely understand the code, but it seems to work, thank you.

The HTML email in question was formed as a bunch of base64 encoded mime
parts for some reason rather than a readable HTML body part.

I got the HTML out of a debug buffer and the URLs that were hanging
emacs when mu4e opened the message were all tracking images from
analytics sites.

Thanks.