"jump" to list-id

[Yuri D'Elia]

Another small hack worth sharing. I'm subscribed to a ton of lists,
which I hide by default in my queries.

However, when I want to browse a specific list, I've found that
searching for the list-id is annoying. For that, I added a new "l"
binding (for "list") in the headers view that shows me a list of recent
list-id that I can look at, similar to the jump-to-maildir command.

Underlying I use "mu find" to extract and process the list-ids, then use
ivy-read to have some fuzzyness while searching. Customize as needed.

(setq my4e~query-lists-command
"mu find -f v flag:list date:1y.. | sed -e '/^$/d' | sort | uniq -c | sort -rn | awk -- '{ if($1 > 3) print $2 }'")

(defun my4e~headers-ask-for-list ()
(let* ((output (shell-command-to-string my4e~query-lists-command))
(lists (split-string output "\n")))
(ivy-read "[mu4e] Jump to list: " lists)))

(defun my4e-headers-jump-to-list (listid)
(interactive
(let ((listid (my4e~headers-ask-for-list)))
(list listid)))
(when listid
(mu4e-mark-handle-when-leaving)
(mu4e-headers-search (format "list:\"%s\"" listid))))

(define-key mu4e-headers-mode-map (kbd "l") 'my4e-headers-jump-to-list)

[Dirk-Jan C. Binnema]

Cool hack! Thanks for sharing.

Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Yuri D'Elia]

On Tue, Dec 27 2016, Dirk-Jan C. Binnema wrote:
>> (define-key mu4e-headers-mode-map (kbd "l") 'my4e-headers-jump-to-list)
>
> Cool hack! Thanks for sharing.

Note that, even though I weed lists by frequency (my cutoff is actually
16) and offer them in order (so that the biggest lists are at the top),
there's a _ton_ of cruft in there.

The list-id is also one of those fields where prefix wildcards would
help, as the domain is at the end.

I think I've asked this in the past, but I'll do it again: why not
duplicate the fields where prefixed wildcards make sense (most headers?)
in a reversed field, and handle prefix matches ourselves?

Is there a way to calculate how much space we would waste in the xapian
DB excluding message bodies? I would kill^H^H^H^Hcode for it.

[Stig Brautaset]

Neat! I really like this. The only thing I don't like is that it's not
appearing in the menu, so I wanted to see if I could work this into a
bookmark:

(add-to-list 'mu4e-bookmarks
(make-mu4e-bookmark
:name "Query For List"
:query (lambda ()
(concat "list:" (my4e~headers-ask-for-list)))
:key ?l))

but it turns out the bookmark queries have to be strings and you can't
use a lambda there. Dirk-Jan, would you be open to a PR that allows
"dynamic" queries for bookmarks?

Stig

[Dirk-Jan C. Binnema]

Hi Yuri,

It's not setting a flag or something, it would require a fairly gross
hack to trick Xapian to support it; we'd need to manipulate database
insertion, as well as massage the queries.

From earlier experiences, that is all rather fragile. With a custom
query parser, it might become a bit easier, so once that is in, we could
take another look at this.

For the specific case of mailing-lists, it's probably easier to add
specific completion for that.

See:
https://lists.xapian.org/pipermail/xapian-discuss/2005-October/001210.html

[Yuri D'Elia]

On Fri, Dec 30 2016, Dirk-Jan C. Binnema wrote:
>> Is there a way to calculate how much space we would waste in the xapian
>> DB excluding message bodies? I would kill^H^H^H^Hcode for it.
>
> It's not setting a flag or something, it would require a fairly gross
> hack to trick Xapian to support it; we'd need to manipulate database
> insertion, as well as massage the queries.

Yes, hacky, but I did use xapian this way in the past. As I built
the queries internally, there was no need to parse anything though.

> From earlier experiences, that is all rather fragile. With a custom
> query parser, it might become a bit easier, so once that is in, we could
> take another look at this.

By the way, is there a way to supply a xapian expression as a single
argument to "mu find"?

When I call find from another program, I actually have to tokenize the
string into arguments, which is suboptimal. It would be nicer if find
itself would just concat the arguments internally and re-tokenize the
string anyway, so that there's no actual difference.

> For the specific case of mailing-lists, it's probably easier to add
> specific completion for that.
>
> See:
> https://lists.xapian.org/pipermail/xapian-discuss/2005-October/001210.html

Heh, I was aware of this, unfortunately. But contrarily to what's
written there, prefix wildcards would be useful also in many places
(german comes to mind). I also use xapian for indexing local
documentation (via "recoll") and I've been bitten countless times with
technical terms and function names.

I wouldn't like, at least technically, to add exceptions to specific
cases such as the list-id.

But re-implementing a custom parser is also annoying.

It's a shame really, as I like the versatility of the query syntax of
xapian itself. I don't think there are other engines with comparable
features out there [?].

[Dirk-Jan C. Binnema]

On Friday Dec 30 2016, Yuri D'Elia wrote:

> By the way, is there a way to supply a xapian expression as a single
> argument to "mu find"?

Not directly; perhaps we can add something for that.

>> See:
>> https://lists.xapian.org/pipermail/xapian-discuss/2005-October/001210.html
>
> Heh, I was aware of this, unfortunately. But contrarily to what's
> written there, prefix wildcards would be useful also in many places
> (german comes to mind). I also use xapian for indexing local
> documentation (via "recoll") and I've been bitten countless times with
> technical terms and function names.

Yeah, they could be quite useful. As would regexp search etc... but
Xapian wasn't really built for that.

> I wouldn't like, at least technically, to add exceptions to specific
> cases such as the list-id.

We already do it for contacts; personally I just receive all mailing
lists in one folder, and then use refiling to archive the few message I
want to keep for later reference. But yeah, different people, different
workflows.

The :mailing-list headers field is also quite useful for keeping things
manageable.

> But re-implementing a custom parser is also annoying.

It's not too bad - I proto-typed one as part of 'mux'. Hopefully in 2017
that becomes something useful.

> It's a shame really, as I like the versatility of the query syntax of
> xapian itself. I don't think there are other engines with comparable
> features out there [?].

There's eg. Lucene, and even sqlite has so full-text capabilities these
days; however, I have no hands-on experience with either.

[Yuri D'Elia]

On Sat, Dec 31 2016, Dirk-Jan C. Binnema wrote:
>> By the way, is there a way to supply a xapian expression as a single
>> argument to "mu find"?
>
> Not directly; perhaps we can add something for that.

I filed an issue (#999).

> We already do it for contacts; personally I just receive all mailing
> lists in one folder, and then use refiling to archive the few message I
> want to keep for later reference. But yeah, different people, different
> workflows.

I similarly do that, but with large volume lists mixed with low volume
ones it's hard to follow everything. In fact, above a certain volume,
it's just inconvenient. I unsubscribe and re-subscribe the list to gmane.

Admittedly, I'm still using gnus for those.

With "notmuch" I could implement some sort of "kill threads" feature by
tagging messages and then back-propagating them with a script with an
after-fetch script.

This is the main reason I was asking for custom flags/tags earlier on,
even if this meant that they are client-only. Combined with some
scripting, which could be either batch or local to the client, you can
reimplement some neat features easily.

What I didn't like in notmuch is the emacs/mail interface. There are
some neat ideas in there, but it's a bit too geared towards tags.

> There's eg. Lucene, and even sqlite has so full-text capabilities these
> days; however, I have no hands-on experience with either.

Lucene is a big dependency.

sqlite would be interesting, but I'm unsure about it's scalability.
But it would make a compelling arguments for message metadata.

[Dirk-Jan C. Binnema]

On Saturday Dec 31 2016, Yuri D'Elia wrote:

> With "notmuch" I could implement some sort of "kill threads" feature by
> tagging messages and then back-propagating them with a script with an
> after-fetch script.
>
> This is the main reason I was asking for custom flags/tags earlier on,
> even if this meant that they are client-only. Combined with some
> scripting, which could be either batch or local to the client, you can
> reimplement some neat features easily.

I’d like to improve scriptability a bit, so you could manipulate things
from mu4e directly without having to implement everything in the
backend.

>> There's eg. Lucene, and even sqlite has so full-text capabilities these
>> days; however, I have no hands-on experience with either.
>
> Lucene is a big dependency.
>
> sqlite would be interesting, but I'm unsure about it's scalability.
> But it would make a compelling arguments for message metadata.

Interestingly, prehistoric versions of mu used a combined sqlite/xapian
approach; but querying became a bit nightmarish.