Query "fragments"

[Yuri D'Elia]

So this is something I've come up to battle with missing prefix
matching, but turns out to be useful for other things as well:

;; setup query fragments
(setq mu4e-query-fragments
'(("%junk" . "maildir:/junk1 OR maildir:/junk2")
("%useless" . "%junk OR flag:trashed OR maildir:/folder3")
("%inbox" . "maildir:/inbox1 OR maildir:/inbox2")
("%work" . "maildir:/something/*")))

;; and then
(setq mu4e-bookmarks
'(("NOT %useless AND flag:unread" "Unread messages" ?u)
("NOT %useless AND flag:flagged" "Flagged messages" ?f)
("NOT %useless AND date:today..now" "Today's messages" ?t)
("NOT %useless AND date:7d..now" "Last 7 days" ?w)))

You can use a "fragment" in any query you make.
A fragment can be any valid xapian query.
Fragments are expanded as (fragment), so the precedence works as
intended. There's no need for extra parentheses.
Fragments can be nested, so they compose properly.

Fragments are *not* shown expanded. Since maildirs and expansions tend
to produce large queries, I don't want to see garbage in my modeline.
I test the expansion manually.

I implement them by using some 'advice, but I'd love this to go mainline.
If the idea is sound, I'll write the PR myself.

;; support for fragment expansion
(defun mu4e~query-fragment-expand-1 (frags str)
(let ((case-fold-search nil))
(replace-regexp-in-string
(regexp-opt (mapcar 'car frags) 'symbols)
(lambda (it) (cdr (assoc it frags)))
str t)))

(defun mu4e~query-fragment-expand (query)
(let (tmp (frags (mapcar (lambda (entry)
(cons (car entry) (concat "(" (cdr entry) ")")))
mu4e-query-fragments)))
;; expand recursively until nothing is substituted
(while (not (string-equal
(setq tmp (mu4e~query-fragment-expand-1 frags query))
query))
(setq query tmp)))
query)

(defun mu4e~proc-find-fragment-expand (args)
(let ((query (car args))
(rest (cdr args)))
(cons (mu4e~query-fragment-expand query) rest)))

(advice-add 'mu4e~proc-find :filter-args 'mu4e~proc-find-fragment-expand)

[Dirk-Jan C. Binnema]

Hi Yuri,

Hmmm - it's an interesting experiment, but to be honest, I'm a bit wary
of adding query pre-processing... There are some improvement possible to
the Xapian query language, but I think those are best addressed with a
custom query parser, on the mu (not mu4e) level.

I think the main practical problem this comes from is the long queries
in the modeline; that's a valid concern, but I think we can handle that
separate from this, perhaps by allowing for a rewriter hook for the
modeline string.

Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Yuri D'Elia]

On Sat, Feb 20 2016, Dirk-Jan C. Binnema <dj...@djcbsoftware.nl> wrote:
> Hmmm - it's an interesting experiment, but to be honest, I'm a bit wary
> of adding query pre-processing... There are some improvement possible to
> the Xapian query language, but I think those are best addressed with a
> custom query parser, on the mu (not mu4e) level.

True. You can obviously see how you can break the substitution as
there's no parsing involved.

As I said, my main gripe was mostly due to the lack of prefix matching,
and mostly for maildir:. As I have multiple accounts with the same
structure, I have to repeat all maildirs whereas I could just use one
pattern.

That being said, after toying with it for a while, I definitely like
being able to define fragments I can use in any query. I have several I
didn't post that select things I look for recurringly.

The fact that this is expanded under-the-hood not only makes the query
instantly readable, but also easily *editable*.

As suggested here, I initially started by defining many bookmarks with
pre-baked queries, but this resulted in too many bookmarks and very long
queries which weren't easy to edit. That wasn't pleasant.

In any case, it has been posted for those who'd like to try.

> I think the main practical problem this comes from is the long queries
> in the modeline; that's a valid concern, but I think we can handle that
> separate from this, perhaps by allowing for a rewriter hook for the
> modeline string.

Compacting the modeline if overlong would be helpful irregardless of
this. Even just ellipsizing the query when exceeding the visual width of
the modeline would be good start (right now the current focus is just
pushed out of view).