mu4e: build complex queries from building blocks

[Matt Price]

I've just started using Nicolas Rougier's mu4e-dashboard and have been enjoying it quite a bit (https://github.com/rougier/mu4e-dashboard/). As I develop my own personal dashboard, I found myself writing quite a few complex and often repetitive search queries. To simplify the process, I introduced two new syntactical elements that I am parsing before I send them to the "mu find" process. 

I find them quite useful and I thought I'd share them here to see if other people would benefit from this expansion. The two new forms are "bm:label" and "bb:label", and they get expanded, respectively, to the query identified by "label" in mu4e-bookmarks and a new defvar, "mu4e-building-blocks".  I introduced the latter because I found that the atomic building blocks were cluttering up my bookmarks menu; and I had to add a new :label property to the bookmark plist so as to write a simple regexp for the parsing function.  The parser is recursive, so you can include "bm:label" or "bb:label" within the building blocks themselves. 

This also makes it easy to combine searches to get union or intersection of your bookmarks, which isn't trivial to do with the existing search rules.

The coe is below -- I was wondering if it seemed like a feature that would be worth upstreaming. 

Thanks,

Matt

(defvar mu4e-building-blocks
  `((:label "inbox" :query "maildir:/inbox AND NOT flag:trashed")
    (:label "ghaddress" :query "contact:notifi...@github.com OR contact:nor...@github.com OR ilcontact:bui...@travis-ci.com OR contact:bui...@travis-ci.org")
    (:label "teachsub" :query "HIS393 OR HIS475 OR NEW113 OR HIS455 OR RLG231 OR RLG239 OR WDW235 OR WDW236 OR HIS101 OR HIS108 OR HIS191 OR \"digital humanities\" OR \"hacking history\" OR \"digital history\"")
    (:label "yesterday" :query "date:2d..today and not date:today..")
    (:label "today" :query "date:today..now")
    (:label "priority" :query "bb:inbox AND NOT bb:ghaddress")
    (:label "github" :query "bb:inbox AND bb:ghaddress"))

  (defun mu4e-dashboard-expand-query (q)
    (let* ((bookmark-re "\\(b[bm]\\):\\(\\w+\\)")
           (match (string-match bookmark-re q))
           (type (match-string 1 q))
           (label (match-string 2 q))
           (expand-list (cond
                         ((equal type "bm") mu4e-bookmarks)
                         ((equal type "bb") mu4e-building-blocks)) )
           (bookmark (if label (--first (equal (plist-get it :label) label) expand-list )))
           (expansion (if bookmark (format "(%s)" (plist-get bookmark :query)) ))
           )
      (message "%s" (match-data))
      (message "%s: %s" q expansion )
      (if expansion  
          (mu4e-dashboard-expand-bm (replace-regexp-in-string bookmark-re expansion  q))
        q))))

[Yuri D'Elia]

On Wed, Dec 01 2021, Matt Price wrote:
> I've just started using Nicolas Rougier's mu4e-dashboard and have been
> enjoying it quite a bit (https://github.com/rougier/mu4e-dashboard/). As
> I develop my own personal dashboard, I found myself writing quite a few
> complex and often repetitive search queries. To simplify the process, I
> introduced two new syntactical elements that I am parsing before I send
> them to the "mu find" process.

I do the same in the mu4e-query-fragments extension, which is a bit more
intrusive (expansion is performed for all mu4e queries).

[Daniel Fleischer]

Matt Price [2021-12-01 Wed 12:12] wrote:

> I've just started using Nicolas Rougier's mu4e-dashboard and have been
> enjoying it quite a bit (https://github.com/rougier/mu4e-dashboard/).
> As I develop my own personal dashboard, I found myself writing quite a
> few complex and often repetitive search queries. To simplify the
> process, I introduced two new syntactical elements that I am parsing
> before I send them to the "mu find" process.

That's very nice; at the least you can try and create a package, even if
they don't become part of mu, I think I'll definitely give them a try.

--

Daniel Fleischer

[Nicolas P. Rougier (gmail)]

Your solution is quite elegant, many thanks for sharing the
code. If you want, you can make a PR to mu4e-dashboard, but it
would better to have it in mu4e.

Nicolas

[Matt Price]

Ah, if I'd known about your package I never would have written my own code.  The advice you use is clever, and definitely makes your solution more comprehensive.

THanks for the pointer!

[Matt Price]

It really does seem like a clear improvement to me. i don't know if there are drawbacks, though. 

Also, Yuri and I have envisioned very similar functionality with different syntax choices. I am somewhat attached to mine, but I don't know which is really better.  Mine also currently depends on dash.el, which perhaps adds a new dependency.

Maybe Dirk will weigh in. I would be happy to prepare a patch if he thinks one or another version is worth supporting in mu4e core.  

[Sebastian Miele]

Matt Price <mopt...@gmail.com>:

>
> It really does seem like a clear improvement to me. i don't know if
> there are drawbacks, though.
>
> Also, Yuri and I have envisioned very similar functionality with
> different syntax choices. I am somewhat attached to mine, but I don't
> know which is really better. Mine also currently depends on dash.el,
> which perhaps adds a new dependency.
>
> Maybe Dirk will weigh in. I would be happy to prepare a patch if he
> thinks one or another version is worth supporting in mu4e core.

In my opinion, the arguably and canonically ultimate feature in that
direction would be an extensible sexp-based syntax for mu queries on the
mu4e/Elisp side. It's Lisp. Something like rx and rx-define for
regular expressions. But creating and integrating such a syntax would
be more involved. It probably should be expanded as late as possible in
mu4e, which probably is somewhere deeper down.

Best wishes
Sebastian

[Dirk-Jan C. Binnema]

Hi all!

Belated response; at least my org-mode todo list did not forget about
this thread...

In general, I like the idea of having some extra way to handle
repetitive / formulaic queries, and the existing 3rd-party
implementations are doing a great job; though I have to admit that my
query needs are fairly limited, so I'm not the best judge!

That said, I agree with Sebastian that for structured / programmatic
handling of queries, it'd be nice to have an sexp-based syntax; note, we
already use sexps as the an intermediate representation when parsing
queries; this representation is not really suitable for human
consumption today, but could be forged into one. Something to consider.

Once that is in place (which may take a while...), we could see if/how
any of the existing mu4e extensions could work with this.

Kind regards,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
gpg: 6987 9CED 1745 9375 0F14 DA98 11DD FEA9 DCC4 A036