Querry contact-fields with more than one contact

[Chris Boo]

Hello everybody,

I think I might have stumbled over a bug in mu/mu4e, or at least unexpected behaviour. It seems that the contact fields of mails with more than one contact per contact field (eg. "To: a...@example.com, b...@example.com") are excluded from queries. So a mail with a "To:" field does not show up when I search for "to:aaa". nor just "aaa". Other fields like subject or body are not affected.
Am I doing something wrong here?

Cheers,

Christian

[Christian Neumann]

Am Sat, 31.01.2015 um 18:26 Uhr, schrieb Chris Boo:
> Am I doing something wrong here?

I don't think so, I have the same problem here.

Christian

[Chris Boo]

Hello,
since the behaviour was confirmed here, I posted an issue over at github: https://github.com/djcb/mu/issues/578

Cheers,
Christian

[Chris Boo]

Hi again,
sorry to bump this up again, but I really cannot help but wonder why I would possibly be the only one who relies on actually finding __all__ correspondences I had with a person (including mails with more than one recipient in the "To:" and "Cc:" fields). Am I missing something? Maybe some more people could be kind enough to let me know if they are affected by this as well?

Cheers,

Christian

[Ken Mankoff]

Hi Chris,

On 2015-02-24 at 15:36, Chris Boo <dachr...@googlemail.com>
wrote:

> Hi again, sorry to bump this up again, but I really cannot help
> but wonder why I would possibly be the only one who relies on
> actually finding __all__ correspondences I had with a person
> (including mails with more than one recipient in the "To:" and
> "Cc:" fields). Am I missing something? Maybe some more people
> could be kind enough to let me know if they are affected by
> this as well?

I can't re-create this.

I just 1) went into my mu MailDir folder and did "mu mkdir test",
and then 2) put the following file into "test/cur/foo".

Message-ID: <rand...@127.0.0.1> Date: Sat, 3 Jan 2015 21:27:34
+1300 Subject: Test Subject From: TestSomeone <som...@test.org>
To: TestMe <m...@test.org>, TestYou <y...@test.org> Email Body here
with a uniqueword

And then 3) did "mu index" and then 4) "mu find TestMe" and "mu
cfind TestMe" and both times the message was found.

Does this work for you? Can you create an MWE (minimum working
example) that demonstrates your bug?

-k.

[Ken Mankoff]

On 2015-02-24 at 16:31, Ken Mankoff <man...@gmail.com> wrote:
> Hi Chris,
>
> On 2015-02-24 at 15:36, Chris Boo <dachr...@googlemail.com>
> wrote:
>> Hi again, sorry to bump this up again, but I really cannot help
>> but wonder why I would possibly be the only one who relies on
>> actually finding __all__ correspondences I had with a person
>> (including mails with more than one recipient in the "To:" and
>> "Cc:" fields). Am I missing something? Maybe some more people
>> could be kind enough to let me know if they are affected by
>> this as well?
>
> I can't re-create this.
>
> I just 1) went into my mu MailDir folder and did "mu mkdir
> test", and then 2) put the following file into "test/cur/foo".

Sorry, I'm having reflow issues with mu4e :(.

Test email contents were (maybe it'll work this time):

Message-ID: <rand...@127.0.0.1> Date: Sat, 3 Jan 2015
21:27:34 +1300 Subject: Test Subject From: TestSomeone
<som...@test.org> To: TestMe <m...@test.org>, TestYou
<y...@test.org>

Email Body here with a uniqueword

-k.

[Ken Mankoff]

If anyone has any suggestsions how to improve mu4e text flow, I'd
love to know it. Test email attached.

-k.

[Chris Boo]

Hi Ken,
thank you very much for your answer and for your MWE! I tried it as you described it using mu version 0.9.11
As you say, it works with "mu find TestMe". BUT for me it is not working using the second name in your example (TestYou):

$ mu find TestYou
mu: no matches for search expression (4)

So, I apologize for actually describing the problem wrongfully in the first mail! The problem appears to be with the __second__ contact in the "To:" field and your example is reproducing the problem for me.

Can you confirm this?

Cheers,

Christian

[Ken Mankoff]

On 2015-02-24 at 20:00, Chris Boo <dachr...@googlemail.com>
wrote:

> So, I apologize for actually describing the problem wrongfully
> in the first mail! The problem appears to be with the
> __second__ contact in the "To:" field and your example is
> reproducing the problem for me.
>
> Can you confirm this?

"mu find testyou" works for me. Also "mu find TestYou", and "mu
find y...@test.org". And "mu cfind..." too.

-k.

[Christian Neumann]

Am Wed, 25.02.2015 um 02:00 Uhr, schrieb Chris Boo:
> As you say, it works with "mu find TestMe". BUT for me it is not working
> using the second name in your example (TestYou):

Same problem here. Using mu 0.9.9.6
Maybe a problem with the used Xapian version? I'm using mu and Xapian as
contained in Debian testing. Xapian version 1.2.19

Christian

[Chris Boo]

Hi all,
I am on Xapian version 1.2.18 as found in the Ubuntu 14.10 repos. I tried the current master branch over at github as well, experiencing the same problems. What versions / distribution are you using, Ken?

Thanks for the help here!

Christian

[Danie Roux]

On Wed, Feb 25, 2015 at 11:07 AM, Chris Boo <dachr...@googlemail.com> wrote:

Hi all,
I am on Xapian version 1.2.18 as found in the Ubuntu 14.10 repos. I tried the current master branch over at github as well, experiencing the same problems. What versions / distribution are you using, Ken?

Christian,

I would be very curious to know if:

$ make tests

... all pass for you on the master branch for you.

My suspicion is that it won't. 

--

Danie

[Chris Boo]

Hi Danie,
running
$ make test
I do not get any errors or failed tests. I would write a failing test myself but have not the first clue about tests in C, sorry.

Any other ideas?

Cheers,
Christian

[Ken Mankoff]

I'm on OS X, using xapian 1.2.19 (from brew) and mu 0.9.11 built
by hand from latest git.

-k.


On 2015-02-25 at 04:07, Chris Boo <dachr...@googlemail.com>
wrote:

[Chris Boo]

Hmmm, this is so weird. What else might it be? Maybe someone more knowledgeable than me could commit a test for this at github? I would do it myself but don't have a clue where to start.

/Christian

[Chris Boo]

Hello again,

this problem is really bugging me a lot and I have been thinking about this quit a bit. Might it be that the problem arises, because somehow contacts are not serialized properly when fed to the database? Eg., a mail with "To: tes...@aaa.de, tes...@bbb.de" is added to the index as "tes...@aaaa.de, tes...@bbb.de" and not as two separate strings. The way I think that cfind works is that the search pattern has to match the beginning of the indexed string, does it not? Hence, this would enable me to find "testMe" but not testYou in the index?

Does this make sense? And if so, how could I find out if this is indeed the case? Is there some possibility to "browse" the database?

Cheers,

Christian

[Ken Mankoff]

Hi Christian,

I can't help since it works on my system, but perhaps the information was entered incorrectly (due to a bug in a previous version of the code?) and now isn't getting updated. Have you tried wiping your DB, or the full re-index, using just the test email we set up previously? Does it occur on a clean/new system?

-k.

[Dirk-Jan C. Binnema]

On Thursday Mar 12 2015, Ken Mankoff wrote:

> Hi Christian,
>
> I can't help since it works on my system, but perhaps the information was entered incorrectly (due to a bug in a previous version of the code?) and now isn't getting updated. Have you tried wiping your DB, or the full re-index, using just the test email we set up previously? Does it occur on a clean/new system?
>
> -k.
>
> On 2015-03-12 at 10:29, Chris Boo <dachr...@googlemail.com> wrote:
>> Hello again,
>>
>> this problem is really bugging me a lot and I have been thinking about this
>> quit a bit. Might it be that the problem arises, because somehow contacts
>> are not serialized properly when fed to the database? Eg., a mail with "To:
>> tes...@aaa.de, tes...@bbb.de" is added to the index as "tes...@aaaa.de,
>> tes...@bbb.de" and not as two separate strings. The way I think that cfind
>> works is that the search pattern has to match the beginning of the indexed
>> string, does it not? Hence, this would enable me to find "testMe" but not
>> testYou in the index?
>>
>> Does this make sense? And if so, how could I find out if this is indeed the
>> case? Is there some possibility to "browse" the database?

Note, the data for `mu cfind' is cached in a local file,
~/.mu/cache/contacts.

Cheers,
Dirk.

--
Dirk-Jan C. Binnema Helsinki, Finland
e:dj...@djcbsoftware.nl w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C

[Chris Boo]

Hello,
thanks again for the replies - much appreciated! I deleted my ~/.mu folder and ran
    $ mu index --maildir="~/maildir/test"
to reindex a maildir with the above mail. I did a few tests running
    $ mu cfind test
with different configurations of the "To" field. I did delete the .mu folder between each try. In the following, I wrote the recipient line I used and the contacts found when running the former command. The sender (TestSomeone) was always properly indexed.

1. Test: as in the above MWE

To: TestMe <m...@test.org>, TestYou <y...@test.org>

Result: TestMe

2. Test: no white space:
To: TestMe <m...@test.org>,TestYou <y...@test.org>
Result: TestMe

3. Test: No names
To:  m...@test.org, y...@test.org
Result: TestMe

4. Test: two "To:" fields in the header
To: TestMe <m...@test.org>
To: TestYou <y...@test.org>
Result: TestMe

5. Test: Split into To and Cc:
To: TestMe <m...@test.org>
Cc: TestYou <y...@test.org>
Result: TestMe, TestYou

Maybe this helps...

Furthermore, here are more details on my specs:
* encoding of the MWE file utf-8-unix
* mu version v0.9.11 (but I tried several ones)
* gcc version 4.9.1
* xapian version 1.2.18
* OS: Ubuntu 14.10

Any leads or clues on this issue are much appreciated!

Cheers,

Christian




To: m...@test.org
Cc: y...@test.org

e:d...@djcbsoftware.nl           w:www.djcbsoftware.nl

[Chris Boo]

Hello,
very strange. Using the exact same configuration as the one which caused the problem on Ubuntu 14.10, I cannot reproduce it on ubuntu 14.04. I would say this marks some change in the distribution as the culprit? Any ideas?

Cheers,

Christian