[VM] Cannot view GMail inbox

[Torstein Krause Johansen]

Dear all,

I'm having problems displaying my GMail inbox and was hoping you could
help me out as I'd really like to start using VM for my IMAP folders
again.

VM logs me into GMail and displays my folders, however when I select
one to view it gives me:

Reading folder INBOX... done
vm-read-index-file-maybe: Wrong type argument: number-or-marker-p, nil

I would really appreciate some help on this because I don't know how
to proceed with this and I'm certain that it "should work" with
GMail/IMAP as others are bound to use this combination too.

I'm using VM 8.2.0a downloaded from launchpad and installed in
/usr/local bin with the normal ./configure && make && make install and
GNU Emacs 23.3.1 installed using the Debian testing packages.

My .vm is (including everything in case there's something obvious
I've missed):

(setq
mail-signature t
mail-signature-file "~/.signature"
message-send-mail-function 'smtpmail-send-it
send-mail-function 'smtpmail-send-it
smtpmail-auth-credentials (expand-file-name "~/.authinfo")
smtpmail-debug-info t
smtpmail-default-smtp-server "smtp.gmail.com"
smtpmail-smtp-server "smtp.gmail.com"
smtpmail-smtp-service 587
smtpmail-starttls-credentials '(("smtp.gmail.com" 587 nil nil))
starttls-use-gnutls t
vm-enable-external-messages t
vm-folder-directory "~/mail"
vm-imap-folder-cache-directory "~/mail/cache"
vm-imap-max-message-size 512
vm-index-file-suffix ".vm"
vm-init-file "~/.vm"
vm-load-headers-only nil
vm-mime-attachment-save-directory "~/tmp"
vm-mutable-frames nil
vm-mutable-windows nil
vm-pop-folder-cache-directory "~/mail/cache"
vm-preview-lines nil
vm-primary-inbox "~/mail/inbox"
vm-stunnel-program "/usr/bin/stunnel4"

vm-imap-account-alist
'(
("imap-ssl:imap.googlemail.com:993:inbox:login:myuser.com:*"
"gmail-work")
)
)

;; Set sent folder to the current year/month every time a new message
;; is composed.
(defun gse-set-mail-archive-file-name()
"Set the sent folder to be based on the current year/month."
(setq mail-archive-file-name
(concat
"~/mail/sent/"
(format-time-string "%Y-%m"))))
(add-hook 'mail-mode-hook 'gse-set-mail-archive-file-name)

;; Nice colours
(require 'u-vm-color)
(add-hook 'vm-summary-mode-hook 'u-vm-color-summary-mode)
(add-hook 'vm-select-message-hook 'u-vm-color-fontify-buffer)
(defadvice vm-decode-mime-message (after u-vm-color activate)
(u-vm-color-fontify-buffer-even-more))

VM is loaded with the following snippet in my .emacs:

(defun tkj-load-vm ()
(interactive)
(add-to-list 'load-path (expand-file-name
"/usr/local/share/emacs/site-lisp/vm"))
(add-to-list 'Info-default-directory-list
(expand-file-name
"/usr/local/share/info/vm"))
(require 'vm-autoloads)
(require 'vm)
(require 'starttls)
(require 'smtpmail)
(vm)
)

Best regards,

-Torstein

[Uday Reddy]

Please remove the setting:

vm-index-file-suffix ".vm"

because there seems to be a bug in the index file handling. I will file
this as a bug report.

Cheers,
Uday

[Torstein Krause Johansen]

Greetings Uday,

and thanks for your reply.

Thanks, that got me a bit further. I'm sorry to say, though, that I
stumbled into another problem which left me with a blank inbox summary
screen:

Checking for new mail for
/home/torstein/mail/cache/imap-cache-6de7902942e2d83f1ded4adb043a2a36...
Logging into the IMAP server...
Updating attributes on the IMAP server... done
Retrieving message attributes and labels...
Retrieving message atrributes and labels... done
Retrieving new messages...
mapcar: Wrong type argument: listp, t

Cheers,

-Torstein

[Uday Reddy]

Torstein Krause Johansen writes:

> Thanks, that got me a bit further. I'm sorry to say, though, that I
> stumbled into another problem which left me with a blank inbox summary
> screen:
>
> Checking for new mail for
> /home/torstein/mail/cache/imap-cache-6de7902942e2d83f1ded4adb043a2a36...
> Logging into the IMAP server...
> Updating attributes on the IMAP server... done
> Retrieving message attributes and labels...
> Retrieving message atrributes and labels... done
> Retrieving new messages...
> mapcar: Wrong type argument: listp, t

I have tried it with your settings and reproduced the error. You have set
`vm-enable-external-messages' to t. You should set it to '(imap).

Can I please urge everybody to try VM with default settings in order to
track down problems with your settings?

I have announced previously a mechanism whereby you can put all your option
settings into a separate .vm.preferences file and block it from loading
initially. That way, you can easily test out and debug your settings.

This is entirely similar to how you test out problems with your .emacs files
by doing emacs -q.

Cheers,
Uday

[Torstein Krause Johansen]

Greetings,

Uday Reddy writes:

> Torstein Krause Johansen writes:
>
> I have tried it with your settings and reproduced the error. You have set
> `vm-enable-external-messages' to t. You should set it to '(imap).

Thank you _so_ much, Uday. Now, VM works great with my GMail
account.

I've still one wee issue, but it's be no means any show stopper for
me, but perhaps something to iron out before a 8.2.0 final release?:

- vm-visit-imap-folder => initial login: if I type the wrong password
(or just hit ENTER), VM refuses to give me a second go, instead it
keeps complaining from my initial mistyping:

completion--some: IMAP protocol error: "CRLF expected"

> Can I please urge everybody to try VM with default settings in order to
> track down problems with your settings?

Note taken.

Cheers,

-Torstein

[Uday Reddy]

Torstein Krause Johansen writes:

> I've still one wee issue, but it's be no means any show stopper for
> me, but perhaps something to iron out before a 8.2.0 final release?:
>
> - vm-visit-imap-folder => initial login: if I type the wrong password
> (or just hit ENTER), VM refuses to give me a second go, instead it
> keeps complaining from my initial mistyping:
>
> completion--some: IMAP protocol error: "CRLF expected"

The error message depends on what the server says. In this case, it looks
like VM couldn't parse the server response.

You can provide me more information about the server response. Please set
vm-imap-keep-trace-buffer to t. Then after you get the error, find the
saved trace buffer and send me the contents of it.

Cheers,
Uday

[Uday Reddy]

Torstein Krause Johansen writes:

> I've still one wee issue, but it's be no means any show stopper for
> me, but perhaps something to iron out before a 8.2.0 final release?:
>
> - vm-visit-imap-folder => initial login: if I type the wrong password
> (or just hit ENTER), VM refuses to give me a second go, instead it
> keeps complaining from my initial mistyping:
>
> completion--some: IMAP protocol error: "CRLF expected"

I was able to reproduce something similar by giving an empty password to
gmail. The session shows:

Starting IMAP over SSL session Mon Aug 08 10:37:33 2011
-- connecting to imap.gmail.com:993
* OK Gimap ready for requests from 92.232.137.113 b21if4769841wef.158
-- connected for folders
VM CAPABILITY
* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 XYZZY SASL-IR AUTH=XOAUTH
VM OK Thats all she wrote! b21if4769841wef.158
VM LOGIN <parameters omitted>
* BYE [UNAVAILABLE] Temporary System Error
VM NO System Error {XDH} (Failure)
Process IMAP over SSL finished

Two problems here.

- First of all, gmail seems to have had an internal crash. How silly is
that? For an empty password?

- Secondly, it dumped some string "{XDH}", which, in IMAP protocol, is
supposed to mean, I am sending you a stream of data and here is its length.
After that there should be an end of line. VM's parser rightly flags up a
syntax error.

I will adjust VM so that if the stuff in braces is not a number, we don't
assume that it is a stream. That should get around this stupid message.

But, please complain to gmail that their server is misbehaving.

Cheers,
Uday

[Torstein Krause Johansen]

Hi again,

sorry for my late reply, work has kept me busy :-)

And in case you're checking the header of this message: yes, I'm using
GMail to post this message. The reason is not that GMail isn't working
with VM now (thanks to your help), but that I haven't set up multiple
SMTP accounts with automatic switching between them.

Since my primary mail is my company mail, GMail just has to take a
back seat right now. But there's perhaps some wizardry to the Emacs
SMTP settings I haven't looked into? Feel free to point me in the
right direction. http://www.emacswiki.org/emacs/MultipleSMTPAccounts
seems like a place to start. Has anyone gotten this to work (with VM)
without any user interaction? I.e. if I reply to a message in
imap/gmail:inbox, it uses the GMail setup and if I'm replying to a
message in imap/someothermail:inbox, it uses someothermail's SMTP
setup?

On 8 August 2011 21:27, Uday Reddy <usr.vm...@gmail.com> wrote:

> Torstein Krause Johansen writes:
>
>> - vm-visit-imap-folder => initial login: if I type the wrong password
>>   (or just hit ENTER), VM refuses to give me a second go, instead it
>>   keeps complaining from my initial mistyping:
>>
>>   completion--some: IMAP protocol error: "CRLF expected"
>
> I was able to reproduce something similar by giving an empty password to
> gmail.  The session shows:

[..]

> Two problems here.
>
> - First of all, gmail seems to have had an internal crash.  How silly is
> that?  For an empty password?

Amazing. Good catch, though!

> - Secondly, it dumped some string "{XDH}", which, in IMAP protocol, is
> supposed to mean, I am sending you a stream of data and here is its length.
> After that there should be an end of line.  VM's parser rightly flags up a
> syntax error.
>
> I will adjust VM so that if the stuff in braces is not a number, we don't
> assume that it is a stream.  That should get around this stupid message.

Great, I had to re-start Emacs whenever I mistyped my password :-)

> But, please complain to gmail that their server is misbehaving.

I'll send them a pedagogical email :-)

Cheers,

-Torstein

--
Torstein Krause Johansen
System architect / Vizrt Online
Vizrt
t...@vizrt.com
www.vizrt.com

[Uday Reddy]

Torstein Krause Johansen writes:

> And in case you're checking the header of this message: yes, I'm using
> GMail to post this message.

Pity, the `from' address wasn't gmail. So, it is now in public domain. My
own feeling is that we should be careful with email addresses and not so
much with the smtp servers, unless you want complete anonymity.

I will defer the question about multiple smtp servers to others that use
them. I myself don't.

> > I will adjust VM so that if the stuff in braces is not a number, we don't
> > assume that it is a stream.  That should get around this stupid message.
>
> Great, I had to re-start Emacs whenever I mistyped my password :-)

Indeed, VM wasn't being kind to wrong passwords. I have done the change
now, which will go into the next release.

Cheers,
Uday

[Torstein Krause Johansen]

Greetings,

On 11 August 2011 19:58, Uday Reddy <usr.vm...@gmail.com> wrote:
> Torstein Krause Johansen writes:

>> Great, I had to re-start Emacs whenever I mistyped my password :-)
>
> Indeed, VM wasn't being kind to wrong passwords.  I have done the change
> now, which will go into the next release.

Excellent!

-Torstein

[Uday Reddy]

Torstein Krause Johansen writes:

> >> Great, I had to re-start Emacs whenever I mistyped my password :-)
> >
> > Indeed, VM wasn't being kind to wrong passwords.  I have done the change
> > now, which will go into the next release.

Incidentally, the recommended method for handling passwords in 8.2.0 is to
store them in .authoinfo.gpg file. Please look up "passwords" in the index
of the manual.

The way VM stores passwords currently is insecure. Anybody who gets an
access to your Emacs session can snoop around and find all your passwords if
they know a bit of elisp.

So, eventually, we will remove password-storing in VM and use standard Emacs
facilities for handling passwords.

Cheers,
Uday

[Terry Jones]

Today I had a weird experience. Someone Cc'd on an email I sent with VM
(8.2.0a) replied to me with a screenshot of the mail as he saw it in Gmail.
Some parts of the mail body had purple text, some black. I'd composed the
mail after pasting text in from a bash window. You could see where I'd
typed new text.

It seems like VM must be sending that extra information. I've been using VM
for donkey's years and have never had anyone tell me my mails looked weird.
I've not changed my VM setup in months. I tried sending the same person an
email containing several paragraphs, one of them pasted from bash, and he
could tell me which paragraph it was.

I don't see anything obvious to explain why this is happening (assuming it
*is* happening). I have vm-send-using-mime set to t, but its documentation
doesn't say anything about this. I've looked at the FCC log I keep of all
outgoing mail, and it just has a regular text/plain mail in it.

Can anyone shed any light on this? I find it completely bizarre! So
bizarre that I can't quite believe it.

Thanks,
Terry

[Tim Cross]

It does sound a little odd, but I have had similar issues when it
comes to cutting and pasting, but not with mail. In my case, the issue
was due to cutting and pasting from another emacs buffer where emacs
was including the text properties in the paste. However, for this to
occur outside of emacs, the emacs text properties (which include the
font-locking info) would need to be translated into something which
another program (ie. in your case, your receiver's web client) can
understand. The only things I can think of which *might* cause this
are packages such as htmlize, which can convert font-lock information
into html representations of the text. I think there may also be a
package which does similar, but for rich text. Alternatively, maybe
your cut and paste is including ANSI escape characters which bash is
using and these are being interpreted by gmail.

Perhaps the first thing to do would be to send a test message to the
list which is constructed in the same way the one you describe was
i.e. with cut and paste from a bash shell. We can then look at the
content and MIME information to see what might be triggering gmail to
render text in different colours.

Tim

--
Tim Cross

[Uday S Reddy]

On 11/30/2011 7:59 PM, Terry Jones wrote:

> Today I had a weird experience. Someone Cc'd on an email I sent with VM
> (8.2.0a) replied to me with a screenshot of the mail as he saw it in Gmail.
> Some parts of the mail body had purple text, some black. I'd composed the
> mail after pasting text in from a bash window. You could see where I'd
> typed new text.

Have you tried sending yourself the same kind of message? Type `D' a
couple of times to see the undecoded MIME.

My guess is that the cut-and-pasted text might be in a different
character set.

Cheers,
Uday

[Uday S Reddy]