Net::IMAP::NoResponseError on a large folder

[ggggggg...@gmail.com]

I know this isn't a problem with Larch, but I wonder if anyone here
has suggestions on workarounds. I'm transferring a single folder
containing a large number of messages (over 100,000), and -- perhaps
not surprisingly -- I get this when I run larch:

[---snip---]
[Aug 13 19:24:40] [info] copying messages from imap.gmail.com/[Gmail]/
All Mail to imap.gmail.com/[Gmail]/All Mail
[Aug 13 19:24:42] [info] xx...@gmail.com@imap.gmail.com: [Gmail]/All
Mail: fetching message headers 1..129845
[Aug 13 20:16:18] [info] xx...@gmail.com@imap.gmail.com:
Net::IMAP::NoResponseError: Some messages could not be FETCHed
(Failure) (will retry)
[Aug 13 21:00:13] [info] xx...@gmail.com@imap.gmail.com:
Net::IMAP::NoResponseError: Some messages could not be FETCHed
(Failure) (will retry)
[Aug 13 21:42:24] [info] xx...@gmail.com@imap.gmail.com:
Net::IMAP::NoResponseError: Some messages could not be FETCHed
(Failure) (will retry)
[Aug 13 22:21:41] [fatal] Net::IMAP::NoResponseError: Some messages
could not be FETCHed (Failure) (giving up)
[Aug 13 22:21:41] [info] 0 message(s) copied, 0 failed, 0 untouched
out of 0 total

There's currently no mechanism for copying messages in batches, and
doing it all at once with this many messages is flaky and memory-
intensive. Has anyone in a similar situation found a workaround?

Gabe

[Ryan Grove]

On Thu, Aug 13, 2009 at 7:53 PM, ga...@pwd.ca<ggggggg...@gmail.com> wrote:
> I know this isn't a problem with Larch, but I wonder if anyone here
> has suggestions on workarounds.  I'm transferring a single folder
> containing a large number of messages (over 100,000), and -- perhaps
> not surprisingly -- I get this when I run larch:
>
> [---snip---]
> [Aug 13 19:24:40] [info] copying messages from imap.gmail.com/[Gmail]/
> All Mail to imap.gmail.com/[Gmail]/All Mail
> [Aug 13 19:24:42] [info] xx...@gmail.com@imap.gmail.com: [Gmail]/All
> Mail: fetching message headers 1..129845
> [Aug 13 20:16:18] [info] xx...@gmail.com@imap.gmail.com:
> Net::IMAP::NoResponseError: Some messages could not be FETCHed
> (Failure) (will retry)

If you're brave, you might want to try installing the very latest
Larch development gem from GitHub:

gem sources -a http://gems.github.com
sudo gem install rgrove-larch

This version downloads message headers in batches and maintains
mailbox and message state during and after transfers (and between
Larch runs) so it can quickly resume where it left off if an error
occurs. It should be better at dealing with huge mailboxes like yours.

Standard caveat: this is an early development version and there may be
bugs, but Larch always operates on the source server in read-only
mode, so at worst only the destination server is at risk if something
goes horribly wrong.

Also, this version of Larch requires SQLite 3. SQLite comes
preinstalled on Mac OS X, but if you're using another OS, you may need
to install it yourself.

- Ryan