rgrove / larch Goto Github PK

Hi Ryan,

first of all thanks for that nice program. It's really a blast and helps me migrating from my IMAP mailbox hosted on our own server to Google.

Actually, I have a lot of old emails in old folders and a lot of subfolders. I recognized, that some folders have not been transferred to labels in gmail and the emails are not there - even though larch said so in it's output. So I decided to migrate each folder.

Well, it would be really cool, if I could define more than one sourcefolder in the config.yaml file. E.g:

gmail_migration:
verbosity: debug

  from: imap://mail.milkno.com
  from-folder: INBOX.Projekte.ruby
  from-folder: INBOX.Projekte.perl
  from-folder: INBOX.Projekte.php
  from-folder: INBOX.Projekte.java
  from-folder: INBOX.Projekte.lisp
  from-folder: INBOX.Projekte.sql 
  from-user: 'mail05'
  from-pass: 'blabla'

  to: imaps://imap.googlemail.com
  to-folder: Projekte
  to-user: '[email protected]'
  to-pass: 'fallera'

That would be helpful. Or did I miss something in the docu?

CHeers

Andy

Gmail doesn't actually delete emails when deleting from IMAP. Gmail instead simply removes an email's labels on delete. To perform a delete, an email needs to be moved to Trash.

http://mail.google.com/support/bin/answer.py?answer=78755

P.S. Thanks for this awesome utility!

Migrating from a dovecot imap server to google mails gives me this
'unicode related' error:

[17:01:33] [info] 192.168.1.13/KLANTEN/Andrea Acuña -> imap.gmail.com/
KLANTEN/Andrea Acuña
[17:01:33] [fatal] Attempt to update object did not result in a single
row modification (SQL: UPDATE mailboxes SET uidvalidity =
1303737107 WHERE (id = 139))

A message with a newline in the subject causes Net::IMAP (and thus Larch) to hang when the message is FETCHed.

This message can be used to reproduce the problem: http://pieisgood.org/misc/larch/subjplusonly.eml

Add search criterion to be able to "filter" copies.

Specifically, I'd like to see date criterion, but you could add a bunch of others too. I wanted to move all messages older than X months old to my "archive" GMail account, which has a mirrored directory structure as my primary account.

This issue happens whenever you have a folder with the same name but different casing between source and destination (ie. "folder" - "Folder"). IMAP protocol is case sensitive, but gmail isn't and throws a fatal error that stops the migration.

I coded a simple patch that adds a new quirk, that checks a regexp for the name.

def mailbox(name, delim = '/')
    retries = 0

    name.gsub!(/^(inbox\/?)/i){ $1.upcase }
    name.gsub!(delim, self.delim)

    name.gsub!(/\s+/, ' ') if @quirks[:gmail]

    # Gmail doesn't allow folders with leading or trailing whitespace.                                                                                       \

    name.strip! if @quirks[:gmail]

    begin
      if @quirks[:gmail]
        # In Gmail folders are not case sensitive                                                                                                             
        update_mailboxes
        @mailboxes.keys.each do |key|
          if key =~ /^#{Regexp.quote(name)}$/i
            return @mailboxes[key]
          end
        end
        raise MailboxNotFoundError, "mailbox not found: #{name}"
      end

      @mailboxes.fetch(name) do
        update_mailboxes
[....]

Larch currently does not verify that the destination server supports all the flags/keywords being transferred from the source server before attempting to set them, which can result in failures.

A user reported on the Larch mailing list that the IMAP URI imaps://mail.example.no results in an invalid URI error, whereas imaps://mail.example.com works fine. Ruby's URI lib appears to be trying (and failing) to validate actual domain names.

Reported on the Larch mailing list at http://groups.google.com/group/larch/browse_thread/thread/294af0368c309bd

I 'am trying to synchronize my personnal IMAP (Dovecot) with my gmail account. The gmail locals are set to french, so Sent Mail folder is written in french with an accent. The problem is i connot use --to-folder option with an accent

Log messages:
creating mailbox: [Gmail]/Messages envoyés (this is the french name of Sent Mail folder)
Net::IMAP::BadResponseError: Invalid Command (will retry)

Looks like it tries to create a folder which should already exist on gmail...

Any clue?
I've set my gmail account to english and used the "Sent Mail" name instead, but i really would like to find a better solution.

First: GREAT WORK!

1. I would like to copy email from the "exchange/Inbox" to the "gmail/Inbox" only if the email is not in "gmail/All Mail". Therefor the inventory comparison should be between "exchange/Inbox" and "gmail/All Mail", but the copy should be from "exchange/Inbox" to "gmail/Inbox". This would allow to archive emails in gmail without having them put back into the inbox with the next sync.

2. Delete-if-older-than option would be nice.

Need to add an --exclude option to specify folder names (or regexes?) to exclude when using --all or --all-subscribed.

Should also add an --exclude-file option to allow exclusions to be specified in a text file.

Reported on the mailing list at http://groups.google.com/group/larch/browse_thread/thread/1dd363a5be6f4ad0?hl=en:

I'm using larch to perform a large migration of a client's Google Apps
account with about 100k emails and a couple hundred labels. Everything went
well until larch started with the following Gmail label (font size increased
to notice the apostrophe): "Teddy’s Bike"

Note the apostrophe, it's not your regular ' apostrophe, it's a different
one: ’

I believe the problem lies in the weird apostrophe, because I had no trouble
with other labels which had the regular apostrophe. I logged larch's output
with the insane option:

S: RUBY0150 OK Success^M 
[17:19:28] [info] imap.gmail.com/Teddy<E2><80><99>s Bike -> imap.gmail.com/Denny<E2><80><99>s Hawthorne 
[17:19:28] [debug] [<] Teddy<E2><80><99>s Bike: getting mailbox status 
C: RUBY0163 STATUS "Teddy&IBk-s Bike" (MESSAGES UIDNEXT UIDVALIDITY)^M 
S: * STATUS "Teddy&IBk-s Bike" (MESSAGES 167 UIDNEXT 170 UIDVALIDITY 607519705)^M 
S: RUBY0163 OK Success^M 
[17:19:28] [fatal] Attempt to update object did not result in a single row modification (SQL: UPDATE `mailboxes` SET `uidvalidity` = 607519705 WHERE (`id` = 62)) 
[17:19:28] [info] 0 message(s) copied, 0 failed, 0 deleted out of 7507 total 
[17:19:28] [debug] performing database maintenance

See https://groups.google.com/forum/#!topic/larch/jc7cllhbnD4

When copying a single folder, all child folders should also be copied.

I updated my gems today and got the latest version of sequel (3.12.1 released in June 9, 2010). Now when running larch I get the following message:

/opt/local/lib/ruby/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:134:in open_db': undefined methodget_current_migration_version' for Sequel::Migrator:Class (NoMethodError)
from /opt/local/lib/ruby/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:36:in init' from /opt/local/lib/ruby/gems/1.9.1/gems/larch-1.1.0.dev.20100209/bin/larch:89:inmodule:Larch'
from /opt/local/lib/ruby/gems/1.9.1/gems/larch-1.1.0.dev.20100209/bin/larch:9:in <top (required)>' from /opt/local/bin/larch:19:inload'
from /opt/local/bin/larch:19:in `

It also happens with 3.12.0, released June 1, 2010. It works fine however with previous versions.

Reported on the Larch mailing list at http://groups.google.com/group/larch/browse_thread/thread/2f8e8dbc2fab0af5

/var/lib/gems/1.8/bin/larch --from imap://server1 --from-user user1@domain1 --from-pass password1 --to imaps://imap.gmail.com --to-user [email protected] --to-pass password2 --all

[Sep 01 17:30:01] [info] [email protected]@imap.gmail.com: creating mailbox: INBOX/blabla/blablabla/blabla blabla 2008
[Sep 01 17:30:02] [info] [email protected]@imap.gmail.com: Net::IMAP::NoResponseError: Folder name is not allowed. (Failure) (will retry)
[Sep 01 17:30:03] [info] [email protected]@imap.gmail.com: Net::IMAP::NoResponseError: Folder name is not allowed. (Failure) (will retry)
[Sep 01 17:30:05] [info] [email protected]@imap.gmail.com: Net::IMAP::NoResponseError: Folder name is not allowed. (Failure) (will retry)
[Sep 01 17:30:09] [fatal] Net::IMAP::NoResponseError: Folder name is not allowed. (Failure) (giving up)
[Sep 01 17:30:09] [info] 0 message(s) copied, 0 failed, 1 untouched out of 1 total

Currently, when Larch encounters a recoverable error, it retries 3 times and then gives up if the error still occurs, halting all mail processing. Ideally, Larch should just skip the problematic message and continue processing other messages.

Add an option to remove email messages from the source account after they are synchronized.

When Gmail returns a "Some messages could not be FETCHed" error in response to a batch fetch operation, Larch should retry the fetch, gradually decreasing the batch size each time until it succeeds and then marking and ignoring the corrupt message.

See http://groups.google.com/group/larch/msg/6e045bba542fbde2

Got the latest source and got bitten by:

undefined method `strip'
when passing "--exclude Trash Spam" on the commandline, without a config file.

Same as reported in:
http://groups.google.com/group/larch/browse_thread/thread/4ec6ee1a5c754c60/6e045bba542fbde2?#6e045bba542fbde2

Hi,

I was expecting that the subfolders of the "from" side would be put as subfolder of "to-folder" on the "to" side.

Is it intentional or a bug? I don't see why the hierarchy would not be kept…

Thanks

1. Use Larch to copy a bunch of messages from server A to server B.
2. Delete those messages on server B.
3. Run Larch again. It will see that the messages have been deleted, but will not recopy them.
4. Run Larch again. This time it will recopy the deleted messages.

Larch should recopy the deleted messages the first time it sees that they're gone.

when scanning folders with only one message in imap/mailbox.rb
variable full_range is 1...2

Since

full_range.max - full_range.min 

is zero

need_full_scan = full_range && full_range.max && full_range.min && full_range.max - full_range.min > 0

is false and nothing is done on the folder.

I suggest to change the definition of need_full_scan as

need_full_scan = full_range && full_range.max && full_range.min && full_range.max - full_range.min >= 0

Hi,
as gmail localizes folder names, it would be very useful to have a "--list-folders" option, in order to find out the names of the source / target servers.
E.g. in Spanish the folder "[Gmail]/All Mail" becomes "[Gmail]/Todos" and in German "[Gmail]/Sent" becomes "[Google Mail]/Gesendet", it took me some time to find this out...
Regards,
Jacob

Further reading:
http://stackoverflow.com/questions/2185391/localized-gmail-imap-folders

Enable connecting behind proxy.
Possibly attaching to the web interface of gmail (rather than imap)

Add an option to delete message from the source after syncing or if they already exist an the destination.

I have attached the output from a run. The first time it complained about 'Sent Messages' so I tried excluding it, but it keeps complaining. I tried renaming the folder on the source server (removed the space) and it works. But I have hundreds of folders containing space in the name. Am I missing something, or could it maybe be the server that cannot handle spaces?

$ bin/larch --from imap://mail.xxxxxxxx.no --to imap://xxxxxx.inmotionhosting.com -u [email protected] -p secret -P secret -U [email protected] -a -V insane --exclude 'Sent Messages'
[21:44:21] [debug] [<] updating mailboxes
[21:44:21] [debug] [<] connecting...
S: * OK IMAP4rev1 server ready at 06/12/11 21:44:24
[21:44:21] [info] [<] connected to mail.xxxxxxxx.no on port 143
C: RUBY0001 CAPABILITY
S: * CAPABILITY IMAP4rev1 IMAP4 AUTH=LOGIN AUTH=CRAM-MD5 IDLE CHILDREN AUTH=NTLM
S: RUBY0001 OK CAPABILITY completed
.....
S: RUBY0002 OK AUTHENTICATE CRAM-MD5.
[21:44:22] [debug] [<] authenticated using CRAM-MD5
C: RUBY0003 LIST "" ""
S: * LIST (\HasChildren) "/" "DIVERSE MAPPER"
S: * LIST (\HasNoChildren) "/" "DIVERSE MAPPER/ANNE BRITH"
........
S: RUBY0003 OK LIST completed
C: RUBY0004 LSUB "" ""
S: * LSUB () "/" "Junk E-mail"
S: * LSUB () "/" "Deleted items"
S: * LSUB () "/" "DIVERSE MAPPER"
........
S: RUBY0004 OK LSUB completed
[21:44:23] [debug] [>] connecting...
S: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2010 Double Precision, Inc. See COPYING for distribution information.
[21:44:23] [info] [>] connected to xxxxxx.inmotionhosting.com on port 143
C: RUBY0001 CAPABILITY
S: * CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS
S: RUBY0001 OK CAPABILITY completed
....
S: RUBY0002 OK LOGIN Ok.
C: RUBY0003 LIST "" ""
S: * LIST (\Noselect) "." ""
S: RUBY0003 OK LIST completed
[21:44:24] [debug] [>] updating mailboxes
C: RUBY0004 LIST "" ""
S: * LIST (\Unmarked \HasChildren) "." "INBOX"
S: * LIST (\HasNoChildren) "." "INBOX.Deleted Messages"
S: * LIST (\HasNoChildren) "." "INBOX.Notes"
S: * LIST (\HasNoChildren) "." "INBOX.Sent"
S: * LIST (\HasNoChildren) "." "INBOX.Trash"
S: * LIST (\HasNoChildren) "." "INBOX.Drafts"
S: * LIST (\HasNoChildren) "." "INBOX.Junk"
S: RUBY0004 OK LIST completed
C: RUBY0005 LSUB "" ""
S: * LSUB (\HasNoChildren) "." "INBOX.Notes"
S: * LSUB (\HasNoChildren) "." "INBOX.Deleted Messages"
S: * LSUB (\Noselect \HasChildren) "." "INBOX"
S: RUBY0005 OK LSUB completed
[21:44:25] [info] [>] creating mailbox: DIVERSE MAPPER.ANNE BRITH
C: RUBY0006 CREATE "DIVERSE MAPPER.ANNE BRITH"
S: RUBY0006 NO Invalid mailbox name.
[21:44:25] [warning] [>] Net::IMAP::NoResponseError: Invalid mailbox name. (will retry)
C: RUBY0007 CREATE "DIVERSE MAPPER.ANNE BRITH"
S: RUBY0007 NO Invalid mailbox name.
[21:44:26] [warning] [>] Net::IMAP::NoResponseError: Invalid mailbox name. (will retry)
C: RUBY0008 CREATE "DIVERSE MAPPER.ANNE BRITH"
S: RUBY0008 NO Invalid mailbox name.
[21:44:29] [warning] [>] Net::IMAP::NoResponseError: Invalid mailbox name. (will retry)
C: RUBY0009 CREATE "DIVERSE MAPPER.ANNE BRITH"
S: RUBY0009 NO Invalid mailbox name.
[21:44:32] [fatal] Net::IMAP::NoResponseError: Invalid mailbox name. (giving up)
[21:44:32] [info] 0 message(s) copied, 0 failed, 0 deleted out of 0 total
[21:44:32] [debug] performing database maintenance

Any clues how to workaorund this ?

[19:36:41] [info] [<] INBOX/supply-infreq/di.se: fetching latest message flags...
[19:36:41] [info] [>] INBOX/supply-infreq/di.se: fetching latest message flags.../usr/lib/ruby/1.9.1/thread.rb:68:in sleep': deadlock detected (fatal) from /usr/lib/ruby/1.9.1/thread.rb:68:inwait'
from /usr/lib/ruby/1.9.1/monitor.rb:97:in wait' from /usr/lib/ruby/1.9.1/net/imap.rb:1170:insend_literal'
from /usr/lib/ruby/1.9.1/net/imap.rb:1373:in send_data' from /usr/lib/ruby/1.9.1/net/imap.rb:1145:insend_data'
from /usr/lib/ruby/1.9.1/net/imap.rb:1072:in block (2 levels) in send_command' from /usr/lib/ruby/1.9.1/net/imap.rb:1070:ineach'
from /usr/lib/ruby/1.9.1/net/imap.rb:1070:in block in send_command' from /usr/lib/ruby/1.9.1/monitor.rb:190:inmon_synchronize'
from /usr/lib/ruby/1.9.1/net/imap.rb:1064:in send_command' from /usr/lib/ruby/1.9.1/net/imap.rb:635:inappend'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:78:in block in append' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:188:insafely'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:72:in append' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:230:inblock in copy_messages'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:104:in block in each_db_message' from /var/lib/gems/1.9.1/gems/sequel-3.9.0/lib/sequel/dataset/actions.rb:15:ineach'
from /var/lib/gems/1.9.1/gems/sequel-3.9.0/lib/sequel/dataset/actions.rb:15:in all' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:104:ineach_db_message'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:188:in copy_messages' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:72:inblock in copy_all'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:126:in block in each_mailbox' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:126:ineach_value'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:126:in each_mailbox' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:60:incopy_all'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/bin/larch:118:in <module:Larch>' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/bin/larch:9:in<top (required)>'
from /var/lib/gems/1.9.1/bin/larch:19:in load' from /var/lib/gems/1.9.1/bin/larch:19:in

[19:36:41] [info] imap.1and1.co.uk/INBOX/supply-infreq/direktronik -> imap.gmail.com/INBOX/supply-infreq/direktronik

...

[19:36:41] [info] [<] INBOX/supply-infreq/di.se: fetching latest message flags...
[19:36:41] [info] [>] INBOX/supply-infreq/di.se: fetching latest message flags.../usr/lib/ruby/1.9.1/thread.rb:68:in sleep': deadlock detected (fatal) from /usr/lib/ruby/1.9.1/thread.rb:68:inwait'
from /usr/lib/ruby/1.9.1/monitor.rb:97:in wait' from /usr/lib/ruby/1.9.1/net/imap.rb:1170:insend_literal'
from /usr/lib/ruby/1.9.1/net/imap.rb:1373:in send_data' from /usr/lib/ruby/1.9.1/net/imap.rb:1145:insend_data'
from /usr/lib/ruby/1.9.1/net/imap.rb:1072:in block (2 levels) in send_command' from /usr/lib/ruby/1.9.1/net/imap.rb:1070:ineach'
from /usr/lib/ruby/1.9.1/net/imap.rb:1070:in block in send_command' from /usr/lib/ruby/1.9.1/monitor.rb:190:inmon_synchronize'
from /usr/lib/ruby/1.9.1/net/imap.rb:1064:in send_command' from /usr/lib/ruby/1.9.1/net/imap.rb:635:inappend'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:78:in block in append' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:188:insafely'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:72:in append' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:230:inblock in copy_messages'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:104:in block in each_db_message' from /var/lib/gems/1.9.1/gems/sequel-3.9.0/lib/sequel/dataset/actions.rb:15:ineach'
from /var/lib/gems/1.9.1/gems/sequel-3.9.0/lib/sequel/dataset/actions.rb:15:in all' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap/mailbox.rb:104:ineach_db_message'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:188:in copy_messages' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:72:inblock in copy_all'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:126:in block in each_mailbox' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:126:ineach_value'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch/imap.rb:126:in each_mailbox' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/lib/larch.rb:60:incopy_all'
from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/bin/larch:118:in <module:Larch>' from /var/lib/gems/1.9.1/gems/larch-1.1.0.dev.20100209/bin/larch:9:in<top (required)>'
from /var/lib/gems/1.9.1/bin/larch:19:in load' from /var/lib/gems/1.9.1/bin/larch:19:in

[19:36:41] [info] imap.1and1.co.uk/INBOX/supply-infreq/direktronik -> imap.gmail.com/INBOX/supply-infreq/direktronik

Feature request from @akumar via Twitter: "estimate of how long the sync is likely to take (or at least the # of objects synced/not synced)."

This would require pre-scanning all mailboxes being synced in order to gather the necessary info so it would add some time to the sync, but should be doable. Cached mailbox state info can potentially reduce the pre-scanning time.

A user has reported that Larch dies consistently with an 'Illegal instruction' error while scanning a large Gmail mailbox. Some preliminary research indicates this error may be coming from SQLite, and may be caused by memory exhaustion, but more investigation is needed.

See http://groups.google.com/group/larch/msg/83bc2f3f90186944

When copying messages that have the \Recent flag set on the source server, Larch tries to set the flag on the destination server as well. However, the IMAP spec prohibits clients from setting this flag. It seems many IMAP servers ignore this, but some (including Zimbra) return an error.

Running the latest --pre version, I get

Net::IMAP::BadResponseError: Invalid system flag \NONJUNK

and sometimes

Net::IMAP::BadResponseError: Invalid system flag \RECEIPT-HANDLED

for certain messages. I'm using a dovecot server, and have been using the Evolution client there.

The BlitzMail IMAP server implementation reportedly fails to adhere to the IMAP standard when sending ENVELOPE responses. It fails to quote or properly escape strings in some cases, which can cause Net::IMAP's response parser to hang.

http://groups.google.com/group/larch/msg/37a8e5f383006b87

Should work just like --all, but only copy subscribed folders.

When the —all option is specified for a copy operation, Larch should recursively sync all folders.

Apparently Gmail doesn't allow whitespace before or after folder names, but many other IMAP servers do. Larch should trim folder names when Gmail is the destination in order to prevent problems.

Larch currently uses some case sensitive folder name comparisons, which can cause problems when dealing with the INBOX folder, which is defined in RFC 3501 as being case insensitive.

See http://groups.google.com/group/larch/browse_thread/thread/79b3797603654d13

I just completed moving email from an IMAP account into Gmail. Unfortunately, I now have a folder in my Gmail account called INBOX/Sent Items while my Gmail's Sent Items is empty.

What is the proper syntax to move email into Gmail's Sent Mail?

I tried:
--from-folder 'INBOX.Sent Items' --to-folder 'Sent Mail'

but that created a new folder on Gmail called [Imap]/Sent Mail.

Logs: https://gist.github.com/619ea0a5c76f6f61ec8e

Mailing list thread: http://groups.google.com/group/larch/browse_thread/thread/a20a7c963fb3e988

when i try to copy emails with blank subject lines to Gmail, i get an error: "Unable to append message to folder."

Any idea how to resolve this?

[Dec 07 03:06:18] [info] copying message: [email protected] - Re: 
[Dec 07 03:06:19] [info] [email protected]@imap.gmail.com: Net::IMAP::NoResponseError: Unable to append message to folder (Failure) (will retry)
[Dec 07 03:06:21] [info] [email protected]@imap.gmail.com: Net::IMAP::NoResponseError: Unable to append message to folder (Failure) (will retry)
[Dec 07 03:06:24] [info] [email protected]@imap.gmail.com: Net::IMAP::NoResponseError: Unable to append message to folder (Failure) (will retry)
[Dec 07 03:06:27] [error] Net::IMAP::NoResponseError: Unable to append message to folder (Failure) (giving up)

Add an option to use the Date header of a message to set the internal date on the destination server rather than using the internal date from the source server (which in some cases is incorrect).

Hi,

I see the feature to delete from source after copying (#22) - but is there the opposite, to delete mails on the target that weren't present on the source?

My purpose is to keep an exact sync of a gmail account on another IMAP server. If I sync and then on gmail, move some items into a folder (label + archive), on the next sync I'll have a copy in both the folder and in the inbox, on the target, which is not ideal for my purposes.

Gmail doesn't properly encode NUL characters in message bodies, which can cause Net::IMAP to hang.

This issue happens whenever you have a folder with the same name but different casing between source and destination (ie. "folder" - "Folder"). IMAP protocol is case sensitive, but gmail isn't and throws a fatal error that stops the migration.

I coded a simple patch that adds a new quirk, that checks a regexp for the name.

def mailbox(name, delim = '/')
    retries = 0

    name.gsub!(/^(inbox\/?)/i){ $1.upcase }
    name.gsub!(delim, self.delim)

    name.gsub!(/\s+/, ' ') if @quirks[:gmail]

    # Gmail doesn't allow folders with leading or trailing whitespace.                                                                                       \

    name.strip! if @quirks[:gmail]

    begin
      if @quirks[:gmail]
        # In Gmail folders are not case sensitive                                                                                                             
        update_mailboxes
        @mailboxes.keys.each do |key|
          if key =~ /^#{Regexp.quote(name)}$/i
            return @mailboxes[key]
          end
        end
        raise MailboxNotFoundError, "mailbox not found: #{name}"
      end

      @mailboxes.fetch(name) do
        update_mailboxes
[....]

Just a missing return

def imap_uid_fetch(set, fields, block_size = FETCH_BLOCK_SIZE, &block) # :yields: fetch_data
    if set.is_a?(Numeric) || (set.is_a?(Range) && set.last < 0)
      data = @imap.safely do
        imap_examine
        @imap.conn.uid_fetch(set, fields)
      end

      yield data unless data.nil?

      # Return was missing, so it continued processing afterwards   
      return
    end

Excerpt from a user email to the Larch mailing list:

"I found that I couldn't transfer the entire tree in go (--all), as
larch choked on one particular folder:

From-server IMAP extract (insane logging):

S: * CAPABILITY IMAP4rev1 UIDPLUS IDLE LOGIN-REFERRALS NAMESPACE QUOTA
CHILDREN AUTH=DIGEST-MD5
S: * LIST (\HasChildren) "/" work/foo
S: * LIST (\HasNoChildren) "/" work/foo/bar
S: * LSUB (\HasChildren) "/" work/foo
S: * LSUB (\HasNoChildren) "/" work/foo/bar

Gmail IMAP extract:

S: * LIST (\HasNoChildren) "/" "work/foo"
S: * LSUB (\HasNoChildren) "/" "work/foo"

Gave this:

[Jun 15 19:40:37] [fatal] mailbox not found: work/foo/bar

I worked around the problem by pulling one folder at a time, which
worked fine, including for the folder which it had originaly choked
on."

A user has reported that the "all" config option is not being respected in the latest Larch dev gem. See http://groups.google.com/group/larch/msg/bc80ea058a9ca123

Hi,
this tool is great!
I used it to migrate a large imap account from googlemail to another provider.
However, I think that there are many users who would like to use the tool without needing to install ruby / gems etc. on their windows machines.
I suggest to release a .exe file created with something like http://www.erikveen.dds.nl/rubyscript2exe/
Regards,
J83

I am running Larch 1.0.2 and getting this error which stops the transfer process.

[fatal] can't dup NilClass