How to migrate from Evolution to Thunderbird/Icedove

Here is a step by step procedure with script to migrate from Evolution to Thunderbird/Icedove. This will keep ALL datas (mail, calendar entries, tasks, contacts) as well as keeping mail status (important, read, replied, etc.) thanks to a script that I developed. This procedure has been done for Debian Sid but has command for Ubuntu Lucid as well. It was tested with Thunderbird/Icedove 3.1 and Evolution 2.28 to 2.32.

1. Backup Evolution and prepare the migration

1.1. Backup

- Put evolution in offline mode
- Backup from the GUI or use :

/usr/lib/evolution/2.32/evolution-backup --backup ~/USERNAME-evolution.tar.gz

If it hangs you can kill it once you saw the following message :
** Message: rm /home/letic/evolution.dir

1.2. Export data that can't be imported straight away

Export all contacts from Evolution by right clicking on each addressbook -> Save to vcard

1.3. Disable completely Evolution

  • Logoff from the session (avoid issue with the clock applet)
  • Log in as the user in a console (Ctrl+Alt+F1)
    mkdir backup
    mv ~/.gconf/apps/evolution ~/backup/evolution_gconf
    # For Evolution >= 2.30
    mv ~/.config/evolution/ ~/backup/evolution_config
    mv ~/.local/share/evolution ~/backup/evolution_data
    # For Lucid (evolution 2.28
    mv ~/.evolution ~/backup/evolution_data
    

    This will make sure that you have no entries left in the calendar and that you won't retrieve your mail by mistake in Evolution.

2. Install Thunderbird/Icedove + lightning/iceowl + enigmail

2.1. On debian

  • Install all required packages
    aptitude install icedove iceowl-extension 
    
  • Optional packages :
    • enigmail for PGP support
    • icedove-l10n-XX and iceowl-l10n-XX where XX stands for your locale

2.2. On Lucid

It is a bit more annoying as the lightning extension is missing.

  • Install Thunderbird
    aptitude install thunderbird 
    
  • Optional packages :
    • enigmail for PGP support
    • enigmail-locale-XX and thunderbird-locale-XX where XX stands for your locale
  • Install the lightning extension (no locales available in Ubuntu)
    • First you require xul-ext-calendar from natty
      wget `http://www.mirrorservice.org/sites/archive.ubuntu.com/ubuntu//pool/universe/l/lightning-extension/xul-ext-calendar-timezones_1.0~b2+build2+nobinonly-0ubuntu2_all.deb`
      dpkg -i xul-ext-calendar-timezones_1.0~b2+build2+nobinonly-0ubuntu2_all.deb
      
    • Then for 32bits
      wget `http://www.mirrorservice.org/sites/archive.ubuntu.com/ubuntu//pool/universe/l/lightning-extension/xul-ext-lightning_1.0~b2+build2+nobinonly-0ubuntu2_i386.deb`
      dpkg -i xul-ext-lightning_1.0~b2+build2+nobinonly-0ubuntu2_i386.deb
      
    • Or for 64 bits
      wget `http://www.mirrorservice.org/sites/archive.ubuntu.com/ubuntu//pool/universe/l/lightning-extension/xul-ext-lightning_1.0~b2+build2+nobinonly-0ubuntu2_amd64.deb`
      dpkg -i xul-ext-lightning_1.0~b2+build2+nobinonly-0ubuntu2_amd64.deb
      

      It is always better to use the packaged version instead of the xpi from the addons website. Especially as there is no 64bits version of lightning available for download on the official website.

3. Prepare Thunderbird/Icedove for the migration

  • Run Thunderbird/Icedove and create a fake imap access manually
  • Optional : set incoming/outgoing mail to UTF-8 in Edit -> Preferences -> Display -> Advanced

3.1. Filtaquilla

Filtaquilla is an addon that enhanced filters to be able to handle regex expressions and even javascript filters. These steps have to be done before importing the mail or the `x-evolution` header won't be in the DB.

  • Go to Edit -> Preferences -> Advanced -> Config editor

    and search for the `mailnews.customDBHeaders` preference and set it to `x-evolution`
  • Get the Filtaquilla plugin from `here`:https://addons.mozilla.org/en-US/thunderbird/addon/filtaquilla/ or run
    wget `https://addons.mozilla.org/thunderbird/downloads/latest/10052/addon-10052-latest.xpi?src=addondetail` -O `filtaquilla-1.1.0-tb+sm.xpi`
    
  • In Tools -> Addons -> Filtaquilla -> Preferences check Javascript
  • Close Thunderbird/Icedove

4. Migrating your data

4.1. Mails

  • Copy mbox files
    cp -ax ~/backup/evolution_data/mail/local/* ~/.icedove/PROFILE/Mail/Local\ Folders/
    
  • Remove all unneeded files
    cd ~/.icedove/PROFILE/Mail/Local\ Folders/
    find . -name `*ibex*` -delete
    find . -name `*.cmeta` -delete
    rm -r folders.db .#evolution.sbd/ Outbox Templates
    
  • Optional : Convert your mail files to UTF-8
    Be careful as this could do more harm than good !
    • First check your mailboxes encoding
      find . -type f -exec file {} \;
      
    • Convert to UTF-8 (for example if ALL you mailboxes are ISO-8859-15)
      find . -type f -print0 | while read -d $'\0' FILE; do iconv -f ISO-8859-15 -t UTF-8 `$FILE` > `$FILE.utf8`; mv `$FILE` `$FILE.old`; mv `$FILE.utf8` `$FILE`; done
      

4.2. Mail status

  • Go to Tools -> Message Filters
  • Select `Local Folders` and add a new message filter :
    • Type : Javascript
    • Matches
    • Content
      logger = 'MESSAGE : ' + message.getStringProperty('subject') + '\n';
      let evohdr = message.getStringProperty('x-evolution');
      logger += 'THUNDERBIRD FLAGS BEFORE : ' + message.flags + '\n';
      evostatus = evohdr.substring(evohdr.length - 4, evohdr.length);
      logger += 'EVOLUTION HEXA : ' + evostatus + '\nSTATUS : \n';
      var hexevostatus = parseInt(evostatus, 16);
      var flags = message.flags;
      deleted = false;
      if (hexevostatus & 1<<0) {
        logger += '   * is answered\n';
        flags = flags | 1<<1
      }
      if (hexevostatus & 1<<1) {
        logger += '   * is deleted\n';
        deleted = true;
      }
      if (hexevostatus & 1<<3) {
        logger += '   * is marked as important\n';
        flags = flags | 1<<2
      }
      if (hexevostatus & 1<<4) {
        logger += '   * is read\n';
        flags = flags | 1<<0
      }
      if (hexevostatus & 1<<5) {
        logger += '   * has attachment(s)\n';
        flags = flags | 1<<28
      }
      if (hexevostatus & 1<<7) {
        logger += '   * is junk\n';
        message.setStringProperty('junkscore', 100);
        message.setStringProperty('junkscoreorigin','user');
      }
      if (hexevostatus & 1<<11) {
        logger += '   * has been marked as not junk (no action)\n';
      }
      if (hexevostatus & 1<<12) {
        logger += '   * has been forwarded\n';
        flags = flags | 1<<12
      }
      message.flags = flags;
      logger += 'THUNDERBIRD FLAGS SET TO : ' + flags + '\n';
      logger +='WILL DELETE ? : ' + deleted;
      Cc[`@mozilla.org/consoleservice;1`]
         .getService(Ci.nsIConsoleService)
         .logStringMessage(logger);
      (deleted)
      
    • Action : Delete
  • Run the Javascript filter on all your imported folders to recover your mail status. It will perform the following action :
Status in Evolution Action in Thunderbird/Icedove
Unread emails Do nothing
Read emails Mark as read
Answered emails Mark as replied
Forwarded emails Mark as transferred
Mark as important Starred
Has attachments Same
Marked as SPAM Manually set the junkscore to 100 and mark for deletion
Deleted Mark them for deletion
  • All mails marked for deletion (junk/deleted) will then be deleted as part of the filter action. All these status are cumulative in both program even if Thunderbird/Icedove is the only one that actually display them. It will keep the flags set by Thunderbird/Icedove during the first import (especially `hasRe` status).
  • Unfortunately there is no way to un a filter on several folders (or all subfolders). This might be possible as an extension but as I had limited time to perform it this wasn't done. The quickest way is to run through your mailbox with the keyboard and type `Alt+T` then R.
  • When restarting the program or clicking on another folder and back the folder's read count will get updated EXCEPT if you selected the threaded display.

4.3. Contacts

  • Get the `More Functions for AddressBook`:http://nic-nac-project.de/~kaosmos/morecols-en.html plugin
    wget http://nic-nac-project.de/~kaosmos/morefunctionsforAB-TB3-0.6.4.1.xpi
    
  • Install the addon in Thunderbird/Icedove
  • Open the addressbook and right click in the blank contact list list (right of the screen) -> `Import vcard/vcf`
  • Once you selected the script it will take some time to import so please click continue the script for as long as it takes

4.4. Calendars

  • Open the calendar and go to `Event and Tasks` -> Import.
  • Your former evolution calendar will be in ~/backup/evolution_data/calendar/[local/]system (if you have several you will find them in ~/backup/evolution_data/calendar/[local/]XXXXXXXXXX.XXXX.X@YOURPC)

4.5. Tasks

  • Same as for the Calendar, open the tasks lists and go to `Event and Tasks` -> Import
  • The tasks are in ~/backup/evolution_data/tasks/system and ~/backup/evolution_data/tasks/XXXXXXXXXX.XXXX.X@YOURPC

4.6. Birthday calendar

If you miss the birthday calendar from Evolution install `Thunderbirthday`:https://addons.mozilla.org/en-us/thunderbird/addon/thunderbirthday/

4.7. Create your mail accounts

That's it you can now create your mail accounts in Thunderbird and get drown by all the mails you have received since closing Evolution.

4.8. Filters

I redid manually my filters because I wanted to clean them up, but a simple XSL to use with xsltproc should be easy to do if you really want to keep your filters. If you need some help to start :
  • Evolution >= 2.30 filters are in a XML file located in : ~/.config/evolution/mail/filters.xml
  • Evolution 2.28 filters are in ~/.evolution/mail/filters.xml
  • Thunderbird/Icedove filters are stored in ~/.icedove/PROFILE/Mail/Local Folders/msgFilterRules.dat, other POP accounts are in : ~/.icedove/PROFILE/Mail/POPACCOUNT/msgFilterRules.dat and IMAP accounts filters are in : ~/.icedove/PROFILE/ImapMail/IMAPACCOUNT/msgFilterRules.dat
  • Thunderbird/Icedove filters file has the following structure :
    version=`9`
    logging=
    name=
    enabled=
    type=
    action=
    actionValue=
    condition=
    name=
    enabled=
    type=
    action=
    actionValue=
    condition=
    ...
    

5. Integrate better in Gnome

5.1. Get calendar events in the gnome-clock applet

Install the `evolution-mirror`:https://addons.mozilla.org/en-US/thunderbird/addon/evolution-mirror/ plugin.

5.2. Integration in Ubuntu

Evolution-mirror should already give you a great integration with gnome but if you want a better integration in Ubuntu see this `post`:http://www.techgarten.com/ubuntu/replace-evolution-thunderbird-completely-ubuntu/

6. References

These are just some links/docs that helped me developing the script, if you wondered where did I found these numbers :
  • `Evolution (Camel) Messages flags`:http://developer.gnome.org/camel/stable/CamelFolderSummary.html
  • Mozilla doesn't have a online code browser so I had to pull the source to get their flags. In the nsMsgMessageFlags.h file you will find the following useful flags definition :
    class NS_NO_VTABLE NS_SCRIPTABLE nsMsgMessageFlags {
     public: 
    
      NS_DECLARE_STATIC_IID_ACCESSOR(NS_MSGMESSAGEFLAGS_IID)
    
      enum { Read = 1U };
    
      enum { Replied = 2U };
    
      enum { Marked = 4U };
      ...
      enum { Forwarded = 4096U };
      ...
      enum { Attachment = 268435456U };
    

Icedove_Advanced_Preferences.png (41.1 KB) Anthony Callegaro, 10/05/2011 18:03

Icedove_about_config.png (9.48 KB) Anthony Callegaro, 10/05/2011 18:06

Icedove_FiltaQuilla_preferences.png (16.5 KB) Anthony Callegaro, 10/05/2011 18:06