Migrating to Zimbra with imapsync

    In addition to such long-awaited features like the Hierarchical Address Book and official support for Ubuntu 18.10 LTS, the release of Zimbra 8.8.12brought us news about the upcoming removal from support of a number of utilities necessary for migration from other mail systems to the Zimbra Collaboration Suite. In particular, support for the ZCS Migration Wizard for Domino was completely discontinued, and the ZCS Migration Wizard for Exchange will now only support importing PST files from Outlook. To implement a full-fledged migration, the developers of Zimbra Collaboration Suite offer to use paid online services, but not every company is ready to provide their mailbox data to third parties for security reasons, and not every company can afford the additional costs associated with migrating to Zimbra. Especially for such users, there is imapsync,

    image

    Imapsync is a free and free utility available in the repositories of any officially supported Zimbra Linux distribution. The purpose of this program is to transfer email between different mail servers and it copes with it perfectly. Thanks to the flexibility of settings, mail can be transferred incrementally and in a fully automatic mode. Let's see how you can use imapsync to migrate from Lotus Domino and MS Exchange to the Zimbra Collaboration Suite.

    Migrating from MS Exchange to Zimbra


    In order for email migration to succeed, you need to complete a series of preparations. First of all, we need to create the same users on the Zimbra server as on the MS Exchange server with the same passwords. In performing this task, it is very useful to auto-configure Zimbra users using AD. The process of such auto-tuning is described in one of our previous articles.

    After all the accounts from AD or Exchange are migrated, you need to install the imapsync package on the server with Zimbra. If you use Ubuntu as the OS for Zimbra, you can install the program using the command sudo apt-get install imapsync -y , in CentOS and RHEL, the installation will be carried out using the commandsudo yum install imapsync -y .

    When the installation of the package and all the dependencies is completed, let's move on to configuring the Exchange server. On it, we will need to enable the IMAP4 service. This is done by running the services.msc file , after which you need to go to Tools> Services -> IMAP4 Microsoft Exchange -> Start. After that, you will need to give the administrator access to user mailboxes in order to avoid the use of user passwords during migration, and also disable or configure speed limit policies so that they do not become an obstacle to migration. Now you can start transferring mail from server to server.

    In our infrastructure, MS Exchange will be hosted on exchange.company.ru, and Zimbra will be hosted on zimbra.company.ru . Users will be transferred one by one using the command line, in which we will run the imapsync utility with a number of parameters necessary for migration. First of all, it should be noted that in Zimbra, as in MS Exchange, administrator accounts have access to the contents of all user mailboxes, which means that in order to migrate to a new mail server, the system administrator does not need to know the passwords from ordinary user accounts. That is why the migration of each user from MS Exchange to Zimbra will be carried out using a command of the following form:

    imapsync --host1 exchange.company.ru --user1 "xxxxx" --authuser1 "yyyyy" --password1 "zzzzz" --nosslcheck --host2 zimbra.company.ru --user2 "XXXXX"  --authuser1 "YYYYY" --password2 "ZZZZZ"

    In this command, we specify the source host ( --host1 ) from which imapsync will pick up account data, as well as the host ( --host2 ) to which this data will be recorded. We also specify the username ( --user1 ) on the first host and the corresponding username ( --user2 ) on the second host. In order to access its data we specify the name of the administrator account ( --authuser1 ) and the administrator password ( --password1 ) of the first host, and in order to record the transmitted data, the name of the administrator account ( --authuser2 ) and password administrator ( --password2 ) of the second host.

    In an ideal world, such a command would be enough to transfer user data between servers with MS Exchange and Zimbra, but, as often happens, the differences between solutions make their own corrections and therefore a simple transfer of data from MS Exchange will lead to terrible chaos in Zimbra mailboxes . The stumbling block in this case is the different names of the folders in the mailboxes. In order to avoid confusion during data transfer, it is necessary to bring our team to approximately the following form:

    imapsync --subscribe_all --maxmessagespersecond 4 --addheader --f1f2 "Inbox=INBOX" \
    --f1f2 "Sent Items=Sent" --f1f2 "Sent items=Sent" --exclude "^Journal|^Tasks|^Trash|Contacts|Calendar|RSS Feeds" \
    --useheader "Message-ID" --buffersize 8192000 --nosyncacls  --allowsizemismatch --usecache --syncinternaldates --skipsize --nofoldersizes \
    --tls1 --port1 143 --user1 "xxxxxxxxxx" --authuser1 "besadmin" --password1 "XXXXXXX" \
    --host2 'xxxxxxx' --user2 "xxxxxx" --authuser2 "XXXXXX" --password2 "XXXX" --tls2 --port2 7143 \
    --regextrans2 "s/Inbox1/INBOX/i" --regextrans2 "s/Sent Items/Sent/i" --regextrans2 "s/sent-mail/Sent/i" --regextrans2 "s/Deleted Items/Trash/i" --regextrans2 "s/Deleted Messages/Trash/" \
    --delete2duplicates --timeout 600 --pidfile "impasync.local.pid" --logfile "imapsync.log" --sep1 "/" --errorsmax "100"

    Note that this is not a universal script that will work flawlessly in any infrastructure. A lot of things in it may depend on various features of your infrastructure. That is why, before starting a full-scale migration, be sure to test the transfer of data between systems on a number of non-critical business accounts.

    Let the abundance of parameters with which imapsync is launched not scare you. most of them are needed precisely so that when transferring folders from the mailbox there is no confusion and all elements from MS Exchange are transferred to Zimbra. For example, the --subscribe_all parameter allows imapsync to get all the folders in your account so that you can work with them later. Option --f1f2allows you to change the name of the folder during synchronization. That is, when synchronizing, for example. the Sent items folder changes its name to Sent, which allows you to perfectly fit its contents into the folder structure of Zimbra Collaboration Suite. The --regextrans2 parameter also allows you to change folder names and is based on the regex mechanism of the Perl programming language, so those who are familiar with this language will easily learn this parameter.

    To explain what the --addheader and --useheader options do, it will be necessary to delve into how exactly the synchronization of letters in imapsync occurs. The fact is that in order to synchronize imapsync, it is necessary to identify letters on both servers in order to decide whether to synchronize them or not. To verify them, this utility uses parameters such as “Message-Id:” and “Received:” (identification number and time of receipt), which are absent in such emails, such as drafts. In order to synchronize them normally, we need the --addheader command. It allows you to add an identification number to them in order to somehow identify and compare such letters during synchronization. The --useheader parameter allows you to compare letters using only one of two parameters during synchronization.

    After writing the perfect script that will allow us to transfer the perfect transfer of accounts from MS Exchange to Zimbra, a significant problem remains, which is related to the fact that we still have to transfer users between servers one at a time. This is not very optimal, especially in those companies where there are many employees and the number of mailboxes is measured in the thousands. However, this process can be significantly optimized by creating a list of accounts, and then write a small script that automatically transfers mail based on account names. To do this, we will create the imap_users file and add all the necessary data to it in the following form:

    domains=(
    domain1\
    domain2)
    users=(
    user1@domain1\
    user2@domain1\
    user3@domain1\
    user1@domain2\
    user2@domain2\
    user3@domain2)

    After that, we will write the script itself. We got something like this:

    . imap_users
    src_srv = exchange.company.ru
    dest_srv = zimbra.company.ru
    for ((i = 0; i < ${#users[@]}; i++ ))
    do
    /usr/bin/imapsync --subscribe_all --maxmessagespersecond 4 --addheader --f1f2 "Inbox=INBOX" \
    --f1f2 "Sent Items=Sent" --f1f2 "Sent items=Sent" --exclude "^Journal|^Tasks|^Trash|Contacts|Calendar|RSS Feeds" \
    --useheader "Message-ID" --buffersize 8192000 --nosyncacls  --allowsizemismatch --usecache --syncinternaldates --skipsize --nofoldersizes \
    --tls1 --port1 143 --host1 $src_srv --user1 ${users[$i]} --authuser1 "besadmin" --password1 "XXXXXXX" --nosslcheck \
    --host2 $dest_srv --user2 ${users[$i]} --authuser2 "XXXXXX" --password2 "XXXX" --tls2 --port2 7143 \
    --regextrans2 "s/Inbox1/INBOX/i" --regextrans2 "s/Sent Items/Sent/i" --regextrans2 "s/sent-mail/Sent/i" --regextrans2 "s/Deleted Items/Trash/i" --regextrans2 "s/Deleted Messages/Trash/" \ 
    --delete2duplicates --timeout 600 --pidfile "impasync.local.pid" --logfile "imapsync.log" --sep1 "/" --errorsmax "100"
    done

    Since the data transfer is incremental, you can first transfer the bulk of the data between the servers, and then, within a short technical break, transfer the small amount of data that has accumulated since the last synchronization, transfer all incoming mail to Zimbra and make it the main mail system. We remind you that before migrating to the new mail system, you need to notify employees about the upcoming transition, conduct training with them, and also prepare memos about where in Zimbra their usual functions are located. Thanks to this, migration will go more smoothly and not create problems for the business.

    Migrating from Lotus / Domino to Zimbra


    In many ways, the migration process from the IBM mail system using imapsync repeats the migration process from MS Exchange and any other mail system. However, in the process of direct migration, many administrators encounter an error with disconnection via IMAP and, accordingly, termination of synchronization. At the same time, no error message appears in the logs. If during the migration from Lotus / Domino you encountered this problem, then the following actions will help to solve this issue:

    • Launch Lotus Notes with administrator privileges and go to the Address Book.
    • In the Address Book settings, change the default message format to “Preferred MIME”
    • In the same Lotus Notes running with administrator privileges, change the format of the user database to Mail (R6), which is taken from the server, and not from the mail client
    • Turn off the Lotus Domino Router from the Domino administration console using the tell router quit command
    • Pack the user database from the Domino administration console using the “load compact userdb.nsf” command
    • Check the database for errors by running the fixup userdb.nsf command from the Domino console (not from the administrator console)
    • From the Domino administration console, run the command “convert -e userdb.nsf” to prepare the database for imap transfer
    • From the Domino administration console, run the command “convert -h userdb.nsf” to add imap headers to all messages.

    After completing all these steps, transferring mail from Lotus / Domino to Zimbra using imapsync should pass without any problems.

    In addition to utilities for migration, Zimbra clients for desktop computers and mobile devices were left without support. Also, HTML web client was removed from support, which was used to ensure compatibility with older browsers and low-power devices. All of them can be easily replaced by third-party customers, due to the fact that Zimbra supports almost all open standards and protocols.

    Recall that migration between Zimbra servers can be done using the free Zextras Migration Tool, which you can download on the Zextras official website .

    For all questions related to the Zextras Suite, you can contact the representative of Zextras Katerina Triandafilidi by e-mail katerina@zextras.com

    Also popular now: