Migrating from Mailman 2.1 to Mailman 3

This guide covers the migration steps required for porting from Mailman 2.1 to Mailman 3.

Why Upgrade?

Mailman 3 is the new version and actively developed, as compared to 2.1, which is now in maintenance mode and won’t receive any feature updates.

Some of the reasons that might convince you to migrate to Mailman 3 include

  • a well tested and rewritten code base,

  • support for REST API (if you need to integrate with other services)

  • support for multiple domains in same installation (without listname collisions)

  • A real database backend for settings and configuration

  • Modern Web UI for list administration and subscription management

  • Support for social logins (and possibly LDAP/openid integration)

  • Much improved and interactive archiver/forum with ability to reply from within the Archiver

Other considerations

Before you upgrade, you should consider a few things like:

  • URLs to archived messages will break, unless you take extra steps to keep them around. Upgrade mechanism makes sure to import all your archived messages in the new system, but, all the URLs to the new messages are going to be different.

    If you need your URLs for Mailman 2 archives to work, you can keep the HTML files generated for the archives around and your web server configuration for the archives intact (possibly with a notice to viewers that it is now a read-only archive, see this list for example).

  • The above mechanism won’t work for private archives since the archives are gated with password and without a Mailman 2 list, there is no password. You can however import them to Mailman 3.

  • Some configuration and settings aren’t available in Mailman 3’s UI yet, so even though those settings will be migrated to Mailman 3, you may not be able to change them from the Web UI today. All of those settings should be exposed in the UI very soon.

  • Importing archives with hyperkitty_import will never send outgoing emails. The messages are only imported from the old archive to the new archive.

  • hyperkitty_import can be run multiple times with an input mbox containing some or all of the same messages. Regardless of other settings, messages which exist in the archive will not be re-imported on subsequent runs.

  • After running import21 to import a list, those members will immediately be able to receive and send mail, and generally participate in the list. They will be members at the level of the mailman3_core database. However if they choose to manage their subscriptions in the web UI, which offers many more options, it will require going to “Sign Up” in the upper right corner of the website, and creating a new Django account. See further discussion about this on the Architecture Page in the section “Understanding User and Member Accounts”.

Before you upgrade

  • Make sure that you have no pending subscription requests as those will not be ported over to Mailman3.

  • Make sure that you don’t have any pending emails in the digest mbox, if there are, you can force send the digests before moving to Mailman 3, as those won’t be upgraded to Mailman3.

Upgrade strategy

As of now, there isn’t a turn key solution to migrate all your lists from Mailman 2 to Mailman 3, although, the process is fairly automated. You need shell access to your installation in order to perform the upgrade.

Mailman 3 is split in two main parts, the Core engine which includes all the lists and their configuration and the Web UI, which includes the archiver and UI for information in Core.

Before you start with migration, you need a working Mailman 3 instance, you can see here for recommendations on installing Mailman.

Some key information that you should know about your Mailman 2 and Mailman 3 installations before you do the migration:

  • Location of list configuration in Mailman 2: This is typically at $var_prefix/lists/LISTNAME/config.pck where $var_prefix is an installation dependent directory which is typically /usr/local/mailman or /var/lib/mailman.

  • Location of list archives in Mailman 2: This is typically at Mailman 2: This is at $var_prefix/archives/private/LISTNAME.mbox/LISTNAME.mbox where $var_prefix is as above.

  • Location of the bin/ commands in Mailman 2: This is at $prefix/bin where $prefix is an installation dependent directory which is typically /usr/local/mailman or /usr/lib/mailman

Steps for migration:

  • Create the list you are trying to migrate in Mailman 3, for the purposes of this guide, we will call it foo-list@example.com

  • Migrate the list configuration from Mailman 2 to Mailman 3 by running the following command [1]

    $ mailman import21 foo-list@example.com /path/to/mailman2/foo-list/config.pck
    
  • Migrate the list archives from Mailman 2 to Mailman 3 by running the following command [2]:

    $ python manage.py hyperkitty_import -l foo-list@example.com $var_prefix/archives/private/foo-list.mbox/foo-list.mbox
    

After this, you will need to rebuild the index for this list:

$ python manage.py update_index_one_list foo-list@example.com

After this, you should be able to search your messages in Hyperkitty.

  • Delete Mailman 2 list:

    $ $prefix/bin/rmlist foo-list
    

After this, you may need some additional steps based on if you want to keep your old archives around or not. To add a notice to your list archives, you can edit index.html available at the root of your mailing list archives.

You may also want to add a redirect at old list page to automatically redirect your users to Mailman3 list-info page. However, this process is fairly manual depending on type of webserver you are using.