The Definitive Mailman Suite Development Setup Guide

Mailman 3 consists of a collection of separate-but-linked projects, each of which has its own development setup guide. This makes sense when you want to focus on a single piece of Mailman, but can make setting up the entire Mailman Suite in one go somewhat confusing. This guide attempts to move all the information currently in the wiki and various package documentation into a single “definitive” guide.

This document currently collates information from the following sources: 1. The Mailman Wiki:

  1. Main package documentation on Readthedocs.io:

Getting prerequisites

For the most part, setup for each project will download any needed packages. However, you will need a few system packages to be sure you’ve got the necessary version of Python and its tools, git (to get the source code), postfix (a mail server), and a few other tools that are used during setup.

On Fedora, you probably want to run:

$ sudo yum install python-setuptools python-devel python-virtualenv python3-devel git gcc nodejs-less postfix

On Debian and Ubuntu, this may be something like:

$ sudo apt-get install python-setuptools python-dev python-virtualenv python3-dev git gcc node-less nodejs postfix

If you prefer, you can substitute Exim4 for Postfix. Postfix is the MTA used by most Mailman developers, but we do support Exim 4. (Sendmail support is very much desired, but the Mailman core developers need contributors with Sendmail expertise to help.)

You will also want tox to run tests. You can get this using “pip install tox”.

HyperKitty also needs sassc. FIXME: add instructions on how to get sassc on a few platforms.

Set up a directory

Setting up the whole Mailman suite means you have to pull code from a bunch of different related repositories. You can put all the code anywhere you want, but you might want to set up a directory to keep all the pieces of mailman together. For example:

$ mkdir ~/mailman
# cd ~/mailman

For the rest of this development guide, we are going to assume you’re using ~/mailman as your directory, but you can use whatever you want.

Set up virtual environments

Mailman core currently requires Python 3, while Postorius and HyperKitty require Python 2.7. The easiest way to handle this is to make sure you have virtualenvs for both environments.

To create the virtualenvs, you’ll use different commands for Python 3 and Python 2.7 as follows:

$ python3.5 -m venv venv-3.5
$ virtualenv venv-2.7

To activate a virtualenv, you need to run the appropriate activate script:

$ source venv-2.7/bin/activate

You must use source (or . if your shell is a pure POSIX shell). Make sure to always be using one of the virtualenvs when you’re setting up and running Mailman.

Set up and run Mailman Core

First, get the code:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/mailman.git

To set up Mailman Core, you’ll need to switch to your Python 3 virtualenv:

$ source venv-3.5/bin/activate

(Change the version number if you’re using a different version of Python 3, currently 3.4, 3.5, and 3.6 are supported.)

Then, go into the mailman directory, run setup, and then run mailman info to be sure everything is set up correctly, and that the right settings are in place:

$ cd mailman
$ python setup.py develop
$ mailman info

You can edit your mailman.cfg file to make any necessary changes. Then start things up:

$ mailman start
$ cd ..

You may want to deactivate your virtualenv now, since you’ll be using a different one for other components:

$ deactivate

Later on, if you need to restart Mailman (i.e. if you get the error “Mailman REST API not available. Please start Mailman core.”) then you can also do that by calling the mailman executable from the venv as follows:

$ ~/mailman/venv-3.5/bin/mailman start

Note that the mailman executable has several sub-commands. One that is particularly useful for debugging is mailman shell.

Set up Mailman Client

Get the code:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/mailmanclient.git

Then set up mailmanclient:

$ source venv-2.7/bin/activate
$ cd mailmanclient
$ python setup.py develop
$ cd ..
$ deactivate

Note that we’re using the Python 2.7 virtualenv here. (mailmanclient can also be run in the Python 3 virtualenv for experimentation. For supported versions of Python, you don’t need to be too careful about which interpreter you use because Python 2 and Python 3 do not share .pyc files.)

Set up Django-mailman3

This package holds the Django libraries and templates used by Postorius and HyperKitty.

Get the code and set it up:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/django-mailman3.git
$ source venv-2.7/bin/activate
$ cd django-mailman3
$ python setup.py develop
$ cd ..
$ deactivate

Set up and run Postorius

The Postorius documentation, including a more extensive setup guide, can be found here: http://postorius.readthedocs.org/

Make sure to install mailmanclient and django-mailman3 before setting up Postorius. (If you’re following this guide in order, you’ve just done that.)

Get the code and run setup. Make sure you’re in the 2.7 venv for Postorius:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/postorius.git
$ source venv-2.7/bin/activate
$ cd postorius
$ python setup.py develop
$ cd ..
$ deactivate

Postorius and HyperKitty both come with example_project directories with basic configuration so you can try them out. For this tutorial, however, we’ll be using a project that combines both instead.

Set up a mail server

To be able to actually receive emails, you need to setup a mail server. Mailman core receives emails over LMTP Protocol, which most of the modern MTAs support. However, setup instructions are provided only for Postfix, Exim4 and qmail. Please refer to the MTA documentation at Mailman Core for the details.

You will also have to add some settings to your django configuration. The setup instructions are provided in django’s email documentation.

Set up and run HyperKitty

Complete guide here: https://hyperkitty.readthedocs.org/en/latest/development.html

Make sure to install mailmanclient and django-mailman3 before setting up Hyperkitty. (If you’re following this guide in order, you’ve just done that.)

Get the code and run setup:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/hyperkitty.git
$ source venv-2.7/bin/activate
$ cd hyperkitty
$ python setup.py develop
$ cd ..
$ deactivate

Postorius and HyperKitty both come with example_project directories with basic configuration so you can try them out. By default, they both use port 8000, so if you do want to run both example projects at the same time, do remember that you’ll need to specify a different port on the command line for one of them.

However, we’re going to run them both in a single Django instance at the end of this guide, so don’t worry about ports right now.

Set up mailman-hyperkitty

mailman-hyperkitty is the package that actually sends the incoming emails to HyperKitty for archiving. Note that this is one of the components that uses Python 3.

Get it and set it up:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/mailman-hyperkitty.git
$ source venv-3.5/bin/activate
$ cd mailman-hyperkitty
$ python setup.py develop
$ cd ..
$ deactivate

You’ll need to fix the default mailman-hyperkitty.cfg file to use the correct url for HyperKitty. If you’re running it on http://localhost:8002 then you need to change base_url to match that.

Run the Mailman Suite (combined hyperkitty+postorius)

You can run HyperKitty and Postorius as separate applications, but many developers are going to want to run them on a single server. The configuration files for this are in a repository called mailman-suite.

The first time you run the suite, you will want to set up a superuser account. This is the account you will use in the web interface to set up your first domains. Please enter an email address otherwise the database won’t be setup correctly and you will run into errors later:

$ cd ~/mailman
$ git clone https://gitlab.com/mailman/mailman-suite.git
$ source venv-2.7/bin/activate
$ cd mailman-suite/mailman-suite_project
$ python manage.py migrate
$ python manage.py createsuperuser

You’ll want to run the following commands in a window where you can leave them running, since it dumps all the django logs to the console:

$ python manage.py runserver

At this point, you should be able to see Mailman Suite running! In the default setup, you can go to http://127.0.0.1:8000 and start poking around. You should be able to use the superuser account you created to log in and create a domain and then some lists.

The default config file uses a dummy email backend created by this line in settings.py:

EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'

Using this backend, all emails will be printed to the Postorius console (rather than sent as email) so you can get the url to verify your email from the console.

Don’t leave the console email backend configured and running once you get to the point where you want to send real emails, though!