Virtualenv Installation

This is a step-by-step installation guide for Mailman Suite, also sometimes referred as Mailman 3 Suite or just Mailman 3. There are more than one ways to have Mailman 3 running on your machine.

Dependencies

Python3.5+.
Mailman requires any version of Python > 3.5.
MTA Setup

Mailman 3 in theory would support any MTA or mail server which can send emails to Mailman over LMTP. Officially, there are configurations for Postfix, Exim4, qmail and sendmail. Mailman Core has a fairly elaborate documentation on setting up your MTA. Look below at `setting up mailman core`_ to find out the location of configuration file mailman.cfg which is mentioned in the documentation above.

The Web Front-end is based on a Python web framework called Django. For email verification and sending out error logs, Django also must be configured to send out emails.

Sass compiler

A sass compiler. Syntactically Awesome Stylesheets or Sass is an extension of CSS, which is used to style webpages, that adds more power to the old CSS syntax. It allows using variables, nested rules, inline imports etc. which CSS originally doesn’t support. Hyperkitty uses this to generate CSS styles.

You can either use the Ruby implementation or C/C++ implementation. Please look at the installation guide for sass to see how you can get one.

For apt based systems:

$ sudo apt install ruby-sass

For yum based systems try:

$ sudo yum install rubygem-sass

or:

$ sudo dnf install rubygem-sass

After installing this, you’d have to configure Django to use these compilers. A basic configuration would look like:

# To be added to Django's settings.py

COMPRESS_PRECOMPILERS = (
  ('text/x-scss', 'sass -t compressed {infile} {outfile}'),
  ('text/x-sass', 'sass -t compressed {infile} {outfile}'),
)

You can replace sass above with whatever is the name of the binary is. For the Ruby implementation it is generally sass, for C/C++ it is sassc.

Python development packages

python3 dev package. This is required for building postorius

For Fedora the package is python3-devel:

$ sudo dnf install python3-devel

For Ubuntu/Debian the package is python3-dev:

$ sudo apt install python3-dev
Fulltext search

A full text search engine like Whoosh or Xapian. Whoosh is the easiest to setup and can be installed in the virutualenv.

See also

Virtualenv setup on how to setup the virtualenv.

Lynx
An HTML to plaintext converter like lynx is required by Mailman Core if you have configured it to convert emails to plaintext.

Virtualenv setup

Virtualenv is Python’s mechanism to create isoated runtime environments.

Hint

If you are not familiar with virtualenv, checkout the user guide for virtualenv.

1. Setup working directory:

# Setup the base installation directory.
$ sudo mkdir -p /opt/mailman

2. Create the virtualenv for Mailman:

$ cd /opt/mailman
$ python3 -m venv venv

3. Activate the virtualenv:

$ source venv/bin/activate

Note

The rest of this documentation assumes that virtualenv is activated, which can be done by step 3 mentioned above. Whether or not virtualenv is activated can be seen by a (venv) before the shell prompt.

Installing Mailman Core

Mailman Core is responsible for sending and receiving emails. It exposes a REST API that different clients can use to interact with over an HTTP protocol. The API itself is an administrative API and it is recommended that you don’t expose it to outside of your host or trusted network. To install Core run:

(venv)$ pip install mailman

This _should_ install the latest release of Mailman Core, which is 3.1.0 as of writing of this document. After this, create a configuration file at /etc/mailman.cfg for Mailman Core.

See also

The further configuration setup for Mailman Core Configuring Mailman Core.

After this, running mailman info will give you an output which looks something like below:

(venv)$ mailman info
GNU Mailman 3.1.0 (Between The Wheels)
Python 3.5.3 (default, Jan 19 2017, 14:11:04)
[GCC 6.3.0 20170118]
config file: /etc/mailman.cfg
db url: sqlite:////var/lib/mailman/data/mailman.db
devmode: DISABLED
REST root url: http://localhost:8001/3.1/
REST credentials: restadmin:restpass

Installing Web UI

Postorius and Hyperkitty are Mailman’s official Web UI and Archiver. Both of them are Django based apps and can be integrated into an existing Django installation/website.

Installing Postorius & Hyperkitty

Postorius and Hyperkitty are Django “apps” and can be directly installed using the commands below:

(venv)$ pip install postorius
(venv)$ pip install hyperkitty

mailman-hyperkitty plugin enables interaction between Core and Hyperkitty. You can install it using:

(venv)$ pip install mailman-hyperkitty

Note that mailman-hyperkitty is a plugin for Mailman Core and not Django.

Setting up Django Project

See also

What is Django?

If you have absolutely no idea about Django, just clone/download this mailman-suite repo . It should have a documented settings.py and a pre-configured urls.py which you can use to run new Django installation. If you find any setting that is not documented, please look at Django’s settings reference for all available settings.

Exact commands would look something like this:

# Download and install the latest release of django.
(venv)$ pip install Django>=1.11

# Clone the repo locally.
(venv)$ git clone https://gitlab.com/mailman/mailman-suite.git
(venv)$ cd mailman-suite/mailman-suite_project/

# Create the tables in the database and load fixtures.
(venv)$ python3 manage.py migrate

# Copy all the static files to one single location.
(venv)$ python3 manage.py collectstatic

# Run the Django's "development" server at localhost:8000
(venv)$ python3 manage.py runserver

Setting up a WSGI server

See also

What is WSGI?

These instructions are to setup your Django website behind a webserver. We are using a uwsgi as the wsgi server to communicate between the webserver and Django. To install uwsgi, run:

(venv)$ pip install uwsgi

Note

The configuration below doesn’t serve static files, so if you are just “trying-it-out” and want static files to be served, you need to add some additional configuration and steps. See serving static files with uwsgi.

Then you can configure it using the following configuration file:

# uwsgi.ini
#
[uwsgi]
# Port on which uwsgi will be listening.
http-socket = 0.0.0.0:8000

# Move to the directory wher the django files are.
chdir = /path-to-django-project-directory/

# Use the wsgi file provided with the django project.
wsgi-file = wsgi.py

# Setup default number of processes and threads per process.
master = true
process = 2
threads = 2

# Drop privielges and don't run as root.
uid = 1000
gid = 1000

# Setup the django_q related worker processes.
attach-daemon = ./manage.py qcluster

# Setup the request log.
req-logger = file:/path-to-logs/logs/uwsgi.log

# Log cron seperately.
logger = cron file:/path-to-logs/logs/uwsgi-cron.log
log-route = cron uwsgi-cron

# Log qcluster commands seperately.
logger = qcluster file:/path-to-logs/logs/uwsgi-qcluster.log
log-route = qcluster uwsgi-daemons

# Last log and it logs the rest of the stuff.
logger = file:/path-to-logs/logs/uwsgi-error.log

You can run uwsgi using the following command:

(venv)$ uwsgi --ini /path/to/uwsgi.ini

Note that in the above configuration, there is a command called python manage.py qcluster which run the django-q processes. You can remove this from here if you want to manage this yourself via some other init process.

Have a look at `uwsgi`_ documentation to learn more about different configuration options. One minor optimization that can be done is to replace:

http-socket = 0.0.0.0:8000

with a more performant option:

uwsgi-socket = 0.0.0.0:8000

However, this requires support for uwsgi protocol in the webserver. Nginx and Apache, the two most popular web-servers have the support and the Nginx configuration below uses uwsgi-socket.

Nginx Configuration

You can reverse proxy the requests to uwsgi server using Nginx. Uwsgi has a special protocol called uwsgi protocol that is available in Nginx as a plugin. Add the following configuration to your sites-availbale in Nginx

Note

This configuration relies on uwsgi listening on a uwsgi-socket instead of a http-socket. Please make sure before using this configuration.

Also, if you must use the http-socket option for some reason, replace the line below with uwsgi_pass 0.0.0.0:8000; with proxy_pass http://0.0.0.0:8000;.

server {

   listen 443 ssl default_server;
   listen [::]:443 ssl default_server;

   server_name MY_SERVER_NAME;
   location /static/ {
        alias /path-to-djang-staticdir/static/;
   }
   ssl_certificate /path-to-ssl-certs/cert.pem;
   ssl_certificate_key /path-to-ssl-certs/privkey.pem;

   location / {
           include uwsgi_params;
           uwsgi_pass 0.0.0.0:8000;

   }

}

Fill in the appropriate paths above in the configuration before using it.