Django for Mailman Admins¶
This is a brief primer on Django to help with administration of Mailman as it is used as a web framework within the web interface of Mailman 3.
Ideally, Django should just be a python dependency and the details should be wrapped behind configurations and settings in Mailman. However, due to the amount of functionality provided to us by Django, it would be a monumental effort to hide it completely. We often resort to referring Django documentation directly in our documentation and also rely on several deployment related features that Django provides.
There are three main topics that is important for system administrators that we are going to cover in this doc, configuration, management commands and running (deployment).
Django expects its configuration to be provided via a
settings module. The
name of the module itself isn’t a constant and can be passed in as
--settings command line parameter in Django commands and also be set using
DJANGO_SETTINGS_MODULE environment variable.
The un-usual thing to note here is that it expects a module not a file
path, so whatever the name of the module is, it should be importable by
Django(in Python), which can be achieved by adding the directory the file
settings.py exists in the
PYTHONPATH environment variable. It works
similar to the
PATH environment varible that unix shell use.
Since it is using Python, you can use more Pythonisms in how the configuration
is defined or loaded. For example, a common pattern is to keep
as close to the one provided by the upstream project, but make the changes
required in a
settings_local.py and then import the configuration in
# settings.py <SNIP> try: from settings_local import * except ImportError # If the import fails, don't worry. pass # settings_local.py DATABASES = ... SECRET_KEY = ...
This way, you can keep the “local” configuration overrides in a separate
file. You can also take this a step further to put secrets into a
secrets.py and import from there in settings.py if you want to separate
secrets from other configuration.
Django upstream settings¶
Django’s upstream settings documentation is probably the most useful place to lookup the variables you might find in the Mailman provided settings.py but don’t fully understand how to configure.
All the settings are applicable directly, including things like LOGGING (to setup logging), DATABASES(to setup backend database configuration), USE_I18N(Internationalization of the interface), EMAIL_BACKEND (how to talk to local or remote MTA for sending emails).
On top of these base settings provided by Django, the configurations for
Postorius, Hyperkitty and django-mailman3 all belong to the same
settings_local.py, if you choose to use that).
There are other third party Django applications, which we reuse for various
functionality in Mailman-web use similar configuration process via
The most important packages for Mailman are:
- django-allauth: Good place to find any configuration related to account management in the Web UI from email sending issues to contents of the email sent for account/email verification.
- Django-haystack: This is used for search indexing of the emails in Hyperkitty. This is a good place to find various settings for setting up different search backends.
- Django-qcluster: This will provide all the asynchronous jobs that are run via cron in Hyperkitty to update various internals in a routine fashion. Use this to see how different backends can be configured to store and run jobs and how to tweak the configuraiton of each backend.
This is probably the most useful (and confusing!) this about Django due to the sheer number of ways that this can be done. Django management commands provide a framework to write and run commands for Django web applications for various manual or other tasks.
Django’s documentation on management commands is a good place to start, but if it is too long for you, this is a short know how for Mailman admins.
There are three variants for running management commands, all of which can be use interchangeably, depending on how you installed. Package managers, Docker images and mailman-web all provide different ways to run the same commands.
python3 manage.py --settings settings --pythonpath <parent dir of settings.py> <commamd>: This command is probably the _most_ common one that you can see. You will notice that it accepts
--pythonpathflags to provide the settings module of Django. You can also set the environment variables
PYTHONPATHinstead of passing the values to command.
django-admin --settings settings --pythonpath <parent dir of settings.py> <command>: This is exactly same as the above, replace the “manage.py” file with an installed command
django-admin, obviating the need to find the
manage.pyfile in your current installation. The same environment variables mentioned in the above marker works here as well.
mailman-web <command>: This is the simplified version of the above two commands, which comes with the mailman-web package. It defaults the settings.py to be present at
/etc/mailman3/settings.pyand automatically sets up the PYTHONPATH for Django to be able to import. You can also set
MAILMAN_WEB_CONFIGenvironment variable to point to a different config file (not module!) and it will take care of adding it to the PYTHONPATH and setting DJANGO_SETTINGS_MODULE.
The documentation of Mailman is slowly going to lean towards using
mailman-web command as it is simple, but third party packages and Django
can emit erorrs that use any of the above three notations. Which is why it is
good to know them. You can use
mailman-web <command> for commands from
createsuperuser for example) or from third
party packages (like
Running Django based web applications follows a Python standard mechanism called WSGI, which describes how a web server communicates with Python applications. There is also ASGI for async based applicaitons, but Mailman currently doesn’t use it.
You can use any WSGI server to run mailman-web. Our current documentation recommends using uwsgi, but gunicorn is also a great option. Some web servers like Apache2 come with wsgi modules, which obviate the need for a separate WSGI server like uwsgi, gunicorn. So, if your current infrastructure is already using Apache2, this might be an option for you.
It is good to refer to uwsgi docs or gunicorn docs if you want to configure the aspects of running Django that aren’t already done in Django, which includes things like, access/error logging, number of threads and processes to run which can handle your expected load, SSL configuration (if you want SSL all the way through, instead of terminating at web server) etc.
Most (all?) WSGI servers will require only the path to the (1) Django’s
settings and (2) WSGI “module”. This wsgi module, is often times just a
wsgi.py file somewhere on PYTHONPATH. If you are using
it comes pre-installed and you can refer to it as
worrying about the file itself.
Since web servers are more hardened in terms of security and more capable of handling high load, _typically_, WSGI servers listen on a local addresses and Web server proxy requests from Internet to the Wsgi server.