Archivers

Mailman supports pluggable archivers, and it comes with several default archivers.

>>> from mailman.app.lifecycle import create_list
>>> mlist = create_list('test@example.com')
>>> from mailman.testing.helpers import (specialized_message_from_string
...   as message_from_string)
>>> msg = message_from_string("""\
... From: aperson@example.org
... To: test@example.com
... Subject: An archived message
... Message-ID: <12345>
... X-Message-ID-Hash: RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
...
... Here is an archived message.
... """)

Archivers support an interface which provides the RFC 2369 List-Archive: header, and one that provides a permalink to the specific message object in the archive. This latter is appropriate for the message footer or for the RFC 5064 Archived-At: header.

If the archiver is not network-accessible, it will return None and the headers will not be added.

Mailman defines a draft spec for how list servers and archivers can interoperate.

>>> archivers = {}
>>> from operator import attrgetter
>>> from mailman.config import config
>>> for archiver in sorted(config.archivers, key=attrgetter('name')):
...     print(archiver.name)
...     print('   ', archiver.list_url(mlist))
...     print('   ', archiver.permalink(mlist, msg))
...     archivers[archiver.name] = archiver
mail-archive
    http://go.mail-archive.dev/test%40example.com
    http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
mhonarc
    http://example.com/.../test@example.com
    http://example.com/.../RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
prototype
    None
    None

Sending the message to the archiver

The prototype archiver archives messages to a maildir.

>>> import os
>>> archivers['prototype'].archive_message(mlist, msg)
>>> archive_path = os.path.join(
...     config.ARCHIVE_DIR, 'prototype', mlist.fqdn_listname, 'new')
>>> len(os.listdir(archive_path))
1

The Mail-Archive.com

The Mail Archive is a public archiver that can be used to archive message for free. Mailman comes with a plugin for this archiver; by enabling it messages to public lists will get sent there automatically.

>>> archiver = archivers['mail-archive']
>>> print(archiver.list_url(mlist))
http://go.mail-archive.dev/test%40example.com
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE

To archive the message, the archiver actually mails the message to a special address at The Mail Archive. The message gets no header or footer decoration.

>>> from mailman.interfaces.archiver import ArchivePolicy
>>> mlist.archive_policy = ArchivePolicy.public
>>> archiver.archive_message(mlist, msg)

>>> from mailman.runners.outgoing import OutgoingRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> outgoing = make_testable_runner(OutgoingRunner, 'out')
>>> outgoing.run()

>>> from operator import itemgetter
>>> messages = list(smtpd.messages)
>>> len(messages)
1

>>> print(messages[0].as_string())
From: aperson@example.org
To: test@example.com
Subject: An archived message
Message-ID: <12345>
X-Message-ID-Hash: RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE
X-Peer: ...
X-MailFrom: test-bounces@example.com
X-RcptTo: archive@mail-archive.dev

Here is an archived message.

>>> smtpd.clear()

However, if the mailing list is not public, the message will never be archived at this service.

>>> mlist.archive_policy = ArchivePolicy.private
>>> print(archiver.list_url(mlist))
None
>>> print(archiver.permalink(mlist, msg))
None
>>> archiver.archive_message(mlist, msg)
>>> list(smtpd.messages)
[]

Additionally, this archiver can handle malformed Message-IDs.

>>> from mailman.utilities.email import add_message_hash
>>> mlist.archive_policy = ArchivePolicy.public
>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> msg['Message-ID'] = '12345>'
>>> add_message_hash(msg)
'YJIGBYRWZFG5LZEBQ7NR25B5HBR2BVD6'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/YJIGBYRWZFG5LZEBQ7NR25B5HBR2BVD6

>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> msg['Message-ID'] = '<12345'
>>> add_message_hash(msg)
'XUFFJNJ2P2WC4NDPQRZFDJMV24POP64B'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/XUFFJNJ2P2WC4NDPQRZFDJMV24POP64B

>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> msg['Message-ID'] = '12345'
>>> add_message_hash(msg)
'RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE

>>> del msg['message-id']
>>> del msg['x-message-id-hash']
>>> add_message_hash(msg)
>>> msg['Message-ID'] = '    12345    '
>>> add_message_hash(msg)
'RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE'
>>> print(archiver.permalink(mlist, msg))
http://go.mail-archive.dev/RSZCG7IGPHFIRW3EMTVMMDNJMNCVCOLE

MHonArc

A MHonArc archiver is also available.

>>> archiver = archivers['mhonarc']
>>> print(archiver.name)
mhonarc

Messages sent to a local MHonArc instance are added to its archive via a subprocess call.

>>> from mailman.testing.helpers import LogFileMark
>>> mark = LogFileMark('mailman.archiver')
>>> archiver.archive_message(mlist, msg)
>>> print('LOG:', mark.readline())
LOG: ... /usr/bin/mhonarc
     -add
     -dbfile .../test@example.com.mbox/mhonarc.db
     -outdir .../mhonarc/test@example.com
     -stderr .../logs/mhonarc
     -stdout .../logs/mhonarc -spammode -umask 022