The incoming runner

This runner’s sole purpose in life is to decide the disposition of the message. It can either be accepted for delivery, rejected (i.e. bounced), held for moderator approval, or discarded.

The runner operates by processing chains on a message/metadata pair in the context of a mailing list. Each mailing list has a default chain for messages posted to the mailing list. This chain is processed with the message eventually ending up in one of the four disposition states described above.

>>> from mailman.app.lifecycle import create_list
>>> mlist = create_list('test@example.com')
>>> print(mlist.posting_chain)
default-posting-chain

Sender addresses

The incoming runner ensures that the sender addresses on the message are registered with the system. This is used for determining nonmember posting privileges. The addresses will not be linked to a user and will be unverified, so if the real user comes along later and claims the address, it will be linked to their user account (and must be verified).

While configurable, the sender addresses by default are those named in the From:, Sender:, and Reply-To: headers, as well as the envelope sender (though we won’t worry about the latter).

>>> from mailman.testing.helpers import (specialized_message_from_string
...   as message_from_string)
>>> msg = message_from_string("""\
... From: zperson@example.com
... Reply-To: yperson@example.com
... Sender: xperson@example.com
... To: test@example.com
... Subject: This is spiced ham
... Message-ID: <bogus>
...
... """)

>>> from zope.component import getUtility
>>> from mailman.interfaces.usermanager import IUserManager
>>> user_manager = getUtility(IUserManager)
>>> print(user_manager.get_address('xperson@example.com'))
None
>>> print(user_manager.get_address('yperson@example.com'))
None
>>> print(user_manager.get_address('zperson@example.com'))
None

Inject the message into the incoming queue, similar to the way the upstream mail server normally would.

>>> from mailman.app.inject import inject_message
>>> filebase = inject_message(mlist, msg)

The incoming runner runs until it is empty.

>>> from mailman.runners.incoming import IncomingRunner
>>> from mailman.testing.helpers import make_testable_runner
>>> incoming = make_testable_runner(IncomingRunner, 'in')
>>> incoming.run()

And now the addresses are known to the system. As mentioned above, they are not linked to a user and are unverified.

>>> for localpart in ('xperson', 'yperson', 'zperson'):
...     email = '{0}@example.com'.format(localpart)
...     address = user_manager.get_address(email)
...     print('{0}; verified? {1}; user? {2}'.format(
...           address.email,
...           ('No' if address.verified_on is None else 'Yes'),
...           user_manager.get_user(email)))
xperson@example.com; verified? No; user? None
yperson@example.com; verified? No; user? None
zperson@example.com; verified? No; user? None

Accepted messages

We have a message that is going to be sent to the mailing list. Once Anne is a member of the mailing list, this message is so perfectly fine for posting that it will be accepted and forward to the pipeline queue.

>>> from mailman.testing.helpers import subscribe
>>> subscribe(mlist, 'Anne')
<Member: Anne Person <aperson@example.com> on test@example.com
         as MemberRole.member>

>>> msg = message_from_string("""\
... From: aperson@example.com
... To: test@example.com
... Subject: My first post
... Message-ID: <first>
...
... First post!
... """)

Inject the message into the incoming queue and run until the queue is empty.

>>> filebase = inject_message(mlist, msg)
>>> incoming.run()

There are no messages left in the incoming queue.

>>> get_queue_messages('in')
[]

Now the message is in the pipeline queue.

>>> messages = get_queue_messages('pipeline')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <first>
Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
X-Message-ID-Hash: 4CMWUN6BHVCMHMDAOSJZ2Q72G5M32MWB
Date: ...
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop;
    banned-address; emergency; member-moderation; nonmember-moderation;
    administrivia; implicit-dest; max-recipients; max-size;
    news-moderation; no-subject; digests; suspicious-header

First post!

>>> from mailman.testing.documentation import dump_msgdata
>>> dump_msgdata(messages[0].msgdata)
_parsemsg    : False
envsender    : noreply@example.com
...

Held messages

The list moderator sets the emergency flag on the mailing list. The built-in chain will now hold all posted messages, so nothing will show up in the pipeline queue.

>>> from mailman.interfaces.chain import ChainEvent
>>> def on_chain(event):
...     if isinstance(event, ChainEvent):
...         print(event)
...         print(event.chain)
...         print('From: {0}\nTo: {1}\nMessage-ID: {2}'.format(
...             event.msg['from'], event.msg['to'],
...             event.msg['message-id']))

>>> mlist.emergency = True

>>> from mailman.testing.helpers import event_subscribers
>>> with event_subscribers(on_chain):
...     filebase = inject_message(mlist, msg)
...     incoming.run()
<mailman.interfaces.chain.HoldEvent ...>
<mailman.chains.hold.HoldChain ...>
From: aperson@example.com
To: test@example.com
Message-ID: <first>

>>> mlist.emergency = False

Discarded messages

Another possibility is that the message would get immediately discarded. The built-in chain does not have such a disposition by default, so let’s craft a new chain and set it as the mailing list’s start chain.

>>> from mailman.chains.base import Chain, Link
>>> from mailman.interfaces.chain import LinkAction
>>> from mailman.config import config
>>> def make_chain(name, target_chain):
...     test_chain = Chain(name, 'Testing {}'.format(target_chain))
...     config.chains[test_chain.name] = test_chain
...     link = Link('truth', LinkAction.jump, target_chain)
...     test_chain.append_link(link)
...     return test_chain

>>> test_chain = make_chain('always-discard', 'discard')
>>> mlist.posting_chain = test_chain.name

>>> msg.replace_header('message-id', '<second>')
>>> with event_subscribers(on_chain):
...     filebase = inject_message(mlist, msg)
...     incoming.run()
<mailman.interfaces.chain.DiscardEvent ...>
<mailman.chains.discard.DiscardChain ...>
From: aperson@example.com
To: test@example.com
Message-ID: <second>

>>> del config.chains[test_chain.name]

Rejected messages

Similar to discarded messages, a message can be rejected, or bounced back to the original sender. Again, the built-in chain doesn’t support this so we’ll just create a new chain that does.

>>> test_chain = make_chain('always-reject', 'reject')
>>> mlist.posting_chain = test_chain.name
>>> msg.replace_header('message-id', '<third>')
>>> with event_subscribers(on_chain):
...     filebase = inject_message(mlist, msg)
...     incoming.run()
<mailman.interfaces.chain.RejectEvent ...>
<mailman.chains.reject.RejectChain ...>
From: aperson@example.com
To: test@example.com
Message-ID: <third>

The rejection message is sitting in the virgin queue waiting to be delivered to the original sender.

>>> messages = get_queue_messages('virgin')
>>> len(messages)
1
>>> print(messages[0].msg.as_string())
Subject: My first post
From: test-owner@example.com
To: aperson@example.com
...

--===============...
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit

[No bounce details are available]
--===============...
Content-Type: message/rfc822
MIME-Version: 1.0

From: aperson@example.com
To: test@example.com
Subject: My first post
Message-ID: <third>
Date: ...

First post!

--===============...
>>> del config.chains['always-reject']