wiki:email-deliverability

Version 18 (modified by Jamie McClelland, 6 years ago) (diff)

--

Email Deliverability Strategies

This week we have been hit hard by email deliverability problems. I think it's time to make some signficant changes to our email strategies. The goals of these changes are:

  • Prioritize sending of individual email over sending of bulk email
  • Reduce the number of times we are blocked
  • Reduce the labor/time it takes to recover from being blocked

Using mail.mayfirst.org

Currently we allow people to relay mail via either their primary server or mail.mayfirst.org. We should alert people of a cut-over time and then dis-allow mail relaying via the primary server.

We should also permanently assign at least three IP addresses to our two mail.mayfirst.org servers and all bulk mail servers and provide a simple script for swapping the smtp bind IP address. See also #3237

Reduce impact of forwarded spam

Like most providers, we allow people to forward mail sent to their MF/PL email address to another email account. As a result, spam sent to a user's MF/PL account is also forwarded, which gives the impression that we are spammers.

Although I've been opposed to any form of email censorship, I think we should make an exception in this case and filter out messages that are tagged by Spamassassin as spam and send these messages to /dev/null. We can make this policy clear in the members control panel whenever someone chooses to forward email sent to them. And we should be clear that if they want to be sure to receive all mail sent to them, they must deliver it to an MF/PL user account. See #7556.

Discourage web site mail relaying via primary host

Any server in our network can relay mail via bulk.mayfirst.org, however, many web sites still send email via localhost. We should implement a policy on primary hosts that limit the number of messages our primary hosts will send to 50 per user per hour.

Improved monitoring

Ross' has recently completed a nagios alert to warn us when the mailq for any server goes over 100.

We should add:

  • Monitor for known mail.log strings that indicate blocking (I have a list already): #831
  • Checking for signs that a web site is compromised (not sure how to do this): not assigned a ticket
  • Check for signs on mail.mayfirst.org that a relay mail account has been compromised (e.g. certain number of messages sent per hour): not assigned a ticket
  • Use the check_dnsbl Nagios plugin: #5736

Pursue bulk mailer status and apply for feedback loops for our email list servers with major mail providers

Many providers allow us to apply for bulk mailer status. We should pursue this for assata.mayfirst.org and leslie.mayfirst.org, our two mailing list servers.

See email-deliverability-status for current status of our bulk mail and feedback loop applications.

Followed up in: #6314

Provide tools for release mailq back log

Although this runs the risk of getting more IP addresses blocked, it would nonetheless be useful if we had easy method for swapping out IP addresses easily to relieve a big mailq backlog. That could happen either by having multiple IP addresses assigned to each bulk mail server that could be swapped around with a command line tool and/or by temporarily relaying mail to a server in another cabinet (e.g. chun or femi or fela).

You can now rotate ip addresses on leslie and assata with the command mf-swap-smtp-bind-address.

Another idea suggested by taggart is to setup multiple relay servers and have our bulk mail servers use round robin DNS to randomly pick the relay servers. We could also setup a "graveyard" mx - which is the relay server that should be sent all deliveries that were deferred on first attempt. That way we keep the queues empty for well-responding destination servers and shunt the problem ones aside. Using transport mapping, we can also send email destined to a particular domain name to a particular server that is not having trouble relaying the mail. See: #6662.

And a yet another idea is to use postfwd to rate limit messages based on criteria (like yahoo messages can only be sent x number at a time).

open deliverability tickets

Open Tickets tagged blocklist or email-deliverability

Ticket Summary Keywords Status Owner Type Priority