wiki:email-deliverability

Version 27 (modified by https://id.mayfirst.org/jaimev, 3 months ago) (diff)

--

Email Deliverability Strategies

See #13114 for additional strategies in progress.

Our goals are:

  • Prioritize sending of individual email over sending of bulk email
  • Reduce the number of times we are blocked
  • Reduce the labor/time it takes to recover from being blocked

Using mail.mayfirst.org

All users should set their email clients (phone and desktop) to use mail.mayfirst.org as the outgoing/smtp server. By focusing on one domain we can better ensure deliverability.

When an IP address gets blocked, see our individual-mail-relay page for instructions on how to change the sending IP.

Use bulk.mayfirst.org

All web sites that send bulk email (e.g. CiviCRM) should be configured to relay through bulk.mayfirst.org (assata) which will relay via our bulk email relay servers.

By using our bulk mail relay servers, we ensure a high volume of legitimate email which offsets bad email.

Reduce impact of forwarded spam

Like most providers, we allow people to forward mail sent to their MF/PL email address to another email account. As a result, spam sent to a user's MF/PL account is also forwarded, which gives the impression that we are spammers.

See #7556 for work on a solution to this problem.

Monitoring

Via Nagios, we currently monitor the following:

spambox deliveries

If our email ends up in the spam box of any of the three major corporate providers, we get an alert.

What do to

See filter-check for more information on how to debug the problem.

mailq

If the mailq goes over 200 messages on any server we get a warning and if it goes over 500 we get an alert.

What to do

Run mailq to review the messages in the queue. Run postcat -q <id> to view the messages in the queue that look suspicious Run mf-mailq-delete <email> to mass delete messages in the mailq that are spam

blocked messages

If we have more than 20 blocked messages that fit a possible pattern of spam we get a Blocklist Status critical alert.

What do to

Run mf-check-blocklist -b to get a human readable report of the blocked messages.

Scan the mail.log to determine who and why we have the problem.

Email relayers

If a single sasl username relays more than 100 messages in a 24 hour period we get an alert.

What do to

Run mf-check-relay-mail-users to see who is being reported. Try to determine if they are sending illegimate email.

Check block lists regularly

See: Use the check_dnsbl Nagios plugin: #5736

Pursue bulk mailer status and apply for feedback loops for our email list servers with major mail providers

See email-deliverability-status for current status of our bulk mail and feedback loop applications.

Followed up in: #6314

Provide tools for release mailq back log

See current process: bulk mail relay.

Thanks to taggart for the idea. We have multiple relay servers and have our bulk mail servers use round robin DNS to randomly pick the relay servers. See: #6662.

We are also rate-limiting outgoing messages when it seems to help.

open deliverability tickets

Open Tickets tagged blocklist or email-deliverability

Ticket Summary Keywords Status Owner Type Priority
#5736 Nagios plugin for checking spam blacklists nagios check_rbl blocklist email-deliverability assigned https://id.mayfirst.org/jamie Feature/Enhancement Request High
#11487 add dkim signing for all outgoing email dkim email-deliverability assigned https://id.mayfirst.org/jamie Feature/Enhancement Request Medium
#13112 automatically block messages that are sending to addresses that don't exist mail spam filter email-deliverability new Bug/Something is broken Medium
#13113 Adding spf records to member domains spf email email-deliverability new Task/To do item Medium
#13114 Improve message delivery email email-deliverability new Task/To do item Medium
#13167 re-think individual mail relay vs bulk mail relay strategy email-deliverability assigned https://id.mayfirst.org/jamie Task/To do item Medium
#13510 forcing tls encryption for all incoming and outgoing connections to postfix email-deliverability, postfix , security assigned https://id.mayfirst.org/jaimev Task/To do item Medium