wiki:bulk-mail-relay

Version 5 (modified by Jamie McClelland, 5 years ago) (diff)

--

Bulk Mail Relay

See #8424 for implementation details.

See the email deliverability page for context.

Big Picture

The over all concept is: most corporate providers allow mail through based on the ratio of delivered mail without complaints to bounced mail and complaints (complaints means people who click the "this is spam" link).

So, our strategy is to use dedicated servers to send email to specific big providers, so we increase the volume of legit mail to help tip the balance when a member (inevitably) sends a bunch of bad addresses or has users click the "this is spam" button.

Details

The following email list servers are currently using this system: assata.mayfirst.org (provides both bulk.mayfirst.org and lists.people-link.net) and leslie.mayfirst.org (provides lists.mayfirst.org and lists.critpath.org). This strategy should also be adopted by morales.mayfirst.org (provides lists.portside.org), mx3.mayfirst.org and mx25.mayfirst.org.

We currently have two servers setup as dedicated mail relay servers (cleveland in XO and rustin in Telehouse). These servers run multi-postfix instances (see #6662 for more information on the setup). That means, each server has multiple instances of postfix running on different IP addresses. NOTE: only rustin is currently in play.

Each of our main email list servers can relay their email to any one of these instances on a domain-by-domain basis by putting a line in the email list server's /etc/postfix/transport.relay file that indicates the domain of the destination email address followed by the domain of the relay server that should send the email.

We have setup special domain names for each of the main mail providers:

  • msn.smtp.mayfirst.org
  • earthlink.smtp.mayfirst.org
  • therest.smtp.mayfirst.org

Future plans include adding:

  • yahoo.smtp.mayfirst.org
  • aol.smtp.mayfirst.org
  • graveyard.smtp.mayfirst.org (see below)

Each of our email list servers is configured to relay mail by domain to the appropriate provider, with all deferred email going to the graveyard. Similarly, deferred email on each of the relay instances also goes to the graveyard. This avoids deferred email build up, which than exerbates the problems on our good IP addresses.

Each domain name will have a 600 second time-to-live and will point to one or more of the IP addresses assigned to the relay servers.

We started by assigning each of these domains to IP addresses that are "warmed up" and currently able to send lots of email.

Over time, we can warm up additional IP addresses by using round robin DNS. In other words, we assign a domain to a single, good IP address 4 times, and to a new IP address once. That means only 20% of the traffic goes to the new IP address so it can build it's reputation slowly over time. We then change the ratio of DNS assignments until it can handle more email.

How Do I…

Add a new IP?

  • Create a Host record that ties the IP address to the host name (e.g. rustin.mayfirst.org or cleveland.mayfirst.org). Set the time to live to 86400 (it's important to have along ttl since some providers do a reverse DNS lookup of the sending IP address and will punish you if the ttl is too short).
  • Create an A record from the $instance.smtp.mayfirst.org to the IP address.
  • Add the IP address to the servers /etc/network/interfaces file and bring it up

Enable a new server to relay through this system?

  • Add the IP address to the puppet file manifests/global.pp, to the variable $mfpl_relaying_ips.
  • Be sure to push to both cleveland and rustin
  • Add the following to the new server's /etc/postfix/main.cf file:
    transport_maps = hash:/etc/postfix/transport.relay
    

If main.cf already has a line for transport_maps, you can add a new value following a comma, e.g.:

transport_maps = hash:/etc/postfix/transport,hash:/etc/postfix/transport.relay

Copy the file /etc/postfix/transport.relay from leslie or assata.