Opened 18 months ago

Last modified 18 months ago

#13113 new Task/To do item

Adding spf records to member domains

Reported by: https://id.mayfirst.org/jaimev Owned by:
Priority: Medium Component: Tech
Keywords: spf email email-deliverability Cc: https://id.mayfirst.org/jamie
Sensitive: no

Description

We'd like to add SPF records to all domains that send mail through MFPL servers.

https://support.mayfirst.org/wiki/faq/email/add-spf-record

The first step will be to ensure that all new hosting orders have an SPF record automatically generated.

The next step will be to automatically add SPF records to all existing hosting orders that are explicitly using our e-mail servers.

Change History (4)

comment:1 Changed 18 months ago by https://id.mayfirst.org/jamie

I've started thinking through the second step of automatically adding SPF records to all existing domains. I realized that many of our groups forward their email to their gmail or other corporate account. Those they take this step presumably send email from their domain via gmail, so we don't want an SPF record for them.

I think we should run the following before creating the SPF record:

  • Generate a distinct list of the dns_zone field for all active records in the database (the dns_zone field is the calculated value of the base domain that takes into account the public suffix list). The value of the dns_zone field will be considered below as $domain
  • If $domain uses a.ns or b.ns as the authoritative name server, continue (use dig to check global name servers)
  • If $domain has an MX record in our database pointing to a domain that ends in .mayfirst.org, continue
  • If $domain does not already have a txt record that starts with 'v=spf', continue
  • If there is no active record in the red_item_email_address table with an email_address that ends in @$domain AND has a corresponding email_address_recipient record that contains an '@', continue

Generate SPF record.

comment:2 Changed 18 months ago by https://id.mayfirst.org/jaimev

I like the above but I have seen that not all members who forward their mail to other providers (like gmail) necessarily use it to send from their own domain, also sometimes within the same hosting order mail is forwarded for some addresses and other logins of the same hosting order use MFPL mail servers directly.

It is not clear to me if the presence of an SPF record authorizing MFPL mail servers to send mail on behalf of the domain necessarily harms an e-mails score if it comes from another server. They will just need to add an spf record for any other servers (like gmail) they intend to send from.

comment:3 Changed 18 months ago by https://id.mayfirst.org/jamie

I think how an incorrect SPF record counts against you varies from provider to provider. However, it is supposed to count against you. The whole point of SPF is to tell the world: "These, and only these servers should ever send email from my address."

Out of curiosity I looked at how our servers treat email send with a valid and invalid SPF record and found this:

# SPF
# Note that the benefit for a valid SPF record is deliberately minimal; it's
# likely that more spammers would quickly move to setting valid SPF records
# otherwise.  The penalties for an *incorrect* record, however, are large.  ;)
ifplugin Mail::SpamAssassin::Plugin::SPF
score SPF_NONE 0
score SPF_HELO_NONE 0
score SPF_PASS -0.001
score SPF_HELO_PASS -0.001
# <gen:mutable>
score SPF_FAIL 0 0.919 0 0.001 # n=0 n=2
score SPF_HELO_FAIL 0 0.001 0 0.001 # n=0 n=2
score SPF_HELO_NEUTRAL 0 0.001 0 0.112 # n=0 n=2
score SPF_HELO_SOFTFAIL 0 0.896 0 0.732 # n=0 n=2
score SPF_NEUTRAL 0 0.652 0 0.779 # n=0 n=2
score SPF_SOFTFAIL 0 0.972 0 0.665 # n=0 n=2

Getting a correct SPF score removes .001 from your spam score. No SPF record does nothing. But getting an incorrect record can add from half a point to a nearly a full point.

I'd prefer to error on the side of not adding SPF records rather than adding incorrect ones for this reason.

comment:4 Changed 18 months ago by https://id.mayfirst.org/jamie

  • Keywords spf email email-deliverability added; SPF mail removed

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.