Opened 4 months ago

Last modified 2 months ago

#14184 assigned Bug/Something is broken

Improve email deliverability and control for listserv (

Reported by: Owned by:
Priority: Medium Component: Tech
Keywords: Cc:
Sensitive: no

Description (last modified by


I'm hoping to just get confirmation here before committing to any action, so as to not break any of the things :D Part of the issue is that I'm not quite sure how to determine the exact sources of email. i.e. how can I be certain that all outbound mail goes through the mayfirst MTA's. See report at end from Certainly some of these are anticipated to be spoofed.

There are actually a number of tasks to complete in regards to this.

  1. SPF
  2. DKIM
  3. DMARC
  4. Reverse routing of email back to ListServ from Postfix (for automatic cleaning of mail lists via return route). Moved to #14277

#1 SPF: ListServ itself is using localhost to deliver mail. My understanding is that postfix is then forwarding all email (by destination) to the various outbound email MFPL servers. In this case, I presume that I want the typical SPF record, only applied at the subdomain level, rather than at the top-level, per e.g.: TXT v=spf1 ~all Now, there is one hesitation I have here, and perhaps just adding is all that is needed. I'm not sure how these particular messages would be being sourced, but according to a domain report from, mumia is a source of email from the domain. For itself, only the mayfirst outbound email servers are referenced, so perhaps a wildcard SPF would be suitable instead?

#2 DKIM: The best documentation I have found here is from ListServ. Being an email dunce, I am not sure however if the DKIM should be applied at the level of ListServ, or postfix. My gut would be that if the only originator for outbound email is the listserv itself, then it makes sense to put it there. This instance has mailman as well however, so...?

#3 DMARC: So far so good here, in that we've set up two emails for receiving reports and can adjust once SPF/DKIM are working properly. Let us know if DMARC reports will be useful to complete this task.

GitLab tickets: (gmail) (gmail) (yahoo deliverability) "sending IPs" for

IP Address    	Hostname 	Volume    	Sender Score 	Very High 	81 	Very High 	99 	Very High 	98 	Very High 	97 	Very High 	99 	Very High 	83 	Very High 	82 	Very High 	96 	Very High 	97 	Very High 	89 	Very High 	96 	Very High 	97 	Very High 	97 	Very High 	96 	Very High 	97 	High 	99 	Very High 	82 	Very High 	82 	Very High 	95 	Very High 	85 	Very High 	97 	Very High 	99 	Moderate 	99 	Very High 	97 	Very High 	44 	Very High 	43 	Very High 	20 	Very High 	20 	Very High 	54 	Very High 	55 	Very High 	51 	Very High 	46 	Very High 	86 	Very High 	83 	Very High 	91 	Very High 	81 	Very High 	86 	Very High 	94 	Very High 	94 	Very High 	86 	Very High 	85 	Very High 	84 	Very High 	85 	Very High 	88 	Very High 	82 	Very High 	82 	Very High 	81 	Very High 	81

Change History (15)

comment:1 Changed 4 months ago by

  • Owner set to
  • Status changed from new to assigned

I have questions about this as well. Copying jamie here but adding just a few details I think I can confirm.

As far as the spf record if you want to use a more limited range you could try using just:

I think you probably want to avoid using directly. We can check the logs and try to identify which software is still delivering mail from there. If you do need to send mail from mumia for any reason then you should also add it to the spf record.

Last edited 4 months ago by (previous) (diff)

comment:2 Changed 4 months ago by

Excellent, ty. For the record, I'd checked Senderscore for, and it is significantly cleaner - and in fact, exactly matches your limited recommendation. Personally, I'm more concerned with cutting off legitimate senders than excluding potentially offending sources - at least for the first go at this. I'm just not sure how to go about it, but thankfully it sounds like you have an idea on where to look for that :)

Sending IPs
IP Address    	Hostname 	Volume    	Sender Score 	Very High 	96 	Very High 	97 	Very High 	89 	Very High 	96 	Very High 	97 	Very High 	97 	Very High 	96 	Very High 	97 	Very High 	97

comment:3 Changed 4 months ago by

Just to note: perhaps my biggest lack of understanding here is "where" (when?) SPF/DKIM, etc. should apply. For example, does it all happen at/apply to the "border" where some MTA finally talks to the target servers (per the MX of the recipient domain's records)? Or, does, e.g. SPF, look "backwards" from that, such that where the message originated might have any bearing? Judging from the recommendations of MayFirst re using the include for, I'm guessing that the border is the appropriate place - at least for SPF. If DKIM applies earlier, would that mean that there might be multiple keys on record, one for each sending program (i.e. mailman + listserv). No need to respond to this directly - just trying to explain my own perspective so you know what I do/don't understand as we proceed here - I'll continue to read up, there's no shortage of info on the huge topic, I just have yet to have dug into a good overarching guide :)

comment:4 Changed 4 months ago by

Yeah, it is a mess of concepts to learn.

it's further complicated by the fact that not all mail servers properly follow the spec :).

Typically, receiving servers will evaluate messages when they are received.

When one server sends email to a second server, the sending servers supplies "envelope" information - including a "MAIL FROM" and a "RCPT TO" value. These values can be totally different from the To: and From: values in the message header. (The envelope RCPT TO will determine what mail box it lands it and the MAIL FROM often is reflected in the Return Path header that is usually hidden in most email clients).

With SPF, it is supposed to apply to the MAIL FROM envelope information. However, in my experience I've found that servers apply SPF rules to the from address in the headers as well.

DKIM, on the other hand, is stricly about the from header. The sending server is supposed to sign the entire message using the private key associated with the domain in the From: header. Then the receiving server should retrieve the public key via dns and validate the signature.


comment:5 Changed 3 months ago by

OK - so, I think we are ready to start rolling out some changes. Things to keep in mind with these suggestions:

  • We are not positive where all email originates.
  • The largest concern is reducing deliverability (obviously), but given this is for a very important asset to Portside, I'm just trying to be extra cautious.
  • The email we are working on at this time is from the subdomain that listserv originates messages from.
  • We are relying heavily on your verification/recommendations here - I'm not an expert on these records by any means.

We are thinking of taking the following next step: Enable DKIM for, and adjust DMARC accordingly:

  • Add domain key: (create with test flag - t=y)
  • Configure L-Soft software to use this key.
  • Update DMARK to the following (we'd like it to ignore SPF failure IF it is otherwise valid with DKIM) Note: DMARK record currently @ which I understand is inhereted by

Proposed DMARC record for: (published at

v=DMARC1; p=none;;; fo=0; adkim=r; aspf=r; pct=1; ri=86400; sp=none

A few things about above: the "percent" flag is 1%. Also, using the following message was also returned, but may be a tool error?

Organization domain and reporting address domains are different. The report receiving domain must also publish a verification record.

Verification record(s) for:
Record location:
Record: v=DMARC1;

Once we have DMARC reports returned after these have rolled out, we plan to move on to adding an SPF record.

comment:6 Changed 3 months ago by

This seems reasonable, especially given that your action is to not take action (I find dmarc overly sensitive and easy to mess up and result in less deliverability).

One point of clarification: are you adding this to the domain name (recommended) or the domain name?

Also, I haven't looked... but do you know how to configure listserv to use dkim? If not we could probably configure postfix on morales to do it, but if you can get listserv to do it that would be ideal.

comment:7 Changed 3 months ago by

Per domains - I've confirmed it as written above, yes: the DKIM record, to the The DMARC record is currently at and seems to be being inherited OK...should we move this to

ListServ has great instructions for adding the key to itself, so yes, I'll go with putting it in there.

comment:8 Changed 3 months ago by

I recommend initially restricting everything to if possible. In my experience:

  • DKIM: these only affect the messages being signed. As long as the public key is properly published, you can't go wrong and in the worse case scenario you will mess up just the emails that are being signed. Remember: DKIM is about the From: address in the email headers.
  • SPF: An incorrect record is worse then no record at all. If you publish an SPF record and you are sending from an IP that is not included, you will be punished. No so great.
  • DMARC: as long as you are not enforcing and just getting reports, you should be safe. But in my limited experience, I have found this one to be quite tricky to get right and once you switch from reporting to enforcement, it can be very painful to get it wrong.

The only possible problem is if the from address on the messages going out is a address instead of a address. If so, your dkim record will need to be on instead of And your SPF record, if you follow the spec, should not matter but in fact I find a lot of email providers seem to check the From: address against SPF anyway, so it might need to as well.

comment:9 Changed 3 months ago by

So, in summary: what I have now appears incorrect. Fortunately, I set up the DKIM key under test configuration and also told ListServ to sign for, so it should in fact be signing nothing if I'm understanding correctly (should be for given "moderator@…", yes?).

Here's the snapshot headers, for reference:

Return-Path: <owner-portside-snapshot*chris**AGARIC*-COM@LISTS.PORTSIDE.ORG>
Received: from
	by (Dovecot) with LMTP id rFuFKt3I/1tJcQAA+yvYHA
	for <>; Thu, 29 Nov 2018 06:09:17 -0500
Received: from (localhost [])
	by (Postfix) with ESMTP id 0EB58139C4
	for <chris@AGARIC.COM>; Thu, 29 Nov 2018 06:09:17 -0500 (EST)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,SHORTCIRCUIT
	autolearn=disabled version=3.4.2
X-Spam-Language: en
Received: from ( [])
	(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by (Postfix) with ESMTPS id DA09546B67
	for <chris@AGARIC.COM>; Thu, 29 Nov 2018 06:09:16 -0500 (EST)
Received: from ( [])
	by (Postfix) with ESMTP id 4A664BDB0
	for <chris@AGARIC.COM>; Thu, 29 Nov 2018 06:09:15 -0500 (EST)
Received: from morales (localhost [])
	by (Postfix) with ESMTP id A5DAA254CB
	for <chris@AGARIC.COM>; Thu, 29 Nov 2018 01:09:14 -1000 (HST)
Received: by LISTS.PORTSIDE.ORG (LISTSERV-TCP/IP release 16.5) with spool id
          28199941 for PORTSIDE-SNAPSHOT@LISTS.PORTSIDE.ORG; Thu, 29 Nov 2018
          01:00:16 -1000
Received: from ( []) by
 (Postfix) with ESMTP id 0B587254A9 for
          <PORTSIDE-SNAPSHOT@LISTS.PORTSIDE.ORG>; Thu, 29 Nov 2018 01:00:16
          -1000 (HST)
Received: from (localhost []) by
          (Postfix) with ESMTP id BABC5BE2F; Thu, 29 Nov 2018 06:00:09 -0500
Received: by (Postfix, from userid 27371) id 5D82BBE32; Thu,
          29 Nov 2018 06:00:09 -0500 (EST)
X-PHP-Originating-Script: 27371:SimpleMailInvoker.php
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="_=_swift_v4_1543489209_3743147083f54b1c56415ce5eb9b5d4e_=_"
X-Mailer: Drupal
X-Virus-Scanned: ClamAV using ClamSMTP
X-Envelope-From: <>
Message-ID:  <6df789c12ea4b113315af5771f15584a@swift.generated>
Date:         Thu, 29 Nov 2018 06:00:09 -0500
Reply-To:     moderator@PORTSIDE.ORG
From:         Portside Snapshot <moderator@PORTSIDE.ORG>
Subject: Portside Snapshot -  November 29, 2018
Precedence: list
List-Help: <>,
List-Unsubscribe: <mailto:PORTSIDE-SNAPSHOT-unsubscribe-request@LISTS.PORTSIDE.ORG>
List-Subscribe: <mailto:PORTSIDE-SNAPSHOT-subscribe-request@LISTS.PORTSIDE.ORG>
List-Archive: <>
X-Virus-Scanned: ClamAV using ClamSMTP

comment:10 Changed 3 months ago by

ListServ now signs for

active 	2018-11-29 11:58:04 	text 		600 		t=y; g=; k=rsa; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC/M96fb2xxDSOEwhLBtTD+RAvxqwl7HcDYkIrIQxyXSPtruyFK0vyODR2LLgS1BcJ8QNdWmaGsjQ20U9KF84aa5WWT3VqUUx7xEdx10zPvgmXm7X+UMpyo2SNv/prdIiYrorm78zcDT2xhHg0EEQwNkc7oDDMn5YVcUgkt5tYSxQIDAQAB 	0 	0 	0 	
active 	2018-11-29 10:17:19 	text 		3600 		v=DMARC1; p=none;;; fo=1; adkim=r; aspf=r; pct=1; ri=86400; sp=none

comment:11 Changed 3 months ago by

OK then, perhaps BOTH and are needed?

Return-Path: <>
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on
X-Spam-Pyzor: Reported 0 times.
X-Spam-Status: No, score=0.2 required=6.0 tests=AM_TRUNCATED,CK_419SIZE,
	SPF_PASS,T_FILL_THIS_FORM_SHORT shortcircuit=no autolearn=disabled
Received: from ( [])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "*", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK))
	by (Postfix) with ESMTPS id 724DD133
	for <>; Thu, 29 Nov 2018 09:50:10 -0800 (PST)
Received: from ( [])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client did not present a certificate)
	by (Postfix) with ESMTPS id 259F11A23EF
	for <>; Thu, 29 Nov 2018 09:50:10 -0800 (PST)
Authentication-Results:; dkim=pass
	reason="1024-bit key; unprotected key"
	header.b=TUgigysj; dkim-adsp=none (unprotected policy);
Received: from ( [])
	by (Postfix) with ESMTP id 547AB140670
	for <chris@WOLCEN.COM>; Thu, 29 Nov 2018 12:50:06 -0500 (EST)
DKIM-Filter: OpenDKIM Filter v2.11.0 547AB140670
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=default; t=1543513809;
Received: from ([])
	by with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256)
	(Exim 4.89)
	(envelope-from <owner-PORTSIDE@LISTS.PORTSIDE.ORG>)
	id 1gSQRr-0003Vg-II
	for chris@WOLCEN.COM; Thu, 29 Nov 2018 09:50:05 -0800
Received: from ( [])
	by (Postfix) with ESMTP id 1B7EF61A3;
	Thu, 29 Nov 2018 12:49:58 -0500 (EST)
Received: from morales (localhost [])
	by (Postfix) with ESMTP id C5359254AC;
	Thu, 29 Nov 2018 07:49:57 -1000 (HST)
Date:         Thu, 29 Nov 2018 07:49:57 -1000
Subject: Welcome to PORTSIDE
To:           Chris Thompson <chris@WOLCEN.COM>
cc:           portside-owner@PORTSIDE.ORG
Message-ID:   <LISTSERV%201811290749577650.A92C@LISTS.PORTSIDE.ORG>
List-Help:    <>,
List-Unsubscribe: <mailto:PORTSIDE-unsubscribe-request@LISTS.PORTSIDE.ORG>
List-Subscribe: <mailto:PORTSIDE-subscribe-request@LISTS.PORTSIDE.ORG>
List-Owner:   <mailto:PORTSIDE-request@LISTS.PORTSIDE.ORG>
List-Archive: <>
X-Filter-Label: newsletter
X-SpamExperts-Class: ham
X-SpamExperts-Evidence: SB/registrar-servers_com (0.000257335705314)
X-Recommended-Action: accept
X-Filter-ID: PqwsvolAWURa0gwxuN3S5aX1D1WTqZz4ZUVZsEKIAZljyiqQxAAa/b4G9zgWT+BUVVs5XW3jz1DS

comment:12 Changed 3 months ago by

Ah yes - it does look like listserv is going to send some email messages from and some are going to come from and to be complete, both should be dkim signed.

comment:13 Changed 3 months ago by

  • Description modified (diff)

comment:14 Changed 2 months ago by

Re No. 1 (SPF): Mention was made of checking logs to see where other messages may be originating. Has this been looked into, or is there something I should do? (journalctl -u postfix on mumia?) Not sure what to look for here. SPF passing (as per below) is for the messages that have been seen coming out of against MFPL SPF, not portside's own records, which don't exist yet.

Re No. 2 (DKIM) Current status: DKIM appears to be functioning properly for email deliveries from ListServ. Google is reporting DMARC and DKIM passing, and varies on SPF (sometimes NEUTRAL, other times PASS).

The DKIM record I first used was incorrect, but correct as of later Sunday. (The g= was changed to g=*, and the DKIM version header had been missing.. not caught originally because some services apparantly treated it improperly).

At present, the DKIM record still indicates t=y (test mode) and thus should not invalidate any improperly signed nor unsigned messages. We'd like to move to removing test, however:

What needs to be done for, e.g. roundcube or postfix outside of the listserv in order to turn this on? If I understand correctly, messages NOT sent by listserv will have no signature, so, for instance, deliverability (depending on other potential DMARC changes) of messages without a signature presumably may decline.

comment:15 Changed 2 months ago by

Congrats on all the progress!

Unfortunately, we have no facility right now to DKIM-sign messages via roundcube or our other outgoing email servers, so at this point, enabling DMARC on is not going to be a good idea (just DKIM-signing should give you a significant boost though).

If there is a way to send listserve email using a from address that uses the domain (instead of, then you could add a DMARC record to just the domain and it should work out since listserv would presumably be the only source of email with that domain.

I'm not sure we have any logs that would be helpful in finding other sources that are sending portside email.

I suspect there aren't any. But a classic example would be: someone with a portside email address has a home verizon email account. Their email program is configured to relay their verizon email through the verizon email servers. Then, they add their portside account and it also is configured to relay through verizon.

This could cause a SPF failure.

This kind of setup seems rare these days and would only affect a single user, who could then change to relay their email via may first.

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.