Opened 6 years ago

Closed 5 years ago

#7045 closed Bug/Something is broken (fixed)

posts not being dlivered to email subswcribers - again

Reported by: https://id.mayfirst.org/portside-submission Owned by: https://id.mayfirst.org/ross
Priority: Urgent Component: Tech
Keywords: listserv aol Cc: bxcohen@…, jschaffner@…, jschaffner14@…, mallory@…
Sensitive: no

Description

Last night again their was a problem with posts not going to email subscribers. Of the five posts sent out, our sample of moderators resulting in the following: Comcast (westcoast) - recd 3 of 5 Comcast (eastcoast) - recd 2 of 5 Gmail (east & midwest) - recd 2 of 5 MSN (east coast) - recd 2 of 5 Earthlink (eastcoast)-recd 2 of 5

This morning, it looks like there was no problem with Portside Labor

can you check delivery logs - was there a glitch in the mail going out from Listserv

Was there any indication of a problem with Portside Labor this morning?

Thanks.

Jay

Change History (15)

comment:1 Changed 6 years ago by https://id.mayfirst.org/dkg

The "again" in the summary seems to refer to an earlier ticket, but i'm not sure which one. perhaps #6712 ? If i'm reading the logs right, #6712 looks like it was caused by google's spam-filtering graylisting service, and it appears to have been resolved by waiting a little while for the graylisting timeouts to kick in.

Do you think this is part of a different pattern? maybe someone else will have a chance to review the delivery logs.

comment:2 follow-up: Changed 6 years ago by https://id.mayfirst.org/portside-submission

Dana,

Thanks for quick response. The "again" refers to problems with the May First IP that is used for Listserv being blocked one month ago. (Ticket #6924)

If this is google spam-filtering, understand how it would impact gmail readers; but not sure how this would also impact Comcast, MSN and Earthlink subscribers.

Jay

comment:3 Changed 6 years ago by https://id.mayfirst.org/portside-submission

not sure if this is related. In the following message the post was sent as it should be, however the sidebar and bottom footer appeared as code.

http://portside.org/2013-04-01/stacked-deck-how-dominance-politics-affluent-business-undermines-economic-mobility

Jay

comment:4 Changed 6 years ago by https://id.mayfirst.org/portside-submission

Yesterday's posts for Portside (5) and Portside Labor (1) went through without any problems.

comment:5 in reply to: ↑ 2 Changed 6 years ago by https://id.mayfirst.org/ross

Replying to https://id.mayfirst.org/portside-submission:

Thanks for quick response. The "again" refers to problems with the May First IP that is used for Listserv being blocked one month ago. (Ticket #6924)

This problem was not with an ip address being blocked Jay. It was a problem with a disk partition filling up. I do not see that same issue right now with the server and am trying to track down the issue.

~/ross

comment:6 Changed 6 years ago by https://id.mayfirst.org/portside-submission

Ross,

Something has to be done to prevent this from happening. Last month the partition filled up, now exactly one month later - same or nearly same problem.

Jay

comment:7 Changed 6 years ago by https://id.mayfirst.org/ross

As I said Jay, the partition did not fill up. This is not the same problem at all. It looks like aol.com has blocked almost all traffic from morales. I have filed a whitelist request and a feedback loop request with aol.

The whitelist request is confirmation number 1364948713-022472 . I have not received an email confirmation yet, so we'll have to wait on that to come through. I'm not sure what we can do in the mean time other than wait for aol to respond.

I have not been able to determine why the other domains might be failing. I did find a few rejected email addresses to gmail.org, these were due to a connection time out.

There remain about 9777 emails in the mailq most of which seem to be to aol. Hopefully we can get whitelisted soon and fix this problem.

~/ross

comment:8 Changed 6 years ago by https://id.mayfirst.org/ross

feed back loop confirmation number 1364951297-262831

comment:9 Changed 6 years ago by https://id.mayfirst.org/portside-submission

Ross,

Thanks. Just sent the following email to the other Portside moderators:

Have been going back-and-forth with May First as to what happened on Sunday.

It seems that the Portside partition on the May First server (Morales) was being flooded with emails that were not going out.

May First has now tracked the problem, and it seems that AOL was/is blocking emails originating from the Morales server.

Here is what May First is doing to resolve the problem. (In addition to what is below, they are also monitoring all of the partitions on Morales to make sure that there is no problem of their filling up.)

Jay

It looks like aol.com has blocked almost all traffic from morales. I have filed a whitelist request and a feedback loop request with aol.

The whitelist request is confirmation number 1364948713-022472 . I have not received an email confirmation yet, so we'll have to wait on that to come through. I'm not sure what we can do in the mean time other than wait for aol to respond.

I have not been able to determine why the other domains might be failing. I did find a few rejected email addresses to gmail.org, these were due to a connection time out.

There remain about 9777 emails in the mailq most of which seem to be to aol. Hopefully we can get whitelisted soon and fix this problem.

comment:10 Changed 6 years ago by https://id.mayfirst.org/ross

Unfortunately, aol rejected my request for whitelisting. I'm trying to figure out how to meet their ownership approval process.

This was their message text:

Your Whitelist request, with the confirmation code 1364951297-262831, 
has been denied.

Your web request has been denied because you do not appear to be the owner
of the IP address(es) in your request.  If you are in fact the owner, one of the 
following criteria must be met before resubmitting your request:

* The RDNS for each IP shares the FBL email domain in common.
    Valid Example: 
    FBL email address is aolfbl@accounting.aol.com
    192.168.1.1 resolves to mailserver1.accounting.aol.com
    192.168.1.2 resolves to mailserver2.accounting.aol.com

*The DOMAIN WHOIS for each IP's RDNS shares the FBL email domain in common. The domain may appear in any of the listed email addresses.

* At least one authoritative nameserver for each IP shares the FBL
  email domain in common.
    Valid Example: 
    FBL email address is aolfbl@accounting.aol.com
    192.168.1.1 authoritative nameserver is ns1.accounting.aol.com
    192.168.1.2 authoritative nameserver is ns1.accounting.aol.com

* The IP WHOIS information for each IP shares the FBL email domain
  in common.  The domain may appear in any of the listed email addresses.
    Valid Example: 
    FBL email address is aolfbl@abuse.aol.com
    192.168.1.1 and 192.168.1.2 IP WHOIS:

    OrgName:    Your Company Name 
    ...
    NetRange:   192.168.1.0 - 192.168.1.255 
    CIDR:       192.168.1.0/24
    NetName:    COMPANY-1

    OrgAbuseHandle: xxxx
    OrgAbuseName:   abuse 
    OrgAbusePhone:  xxxx
    OrgAbuseEmail:  abuse@abuse.aol.com
    ...

* The ASN WHOIS information for each IP shares the FBL email domain
  in common.  The domain may appear in any of the listed email addresses.
    Valid Example: 
    FBL email address is aolfbl@abuse.aol.com
    192.168.1.1 and 192.168.1.2 ASN WHOIS:

    OrgName:    Your Company Name 
    ...
    NetRange:   192.168.1.0 - 192.168.1.255 
    CIDR:       192.168.1.0/24
    NetName:    COMPANY-1

    OrgAbuseHandle: xxxx
    OrgAbuseName:   abuse 
    OrgAbusePhone:  xxxx
    OrgAbuseEmail:  abuse@abuse.aol.com
    ...

More about feedback loops can be found here: http://postmaster-us.info.aol.com/fbl/index.html.

comment:11 Changed 6 years ago by https://id.mayfirst.org/ross

  • Owner set to https://id.mayfirst.org/ross
  • Status changed from new to assigned

comment:12 Changed 6 years ago by https://id.mayfirst.org/portside-submission

Ross,

Send me the message to AOL. I will re-send from the Portside queue (portside@…) Maybe that will get through.

Jay

comment:13 Changed 6 years ago by https://id.mayfirst.org/ross

  • Keywords listserv aol workers added

This still seems to be a problem. I'm going to work with jamie tomorrow and see if we can come to some reasonable solution about aol.

comment:14 Changed 5 years ago by https://id.mayfirst.org/ross

  • Keywords workers removed

Over the last few weeks, we've been periodically pushing aol emails through a secondary relay on rustin.mayfirst.org to warm up the server for handling all aol.com email addresses. The process involves a postfix-multi-relay. To turn the relay on and off (for warming up rustin), I run the following two commands with a few seconds in between:

0 morales:/etc/postfix#  mv transport.relay{,.old} && mv transport.relay{.aol,} && postmap transport.relay && postfix flush
0 morales:/etc/postfix#
0 morales:/etc/postfix# mv transport.relay{,.aol} && mv transport.relay{.old,} && postmap transport.relay && postfix flush
0 morales:/etc/postfix#

The file transport.relay.aol looks like this:

0 morales:/etc/postfix# cat transport.relay.aol
# we're testing the sending of aol email to b.rustin.mayfirst.org
aol.com smtp:b.rustin.mayfirst.org
0 morales:/etc/postfix

I'm not exactly sure how long we need to warm up rustin, but I think we're almost ready to make this relay live.

~/ross

comment:15 Changed 5 years ago by https://id.mayfirst.org/ross

  • Resolution set to fixed
  • Status changed from assigned to closed

I've just switched the transport.rely for aol to rustin on a permanent basis. Hopefully this will keep aol emails flowing consistently.

~/ross

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.