wiki:filter-check

Filter check

May First/People Link regularly checks to see if our servers are sending email messages into the spam/junk boxes of corporate providers. These scripts are installed and configured on gil.mayfirst.org.

There are three parts to the check:

  • mf-monitor-filtercheck - This script runs on a cron job and updates the nagios server if the status changes. When run on the terminal it will take 45 minutes to complete (to leave enough time for all messages to be delivered) and will output either OK: (if all messages landed in the inbox), WARNING: (if not all messages landed in the inbox and the last time it ran that status was OK:) or CRITICAL: (if not all messages landed in the inbox and the last time it ran the status was not OK:).
  • mf-filtercheck - This script is the one executed by mf-monitor-filtercheck - by default it provides verbose output so you can see which corporate providers are filtering our email. You can pass the environment variable MF_SLEEP=60 if you want the script to sleep for only 60 seconds instead of the full 45 minutes.
  • filter-check - This is the generic script that powers mf-filercheck. See more below...

What to do if the nagios monitor alert Ensure Email goes to Inbox goes off

Check the logs.

journalctl --since "4 hours ago"| grep filter-check

Or

cat /var/log/syslog | grep filter-check

If the exit code is "1" it means the message landed in the spambox. If the exit code is 2 it means it was not received. If the exit code was 255 it means there was an error.

If the exit code is "1" - you can retest to see if this is happening consistently:

filter-check --sendto SENDTO --sendvia SENDVIA--sleep 30

For example, if the error is:

0 gil:~# grep filter-check /var/log/syslog
Jan 11 09:14:37 gil filter-check: Exit code '1' fetching messageid '<20180111131202.9683.90646@gil.mayfirst.org>' from 'gmail' via 'localhost'
0 gil:~#

That means a message sent to gmail via localhost ended up in the spam box. You can check to see if that is still happening with:

0 gil:~# filter-check --sendto gmail --sendvia localhost --sleep 30
Message sent (Thu, 11 Jan 2018 15:13:32 -0000)
Message-id: <20180111151332.17519.37006@gil.mayfirst.org>
Sleeping for 30.0 seconds.
Found in: Inbox

ARC-Authentication-Results: i=1; mx.google.com;
       spf=neutral (google.com: 216.66.23.48 is neither permitted nor denied by best guess record for domain of jamie@workingdirectory.net) smtp.mailfrom=jamie@workingdirectory.net
Received-SPF: neutral (google.com: 216.66.23.48 is neither permitted nor denied by best guess record for domain of jamie@workingdirectory.net) client-ip=216.66.23.48;
Authentication-Results: mx.google.com;
       spf=neutral (google.com: 216.66.23.48 is neither permitted nor denied by best guess record for domain of jamie@workingdirectory.net) smtp.mailfrom=jamie@workingdirectory.net
0 gil:~# 

This demonstrates that now messages are being properly delivered.

If the exit code is "2" - you can manually login to the provider (see keyringer for all passwords) and see if the message eventually made it. Then you can debug the delay.

If the exit code is "3" - you can try to re-run manually using filter-check (like with exit code 1) to see if you can re-create the error.

filter-check

The filter-check program sends email via our servers to various corporate providers and checks to see if they land in the inbox or the spam box.

Current providers checked: Gmail, Yahoo and outlook.com.

Basic usage:

usage: filter-check [-h] [--sendvia SENDVIA] [--sendto SENDTO] [--quiet]
                    [--gtube] [--headers] [--sleep SLEEP]
                    [--emailfrom EMAILFROM] [--subject SUBJECT] [--msg MSG]
                    [--sendonly] [--fetchfrom FETCHFROM]
                    [--messageid MESSAGEID]

Check deliverability to various mail providers. Configuration file is in
~/.filter-check.conf

optional arguments:
  -h, --help            show this help message and exit
  --sendvia SENDVIA     the host to relay the email
  --sendto SENDTO       the host to send the email
  --quiet               surpress all output, use exit codes only
  --gtube               use spamassassin code to identify the message as spam
                        (for debugging)
  --headers             output full headers, instead of brief headers
  --sleep SLEEP         number of seconds to wait between sending and checking
                        email
  --emailfrom EMAILFROM
                        send the test message from this email
  --subject SUBJECT     specify the subject for the test message
  --msg MSG             specify the body of the test message
  --sendonly            don't fetch the message, only send it and output the
                        message-id sent
  --fetchfrom FETCHFROM
                        don't send a message, only fetch a message from this
                        host matching the passed messageid
  --messageid MESSAGEID
                        when using fetchfrom, fetch the messageid specified in
                        this option

Exit codes: 0 if message is found in Inbox, 1 if found in spam box, 2 if not
delivered, 255 if error.

Code is available via git clone git://git.mayfirst.org/mfpl/filter-check

Last modified 10 months ago Last modified on Nov 21, 2018, 10:32:55 AM