= redacting messages in mailman archives =

https://lists.mayfirst.org/ runs mailman.  Sometimes information gets posted to a mailman list that should not be public.  Once private information is posted, it has already been leaked.  But people may want to redact information from our publicly-accessible archives or from follow-on list digest mailings, etc.  We should do this sparingly, and only with good reason (e.g. someone posting someone else's private information to a publicly archived list).

This page contains information on how an MF/PL administrator can redact messages from a mailman archive to avoid further exposure.  It does not solve the problem that the data has already leaked.

Connect to leslie.mayfirst.org and switch to the mailman user:

{{{
ssh -t root@leslie.mayfirst.org su - list
}}}

FIXME: can we lock these files so that mailman doesn't clobber them while we edit?  This could be risky on a high-traffic list.

Whether this list is private or publicly archived, the mbox to edit is in archives/private/.

Assuming that the message has just recently been sent, edit both the pending digest mail and the current archive:

{{{
cd /var/lib/mailman
$EDITOR archives/private/$listname.mbox/$listname.mbox lists/$listname/digest.mbox
}}}

If the message hasn't been recently sent, you needn't edit the digest mail.

Things to watch out for when editing these files:
 * avoid removal of entire messages, since that will screw up the public-facing URLs. You may just replace the message body with a body that says "message redacted", for example.
 * If you are doing significant surgery, please pay attention to the Content-Type headers, etc, so that the message ends up cleanly formatted
 * Avoid creating new lines that start with the word `From` (with any capitalization), since this is how mbox indicates the start of a new e-mail message.

Once this is done (still as `list@leslie`) ask mailman to re-build all the archives for that list, but first backup and remove the the old archives to avoid duplications:

{{{
mkdir ~/ticketXXXX
tar cvfz ~/ticketXXXX/$listname.tar.gz archives/private/$listname/
rm -rf archives/private/$listname/
./bin/arch $listname
}}}

(anyone know of a way to do this in a more targeted fashion?  for long lists or high-traffic lists, this step could take ages!)

You may also be interested in [http://wiki.list.org/pages/viewpage.action?pageId=4030681 the upstream documentation about removing/redacting messages in mailman] for more detail.