wiki:mailman-redact

redacting messages in mailman archives

https://lists.mayfirst.org/ runs mailman. Sometimes information gets posted to a mailman list that should not be public. Once private information is posted, it has already been leaked. But people may want to redact information from our publicly-accessible archives or from follow-on list digest mailings, etc. We should do this sparingly, and only with good reason (e.g. someone posting someone else's private information to a publicly archived list).

This page contains information on how an MF/PL administrator can redact messages from a mailman archive to avoid further exposure. It does not solve the problem that the data has already leaked.

Connect to leslie.mayfirst.org and switch to the mailman user:

ssh -t root@leslie.mayfirst.org su - list

FIXME: can we lock these files so that mailman doesn't clobber them while we edit? This could be risky on a high-traffic list.

Whether this list is private or publicly archived, the mbox to edit is in archives/private/.

Assuming that the message has just recently been sent, edit both the pending digest mail and the current archive:

cd /var/lib/mailman
$EDITOR archives/private/$listname.mbox/$listname.mbox lists/$listname/digest.mbox

If the message hasn't been recently sent, you needn't edit the digest mail.

Things to watch out for when editing these files:

  • avoid removal of entire messages, since that will screw up the public-facing URLs. You may just replace the message body with a body that says "message redacted", for example.
  • If you are doing significant surgery, please pay attention to the Content-Type headers, etc, so that the message ends up cleanly formatted
  • Avoid creating new lines that start with the word From (with any capitalization), since this is how mbox indicates the start of a new e-mail message.

Once this is done (still as list@leslie) ask mailman to re-build all the archives for that list, but first backup and remove the the old archives to avoid duplications:

mkdir ~/ticketXXXX
tar cvfz ~/ticketXXXX/$listname.tar.gz archives/private/$listname/
rm -rf archives/private/$listname/
./bin/arch $listname

(anyone know of a way to do this in a more targeted fashion? for long lists or high-traffic lists, this step could take ages!)

You may also be interested in the upstream documentation about removing/redacting messages in mailman for more detail.

See also https://wikitech.wikimedia.org/wiki/Remove_a_message_from_mailing_list_archive

Last modified 5 years ago Last modified on Aug 6, 2014, 12:56:29 AM