Opened 3 years ago

Closed 3 years ago

#11539 closed Bug/Something is broken (fixed)

mailman encoding problems after wheezy jessie upgrade

Reported by: JaimeV Owned by: JaimeV
Priority: High Component: Tech
Keywords: mailman, encoding Cc: Jamie McClelland
Sensitive: no


Apparently mentioned in the upgrade instructions. Debian's modification of mailman to make all strings utf-8 causes problems with existing lists whose descriptions saved through the web interface include non ASCII strings.

This bug report explains how this problem can cause all messages to a list to be shunted.

Ticket #11527 is an example of this problem occurring in leslie.

Debian upgrade instructions:

0 leslie:~# zless /usr/share/doc/mailman/NEWS.Debian.gz | head -n12
mailman (1:2.1.16-1exp1) experimental; urgency=low

  This version has changed the encoding of most strings, templates
  and pages to UTF-8 to meet the Debian release goal of full UTF-8
  support in all packages. It also no longer automatically converts
  mails to ISO-8859-1.

  If you have been using any nōn-ASCII strings in places such as
  the mailing list description, these were be stored wrongly in the
  list configuration file (config.pck), so you will need to change
  those (e.g. via the webinterface) again in order to have them be
  displayed correctly.

So we need a way to identify lists that might be suffering this problem and help members manually fix problematic characters through the mailman admin web interface.

Change History (6)

comment:1 Changed 3 years ago by JaimeV

Cc: Jamie McClelland added
Owner: set to JaimeV
Status: newassigned

Just got a member complaint about two more lists affected by this bug.

comment:2 Changed 3 years ago by Jamie McClelland

I've spent some time debugging and am having a hard time.

There are useful tools for tools for outputting the description of a list, e.g.:

0 leslie:~# withlist support                                                                  |
Loading list support (unlocked)
The variable `m' is the support MailList instance
>>> m.description                                                                             |
'Support list \xc3\xba'

(I added ú to the description which is why we get those escaped characters.)


0 leslie:~# config_list -o - support | grep description | grep -v '^#'                        |
description = "Support list ú"
0 leslie:~# 

But I can't distinguish between a "fixed" version and a not fixed version.

Also, I can't trigger the error without sending a message to the list (sending a message that is held for moderation doesn't trigger it). So it's hard to tell whether or not it has been fixed.

comment:3 Changed 3 years ago by Jamie McClelland

I wrote this one liner to report all the lists that have experienced the bug:

0 leslie:/var/lib/mailman/logs# for id in $(grep -B1 "codec can't decode" error | grep SHUNTING | awk '{print $7}'); do /var/lib/mailman/bin/dumpdb /var/lib/mailman/qfiles/shunt/$id.pck |grep listname; done | sort |uniq
    'listname': 'boletin',
    'listname': 'caravanerxs',
    'listname': 'difusion',
    'listname': 'palestinaymexico',
    'listname': 'sl-online',
    'listname': 'solidaridad-venezuela',
    'listname': 'tierracomun',
0 leslie:/var/lib/mailman/logs#

comment:4 Changed 3 years ago by Jamie McClelland

I have manually visited the web configuration page for each of these lists, logged in, and resaved the general settings. I'm not sure if this has fixed it for each list.

comment:5 Changed 3 years ago by JaimeV

I don't think just saving the general settings will stop the error but now that you've identified the the potentially affected lists I was able to access each of their interfaces and correct the problem characters manually.

comment:6 Changed 3 years ago by JaimeV

Resolution: fixed
Status: assignedclosed

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.