Opened 6 years ago

Last modified 5 years ago

#7425 assigned Task/To do item

Potential errors during wheezy upgrades

Reported by: https://id.mayfirst.org/ross Owned by: https://id.mayfirst.org/ross
Priority: Medium Component: Tech
Keywords: drupal-errors php5.4 Cc: support-team@…
Sensitive: no

Description

There look to be a bunch of drupal 6 errors after we upgrade to wheezy/php5.4. I think we should begin logging these errors and any possible solutions. In my first venture in this regard, I get quite a few views errors that seem that they will not be fixed https://drupal.org/node/1555446

strict warning: Non-static method view::load() should not be called statically in /home/members/nacla/sites/web.nacla.org/web/sites/all/modules/contrib/views/views.module on line 906.
strict warning: Declaration of views_handler_filter::options_validate() should be compatible with views_handler::options_validate($form, &$form_state) in /home/members/nacla/sites/web.nacla.org/web/sites/all/modules/contrib/views/handlers/views_handler_filter.inc on line 607.
strict warning: Declaration of views_handler_filter::options_submit() should be compatible with views_handler::options_submit($form, &$form_state) in /home/members/nacla/sites/web.nacla.org/web/sites/all/modules/contrib/views/handlers/views_handler_filter.inc on line 607.
strict warning: Declaration of views_plugin_style_default::options() should be compatible with views_object::options() in /home/members/nacla/sites/web.nacla.org/web/sites/all/modules/contrib/views/plugins/views_plugin_style_default.inc on line 24.

Attachments (2)

views-master-patch-php5.4.patch (15.5 KB) - added by https://id.mayfirst.org/ross 5 years ago.
master-views-php5.4.2014-03-13.patch (16.1 KB) - added by https://id.mayfirst.org/ross 5 years ago.
Revised master patch.

Download all attachments as: .zip

Change History (33)

comment:1 Changed 6 years ago by https://id.mayfirst.org/dskallman

  • Owner set to https://id.mayfirst.org/ross
  • Status changed from new to assigned

Ross, assigning to you. If not you, please re-assign to the right person.

comment:2 Changed 6 years ago by https://id.mayfirst.org/ross

  • Summary changed from Expected drupal errors during wheezy upgrade to Potential errors during wheezy upgrades

comment:3 Changed 6 years ago by https://id.mayfirst.org/ross

The dbndns package is not in wheezy as of 06-27-2013. This will cause problems at least for viewsic (maybe other servers) on upgrade. If it is not in wheezy backports by the time we upgrade, we'll need to add sid sources and preferences. On haring, we added the following:

0 haring:/usr/local/etc/red# cat /etc/apt/sources.list.d/sid.sources.list 
deb http://http.us.debian.org/debian sid main
0 haring:/usr/local/etc/red#
0 haring:/usr/local/etc/red# cat /etc/apt/preferences.d/sid 
Package: *
Pin: release n=sid
Pin-Priority: 200
0 haring:/usr/local/etc/red# 

comment:4 follow-up: Changed 6 years ago by https://id.mayfirst.org/dkg

DebianPackage:dbndns has not been in squeeze or wheezy for years, due to DebianBug:516394, which appears to be a matter of lengthy dispute over how vulnerable dbndns is to cache poisoning. since this dispute remains unresolved, i do not expect dbndns to end up in wheezy.

I recommend not adding sid to every machine's sources.list -- i think it's particularly risky to link to the entire unstable archive, since that means that an apt-get install foo (if foo is only in unstable and not in stable) is likely to try to go ahead and bring other unstable packages into our production equipment.

The same version of dbndns that worked for squeeze will continue to work for wheezy if this is an upgrade process. where did the versions on squeeze systems come from? Can we replicate that?

If we really need dbndns to come from an apt repository, we should use an MF/PL-specific apt repository instead of the entire debian unstable.

comment:5 in reply to: ↑ 4 ; follow-up: Changed 6 years ago by https://id.mayfirst.org/ross

Replying to https://id.mayfirst.org/dkg:

I recommend not adding sid to every machine's sources.list --

I was only suggesting this for machines that need dbndns.

The same version of dbndns that worked for squeeze will continue to work for wheezy if this is an upgrade process. where did the versions on squeeze systems come from? Can we replicate that?

Ah, this is good information. I didn't realize viewsic was using the lenny version of dbndns.

0 viewsic:~# dpkg -l dbndns
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                        Version                     Description
+++-===========================-===========================-======================================================================
ii  dbndns                      1:1.05-4+lenny1             Debian fork of djbdns, a collection of Domain Name System tools
0 viewsic:~# 

I'm just trying to document problems we might run into in the upgrade process. The current process was not an upgrade but a server created as a wheezy server, so I still do not know what will happen during an upgrade, but it's something we should certainly be aware of when we do the upgrades.

comment:6 in reply to: ↑ 5 Changed 6 years ago by https://id.mayfirst.org/dkg

Replying to https://id.mayfirst.org/ross:

Replying to https://id.mayfirst.org/dkg:

I recommend not adding sid to every machine's sources.list --

I was only suggesting this for machines that need dbndns.

I recommend not adding sid to any machine's sources.list if the only thing they need from that repository is dbndns. dbndns is an isolated package that has very few explicit dependencies and a very stable upstream. If we can find other ways (not involving sid) to get the package onto the machines that need this package, we should do so.

comment:7 in reply to: ↑ description Changed 6 years ago by https://id.mayfirst.org/ross

Replying to https://id.mayfirst.org/ross:

strict warning: Non-static method view::load() should not be called statically in /home/members/nacla/sites/web.nacla.org/web/sites/all/modules/contrib/views/views.module on line 906.

Here are some drupal issues covering some php5.4 views strict warning errors:

comment:8 Changed 6 years ago by https://id.mayfirst.org/ross

The mysql_insert_id() function seems to produce unpredictable results in php5.4 as well. It's listed as depreciated as of php5.5, but the function failed to return correct results in drupal when moved from php5.3 to php5.4. I was unable to discern the cause of the failure.

comment:9 follow-up: Changed 5 years ago by https://id.mayfirst.org/ross

Steve has pointed out a mediawiki gotcha for wheezy upgrades, since wheezy uses mediawiki 1.19 and squeeze 1.15, see #7992. It looks like each currently installed mediawiki web app will need to be upgraded, the current list of such implementations is here:

mysql> select red_item_web_app.item_id as item_id, item_host from red_item_web_app join red_item on red_item.item_id = red_item_web_app.item_id where web_app_name = "mediawiki" and item_host != "chelsea.mayfirst.org" and item_status = "active";
+---------+-----------------------+
| item_id | item_host             |
+---------+-----------------------+
|   87218 | rose.mayfirst.org     |
|   87907 | rose.mayfirst.org     |
|   88779 | rodolpho.mayfirst.org |
|   90051 | rose.mayfirst.org     |
|   93087 | proudhon.mayfirst.org |
|   93798 | buffy.mayfirst.org    |
|   95636 | buffy.mayfirst.org    |
+---------+-----------------------+
7 rows in set (0.00 sec)

mysql> 

This is a process srevilak used for a prior mediawiki upgrade, which might help think through the necessary steps.

comment:10 Changed 5 years ago by https://id.mayfirst.org/ross

varnish will move from version 2 to version 3 during the wheezy upgrade. I know there are some configuration changes in varnish version 3. We will need to evaluate those changes against our current configs before upgrading our varnish servers.

Also note this varnish bug report. Apparently an empty GET request can crash the server.

Here are the changes to varnish 3 https://www.varnish-cache.org/docs/3.0/installation/upgrade.html

Last edited 5 years ago by https://id.mayfirst.org/ross (previous) (diff)

comment:11 in reply to: ↑ 9 Changed 5 years ago by https://id.mayfirst.org/srevilak

Replying to https://id.mayfirst.org/ross:

Steve has pointed out a mediawiki gotcha for wheezy upgrades, since wheezy uses mediawiki 1.19 and squeeze 1.15, see #7992.

ticket:7992#comment:12 contains most of the gory details about wheezy's mediawiki. Short story:

  • mediawiki 1.19 (wheezy) is a lot different than mediawiki 1.15 (squeeze)
  • our approach to customizing 1.15's install might not work for 1.19 (have to work on this)
Last edited 5 years ago by https://id.mayfirst.org/srevilak (previous) (diff)

comment:12 Changed 5 years ago by https://id.mayfirst.org/ross

It appears that sqlite extensions are not enabled by default in wheezy's php version.

Here's what our squeeze servers look like:

0 mandela:/etc/php5# grep -r extension ./* | grep sqlite
./cgi/php.ini:;sqlite3.extension_dir =
./cgi/conf.d/pdo_sqlite.ini:extension=pdo_sqlite.so
./cgi/conf.d/sqlite.ini:extension=sqlite.so
./cgi/conf.d/sqlite3.ini:extension=sqlite3.so
./cli/php.ini:;sqlite3.extension_dir =
./cli/conf.d/pdo_sqlite.ini:extension=pdo_sqlite.so
./cli/conf.d/sqlite.ini:extension=sqlite.so
./cli/conf.d/sqlite3.ini:extension=sqlite3.so
./conf.d/pdo_sqlite.ini:extension=pdo_sqlite.so
./conf.d/sqlite.ini:extension=sqlite.so
./conf.d/sqlite3.ini:extension=sqlite3.so
0 mandela:/etc/php5#

and here's rose after upgrade (the same is true for chelsea's default wheezy installation):

0 rose:/etc/php5# grep -r extension ./* | grep sqlite
./apache2/php.ini:;sqlite3.extension_dir =
./cgi/php.ini:;sqlite3.extension_dir =
./cli/php.ini:;sqlite3.extension_dir =
./mods-available/pdo_sqlite.ini:extension=pdo_sqlite.so
./mods-available/sqlite3.ini:extension=sqlite3.so
0 rose:/etc/php5#

Red relies on sqlite for handling email auto-responders so this needs to be resolved. I'm not clear as to whether or not this should be handled by puppet. I'm going to try to fix rose by hand for #8016.

comment:13 Changed 5 years ago by https://id.mayfirst.org/jamie

#8016 should now be fixed in red.

comment:14 follow-up: Changed 5 years ago by https://id.mayfirst.org/jamie

We received a report via irc that a D6 site was broken on rose (rainforestfoundation.org). Upgrading views, ctools, panels and views to their dev version fixed the problem, although it's not clear if all three needed to be upgraded.

comment:15 in reply to: ↑ 14 ; follow-up: Changed 5 years ago by https://id.mayfirst.org/ross

Replying to https://id.mayfirst.org/jamie:

We received a report via irc that a D6 site was broken on rose (rainforestfoundation.org). Upgrading views, ctools, panels and views to their dev version fixed the problem, although it's not clear if all three needed to be upgraded.

The continuing watchdog errors experienced by rainforestfoundation.org is the first documented problem in this ticket, and it seems as if there will be no solution offered.

comment:16 in reply to: ↑ 15 Changed 5 years ago by https://id.mayfirst.org/dkg

Replying to https://id.mayfirst.org/ross:

The continuing watchdog errors experienced by rainforestfoundation.org is the first documented problem in this ticket, and it seems as if there will be no solution offered.

What are the continuing watchdog errors? Is there an upstream bug report or other external documentation?

comment:17 Changed 5 years ago by https://id.mayfirst.org/ross

Here are the errors:

Declaration of views_plugin_style_default::options() should be compatible with views_object::options()
Non-static method views_many_to_one_helper::option_definition() should not be called statically, assuming $this from incompatible context

And the ticket description contains the link to the upstream report.

comment:18 Changed 5 years ago by https://id.mayfirst.org/dkg

Thanks! the upstream error report suggests that the only reason these errors will not get fixed is because drupal 6 needs to retain backward compatibility with PHP 4. D7 apparently has patches for this (and similar) issues already incorporated. The report also suggests that these same warnings were appearing when running D6 against PHP 5.3 (so maybe the warnings are happening already on other squeeze instances?)

Fortunately, we have no instances of PHP 4 on MF/PL servers to worry about, and we distribute our own copies of drupal. This means we have a good mechanism in place to distribute the patches against D6 that would clear these warnings up if they are a result of core drupal code. Perhaps we should consider this kind of distribution mechanism for important modules like Views as well.

comment:19 Changed 5 years ago by https://id.mayfirst.org/jamie

I agree with dkg - I think we need to write and make available our own copies of the Drupal 6 modules that we know will cause problems (minimally views, but I suspect this list will grow).

To accomodate this work, I think we need adjust our MOSH upgrade schedule (push it all back at least one month). This way we can send a lowdown announcement now warning people about the imminent upgrade (providing two months notice before the next mosh is upgraded) and reference the wiki page with the upgrade schedule and have a few weeks to updates the drupal 6 modules.

At least one month before we upgrade malcolm (the next MOSH on the list), we should:

  • Have working copies of views (and any other module we know will cause errors with Drupal 6) - available on all MOSH'es so groups can symlink to them or copy them: #8024
  • Modified version of our email to groups (the current email only mentions Drupal versions earlier than Drupal 6, it should mention Drupal 6 with a warning about upgrading the modules that need upgrading)
  • A targetted email to Drupal 6 members (currently we are only planning to send targetted emails to Drupal 5 users and below): #8010 (for building this list)

How does that sound? Is this realistic?

comment:20 Changed 5 years ago by https://id.mayfirst.org/dkg

From the point of view of getting support from our upstream distro, debian wheezy was released on 2013-05-04. The usual arrangement is that the previous release (squeeze, in this case) will be supported for one additional year, so we really do need to be transitioned to wheezy by the end of April, 2014.

While i like this proposal for D6, I don't have any time or expertise to contribute to the project. i also observe we didn't do this packaging and distribution work for common drupal modules during D6's prime years of activity. Will we also be providing easily-targetable common modules for D7, or expecting users to continue using drush to manage their own modules?

how will drush interact with a D6 installation that has some modules linked in from an unwritable location?

comment:21 Changed 5 years ago by https://id.mayfirst.org/ross

I do not think packaging our own version of modules is a good idea at all. A number of drupal 6 sites have been running on wheezy for a while now without this views problem getting in the way of site functionality.

I believe we should inform the membership of the potential problem, encourage them to update all their modules, let them know to turn of error reporting to the display, and move forward.

If we want to fix views for members, the only realistic option I see is writing a patch and instructing members how to patch their views module and offering to run the patch for them if they want. More than this is a recipe for total support-team overload and a level of confusion worse than some errors that we understand.

We also need to be aware of and test views 2 vs. views 3 to see the differences between them.

~/ross

comment:22 Changed 5 years ago by https://id.mayfirst.org/jamie

On rose, the rainforestfoundation's D6 site broke until views were upgraded to the dev version (it was already running the latest stable version). The latest stable version is views 2. The dev version is views 3.

I think we need to be ready for this problem. You've convinced me about not maintaining these as symlinks to root-owned directories.

But I think we should at least be able to provide a git repo and a download address. I don't think it commits us to significant maintenance beyond the creation of the initial copy. And, it provides us with an easy set of directions to groups.

The alternative would be to instruct people to upgrade to the dev version of view - views 3. Which is a viable route.

What do other people think?

comment:23 Changed 5 years ago by https://id.mayfirst.org/jamie

Note: autoreponder db needs to be upgraded: https://support.mayfirst.org/ticket/8027

comment:24 Changed 5 years ago by https://id.mayfirst.org/ross

Actually, there are two stable releases for views in drupal 6. One is views 2 and the other is views 3, each of these stable releases has a development version.

And, just for the record. nacla.org is running views-2.x on a wheezy server without the site breaking.

0 naclaweb@kahlo:~/web.nacla.org/web$ uname -a
Linux kahlo 3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2 x86_64 GNU/Linux
0 naclaweb@kahlo:~/web.nacla.org/web$ drush pm-info views
 Extension        :  views                                                                                                           
 Project          :  views                                                                                                           
 Type             :  module                                                                                                          
 Title            :  Views                                                                                                           
 Description      :  Create customized lists and queries from your database.                                                         
 Version          :  6.x-2.16                                                                                                        
 Package          :  Views                                                                                                           
 Core             :  6.x                                                                                                             
 PHP              :  4.3.5                                                                                                           
 Status           :  enabled                                                                                                         
 Path             :  sites/all/modules/contrib/views                                                                                 
 Schema version   :  6013                                                                                                            
 Requires         :  none                                                                                                            
 Required by      :  features_test, ffpc, img_assist, na_views, tagadelic_views, views_attach, views_bulk_operations, views_content, 
                     views_customfield, views_export, views_slideshow, views_ui, wysiwyg_imageupload_browser,                        
                     views_slideshow_singleframe, views_slideshow_thumbnailhover                                                     


0 naclaweb@kahlo:~/web.nacla.org/web$ 

I am going to run some tests on our infrastructure against views versions and our wheezy servers to get a more accurate picture of what's going on. I think this should happen before we make any decisions.

~/ross

comment:25 Changed 5 years ago by https://id.mayfirst.org/jamie

Nice Ross - thanks! I didn't realize there were stable versions of views 3 available.

Note - Jack updated ctools, panels, and views to their dev versions to fix the breakage. Perhaps it wasn't views that was causing the breakage?

comment:26 Changed 5 years ago by https://id.mayfirst.org/jamie

Based on Ross' tests and some info I've gained from bart from linksunten.indymedia.org - it seems that panels and draggableviews are the only 3rd party modules we've identified that are known to break, and both will work if ugpraded to the current dev version.

Changed 5 years ago by https://id.mayfirst.org/ross

comment:27 Changed 5 years ago by https://id.mayfirst.org/jamie

We found an error with calendar views - minor, but easily fixable. The patch is available (comment #78). For full info see the Drupal issue.

The symptom is: Warning: Illegal string offset 'data' in template_preprocess_calendar_month

Changed 5 years ago by https://id.mayfirst.org/ross

Revised master patch.

comment:28 Changed 5 years ago by https://id.mayfirst.org/ross

I'm adding a new version of the master patch to incorporate this fix https://drupal.org/node/2120727 pointed out by jackalope.

comment:29 Changed 5 years ago by https://id.mayfirst.org/rootwork

Because I had had to figure this out on my own (see my comment in #8010) I didn't get a chance to try your master patch. I also had some other modules that clients were using that needed to be upgraded to be fixed anyway. So for anyone else who wants to do this the drush way:

drush en update -y

drush up ctools-1.x-dev panels-3.x-dev views-2.x-dev -y

drush up date-2.x-dev backup_migrate-2.x-dev -y

cd sites/all/modules/contrib/views

curl https://drupal.org/files/issues/master-views-php5.4.2014-03-13.patch | git apply

drush cc all

drush dis update -y

Commentary:

  • If you have update notifications enabled and want to leave it enabled, ignore the first and last lines
  • date and backup_migrate were relatively commonly-used modules we needed to upgrade. Ignore if you don't use these. We also needed to update content_profile (to dev), openlayers_proximity (to dev) and do some custom editing of an openlayers_proximity file. Those are much less-used, though, so not included above.
  • Where your views module exists in your directory structure will vary. You may not have a contrib folder; it might simply be in sites/all/modules. If you are running a multisite install, it will be in sites/[siteurl]/modules. If you are running off of a distro or installation profile, it will be in profiles/[profilename]/modules/contrib.
  • The final cache-clear seemed necessary to get the patch to views to "take," at least in our heavily-cached (advagg, boost, etc.) setup.

So for anyone else who runs into this, hope it helps. It wasn't a super-fun way for me to spend my Saturday evening :/

comment:30 Changed 5 years ago by https://id.mayfirst.org/jamie

Thanks for the update Ivan. As you can imagine, it's pretty hard to cover all the different possibilities of Drupal configurations. By doing the upgrade slowly, over time, we hoped to maximize the amount of information we could gather to help members. And, as seen on this ticket, we made every effort we could to post what we've found. Sorry this didn't make it into the main wiki page. I've added it now.

comment:31 Changed 5 years ago by https://id.mayfirst.org/jackaponte

Yet another Drupal 6 core patch that could be useful, this one for the taxonomy module: https://drupal.org/node/996124#comment-7531599

Please login to add comments to this ticket.

Note: See TracTickets for help on using tickets.