wiki:faq/shared-varnish-server

Shared Varnish Caching

If you find that your site runs slowly and you've turned on all the CMS specific caching features available to you, it might be time to use our shared varnish server. Varnish helps manage higher traffic websites by creating a cache of all the previous page loads and serving them up with a minimal amount of processing. In order to hook into our shared varnish instance, you'll need to make some changes to your DNS entries, and a support team member will need to add a file to our varnish server (currently dolores.mayfirst.org).

Changing DNS Entries

Visit the control panel and the hosting order that should use varnish caching. Change the DNS entry for the appropriate domains. For example, you might want

Domain Name IP Address
example.org 209.51.180.234
www.example.org 209.51.180.234
example.com 209.51.180.234
www.example.com 209.51.180.234
example.mayfirst.org doesn't change

Notice that the last entry example.mayfirst.org did not change. This domain can serve as a direct link to the server in case anything goes awry.

Modify Your Config Files

Some CMS's require you to modify your configuration files. You should check to make sure you have the correct configurations set.

For Mediawiki

In your LocalSettings.php, you'll need to add the following:

$wgUseSquid = true;
$wgSquidServers = array('209.51.180.234:80');

For Drupal

  • Drupal 6 requires the Pressflow distribution.
  • Drupal 7: According to Propeller-heads Unite!, you may need to add the following to your settings.php file:
    # Tell Drupal it's behind a proxy
    $conf['reverse_proxy'] = TRUE;
    # Tell Drupal what addresses the proxy server(s) use
    $conf['reverse_proxy_addresses'] = array('209.51.180.234');
    

You may also want these additional variables set from drupal.org

$conf['page_cache_invoke_hooks'] = false;
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 21600;
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['omit_vary_cookie'] = true;

Additionally, the Drupal varnish plugin provides added functionality.

Wordpress

You shouldn't need to take any special steps for wordpress caching. Though please make sure to turn off W3 Total Cache before using our shared varnish server.

Testing

To see if Varnish is working, type your domain into the Is Varnish Working site.

Support Team

Our current varnish server is dolores.mayfirst.org. A support team member needs to make the following configuration changes for each new hosting order. If you need these changes made and you aren't a member of the support team, please open a new ticket explaining what you need to happen.

In your puppet configuration at puppet/modules/mayfirst/files/varnish you will see three important directories:

  • backends
  • mfpl-includes
  • sites

Ensure you have a backend

The first step is to check that the virtual server on which the desired varnish site has a backend. If it does not you will need to create one, the dorothy backend looks like this:

0 ross@virilio:~/projects/puppet/modules/mayfirst/files/varnish/backends$ cat dorothy 
backend dorothy {
    .host = "216.66.23.40";
    .port = "80";
}
0 ross@virilio:~/projects/puppet/modules/mayfirst/files/varnish/backends$

Adding your site vcl file

The backend tells varnish which IP Address to listen on. Once you've established that you have a backend for your server, you can add your specific site from the sites directory. Each domain requires it's own site file, which looks something like this:

0 ross@virilio:~/projects/puppet/modules/mayfirst/files/varnish/sites$ cat bdsmovement.vcl 
# Drop any cookies sent to Wordpress.
sub vcl_recv {
    if (req.http.host ~ "^(www\.)?(bdsmovement)\.net" ) {
        set req.backend = dorothy;
        include "/etc/varnish/mfpl-includes/wordpress/recv.vcl";
    }
}

# Drop any cookies Wordpress tries to send back to the client.
sub vcl_fetch {
    if (req.http.host ~ "^(www\.)?(bdsmovement)\.net" ) {
        include "/etc/varnish/mfpl-includes/wordpress/fetch.vcl";
    }
}
0 ross@virilio:~/projects/puppet/modules/mayfirst/files/varnish/sites$

Modifying your site vcl file

NOTE: In order for varnish to function properly you must include the following line in the sub vcl_recv directive.

set req.backend = BACKEND_NAME

You will need to create a file YOURSITE.vcl and make the necessary modifications. Specifically, you will need to make the req.http.host in both vcl_recv and vcl_fetch match the domain you're caching.

Also...based on your specific cms, you'll need to add mfpl-includes.

We have both standard wordpress and drupal caching components in the directory mfpl-includes. For most configurations you can simply use /etc/varnish/mfpl-include/(wordpress|drupal).recv.vcl in the vcl_recv directive and /etc/varnish/mfpl-include/(wordpress|drupal).fetch.vcl in the vcl_fetch directive.

Modifying varnish server manifest

The manifest of the specific varnish server will need to be modified. dolores.pp currently looks like this:

  class { "m_varnish":
    listen_ip => "209.51.180.234",
    backends => [ "cloudips", "kerr", "dorothy" ],
    sites => [ "bofa.vcl", "occupychi.vcl", "chicagospring.vcl", "bdsmovement.vcl" ]
  }

  m_varnish::backend { "dorothy": }
  m_varnish::backend { "cloudips": }
  m_varnish::backend { "kerr": }
  m_varnish::site { "bofa.vcl": }
  m_varnish::site { "occupychi.vcl": }
  m_varnish::site { "chicagospring.vcl": }
  m_varnish::site { "bdsmovement.vcl": }

You will need to make up to four modifications to this manifest:

  1. Change the m_varnish backends line by adding , "YOUR_BACKEND_NAME" to the backends array (if it doesn't already exist). Warning! Varnish will not restart correctly if you aren't actually referencing this backend from any site.vcl.
  2. Change the m_varnish sites line by adding , "YOUR_SITENAME.vcl" to the sites array.
  3. Add the line m_varnish::backend { "YOUR_BACKEND_NAME" } if it doesn't already exist.
  4. Add m_varnish::site { "YOUR_SITENAME.vcl" }

After pushing your changes to puppet, make sure varnish restarts without error and ensure the DNS changes have been made to point to the varnish server's (dolores) IP Address. Then enjoy the benefits of sharing varnish!

Last modified 2 years ago Last modified on Jan 4, 2017, 4:55:22 PM