Version 34 (modified by 13 years ago) ( diff ) | ,
---|
Apache Solr on mirabal.mayfirst.org
Apache Solr is an indexing and searching program that can replace the Drupal core search engine with a much improved system.
mirabal.mayfirst.org is a dedicated Apache Solr Server that is available for use by any May First/People Link member.
Security
Solr does not come with a generic authentification mechanism. The Solr installation on mirabal uses a two-level security system. The connection between the web and the Solr server is secured by ssh against attacks from the web. The instances (denoted by <sitename>) on the Solr server are protected by random admin paths (denoted by <sitepath>) from one another. Random admin paths are generated by
pwgen --secure 20
Reliability
Sometimes Solr crashes Tomcat if there are too many requests at once. So we use monit to monitor Solr and restart Tomcat automatically if it crashed.
We are currently using autossh to connect the web servers and the Solr server. We try to either combine autossh with Gnu screen or replace it with ssh and runit in the future.
Web site Administrators
If you run a Drupal web site, you can use Solr. The first step is to determine if you primary host is connected to our Solr server. The following hosts are currently connected:
- lucy.mayfirst.org
- lumumba.mayfirst.org
- june.mayfirst.org
- bethemedia.org.uk (209.44.112.176)
- at.indymedia.org
If your primary server is not listed, please open a ticket or contact the support team to request having your primary host added.
Enable Apache Solr on Drupal
To enable Solr you have to install Apache Solr Drupal module which is done best through drush:
drush dl apachesolr drush solr-phpclient
Configure Apache Solr on Drupal
Once your host is solr-enabled then you can then connect to the Apache Solr server. Enter on your website at admin/settings/apachesolr:
Solr host name: localhost Solr port: 9080 Solr path: /solr/<sitepath>
Add TcpForward exception
MFPL servers by default do not allow TCP Forwarding. You can create an exception by adding:
$sshd_tcp_forwarding = "yes"
To the server's puppet configuration before the include ssh
line.
Administrators
Below are the steps administrators must take to provide solr access.
On the primary server
Create a new user on the server (called $SERVER from now on) with the user name solr-ssh and add:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK9Rnog/DuOeFN0gusPadGHtAFs/vrfuNyxsNpakycA/+hkKbgEid/xR4Tbbs25ak03bBJePdCf/PpORcB2rgiFwte4fOAfoXX/VHBnqZLFizhZKPCtG0gstQXTwdvOkx+8p5yPODVQvkqeOJCxF7EAI5B9VgcoJWdh2tQX+e07v/DgRtTs+01re0ZmVxmpGpgRWICdaPms9Hh/DwJT7gs19TpPv1qBgzDTN/z12b/6BRsShk/eEGVwslGF3meFA5+saWPmCSJKKK3Pg0btp3LxLkZfWUtdnVr7ASeJlevnthOPlSMp0ITs6oznhQEjycWK+nBYNUrK+cguiMepe/t solr-ssh@mirabal
to the new user's ~/.ssh/authorized_keys file. mirabal will connect to $SERVER with these credentials and provide an ssh tunnel to the Apache Solr server.
On Mirabal
There are two things to do on mirabal.mayfirst.org: add a ssh tunnel to $SERVER and create a new Apache Solr site in Apache Tomcat.
Create SSH tunnel
Log as solr-ssh into $SERVER from solr-ssh on mirabal.mayfirst.org
sudo -u solr-ssh ssh -p $SERVER_SSH_PORT solr-ssh@$SERVER
End the connection with exit
On mirabal, add a SSH tunnel by adding the login credentials (-p $SERVER_SSH_PORT solr-ssh@$SERVER) to
/etc/default/solr-autossh
Restart the script with
service solr-autossh restart
Test SSH tunnel
Check with
service solr-autossh status
that the tunnel exists.
Log as solr-ssh into $SERVER and download with
wget localhost:9080
the Apache Tomcat welcome page from mirabal.mayfirst.org.
Create new Solr configuration
Let's assume your site is called <sitename>.
Create a directory for the new Solr core
mkdir /usr/share/solr/<sitename>
Copy an existing Solr core
cp -a /etc/solr/testsite /etc/solr/<sitename>
If you need a different configuration than the Apache Solr Drupal module provides you probably have to change at least schema.xml and solrconfig.xml in /etc/solr/<sitename>/conf
.
Change the symbolic link to the new configuration directory
ln -sf /etc/solr/<sitename>/conf /usr/share/solr/<sitename>
Create a new data directory
mkdir /var/lib/solr/data/<sitename>
Change ownership to tomcat6
chown -R tomcat6:tomcat6 /var/lib/solr/data
Change the symbolic link to the new data directory
ln -sf /var/lib/solr/data/<sitename> /usr/share/solr/<sitename>/data
Register new Solr core
Register a new core
<core name="<sitepath>" instanceDir="<sitename>" />
in
/etc/solr/solr-multicore.xml
Restart Apache Tomcat
Restart Apache Tomcat with
service tomcat6 restart
Test new Solr site
Check that the new core is accessible with:
curl http://localhost:8080/solr/<sitepath>/admin/.