wiki:apachesolr

Apache Solr on abernathy.mayfirst.org

Apache Solr is an indexing and searching program that can replace the Drupal core search engine with a much improved system.

abernathy.mayfirst.org is a dedicated Apache Solr Server that is available for use by any May First/People Link member. Feel free to contact the publicly archived support list, open a ticket or contact the support team if you want to use Apache Solr on your website.

Security

Solr does not come with a generic authentification mechanism. The Solr installation on abernathy uses a two-level security system. The connection between the Drupal web server (denoted by $SERVERNAME) and the Solr server is secured by ssh against attacks from the web. The instances (denoted by $CORENAME) on the Solr server are protected by random admin paths (denoted by $COREPATH) from one another. The random admin paths are generated by core_ followed by pwgen -1 --secure 30

Reliability

The ssh connections between the web server and the Solr server is supervised by runit. If there is a problem ssh will end the connection and runit will restart ssh. Sometimes Solr crashes Tomcat if there are too many requests at once. So we use monit to monitor Solr and restart Tomcat automatically if it crashed.

Website Administrators

If you run a Drupal website, you can use Solr. The first step is to determine whether your primary host $SERVERHOST is connected to our Solr server. The following hosts are currently connected:

  • at.indymedia.org
  • bethemedia.org.uk
  • lumumba.mayfirst.org
  • sojourner.mayfirst.org
  • yser.mayfirst.org

If your primary server is not listed, please open a ticket or contact the support team to request having your primary host added.

Enable Apache Solr on Drupal

To enable Solr you have to install Apache Solr Drupal module which is done best through drush:

drush dl apachesolr
drush solr-phpclient

Configure Apache Solr on Drupal

Once your host is solr-enabled then you can then connect to the Apache Solr server. Enter on your website at admin/settings/apachesolr:

Solr host name: localhost
Solr port: 9080
Solr path: /solr/$COREPATH

Add TcpForward exception

MFPL servers by default do not allow TCP Forwarding. You can create an exception by adding:

  $sshd_tcp_forwarding = "yes"

To the server's puppet configuration before the include ssh line.

Administrators

Below are the steps administrators must take to provide Solr access.

On the primary server

Create a new user on the server (called $SERVERNAME with host $SERVERHOST from now on) with the user name solr-ssh and add:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK9Rnog/DuOeFN0gusPadGHtAFs/vrfuNyxsNpakycA/+hkKbgEid/xR4Tbbs25ak03bBJePdCf/PpORcB2rgiFwte4fOAfoXX/VHBnqZLFizhZKPCtG0gstQXTwdvOkx+8p5yPODVQvkqeOJCxF7EAI5B9VgcoJWdh2tQX+e07v/DgRtTs+01re0ZmVxmpGpgRWICdaPms9Hh/DwJT7gs19TpPv1qBgzDTN/z12b/6BRsShk/eEGVwslGF3meFA5+saWPmCSJKKK3Pg0btp3LxLkZfWUtdnVr7ASeJlevnthOPlSMp0ITs6oznhQEjycWK+nBYNUrK+cguiMepe/t solr-ssh@abernathy

to the new user's ~/.ssh/authorized_keys file. abernathy will connect to $SERVERHOST with these credentials and provide an ssh tunnel to the Apache Solr server.

On Abernathy

There are two things to do on abernathy.mayfirst.org: add a ssh tunnel from abernathy to the web server and create a new Apache Solr instance in Apache Tomcat.

Create SSH tunnel

Log as solr-ssh into $SERVERHOST from solr-ssh on abernathy.mayfirst.org

sudo -u solr-ssh ssh solr-ssh@$SERVERHOST

End the connection with exit

On abernathy, add a SSH tunnel by using solr_addssh (see solr_addssh --man for help and more options)

solr_addssh $SERVERNAME $SERVERHOST

You can list port forwardings with solr_listssh and delete (stop/disable/remove) port forwarding with solr_delssh $SERVERNAME (add --man for help and more options).

Test SSH tunnel

Log as solr-ssh into $SERVERHOST and try

curl http://localhost:9080

to see the Apache Tomcat welcome page from abernathy.mayfirst.org.

Create new Solr configuration

Use solr_addcore to create a new Solr instance

solr_addcore $CORENAME

The script will output the Solr admin path ($COREPATH) for the new Solr instance.

You can list instances with solr_listcore and delete an instance with solr_delcore $CORENAME (add --man for help and more options).

Restart Apache Tomcat

Restart Apache Tomcat with

service tomcat6 restart

Test new Solr core

Check that the new core is accessible with:

curl http://localhost:8080/solr/$COREPATH/admin/.

Troubleshooting

Make sure that the ssh tunnel is up

sv status solr-ssh-$SERVERNAME

If the tunnel is down you should have a look at the logs: tail -f /var/log/supervise/solr-ssh-$SERVERNAME/current

For more information add -v to the ssh command in /etc/sv/solr-ssh-$SERVERNAME/run and check the logs again.

Check the Solr cores with

solr_checkcore

(add --man for help and more options)

Last modified 8 months ago Last modified on Oct 19, 2018, 5:18:35 PM