Changes between Version 61 and Version 62 of apachesolr


Ignore:
Timestamp:
Mar 1, 2020, 1:22:03 PM (6 years ago)
Author:
ekes
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • apachesolr

    v61 v62  
    55[http://lucene.apache.org/solr/ Apache Solr] is an indexing and searching program that can replace the Drupal core search engine with a much improved system.
    66
    7 abernathy.mayfirst.org is a dedicated Apache Solr Server that is available for use by any May First/People Link member. Feel free to contact the publicly archived [https://lists.mayfirst.org/mailman/listinfo/solr support list], open a [/newticket ticket] or contact the [wiki:chat support team] if you want to use Apache Solr on your website.
    8 
    9 == Security ==
    10 
    11 Solr does not come with a generic authentification mechanism. The Solr installation on abernathy uses a two-level security system. The connection between the Drupal web server (denoted by $SERVERNAME) and the Solr server is secured by ssh against attacks from the web. The instances (denoted by $CORENAME) on the Solr server are protected by random admin paths (denoted by $COREPATH) from one another. The random admin paths are generated by ''core_'' followed by {{{pwgen -1 --secure 30}}}
    12 
    13 == Reliability ==
    14 
    15 The ssh connections between the web server and the Solr server is supervised by [http://smarden.org/runit/ runit]. If there is a problem ssh will end the connection and runit will restart ssh. Sometimes Solr crashes Tomcat if there are too many requests at once. So we use [http://mmonit.com/monit/ monit] to monitor Solr and restart Tomcat automatically if it crashed.
    16 
    17 == Website Administrators ==
    18 
    19 If you run a Drupal website, you can use Solr. The first step is to determine whether your primary host $SERVERHOST is connected to our Solr server. The following hosts are currently connected:
    20 
    21  * at.indymedia.org
    22  * bethemedia.org.uk
    23  * lumumba.mayfirst.org
    24  * sojourner.mayfirst.org
    25  * yser.mayfirst.org
    26 
    27 If your primary server is not listed, please open a [/newticket ticket] or contact the [wiki:chat support team] to request having your primary host added.
     7abernathy.mayfirst.org is a dedicated Apache Solr Server that is available for use by any May First/People Link member. Open a [/newticket ticket] or contact the [wiki:chat support team] if you want to use Apache Solr on your website.
    288
    299=== Enable Apache Solr on Drupal ===
    3010
    31 To enable Solr you have to install [http://drupal.org/project/apachesolr Apache Solr Drupal module] which is done best through drush:
     11==== Drupal 7: ApacheSolr module ====
     12
     13To enable Solr you have to install [https://drupal.org/project/apachesolr Apache Solr Drupal module] which is done best through drush:
    3214
    3315{{{
    … …  
    3618}}}
    3719
     20==== Drupal 7 & 8: Search API with Search API Solr ====
     21
     22[https://drupal.org/project/search_api Search API module] with the [https://drupal.org/project/search_api_solr Search API Solr module]
     23
    3824=== Configure Apache Solr on Drupal ===
    3925
    40 Once your host is solr-enabled then you can then connect to the Apache Solr server. Enter on your website at ''admin/settings/apachesolr'':
     26You will need to arrange a search index on abernathy (see contact details above). When you have access details these will need to be added to the module's configuration.
    4127
    4228{{{
    43 Solr host name: localhost
    44 Solr port: 9080
    45 Solr path: /solr/$COREPATH
     29Server: abernathy.mayfirst.org
     30Connection: https
     31Core: /solr/solr_name_provided_by_admin
     32Basic Auth:
     33 Username and Password provided
    4634}}}
    4735
    48 === Add TcpForward exception ===
    4936
    50 MFPL servers by default do not allow TCP Forwarding. You can create an exception by adding:
    51 
    52 {{{
    53   $sshd_tcp_forwarding = "yes"
    54 }}}
    55 
    56 To the server's [wiki:puppet puppet configuration] ''before'' the {{{include ssh}}} line.
     37If you are using Drupal 7 the search index ('core') will already be configured. If you are using Drupal 8 you will need to export the configuration files from the server 'files' tab, and pass these back to the solr admin. If you update you field schema on Drupal (change entities, fields, and most importantly enabled languages) you might have to download the 'files' configuration again and get the solr core configuration update.
    5738
    5839== Administrators ==
    5940
    60 Below are the steps administrators must take to provide Solr access.
     41See instructions in ansible abernathy.mayfirst.org host vars.
    6142
    62 === On the primary server ===
     43Presently:
    6344
    64 Create a new user on the server (called $SERVERNAME with host $SERVERHOST from now on) with the user name solr-ssh and add:
    6545
    6646{{{
    67 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDK9Rnog/DuOeFN0gusPadGHtAFs/vrfuNyxsNpakycA/+hkKbgEid/xR4Tbbs25ak03bBJePdCf/PpORcB2rgiFwte4fOAfoXX/VHBnqZLFizhZKPCtG0gstQXTwdvOkx+8p5yPODVQvkqeOJCxF7EAI5B9VgcoJWdh2tQX+e07v/DgRtTs+01re0ZmVxmpGpgRWICdaPms9Hh/DwJT7gs19TpPv1qBgzDTN/z12b/6BRsShk/eEGVwslGF3meFA5+saWPmCSJKKK3Pg0btp3LxLkZfWUtdnVr7ASeJlevnthOPlSMp0ITs6oznhQEjycWK+nBYNUrK+cguiMepe/t solr-ssh@abernathy
     47# To add users with cores.
     48#
     49# 1. Add server to the firewall rules if it's not already there.
     50# 2. Add core name to solr_cores: with obscured name.
     51#    Advice is one core per index. For a lot of uses multiple indexes can
     52#    go on the same core, but it could cause issues depending on requirements.
     53# 3. Add a username:pass to solr_auth_users: with password.
     54# 4. Add username: "users" role in solr_auth_users_roles.
     55#
     56# 5. On the server a core will have been created with the 'example' config.
     57#    /var/solr/data/core_name/conf
     58#    Replace this with the configuration required:
     59#     - for Drupal 7 search_api_solr there is a copy of the config in /root
     60#     - for Drupal 8 search_api_solr it require the site admin to export the configuration from their site administration interface; this can change if the schema or languages change. After configuring the core there is a tab to download it.
     61#       https://git.drupalcode.org/project/search_api_solr/blob/8.x-3.x/INSTALL.md
     62#     - for WP?
     63#   When a configuration has been replaced reload the core, can be done from admin ui.
    6864}}}
    6965
    70 to the new user's ''~/.ssh/authorized_keys'' file. abernathy will connect to $SERVERHOST with these credentials and provide an ssh tunnel to the Apache Solr server.
    71 
    72 === On Abernathy  ===
    73 
    74 There are two things to do on abernathy.mayfirst.org: add a ssh tunnel from abernathy to the web server and create a new Apache Solr instance in Apache Tomcat.
    75 
    76 ==== Create SSH tunnel ====
    77 
    78 Log as solr-ssh into $SERVERHOST from solr-ssh on abernathy.mayfirst.org
    79 {{{
    80 sudo -u solr-ssh ssh solr-ssh@$SERVERHOST
    81 }}}
    82 End the connection with {{{exit}}}
    83 
    84 On abernathy, add a SSH tunnel by using {{{solr_addssh}}} (see {{{solr_addssh --man}}} for help and more options)
    85 {{{
    86 solr_addssh $SERVERNAME $SERVERHOST
    87 }}}
    88 
    89 You can list port forwardings with {{{solr_listssh}}} and delete (stop/disable/remove) port forwarding with {{{solr_delssh $SERVERNAME}}} (add {{{--man}}} for help and more options).
    90 
    91 ==== Test SSH tunnel ====
    92 
    93 Log as solr-ssh into $SERVERHOST and try
    94 {{{
    95 curl http://localhost:9080
    96 }}}
    97 to see the Apache Tomcat welcome page from abernathy.mayfirst.org.
    98 
    99 ==== Create new Solr configuration ====
    100 
    101 Use {{{solr_addcore}}} to create a new Solr instance
    102 
    103 {{{solr_addcore $CORENAME}}}
    104 
    105 The script will output the Solr admin path ($COREPATH) for the new Solr instance.
    106 
    107 You can list instances with {{{solr_listcore}}} and delete an instance with {{{solr_delcore $CORENAME}}} (add {{{--man}}} for help and more options).
    108 
    109 ==== Restart Apache Tomcat ====
    110 
    111 Restart Apache Tomcat with
    112 {{{
    113 service tomcat6 restart
    114 }}}
    115 
    116 ==== Test new Solr core ====
    117 
    118 Check that the new core is accessible with:
    119 {{{
    120 curl http://localhost:8080/solr/$COREPATH/admin/.
    121 }}}
    122 
    123 === Troubleshooting ===
    124 
    125 Make sure that the ssh tunnel is ''up''
    126 {{{
    127 sv status solr-ssh-$SERVERNAME
    128 }}}
    129 
    130 If the tunnel is ''down'' you should have a look at the logs:
    131 {{{tail -f /var/log/supervise/solr-ssh-$SERVERNAME/current}}}
    132 
    133 For more information add ''-v'' to the ssh command in {{{/etc/sv/solr-ssh-$SERVERNAME/run}}} and check the logs again.
    134 
    135 Check the Solr cores with
    136 {{{
    137 solr_checkcore
    138 }}}
    139 (add {{{--man}}} for help and more options)