/*! \page install_solr_page Install and Configure Solr
[TOC]
\section install_solr_overview Overview
Autopsy uses Apache Solr to store keyword text indexes. A central server is needed in a multi-user cluster to maintain and search the indexes.
A new text index is created for each case. The index can be stored either on shared storage or on the local drive of the Solr server(s) (large amount of local storage is required).
Solr's embedded ZooKeeper is also used as a coordination service for Autopsy.
\section install_solr_prereq Prerequisites
We have prepared a pre-packaged version of Solr which packages Solr as Windows service.
You will need:
• A 64-bit version of the Java 8 Runtime Environment (JRE) from https://github.com/ojdkbuild/ojdkbuild. (Download links)
• Pre-packaged Autopsy version of Solr (LINK TO SOURCEFORGE HERE)
• A network-accessible machine to install Solr on. Note that the Solr process will need to write data out to the main shared storage drive, and needs adequate permissions to write to this location, which may be across a network.
\section install_solr_install Installation
\subsection install_solr_jre JRE Installation
Solr requires a Java Runtime Environment (JRE), which may already be installed. You can test this by running "where java" from the command line. If you see output similar to the results below, you have a JRE.
\image html wherejava.PNG
If you need the JRE, use the link in the \ref install_solr_prereq section above to download an installer. Accept the default settings during installation.
\subsection install_solr_config Solr Configuration
Follow these steps to configure Solr:
- Extract the solr-8.6.3.zip archive from the location given in the \ref install_solr_prereq section into a directory of your choice. The rest of this document assumes that the archive is extracted into C:\solr-8.6.3 directory.
- Go to C:\solr-8.6.3\bin directory and open the solr.in.cmd file in a text editor.
\image html solr_config_folder.png
- Search for each "TODO" and specify a valid path for each of the required configuration parameters. These parameters will be described in detail \ref install_solr_params "below".
\image html solr_config_todo.png
\image html solr_config_param.png
\subsubsection install_solr_params Solr Configuration Parameters
Required Solr Configuration Parameters:
- JAVA_HOME – path to 64-bit JRE installation. For example "JAVA_HOME=C:\Program Files\Java\jre1.8.0_151" or "JAVA_HOME=C:\Program Files\ojdkbuild\java-1.8.0-openjdk-1.8.0.222-1"
- DEFAULT_CONFDIR – path to Autopsy configuration directory. If the Solr archive was extracted into C:\solr-8.6.3 directory, then this path will be “C:\ solr-8.6.3\server\solr\configsets\AutopsyConfig\conf”.
- Dbootstrap_confdir – same path as DEFAULT_CONFDIR
- SOLR_JAVA_MEM - Solr JVM heap size should be somewhere between one third and one half of the total RAM available on the machine. A rule of thumb would be to set the heap size to "set SOLR_JAVA_MEM=-Xms2G -Xmx14G" for a machine with 32GB of RAM or more, and "set SOLR_JAVA_MEM=-Xms2G -Xmx8G" for a machine with 16GB of RAM.
- SOLR_DATA_HOME – location where Solr indexes will be stored. If this is not configured, the indexes will be stored in the "C:\solr-8.6.3\server\solr" directory. NOTE: for Autopsy cases consisting of large number of data sources, Solr indexes can get very large (hundreds of GBs, or TBs) so they should probably be stored on a larger network share.
Optional Solr Configuration Parameters:
- SOLR_HOST – by default, the Solr node name is “localhost”. If multiple Solr nodes are going to be used as part of Solr Cloud, then specify the current computer’s host name as SOLR_HOST variable.
\subsubsection install_sorl_index_file_loc Solr Text Index File Location
Important note: previous versions of Autopsy (Autopsy 4.17.0 and earlier) were storing the Solr text indexes in the case output directory. As a result, the Solr indexes would get deleted if a user deleted the case output directory. Solr 8 (i.e. Autpsy 4.18.0 and later) no longer stores the Solr text index files in the case output directory but instead stores them in location defined by the SOLR_DATA_HOME parameter. As a consequence, if a user choses to manually delete case output directories (for example, to free up disk space), the Solr index directories located in SOLR_DATA_HOME need to be manually deleted as well.
Text index for an Autopsy case will follow a naming structure according to following rules: [Autopsy case name] [Case creation time stamp] [Text index creation time stamp] [shardX_replica_nY]. For example, the text index for an Autopsy case "Test Case" will be located in the following directory inside SOLR_DATA_HOME:
\image html solr_config_case.png
\subsection install_solr_service Windows Service Installation
At this point Solr has been configured and ready to be installed as Windows service. Open a command line console as Administrator and navigate to C:\solr-8.6.3\bin directory. From there, run the following command: "nssm install Solr_8.6.3".
\image html solr_install_1.png
An NSSM UI window will appear. Click the “Path” navigation button:
\image html solr_install_2.png
And select the “C:\solr-8.6.3\bin\solr.cmd” file. NOTE: Make sure you don’t select the “solr.in.cmd” file by accident.
In the “Arguments” parameter, type in “start –f –c”:
\image html solr_install_3.png
Optionally, configure service’s display name, startup type, and account info:
\image html solr_install_4.png
\subsection install_solr_service_user Configure Service User
Back in Pick Your User Accounts (TODO FIX), you should have decided what user to run Solr as. To configure Solr to run as that user, you'll use Windows Service Manager.
Switch to the Log On tab to change the logon credentials to the chosen user who will have access to the shared storage.
- If you specify a domain account, the account name will be in the form of DOMAINNAME\username as shown in the example below
\image html solr_user_1.png
Click “Install Service”. You should see the following UI window appear:
\image html solr_user_2.png
\section install_solr_start Start Solr Service
At this point the Solr service has been configured and installed. You can verify this by opening Windows “Services” window:
\image html solr_start_1.png
Start the “Solr_8.6.3” service, and verify that the service status changes to “Running”.
\image html solr_start_2.png
\section install_solr_testing
There are two tests that you should perform to confirm that the Solr machine is configured correctly.
- Web Interface: You should attempt to access the Solr admin panel in a web browser. On the Solr machine, navigate to http://localhost:8983/solr/#/ and verify that Solr admin console gets displayed. You should also attempt to access the Solr admin panel in a web browser from another machine on the network. Replace the IP address in the following URL with the IP address or the host name that the Solr service is running on
\image html solr_testing_1.png
If the service is appropriately started but you are unable to see the screenshot above, then it could be that port 8983 for Solr and port 9983 for ZooKeeper are blocked by your firewall. Contact your network administrator to open these ports.
- Shared Storage: Log into the Solr computer as the user you decided to run the Solr service as and attempt to access the shared storage paths. Ensure that you can access the UNC paths (or drive letters if you have hardware NAS). If everything is configured correctly you should be able to access the storage paths without having to provide credentials. If you are prompted for a password to access the shared storage, then either enter the password and choose to save the credentials or reconfigure the setup so that the same passwords are used, etc. See Storing Credentials (TODO) for steps on storing credentials. If you needed to store the credentials, then you should restart the service or reboot the computer (we have observed that a running service does not get the updated credentials).
\section install_solr_autopsy
Once the infrastructure is in place, you can configure Autopsy clients to use them.
- Install Autopsy on each client system. Use the normal installer and pick the defaults.
- Test that the user has access to the shared storage by opening the shared storage folders using Windows Explorer. If a password prompt is given, then enter the password and store the credentials (see Storing Credentials).
- Start Autopsy and open the multi-user settings panel from "Tools", "Options", "Multi-user". As shown in the screenshot below, you can then enter all of the address and authentication information for the network-based services. Note that in order to create or open Multi-user cases, "Enable Multi-user cases" must be checked and the settings below must be correct.
- To use a Solr 8 server, configure the Solr 8 Service and the ZooKeeper service connection info. ZooKeeper connection info is required. The ZooKeeper port number is 1000 higher than Solr service port number. By default, Solr service port is 8983 so the embedded ZooKeeper port is 9983.
\image html solr_autopsy.png
- For each setting, press the "Test Connection" button to ensure that Autopsy can communicate with each service. If any fail, then refer to the specific setup page for testing options. Also check that a firewall is not blocking the communications.
- NOTE: None of these tests are for permissions on the shared storage because Autopsy does not know about the shared storage. It can't test that until you make a case.
- Make a test case (see Creating Multi-user cases (TODO)). You can add a single file in as a logical data source. The key concept is to look for errors.
- If you find errors, look for errors in the log file on the Autopsy client.
- If you followed all of the previous steps in all of the previous pages, then a common error at this point is that Solr cannot access the shared storage and it is running as a Service account. When this happens, you'll see an error message about Solr not being able to create or access a "core". If this happens, review what user Solr should be running as (see Solr Service) and change the shared storage configuration or ensure that credentials are stored.
*/