mstoeck3/autopsy-flatpak
1
0
Fork 0
mirror of https://github.com/overcuriousity/autopsy-flatpak.git synced 2025-07-06 21:00:22 +00:00
Code Issues Packages Projects Releases Wiki Activity

Updated docs to reflect Solr can be local and impact on Single User Cases

Browse Source
This commit is contained in:
Brian Carrier 2021-01-22 07:57:48 -05:00
parent 5649231204
commit aa672a228a
6 changed files with 57 additions and 34 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/doxygen-user/main.dox
View File

@ -13,6 +13,8 @@ Help Topics
The following topics are available here:
- \subpage installation_page
- Notable Upgrades
- \subpage upgrade_solr8_page
- Configuration
- \subpage config_page
- \subpage performance_page

2
docs/doxygen-user/multi-user/installSharedStorage.dox
View File

@ -4,7 +4,7 @@
The cluster will need shared storage that can be accessed from:
- Autopsy clients
- Solr server
- Solr server (depending on configuration)
This shared storage will be used for both data sources and case outputs, so you will need lots of space.

28
docs/doxygen-user/multi-user/installSolr.dox
View File

@ -9,19 +9,21 @@ A new text index is created for each case. The index can be stored either on sha
Solr's embedded ZooKeeper is also used as a coordination service for Autopsy.
If you have already installed Solr 4 with a previous version of Autopsy, please see the \ref upgrade_solr_page page for information on how open older cases after the upgrade and migrate data.
If you have already installed Solr 4 with a previous version of Autopsy, please see the \ref upgrade_solr8_page page for information on how open older cases after the upgrade and migrate data.
NOTE: This document assumes you will be running Solr on Windows as a service. You can run it as a non-service or on another platform, but you'll need to understand the steps in this document to make that happen.
\section install_solr_prereq Prerequisites
We have prepared a pre-packaged version of Solr which packages Solr as Windows service.
You will need:
<ul>
<li>A 64-bit version of the Java 8 Runtime Environment (JRE) from <a href="https://github.com/ojdkbuild/ojdkbuild">https://github.com/ojdkbuild/ojdkbuild</a>. (<a href="https://github.com/ojdkbuild/ojdkbuild/blob/master/README.md">Download links</a>)
<li>Pre-packaged Autopsy version of Solr from https://sourceforge.net/projects/autopsy/files/CollaborativeServices/Solr/bitnami-solr-4.10.3-0-windows-installer.exe/download
<li>A network-accessible machine to install Solr on. Note that the Solr process will need to write data out to the main shared storage drive, and needs adequate permissions to write to this location, which may be across a network.
<li>A 64-bit version of the Java 8 Runtime Environment (JRE) from <a href="https://github.com/ojdkbuild/ojdkbuild">https://github.com/ojdkbuild/ojdkbuild</a>. (<a href="https://github.com/ojdkbuild/ojdkbuild/blob/master/README.md">Download links</a>)
<li>Pre-packaged Autopsy version of Solr from <a href="https://sourceforge.net/projects/autopsy/files/CollaborativeServices/Solr/SOLR_8.6.3_AutopsyService.zip/download">here</a>. This contains Solr, <a href="https://nssm.cc/">NSSM</a> to make it run as a service, and the needed schema config files.
<li>A network-accessible machine to install Solr on. Note that the Solr process may need to write to a shared storage drive (if that is how you configure it) and will therefore need adequate permissions.
</ul>
\section install_solr_install Installation
\section install_solr_install Solr Installation
\subsection install_solr_jre JRE Installation
@ -70,9 +72,11 @@ Text index for an Autopsy case will follow a naming structure according to follo
\image html solr_config_case.png
\subsection install_solr_service Solr Service Installation
\section install_solr_service Solr Windows Service Installation
At this point Solr has been configured and ready to be installed as Windows service. Open a command line console as Administrator and navigate to the \c "C:\solr-8.6.3\bin" directory. From there, run the following command: \c "nssm install Solr_8.6.3".
At this point Solr has been configured and ready to use. The last step is to configure it as a Windows service so that it starts each time the computer starts.
Open a command line console as Administrator and navigate to the \c "C:\solr-8.6.3\bin" directory. From there, run the following command: \c "nssm install Solr_8.6.3".
\image html solr_install_1.png
@ -101,7 +105,7 @@ Click \c "Install Service". You should see the following UI window appear:
\image html solr_user_2.png
\section install_solr_start Start Solr Service
\subsection install_solr_start Start Solr Service
At this point the Solr service has been configured and installed. You can verify this by opening Windows "Services" window:
@ -125,11 +129,11 @@ If the service is appropriately started but you are unable to see the screenshot
<li><b>Shared Storage:</b> Log in to the Solr computer as the user you decided to run the Solr service as and attempt to access the shared storage paths. Ensure that you can access the UNC paths (or drive letters if you have hardware NAS). If everything is configured correctly you should be able to access the storage paths without having to provide credentials. If you are prompted for a password to access the shared storage, then either enter the password and choose to save the credentials or reconfigure the setup so that the same passwords are used. See the \ref multiuser_users_store section for steps on storing credentials. If you needed to store the credentials, then you should restart the service or reboot the computer (we have observed that a running service does not get the updated credentials).
</ul>
\section install_solr_autopsy Configuring Autopsy
\section install_solr_autopsy Configuring Autopsy Clients
Once the rest of the services are configured you will \ref install_multiuserclient_page "configure Autopsy to enable multi-user cases". For the Solr 8 server, configure the Solr 8 Service and the ZooKeeper service connection info. ZooKeeper connection info is required. The ZooKeeper port number is 1000 higher than Solr service port number. By default, Solr service port is 8983 making the embedded ZooKeeper port 9983. You may also use a \ref install_solr_standalone_zk "standalone ZooKeeper service".
\section install_sorl_adding_nodes Adding More Solr Nodes
\section install_sorl_adding_nodes Adding More Solr Nodes (SolrCloud)
Solr 8 has ability for multiple Solr nodes to work together as a Solr cluster. In this mode (SolrCloud mode) each Solr collection/index is split across all of the available Solr nodes. This is called sharding. For example, if there are 4 Solr nodes in a SolrCloud cluster, then the text index will be split across the 4 Solr nodes, thus greatly reducing the load on each individual Solr server and improving Solr indexing and searching performance.

5
docs/doxygen-user/multi-user/installSystems.dox
View File

@ -14,8 +14,7 @@ You can run each of these on their own dedicated VM, but that is not necessary.
Apache Solr uses a lot of memory, so we recommend keeping it by itself. The exception is if you are using Windows File Sharing for shared storage. You can get better Solr performance if it is writing to local storage instead of over the network. So, you can consider using the same computer for both Solr and shared storage.
Also note that because all computers need to access the shared storage at the same path, you cannot mix operating systems. A Linux system running Solr will not be able to access the shared storage at the same path as a Windows Autopsy client.
Also note that because all computers need to access the shared storage at the same path, you cannot mix operating systems.
We recommend:
@ -44,4 +43,4 @@ You will have lots of important data on the system. Ensure that the following a
- Zookeeper data on the Solr server (see \ref install_solr_backup)
*/
*/

8
docs/doxygen-user/multi-user/installUsers.dox
View File

@ -29,9 +29,13 @@ The user account that Autopsy runs as will need access to the shared storage. Th
\subsection multiuser_users_solr Solr Service
Solr will run as a Windows service and will need access to the shared storage. The default user, which is "LocalService", will not have access to network-based storage.
Solr will run as a Windows service and may need access to shared storage if it does not have enough local storage. Solr performs best when it has fast access to storage, so keeping the indexes on local SSD drives is best. But, some clusters will need to store the indexes on the same shared storage that are used for images and other case outputs.
So, if you have network-based shared storage, you have three options:
NOTE: Autopsy 4.17.0 and prior required that indexes were stored on the shared storage drives. Autopsy 4.18.0 and beyond (which now use Solr 8) can use either local storae or shared storage.
If you are using local storage for Solr, then you can run the Solr service as "LocalService".
If you are going to use network storage for Solr, then you have three options:
- <b>NetworkService</b>: If you are on a domain, you may be able to run Solr as the "NetworkService" account. This account has access to the network, but the challenge can be granting access for this account to the shared storage.
- If your shared storage is a Windows file share, you'll need to grant access to the computer account running Solr as follows:
<ol> <li> Right click on the shared storage folder choose "Properties" and select the "Security" tab.

46
docs/doxygen-user/multi-user/upgradeSolr.dox → docs/doxygen-user/multi-user/upgradeSolr8.dox
View File

@ -1,26 +1,36 @@
/*! \page upgrade_solr_page Upgrading Solr
/*! \page upgrade_solr8_page Upgrading to Autopsy 4.18.0 (with Solr 8)
[TOC]
\section solr_upgrade_solr4 Upgrading from Solr 4 from Solr 8
Autopsy 4.18.0 and beyond will make new cases with Solr 8 instead of Solr 4. Because Solr 8 is not backward compatable with Solr 4, this will have some impact on you. Notably:
To upgrade from Solr 4 to Solr 8, first follow the instructions on the \ref install_solr_page page. At the end of that process, all new cases will be created using Solr 8. If you need to open older cases that were created using Solr 4, please see the following sections which describe how to set up Solr 4 and Solr 8 in parallel and how to migrate Zookeeper data.
- Cases made with 4.17.0 and earlier will continue to use Solr 4 indexes and can be opened with Autopsy 4.18.0+
- Cases made with 4.18.0 and later will use Solr 8 and cannot be opened by Autopsy 4.17.0 and earlier.
The main take away is that you can open older cases, but new cases can't be opened by older software.
\subsection install_solr_solr_4 Legacy (Solr 4) Autopsy Cases and Text Indexes
If you are using single-user cases, there is nothing else for you to do. Autopsy ships with both Solr 4 and 8 embedded.
Things to keep in mind regarding backwards compatibility with existing Solr 4 multi-user cases (cases created with Autopsy versions 4.17.0 and earlier):
If you are using a multi-user cluster, then you'll need to install a new Solr 8 server and may choose to keep Solr 4 around too.
<ul>
<li>New text indexes can only be created using Solr 8 (Autopsy version 4.18.0 and later).
<li>Autopsy is able to open and search existing Solr 4 single-user cases. This functionality comes "out of the box" and nothing needs to be configured or installed to achieve this.
<li>Solr 8 multi-user server is unable to open existing legacy Solr 4 multi-user cases because Solr indexes are only backward compatible one major version back (i.e. Solr 8 is only able to open indexes created using Solr 8 or Solr 7).
</ul>
To overcome this limitation on older multi-user cases, there is an option of configuring Autopsy to be able to connect to both Solr 8 and Solr 4 multi-user servers. If a multi-user Solr 4 server is configured, you will be able to open and search existing Solr 4 cases, as well as add data sources to the existing Solr 4 cases, thus adding more data to the existing legacy Solr 4 text indexes. However, new text indexes can only be created using Solr 8. Note that a Solr 4 server is unable to open text indexes created using a Solr 8 server.
\section solr8_upgrade_cluster Multi-User Cluster Options
\subsection install_solr_parallel Running Solr 8 and Solr 4 Multi-user Servers in Parallel
There are two considerations when ugprading a multi-user cluster:
- Which versions of Solr servers to run
- Where to store ZooKeeper data
It is possible to configure Autopsy to be able to connect to both Solr 8 and Solr 4 multi-user servers. Start Autopsy and open the multi-user settings panel from "Tools", "Options", "Multi-user". Note that in order to create or open Multi-user cases, "Enable Multi-user cases" must be checked and the settings below must be correct.
The following sections cover these topics in more detail.
\subsection solr8_upgrade_cluster_solr Multiple Solr Servers
You will most likely want to have both Solr 4 and 8 servers running so that you can open older cases. If so, then:
- Get a new server and install Solr 8 on it using the \ref install_solr_page page instructions.
- Configure each Autopsy client to have the addresses for both servers (see below)
If you want to have only Solr 8, then simply follow the instructions for Solr 8 and get rid of the Solr 4 server. You will not be able to open older cases. It is possible to upgrade Solr indexes, but we have not tested this.
To configure Autopsy to be able to connect to both Solr 8 and Solr 4 multi-user servers. Start Autopsy and open the multi-user settings panel from "Tools", "Options", "Multi-user". Note that in order to create or open Multi-user cases, "Enable Multi-user cases" must be checked and the settings below must be correct.
It is recommended that you run Solr 8 and Solr 4 servers on separate machines. In the example below, Solr 8 server is running on a machine with host name "Solr1" and Solr 4 server is running on a machine with host name "Solr6".
@ -32,9 +42,13 @@ Once both the Solr 8 and Solr 4 multi-user server connection info is entered and
If you intend to run Solr 4 and Solr 8 servers on the same machine at the same time, you need to change the port of the Solr 8 service using the setting <b>SOLR_PORT</b> in \c "C:\solr-8.6.3\bin\solr.in.cmd" file (assuming that the Solr package ZIP was extracted into \c "C:\solr-8.6.3\" directory). By default the Solr service starts on port 8983.
\subsection install_solr_zk_migration Migrating existing ZooKeeper data to Solr 8 server
\subsection install_solr_zk_migration Migrating Of ZooKeeper Data
If you have an existing Solr 4 server that you have been using in Multi-User (MU) mode and you wish to migrate this data onto your new Solr 8 server, we have developed a migration utility that performs this task. Note that the utility is able to migrate ZooKeeper data from any one ZooKeeper server to another ZooKeeper server, and is not tied to migrating Solr 4 ZooKeeper data to a Sol 8 ZooKeeper server.
In addition to text indexing, the Solr service also stores "coordination" data using Apache ZooKeeper. You'll want to move this data if you get rid of your Solr 4 server. This data allows you to:
- Know what multi-user cases you can open
- Know which disk images were already processed for auto-ingest
You can continue to use your Solr 4 instance of ZooKeeper, but we also have a utility that allows you to migrate the data to a new server, such as the one running Solr 8 (or a stand-alone instance).
In our example we will be migrating ZooKeeper data from a ZooKeeper server running on a Solr 4 server (on machine "Solr6") to a brand new ZooKeeper server running on a Solr 8 server (on machine "Solr1").
@ -73,4 +87,4 @@ To verify that the ZooKeeper data has been copied from the Solr6 server to the S
\image html solr_zk_migration_5.png
*/
*/