Merge branch 'develop' into 2131-refactor-TextExtractors

Core/src/org/sleuthkit/autopsy/casemodule/Bundle.properties

@@ -159,7 +159,7 @@ GeneralFilter.rawImageDesc.text=Raw Images (*.img, *.dd, *.001, *.aa, *.raw, *.b
 GeneralFilter.encaseImageDesc.text=Encase Images (*.e01)
 GeneralFilter.virtualMachineImageDesc.text=Virtual Machines (*.vmdk, *.vhd)
 GeneralFilter.executableDesc.text=Executables (*.exe)
-ImageDSProcessor.dsType.text=Image or VM File
+ImageDSProcessor.dsType.text=Disk Image or VM File
 ImageDSProcessor.allDesc.text=All Supported Types
 ImageFilePanel.moduleErr=Module Error
 ImageFilePanel.moduleErr.msg=A module caused an error listening to ImageFilePanel updates. See log to determine which module. Some data could be incomplete.

ImageGallery/src/org/sleuthkit/autopsy/imagegallery/actions/OpenHelpAction.java

@@ -44,7 +44,7 @@ public final class OpenHelpAction implements ActionListener {
     @Override
     public void actionPerformed(ActionEvent e) {
         try {
-            Desktop.getDesktop().browse(URI.create("http://sleuthkit.org/autopsy/docs/user-docs/4.1/image_gallery_page.html")); //NON-NLS
+            Desktop.getDesktop().browse(URI.create("http://sleuthkit.org/autopsy/docs/user-docs/4.2/image_gallery_page.html")); //NON-NLS
         } catch (IOException ex) {
             Logger.getLogger(OpenHelpAction.class.getName()).log(Level.SEVERE, "failed to open help page", ex); //NON-NLS
         }

NEWS.txt

@@ -1,14 +1,37 @@
+---------------- VERSION 4.3.0  --------------
+Improvements:
+- Creation and analysis (e.g., keyword search) of virtual files for slack 
+space.
+- A preloader in an Android device image does not prevent adding the image as 
+a data source (reading of secondary GPT tables supported).
+- Ability to add data sources with no file systems or unsupported file systems 
+as "unallocated space image files" for carving, keyword search, etc.
+- File extension mismatch analysis can be configured to check all file types, 
+all file types except text files, or only multimedia and executable files.
+- Column order changes in table views are "sticky" for each type of tree view 
+item.
+- Tree view has new file types by MIME type sub tree.
+- Bulk adding list of keywords to a keyword list.
+- Highlighting of tagged items in table views.
+- Toolbar button for Image/Video Gallery
+- New "Experimental" module (activate via Tools, Plugins) with auto ingest 
+feature.
+- Assorted bug fixes and minor enhancements. 
+
+
 ---------------- VERSION 4.2.0  --------------
 Improvements:
 - Credit card account search.
 - Encoding/decoding of extracted files to avoid anti-virus alerts/quarantine.
-- Ingest history (start time, end time, status, which versions of which ingest modules were run). 
+- Ingest history (start time, end time, status, which versions of which ingest 
+modules were run). 
 - Ingest history used to warn before doing redundant analysis. 
 - Options panel for managing custom tag names.
 - Options panel for setting external viewer associations.
 - Keyboard shortcut for applying Bookmark tags.
 - Improved PhotoRec carver ingest module cancellation responsiveness.
-- Results content viewer formats dates.
+- Results content viewer formats dates instead of showing raw seconds since 
+epoch.
 - Update to PostgreSQL 9.5.
 - Assorted bug fixes and minor enhancements. 
 

docs/doxygen-user/data_sources.dox

@@ -1,13 +1,13 @@
 /*! \page ds_page Data Sources
 
 
-A data source the thing you want to analyze. It can be a disk image, some logical files, a local drive, etc. You must open a case prior to adding a data source to Autopsy. 
+A data source is the thing you want to analyze. It can be a disk image, some logical files, a local disk, etc. You must open a case prior to adding a data source to Autopsy. 
 
-Autopsy supports three types of data sources:
-- Disk Image: A file (or set of files) that is a byte-for-byte copy of a hard drive or media card.  (see \ref ds_img)
-- Local Drive: Local storage device (local drive, USB-attached drive, etc.).  (see \ref ds_local)
+Autopsy supports four types of data sources:
+- Disk Image or VM File: A file (or set of files) that is a byte-for-byte copy of a hard drive or media card, or a virtual machine image.  (see \ref ds_img)
+- Local Disk: Local storage device (local drive, USB-attached drive, etc.).  (see \ref ds_local)
 - Logical Files: Local files or folders. (see \ref ds_log)
-
+- Unallocated Space Image Files: Any type of file that does not contain a file system but you want to run through ingest (see \ref ds_unalloc)
 
 
 \section ds_add Adding a Data Source
@@ -52,7 +52,7 @@ Autopsy supports disk images in the following formats:
 
 To add a disk image:
 
--# Choose "Image File" from the pull down.
+-# Choose "Disk Image or VM File" from the pull down.
 -# Browse to the first file in the disk image. You need to specify only the first file and Autopsy will find the rest.  
 -# Choose the timezone that the disk image came from.  This is most important for when adding FAT file systems because it does not store timezone information and Autopsy will not know how to normalize to UTC.
 -# Choose to perform orphan file finding on FAT file systems.  This can be a time intensive process because it will require that Autopsy look at each sector in the device.  
@@ -90,4 +90,13 @@ To add logical files:
 
 All of the files that you added in the panel will be grouped together into a single data source, called "LogicalFileSet" in the main UI. 
 
+\section ds_unalloc Adding an Unallocated Space Image File
+
+\image html unallocated_space_options.PNG
+
+To add unallocated space image files:
+-# Choose "Unallocated Space Image File" from the pull down
+-# Browse to the file
+-# Choose whether to break the image up into chunks. Breaking the image up will give better performance since the chunks can be processed in parallel,  but there is a chance that keywords or carved files that span chunk boundaries will be missed.
+
 */

docs/doxygen-user/extension_mismatch.dox

@@ -21,7 +21,7 @@ Note that you can get a lot of false positives with this module. You can add you
 Ingest Settings
 ------
 
-In the ingest settings, the user can choose if the module should skip files without extensions and skip text files. Both of these options are enabled by default.
+In the ingest settings, the user can choose whether to run on all files, all files except text files, or only multimedia or executable files. Additionally, the user can choose to skip all files without an extension, and to skip any known files identified by the hash lookup module, if it is enabled.
 
 \image html extension-mismatch-detected-ingest-settings.PNG
 

docs/doxygen-user/image_gallery.dox

@@ -12,7 +12,7 @@ Quick Start
 ===========
 1. The Image Gallery tool can be configured to collect data about images/videos as ingest runs or all at once after ingest.  To change this setting go to "Tools", "Options", "Image /Video Gallery".  This setting is saved per case, but cannot be changed during ingest. See the Options window for more details
 2. Create a case as normal and add a disk image (or folder of files) as a data source.  Ensure that you have the hash lookup module enabled with NSRL and known bad hashsets, the EXIF module enabled, and the File Type module enabled. 
-3. Click "Tools", "View Images/Videos" in the menu.  This will open the  Autopsy Image/Video Analysis tool in a new window.
+3. Click the "View Images/Videos" button or select "View Images/Videos" in the "Tools" menu.  This will open the  Autopsy Image/Video Analysis tool in a new window.
 4. Groups of images will be presented as they are analyzed by the background ingest modules.  You can later resort and regroup, but it is required to keep it grouped by folder while ingest is still ongoing. 
 5. As each group is reviewed, the next highest priority group is presented, according to a sorting criteria (the default is the density of hash set hits).
 6. Images that were hits from hashsets, will have a dashed border around them.

docs/doxygen-user/keyword_search.dox

@@ -31,6 +31,13 @@ Autopsy supports importing Encase tab-delimited lists as well as lists created p
 <b>Lists tab</b> \n
 \image html keyword-search-configuration-dialog.PNG
 
+The Lists tab is used to create/import and add content to keyword lists. Once a keyword list is selected, the "New Keywords" button can be used to add one or more entries to the list. 
+
+<br>
+\image html keyword-search-configuration-new-keywords.PNG
+
+New entries can be typed into the dialog or pasted from the clipboard. All entries added at once must be the same type of match (exact, substring, or regex), but the dialog can be used multiple times to add keywords to the keyword list.
+
 <br>
 <b>String extraction setting</b> \n
 The string extraction setting defines how strings are extracted from files from which text cannot be extracted because their file formats are not supported. This is the case with arbitrary binary files (such as the page file) and chunks of unallocated space that represent deleted files.

docs/doxygen-user/tagging.dox

@@ -22,12 +22,19 @@ You can create arbitrary tag names. Autopsy remembers your tag names from previo
 
 \image html tagging-3.PNG
 
-You can apply tags to groups of items at once. Select multiple items in the Blackboard, right click, and add the appropriate tag.
+If you just want to tag the item with the default "Bookmark" tag, you can also use the keyboard shortcut control+B instead of going through the menus. 
+
+You can also apply tags to groups of items at once. Select multiple items in the Blackboard, right click, and add the appropriate tag.
 Items may have more than one tag.
 
-Tagged results are shown in the "Results" portion of the tree under "Tags".
+Tagged results are shown in the "Results" portion of the tree under "Tags". Tagged items are also highlighted in the Results Viewer.
 
 <br>
 \image html tagging-4.PNG
 <br>
+
+The list of tags can be edited through the Tags tab on the Options menu. 
+<br>
+\image html tagging-5.PNG
+<br>
 */

docs/doxygen-user/uilayout.dox

@@ -43,7 +43,7 @@ An example of the single file extraction option is shown below.
 \subsection ui_tree_views Views
 
 Views filter all the files in the case by some external property of the file, not by any internal analysis of the file. 
-- <b>File Type</b>  Sorts files by file extension, and shows them in the appropriate group. For example, .mp3 and .wav both end up in the "Audio" group. 
+- <b>File Type</b>  Sorts files by file extension or MIME type, and shows them in the appropriate group. For example, .mp3 and .wav both end up in the "Audio" group. 
 - <b>Recent Files</b>  Displays files that are accessed within the last seven days the user had the device.
 - <b>Deleted Files</b>  Displays files that have been deleted but the names have been recovered.
 - <b>File Size</b>  Sorts files based upon size. This can give you an idea where to look for files you are interested in.
@@ -55,6 +55,7 @@ Views filter all the files in the case by some external property of the file, no
 - <b>Hashset Hits:</b> Hashset hits show up here 
 - <b>E-Mail Messages:</b> Email messages show up here
 - <b>Interesting Items:</b> Things deemed interesting show up here
+- <b>Accounts:</b> Credit card accounts show up here
 - <b>Tags:</b> Any item you tag shows up here so you can find it again easily
 
 \subsection ui_tree_reports Reports
@@ -73,7 +74,7 @@ The Result Viewer windows are in the upper right area of the interface and displ
 \subsection right_click_functions Right Click Functions
 Viewers in Result Viewers have certain right-click functions built-in into them that can be accessed when a node a certain type is selected (a file, directory or a result).
 Here are some examples that you may see:
-\li Open File in External Viewer: Opens the selected file in an "external" application as defined by the local OS. For example, HTML files may be opened by IE or Firefox, depending on what the local system is configured to use.
+\li Open File in External Viewer: Opens the selected file in an "external" application as defined by the local OS or through the External Viewer tab on the Options menu. For example, HTML files may be opened by IE or Firefox, depending on what the local system is configured to use.
 \li View in New Window: Opens the content in a new internal Content Viewer (instead of in the default location in the lower right).
 \li Extract: Make a local copy of the file or directory for further analysis.
 \li Search for files with the same MD5 Hash: Searches the entire file-system for any files with the same MD5 Hash as the one selected.