/*! \page ingest_page Creating Ingest Modules

\section ingest_overview Overview

Autopsy provides ingest framework in org.sleuthkit.autopsy.ingest.

Ingest modules (also referred as ingest services) are designed to be pluggable and they can be added to the Autopsy ingest pipeline
as jar files and they will be automatically recognized next time Autopsy starts.

This document outlines steps necessary to implement a functional ingest module.

\subsection ingest_interface Interfaces

Implement one of the interfaces:

- org.sleuthkit.autopsy.ingest.IngestServiceImage (for modules that are interested in the image as a whole, or modules that selectively pick and analyze data from the image) 
- org.sleuthkit.autopsy.ingest.IngestServiceAbstractFile (for modules that should process every file in the image).

Another difference between the two is that org.sleuthkit.autopsy.ingest.IngestServiceImage services run each in their own threads, and can run in parallel to other image services.
File services execute within the same worker thread and they run in series, every service is called for every file.

For this reason, org.sleuthkit.autopsy.ingest.IngestServiceAbstractFile services are singleton instances
and org.sleuthkit.autopsy.ingest.IngestServiceAbstractFile are not singletons -- there could be multiple instances of 
an image based service, because multiple images can be analyzed at the same time by multiple instances of the same image service class
(NOTE: this design might change in the future to limit number of image services executing at the same time and to introduce a better service dependency system).

org.sleuthkit.autopsy.ingest.IngestServiceAbstract defines common methods for both types of services, please refer to API documentation for details on the required methods.

\subsection ingest_interface Implementation details.

org.sleuthkit.autopsy.ingest.example package for sample service skeleton code.

There is a static getDefault() method that are not part of the interface, that every module (whether an image or a file service)
needs to implement to return the registered static instance of the service.
Refer to example code in org.sleuthkit.autopsy.ingest.example.ExampleAbstractFileIngestService.getDefault()

In case of file ingest service, there needs to be a private constructor to ensure only and only (singleton) instance.
Ensure the default public file service constructor is overridden with the private one.

In case of the image ingest service, the constructor should be public.

The most important method is the process() method, which takes either a file or an image as an argument (depending on the type of the service).
A service either performs work in the process method, 
or enqueues the work for later processing (more common if the service manages its own internal threads).

init() method is invoked on a service (by ingest manager) every time ingest starts, and the service is not already running.
A service should support multiple invocations of init() throughout the application life-cycle.

complete() method is invoked on a service when ingest completed.  The service should perform any resource (file, handles, caches)
cleanup here.

stop() method is invoked on a service when ingest is interrupted (by the user or by the system).
The method is similar to complete(), in that the service should perform any cleanup work.
The difference is if there is pending data to be processed or pending results to be reported by the service; 
the results should be rejected and ignored if stop() is invoked and the service should terminate as early as possible.

Services should post inbox messages to the user when stop() or complete() is invoked (refer to the examples).

Every service should support multiple init() - process() - complete(), or init() - process() - stop() invocations.
The services should also support multiple init() - complete() and init() - stop() invocations, 
which can occur if ingest is started but no work is enqueued for the particular service.


Module developers are encouraged to use the standard java.util.logging.Logger infrastructure to log errors to the Autopsy log.


\subsection ingest_registration Service Registration

Ingest service class / module should register itself using Netbeans Lookup infrastructure 
in layer.xml file in the same package where the ingest module is located.

Example image ingest service registration:

<file name="org-sleuthkit-autopsy-ingest-example-ExampleImageIngestService.instance">
<attr name="instanceOf" stringvalue="org.sleuthkit.autopsy.ingest.IngestServiceImage"/>
<attr name="instanceCreate" methodvalue="org.sleuthkit.autopsy.ingest.example.ExampleImageIngestService.getDefault"/>
<attr name="position" intvalue="1000"/>
</file>
      
File image ingest service registration:
  
<file name="org-sleuthkit-autopsy-ingest-example-ExampleAbstractFileIngestService.instance">
<attr name="instanceOf" stringvalue="org.sleuthkit.autopsy.ingest.IngestServiceAbstractFile"/>
<attr name="instanceCreate" methodvalue="org.sleuthkit.autopsy.ingest.example.ExampleAbstractFileIngestService.getDefault"/>
<attr name="position" intvalue="1100"/>
</file>

Note the "position" attribute.  The attribute determines the ordering of the module in the ingest pipeline.
Services with lower position attribute will execute first.
Use high numbers (higher than 1000) or non-core services.
If your module depends on results from another module, use a higher position attribute to enforce the dependency.
Note: we plan to implement a more flexible and robust module dependency system in future versions of the Autopsy ingest framework.

\subsection ingest_configuration Service Configuration

Ingest modules typically require configuration before they are executed.
There are 2 levels of configuration: simple and advanced.


Module configuration is decentralized and module-specific; every module maintains its
own configuration state and is responsible for implementing its own JPanels to render
and present the configuration to the user. 

The configuration methods are defined in the ingest modules interfaces.
For example, to implement simple configuration, module should return true in:

org.sleuthkit.autopsy.ingest.IngestServiceAbstract.hasSimpleConfiguration()

org.sleuthkit.autopsy.ingest.IngestServiceAbstract.getSimpleConfiguration()
should then return javax.swing.JPanel instance.
 
There are method hooks defined in the ingest service interface that are used to hint the module when the configuration should be saved internally by the module.
For example, to save the simple configuration state, the module should implement
org.sleuthkit.autopsy.ingest.IngestServiceAbstract.saveSimpleConfiguration()


\subsection ingest_events Sending Service Events

Service should notify listeners of new data available periodically by invoking org.sleuthkit.autopsy.ingest.IngestManagerProxy.fireServiceDataEvent() method.
The method accepts org.sleuthkit.autopsy.ingest.ServiceDataEvent parameter.  
The artifacts passed in a single event should be of the same type, 
which is enforced by the org.sleuthkit.autopsy.ingest.ServiceDataEvent constructor.

\subsection ingest_events Data Posting Intervals

The timing as to when a service posts results data is module-implementation-specific.  
In a simple case, service may post new data as soon as the data is available 
- the case for simple services that take a relatively short amount of time to execute and new data is expected 
to arrive in the order of seconds.

Another possibility is to post data in fixed time-intervals (e.g. for a service that takes minutes to produce results
and for a service that maintains internal threads to perform work).
There exist a global update setting that specifies maximum time interval for the service to post data.  
User may adjust the interval for more frequent, real-time updates.  Services that post data in periodic intervals should post their data according to this setting.  
The setting is retrieved by the module using getUpdateFrequency() method in org.sleuthkit.autopsy.ingest.IngestManagerProxy class.


\subsection ingest_messages Inbox messages
In addition to data events, ingest services should send ingest messages about interesting events.
Examples of such events include service status (started, stopped) or information about new data.
The messages include the source service, message subject, message details, unique message id (in the context of the originating service) and a uniqueness attribute, used to group similar messages together and to determine the overall importance priority) of the message.
A message group with a higher number of aggregate messages with the same uniqueness is considered a lower priority.

Ingest messages have different types: there are info messages, warning messages, error messages and data messages.
The data messages contain encapsulated blackboard artifacts and attributes. The passed in data is used by the ingest inbox GUI widget to navigate to the artifact view in the directory tree, if requested by the user.

Ingest message API is defined in org.sleuthkit.autopsy.ingest.IngestMessage class.  The class also contains factory methods to create new messages.
Messages are posted using org.sleuthkit.autopsy.ingest.IngestManagerProxy postMessage() method, which accepts a message created using of the factory methods.

The recipient of the ingest messages is the Ingest Inbox viewer widget component, from the org.sleuthkit.autopsy.ingest.IngestManager package.


*/