From a368694c18c7df281f6fa1898663ca91c69bdf4a Mon Sep 17 00:00:00 2001 From: apriestman Date: Mon, 9 Aug 2021 15:13:41 -0400 Subject: [PATCH] Update python tutorials --- docs/doxygen/modDSIngestTutorial.dox | 39 ++++++------ docs/doxygen/modFileIngestTutorial.dox | 76 ++++++++++-------------- docs/doxygen/modReportModuleTutorial.dox | 2 +- 3 files changed, 54 insertions(+), 63 deletions(-) diff --git a/docs/doxygen/modDSIngestTutorial.dox b/docs/doxygen/modDSIngestTutorial.dox index c187f7ab33..ed8f128c13 100644 --- a/docs/doxygen/modDSIngestTutorial.dox +++ b/docs/doxygen/modDSIngestTutorial.dox @@ -76,29 +76,28 @@ With our connection in hand, we can do some queries. In our sample database, we stmt = dbConn.createStatement() resultSet = stmt.executeQuery("SELECT * FROM contacts")\endverbatim -For each row, we are going to get the values for the name, e-mail, and phone number and make a TSK_CONTACT artifact. Recall from the first tutorial that posting artifacts to the blackboard allows modules to communicate with each other and also allows you to easily display data to the user. The TSK_CONTACT artifact is for storing contact information. +For each row, we are going to get the values for the name, e-mail, and phone number and make a TSK_CONTACT artifact. Recall from the first tutorial that posting artifacts to the blackboard allows modules to communicate with each other and also allows you to easily display data to the user. The TSK_CONTACT artifact is for storing contact information. The artifact catalog shows that TSK_CONTACT is a data artifact, so we will be using the newDataArtifact() method to create each one. The basic approach in our example is to make an artifact of a given type (TSK_CONTACT) and have it be associated with the database it came from. We then make attributes for the name, email, and phone. The following code does this for each row in the database: \verbatim while resultSet.next(): - - # Make an artifact on the blackboard and give it attributes - art = file.newArtifact(BlackboardArtifact.ARTIFACT_TYPE.TSK_CONTACT) - - name = resultSet.getString("name") - art.addAttribute(BlackboardAttribute( - BlackboardAttribute.ATTRIBUTE_TYPE.TSK_NAME_PERSON.getTypeID(), - ContactsDbIngestModuleFactory.moduleName, name)) - - email = resultSet.getString("email") - art.addAttribute(BlackboardAttribute( - BlackboardAttribute.ATTRIBUTE_TYPE.TSK_EMAIL.getTypeID(), - ContactsDbIngestModuleFactory.moduleName, email)) - - phone = resultSet.getString("phone") - art.addAttribute(BlackboardAttribute( - BlackboardAttribute.ATTRIBUTE_TYPE.TSK_PHONE_NUMBER.getTypeID(), - ContactsDbIngestModuleFactory.moduleName, phone))\endverbatim + try: + name = resultSet.getString("name") + email = resultSet.getString("email") + phone = resultSet.getString("phone") + except SQLException as e: + self.log(Level.INFO, "Error getting values from contacts table (" + e.getMessage() + ")") + + + # Make an artifact on the blackboard, TSK_CONTACT and give it attributes for each of the fields + art = file.newDataArtifact(BlackboardArtifact.Type.TSK_CONTACT, Arrays.asList( + BlackboardAttribute(BlackboardAttribute.Type.TSK_NAME_PERSON, + ContactsDbIngestModuleFactory.moduleName, name), + BlackboardAttribute(BlackboardAttribute.Type.TSK_EMAIL, + ContactsDbIngestModuleFactory.moduleName, email), + BlackboardAttribute(BlackboardAttribute.Type.TSK_PHONE_NUMBER, + ContactsDbIngestModuleFactory.moduleName, phone) + ))\endverbatim That's it. We've just found the databases, queried them, and made artifacts for the user to see. There are some final things though. First, we should fire off an event so that the UI updates and refreshes with the new artifacts. We can fire just one event after each database is parsed (or you could fire one for each artifact - it's up to you). @@ -113,6 +112,8 @@ stmt.close() dbConn.close() os.remove(lclDbPath)\endverbatim +The final version of findContactsDb.py can be found on github. + \subsection python_tutorial2_niceties Niceties Data source-level ingest modules can run for quite some time. Therefore, data source-level ingest modules should do some additional things that file-level ingest modules do not need to. diff --git a/docs/doxygen/modFileIngestTutorial.dox b/docs/doxygen/modFileIngestTutorial.dox index 8079718137..dadc31305b 100644 --- a/docs/doxygen/modFileIngestTutorial.dox +++ b/docs/doxygen/modFileIngestTutorial.dox @@ -75,75 +75,65 @@ The process() method is passed in a reference to an AbstractFile Object. With th Now that we have found the files, we want to do something with them. In our situation, we just want to alert the user to them. We do this by making an "Interesting Item" blackboard artifact. The Blackboard is where ingest modules can communicate with each other and with the Autopsy GUI. The blackboard has a set of artifacts on it and each artifact:

-For our example, we are going to make an artifact of type "TSK_INTERESTING_FILE" whenever we find a big and round file. These are one of the most generic artifact types and are simply a way of alerting the user that a file is interesting for some reason. Once you make the artifact, it will be shown in the UI. The below code makes an artifact for the file and puts it into the set of "Big and Round Files". You can create whatever set names you want. The Autopsy GUI organizes Interesting Files by their set name. +A list of standard artifact types can be found in the artifact catalog. It is important to note the catagory for the artifact you want to since this affects which method you will use to create the artifact. + +For our example, we are going to make an artifact of type "TSK_INTERESTING_FILE", which is an analysis result, whenever we find a big and round file. These are one of the most generic artifact types and are simply a way of alerting the user that a file is interesting for some reason. Once you make the artifact, it will be shown in the UI. The below code makes an artifact for the file and puts it into the set of "Big and Round Files". You can create whatever set names you want. The Autopsy GUI organizes Interesting Files by their set name. \verbatim - art = file.newArtifact(BlackboardArtifact.ARTIFACT_TYPE.TSK_INTERESTING_FILE_HIT) - att = BlackboardAttribute(BlackboardAttribute.ATTRIBUTE_TYPE.TSK_SET_NAME.getTypeID(), - FindBigRoundFilesIngestModuleFactory.moduleName, "Big and Round Files") - art.addAttribute(att)\endverbatim + art = file.newAnalysisResult(BlackboardArtifact.Type.TSK_INTERESTING_FILE_HIT, Score.SCORE_LIKELY_NOTABLE, + None, "Big and Round Files", None, + Arrays.asList( + BlackboardAttribute(BlackboardAttribute.Type.TSK_SET_NAME, + FindBigRoundFilesIngestModuleFactory.moduleName, + "Big and Round Files"))).getAnalysisResult()\endverbatim -The above code adds the artifact and a single attribute to the blackboard in the embedded database, but it does not notify other modules or the UI. The UI will eventually refresh, but it is faster to fire an event with this: +The above code adds the artifact and a single attribute to the blackboard in the embedded database, but it does not notify other modules or the UI. Calling postArtifact() will let the tree viewer and other parts of the UI know that a refresh may be necessary, and passes the newly created artifacts to other modules that may do further processing on it. \verbatim - IngestServices.getInstance().fireModuleDataEvent( - ModuleDataEvent(FindBigRoundFilesIngestModuleFactory.moduleName, - BlackboardArtifact.ARTIFACT_TYPE.TSK_INTERESTING_FILE_HIT, None))\endverbatim + blackboard.postArtifact(art, FindBigRoundFilesIngestModuleFactory.moduleName)\endverbatim That's it. Your process() method should look something like this: \verbatim def process(self, file): + # Use blackboard class to index blackboard artifacts for keyword search + blackboard = Case.getCurrentCase().getSleuthkitCase().getBlackboard() + # Skip non-files - - if ((file.getType() == TskData.TSK_DB_FILES_TYPE_ENUM.UNALLOC_BLOCKS) or - - (file.getType() == TskData.TSK_DB_FILES_TYPE_ENUM.UNUSED_BLOCKS) or - + if ((file.getType() == TskData.TSK_DB_FILES_TYPE_ENUM.UNALLOC_BLOCKS) or + (file.getType() == TskData.TSK_DB_FILES_TYPE_ENUM.UNUSED_BLOCKS) or (file.isFile() == False)): - return IngestModule.ProcessResult.OK - - - # Look for files bigger than 10MB that are a multiple of 4096 - - if ((file.getSize() > 10485760) and ((file.getSize() % 4096) == 0)): - - + # Look for files bigger than 10MB that are a multiple of 4096 + if ((file.getSize() > 10485760) and ((file.getSize() % 4096) == 0)): # Make an artifact on the blackboard. TSK_INTERESTING_FILE_HIT is a generic type of - # artifact. Refer to the developer docs for other examples. + art = file.newAnalysisResult(BlackboardArtifact.Type.TSK_INTERESTING_FILE_HIT, Score.SCORE_LIKELY_NOTABLE, + None, "Big and Round Files", None, + Arrays.asList( + BlackboardAttribute(BlackboardAttribute.Type.TSK_SET_NAME, + FindBigRoundFilesIngestModuleFactory.moduleName, + "Big and Round Files"))).getAnalysisResult() - art = file.newArtifact(BlackboardArtifact.ARTIFACT_TYPE.TSK_INTERESTING_FILE_HIT) + try: + # post the artifact for listeners of artifact events + blackboard.postArtifact(art, FindBigRoundFilesIngestModuleFactory.moduleName) + except Blackboard.BlackboardException as e: + self.log(Level.SEVERE, "Error indexing artifact " + art.getDisplayName()) - att = BlackboardAttribute(BlackboardAttribute.ATTRIBUTE_TYPE.TSK_SET_NAME.getTypeID(), - - FindBigRoundFilesIngestModuleFactory.moduleName, "Big and Round Files") - - art.addAttribute(att) - - - - # Fire an event to notify the UI and others that there is a new artifact - - IngestServices.getInstance().fireModuleDataEvent( - - ModuleDataEvent(FindBigRoundFilesIngestModuleFactory.moduleName, - - BlackboardArtifact.ARTIFACT_TYPE.TSK_INTERESTING_FILE_HIT, None)) - - - - return IngestModule.ProcessResult.OK\endverbatim + return IngestModule.ProcessResult.OK\endverbatim Save this file and run the module on some of your data. If you have any big and round files, you should see an entry under the "Interesting Items" node in the tree. \image html bigAndRoundFiles.png +The full big and round file module along with test data can be found on github. + \subsection python_tutorial1_debug Debugging and Development Tips Whenever you have syntax errors or other errors in your script, you will get some form of dialog from Autopsy when you try to run ingest modules. If that happens, fix the problem and run ingest modules again. You don't need to restart Autopsy each time! diff --git a/docs/doxygen/modReportModuleTutorial.dox b/docs/doxygen/modReportModuleTutorial.dox index ed2b184cf0..d620076df5 100644 --- a/docs/doxygen/modReportModuleTutorial.dox +++ b/docs/doxygen/modReportModuleTutorial.dox @@ -45,7 +45,7 @@ A third approach is to call org.sleuthkit.autopsy.casemodule.Case.getDataSources \subsubsection python_tutorial3_getting_artifacts Getting Blackboard Artifacts -The blackboard is where modules store their analysis results. If you want to include them in your report, then there are several methods that you could use. If you want all artifacts of a given type, then you can use SleuthkitCase.getBlackboardArtifacts(). There are many variations of this method that take different arguments. Look at them to find the one that is most convenient for you. +The blackboard is where modules store their analysis results. If you want to include them in your report, then there are several methods that you could use. If you want all artifacts of a given type, then you can use getDataArtifacts()or Blackboard.getAnalysisResultsByType(). There are variations of these methods that take different arguments. Look at them to find the one that is most convenient for you. \subsubsection python_tutorial3_getting_tags Getting Tagged Files or Artifacts