Merge pull request #24 from overcuriousity/claude/clarify-timestamp-format-cBGwJ

Claude/clarify timestamp format c b gw j

CLAUDE.md

@@ -145,6 +145,14 @@ The codebase is organized into focused, single-responsibility modules to make it
 
 **Integrity System**: Every note automatically gets:
 1. SHA256 hash of `timestamp:content` (via `Note.calculate_hash()`)
+   - **Timestamp Format**: Unix epoch timestamp as float (seconds since 1970-01-01 00:00:00 UTC)
+   - **Hash Input Format**: `"{timestamp}:{content}"` where timestamp is converted to string using Python's default str() conversion
+   - **Example**: For content "Suspicious process detected" with timestamp 1702345678.123456, the hash input is:
+     ```
+     1702345678.123456:Suspicious process detected
+     ```
+   - This ensures integrity of both WHAT was said (content) and WHEN it was said (timestamp)
+   - The exact float precision is preserved in the hash, making timestamps forensically tamper-evident
 2. Optional GPG clearsign signature (if `pgp_enabled` in settings and GPG available)
 
 **Tag System**: Regex-based hashtag extraction (`#word`)

README.md

@@ -20,7 +20,7 @@ trace "IR team gained shell access. Initial persistence checks running."
 trace "Observed outbound connection to 192.168.1.55 on port 80. #suspicious #network"
 ```
 
-**System Integrity Chain:** Each command-line note is immediately stamped, concatenated with its content, and hashed using SHA256 before storage. This ensures a non-repudiable log entry.
+**System Integrity Chain:** Each command-line note is immediately stamped with a Unix epoch timestamp (seconds since 1970-01-01 00:00:00 UTC as float, e.g., `1702345678.123456`), concatenated with its content in the format `"{timestamp}:{content}"`, and hashed using SHA256 before storage. This ensures a non-repudiable log entry with forensically tamper-evident timestamps.
 
 ## CLI Command Reference
 
@@ -195,7 +195,7 @@ After this, you can log with just: `t "Your note here"`
 
 | Feature | Description | Operational Impact |
 | :--- | :--- | :--- |
-| **Integrity Hashing** | SHA256 applied to every log entry (content + timestamp). | **Guaranteed log integrity.** No modification possible post-entry. |
+| **Integrity Hashing** | SHA256 applied to every log entry using format `"{unix_timestamp}:{content}"`. Timestamp is Unix epoch as float (e.g., `1702345678.123456`). | **Guaranteed log integrity.** No modification possible post-entry. Timestamps are forensically tamper-evident with full float precision. |
 | **GPG Signing** | Optional PGP/GPG signature applied to notes. | **Non-repudiation** for formal evidence handling. |
 | **IOC Extraction** | Automatic parsing of IPv4, FQDNs, URLs, hashes, and email addresses. | **Immediate intelligence gathering** from raw text. |
 | **Tag System** | Supports `#hashtags` for classification and filtering. | **Efficient triage** of large log sets. |
@@ -208,20 +208,33 @@ After this, you can log with just: `t "Your note here"`
 ### Layer 1: Note-Level Integrity (Always Active)
 
 **Process:**
-1. **Timestamp Generation** - Precise Unix timestamp captured at note creation
-2. **Content Hashing** - SHA256 hash computed from `timestamp:content`
+1. **Timestamp Generation** - Precise Unix epoch timestamp (float) captured at note creation
+   - Format: Seconds since 1970-01-01 00:00:00 UTC (e.g., `1702345678.123456`)
+   - Full float precision preserved for forensic tamper-evidence
+2. **Content Hashing** - SHA256 hash computed from `"{timestamp}:{content}"`
 3. **Optional Signature** - Hash is signed with investigator's GPG private key
 
 **Mathematical Representation:**
 ```
-hash = SHA256(timestamp + ":" + content)
+timestamp = Unix epoch time as float (e.g., 1702345678.123456)
+hash_input = "{timestamp}:{content}"
+hash = SHA256(hash_input)
 signature = GPG_Sign(hash, private_key)
 ```
 
+**Example:**
+```
+Content: "Suspicious process detected"
+Timestamp: 1702345678.123456
+Hash input: "1702345678.123456:Suspicious process detected"
+Hash: SHA256 of above = a3f5b2c8d9e1f4a7b6c3d8e2f5a9b4c7d1e6f3a8b5c2d9e4f7a1b8c6d3e0f5a2
+```
+
 **Security Properties:**
 - **Temporal Integrity**: Timestamp is cryptographically bound to content (cannot backdate notes)
 - **Tamper Detection**: Any modification to content or timestamp invalidates the hash
 - **Non-Repudiation**: GPG signature proves who created the note (if signing enabled)
+- **Hash Reproducibility**: Exported markdown includes Unix timestamp for independent verification
 - **Efficient Storage**: Signing only the hash (64 hex chars) instead of full content
 
 ### Layer 2: Export-Level Integrity (On Demand)
@@ -328,6 +341,26 @@ Individual note signatures are embedded in the markdown export. To verify a spec
 - The GPG signature proves who created that hash
 - Together: Proves this specific content was created by this investigator at this time
 
+**Hash Verification (Manual):**
+
+To independently verify a note's hash from the markdown export:
+
+1. Locate the note in the export file and extract:
+   - Unix Timestamp (e.g., `1702345678.123456`)
+   - Content (e.g., `"Suspicious process detected"`)
+   - Claimed Hash (e.g., `a3f5b2c8...`)
+
+2. Recompute the hash:
+   ```bash
+   # Using Python
+   python3 -c "import hashlib; print(hashlib.sha256(b'1702345678.123456:Suspicious process detected').hexdigest())"
+
+   # Using command-line tools
+   echo -n "1702345678.123456:Suspicious process detected" | sha256sum
+   ```
+
+3. Compare the computed hash with the claimed hash - they must match exactly
+
 ### Cryptographic Trust Model
 
 ```

trace/cli.py

@@ -364,9 +364,14 @@ def export_markdown(output_file: str = "export.md"):
         sys.exit(1)
 
 def format_note_for_export(note: Note) -> str:
-    """Format a single note for export (returns string instead of writing to file)"""
+    """Format a single note for export (returns string instead of writing to file)
+
+    Includes Unix timestamp for hash reproducibility - anyone can recompute the hash
+    using the formula: SHA256("{unix_timestamp}:{content}")
+    """
     lines = []
     lines.append(f"- **{time.ctime(note.timestamp)}**\n")
+    lines.append(f"  - Unix Timestamp: `{note.timestamp}` (for hash verification)\n")
     lines.append(f"  - Content:\n")
     # Properly indent multi-line content
     for line in note.content.splitlines():

trace/crypto.py

@@ -43,12 +43,25 @@ class Crypto:
             return False, "Not a GPG signed message"
 
         try:
+            # Force English output for consistent parsing across locales
+            # Linux/macOS: LC_ALL/LANG variables control GPG's output language
+            # Windows: GPG may ignore these, but encoding='utf-8' + errors='replace' provides robustness
+            import os
+            env = os.environ.copy()
+            # Use C.UTF-8 for English messages with UTF-8 encoding support
+            # Falls back gracefully via errors='replace' if locale not available
+            env['LC_ALL'] = 'C.UTF-8'
+            env['LANG'] = 'C.UTF-8'
+
             proc = subprocess.Popen(
                 ['gpg', '--verify'],
                 stdin=subprocess.PIPE,
                 stdout=subprocess.PIPE,
                 stderr=subprocess.PIPE,
-                text=True
+                text=True,
+                encoding='utf-8',
+                errors='replace',  # Handle encoding issues on any platform
+                env=env
             )
             stdout, stderr = proc.communicate(input=signed_content, timeout=10)
 
@@ -171,5 +184,25 @@ class Crypto:
 
     @staticmethod
     def hash_content(content: str, timestamp: float) -> str:
+        """Calculate SHA256 hash of timestamp:content.
+
+        Hash input format: "{timestamp}:{content}"
+        - timestamp: Unix epoch timestamp as float (seconds since 1970-01-01 00:00:00 UTC)
+          Example: 1702345678.123456
+        - The float is converted to string using Python's default str() conversion
+        - Colon (':') separator between timestamp and content
+        - Ensures integrity of both WHAT was said and WHEN it was said
+
+        Args:
+            content: The note content to hash
+            timestamp: Unix epoch timestamp as float
+
+        Returns:
+            SHA256 hash as hexadecimal string (64 characters)
+
+        Example:
+            >>> hash_content("Suspicious process detected", 1702345678.123456)
+            Computes SHA256 of: "1702345678.123456:Suspicious process detected"
+        """
         data = f"{timestamp}:{content}".encode('utf-8')
         return hashlib.sha256(data).hexdigest()

trace/models/__init__.py

@@ -12,6 +12,9 @@ from .extractors import TagExtractor, IOCExtractor
 @dataclass
 class Note:
     content: str
+    # Unix timestamp: seconds since 1970-01-01 00:00:00 UTC as float
+    # Example: 1702345678.123456
+    # This exact float value (with full precision) is used in hash calculation
     timestamp: float = field(default_factory=time.time)
     note_id: str = field(default_factory=lambda: str(uuid.uuid4()))
     content_hash: str = ""
@@ -28,7 +31,16 @@ class Note:
         self.iocs = IOCExtractor.extract_iocs(self.content)
 
     def calculate_hash(self):
-        # We hash the content + timestamp to ensure integrity of 'when' it was said
+        """Calculate SHA256 hash of timestamp:content.
+
+        Hash input format: "{timestamp}:{content}"
+        - timestamp: Unix epoch timestamp as float (e.g., "1702345678.123456")
+        - The float is converted to string using Python's default str() conversion
+        - Colon separator between timestamp and content
+        - Ensures integrity of both WHAT was said and WHEN it was said
+
+        Example hash input: "1702345678.123456:Suspicious process detected"
+        """
         data = f"{self.timestamp}:{self.content}".encode('utf-8')
         self.content_hash = hashlib.sha256(data).hexdigest()
 

trace/tui/handlers/export_handler.py

@@ -222,15 +222,23 @@ class ExportHandler:
 
     @staticmethod
     def _write_note_markdown(f, note: Note):
-        """Helper to write a note in markdown format"""
+        """Helper to write a note in markdown format
+
+        Includes Unix timestamp for hash reproducibility - anyone can recompute the hash
+        using the formula: SHA256("{unix_timestamp}:{content}")
+        """
         f.write(f"- **{time.ctime(note.timestamp)}**\n")
-        f.write(f"  - Content: {note.content}\n")
+        f.write(f"  - Unix Timestamp: `{note.timestamp}` (for hash verification)\n")
+        f.write(f"  - Content:\n")
+        # Properly indent multi-line content
+        for line in note.content.splitlines():
+            f.write(f"    {line}\n")
         if note.tags:
             tags_str = " ".join([f"#{tag}" for tag in note.tags])
             f.write(f"  - Tags: {tags_str}\n")
-        f.write(f"  - Hash: `{note.content_hash}`\n")
+        f.write(f"  - SHA256 Hash (timestamp:content): `{note.content_hash}`\n")
         if note.signature:
-            f.write("  - **Signature Verified:**\n")
+            f.write("  - **GPG Signature of Hash:**\n")
             f.write("    ```\n")
             for line in note.signature.splitlines():
                 f.write(f"    {line}\n")

trace/tui_app.py

@@ -119,13 +119,13 @@ class TUI:
         self.flash_time = time.time()
 
     def verify_note_signature(self):
-        """Show detailed verification dialog for current note"""
+        """Show signature verification and print raw signature to terminal"""
         if not self.current_note:
             return
 
         verified, info = self.current_note.verify_signature()
 
-        # Prepare dialog content
+        # Handle unsigned notes
         if not self.current_note.signature:
             title = "Note Signature Status"
             message = [
@@ -134,56 +134,95 @@ class TUI:
                 "To sign notes, enable GPG signing in settings",
                 "and ensure you have a GPG key configured."
             ]
-        elif verified:
-            title = "✓ Signature Verified"
-            message = [
-                "The note's signature is valid.",
-                "",
-                f"Signed by: {info}",
-                "",
-                "This note has not been tampered with since signing."
-            ]
-        else:
-            title = "✗ Signature Verification Failed"
-            message = [
-                "The note's signature could not be verified.",
-                "",
-                f"Reason: {info}",
-                "",
-                "Possible causes:",
-                "- Public key not in keyring",
-                "- Note content was modified after signing",
-                "- Signature is corrupted"
-            ]
+            self._show_simple_dialog(title, message)
+            return
 
-        # Display dialog (reuse pattern from other dialogs)
+        # Temporarily exit curses to print signature to terminal
+        curses.endwin()
+
+        try:
+            # Print verification status
+            print("\n" + "=" * 70)
+            if verified:
+                print(f"✓ SIGNATURE VERIFIED - Signed by: {info}")
+            else:
+                print(f"✗ SIGNATURE VERIFICATION FAILED - Reason: {info}")
+            print("=" * 70)
+            print("\nRAW PGP SIGNATURE (select and copy from terminal):")
+            print("-" * 70)
+
+            # Print the actual signature
+            print(self.current_note.signature)
+
+            print("-" * 70)
+            print("\nPress Enter to return to trace...")
+            input()
+
+        finally:
+            # Restore curses mode
+            self.stdscr.refresh()
+            curses.doupdate()
+
+    def _show_simple_dialog(self, title, message_lines):
+        """Display a simple scrollable dialog with the given title and message lines"""
         h, w = self.stdscr.getmaxyx()
-        dialog_h = min(len(message) + 6, h - 4)
-        dialog_w = min(max(len(line) for line in [title] + message) + 6, w - 4)
+        dialog_h = min(h - 4, len(message_lines) + 8)
+        dialog_w = min(w - 4, max(len(title) + 4, max((len(line) for line in message_lines), default=40) + 4))
         start_y = (h - dialog_h) // 2
         start_x = (w - dialog_w) // 2
 
-        # Create dialog window
         dialog = curses.newwin(dialog_h, dialog_w, start_y, start_x)
-        dialog.box()
 
-        # Title
-        dialog.attron(curses.A_BOLD)
-        title_x = (dialog_w - len(title)) // 2
-        dialog.addstr(1, title_x, title)
-        dialog.attroff(curses.A_BOLD)
+        scroll_offset = 0
+        max_scroll = max(0, len(message_lines) - (dialog_h - 6))
 
-        # Message
-        for i, line in enumerate(message):
-            dialog.addstr(3 + i, 2, line)
+        while True:
+            dialog.clear()
+            dialog.box()
 
-        # Footer
-        footer = "Press any key to close"
-        footer_x = (dialog_w - len(footer)) // 2
-        dialog.addstr(dialog_h - 2, footer_x, footer, curses.color_pair(3))
+            # Title
+            dialog.attron(curses.A_BOLD)
+            title_x = max(2, (dialog_w - len(title)) // 2)
+            try:
+                dialog.addstr(1, title_x, title[:dialog_w - 4])
+            except curses.error:
+                pass
+            dialog.attroff(curses.A_BOLD)
 
-        dialog.refresh()
-        dialog.getch()
+            # Display visible lines
+            visible_lines = message_lines[scroll_offset:scroll_offset + dialog_h - 6]
+            for i, line in enumerate(visible_lines):
+                try:
+                    truncated_line = line[:dialog_w - 4]
+                    dialog.addstr(3 + i, 2, truncated_line)
+                except curses.error:
+                    pass
+
+            # Footer
+            if max_scroll > 0:
+                footer = f"↑/↓ Scroll  Any other key to close"
+            else:
+                footer = "Press any key to close"
+            footer_x = max(2, (dialog_w - len(footer)) // 2)
+            try:
+                dialog.addstr(dialog_h - 2, footer_x, footer[:dialog_w - 4], curses.color_pair(3))
+            except curses.error:
+                pass
+
+            dialog.refresh()
+
+            # Handle input
+            key = dialog.getch()
+            if key == curses.KEY_UP and scroll_offset > 0:
+                scroll_offset -= 1
+            elif key == curses.KEY_DOWN and scroll_offset < max_scroll:
+                scroll_offset += 1
+            elif key == curses.KEY_PPAGE:
+                scroll_offset = max(0, scroll_offset - (dialog_h - 6))
+            elif key == curses.KEY_NPAGE:
+                scroll_offset = min(max_scroll, scroll_offset + (dialog_h - 6))
+            else:
+                break
 
     def _save_nav_position(self):
         """Save current navigation position before changing views"""