This commit implements a complete visual redesign and refactoring to improve
consistency, accessibility, and maintainability of the TUI interface.
## New Visual Constants Module
- Created trace/tui/visual_constants.py with centralized constants:
- Layout: Screen positioning and structure (header, content, footer)
- Spacing: Padding and margins (dialogs, horizontal padding)
- ColumnWidths: Fixed column widths for lists (tags, IOCs, content)
- DialogSize: Standard dialog dimensions (small, medium, large)
- Icons: Unicode symbols used throughout UI
- Timing: Animation and feedback timing constants
## Color System Improvements
- Fixed duplicate color definitions (removed from tui_app.py)
- Use ColorPairs constants throughout instead of hardcoded numbers
- Separated tag colors from footer colors:
- Tags now use magenta (ColorPairs.TAG) instead of yellow
- Footers keep yellow (ColorPairs.WARNING) for consistency
- Updated TAG_SELECTED to magenta on cyan
- All 129+ color_pair() calls now use semantic ColorPairs constants
## Accessibility Enhancements
- Added warning icons (⚠) to all IOC displays for visual + color cues
- Tag highlighting now uses distinct magenta color
- Improved color semantics reduce reliance on color alone
- Smart text truncation at word boundaries for better readability
## Layout & Spacing Standardization
- Replaced magic numbers with Layout/Spacing constants:
- Footer positioning: height - Layout.FOOTER_OFFSET_FROM_BOTTOM
- Content area: Layout.CONTENT_START_Y
- Truncation: width - Spacing.HORIZONTAL_PADDING
- Dialog margins: Spacing.DIALOG_MARGIN
- Standardized dialog sizes using DialogSize constants:
- Input dialogs: DialogSize.MEDIUM
- Multiline dialogs: DialogSize.LARGE
- Confirm dialogs: DialogSize.SMALL (with dynamic width)
- Settings dialog: DialogSize.MEDIUM
## User Experience Improvements
- Enhanced footer command organization with visual grouping:
- Used Icons.SEPARATOR_GROUP (│) to group related commands
- Example: "[n] Add Note │ [t] Tags [i] IOCs │ [v] View [e] Export"
- Smart content truncation (_safe_truncate):
- Added word_break parameter (default True)
- Breaks at word boundaries when >60% text retained
- Maintains Unicode safety while improving readability
- Improved empty state messages:
- New _draw_empty_state() helper for consistent visual structure
- Centered boxes with proper spacing
- Clear call-to-action hints
- Applied to "No cases found" and "No cases match filter"
## Code Quality & Maintainability
- Eliminated hardcoded spacing values throughout 3,468-line file
- Used Icons constants for all Unicode symbols (─│┌└◆●○▸⚠⌗◈✓✗?)
- Fixed circular import issues with delayed global imports in TUI.__init__
- Updated comments to reflect new ColorPairs constants
- Consistent use of f-strings for footer construction
## Visual Consistency
- Replaced all "─" literals with Icons.SEPARATOR_H
- Standardized truncation widths (width - Spacing.HORIZONTAL_PADDING)
- Consistent use of ColumnWidths for tag (30) and IOC (50+2) displays
- All dialogs now use standard sizes from visual_constants
## Testing
- Verified no syntax errors in all modified files
- Confirmed successful module imports
- Tested CLI functionality (--help, --list)
- Backward compatibility maintained
This establishes a strong foundation for future UI enhancements while
significantly improving code maintainability and visual consistency.
The TUI export handler (_write_note_markdown) was missing the Unix timestamp
that was added to CLI exports. This ensures consistency across all export paths.
Changes:
- Updated _write_note_markdown() in trace/tui/handlers/export_handler.py
- Now includes "Unix Timestamp: `{timestamp}` (for hash verification)" line
- Matches the format from CLI exports in trace/cli.py
- Multi-line content is properly indented
- Hash label updated to "SHA256 Hash (timestamp:content)" for clarity
All export paths (CLI --export, TUI case export, TUI evidence export) now
include the Unix timestamp needed for independent hash verification.
Changes:
- Modified export format to include Unix timestamp for hash reproducibility
- Each note now shows: "Unix Timestamp: `{timestamp}` (for hash verification)"
- This allows independent verification using: SHA256("{timestamp}:{content}")
- Updated README.md with comprehensive timestamp format documentation:
- Clarified that timestamps are Unix epoch (seconds since 1970-01-01 UTC) as float
- Added example: 1702345678.123456
- Documented exact hash input format: "{timestamp}:{content}"
- Added "Hash Verification (Manual)" section with step-by-step verification instructions
- Included examples using Python and command-line tools
- Updated Core Features table with timestamp format details
- Enhanced Layer 1 integrity documentation with concrete examples
These changes ensure hash reproducibility from exported markdown files,
critical for forensic chain of custody and independent verification.
Added comprehensive documentation to make it crystal clear that:
- Timestamps are Unix epoch timestamps (seconds since 1970-01-01 00:00:00 UTC) stored as floats
- Hash input format is "{timestamp}:{content}" with float-to-string conversion
- Example: "1702345678.123456:Suspicious process detected"
- Full float precision is preserved, ensuring forensic tamper-evidence
Updated documentation in:
- trace/models/__init__.py: Added field comments and detailed docstring for calculate_hash()
- trace/crypto.py: Added comprehensive docstring for hash_content() with examples
- CLAUDE.md: Added detailed explanation in Integrity System section
Simplified approach: When pressing 'V' in note details, the TUI temporarily
exits to terminal mode and prints the raw PGP signature directly to stdout.
This allows users to:
- Select and copy the signature using their terminal's native copy/paste
- Works with any shell (bash, fish, zsh, etc.)
- No dependency on clipboard tools (xclip, xsel, pbcopy, clip)
- No complex shell-specific commands needed
The signature is displayed with verification status and clear formatting,
then waits for Enter to return to the TUI.
This is the simplest and most universal solution for viewing and copying
PGP signatures for external verification in Kleopatra or GPG tools.
Enhanced the signature verification dialog with:
1. Clear clipboard status feedback:
- Shows success: "✓ Clipboard: Copied successfully (using xclip)"
- Shows failure: "✗ Clipboard: Failed to copy"
- Provides installation instructions for Linux (xclip/xsel)
2. Direct GPG verification commands:
- Linux/macOS: gpg --verify <(cat ~/.trace/last_signature.txt)
- Windows PowerShell: Get-Content ~/.trace/last_signature.txt | gpg --verify
- Also includes simple view commands (cat/Get-Content)
3. Better organized dialog sections:
- EXPORT STATUS section showing clipboard and file status
- VERIFY WITH GPG section with platform-specific commands
Users now get immediate, clear feedback about whether clipboard copy worked
and can easily copy/paste the verification commands to verify signatures
externally in Kleopatra or command-line GPG tools.
When pressing 'V' on a note detail view, the PGP signature is now:
1. Automatically copied to system clipboard (if available)
2. Saved to ~/.trace/last_signature.txt for manual access
This solves the copy/paste problem where signatures displayed in the TUI
modal couldn't be selected and copied. Users can now:
- Paste directly from clipboard into Kleopatra/GPG tools (if clipboard worked)
- Access the signature file from another terminal if needed
Platform support:
- Linux: Uses xclip or xsel (falls back to file if not available)
- macOS: Uses pbcopy (falls back to file if not available)
- Windows: Uses clip (falls back to file if not available)
The dialog shows clear feedback about:
- Whether clipboard copy succeeded
- File location and how to access it manually
- Verification status (verified/failed/unsigned)
No external dependencies added - uses only stdlib subprocess calls.
Modified the verify_note_signature() function to show the complete raw PGP
signature text when users press 'V' in the note details view. This allows
users to copy/paste signatures into external tools like Kleopatra for
independent verification.
Changes:
- Added raw signature display after verification status info
- Implemented scrollable dialog with arrow keys and Page Up/Down support
- Added clear separator and label for the signature section
- Shows signature for verified, failed, and unsigned notes (when present)
Users can now easily copy PGP signatures for external verification workflows.
Clarified how the locale override works across different platforms:
- Linux/macOS: LC_ALL and LANG variables control GPG output language
- Windows: GPG may ignore locale variables, but the code remains robust
through explicit encoding='utf-8' and errors='replace' parameters
The combination of:
1. Setting LC_ALL/LANG to C.UTF-8 (works on Linux/macOS)
2. Explicit encoding='utf-8' parameter
3. errors='replace' for graceful handling
...ensures the code works reliably on all platforms even if the locale
setting is not fully respected by GPG.
Issue: GPG verification crashed with UnicodeDecodeError when signatures
contained international characters (German ö, Turkish ü, etc.) in the
signed content.
Error: "'utf-8' codec can't decode byte 0xf6 in position 160"
Root cause: subprocess.Popen was using default text decoding without
handling encoding errors gracefully.
Solution:
1. Changed LC_ALL/LANG from 'C' to 'C.UTF-8' to ensure GPG uses UTF-8
2. Added explicit encoding='utf-8' parameter to Popen
3. Added errors='replace' to replace invalid UTF-8 bytes instead of crashing
This allows the verification to proceed even if GPG's output contains
characters that don't decode cleanly, ensuring robustness with
multilingual content.
Root cause: GPG outputs verification messages in the user's locale
(e.g., German "Gute Signatur von" instead of "Good signature from"),
causing the parsing logic to fail and display "Unknown signer".
Solution: Force LC_ALL=C and LANG=C environment variables when
calling 'gpg --verify' to ensure consistent English output across
all locales.
This ensures the code can reliably parse "Good signature from"
regardless of the user's system language settings.
The verify_signature() function had a logic bug where the break statement
was executed even when quote parsing failed, preventing fallback to the
key ID extraction.
The break was positioned to execute whenever "Good signature from" appeared,
regardless of whether the signer name was successfully extracted from quotes.
This left signer_info as "Unknown signer" when quote parsing failed.
Fix: Move break inside the successful parsing block (if len(parts) >= 2)
so it only breaks after successfully extracting the signer name. If quote
parsing fails, the loop continues and can fall back to extracting the key
ID from the "using" line.
Testing: Created comprehensive test suite verifying correct extraction of
signer information and proper handling of edge cases.
Added comprehensive CLI Command Reference section documenting:
- Context management commands (--show-context, --list, --switch-*)
- Case/evidence creation commands (--new-case, --new-evidence)
- Advanced note-taking features (--stdin, --case, --evidence overrides)
- Export commands
Placed section prominently after the hot logging feature overview
to maintain focus on the primary use case while documenting the
full CLI surface area.
Added CLI features for rapid observation logging during live forensic analysis:
Context Management:
- --show-context: Display active case and evidence
- --list: List all cases and evidence in hierarchy
- --switch-case: Change active case (by ID or case number)
- --switch-evidence: Change active evidence (by ID or name)
Case/Evidence Creation:
- --new-case: Create new case with optional --name and --investigator
- --new-evidence: Create new evidence with optional --description
Enhanced Note-Taking:
- --stdin: Read note content from stdin for piping command output
- --case: Add note to specific case without changing active context
- --evidence: Add note to specific evidence without changing active context
All context switching commands support both UUIDs and human-friendly identifiers
(case numbers for cases, names for evidence). Creating new items automatically
sets them as active. Override flags allow temporary context changes without
modifying the active state.
Updated CLAUDE.md with comprehensive CLI command examples organized by workflow.
Remove auto-selection when only one key exists. Users should always
be explicitly aware of which key is being used and have the option
to choose the default key (option 0) instead.
trace/gpg_wizard.py:79-95
The wizard was checking if 'pgp_enabled' key existed in settings dict,
but StateManager.get_settings() always returns this key (as a default
value when settings.json doesn't exist). This caused the wizard to
think it had already run, even on first launch.
Fix: Check if settings file exists instead of checking for key presence.
trace/gpg_wizard.py:120
Fixed rendering issue where the Case Notes section would overlap with the
"No evidence items" message. The problem occurred because the y_pos variable
was not updated after drawing the 2-line message, causing subsequent content
to render at the wrong vertical position.
Changes:
- Adjusted "No evidence items" message to start at y_pos (not y_pos + 1)
for consistency with evidence item rendering
- Added y_pos += 2 after the message to account for the 2 lines used
- Fixed boundary check from y_pos + 2 to y_pos + 1
Also verified all other views (case list, tags, IOCs) handle empty states
correctly and don't have similar overlap issues.
Resolves visual bug reported in case detail view.
When exporting to markdown (--export), the entire export document is now
signed with GPG if signing is enabled in settings.
Features:
- Builds export content in memory before signing
- Signs the complete document as one GPG clearsigned block
- Individual note signatures are preserved within the export
- Provides two layers of verification:
1. Document-level: Verifies entire export hasn't been modified
2. Note-level: Verifies individual notes haven't been tampered with
Verification workflow:
- Entire export: gpg --verify export.md
- Individual notes: Extract signature blocks and verify separately
Changes:
- Renamed write_note() to format_note_for_export() returning string
- Export content built in memory before file write
- Signs complete export if pgp_enabled=True
- Shows verification instructions after successful export
Example output:
✓ Export signed with GPG
✓ Exported to case-2024-001.md
To verify the export:
gpg --verify case-2024-001.md
This commit adds comprehensive GPG signature verification functionality
and a first-run setup wizard for configuring GPG signing:
**GPG Verification Features:**
- Added `Crypto.verify_signature()` to verify GPG clearsigned messages
- Added `Crypto.is_gpg_available()` to detect GPG installation
- Added `Note.verify_signature()` method to verify note signatures
- Verification returns status (verified/failed/unsigned) and signer info
**TUI Enhancements:**
- Display verification symbols in note lists: ✓ (verified), ✗ (failed), ? (unsigned)
- Updated note detail view to show verification status with signer information
- Added 'v' key binding in note detail view to trigger verification dialog
- Verification dialog shows detailed status and helpful error messages
**First-Run Wizard:**
- Created `gpg_wizard.py` module with interactive setup wizard
- Wizard runs on first application startup (when settings.json doesn't exist)
- Detects GPG availability and informs user if not installed
- Lists available secret keys and allows user to select signing key
- Gracefully handles missing GPG or no available keys
- Settings can be manually edited later via ~/.trace/settings.json
**Implementation Details:**
- GPG key ID is now stored in settings as `gpg_key_id`
- All note displays show verification status for better chain-of-custody
- External verification possible via standard GPG tools on exported notes
- Follows existing codebase patterns (atomic writes, graceful degradation)
Files modified:
- trace/crypto.py: Added verification and availability check functions
- trace/models/__init__.py: Added Note.verify_signature() method
- trace/gpg_wizard.py: New first-run setup wizard module
- trace/cli.py: Integrated wizard before TUI launch
- trace/tui_app.py: Added verification display and dialog
The settings dialog window height was too small (12 lines), causing the
footer to overlap with the 'Save' option at position 10. Users couldn't
see or select the Save button, preventing GPG key configuration from
being persisted.
Changes:
- Increased window height from 12 to 15 lines
- Adjusted y position to keep dialog centered
- Now all 4 options (GPG Signing, Select GPG Key, Save, Cancel) are
fully visible with the footer below them
This was a pre-existing UI bug, not introduced by the restructuring.
Resolved naming conflict between trace/tui.py (file) and trace/tui/ (package).
Python prioritizes packages over modules with the same name, causing import failures.
Changes:
- Renamed trace/tui.py to trace/tui_app.py
- Updated trace/cli.py to import from tui_app
- Updated trace/tui/__init__.py to re-export from tui_app for backward compatibility
This allows both direct imports (from trace.tui_app) and package imports (from trace.tui)
to work correctly, maintaining backward compatibility while supporting the new modular structure.
Major refactoring to organize code into focused, single-responsibility modules
that are easier for AI coding agents and developers to navigate and modify.
**Module Reorganization:**
Models Package (trace/models/):
- Moved models.py content into models/__init__.py
- Extracted IOC extraction into models/extractors/ioc_extractor.py (236 lines)
- Extracted tag extraction into models/extractors/tag_extractor.py (34 lines)
- Reduced duplication and improved maintainability
Storage Package (trace/storage_impl/):
- Split storage.py (402 lines) into focused modules:
- storage.py: Main Storage class (112 lines)
- state_manager.py: StateManager for context/settings (92 lines)
- lock_manager.py: Cross-platform file locking (87 lines)
- demo_data.py: Demo case creation (143 lines)
- Added backward-compatible wrapper at trace/storage.py
TUI Utilities (trace/tui/):
- Created rendering package:
- colors.py: Color pair constants and initialization (43 lines)
- text_renderer.py: Text rendering with highlighting (137 lines)
- Created handlers package:
- export_handler.py: Export functionality (238 lines)
- Main tui.py (3307 lines) remains for future refactoring
**Benefits:**
- Smaller, focused files (most < 250 lines)
- Clear single responsibilities
- Easier to locate and modify specific functionality
- Better separation of concerns
- Reduced cognitive load for AI agents
- All tests pass, no features removed
**Testing:**
- All existing tests pass
- Imports verified
- CLI and storage functionality tested
- Backward compatibility maintained
Updated CLAUDE.md to document new architecture and AI optimization strategy.
When navigating between views (case list -> case detail -> evidence detail)
and pressing 'b' to go back, the previously selected item is now restored
instead of always jumping to the top of the list.
Implementation:
- Added nav_history dict to track selected indices per view/context
- _save_nav_position() saves current index before navigating away
- _restore_nav_position() restores index when returning to a view
- Works across all navigation paths: cases, evidence, tags, IOCs, notes
This improves UX by maintaining user context during navigation.
Location: trace/tui.py
This commit addresses 20 bugs discovered during comprehensive code review,
focusing on data integrity, concurrent access, and user experience.
CRITICAL FIXES:
- Fix GPG key listing to support keys with multiple UIDs (crypto.py:40)
- Implement cross-platform file locking to prevent concurrent access corruption (storage.py)
- Fix evidence detail delete logic that could delete wrong note (tui.py:2481-2497)
- Add corrupted JSON handling with user prompt and automatic backup (storage.py, tui.py)
DATA INTEGRITY:
- Fix IOC/Hash pattern false positives by checking longest hashes first (models.py:32-95)
- Fix URL pattern to exclude trailing punctuation (models.py:81, 152, 216)
- Improve IOC overlap detection with proper range tracking (models.py)
- Fix note deletion to use note_id instead of object identity (tui.py:2498-2619)
- Add state validation to detect and clear orphaned references (storage.py:355-384)
SCROLLING & NAVIGATION:
- Fix evidence detail view to support full scrolling instead of "last N" (tui.py:816-847)
- Fix filter reset index bounds bug (tui.py:1581-1654)
- Add scroll_offset validation after all operations (tui.py:1608-1654)
- Fix division by zero in scroll calculations (tui.py:446-478)
- Validate selection bounds across all views (tui.py:_validate_selection_bounds)
EXPORT & CLI:
- Fix multi-line note export with proper markdown indentation (cli.py:129-143)
- Add stderr warnings for GPG signature failures (cli.py:61, 63)
- Validate active context and show warnings in CLI (cli.py:12-44)
TESTING:
- Update tests to support new lock file mechanism (test_models.py)
- All existing tests pass with new changes
Breaking changes: None
Backward compatible: Yes (existing data files work unchanged)
When adding filter support, the display logic correctly used filtered
lists but the interaction handlers (Enter, 'v', 'd' keys) and navigation
(max_idx calculations) still used unfiltered lists. This caused:
- Wrong item selected when filter active
- Potential out-of-bounds errors
- Inconsistent behavior between display and action
Fixed in all affected views:
1. evidence_detail:
- Enter key navigation now uses filtered notes
- 'v' key (notes modal) now uses filtered notes
- Delete handler now uses filtered notes
- max_idx navigation now uses filtered notes
2. tag_notes_list:
- Enter key navigation now uses filtered notes
- Delete handler now uses filtered notes
- max_idx navigation now uses filtered notes
3. ioc_notes_list:
- Enter key navigation now uses filtered notes
- Delete handler now uses filtered notes
- max_idx navigation now uses filtered notes
4. tags_list:
- Enter key navigation now uses filtered tags
- max_idx navigation now uses filtered tags
5. ioc_list:
- Enter key navigation now uses filtered IOCs
- max_idx navigation now uses filtered IOCs
All views now consistently respect active filters across display,
navigation, and interaction handlers.
