Clarified how the locale override works across different platforms:
- Linux/macOS: LC_ALL and LANG variables control GPG output language
- Windows: GPG may ignore locale variables, but the code remains robust
through explicit encoding='utf-8' and errors='replace' parameters
The combination of:
1. Setting LC_ALL/LANG to C.UTF-8 (works on Linux/macOS)
2. Explicit encoding='utf-8' parameter
3. errors='replace' for graceful handling
...ensures the code works reliably on all platforms even if the locale
setting is not fully respected by GPG.
Issue: GPG verification crashed with UnicodeDecodeError when signatures
contained international characters (German ö, Turkish ü, etc.) in the
signed content.
Error: "'utf-8' codec can't decode byte 0xf6 in position 160"
Root cause: subprocess.Popen was using default text decoding without
handling encoding errors gracefully.
Solution:
1. Changed LC_ALL/LANG from 'C' to 'C.UTF-8' to ensure GPG uses UTF-8
2. Added explicit encoding='utf-8' parameter to Popen
3. Added errors='replace' to replace invalid UTF-8 bytes instead of crashing
This allows the verification to proceed even if GPG's output contains
characters that don't decode cleanly, ensuring robustness with
multilingual content.
Root cause: GPG outputs verification messages in the user's locale
(e.g., German "Gute Signatur von" instead of "Good signature from"),
causing the parsing logic to fail and display "Unknown signer".
Solution: Force LC_ALL=C and LANG=C environment variables when
calling 'gpg --verify' to ensure consistent English output across
all locales.
This ensures the code can reliably parse "Good signature from"
regardless of the user's system language settings.
The verify_signature() function had a logic bug where the break statement
was executed even when quote parsing failed, preventing fallback to the
key ID extraction.
The break was positioned to execute whenever "Good signature from" appeared,
regardless of whether the signer name was successfully extracted from quotes.
This left signer_info as "Unknown signer" when quote parsing failed.
Fix: Move break inside the successful parsing block (if len(parts) >= 2)
so it only breaks after successfully extracting the signer name. If quote
parsing fails, the loop continues and can fall back to extracting the key
ID from the "using" line.
Testing: Created comprehensive test suite verifying correct extraction of
signer information and proper handling of edge cases.
Added comprehensive CLI Command Reference section documenting:
- Context management commands (--show-context, --list, --switch-*)
- Case/evidence creation commands (--new-case, --new-evidence)
- Advanced note-taking features (--stdin, --case, --evidence overrides)
- Export commands
Placed section prominently after the hot logging feature overview
to maintain focus on the primary use case while documenting the
full CLI surface area.
Added CLI features for rapid observation logging during live forensic analysis:
Context Management:
- --show-context: Display active case and evidence
- --list: List all cases and evidence in hierarchy
- --switch-case: Change active case (by ID or case number)
- --switch-evidence: Change active evidence (by ID or name)
Case/Evidence Creation:
- --new-case: Create new case with optional --name and --investigator
- --new-evidence: Create new evidence with optional --description
Enhanced Note-Taking:
- --stdin: Read note content from stdin for piping command output
- --case: Add note to specific case without changing active context
- --evidence: Add note to specific evidence without changing active context
All context switching commands support both UUIDs and human-friendly identifiers
(case numbers for cases, names for evidence). Creating new items automatically
sets them as active. Override flags allow temporary context changes without
modifying the active state.
Updated CLAUDE.md with comprehensive CLI command examples organized by workflow.
Remove auto-selection when only one key exists. Users should always
be explicitly aware of which key is being used and have the option
to choose the default key (option 0) instead.
trace/gpg_wizard.py:79-95
The wizard was checking if 'pgp_enabled' key existed in settings dict,
but StateManager.get_settings() always returns this key (as a default
value when settings.json doesn't exist). This caused the wizard to
think it had already run, even on first launch.
Fix: Check if settings file exists instead of checking for key presence.
trace/gpg_wizard.py:120
Fixed rendering issue where the Case Notes section would overlap with the
"No evidence items" message. The problem occurred because the y_pos variable
was not updated after drawing the 2-line message, causing subsequent content
to render at the wrong vertical position.
Changes:
- Adjusted "No evidence items" message to start at y_pos (not y_pos + 1)
for consistency with evidence item rendering
- Added y_pos += 2 after the message to account for the 2 lines used
- Fixed boundary check from y_pos + 2 to y_pos + 1
Also verified all other views (case list, tags, IOCs) handle empty states
correctly and don't have similar overlap issues.
Resolves visual bug reported in case detail view.
When exporting to markdown (--export), the entire export document is now
signed with GPG if signing is enabled in settings.
Features:
- Builds export content in memory before signing
- Signs the complete document as one GPG clearsigned block
- Individual note signatures are preserved within the export
- Provides two layers of verification:
1. Document-level: Verifies entire export hasn't been modified
2. Note-level: Verifies individual notes haven't been tampered with
Verification workflow:
- Entire export: gpg --verify export.md
- Individual notes: Extract signature blocks and verify separately
Changes:
- Renamed write_note() to format_note_for_export() returning string
- Export content built in memory before file write
- Signs complete export if pgp_enabled=True
- Shows verification instructions after successful export
Example output:
✓ Export signed with GPG
✓ Exported to case-2024-001.md
To verify the export:
gpg --verify case-2024-001.md
This commit adds comprehensive GPG signature verification functionality
and a first-run setup wizard for configuring GPG signing:
**GPG Verification Features:**
- Added `Crypto.verify_signature()` to verify GPG clearsigned messages
- Added `Crypto.is_gpg_available()` to detect GPG installation
- Added `Note.verify_signature()` method to verify note signatures
- Verification returns status (verified/failed/unsigned) and signer info
**TUI Enhancements:**
- Display verification symbols in note lists: ✓ (verified), ✗ (failed), ? (unsigned)
- Updated note detail view to show verification status with signer information
- Added 'v' key binding in note detail view to trigger verification dialog
- Verification dialog shows detailed status and helpful error messages
**First-Run Wizard:**
- Created `gpg_wizard.py` module with interactive setup wizard
- Wizard runs on first application startup (when settings.json doesn't exist)
- Detects GPG availability and informs user if not installed
- Lists available secret keys and allows user to select signing key
- Gracefully handles missing GPG or no available keys
- Settings can be manually edited later via ~/.trace/settings.json
**Implementation Details:**
- GPG key ID is now stored in settings as `gpg_key_id`
- All note displays show verification status for better chain-of-custody
- External verification possible via standard GPG tools on exported notes
- Follows existing codebase patterns (atomic writes, graceful degradation)
Files modified:
- trace/crypto.py: Added verification and availability check functions
- trace/models/__init__.py: Added Note.verify_signature() method
- trace/gpg_wizard.py: New first-run setup wizard module
- trace/cli.py: Integrated wizard before TUI launch
- trace/tui_app.py: Added verification display and dialog
The settings dialog window height was too small (12 lines), causing the
footer to overlap with the 'Save' option at position 10. Users couldn't
see or select the Save button, preventing GPG key configuration from
being persisted.
Changes:
- Increased window height from 12 to 15 lines
- Adjusted y position to keep dialog centered
- Now all 4 options (GPG Signing, Select GPG Key, Save, Cancel) are
fully visible with the footer below them
This was a pre-existing UI bug, not introduced by the restructuring.
Resolved naming conflict between trace/tui.py (file) and trace/tui/ (package).
Python prioritizes packages over modules with the same name, causing import failures.
Changes:
- Renamed trace/tui.py to trace/tui_app.py
- Updated trace/cli.py to import from tui_app
- Updated trace/tui/__init__.py to re-export from tui_app for backward compatibility
This allows both direct imports (from trace.tui_app) and package imports (from trace.tui)
to work correctly, maintaining backward compatibility while supporting the new modular structure.
Major refactoring to organize code into focused, single-responsibility modules
that are easier for AI coding agents and developers to navigate and modify.
**Module Reorganization:**
Models Package (trace/models/):
- Moved models.py content into models/__init__.py
- Extracted IOC extraction into models/extractors/ioc_extractor.py (236 lines)
- Extracted tag extraction into models/extractors/tag_extractor.py (34 lines)
- Reduced duplication and improved maintainability
Storage Package (trace/storage_impl/):
- Split storage.py (402 lines) into focused modules:
- storage.py: Main Storage class (112 lines)
- state_manager.py: StateManager for context/settings (92 lines)
- lock_manager.py: Cross-platform file locking (87 lines)
- demo_data.py: Demo case creation (143 lines)
- Added backward-compatible wrapper at trace/storage.py
TUI Utilities (trace/tui/):
- Created rendering package:
- colors.py: Color pair constants and initialization (43 lines)
- text_renderer.py: Text rendering with highlighting (137 lines)
- Created handlers package:
- export_handler.py: Export functionality (238 lines)
- Main tui.py (3307 lines) remains for future refactoring
**Benefits:**
- Smaller, focused files (most < 250 lines)
- Clear single responsibilities
- Easier to locate and modify specific functionality
- Better separation of concerns
- Reduced cognitive load for AI agents
- All tests pass, no features removed
**Testing:**
- All existing tests pass
- Imports verified
- CLI and storage functionality tested
- Backward compatibility maintained
Updated CLAUDE.md to document new architecture and AI optimization strategy.
When navigating between views (case list -> case detail -> evidence detail)
and pressing 'b' to go back, the previously selected item is now restored
instead of always jumping to the top of the list.
Implementation:
- Added nav_history dict to track selected indices per view/context
- _save_nav_position() saves current index before navigating away
- _restore_nav_position() restores index when returning to a view
- Works across all navigation paths: cases, evidence, tags, IOCs, notes
This improves UX by maintaining user context during navigation.
Location: trace/tui.py
This commit addresses 20 bugs discovered during comprehensive code review,
focusing on data integrity, concurrent access, and user experience.
CRITICAL FIXES:
- Fix GPG key listing to support keys with multiple UIDs (crypto.py:40)
- Implement cross-platform file locking to prevent concurrent access corruption (storage.py)
- Fix evidence detail delete logic that could delete wrong note (tui.py:2481-2497)
- Add corrupted JSON handling with user prompt and automatic backup (storage.py, tui.py)
DATA INTEGRITY:
- Fix IOC/Hash pattern false positives by checking longest hashes first (models.py:32-95)
- Fix URL pattern to exclude trailing punctuation (models.py:81, 152, 216)
- Improve IOC overlap detection with proper range tracking (models.py)
- Fix note deletion to use note_id instead of object identity (tui.py:2498-2619)
- Add state validation to detect and clear orphaned references (storage.py:355-384)
SCROLLING & NAVIGATION:
- Fix evidence detail view to support full scrolling instead of "last N" (tui.py:816-847)
- Fix filter reset index bounds bug (tui.py:1581-1654)
- Add scroll_offset validation after all operations (tui.py:1608-1654)
- Fix division by zero in scroll calculations (tui.py:446-478)
- Validate selection bounds across all views (tui.py:_validate_selection_bounds)
EXPORT & CLI:
- Fix multi-line note export with proper markdown indentation (cli.py:129-143)
- Add stderr warnings for GPG signature failures (cli.py:61, 63)
- Validate active context and show warnings in CLI (cli.py:12-44)
TESTING:
- Update tests to support new lock file mechanism (test_models.py)
- All existing tests pass with new changes
Breaking changes: None
Backward compatible: Yes (existing data files work unchanged)
When adding filter support, the display logic correctly used filtered
lists but the interaction handlers (Enter, 'v', 'd' keys) and navigation
(max_idx calculations) still used unfiltered lists. This caused:
- Wrong item selected when filter active
- Potential out-of-bounds errors
- Inconsistent behavior between display and action
Fixed in all affected views:
1. evidence_detail:
- Enter key navigation now uses filtered notes
- 'v' key (notes modal) now uses filtered notes
- Delete handler now uses filtered notes
- max_idx navigation now uses filtered notes
2. tag_notes_list:
- Enter key navigation now uses filtered notes
- Delete handler now uses filtered notes
- max_idx navigation now uses filtered notes
3. ioc_notes_list:
- Enter key navigation now uses filtered notes
- Delete handler now uses filtered notes
- max_idx navigation now uses filtered notes
4. tags_list:
- Enter key navigation now uses filtered tags
- max_idx navigation now uses filtered tags
5. ioc_list:
- Enter key navigation now uses filtered IOCs
- max_idx navigation now uses filtered IOCs
All views now consistently respect active filters across display,
navigation, and interaction handlers.
The delete handler was checking case notes first, then evidence,
but the display order is evidence first, then notes. This caused:
- Selecting evidence + pressing 'd' -> asked to delete a note
- Selecting note + pressing 'd' -> asked to delete evidence
Fixed by reordering the checks to match display order:
1. Check if index < len(evidence) -> delete evidence
2. Otherwise check if in notes range -> delete note
Now delete correctly targets the selected item type.
Based on UX analysis, added three major improvements:
1. Context-sensitive filtering everywhere ('/' key):
- evidence_detail: Filter notes by content
- tags_list: Filter tags by name
- tag_notes_list: Filter notes by content
- ioc_list: Filter IOCs by value or type
- ioc_notes_list: Filter notes by content
- All views show active filter in footer
2. Extended delete support ('d' key):
- note_detail: Delete current note and return to previous view
- tag_notes_list: Delete selected note from filtered view
- ioc_notes_list: Delete selected note from filtered view
- Finds and removes note from parent case/evidence
3. Markdown export for case/evidence ('e' key):
- case_detail: Export entire case + all evidence to markdown
- evidence_detail: Export single evidence item to markdown
- Files saved to ~/.trace/exports/ with timestamps
- Complements existing IOC export functionality
All changes maintain consistent UX patterns across views and
provide clear feedback through updated footers showing available
actions in each context.
Documents findings from comprehensive menu interface review across all
9 views in the TUI. Identifies inconsistencies with filter, delete,
and export functionality.
Clarifications from user:
- 'n' and 'a' keys correctly limited to case/evidence contexts
- Filter should work everywhere (context-sensitive)
- Delete should work for all note views, not tag/IOC lists
- Export should extend to case/evidence markdown exports
Both views now support dual navigation options:
- Enter: Opens note_detail view (single note focus)
- 'v': Opens notes modal with selected note highlighted
Previously, case_detail would open the notes modal from the beginning
even when a case note was selected. Now it intelligently jumps to the
selected case note, matching the behavior in evidence_detail view.
This provides a consistent, predictable user experience across both
views where notes can be selected and viewed.
Now provides two ways to access notes in evidence_detail view:
- Enter: Opens note_detail view (single note focus)
- 'v': Opens notes modal with selected note highlighted (all notes)
The 'v' key now intelligently jumps to the selected note when
opening the modal, providing context while still showing all notes.
This gives users flexibility in how they want to view their notes.
Previously, pressing Enter on a note in evidence_detail view would
open the notes modal (same as 'v' key), while in other views
(case_detail, tag_notes_list, ioc_notes_list) it would open the
note_detail view. This created confusing and inconsistent behavior.
Now Enter consistently opens note_detail view across all contexts:
- Enter: Opens note detail view (full note content)
- 'v': Opens notes modal (scrollable list of all notes)
This aligns the implementation with the help text which already
documented the correct behavior.
Added full Unicode character support to the TUI's multiline input dialog.
Previously, only ASCII characters (32-126) were captured when typing notes.
Changes:
- Added UTF-8 multibyte character handling to _multiline_input_dialog()
- Properly collects and decodes 2/3/4-byte UTF-8 sequences
- Added explicit UTF-8 encoding to all file I/O operations
- Added ensure_ascii=False to JSON serialization for proper Unicode preservation
This fix allows users to enter umlauts (ä, ö, ü), accented characters
(é, à, ñ), and other Unicode characters in notes, case names, and all
other text input fields.
Tested with German umlauts, Asian characters, Cyrillic, and special chars.
Critical Fixes:
- Fixed IOC extraction order: URLs now checked before domains to prevent duplicates
- Fixed overlapping IOC highlights with overlap detection
- Fixed IPv4 pattern to validate octets (0-255) preventing invalid IPs like 999.999.999.999
- Fixed IPv6 pattern to support compressed format (::)
- Fixed hash extraction order: SHA256 -> SHA1 -> MD5 to prevent misclassification
High Priority Fixes:
- Added 10s timeout to all GPG subprocess calls to prevent hangs
- Fixed indentation inconsistency in storage.py:253
Performance Improvements:
- Removed 8 time.sleep(0.1) calls from demo case creation (800ms faster startup)
Robustness Improvements:
- Added error handling to export_markdown() for IOError/OSError/PermissionError
- Implemented atomic writes for state file (set_active)
- Implemented atomic writes for settings file (set_setting)
All changes tested and verified with unit tests.
