semanticsearch #6
670
README.md
670
README.md
@ -1,232 +1,149 @@
|
||||
# ForensicPathways
|
||||
|
||||
Ein kuratiertes Verzeichnis für Digital Forensics und Incident Response (DFIR) Tools, Methoden und Konzepte mit KI-gestützten Workflow-Empfehlungen.
|
||||
Ein umfassendes Verzeichnis digitaler Forensik- und Incident-Response-Tools mit KI-gestützten Empfehlungen basierend auf der NIST SP 800-86 Methodik.
|
||||
|
||||
## ✨ Funktionen
|
||||
## Lizenz
|
||||
|
||||
### 🎯 Hauptansichten
|
||||
- **Kachelansicht (Grid View):** Übersichtliche Kartenansicht aller Tools/Methoden
|
||||
- **Matrix-Ansicht:** Interaktive Matrix nach forensischen Domänen und Untersuchungsphasen (NIST Framework)
|
||||
- **Forensic-AI:** AI-gestützte Workflow-Empfehlungen basierend auf Szenario-Beschreibungen
|
||||
Dieses Projekt ist unter der BSD-3-Clause-Lizenz lizenziert.
|
||||
|
||||
### 🔍 Navigation & Filterung
|
||||
- **Tag-System:** Intelligente Filterung nach Kategorien und Eigenschaften
|
||||
- **Volltext-Suche:** Durchsuchen von Namen, Beschreibungen und Tags
|
||||
- **Domain/Phase-Filter:** Filterung nach forensischen Bereichen und Ermittlungsphasen
|
||||
## Funktionen
|
||||
|
||||
### 📚 Inhaltstypen
|
||||
- **Software/Tools:** Open Source und proprietäre forensische Software
|
||||
- **Methoden:** Bewährte forensische Verfahren und Prozesse
|
||||
- **Konzepte:** Grundlegendes Fachwissen und theoretische Grundlagen
|
||||
### Kernfunktionalität
|
||||
- **Umfassende Tool-Datenbank**: 100+ forensische Tools kategorisiert nach Domänen, Phasen und Skill-Levels
|
||||
- **NIST SP 800-86 Integration**: Vier-Phasen-Methodik (Sammlung → Auswertung → Analyse → Berichterstattung)
|
||||
- **Multiple Ansichtsmodi**: Kachelansicht, Matrix-Übersicht und KI-gestützte Empfehlungen
|
||||
- **Erweiterte Suche**: Textsuche, semantische Embedding-basierte Suche und Multi-Kriterien-Filterung
|
||||
- **Responsive Design**: Dark/Light-Mode-Unterstützung, mobile-optimierte Benutzeroberfläche
|
||||
|
||||
### 📖 Knowledgebase
|
||||
- **Erweiterte Dokumentation:** Detaillierte Artikel zu Tools und Methoden
|
||||
- **Praktische Anleitungen:** Installation, Konfiguration und Best Practices
|
||||
- **Markdown-basiert:** Einfache Erstellung und Wartung von Inhalten
|
||||
### KI-gestützte Analyse
|
||||
- **Micro-Task-Pipeline**: Intelligente Tool-Auswahl durch mehrere KI-Analyseschritte
|
||||
- **Semantische Suche**: Vector-Embeddings für natürlichsprachige Tool-Entdeckung
|
||||
- **Konfidenz-Bewertung**: Transparente Vertrauensmetriken für KI-Empfehlungen
|
||||
- **Audit-Trail**: Vollständige Entscheidungstransparenz mit detaillierter Protokollierung
|
||||
- **Rate Limiting**: Intelligente Warteschlangenverwaltung und nutzerbasierte Ratenbegrenzung
|
||||
|
||||
### 🤝 Contribution-System
|
||||
- **Tool/Methoden-Beiträge:** Webformular für neue Einträge
|
||||
- **Knowledgebase-Artikel:** Artikel-Editor mit Datei-Upload
|
||||
- **Git-Integration:** Automatische Issue-Erstellung für Review-Prozess
|
||||
- **File-Management:** Nextcloud-Integration für Medien-Uploads
|
||||
### Zusammenarbeit & Beiträge
|
||||
- **Tool-Beiträge**: Neue Tools einreichen oder bestehende über Git-Integration bearbeiten
|
||||
- **Knowledgebase**: Community-beigetragene Artikel und Dokumentation
|
||||
- **File-Upload-System**: Nextcloud-Integration für Medien-Anhänge
|
||||
- **Authentifizierung**: OIDC-Integration mit konfigurierbaren Anbietern
|
||||
|
||||
### 🔐 Authentifizierung
|
||||
- **OIDC-Integration:** Single Sign-On mit OpenID Connect
|
||||
- **Berechtigungssteuerung:** Schutz für AI-Features und Contribution-System
|
||||
- **Session-Management:** Sichere JWT-basierte Sessions
|
||||
### Enterprise-Funktionen
|
||||
- **Warteschlangenverwaltung**: Ratenbegrenzte KI-Verarbeitung mit Echtzeit-Status-Updates
|
||||
- **Audit-Protokollierung**: Umfassender forensischer Audit-Trail für KI-Entscheidungsfindung
|
||||
- **Multi-Provider-Unterstützung**: Konfigurierbare KI-Services (Mistral AI, Ollama, OpenAI)
|
||||
- **Git-Integration**: Automatisierte Issue-Erstellung für Beiträge (Gitea, GitHub, GitLab)
|
||||
|
||||
## 🛠 Technische Grundlage
|
||||
## Datenmodell
|
||||
|
||||
- **Framework:** Astro 4.x mit TypeScript
|
||||
- **Styling:** CSS Custom Properties mit Dark/Light Mode
|
||||
- **API:** Node.js Backend mit Astro API Routes
|
||||
- **Datenbank:** YAML-basierte Konfiguration (tools.yaml)
|
||||
Das System verwendet eine YAML-basierte Konfiguration in `src/data/tools.yaml`:
|
||||
|
||||
## 📋 Voraussetzungen
|
||||
```yaml
|
||||
tools:
|
||||
- name: Tool Name
|
||||
type: software|method|concept
|
||||
description: Detaillierte Beschreibung
|
||||
skillLevel: novice|beginner|intermediate|advanced|expert
|
||||
url: https://tool-homepage.com
|
||||
domains: [incident-response, static-investigations, ...]
|
||||
phases: [data-collection, examination, analysis, reporting]
|
||||
platforms: [Windows, Linux, macOS]
|
||||
license: Lizenztyp
|
||||
tags: [gui, commandline, ...]
|
||||
related_concepts: [konzept1, konzept2]
|
||||
# Optionale Felder
|
||||
projectUrl: https://hosted-instance.com # Für CC24-Server gehostete Tools
|
||||
knowledgebase: true # Hat KB-Artikel
|
||||
accessType: download|hosted|cloud
|
||||
|
||||
- **Node.js:** Version 18.x oder höher
|
||||
- **npm:** Version 8.x oder höher
|
||||
- **Nginx:** Für Reverse Proxy (Produktion)
|
||||
domains:
|
||||
- id: incident-response
|
||||
name: Incident Response & Breach-Untersuchung
|
||||
|
||||
phases:
|
||||
- id: data-collection
|
||||
name: Datensammlung
|
||||
description: Imaging, Akquisition, Remote-Collection-Tools
|
||||
|
||||
## 🔧 Externe Abhängigkeiten (Optional)
|
||||
|
||||
### OIDC Provider
|
||||
- **Zweck:** Benutzerauthentifizierung
|
||||
- **Beispiel:** Nextcloud, Keycloak, Auth0
|
||||
- **Konfiguration:** `OIDC_ENDPOINT`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`
|
||||
|
||||
### Nextcloud
|
||||
- **Zweck:** File-Upload für Knowledgebase-Beiträge
|
||||
- **Features:** Medien-Management, öffentliche Links
|
||||
- **Konfiguration:** `NEXTCLOUD_ENDPOINT`, `NEXTCLOUD_USERNAME`, `NEXTCLOUD_PASSWORD`
|
||||
|
||||
### AI Service (Mistral/OpenAI-kompatibel)
|
||||
- **Zweck:** KI-gestützte Tool-Empfehlungen
|
||||
- **Konfiguration:** `AI_ANALYZER_ENDPOINT`, `AI_ANALYZER_API_KEY`, `AI_ANALYZER_MODEL`
|
||||
|
||||
### Uptime Kuma
|
||||
- **Zweck:** Status-Monitoring für gehostete Services
|
||||
- **Integration:** Status-Badges in der Service-Übersicht
|
||||
|
||||
### Git Provider (Gitea/GitHub/GitLab)
|
||||
- **Zweck:** Issue-Erstellung für Contributions
|
||||
- **Konfiguration:** `GIT_PROVIDER`, `GIT_API_ENDPOINT`, `GIT_API_TOKEN`
|
||||
|
||||
## 🚀 Installation
|
||||
|
||||
### Lokale Entwicklung
|
||||
|
||||
```bash
|
||||
# Repository klonen
|
||||
git clone https://git.cc24.dev/mstoeck3/forensic-pathways.git
|
||||
cd forensic-pathways
|
||||
|
||||
# Dependencies installieren
|
||||
npm install
|
||||
|
||||
# Umgebungsvariablen konfigurieren
|
||||
cp .env.example .env
|
||||
# .env bearbeiten (siehe Konfiguration unten)
|
||||
|
||||
npm run astro build
|
||||
|
||||
# Development Server starten
|
||||
npm run dev
|
||||
scenarios:
|
||||
- id: scenario:memory_dump
|
||||
icon: 🧠
|
||||
friendly_name: RAM-Analyse
|
||||
```
|
||||
|
||||
Die Seite ist dann unter `http://localhost:4321` verfügbar.
|
||||
## AI Concept
|
||||
|
||||
### Produktions-Deployment
|
||||
### Micro-Task Architecture
|
||||
The AI system uses a sophisticated pipeline that breaks complex analysis into focused micro-tasks:
|
||||
|
||||
#### 1. System vorbereiten
|
||||
1. **Scenario Analysis**: Understanding the forensic context
|
||||
2. **Investigation Approach**: Determining optimal methodology
|
||||
3. **Critical Considerations**: Identifying potential challenges
|
||||
4. **Tool Selection**: Phase-specific or problem-specific recommendations
|
||||
5. **Background Knowledge**: Relevant concepts and prerequisites
|
||||
6. **Final Synthesis**: Integrated recommendations with confidence scoring
|
||||
|
||||
### Confidence Scoring
|
||||
Each recommendation includes transparent confidence metrics:
|
||||
- **Semantic Relevance**: Vector similarity between query and tool descriptions
|
||||
- **Task Suitability**: AI-assessed fitness for the specific scenario
|
||||
- **Uncertainty Factors**: Potential limitations and considerations
|
||||
- **Strength Indicators**: Why the tool is well-suited
|
||||
|
||||
## NIST SP 800-86 Phases
|
||||
|
||||
The system organizes tools according to the four-phase NIST methodology:
|
||||
|
||||
1. **Data Collection**: Imaging, acquisition, and evidence preservation
|
||||
2. **Examination**: Parsing, extraction, and initial data processing
|
||||
3. **Analysis**: Deep investigation, correlation, and insight generation
|
||||
4. **Reporting**: Documentation, visualization, and presentation
|
||||
|
||||
Each tool is mapped to appropriate phases, enabling workflow-based recommendations.
|
||||
|
||||
## Deployment
|
||||
|
||||
### Production Setup
|
||||
|
||||
1. **Build and Deploy**:
|
||||
```bash
|
||||
# System-Updates
|
||||
sudo apt update && sudo apt upgrade -y
|
||||
|
||||
# Node.js installieren (Ubuntu/Debian)
|
||||
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
|
||||
sudo apt-get install -y nodejs
|
||||
|
||||
# Nginx installieren
|
||||
sudo apt install nginx -y
|
||||
|
||||
# Systemd für Service-Management
|
||||
sudo systemctl enable nginx
|
||||
npm run build
|
||||
sudo ./deploy.sh # Copies dist/ to /var/www/forensic-pathways
|
||||
```
|
||||
|
||||
#### 2. Anwendung installieren
|
||||
|
||||
2. **Configuration**:
|
||||
```bash
|
||||
# Klonen des Repositorys
|
||||
sudo git clone https://git.cc24.dev/mstoeck3/forensic-pathways /opt/forensic-pathways
|
||||
cd /opt/forensic-pathways
|
||||
|
||||
# Abhängigkeiten installieren
|
||||
sudo npm install
|
||||
|
||||
# Production-Build erstellen
|
||||
sudo npm run build
|
||||
npm run astro build
|
||||
|
||||
# Berechtigungen setzen
|
||||
sudo chown -R www-data:www-data /opt/forensic-pathways
|
||||
cd /var/www/forensic-pathways
|
||||
sudo cp .env.example .env
|
||||
sudo nano .env # Configure AI services, authentication, etc.
|
||||
```
|
||||
|
||||
#### 3. Umgebungsvariablen konfigurieren
|
||||
3. **Systemd Service** (`/etc/systemd/system/forensic-pathways.service`):
|
||||
```ini
|
||||
[Unit]
|
||||
Description=ForensicPathways
|
||||
After=network.target
|
||||
|
||||
Erstelle `/opt/forensic-pathways/.env`:
|
||||
[Service]
|
||||
Type=simple
|
||||
User=www-data
|
||||
WorkingDirectory=/var/www/forensic-pathways
|
||||
ExecStart=/usr/bin/node server/entry.mjs
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
Environment=NODE_ENV=production
|
||||
|
||||
```bash
|
||||
# ===========================================
|
||||
# ForensicPathways Environment Configuration
|
||||
# ===========================================
|
||||
|
||||
# Authentication & OIDC (Required)
|
||||
AUTH_SECRET=change-this-to-a-strong-secret-key-in-production
|
||||
OIDC_ENDPOINT=https://your-oidc-provider.com
|
||||
OIDC_CLIENT_ID=your-oidc-client-id
|
||||
OIDC_CLIENT_SECRET=your-oidc-client-secret
|
||||
|
||||
# Auth Scopes - set to true in prod
|
||||
AUTHENTICATION_NECESSARY_CONTRIBUTIONS=true
|
||||
AUTHENTICATION_NECESSARY_AI=true
|
||||
|
||||
# Application Configuration (Required)
|
||||
PUBLIC_BASE_URL=https://your-domain.com
|
||||
NODE_ENV=production
|
||||
|
||||
# AI Service Configuration (Required for AI features)
|
||||
AI_ANALYZER_MODEL=mistral-large-latest
|
||||
AI_ANALYZER_ENDPOINT=https://api.mistral.ai
|
||||
AI_ANALYZER_API_KEY=your-mistral-api-key
|
||||
AI_RATE_LIMIT_DELAY_MS=1000
|
||||
|
||||
# Git Integration (Required for contributions)
|
||||
GIT_REPO_URL=https://git.cc24.dev/mstoeck3/forensic-pathways
|
||||
GIT_PROVIDER=gitea
|
||||
GIT_API_ENDPOINT=https://git.cc24.dev/api/v1
|
||||
GIT_API_TOKEN=your-git-api-token
|
||||
|
||||
# File Upload Configuration (Optional)
|
||||
LOCAL_UPLOAD_PATH=./public/uploads
|
||||
|
||||
# Nextcloud Integration (Optional)
|
||||
NEXTCLOUD_ENDPOINT=https://your-nextcloud.com
|
||||
NEXTCLOUD_USERNAME=your-username
|
||||
NEXTCLOUD_PASSWORD=your-password
|
||||
NEXTCLOUD_UPLOAD_PATH=/kb-media
|
||||
NEXTCLOUD_PUBLIC_URL=https://your-nextcloud.com/s/
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
```bash
|
||||
# Berechtigungen sichern
|
||||
sudo chmod 600 /opt/forensic-pathways/.env
|
||||
sudo chown www-data:www-data /opt/forensic-pathways/.env
|
||||
```
|
||||
|
||||
#### 4. Nginx konfigurieren
|
||||
|
||||
Erstelle `/etc/nginx/sites-available/forensic-pathways`:
|
||||
|
||||
4. **Nginx Configuration**:
|
||||
```nginx
|
||||
server {
|
||||
listen 80;
|
||||
server_name ihre-domain.de;
|
||||
server_name forensic-pathways.yourdomain.com;
|
||||
|
||||
# Redirect HTTP to HTTPS
|
||||
return 301 https://$server_name$request_uri;
|
||||
}
|
||||
|
||||
server {
|
||||
listen 443 ssl http2;
|
||||
server_name ihre-domain.de;
|
||||
|
||||
# SSL Konfiguration (Let's Encrypt empfohlen)
|
||||
ssl_certificate /etc/letsencrypt/live/ihre-domain.de/fullchain.pem;
|
||||
ssl_certificate_key /etc/letsencrypt/live/ihre-domain.de/privkey.pem;
|
||||
|
||||
# Security Headers
|
||||
add_header X-Frame-Options DENY;
|
||||
add_header X-Content-Type-Options nosniff;
|
||||
add_header X-XSS-Protection "1; mode=block";
|
||||
add_header Referrer-Policy "strict-origin-when-cross-origin";
|
||||
|
||||
# Static Files
|
||||
location / {
|
||||
try_files $uri $uri/ @nodejs;
|
||||
root /opt/forensic-pathways/dist;
|
||||
index index.html;
|
||||
|
||||
# Cache static assets
|
||||
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff2?|ttf)$ {
|
||||
expires 1y;
|
||||
add_header Cache-Control "public, immutable";
|
||||
}
|
||||
}
|
||||
|
||||
# API Routes to Node.js
|
||||
location @nodejs {
|
||||
proxy_pass http://localhost:4321;
|
||||
proxy_http_version 1.1;
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
@ -236,251 +153,162 @@ server {
|
||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||
proxy_set_header X-Forwarded-Proto $scheme;
|
||||
proxy_cache_bypass $http_upgrade;
|
||||
proxy_read_timeout 300s;
|
||||
proxy_connect_timeout 75s;
|
||||
}
|
||||
|
||||
# Upload limit
|
||||
client_max_body_size 50M;
|
||||
}
|
||||
```
|
||||
|
||||
5. **Enable and Start**:
|
||||
```bash
|
||||
# Site aktivieren
|
||||
sudo ln -s /etc/nginx/sites-available/forensic-pathways /etc/nginx/sites-enabled/
|
||||
sudo nginx -t
|
||||
sudo systemctl enable forensic-pathways
|
||||
sudo systemctl start forensic-pathways
|
||||
sudo systemctl reload nginx
|
||||
```
|
||||
|
||||
#### 5. Systemd Service einrichten
|
||||
### Environment Configuration
|
||||
|
||||
Erstelle `/etc/systemd/system/forensic-pathways.service`:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=ForensicPathways DFIR Guide
|
||||
After=network.target nginx.service
|
||||
Wants=nginx.service
|
||||
|
||||
[Service]
|
||||
Type=exec
|
||||
User=www-data
|
||||
Group=www-data
|
||||
WorkingDirectory=/opt/forensic-pathways
|
||||
Environment=NODE_ENV=production
|
||||
ExecStart=/usr/bin/node ./dist/server/entry.mjs
|
||||
Restart=always
|
||||
RestartSec=10
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
# Security
|
||||
NoNewPrivileges=yes
|
||||
PrivateTmp=yes
|
||||
ProtectSystem=strict
|
||||
ProtectHome=yes
|
||||
ReadWritePaths=/opt/forensic-pathways
|
||||
CapabilityBoundingSet=
|
||||
|
||||
# Resource Limits
|
||||
LimitNOFILE=65536
|
||||
MemoryMax=512M
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
Key configuration in `.env`:
|
||||
|
||||
```bash
|
||||
# Service aktivieren und starten
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable forensic-pathways
|
||||
sudo systemctl start forensic-pathways
|
||||
# Core Application
|
||||
PUBLIC_BASE_URL=https://forensic-pathways.yourdomain.com
|
||||
AUTH_SECRET=your-secure-random-secret
|
||||
|
||||
# Status prüfen
|
||||
sudo systemctl status forensic-pathways
|
||||
# AI Services (Required)
|
||||
AI_ANALYZER_ENDPOINT=https://api.mistral.ai/v1/chat/completions
|
||||
AI_ANALYZER_API_KEY=your-api-key
|
||||
AI_ANALYZER_MODEL=mistral/mistral-small-latest
|
||||
|
||||
# Vector Embeddings (Recommended)
|
||||
AI_EMBEDDINGS_ENABLED=true
|
||||
AI_EMBEDDINGS_ENDPOINT=https://api.mistral.ai/v1/embeddings
|
||||
AI_EMBEDDINGS_MODEL=mistral-embed
|
||||
|
||||
# Authentication (Optional)
|
||||
AUTHENTICATION_NECESSARY_AI=false
|
||||
OIDC_ENDPOINT=https://your-keycloak.com/auth/realms/your-realm
|
||||
OIDC_CLIENT_ID=forensic-pathways
|
||||
```
|
||||
|
||||
## 🔧 Konfiguration
|
||||
## Externe Abhängigkeiten (Optionale Features)
|
||||
|
||||
### Minimalkonfiguration (ohne Auth)
|
||||
### File-Upload-System
|
||||
- **Nextcloud**: Primärer Speicher für Beitrags-Anhänge
|
||||
- **Lokaler Fallback**: Automatischer Fallback zu lokalem Speicher bei Nextcloud-Ausfall
|
||||
|
||||
### Authentifizierungsanbieter
|
||||
- **Keycloak**: Empfohlener OIDC-Provider
|
||||
- **Andere OIDC**: Jeder OIDC-konforme Provider (Auth0, Azure AD, etc.)
|
||||
|
||||
### Git-Integration
|
||||
- **Gitea**: Primärer Git-Provider für Beiträge
|
||||
- **GitHub/GitLab**: Alternative Git-Provider unterstützt
|
||||
|
||||
### Monitoring
|
||||
- **Uptime Kuma**: Service-Monitoring und Gesundheitschecks (optional)
|
||||
|
||||
### KI-Services
|
||||
- **Mistral AI**: Empfohlen für Produktion (API-Schlüssel erforderlich)
|
||||
- **Ollama**: Lokale Deployment-Option (kein API-Schlüssel benötigt)
|
||||
- **OpenAI**: Alternative kommerzielle Anbieter
|
||||
|
||||
## Knowledgebase-System
|
||||
|
||||
### Artikel hinzufügen
|
||||
|
||||
Knowledgebase-Artikel werden in `src/content/knowledgebase/` als Markdown-Dateien mit Frontmatter gespeichert:
|
||||
|
||||
```markdown
|
||||
---
|
||||
title: "Tool-Konfigurationsanleitung"
|
||||
description: "Schritt-für-Schritt-Setup-Anweisungen"
|
||||
last_updated: 2024-01-15
|
||||
author: "Ihr Name"
|
||||
difficulty: intermediate
|
||||
|
||||
# Tool-Zuordnung (optional)
|
||||
tool_name: "Autopsy"
|
||||
related_tools: ["Volatility 3", "YARA"]
|
||||
|
||||
# Kategorisierung
|
||||
categories: ["konfiguration", "setup"]
|
||||
tags: ["gui", "installation", "windows"]
|
||||
|
||||
published: true
|
||||
---
|
||||
|
||||
# Tool-Konfigurationsanleitung
|
||||
|
||||
Ihr Artikel-Inhalt hier...
|
||||
|
||||
## Voraussetzungen
|
||||
- Systemanforderungen
|
||||
- Abhängigkeiten
|
||||
|
||||
## Installationsschritte
|
||||
1. Download von offizieller Quelle
|
||||
2. Installer ausführen
|
||||
3. Einstellungen konfigurieren
|
||||
|
||||
## Häufige Probleme
|
||||
Lösungen für typische Probleme...
|
||||
```
|
||||
|
||||
### Artikel-Struktur-Richtlinien
|
||||
|
||||
**Erforderliche Felder**:
|
||||
- `title`: Klarer, beschreibender Titel
|
||||
- `description`: Einzeilige Zusammenfassung für Auflistungen
|
||||
- `last_updated`: Artikel-Änderungsdatum
|
||||
- `published`: Boolean-Flag für Sichtbarkeit
|
||||
|
||||
**Optionale Felder**:
|
||||
- `tool_name`: Zuordnung zu spezifischem Tool aus Datenbank
|
||||
- `author`: Mitwirkender Name (Standard: "Anon")
|
||||
- `difficulty`: Komplexitätslevel passend zu Tool-Skill-Levels
|
||||
- `categories`: Breite Klassifizierungen
|
||||
- `tags`: Spezifische Stichwörter für Entdeckung
|
||||
- `related_tools`: Array verwandter Tool-Namen
|
||||
|
||||
**Inhalt-Richtlinien**:
|
||||
- Standard-Markdown-Formatierung verwenden
|
||||
- Praktische Beispiele und Code-Snippets einschließen
|
||||
- Screenshots oder Diagramme bei Bedarf hinzufügen
|
||||
- Zu verwandten Tools mit `[Tool Name](/tools/tool-slug)` Format verlinken
|
||||
- Troubleshooting-Abschnitte für komplexe Tools einschließen
|
||||
|
||||
### Automatische Verarbeitung
|
||||
|
||||
1. Artikel werden automatisch beim Build indexiert
|
||||
2. Tool-Zuordnungen erstellen bidirektionale Links
|
||||
3. Suche umfasst Volltext-Inhalt und Metadaten
|
||||
4. Verwandte Artikel erscheinen in Tool-Detail-Ansichten
|
||||
|
||||
## Entwicklung
|
||||
|
||||
```bash
|
||||
# Nur für Tests geeignet
|
||||
AUTHENTICATION_NECESSARY=false
|
||||
PUBLIC_BASE_URL=http://localhost:4321
|
||||
# Setup
|
||||
npm install
|
||||
cp .env.example .env
|
||||
|
||||
# Entwicklung
|
||||
npm run dev
|
||||
|
||||
# Build
|
||||
npm run build
|
||||
|
||||
# Deploy
|
||||
sudo ./deploy.sh
|
||||
```
|
||||
|
||||
### Tools-Datenbank
|
||||
## Konfigurationsübersicht
|
||||
|
||||
Die Tools werden in `src/data/tools.yaml` verwaltet. Vollständiges Beispiel:
|
||||
Die `.env.example`-Datei enthält umfassende Konfigurationsoptionen für alle Features. Die meisten Optionen haben sinnvolle Standardwerte, wobei nur die KI-Service-Konfiguration für volle Funktionalität erforderlich ist.
|
||||
|
||||
```yaml
|
||||
tools:
|
||||
- name: Autopsy
|
||||
type: software # software|method|concept
|
||||
description: >-
|
||||
Die führende Open-Source-Alternative zu kommerziellen Forensik-Suiten mit
|
||||
intuitiver grafischer Oberfläche. Besonders stark in der Timeline-Analyse,
|
||||
Keyword-Suche und dem Carving gelöschter Dateien. Die modulare
|
||||
Plugin-Architektur erlaubt Erweiterungen für spezielle
|
||||
Untersuchungsszenarien.
|
||||
icon: 📦
|
||||
skillLevel: intermediate # novice|beginner|intermediate|advanced|expert
|
||||
url: https://www.autopsy.com/
|
||||
domains:
|
||||
- incident-response
|
||||
- static-investigations
|
||||
- malware-analysis
|
||||
- mobile-forensics
|
||||
- cloud-forensics
|
||||
phases:
|
||||
- examination
|
||||
- analysis
|
||||
platforms:
|
||||
- Windows
|
||||
- Linux
|
||||
related_concepts:
|
||||
- SQL Query Fundamentals
|
||||
- Hash Functions & Digital Signatures
|
||||
accessType: download # download|web|api|cli|service
|
||||
license: Apache 2.0
|
||||
knowledgebase: false # true für erweiterte Dokumentation
|
||||
tags:
|
||||
- gui
|
||||
- filesystem
|
||||
- timeline-analysis
|
||||
- carving
|
||||
- artifact-extraction
|
||||
- keyword-search
|
||||
# Optional: Für gehostete Services
|
||||
projectUrl: https://autopsy.ihre-domain.de
|
||||
statusUrl: https://status.ihre-domain.de/api/badge/1/status
|
||||
|
||||
# Beispiel Methode
|
||||
- name: Live Response Methodology
|
||||
type: method
|
||||
description: >-
|
||||
Strukturierte Vorgehensweise zur Sammlung volatiler Daten
|
||||
von laufenden Systemen ohne Shutdown.
|
||||
icon: 📋
|
||||
skillLevel: advanced
|
||||
url: https://www.sans.org/white-papers/live-response/
|
||||
domains:
|
||||
- incident-response
|
||||
phases:
|
||||
- data-collection
|
||||
related_concepts:
|
||||
- Memory Forensics Fundamentals
|
||||
tags:
|
||||
- volatile-data
|
||||
- live-analysis
|
||||
- methodology
|
||||
knowledgebase: true
|
||||
|
||||
# Beispiel Konzept
|
||||
- name: Hash Functions & Digital Signatures
|
||||
type: concept
|
||||
description: >-
|
||||
Kryptographische Grundlagen für Datenintegrität und
|
||||
Authentifizierung in der digitalen Forensik.
|
||||
icon: 🔐
|
||||
skillLevel: intermediate
|
||||
url: https://en.wikipedia.org/wiki/Cryptographic_hash_function
|
||||
domains:
|
||||
- incident-response
|
||||
- static-investigations
|
||||
- malware-analysis
|
||||
phases:
|
||||
- data-collection
|
||||
- examination
|
||||
tags:
|
||||
- cryptography
|
||||
- data-integrity
|
||||
- evidence-preservation
|
||||
knowledgebase: false
|
||||
## Architektur
|
||||
|
||||
# Konfiguration der Domänen
|
||||
domains:
|
||||
- id: incident-response
|
||||
name: Incident Response & Breach-Untersuchung
|
||||
- id: static-investigations
|
||||
name: Datenträgerforensik & Ermittlungen
|
||||
- id: malware-analysis
|
||||
name: Malware-Analyse & Reverse Engineering
|
||||
- id: mobile-forensics
|
||||
name: Mobile Geräte & App-Forensik
|
||||
- id: cloud-forensics
|
||||
name: Cloud & Virtuelle Umgebungen
|
||||
|
||||
# Konfiguration der Phasen (NIST Framework)
|
||||
phases:
|
||||
- id: data-collection
|
||||
name: Datensammlung
|
||||
description: Imaging, Acquisition, Remote Collection Tools
|
||||
- id: examination
|
||||
name: Auswertung
|
||||
description: Parsing, Extraction, Initial Analysis Tools
|
||||
- id: analysis
|
||||
name: Analyse
|
||||
description: Deep Analysis, Correlation, Visualization Tools
|
||||
- id: reporting
|
||||
name: Bericht & Präsentation
|
||||
description: Documentation, Visualization, Presentation Tools
|
||||
|
||||
# Domänenübergreifende Kategorien
|
||||
domain-agnostic-software:
|
||||
- id: collaboration-general
|
||||
name: Übergreifend & Kollaboration
|
||||
description: Cross-cutting tools and collaboration platforms
|
||||
- id: specific-os
|
||||
name: Betriebssysteme
|
||||
description: Operating Systems which focus on forensics
|
||||
```
|
||||
|
||||
## 📦 Updates
|
||||
|
||||
```bash
|
||||
# Repository aktualisieren
|
||||
cd /opt/forensic-pathways
|
||||
sudo git pull
|
||||
|
||||
# Dependencies aktualisieren
|
||||
sudo npm install
|
||||
|
||||
# Rebuild
|
||||
sudo npm run build
|
||||
|
||||
# Service neustarten
|
||||
sudo systemctl restart forensic-pathways
|
||||
```
|
||||
|
||||
## 💾 Backup
|
||||
|
||||
Wichtige Dateien für Backup:
|
||||
|
||||
```bash
|
||||
/opt/forensic-pathways/src/data/tools.yaml
|
||||
/opt/forensic-pathways/.env
|
||||
/etc/nginx/sites-available/forensic-pathways
|
||||
/etc/systemd/system/forensic-pathways.service
|
||||
```
|
||||
|
||||
## 🤝 Beiträge
|
||||
|
||||
Contributions sind willkommen! Bitte:
|
||||
|
||||
1. Issue im Repository erstellen
|
||||
2. Feature-Branch erstellen
|
||||
3. Pull Request öffnen
|
||||
4. Tests durchführen
|
||||
|
||||
## 📞 Support
|
||||
|
||||
Bei Problemen oder Fragen:
|
||||
|
||||
- **Issues:** [Repository Issues](https://git.cc24.dev/mstoeck3/forensic-pathways/issues)
|
||||
- **Dokumentation:** Siehe `/knowledgebase` auf der Website
|
||||
|
||||
## 📄 Lizenz
|
||||
|
||||
Dieses Projekt steht unter der **BSD-3-Clause** Lizenz.
|
||||
- **Frontend**: Astro mit TypeScript, responsive CSS
|
||||
- **Backend**: Node.js API-Routen mit intelligenter Ratenbegrenzung
|
||||
- **KI-Pipeline**: Micro-Task-Architektur mit Audit-Protokollierung
|
||||
- **Daten**: YAML-basierte Tool-Datenbank mit Git-basierten Beiträgen
|
||||
- **Suche**: Dual-Mode Text- und semantische Vector-Suche
|
||||
- **Auth**: OIDC-Integration mit Session-Management
|
||||
@ -1,389 +0,0 @@
|
||||
// src/js/auditTrailRenderer.js
|
||||
|
||||
import { auditService } from '../../src/utils/auditService.js';
|
||||
|
||||
export class AuditTrailRenderer {
|
||||
constructor(containerId, options = {}) {
|
||||
this.containerId = containerId;
|
||||
this.options = {
|
||||
title: options.title || 'KI-Entscheidungspfad',
|
||||
collapsible: options.collapsible !== false,
|
||||
defaultExpanded: options.defaultExpanded || false,
|
||||
...options
|
||||
};
|
||||
this.componentId = `audit-trail-${Date.now()}-${Math.random().toString(36).substr(2, 6)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render audit trail from raw audit data
|
||||
* FIXED: Proper Promise handling
|
||||
*/
|
||||
render(rawAuditTrail) {
|
||||
const container = document.getElementById(this.containerId);
|
||||
if (!container) {
|
||||
console.error(`[AUDIT RENDERER] Container ${this.containerId} not found`);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!rawAuditTrail || !Array.isArray(rawAuditTrail) || rawAuditTrail.length === 0) {
|
||||
this.renderEmpty();
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
console.log('[AUDIT RENDERER] Processing audit trail...', rawAuditTrail.length, 'entries');
|
||||
|
||||
// Process audit trail using the centralized service (synchronous)
|
||||
const processedAudit = auditService.processAuditTrail(rawAuditTrail);
|
||||
|
||||
console.log('[AUDIT RENDERER] Processed audit:', processedAudit);
|
||||
|
||||
if (processedAudit && processedAudit.phases && processedAudit.phases.length > 0) {
|
||||
this.renderProcessed(processedAudit);
|
||||
// Attach event handlers after DOM is updated
|
||||
setTimeout(() => this.attachEventHandlers(), 0);
|
||||
} else {
|
||||
console.warn('[AUDIT RENDERER] No processed audit data');
|
||||
this.renderEmpty();
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('[AUDIT RENDERER] Failed to render audit trail:', error);
|
||||
this.renderError(error);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Render processed audit trail
|
||||
*/
|
||||
renderProcessed(processedAudit) {
|
||||
const container = document.getElementById(this.containerId);
|
||||
if (!container) return;
|
||||
|
||||
const detailsId = `${this.componentId}-details`;
|
||||
|
||||
console.log('[AUDIT RENDERER] Rendering processed audit with', processedAudit.phases.length, 'phases');
|
||||
|
||||
container.innerHTML = `
|
||||
<div class="audit-trail-container">
|
||||
<div class="audit-trail-header ${this.options.collapsible ? 'clickable' : ''}"
|
||||
${this.options.collapsible ? `data-target="${detailsId}"` : ''}>
|
||||
<div class="audit-trail-title">
|
||||
<div class="audit-icon">
|
||||
<div class="audit-icon-gradient">✓</div>
|
||||
<h4>${this.options.title}</h4>
|
||||
</div>
|
||||
|
||||
<div class="audit-stats">
|
||||
<div class="stat-item">
|
||||
<div class="stat-dot stat-time"></div>
|
||||
<span>${auditService.formatDuration(processedAudit.totalTime)}</span>
|
||||
</div>
|
||||
<div class="stat-item">
|
||||
<div class="stat-dot" style="background-color: ${auditService.getConfidenceColor(processedAudit.avgConfidence)}"></div>
|
||||
<span>${processedAudit.avgConfidence}% Vertrauen</span>
|
||||
</div>
|
||||
<div class="stat-item">
|
||||
<span>${processedAudit.stepCount} Schritte</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
${this.options.collapsible ? `
|
||||
<div class="toggle-icon">
|
||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
||||
<polyline points="6 9 12 15 18 9"/>
|
||||
</svg>
|
||||
</div>
|
||||
` : ''}
|
||||
</div>
|
||||
|
||||
<div id="${detailsId}" class="audit-trail-details ${this.options.collapsible && !this.options.defaultExpanded ? 'collapsed' : ''}">
|
||||
${this.renderSummary(processedAudit)}
|
||||
${this.renderProcessFlow(processedAudit)}
|
||||
${this.renderTechnicalDetails(processedAudit)}
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
|
||||
console.log('[AUDIT RENDERER] HTML rendered successfully');
|
||||
}
|
||||
|
||||
/**
|
||||
* Render audit summary section
|
||||
*/
|
||||
renderSummary(audit) {
|
||||
return `
|
||||
<div class="audit-summary">
|
||||
<div class="summary-header">📊 Analyse-Qualität</div>
|
||||
<div class="summary-grid">
|
||||
<div class="summary-stat">
|
||||
<div class="summary-value success">${audit.highConfidenceSteps}</div>
|
||||
<div class="summary-label">Hohe Sicherheit</div>
|
||||
</div>
|
||||
<div class="summary-stat">
|
||||
<div class="summary-value ${audit.lowConfidenceSteps > 0 ? 'warning' : 'success'}">
|
||||
${audit.lowConfidenceSteps}
|
||||
</div>
|
||||
<div class="summary-label">Unsichere Schritte</div>
|
||||
</div>
|
||||
<div class="summary-stat">
|
||||
<div class="summary-value">${auditService.formatDuration(audit.totalTime)}</div>
|
||||
<div class="summary-label">Verarbeitungszeit</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
${audit.summary.keyInsights && audit.summary.keyInsights.length > 0 ? `
|
||||
<div class="insights-section">
|
||||
<div class="insights-header success">✓ Erkenntnisse:</div>
|
||||
<ul class="insights-list">
|
||||
${audit.summary.keyInsights.map(insight => `<li>${this.escapeHtml(insight)}</li>`).join('')}
|
||||
</ul>
|
||||
</div>
|
||||
` : ''}
|
||||
|
||||
${audit.summary.potentialIssues && audit.summary.potentialIssues.length > 0 ? `
|
||||
<div class="insights-section">
|
||||
<div class="insights-header warning">⚠ Hinweise:</div>
|
||||
<ul class="insights-list">
|
||||
${audit.summary.potentialIssues.map(issue => `<li>${this.escapeHtml(issue)}</li>`).join('')}
|
||||
</ul>
|
||||
</div>
|
||||
` : ''}
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render process flow section
|
||||
*/
|
||||
renderProcessFlow(audit) {
|
||||
if (!audit.phases || audit.phases.length === 0) {
|
||||
return '<div class="audit-process-flow"><p>Keine Phasen verfügbar</p></div>';
|
||||
}
|
||||
|
||||
return `
|
||||
<div class="audit-process-flow">
|
||||
${audit.phases.map((phase, index) => `
|
||||
<div class="phase-group ${index === audit.phases.length - 1 ? 'last-phase' : ''}">
|
||||
<div class="phase-header">
|
||||
<div class="phase-info">
|
||||
<span class="phase-icon">${phase.icon || '📋'}</span>
|
||||
<span class="phase-name">${phase.displayName || phase.name}</span>
|
||||
</div>
|
||||
<div class="phase-divider"></div>
|
||||
<div class="phase-stats">
|
||||
<div class="confidence-bar">
|
||||
<div class="confidence-fill"
|
||||
style="width: ${phase.avgConfidence || 0}%; background-color: ${auditService.getConfidenceColor(phase.avgConfidence || 0)}">
|
||||
</div>
|
||||
</div>
|
||||
<span class="confidence-text">${phase.avgConfidence || 0}%</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="phase-entries">
|
||||
${(phase.entries || []).map(entry => `
|
||||
<div class="audit-entry">
|
||||
<div class="entry-main">
|
||||
<span class="entry-action">${auditService.getActionDisplayName(entry.action)}</span>
|
||||
<div class="entry-meta">
|
||||
<div class="confidence-indicator"
|
||||
style="background-color: ${auditService.getConfidenceColor(entry.confidence || 0)}">
|
||||
</div>
|
||||
<span class="confidence-value">${entry.confidence || 0}%</span>
|
||||
<span class="processing-time">${entry.processingTimeMs || 0}ms</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
${(entry.inputSummary && entry.inputSummary !== 'null') || (entry.outputSummary && entry.outputSummary !== 'null') ? `
|
||||
<div class="entry-details">
|
||||
${entry.inputSummary && entry.inputSummary !== 'null' ? `
|
||||
<div class="detail-item"><strong>Input:</strong> ${this.escapeHtml(entry.inputSummary)}</div>
|
||||
` : ''}
|
||||
${entry.outputSummary && entry.outputSummary !== 'null' ? `
|
||||
<div class="detail-item"><strong>Output:</strong> ${this.escapeHtml(entry.outputSummary)}</div>
|
||||
` : ''}
|
||||
</div>
|
||||
` : ''}
|
||||
</div>
|
||||
`).join('')}
|
||||
</div>
|
||||
</div>
|
||||
`).join('')}
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render technical details section
|
||||
*/
|
||||
renderTechnicalDetails(audit) {
|
||||
const technicalId = `${this.componentId}-technical`;
|
||||
|
||||
return `
|
||||
<div class="technical-toggle">
|
||||
<button class="technical-toggle-btn" data-target="${technicalId}">
|
||||
🔧 Technische Details anzeigen
|
||||
</button>
|
||||
<div id="${technicalId}" class="technical-details collapsed">
|
||||
${(audit.phases || []).map(phase =>
|
||||
(phase.entries || []).map(entry => `
|
||||
<div class="technical-entry">
|
||||
<div class="technical-header">
|
||||
<span class="technical-phase">${entry.phase}/${entry.action}</span>
|
||||
<span class="technical-time">
|
||||
${new Date(entry.timestamp).toLocaleTimeString('de-DE', {
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
second: '2-digit'
|
||||
})} • ${entry.processingTimeMs || 0}ms
|
||||
</span>
|
||||
</div>
|
||||
<div class="technical-content">
|
||||
<div class="technical-row">
|
||||
<strong>Confidence:</strong> ${entry.confidence || 0}%
|
||||
</div>
|
||||
${entry.metadata && Object.keys(entry.metadata).length > 0 ? `
|
||||
<div class="technical-row">
|
||||
<strong>Metadata:</strong> ${this.escapeHtml(JSON.stringify(entry.metadata))}
|
||||
</div>
|
||||
` : ''}
|
||||
</div>
|
||||
</div>
|
||||
`).join('')
|
||||
).join('')}
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Attach event handlers for interactions
|
||||
*/
|
||||
attachEventHandlers() {
|
||||
console.log('[AUDIT RENDERER] Attaching event handlers...');
|
||||
|
||||
// Handle collapsible header
|
||||
if (this.options.collapsible) {
|
||||
const header = document.querySelector(`[data-target="${this.componentId}-details"]`);
|
||||
const details = document.getElementById(`${this.componentId}-details`);
|
||||
const toggleIcon = header?.querySelector('.toggle-icon svg');
|
||||
|
||||
if (header && details && toggleIcon) {
|
||||
// Remove existing listeners
|
||||
header.replaceWith(header.cloneNode(true));
|
||||
const newHeader = document.querySelector(`[data-target="${this.componentId}-details"]`);
|
||||
const newToggleIcon = newHeader?.querySelector('.toggle-icon svg');
|
||||
|
||||
if (newHeader && newToggleIcon) {
|
||||
newHeader.addEventListener('click', () => {
|
||||
const isCollapsed = details.classList.contains('collapsed');
|
||||
|
||||
if (isCollapsed) {
|
||||
details.classList.remove('collapsed');
|
||||
newToggleIcon.style.transform = 'rotate(180deg)';
|
||||
} else {
|
||||
details.classList.add('collapsed');
|
||||
newToggleIcon.style.transform = 'rotate(0deg)';
|
||||
}
|
||||
});
|
||||
console.log('[AUDIT RENDERER] Collapsible header handler attached');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Handle technical details toggle
|
||||
const technicalBtn = document.querySelector(`[data-target="${this.componentId}-technical"]`);
|
||||
const technicalDetails = document.getElementById(`${this.componentId}-technical`);
|
||||
|
||||
if (technicalBtn && technicalDetails) {
|
||||
// Remove existing listener
|
||||
technicalBtn.replaceWith(technicalBtn.cloneNode(true));
|
||||
const newTechnicalBtn = document.querySelector(`[data-target="${this.componentId}-technical"]`);
|
||||
|
||||
if (newTechnicalBtn) {
|
||||
newTechnicalBtn.addEventListener('click', () => {
|
||||
const isCollapsed = technicalDetails.classList.contains('collapsed');
|
||||
|
||||
if (isCollapsed) {
|
||||
technicalDetails.classList.remove('collapsed');
|
||||
newTechnicalBtn.textContent = '🔧 Technische Details ausblenden';
|
||||
} else {
|
||||
technicalDetails.classList.add('collapsed');
|
||||
newTechnicalBtn.textContent = '🔧 Technische Details anzeigen';
|
||||
}
|
||||
});
|
||||
console.log('[AUDIT RENDERER] Technical details handler attached');
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render empty state
|
||||
*/
|
||||
renderEmpty() {
|
||||
const container = document.getElementById(this.containerId);
|
||||
if (container) {
|
||||
container.innerHTML = `
|
||||
<div class="audit-trail-container">
|
||||
<div class="audit-trail-header">
|
||||
<div class="audit-icon">
|
||||
<div class="audit-icon-gradient">ⓘ</div>
|
||||
<h4>Kein Audit-Trail verfügbar</h4>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Render error state
|
||||
*/
|
||||
renderError(error) {
|
||||
const container = document.getElementById(this.containerId);
|
||||
if (container) {
|
||||
container.innerHTML = `
|
||||
<div class="audit-trail-container">
|
||||
<div class="audit-trail-header">
|
||||
<div class="audit-icon">
|
||||
<div class="audit-icon-gradient" style="background: var(--color-error);">✗</div>
|
||||
<h4>Audit-Trail Fehler</h4>
|
||||
</div>
|
||||
</div>
|
||||
<div class="audit-summary">
|
||||
<p style="color: var(--color-error);">
|
||||
Fehler beim Laden der Audit-Informationen: ${this.escapeHtml(error.message)}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Utility method to escape HTML
|
||||
*/
|
||||
escapeHtml(text) {
|
||||
if (typeof text !== 'string') return String(text);
|
||||
const div = document.createElement('div');
|
||||
div.textContent = text;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear the audit trail display
|
||||
*/
|
||||
clear() {
|
||||
const container = document.getElementById(this.containerId);
|
||||
if (container) {
|
||||
container.innerHTML = '';
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get container element
|
||||
*/
|
||||
getContainer() {
|
||||
return document.getElementById(this.containerId);
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@ -354,7 +354,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
let semanticSearchAvailable = false;
|
||||
let lastSemanticResults = null;
|
||||
|
||||
// Check embeddings availability
|
||||
async function checkEmbeddingsAvailability() {
|
||||
try {
|
||||
const res = await fetch('/api/ai/embeddings-status');
|
||||
@ -363,15 +362,13 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
|
||||
if (semanticSearchAvailable) {
|
||||
elements.semanticContainer.classList.remove('hidden');
|
||||
elements.semanticCheckbox.disabled = false; // 👈 re-enable
|
||||
elements.semanticCheckbox.disabled = false;
|
||||
}
|
||||
} catch (err) {
|
||||
console.error('[EMBEDDINGS] Status check failed:', err);
|
||||
// leave the checkbox disabled
|
||||
}
|
||||
}
|
||||
|
||||
// Semantic search function
|
||||
async function performSemanticSearch(query) {
|
||||
if (!semanticSearchAvailable || !query.trim()) {
|
||||
return null;
|
||||
@ -576,7 +573,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
}
|
||||
}
|
||||
|
||||
// FIXED: Consolidated filtering logic with semantic search support
|
||||
async function filterTools() {
|
||||
const searchTerm = elements.searchInput.value.trim().toLowerCase();
|
||||
const selectedDomain = elements.domainSelect.value;
|
||||
@ -594,7 +590,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
let filteredTools = window.toolsData;
|
||||
let semanticResults = null;
|
||||
|
||||
// CONSOLIDATED: Use semantic search if enabled and search term exists
|
||||
if (semanticSearchEnabled && semanticSearchAvailable && searchTerm) {
|
||||
semanticResults = await performSemanticSearch(searchTerm);
|
||||
lastSemanticResults = semanticResults;
|
||||
@ -605,7 +600,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
} else {
|
||||
lastSemanticResults = null;
|
||||
|
||||
// Traditional text-based search
|
||||
if (searchTerm) {
|
||||
filteredTools = window.toolsData.filter(tool =>
|
||||
tool.name.toLowerCase().includes(searchTerm) ||
|
||||
@ -615,7 +609,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
}
|
||||
}
|
||||
|
||||
// Apply additional filters to the results
|
||||
filteredTools = filteredTools.filter(tool => {
|
||||
if (selectedDomain && !(tool.domains || []).includes(selectedDomain)) {
|
||||
return false;
|
||||
@ -666,9 +659,8 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
);
|
||||
}
|
||||
|
||||
/* existing code continues */
|
||||
const finalResults = semanticSearchEnabled && lastSemanticResults
|
||||
? filteredTools // now properly re-sorted
|
||||
? filteredTools
|
||||
: (searchTerm && window.prioritizeSearchResults
|
||||
? window.prioritizeSearchResults(filteredTools, searchTerm)
|
||||
: filteredTools);
|
||||
@ -726,7 +718,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
filterTagCloud();
|
||||
}
|
||||
|
||||
// Event listeners
|
||||
elements.searchInput.addEventListener('input', (e) => {
|
||||
const hasValue = e.target.value.length > 0;
|
||||
elements.clearSearch.classList.toggle('hidden', !hasValue);
|
||||
@ -741,7 +732,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
filterTools();
|
||||
});
|
||||
|
||||
// Semantic search checkbox handler
|
||||
if (elements.semanticCheckbox) {
|
||||
elements.semanticCheckbox.addEventListener('change', (e) => {
|
||||
semanticSearchEnabled = e.target.checked;
|
||||
@ -812,7 +802,6 @@ const sortedTags = Object.entries(tagFrequency)
|
||||
window.clearTagFilters = resetTags;
|
||||
window.clearAllFilters = resetAllFilters;
|
||||
|
||||
// Initialize
|
||||
checkEmbeddingsAvailability();
|
||||
initializeCollapsible();
|
||||
initTagCloud();
|
||||
|
||||
@ -20,7 +20,7 @@ const AI_ANALYZER_API_KEY = getEnv('AI_ANALYZER_API_KEY');
|
||||
const AI_ANALYZER_MODEL = getEnv('AI_ANALYZER_MODEL');
|
||||
|
||||
const rateLimitStore = new Map<string, { count: number; resetTime: number }>();
|
||||
const RATE_LIMIT_WINDOW = 60 * 1000; // 1 minute
|
||||
const RATE_LIMIT_WINDOW = 60 * 1000;
|
||||
const RATE_LIMIT_MAX = 5;
|
||||
|
||||
function sanitizeInput(input: string): string {
|
||||
|
||||
@ -198,15 +198,12 @@ const phases = data.phases;
|
||||
<script define:vars={{ toolsData: data.tools, phases: data.phases }}>
|
||||
window.toolsData = toolsData;
|
||||
|
||||
// CONSOLIDATED: Approach selection - Pure navigation aid
|
||||
window.selectApproach = function(approach) {
|
||||
console.log(`Selected approach: ${approach}`);
|
||||
|
||||
// Clear any existing AI results
|
||||
const aiResults = document.getElementById('ai-results');
|
||||
if (aiResults) aiResults.style.display = 'none';
|
||||
|
||||
// Update visual selection state
|
||||
document.querySelectorAll('.approach-card').forEach(card => {
|
||||
card.classList.remove('selected');
|
||||
});
|
||||
@ -214,14 +211,12 @@ const phases = data.phases;
|
||||
const selectedCard = document.querySelector(`.approach-card.${approach}`);
|
||||
if (selectedCard) selectedCard.classList.add('selected');
|
||||
|
||||
// Hide all approach sections first (ensures mutual exclusivity)
|
||||
const methodologySection = document.getElementById('methodology-section');
|
||||
const targetedSection = document.getElementById('targeted-section');
|
||||
|
||||
if (methodologySection) methodologySection.classList.remove('active');
|
||||
if (targetedSection) targetedSection.classList.remove('active');
|
||||
|
||||
// Show the selected approach section (navigation aid only)
|
||||
if (approach === 'methodology') {
|
||||
if (methodologySection) {
|
||||
methodologySection.classList.add('active');
|
||||
@ -235,11 +230,9 @@ const phases = data.phases;
|
||||
}
|
||||
};
|
||||
|
||||
// CONSOLIDATED: Phase selection - Sets unified filter dropdown
|
||||
window.selectPhase = function(phase) {
|
||||
console.log(`Selected NIST phase: ${phase}`);
|
||||
|
||||
// Update visual selection of phase cards
|
||||
document.querySelectorAll('.phase-card').forEach(card => {
|
||||
card.classList.remove('active');
|
||||
});
|
||||
@ -249,23 +242,19 @@ const phases = data.phases;
|
||||
selectedCard.classList.add('active');
|
||||
}
|
||||
|
||||
// CONSOLIDATED: Set the unified phase-select dropdown
|
||||
const phaseSelect = document.getElementById('phase-select');
|
||||
if (phaseSelect) {
|
||||
phaseSelect.value = phase;
|
||||
|
||||
// Trigger the change event to activate unified filtering
|
||||
const changeEvent = new Event('change', { bubbles: true });
|
||||
phaseSelect.dispatchEvent(changeEvent);
|
||||
}
|
||||
|
||||
// Switch to grid view to show filtered results
|
||||
const gridToggle = document.querySelector('.view-toggle[data-view="grid"]');
|
||||
if (gridToggle && !gridToggle.classList.contains('active')) {
|
||||
gridToggle.click();
|
||||
}
|
||||
|
||||
// Scroll to filtered results
|
||||
setTimeout(() => {
|
||||
window.scrollToElementById('tools-grid');
|
||||
}, 200);
|
||||
@ -309,17 +298,14 @@ const phases = data.phases;
|
||||
const filtersSection = document.getElementById('filters-section');
|
||||
const noResults = document.getElementById('no-results');
|
||||
|
||||
// FIXED: Hide approach sections when switching to ANY view mode
|
||||
const methodologySection = document.getElementById('methodology-section');
|
||||
const targetedSection = document.getElementById('targeted-section');
|
||||
|
||||
// Hide all main content areas
|
||||
if (toolsGrid) toolsGrid.style.display = 'none';
|
||||
if (matrixContainer) matrixContainer.style.display = 'none';
|
||||
if (aiInterface) aiInterface.style.display = 'none';
|
||||
if (noResults) noResults.style.display = 'none';
|
||||
|
||||
// FIXED: Hide approach sections when switching to view modes
|
||||
if (methodologySection) methodologySection.classList.remove('active');
|
||||
if (targetedSection) targetedSection.classList.remove('active');
|
||||
|
||||
@ -335,18 +321,14 @@ const phases = data.phases;
|
||||
case 'ai':
|
||||
if (aiInterface) aiInterface.style.display = 'block';
|
||||
|
||||
// FIXED: Show filters but hide everything except view controls
|
||||
if (filtersSection) {
|
||||
filtersSection.style.display = 'block';
|
||||
|
||||
// Hide all filter sections except the last one (view controls)
|
||||
const filterSections = filtersSection.querySelectorAll('.filter-section');
|
||||
filterSections.forEach((section, index) => {
|
||||
if (index === filterSections.length - 1) {
|
||||
// Keep view controls visible
|
||||
section.style.display = 'block';
|
||||
} else {
|
||||
// Hide other filter sections
|
||||
section.style.display = 'none';
|
||||
}
|
||||
});
|
||||
@ -354,7 +336,6 @@ const phases = data.phases;
|
||||
break;
|
||||
}
|
||||
|
||||
// FIXED: Reset filter sections visibility when not in AI view
|
||||
if (view !== 'ai' && filtersSection) {
|
||||
const filterSections = filtersSection.querySelectorAll('.filter-section');
|
||||
filterSections.forEach(section => {
|
||||
@ -363,7 +344,6 @@ const phases = data.phases;
|
||||
}
|
||||
}
|
||||
|
||||
// Navigation functions for AI recommendations (unchanged)
|
||||
window.navigateToGrid = function(toolName) {
|
||||
console.log('Navigating to grid for tool:', toolName);
|
||||
|
||||
@ -485,8 +465,6 @@ const phases = data.phases;
|
||||
}, 100);
|
||||
}
|
||||
|
||||
// REPLACE the existing toolsFiltered event listener in index.astro with this enhanced version:
|
||||
|
||||
window.addEventListener('toolsFiltered', (event) => {
|
||||
const { tools: filtered, semanticSearch } = event.detail;
|
||||
const currentView = document.querySelector('.view-toggle.active')?.getAttribute('data-view');
|
||||
@ -504,11 +482,9 @@ const phases = data.phases;
|
||||
if (semanticSearch && filtered.length > 0) {
|
||||
console.log('[SEMANTIC] Reordering tools by semantic similarity');
|
||||
|
||||
// FIXED: Create ordered array of cards based on semantic similarity
|
||||
const orderedCards = [];
|
||||
const remainingCards = [];
|
||||
|
||||
// First pass: collect cards in semantic order
|
||||
filtered.forEach(tool => {
|
||||
const toolName = tool.name.toLowerCase();
|
||||
const matchingCard = Array.from(allToolCards).find(card =>
|
||||
@ -520,12 +496,10 @@ const phases = data.phases;
|
||||
orderedCards.push(matchingCard);
|
||||
visibleCount++;
|
||||
|
||||
// Add semantic indicators if available
|
||||
if (tool._semanticSimilarity) {
|
||||
matchingCard.setAttribute('data-semantic-similarity', tool._semanticSimilarity.toFixed(3));
|
||||
matchingCard.setAttribute('data-semantic-rank', tool._semanticRank || '');
|
||||
|
||||
// Visual indication of semantic ranking (subtle)
|
||||
const header = matchingCard.querySelector('.tool-card-header h3');
|
||||
if (header && tool._semanticRank <= 3) {
|
||||
const existingIndicator = header.querySelector('.semantic-rank-indicator');
|
||||
@ -551,7 +525,6 @@ const phases = data.phases;
|
||||
}
|
||||
});
|
||||
|
||||
// Second pass: hide non-matching cards and collect them
|
||||
allToolCards.forEach(card => {
|
||||
const toolName = card.getAttribute('data-tool-name');
|
||||
if (!filteredNames.has(toolName)) {
|
||||
@ -560,18 +533,15 @@ const phases = data.phases;
|
||||
}
|
||||
});
|
||||
|
||||
// Reorder DOM: semantic results first, then hidden cards
|
||||
const allCards = [...orderedCards, ...remainingCards];
|
||||
allCards.forEach(card => {
|
||||
toolsContainer.appendChild(card);
|
||||
});
|
||||
|
||||
} else {
|
||||
// FIXED: Standard filtering without semantic ordering
|
||||
allToolCards.forEach(card => {
|
||||
const toolName = card.getAttribute('data-tool-name');
|
||||
|
||||
// Clean up any semantic indicators
|
||||
card.removeAttribute('data-semantic-similarity');
|
||||
card.removeAttribute('data-semantic-rank');
|
||||
const semanticIndicator = card.querySelector('.semantic-rank-indicator');
|
||||
@ -587,10 +557,8 @@ const phases = data.phases;
|
||||
}
|
||||
});
|
||||
|
||||
// Restore original order when not using semantic search
|
||||
if (!semanticSearch) {
|
||||
const originalOrder = Array.from(allToolCards).sort((a, b) => {
|
||||
// Get original indices from data attributes or DOM order
|
||||
const aIndex = Array.from(allToolCards).indexOf(a);
|
||||
const bIndex = Array.from(allToolCards).indexOf(b);
|
||||
return aIndex - bIndex;
|
||||
@ -602,14 +570,12 @@ const phases = data.phases;
|
||||
}
|
||||
}
|
||||
|
||||
// Show/hide no results message
|
||||
if (visibleCount === 0) {
|
||||
noResults.style.display = 'block';
|
||||
} else {
|
||||
noResults.style.display = 'none';
|
||||
}
|
||||
|
||||
// Log semantic search info
|
||||
if (semanticSearch) {
|
||||
console.log(`[SEMANTIC] Displayed ${visibleCount} tools in semantic order`);
|
||||
}
|
||||
|
||||
@ -1,10 +1,10 @@
|
||||
// src/utils/aiPipeline.ts - Enhanced with Proper Confidence Scoring
|
||||
// src/utils/aiPipeline.ts
|
||||
|
||||
import { getCompressedToolsDataForAI } from './dataService.js';
|
||||
import { embeddingsService, type EmbeddingData, type SimilarityResult } from './embeddings.js';
|
||||
import { AI_PROMPTS, getPrompt } from '../config/prompts.js';
|
||||
import { isToolHosted } from './toolHelpers.js';
|
||||
import { auditService } from './auditService.js'; // Add this import
|
||||
import { auditService } from './auditService.js';
|
||||
|
||||
interface AIConfig {
|
||||
endpoint: string;
|
||||
@ -188,7 +188,6 @@ class ImprovedMicroTaskAIPipeline {
|
||||
metadata
|
||||
};
|
||||
|
||||
// Add to context audit trail instead of temp storage
|
||||
if (!context.auditTrail) {
|
||||
context.auditTrail = [];
|
||||
}
|
||||
@ -209,25 +208,6 @@ class ImprovedMicroTaskAIPipeline {
|
||||
|
||||
console.log('[AUDIT] Merged temporary entries into context');
|
||||
}
|
||||
/**
|
||||
private summarizeForAudit(data: any): any {
|
||||
if (this.auditConfig.detailLevel === 'minimal') {
|
||||
if (typeof data === 'string' && data.length > 100) {
|
||||
return data.slice(0, 100) + '...[truncated]';
|
||||
}
|
||||
if (Array.isArray(data) && data.length > 3) {
|
||||
return [...data.slice(0, 3), `...[${data.length - 3} more items]`];
|
||||
}
|
||||
} else if (this.auditConfig.detailLevel === 'standard') {
|
||||
if (typeof data === 'string' && data.length > 500) {
|
||||
return data.slice(0, 500) + '...[truncated]';
|
||||
}
|
||||
if (Array.isArray(data) && data.length > 10) {
|
||||
return [...data.slice(0, 10), `...[${data.length - 10} more items]`];
|
||||
}
|
||||
}
|
||||
return data;
|
||||
}**/
|
||||
|
||||
private calculateSelectionConfidence(result: any, candidateCount: number): number {
|
||||
if (!result || !result.selectedTools) return 30;
|
||||
@ -388,7 +368,6 @@ class ImprovedMicroTaskAIPipeline {
|
||||
|
||||
context.embeddingsSimilarities = new Map<string, number>();
|
||||
|
||||
// Always try to initialize embeddings - let the service decide if it should be enabled
|
||||
try {
|
||||
console.log('[AI PIPELINE] Attempting embeddings initialization...');
|
||||
await embeddingsService.waitForInitialization();
|
||||
@ -1194,7 +1173,6 @@ ${JSON.stringify(conceptsToSend, null, 2)}`;
|
||||
})) || []
|
||||
};
|
||||
|
||||
// Process audit trail before returning
|
||||
const processedAuditTrail = this.auditConfig.enabled && context.auditTrail
|
||||
? context.auditTrail
|
||||
: [];
|
||||
@ -1237,7 +1215,7 @@ ${JSON.stringify(conceptsToSend, null, 2)}`;
|
||||
...base,
|
||||
recommended_tools: recommendedToolsWithConfidence,
|
||||
workflow_suggestion: finalContent,
|
||||
auditTrail: processedAuditTrail // Always include audit trail array
|
||||
auditTrail: processedAuditTrail
|
||||
};
|
||||
} else {
|
||||
const recommendedToolsWithConfidence = context.selectedTools?.map(st => {
|
||||
@ -1279,7 +1257,7 @@ ${JSON.stringify(conceptsToSend, null, 2)}`;
|
||||
...base,
|
||||
recommended_tools: recommendedToolsWithConfidence,
|
||||
additional_considerations: finalContent,
|
||||
auditTrail: processedAuditTrail // Always include audit trail array
|
||||
auditTrail: processedAuditTrail
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,12 +1,10 @@
|
||||
// src/utils/auditService.ts - Centralized Audit Trail Management
|
||||
// src/utils/auditService.ts
|
||||
|
||||
|
||||
function env(key: string, fallback: string | undefined = undefined): string | undefined {
|
||||
// during dev/server-side rendering
|
||||
if (typeof process !== 'undefined' && process.env?.[key] !== undefined) {
|
||||
return process.env[key];
|
||||
}
|
||||
// during client build / browser
|
||||
if (typeof import.meta !== 'undefined' && (import.meta as any).env?.[key] !== undefined) {
|
||||
return (import.meta as any).env[key];
|
||||
}
|
||||
@ -67,7 +65,6 @@ class AuditService {
|
||||
private config: AuditConfig;
|
||||
private tempEntries: AuditEntry[] = [];
|
||||
|
||||
// Phase configuration with German translations
|
||||
private readonly phaseConfig = {
|
||||
'initialization': { icon: '🚀', displayName: 'Initialisierung' },
|
||||
'retrieval': { icon: '🔍', displayName: 'Datensuche' },
|
||||
@ -77,7 +74,6 @@ class AuditService {
|
||||
'completion': { icon: '✅', displayName: 'Finalisierung' }
|
||||
};
|
||||
|
||||
// Action translations
|
||||
private readonly actionTranslations = {
|
||||
'pipeline-start': 'Analyse gestartet',
|
||||
'embeddings-search': 'Ähnliche Tools gesucht',
|
||||
@ -95,7 +91,6 @@ class AuditService {
|
||||
}
|
||||
|
||||
private loadConfig(): AuditConfig {
|
||||
// use the helper if you added it
|
||||
const enabledFlag =
|
||||
(typeof import.meta !== 'undefined' &&
|
||||
(import.meta as any).env?.PUBLIC_FORENSIC_AUDIT_ENABLED) ?? 'false';
|
||||
@ -172,7 +167,6 @@ class AuditService {
|
||||
try {
|
||||
console.log('[AUDIT] Processing', rawAuditTrail.length, 'audit entries');
|
||||
|
||||
// Calculate summary statistics with safe defaults
|
||||
const totalTime = rawAuditTrail.reduce((sum, entry) => sum + (entry.processingTimeMs || 0), 0);
|
||||
const validConfidenceEntries = rawAuditTrail.filter(entry => typeof entry.confidence === 'number');
|
||||
const avgConfidence = validConfidenceEntries.length > 0
|
||||
@ -182,7 +176,6 @@ class AuditService {
|
||||
const highConfidenceSteps = rawAuditTrail.filter(entry => (entry.confidence || 0) >= 80).length;
|
||||
const lowConfidenceSteps = rawAuditTrail.filter(entry => (entry.confidence || 0) < 60).length;
|
||||
|
||||
// Group entries by phase with safe handling
|
||||
const groupedEntries = rawAuditTrail.reduce((groups, entry) => {
|
||||
const phase = entry.phase || 'unknown';
|
||||
if (!groups[phase]) groups[phase] = [];
|
||||
@ -190,7 +183,6 @@ class AuditService {
|
||||
return groups;
|
||||
}, {} as Record<string, AuditEntry[]>);
|
||||
|
||||
// Process phases with error handling
|
||||
const phases = Object.entries(groupedEntries).map(([phase, entries]) => {
|
||||
const phaseConfig = this.phaseConfig[phase] || { icon: '📋', displayName: phase };
|
||||
const validEntries = entries.filter(entry => entry && typeof entry === 'object');
|
||||
@ -211,9 +203,8 @@ class AuditService {
|
||||
.map(e => this.compressEntry(e))
|
||||
.filter((e): e is CompressedAuditEntry => e !== null)
|
||||
};
|
||||
}).filter(phase => phase.entries.length > 0); // Only include phases with valid entries
|
||||
}).filter(phase => phase.entries.length > 0);
|
||||
|
||||
// Generate analysis summary
|
||||
const summary = this.generateSummary(rawAuditTrail, avgConfidence, lowConfidenceSteps);
|
||||
|
||||
const result: ProcessedAuditTrail = {
|
||||
@ -261,12 +252,9 @@ class AuditService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Compress data based on detail level
|
||||
*/
|
||||
private compressData(data: any): any {
|
||||
if (this.config.detailLevel === 'verbose') {
|
||||
return data; // Keep full data
|
||||
return data;
|
||||
} else if (this.config.detailLevel === 'standard') {
|
||||
return this.summarizeForStorage(data);
|
||||
} else {
|
||||
@ -274,9 +262,6 @@ class AuditService {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Summarize data for display purposes
|
||||
*/
|
||||
private summarizeData(data: any): string {
|
||||
if (data === null || data === undefined) return 'null';
|
||||
if (typeof data === 'string') {
|
||||
@ -327,15 +312,11 @@ class AuditService {
|
||||
return data;
|
||||
}
|
||||
|
||||
/**
|
||||
* Generate analysis summary
|
||||
*/
|
||||
private generateSummary(entries: AuditEntry[], avgConfidence: number, lowConfidenceSteps: number): {
|
||||
analysisQuality: 'excellent' | 'good' | 'fair' | 'poor';
|
||||
keyInsights: string[];
|
||||
potentialIssues: string[];
|
||||
} {
|
||||
// Determine analysis quality
|
||||
let analysisQuality: 'excellent' | 'good' | 'fair' | 'poor';
|
||||
if (avgConfidence >= 85 && lowConfidenceSteps === 0) {
|
||||
analysisQuality = 'excellent';
|
||||
@ -347,7 +328,6 @@ class AuditService {
|
||||
analysisQuality = 'poor';
|
||||
}
|
||||
|
||||
// Generate key insights
|
||||
const keyInsights: string[] = [];
|
||||
const embeddingsUsed = entries.some(e => e.action === 'embeddings-search');
|
||||
if (embeddingsUsed) {
|
||||
@ -362,7 +342,6 @@ class AuditService {
|
||||
}
|
||||
}
|
||||
|
||||
// Identify potential issues
|
||||
const potentialIssues: string[] = [];
|
||||
if (lowConfidenceSteps > 2) {
|
||||
potentialIssues.push(`${lowConfidenceSteps} Analyseschritte mit niedriger Konfidenz`);
|
||||
@ -380,16 +359,10 @@ class AuditService {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Get translated action name
|
||||
*/
|
||||
getActionDisplayName(action: string): string {
|
||||
return this.actionTranslations[action] || action;
|
||||
}
|
||||
|
||||
/**
|
||||
* Format duration for display
|
||||
*/
|
||||
formatDuration(ms: number): string {
|
||||
if (ms < 1000) return '< 1s';
|
||||
if (ms < 60000) return `${Math.ceil(ms / 1000)}s`;
|
||||
@ -398,30 +371,20 @@ class AuditService {
|
||||
return seconds > 0 ? `${minutes}m ${seconds}s` : `${minutes}m`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get confidence color for UI
|
||||
*/
|
||||
getConfidenceColor(confidence: number): string {
|
||||
if (confidence >= 80) return 'var(--color-accent)';
|
||||
if (confidence >= 60) return 'var(--color-warning)';
|
||||
return 'var(--color-error)';
|
||||
}
|
||||
|
||||
/**
|
||||
* Check if audit is enabled
|
||||
*/
|
||||
isEnabled(): boolean {
|
||||
return this.config.enabled;
|
||||
}
|
||||
|
||||
/**
|
||||
* Get current configuration
|
||||
*/
|
||||
getConfig(): AuditConfig {
|
||||
return { ...this.config };
|
||||
}
|
||||
}
|
||||
|
||||
// Export singleton instance
|
||||
export const auditService = new AuditService();
|
||||
export type { ProcessedAuditTrail, CompressedAuditEntry };
|
||||
@ -37,19 +37,17 @@ class EmbeddingsService {
|
||||
private readonly embeddingsPath = path.join(process.cwd(), 'data', 'embeddings.json');
|
||||
private readonly batchSize: number;
|
||||
private readonly batchDelay: number;
|
||||
private enabled: boolean = false; // Make mutable again
|
||||
private enabled: boolean = false;
|
||||
|
||||
constructor() {
|
||||
this.batchSize = parseInt(process.env.AI_EMBEDDINGS_BATCH_SIZE || '20', 10);
|
||||
this.batchDelay = parseInt(process.env.AI_EMBEDDINGS_BATCH_DELAY_MS || '1000', 10);
|
||||
|
||||
// Don't call async method from constructor - handle in initialize() instead
|
||||
this.enabled = true; // Start optimistically enabled for development
|
||||
this.enabled = true;
|
||||
}
|
||||
|
||||
private async checkEnabledStatus(): Promise<void> {
|
||||
try {
|
||||
// Add debugging to see what's actually in process.env
|
||||
console.log('[EMBEDDINGS] Debug env check:', {
|
||||
AI_EMBEDDINGS_ENABLED: process.env.AI_EMBEDDINGS_ENABLED,
|
||||
envKeys: Object.keys(process.env).filter(k => k.includes('EMBEDDINGS')).length,
|
||||
@ -59,7 +57,6 @@ class EmbeddingsService {
|
||||
const envEnabled = process.env.AI_EMBEDDINGS_ENABLED;
|
||||
|
||||
if (envEnabled === 'true') {
|
||||
// Check if we have the required API configuration
|
||||
const endpoint = process.env.AI_EMBEDDINGS_ENDPOINT;
|
||||
const model = process.env.AI_EMBEDDINGS_MODEL;
|
||||
|
||||
@ -74,7 +71,6 @@ class EmbeddingsService {
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if embeddings file exists
|
||||
try {
|
||||
await fs.stat(this.embeddingsPath);
|
||||
console.log('[EMBEDDINGS] Existing embeddings file found - enabling');
|
||||
@ -103,7 +99,6 @@ class EmbeddingsService {
|
||||
}
|
||||
|
||||
private async performInitialization(): Promise<void> {
|
||||
// 1️⃣ Respect the on/off switch that the newer code introduced
|
||||
await this.checkEnabledStatus();
|
||||
if (!this.enabled) {
|
||||
console.log('[EMBEDDINGS] Embeddings disabled, skipping initialization');
|
||||
@ -114,14 +109,11 @@ class EmbeddingsService {
|
||||
try {
|
||||
console.log('[EMBEDDINGS] Initializing embeddings system…');
|
||||
|
||||
// Make sure the data folder exists
|
||||
await fs.mkdir(path.dirname(this.embeddingsPath), { recursive: true });
|
||||
|
||||
// Load current tools / concepts and generate a hash
|
||||
const toolsData = await getCompressedToolsDataForAI();
|
||||
const currentDataHash = await this.hashToolsFile(); // <- keep the old helper
|
||||
const currentDataHash = await this.hashToolsFile();
|
||||
|
||||
// Try to read an existing file
|
||||
const existing = await this.loadEmbeddings();
|
||||
console.log('[EMBEDDINGS] Current hash:', currentDataHash);
|
||||
console.log('[EMBEDDINGS] Existing file version:', existing?.version);
|
||||
@ -138,8 +130,7 @@ class EmbeddingsService {
|
||||
this.embeddings = existing.embeddings;
|
||||
} else {
|
||||
console.log('[EMBEDDINGS] Generating new embeddings…');
|
||||
// 2️⃣ Build and persist new vectors
|
||||
await this.generateEmbeddings(toolsData, currentDataHash); // <- old helper
|
||||
await this.generateEmbeddings(toolsData, currentDataHash);
|
||||
}
|
||||
|
||||
this.isInitialized = true;
|
||||
@ -147,9 +138,8 @@ class EmbeddingsService {
|
||||
} catch (err) {
|
||||
console.error('[EMBEDDINGS] Failed to initialize:', err);
|
||||
this.isInitialized = false;
|
||||
throw err; // Let the caller know – same behaviour as before
|
||||
throw err;
|
||||
} finally {
|
||||
// 3️⃣ Always clear the promise so subsequent calls don't hang
|
||||
this.initializationPromise = null;
|
||||
}
|
||||
}
|
||||
@ -157,7 +147,7 @@ class EmbeddingsService {
|
||||
private async hashToolsFile(): Promise<string> {
|
||||
const file = path.join(process.cwd(), 'src', 'data', 'tools.yaml');
|
||||
const raw = await fs.readFile(file, 'utf8');
|
||||
return crypto.createHash('sha256').update(raw).digest('hex'); // 64-char hex
|
||||
return crypto.createHash('sha256').update(raw).digest('hex');
|
||||
}
|
||||
|
||||
private async loadEmbeddings(): Promise<EmbeddingsDatabase | null> {
|
||||
@ -298,7 +288,6 @@ class EmbeddingsService {
|
||||
}
|
||||
|
||||
async waitForInitialization(): Promise<void> {
|
||||
// Always re-check environment status first in case variables loaded after initial check
|
||||
await this.checkEnabledStatus();
|
||||
|
||||
if (!this.enabled || this.isInitialized) {
|
||||
@ -313,7 +302,6 @@ class EmbeddingsService {
|
||||
return this.initialize();
|
||||
}
|
||||
|
||||
// Force re-check of environment status (useful for development)
|
||||
async forceRecheckEnvironment(): Promise<void> {
|
||||
this.enabled = false;
|
||||
this.isInitialized = false;
|
||||
@ -342,7 +330,6 @@ class EmbeddingsService {
|
||||
}
|
||||
|
||||
try {
|
||||
// If we have embeddings data, use it
|
||||
if (this.isInitialized && this.embeddings.length > 0) {
|
||||
console.log(`[EMBEDDINGS] Using embeddings data for similarity search: ${query}`);
|
||||
|
||||
@ -394,7 +381,6 @@ class EmbeddingsService {
|
||||
return results;
|
||||
|
||||
} else {
|
||||
// Fallback: generate mock similarity results from actual tools data
|
||||
console.log(`[EMBEDDINGS] No embeddings data, using fallback text matching: ${query}`);
|
||||
|
||||
const { getToolsData } = await import('./dataService.js');
|
||||
@ -407,17 +393,14 @@ class EmbeddingsService {
|
||||
.map((tool: any) => {
|
||||
let similarity = 0;
|
||||
|
||||
// Name matching
|
||||
if (tool.name.toLowerCase().includes(queryLower)) {
|
||||
similarity += 0.8;
|
||||
}
|
||||
|
||||
// Description matching
|
||||
if (tool.description && tool.description.toLowerCase().includes(queryLower)) {
|
||||
similarity += 0.6;
|
||||
}
|
||||
|
||||
// Tag matching
|
||||
if (tool.tags && Array.isArray(tool.tags)) {
|
||||
const matchingTags = tool.tags.filter((tag: string) =>
|
||||
tag.toLowerCase().includes(queryLower) || queryLower.includes(tag.toLowerCase())
|
||||
@ -427,7 +410,6 @@ class EmbeddingsService {
|
||||
}
|
||||
}
|
||||
|
||||
// Word-level matching
|
||||
const toolText = `${tool.name} ${tool.description || ''} ${(tool.tags || []).join(' ')}`.toLowerCase();
|
||||
const matchingWords = queryWords.filter(word => toolText.includes(word));
|
||||
if (queryWords.length > 0) {
|
||||
@ -439,7 +421,7 @@ class EmbeddingsService {
|
||||
type: 'tool' as const,
|
||||
name: tool.name,
|
||||
content: toolText,
|
||||
embedding: [], // Empty for fallback
|
||||
embedding: [],
|
||||
metadata: {
|
||||
domains: tool.domains || [],
|
||||
phases: tool.phases || [],
|
||||
@ -465,10 +447,7 @@ class EmbeddingsService {
|
||||
}
|
||||
|
||||
isEnabled(): boolean {
|
||||
// If not enabled and not initialized, try re-checking environment
|
||||
// This handles the case where environment variables loaded after initial check
|
||||
if (!this.enabled && !this.isInitialized) {
|
||||
// Don't await this, just trigger it and return current status
|
||||
this.checkEnabledStatus().catch(console.error);
|
||||
}
|
||||
|
||||
@ -477,7 +456,7 @@ class EmbeddingsService {
|
||||
|
||||
getStats(): { enabled: boolean; initialized: boolean; count: number } {
|
||||
return {
|
||||
enabled: this.enabled, // Always true during development
|
||||
enabled: this.enabled,
|
||||
initialized: this.isInitialized,
|
||||
count: this.embeddings.length
|
||||
};
|
||||
|
||||
@ -338,7 +338,7 @@ export class NextcloudUploader {
|
||||
info: {
|
||||
path: remotePath,
|
||||
exists: true,
|
||||
response: text.substring(0, 200) + '...' // Truncated for safety
|
||||
response: text.substring(0, 200) + '...'
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user