Merge pull request 'semanticsearch' (#6) from semanticsearch into main
Reviewed-on: #6
This commit is contained in:
commit
358a30072e
670
README.md
670
README.md
@ -1,232 +1,149 @@
|
|||||||
# ForensicPathways
|
# ForensicPathways
|
||||||
|
|
||||||
Ein kuratiertes Verzeichnis für Digital Forensics und Incident Response (DFIR) Tools, Methoden und Konzepte mit KI-gestützten Workflow-Empfehlungen.
|
Ein umfassendes Verzeichnis digitaler Forensik- und Incident-Response-Tools mit KI-gestützten Empfehlungen basierend auf der NIST SP 800-86 Methodik.
|
||||||
|
|
||||||
## ✨ Funktionen
|
## Lizenz
|
||||||
|
|
||||||
### 🎯 Hauptansichten
|
Dieses Projekt ist unter der BSD-3-Clause-Lizenz lizenziert.
|
||||||
- **Kachelansicht (Grid View):** Übersichtliche Kartenansicht aller Tools/Methoden
|
|
||||||
- **Matrix-Ansicht:** Interaktive Matrix nach forensischen Domänen und Untersuchungsphasen (NIST Framework)
|
|
||||||
- **Forensic-AI:** AI-gestützte Workflow-Empfehlungen basierend auf Szenario-Beschreibungen
|
|
||||||
|
|
||||||
### 🔍 Navigation & Filterung
|
## Funktionen
|
||||||
- **Tag-System:** Intelligente Filterung nach Kategorien und Eigenschaften
|
|
||||||
- **Volltext-Suche:** Durchsuchen von Namen, Beschreibungen und Tags
|
|
||||||
- **Domain/Phase-Filter:** Filterung nach forensischen Bereichen und Ermittlungsphasen
|
|
||||||
|
|
||||||
### 📚 Inhaltstypen
|
### Kernfunktionalität
|
||||||
- **Software/Tools:** Open Source und proprietäre forensische Software
|
- **Umfassende Tool-Datenbank**: 100+ forensische Tools kategorisiert nach Domänen, Phasen und Skill-Levels
|
||||||
- **Methoden:** Bewährte forensische Verfahren und Prozesse
|
- **NIST SP 800-86 Integration**: Vier-Phasen-Methodik (Sammlung → Auswertung → Analyse → Berichterstattung)
|
||||||
- **Konzepte:** Grundlegendes Fachwissen und theoretische Grundlagen
|
- **Multiple Ansichtsmodi**: Kachelansicht, Matrix-Übersicht und KI-gestützte Empfehlungen
|
||||||
|
- **Erweiterte Suche**: Textsuche, semantische Embedding-basierte Suche und Multi-Kriterien-Filterung
|
||||||
|
- **Responsive Design**: Dark/Light-Mode-Unterstützung, mobile-optimierte Benutzeroberfläche
|
||||||
|
|
||||||
### 📖 Knowledgebase
|
### KI-gestützte Analyse
|
||||||
- **Erweiterte Dokumentation:** Detaillierte Artikel zu Tools und Methoden
|
- **Micro-Task-Pipeline**: Intelligente Tool-Auswahl durch mehrere KI-Analyseschritte
|
||||||
- **Praktische Anleitungen:** Installation, Konfiguration und Best Practices
|
- **Semantische Suche**: Vector-Embeddings für natürlichsprachige Tool-Entdeckung
|
||||||
- **Markdown-basiert:** Einfache Erstellung und Wartung von Inhalten
|
- **Konfidenz-Bewertung**: Transparente Vertrauensmetriken für KI-Empfehlungen
|
||||||
|
- **Audit-Trail**: Vollständige Entscheidungstransparenz mit detaillierter Protokollierung
|
||||||
|
- **Rate Limiting**: Intelligente Warteschlangenverwaltung und nutzerbasierte Ratenbegrenzung
|
||||||
|
|
||||||
### 🤝 Contribution-System
|
### Zusammenarbeit & Beiträge
|
||||||
- **Tool/Methoden-Beiträge:** Webformular für neue Einträge
|
- **Tool-Beiträge**: Neue Tools einreichen oder bestehende über Git-Integration bearbeiten
|
||||||
- **Knowledgebase-Artikel:** Artikel-Editor mit Datei-Upload
|
- **Knowledgebase**: Community-beigetragene Artikel und Dokumentation
|
||||||
- **Git-Integration:** Automatische Issue-Erstellung für Review-Prozess
|
- **File-Upload-System**: Nextcloud-Integration für Medien-Anhänge
|
||||||
- **File-Management:** Nextcloud-Integration für Medien-Uploads
|
- **Authentifizierung**: OIDC-Integration mit konfigurierbaren Anbietern
|
||||||
|
|
||||||
### 🔐 Authentifizierung
|
### Enterprise-Funktionen
|
||||||
- **OIDC-Integration:** Single Sign-On mit OpenID Connect
|
- **Warteschlangenverwaltung**: Ratenbegrenzte KI-Verarbeitung mit Echtzeit-Status-Updates
|
||||||
- **Berechtigungssteuerung:** Schutz für AI-Features und Contribution-System
|
- **Audit-Protokollierung**: Umfassender forensischer Audit-Trail für KI-Entscheidungsfindung
|
||||||
- **Session-Management:** Sichere JWT-basierte Sessions
|
- **Multi-Provider-Unterstützung**: Konfigurierbare KI-Services (Mistral AI, Ollama, OpenAI)
|
||||||
|
- **Git-Integration**: Automatisierte Issue-Erstellung für Beiträge (Gitea, GitHub, GitLab)
|
||||||
|
|
||||||
## 🛠 Technische Grundlage
|
## Datenmodell
|
||||||
|
|
||||||
- **Framework:** Astro 4.x mit TypeScript
|
Das System verwendet eine YAML-basierte Konfiguration in `src/data/tools.yaml`:
|
||||||
- **Styling:** CSS Custom Properties mit Dark/Light Mode
|
|
||||||
- **API:** Node.js Backend mit Astro API Routes
|
|
||||||
- **Datenbank:** YAML-basierte Konfiguration (tools.yaml)
|
|
||||||
|
|
||||||
## 📋 Voraussetzungen
|
```yaml
|
||||||
|
tools:
|
||||||
|
- name: Tool Name
|
||||||
|
type: software|method|concept
|
||||||
|
description: Detaillierte Beschreibung
|
||||||
|
skillLevel: novice|beginner|intermediate|advanced|expert
|
||||||
|
url: https://tool-homepage.com
|
||||||
|
domains: [incident-response, static-investigations, ...]
|
||||||
|
phases: [data-collection, examination, analysis, reporting]
|
||||||
|
platforms: [Windows, Linux, macOS]
|
||||||
|
license: Lizenztyp
|
||||||
|
tags: [gui, commandline, ...]
|
||||||
|
related_concepts: [konzept1, konzept2]
|
||||||
|
# Optionale Felder
|
||||||
|
projectUrl: https://hosted-instance.com # Für CC24-Server gehostete Tools
|
||||||
|
knowledgebase: true # Hat KB-Artikel
|
||||||
|
accessType: download|hosted|cloud
|
||||||
|
|
||||||
- **Node.js:** Version 18.x oder höher
|
domains:
|
||||||
- **npm:** Version 8.x oder höher
|
- id: incident-response
|
||||||
- **Nginx:** Für Reverse Proxy (Produktion)
|
name: Incident Response & Breach-Untersuchung
|
||||||
|
|
||||||
|
phases:
|
||||||
|
- id: data-collection
|
||||||
|
name: Datensammlung
|
||||||
|
description: Imaging, Akquisition, Remote-Collection-Tools
|
||||||
|
|
||||||
## 🔧 Externe Abhängigkeiten (Optional)
|
scenarios:
|
||||||
|
- id: scenario:memory_dump
|
||||||
### OIDC Provider
|
icon: 🧠
|
||||||
- **Zweck:** Benutzerauthentifizierung
|
friendly_name: RAM-Analyse
|
||||||
- **Beispiel:** Nextcloud, Keycloak, Auth0
|
|
||||||
- **Konfiguration:** `OIDC_ENDPOINT`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET`
|
|
||||||
|
|
||||||
### Nextcloud
|
|
||||||
- **Zweck:** File-Upload für Knowledgebase-Beiträge
|
|
||||||
- **Features:** Medien-Management, öffentliche Links
|
|
||||||
- **Konfiguration:** `NEXTCLOUD_ENDPOINT`, `NEXTCLOUD_USERNAME`, `NEXTCLOUD_PASSWORD`
|
|
||||||
|
|
||||||
### AI Service (Mistral/OpenAI-kompatibel)
|
|
||||||
- **Zweck:** KI-gestützte Tool-Empfehlungen
|
|
||||||
- **Konfiguration:** `AI_ANALYZER_ENDPOINT`, `AI_ANALYZER_API_KEY`, `AI_ANALYZER_MODEL`
|
|
||||||
|
|
||||||
### Uptime Kuma
|
|
||||||
- **Zweck:** Status-Monitoring für gehostete Services
|
|
||||||
- **Integration:** Status-Badges in der Service-Übersicht
|
|
||||||
|
|
||||||
### Git Provider (Gitea/GitHub/GitLab)
|
|
||||||
- **Zweck:** Issue-Erstellung für Contributions
|
|
||||||
- **Konfiguration:** `GIT_PROVIDER`, `GIT_API_ENDPOINT`, `GIT_API_TOKEN`
|
|
||||||
|
|
||||||
## 🚀 Installation
|
|
||||||
|
|
||||||
### Lokale Entwicklung
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Repository klonen
|
|
||||||
git clone https://git.cc24.dev/mstoeck3/forensic-pathways.git
|
|
||||||
cd forensic-pathways
|
|
||||||
|
|
||||||
# Dependencies installieren
|
|
||||||
npm install
|
|
||||||
|
|
||||||
# Umgebungsvariablen konfigurieren
|
|
||||||
cp .env.example .env
|
|
||||||
# .env bearbeiten (siehe Konfiguration unten)
|
|
||||||
|
|
||||||
npm run astro build
|
|
||||||
|
|
||||||
# Development Server starten
|
|
||||||
npm run dev
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Die Seite ist dann unter `http://localhost:4321` verfügbar.
|
## AI Concept
|
||||||
|
|
||||||
### Produktions-Deployment
|
### Micro-Task Architecture
|
||||||
|
The AI system uses a sophisticated pipeline that breaks complex analysis into focused micro-tasks:
|
||||||
|
|
||||||
#### 1. System vorbereiten
|
1. **Scenario Analysis**: Understanding the forensic context
|
||||||
|
2. **Investigation Approach**: Determining optimal methodology
|
||||||
|
3. **Critical Considerations**: Identifying potential challenges
|
||||||
|
4. **Tool Selection**: Phase-specific or problem-specific recommendations
|
||||||
|
5. **Background Knowledge**: Relevant concepts and prerequisites
|
||||||
|
6. **Final Synthesis**: Integrated recommendations with confidence scoring
|
||||||
|
|
||||||
|
### Confidence Scoring
|
||||||
|
Each recommendation includes transparent confidence metrics:
|
||||||
|
- **Semantic Relevance**: Vector similarity between query and tool descriptions
|
||||||
|
- **Task Suitability**: AI-assessed fitness for the specific scenario
|
||||||
|
- **Uncertainty Factors**: Potential limitations and considerations
|
||||||
|
- **Strength Indicators**: Why the tool is well-suited
|
||||||
|
|
||||||
|
## NIST SP 800-86 Phases
|
||||||
|
|
||||||
|
The system organizes tools according to the four-phase NIST methodology:
|
||||||
|
|
||||||
|
1. **Data Collection**: Imaging, acquisition, and evidence preservation
|
||||||
|
2. **Examination**: Parsing, extraction, and initial data processing
|
||||||
|
3. **Analysis**: Deep investigation, correlation, and insight generation
|
||||||
|
4. **Reporting**: Documentation, visualization, and presentation
|
||||||
|
|
||||||
|
Each tool is mapped to appropriate phases, enabling workflow-based recommendations.
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
### Production Setup
|
||||||
|
|
||||||
|
1. **Build and Deploy**:
|
||||||
```bash
|
```bash
|
||||||
# System-Updates
|
npm run build
|
||||||
sudo apt update && sudo apt upgrade -y
|
sudo ./deploy.sh # Copies dist/ to /var/www/forensic-pathways
|
||||||
|
|
||||||
# Node.js installieren (Ubuntu/Debian)
|
|
||||||
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
|
|
||||||
sudo apt-get install -y nodejs
|
|
||||||
|
|
||||||
# Nginx installieren
|
|
||||||
sudo apt install nginx -y
|
|
||||||
|
|
||||||
# Systemd für Service-Management
|
|
||||||
sudo systemctl enable nginx
|
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 2. Anwendung installieren
|
2. **Configuration**:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Klonen des Repositorys
|
cd /var/www/forensic-pathways
|
||||||
sudo git clone https://git.cc24.dev/mstoeck3/forensic-pathways /opt/forensic-pathways
|
sudo cp .env.example .env
|
||||||
cd /opt/forensic-pathways
|
sudo nano .env # Configure AI services, authentication, etc.
|
||||||
|
|
||||||
# Abhängigkeiten installieren
|
|
||||||
sudo npm install
|
|
||||||
|
|
||||||
# Production-Build erstellen
|
|
||||||
sudo npm run build
|
|
||||||
npm run astro build
|
|
||||||
|
|
||||||
# Berechtigungen setzen
|
|
||||||
sudo chown -R www-data:www-data /opt/forensic-pathways
|
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 3. Umgebungsvariablen konfigurieren
|
3. **Systemd Service** (`/etc/systemd/system/forensic-pathways.service`):
|
||||||
|
```ini
|
||||||
|
[Unit]
|
||||||
|
Description=ForensicPathways
|
||||||
|
After=network.target
|
||||||
|
|
||||||
Erstelle `/opt/forensic-pathways/.env`:
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
User=www-data
|
||||||
|
WorkingDirectory=/var/www/forensic-pathways
|
||||||
|
ExecStart=/usr/bin/node server/entry.mjs
|
||||||
|
Restart=always
|
||||||
|
RestartSec=10
|
||||||
|
Environment=NODE_ENV=production
|
||||||
|
|
||||||
```bash
|
[Install]
|
||||||
# ===========================================
|
WantedBy=multi-user.target
|
||||||
# ForensicPathways Environment Configuration
|
|
||||||
# ===========================================
|
|
||||||
|
|
||||||
# Authentication & OIDC (Required)
|
|
||||||
AUTH_SECRET=change-this-to-a-strong-secret-key-in-production
|
|
||||||
OIDC_ENDPOINT=https://your-oidc-provider.com
|
|
||||||
OIDC_CLIENT_ID=your-oidc-client-id
|
|
||||||
OIDC_CLIENT_SECRET=your-oidc-client-secret
|
|
||||||
|
|
||||||
# Auth Scopes - set to true in prod
|
|
||||||
AUTHENTICATION_NECESSARY_CONTRIBUTIONS=true
|
|
||||||
AUTHENTICATION_NECESSARY_AI=true
|
|
||||||
|
|
||||||
# Application Configuration (Required)
|
|
||||||
PUBLIC_BASE_URL=https://your-domain.com
|
|
||||||
NODE_ENV=production
|
|
||||||
|
|
||||||
# AI Service Configuration (Required for AI features)
|
|
||||||
AI_ANALYZER_MODEL=mistral-large-latest
|
|
||||||
AI_ANALYZER_ENDPOINT=https://api.mistral.ai
|
|
||||||
AI_ANALYZER_API_KEY=your-mistral-api-key
|
|
||||||
AI_RATE_LIMIT_DELAY_MS=1000
|
|
||||||
|
|
||||||
# Git Integration (Required for contributions)
|
|
||||||
GIT_REPO_URL=https://git.cc24.dev/mstoeck3/forensic-pathways
|
|
||||||
GIT_PROVIDER=gitea
|
|
||||||
GIT_API_ENDPOINT=https://git.cc24.dev/api/v1
|
|
||||||
GIT_API_TOKEN=your-git-api-token
|
|
||||||
|
|
||||||
# File Upload Configuration (Optional)
|
|
||||||
LOCAL_UPLOAD_PATH=./public/uploads
|
|
||||||
|
|
||||||
# Nextcloud Integration (Optional)
|
|
||||||
NEXTCLOUD_ENDPOINT=https://your-nextcloud.com
|
|
||||||
NEXTCLOUD_USERNAME=your-username
|
|
||||||
NEXTCLOUD_PASSWORD=your-password
|
|
||||||
NEXTCLOUD_UPLOAD_PATH=/kb-media
|
|
||||||
NEXTCLOUD_PUBLIC_URL=https://your-nextcloud.com/s/
|
|
||||||
```
|
```
|
||||||
|
|
||||||
```bash
|
4. **Nginx Configuration**:
|
||||||
# Berechtigungen sichern
|
|
||||||
sudo chmod 600 /opt/forensic-pathways/.env
|
|
||||||
sudo chown www-data:www-data /opt/forensic-pathways/.env
|
|
||||||
```
|
|
||||||
|
|
||||||
#### 4. Nginx konfigurieren
|
|
||||||
|
|
||||||
Erstelle `/etc/nginx/sites-available/forensic-pathways`:
|
|
||||||
|
|
||||||
```nginx
|
```nginx
|
||||||
server {
|
server {
|
||||||
listen 80;
|
listen 80;
|
||||||
server_name ihre-domain.de;
|
server_name forensic-pathways.yourdomain.com;
|
||||||
|
|
||||||
# Redirect HTTP to HTTPS
|
|
||||||
return 301 https://$server_name$request_uri;
|
|
||||||
}
|
|
||||||
|
|
||||||
server {
|
|
||||||
listen 443 ssl http2;
|
|
||||||
server_name ihre-domain.de;
|
|
||||||
|
|
||||||
# SSL Konfiguration (Let's Encrypt empfohlen)
|
|
||||||
ssl_certificate /etc/letsencrypt/live/ihre-domain.de/fullchain.pem;
|
|
||||||
ssl_certificate_key /etc/letsencrypt/live/ihre-domain.de/privkey.pem;
|
|
||||||
|
|
||||||
# Security Headers
|
|
||||||
add_header X-Frame-Options DENY;
|
|
||||||
add_header X-Content-Type-Options nosniff;
|
|
||||||
add_header X-XSS-Protection "1; mode=block";
|
|
||||||
add_header Referrer-Policy "strict-origin-when-cross-origin";
|
|
||||||
|
|
||||||
# Static Files
|
|
||||||
location / {
|
location / {
|
||||||
try_files $uri $uri/ @nodejs;
|
|
||||||
root /opt/forensic-pathways/dist;
|
|
||||||
index index.html;
|
|
||||||
|
|
||||||
# Cache static assets
|
|
||||||
location ~* \.(js|css|png|jpg|jpeg|gif|ico|svg|woff2?|ttf)$ {
|
|
||||||
expires 1y;
|
|
||||||
add_header Cache-Control "public, immutable";
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
# API Routes to Node.js
|
|
||||||
location @nodejs {
|
|
||||||
proxy_pass http://localhost:4321;
|
proxy_pass http://localhost:4321;
|
||||||
proxy_http_version 1.1;
|
proxy_http_version 1.1;
|
||||||
proxy_set_header Upgrade $http_upgrade;
|
proxy_set_header Upgrade $http_upgrade;
|
||||||
@ -236,251 +153,162 @@ server {
|
|||||||
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
|
||||||
proxy_set_header X-Forwarded-Proto $scheme;
|
proxy_set_header X-Forwarded-Proto $scheme;
|
||||||
proxy_cache_bypass $http_upgrade;
|
proxy_cache_bypass $http_upgrade;
|
||||||
proxy_read_timeout 300s;
|
|
||||||
proxy_connect_timeout 75s;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
# Upload limit
|
|
||||||
client_max_body_size 50M;
|
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
5. **Enable and Start**:
|
||||||
```bash
|
```bash
|
||||||
# Site aktivieren
|
sudo systemctl enable forensic-pathways
|
||||||
sudo ln -s /etc/nginx/sites-available/forensic-pathways /etc/nginx/sites-enabled/
|
sudo systemctl start forensic-pathways
|
||||||
sudo nginx -t
|
|
||||||
sudo systemctl reload nginx
|
sudo systemctl reload nginx
|
||||||
```
|
```
|
||||||
|
|
||||||
#### 5. Systemd Service einrichten
|
### Environment Configuration
|
||||||
|
|
||||||
Erstelle `/etc/systemd/system/forensic-pathways.service`:
|
Key configuration in `.env`:
|
||||||
|
|
||||||
```ini
|
|
||||||
[Unit]
|
|
||||||
Description=ForensicPathways DFIR Guide
|
|
||||||
After=network.target nginx.service
|
|
||||||
Wants=nginx.service
|
|
||||||
|
|
||||||
[Service]
|
|
||||||
Type=exec
|
|
||||||
User=www-data
|
|
||||||
Group=www-data
|
|
||||||
WorkingDirectory=/opt/forensic-pathways
|
|
||||||
Environment=NODE_ENV=production
|
|
||||||
ExecStart=/usr/bin/node ./dist/server/entry.mjs
|
|
||||||
Restart=always
|
|
||||||
RestartSec=10
|
|
||||||
StandardOutput=journal
|
|
||||||
StandardError=journal
|
|
||||||
|
|
||||||
# Security
|
|
||||||
NoNewPrivileges=yes
|
|
||||||
PrivateTmp=yes
|
|
||||||
ProtectSystem=strict
|
|
||||||
ProtectHome=yes
|
|
||||||
ReadWritePaths=/opt/forensic-pathways
|
|
||||||
CapabilityBoundingSet=
|
|
||||||
|
|
||||||
# Resource Limits
|
|
||||||
LimitNOFILE=65536
|
|
||||||
MemoryMax=512M
|
|
||||||
|
|
||||||
[Install]
|
|
||||||
WantedBy=multi-user.target
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Service aktivieren und starten
|
# Core Application
|
||||||
sudo systemctl daemon-reload
|
PUBLIC_BASE_URL=https://forensic-pathways.yourdomain.com
|
||||||
sudo systemctl enable forensic-pathways
|
AUTH_SECRET=your-secure-random-secret
|
||||||
sudo systemctl start forensic-pathways
|
|
||||||
|
|
||||||
# Status prüfen
|
# AI Services (Required)
|
||||||
sudo systemctl status forensic-pathways
|
AI_ANALYZER_ENDPOINT=https://api.mistral.ai/v1/chat/completions
|
||||||
|
AI_ANALYZER_API_KEY=your-api-key
|
||||||
|
AI_ANALYZER_MODEL=mistral/mistral-small-latest
|
||||||
|
|
||||||
|
# Vector Embeddings (Recommended)
|
||||||
|
AI_EMBEDDINGS_ENABLED=true
|
||||||
|
AI_EMBEDDINGS_ENDPOINT=https://api.mistral.ai/v1/embeddings
|
||||||
|
AI_EMBEDDINGS_MODEL=mistral-embed
|
||||||
|
|
||||||
|
# Authentication (Optional)
|
||||||
|
AUTHENTICATION_NECESSARY_AI=false
|
||||||
|
OIDC_ENDPOINT=https://your-keycloak.com/auth/realms/your-realm
|
||||||
|
OIDC_CLIENT_ID=forensic-pathways
|
||||||
```
|
```
|
||||||
|
|
||||||
## 🔧 Konfiguration
|
## Externe Abhängigkeiten (Optionale Features)
|
||||||
|
|
||||||
### Minimalkonfiguration (ohne Auth)
|
### File-Upload-System
|
||||||
|
- **Nextcloud**: Primärer Speicher für Beitrags-Anhänge
|
||||||
|
- **Lokaler Fallback**: Automatischer Fallback zu lokalem Speicher bei Nextcloud-Ausfall
|
||||||
|
|
||||||
|
### Authentifizierungsanbieter
|
||||||
|
- **Keycloak**: Empfohlener OIDC-Provider
|
||||||
|
- **Andere OIDC**: Jeder OIDC-konforme Provider (Auth0, Azure AD, etc.)
|
||||||
|
|
||||||
|
### Git-Integration
|
||||||
|
- **Gitea**: Primärer Git-Provider für Beiträge
|
||||||
|
- **GitHub/GitLab**: Alternative Git-Provider unterstützt
|
||||||
|
|
||||||
|
### Monitoring
|
||||||
|
- **Uptime Kuma**: Service-Monitoring und Gesundheitschecks (optional)
|
||||||
|
|
||||||
|
### KI-Services
|
||||||
|
- **Mistral AI**: Empfohlen für Produktion (API-Schlüssel erforderlich)
|
||||||
|
- **Ollama**: Lokale Deployment-Option (kein API-Schlüssel benötigt)
|
||||||
|
- **OpenAI**: Alternative kommerzielle Anbieter
|
||||||
|
|
||||||
|
## Knowledgebase-System
|
||||||
|
|
||||||
|
### Artikel hinzufügen
|
||||||
|
|
||||||
|
Knowledgebase-Artikel werden in `src/content/knowledgebase/` als Markdown-Dateien mit Frontmatter gespeichert:
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
---
|
||||||
|
title: "Tool-Konfigurationsanleitung"
|
||||||
|
description: "Schritt-für-Schritt-Setup-Anweisungen"
|
||||||
|
last_updated: 2024-01-15
|
||||||
|
author: "Ihr Name"
|
||||||
|
difficulty: intermediate
|
||||||
|
|
||||||
|
# Tool-Zuordnung (optional)
|
||||||
|
tool_name: "Autopsy"
|
||||||
|
related_tools: ["Volatility 3", "YARA"]
|
||||||
|
|
||||||
|
# Kategorisierung
|
||||||
|
categories: ["konfiguration", "setup"]
|
||||||
|
tags: ["gui", "installation", "windows"]
|
||||||
|
|
||||||
|
published: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# Tool-Konfigurationsanleitung
|
||||||
|
|
||||||
|
Ihr Artikel-Inhalt hier...
|
||||||
|
|
||||||
|
## Voraussetzungen
|
||||||
|
- Systemanforderungen
|
||||||
|
- Abhängigkeiten
|
||||||
|
|
||||||
|
## Installationsschritte
|
||||||
|
1. Download von offizieller Quelle
|
||||||
|
2. Installer ausführen
|
||||||
|
3. Einstellungen konfigurieren
|
||||||
|
|
||||||
|
## Häufige Probleme
|
||||||
|
Lösungen für typische Probleme...
|
||||||
|
```
|
||||||
|
|
||||||
|
### Artikel-Struktur-Richtlinien
|
||||||
|
|
||||||
|
**Erforderliche Felder**:
|
||||||
|
- `title`: Klarer, beschreibender Titel
|
||||||
|
- `description`: Einzeilige Zusammenfassung für Auflistungen
|
||||||
|
- `last_updated`: Artikel-Änderungsdatum
|
||||||
|
- `published`: Boolean-Flag für Sichtbarkeit
|
||||||
|
|
||||||
|
**Optionale Felder**:
|
||||||
|
- `tool_name`: Zuordnung zu spezifischem Tool aus Datenbank
|
||||||
|
- `author`: Mitwirkender Name (Standard: "Anon")
|
||||||
|
- `difficulty`: Komplexitätslevel passend zu Tool-Skill-Levels
|
||||||
|
- `categories`: Breite Klassifizierungen
|
||||||
|
- `tags`: Spezifische Stichwörter für Entdeckung
|
||||||
|
- `related_tools`: Array verwandter Tool-Namen
|
||||||
|
|
||||||
|
**Inhalt-Richtlinien**:
|
||||||
|
- Standard-Markdown-Formatierung verwenden
|
||||||
|
- Praktische Beispiele und Code-Snippets einschließen
|
||||||
|
- Screenshots oder Diagramme bei Bedarf hinzufügen
|
||||||
|
- Zu verwandten Tools mit `[Tool Name](/tools/tool-slug)` Format verlinken
|
||||||
|
- Troubleshooting-Abschnitte für komplexe Tools einschließen
|
||||||
|
|
||||||
|
### Automatische Verarbeitung
|
||||||
|
|
||||||
|
1. Artikel werden automatisch beim Build indexiert
|
||||||
|
2. Tool-Zuordnungen erstellen bidirektionale Links
|
||||||
|
3. Suche umfasst Volltext-Inhalt und Metadaten
|
||||||
|
4. Verwandte Artikel erscheinen in Tool-Detail-Ansichten
|
||||||
|
|
||||||
|
## Entwicklung
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Nur für Tests geeignet
|
# Setup
|
||||||
AUTHENTICATION_NECESSARY=false
|
npm install
|
||||||
PUBLIC_BASE_URL=http://localhost:4321
|
cp .env.example .env
|
||||||
|
|
||||||
|
# Entwicklung
|
||||||
|
npm run dev
|
||||||
|
|
||||||
|
# Build
|
||||||
|
npm run build
|
||||||
|
|
||||||
|
# Deploy
|
||||||
|
sudo ./deploy.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
### Tools-Datenbank
|
## Konfigurationsübersicht
|
||||||
|
|
||||||
Die Tools werden in `src/data/tools.yaml` verwaltet. Vollständiges Beispiel:
|
Die `.env.example`-Datei enthält umfassende Konfigurationsoptionen für alle Features. Die meisten Optionen haben sinnvolle Standardwerte, wobei nur die KI-Service-Konfiguration für volle Funktionalität erforderlich ist.
|
||||||
|
|
||||||
```yaml
|
## Architektur
|
||||||
tools:
|
|
||||||
- name: Autopsy
|
|
||||||
type: software # software|method|concept
|
|
||||||
description: >-
|
|
||||||
Die führende Open-Source-Alternative zu kommerziellen Forensik-Suiten mit
|
|
||||||
intuitiver grafischer Oberfläche. Besonders stark in der Timeline-Analyse,
|
|
||||||
Keyword-Suche und dem Carving gelöschter Dateien. Die modulare
|
|
||||||
Plugin-Architektur erlaubt Erweiterungen für spezielle
|
|
||||||
Untersuchungsszenarien.
|
|
||||||
icon: 📦
|
|
||||||
skillLevel: intermediate # novice|beginner|intermediate|advanced|expert
|
|
||||||
url: https://www.autopsy.com/
|
|
||||||
domains:
|
|
||||||
- incident-response
|
|
||||||
- static-investigations
|
|
||||||
- malware-analysis
|
|
||||||
- mobile-forensics
|
|
||||||
- cloud-forensics
|
|
||||||
phases:
|
|
||||||
- examination
|
|
||||||
- analysis
|
|
||||||
platforms:
|
|
||||||
- Windows
|
|
||||||
- Linux
|
|
||||||
related_concepts:
|
|
||||||
- SQL Query Fundamentals
|
|
||||||
- Hash Functions & Digital Signatures
|
|
||||||
accessType: download # download|web|api|cli|service
|
|
||||||
license: Apache 2.0
|
|
||||||
knowledgebase: false # true für erweiterte Dokumentation
|
|
||||||
tags:
|
|
||||||
- gui
|
|
||||||
- filesystem
|
|
||||||
- timeline-analysis
|
|
||||||
- carving
|
|
||||||
- artifact-extraction
|
|
||||||
- keyword-search
|
|
||||||
# Optional: Für gehostete Services
|
|
||||||
projectUrl: https://autopsy.ihre-domain.de
|
|
||||||
statusUrl: https://status.ihre-domain.de/api/badge/1/status
|
|
||||||
|
|
||||||
# Beispiel Methode
|
|
||||||
- name: Live Response Methodology
|
|
||||||
type: method
|
|
||||||
description: >-
|
|
||||||
Strukturierte Vorgehensweise zur Sammlung volatiler Daten
|
|
||||||
von laufenden Systemen ohne Shutdown.
|
|
||||||
icon: 📋
|
|
||||||
skillLevel: advanced
|
|
||||||
url: https://www.sans.org/white-papers/live-response/
|
|
||||||
domains:
|
|
||||||
- incident-response
|
|
||||||
phases:
|
|
||||||
- data-collection
|
|
||||||
related_concepts:
|
|
||||||
- Memory Forensics Fundamentals
|
|
||||||
tags:
|
|
||||||
- volatile-data
|
|
||||||
- live-analysis
|
|
||||||
- methodology
|
|
||||||
knowledgebase: true
|
|
||||||
|
|
||||||
# Beispiel Konzept
|
|
||||||
- name: Hash Functions & Digital Signatures
|
|
||||||
type: concept
|
|
||||||
description: >-
|
|
||||||
Kryptographische Grundlagen für Datenintegrität und
|
|
||||||
Authentifizierung in der digitalen Forensik.
|
|
||||||
icon: 🔐
|
|
||||||
skillLevel: intermediate
|
|
||||||
url: https://en.wikipedia.org/wiki/Cryptographic_hash_function
|
|
||||||
domains:
|
|
||||||
- incident-response
|
|
||||||
- static-investigations
|
|
||||||
- malware-analysis
|
|
||||||
phases:
|
|
||||||
- data-collection
|
|
||||||
- examination
|
|
||||||
tags:
|
|
||||||
- cryptography
|
|
||||||
- data-integrity
|
|
||||||
- evidence-preservation
|
|
||||||
knowledgebase: false
|
|
||||||
|
|
||||||
# Konfiguration der Domänen
|
- **Frontend**: Astro mit TypeScript, responsive CSS
|
||||||
domains:
|
- **Backend**: Node.js API-Routen mit intelligenter Ratenbegrenzung
|
||||||
- id: incident-response
|
- **KI-Pipeline**: Micro-Task-Architektur mit Audit-Protokollierung
|
||||||
name: Incident Response & Breach-Untersuchung
|
- **Daten**: YAML-basierte Tool-Datenbank mit Git-basierten Beiträgen
|
||||||
- id: static-investigations
|
- **Suche**: Dual-Mode Text- und semantische Vector-Suche
|
||||||
name: Datenträgerforensik & Ermittlungen
|
- **Auth**: OIDC-Integration mit Session-Management
|
||||||
- id: malware-analysis
|
|
||||||
name: Malware-Analyse & Reverse Engineering
|
|
||||||
- id: mobile-forensics
|
|
||||||
name: Mobile Geräte & App-Forensik
|
|
||||||
- id: cloud-forensics
|
|
||||||
name: Cloud & Virtuelle Umgebungen
|
|
||||||
|
|
||||||
# Konfiguration der Phasen (NIST Framework)
|
|
||||||
phases:
|
|
||||||
- id: data-collection
|
|
||||||
name: Datensammlung
|
|
||||||
description: Imaging, Acquisition, Remote Collection Tools
|
|
||||||
- id: examination
|
|
||||||
name: Auswertung
|
|
||||||
description: Parsing, Extraction, Initial Analysis Tools
|
|
||||||
- id: analysis
|
|
||||||
name: Analyse
|
|
||||||
description: Deep Analysis, Correlation, Visualization Tools
|
|
||||||
- id: reporting
|
|
||||||
name: Bericht & Präsentation
|
|
||||||
description: Documentation, Visualization, Presentation Tools
|
|
||||||
|
|
||||||
# Domänenübergreifende Kategorien
|
|
||||||
domain-agnostic-software:
|
|
||||||
- id: collaboration-general
|
|
||||||
name: Übergreifend & Kollaboration
|
|
||||||
description: Cross-cutting tools and collaboration platforms
|
|
||||||
- id: specific-os
|
|
||||||
name: Betriebssysteme
|
|
||||||
description: Operating Systems which focus on forensics
|
|
||||||
```
|
|
||||||
|
|
||||||
## 📦 Updates
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Repository aktualisieren
|
|
||||||
cd /opt/forensic-pathways
|
|
||||||
sudo git pull
|
|
||||||
|
|
||||||
# Dependencies aktualisieren
|
|
||||||
sudo npm install
|
|
||||||
|
|
||||||
# Rebuild
|
|
||||||
sudo npm run build
|
|
||||||
|
|
||||||
# Service neustarten
|
|
||||||
sudo systemctl restart forensic-pathways
|
|
||||||
```
|
|
||||||
|
|
||||||
## 💾 Backup
|
|
||||||
|
|
||||||
Wichtige Dateien für Backup:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
/opt/forensic-pathways/src/data/tools.yaml
|
|
||||||
/opt/forensic-pathways/.env
|
|
||||||
/etc/nginx/sites-available/forensic-pathways
|
|
||||||
/etc/systemd/system/forensic-pathways.service
|
|
||||||
```
|
|
||||||
|
|
||||||
## 🤝 Beiträge
|
|
||||||
|
|
||||||
Contributions sind willkommen! Bitte:
|
|
||||||
|
|
||||||
1. Issue im Repository erstellen
|
|
||||||
2. Feature-Branch erstellen
|
|
||||||
3. Pull Request öffnen
|
|
||||||
4. Tests durchführen
|
|
||||||
|
|
||||||
## 📞 Support
|
|
||||||
|
|
||||||
Bei Problemen oder Fragen:
|
|
||||||
|
|
||||||
- **Issues:** [Repository Issues](https://git.cc24.dev/mstoeck3/forensic-pathways/issues)
|
|
||||||
- **Dokumentation:** Siehe `/knowledgebase` auf der Website
|
|
||||||
|
|
||||||
## 📄 Lizenz
|
|
||||||
|
|
||||||
Dieses Projekt steht unter der **BSD-3-Clause** Lizenz.
|
|
||||||
@ -1,389 +0,0 @@
|
|||||||
// src/js/auditTrailRenderer.js
|
|
||||||
|
|
||||||
import { auditService } from '../../src/utils/auditService.js';
|
|
||||||
|
|
||||||
export class AuditTrailRenderer {
|
|
||||||
constructor(containerId, options = {}) {
|
|
||||||
this.containerId = containerId;
|
|
||||||
this.options = {
|
|
||||||
title: options.title || 'KI-Entscheidungspfad',
|
|
||||||
collapsible: options.collapsible !== false,
|
|
||||||
defaultExpanded: options.defaultExpanded || false,
|
|
||||||
...options
|
|
||||||
};
|
|
||||||
this.componentId = `audit-trail-${Date.now()}-${Math.random().toString(36).substr(2, 6)}`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Render audit trail from raw audit data
|
|
||||||
* FIXED: Proper Promise handling
|
|
||||||
*/
|
|
||||||
render(rawAuditTrail) {
|
|
||||||
const container = document.getElementById(this.containerId);
|
|
||||||
if (!container) {
|
|
||||||
console.error(`[AUDIT RENDERER] Container ${this.containerId} not found`);
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (!rawAuditTrail || !Array.isArray(rawAuditTrail) || rawAuditTrail.length === 0) {
|
|
||||||
this.renderEmpty();
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
console.log('[AUDIT RENDERER] Processing audit trail...', rawAuditTrail.length, 'entries');
|
|
||||||
|
|
||||||
// Process audit trail using the centralized service (synchronous)
|
|
||||||
const processedAudit = auditService.processAuditTrail(rawAuditTrail);
|
|
||||||
|
|
||||||
console.log('[AUDIT RENDERER] Processed audit:', processedAudit);
|
|
||||||
|
|
||||||
if (processedAudit && processedAudit.phases && processedAudit.phases.length > 0) {
|
|
||||||
this.renderProcessed(processedAudit);
|
|
||||||
// Attach event handlers after DOM is updated
|
|
||||||
setTimeout(() => this.attachEventHandlers(), 0);
|
|
||||||
} else {
|
|
||||||
console.warn('[AUDIT RENDERER] No processed audit data');
|
|
||||||
this.renderEmpty();
|
|
||||||
}
|
|
||||||
} catch (error) {
|
|
||||||
console.error('[AUDIT RENDERER] Failed to render audit trail:', error);
|
|
||||||
this.renderError(error);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
/**
|
|
||||||
* Render processed audit trail
|
|
||||||
*/
|
|
||||||
renderProcessed(processedAudit) {
|
|
||||||
const container = document.getElementById(this.containerId);
|
|
||||||
if (!container) return;
|
|
||||||
|
|
||||||
const detailsId = `${this.componentId}-details`;
|
|
||||||
|
|
||||||
console.log('[AUDIT RENDERER] Rendering processed audit with', processedAudit.phases.length, 'phases');
|
|
||||||
|
|
||||||
container.innerHTML = `
|
|
||||||
<div class="audit-trail-container">
|
|
||||||
<div class="audit-trail-header ${this.options.collapsible ? 'clickable' : ''}"
|
|
||||||
${this.options.collapsible ? `data-target="${detailsId}"` : ''}>
|
|
||||||
<div class="audit-trail-title">
|
|
||||||
<div class="audit-icon">
|
|
||||||
<div class="audit-icon-gradient">✓</div>
|
|
||||||
<h4>${this.options.title}</h4>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="audit-stats">
|
|
||||||
<div class="stat-item">
|
|
||||||
<div class="stat-dot stat-time"></div>
|
|
||||||
<span>${auditService.formatDuration(processedAudit.totalTime)}</span>
|
|
||||||
</div>
|
|
||||||
<div class="stat-item">
|
|
||||||
<div class="stat-dot" style="background-color: ${auditService.getConfidenceColor(processedAudit.avgConfidence)}"></div>
|
|
||||||
<span>${processedAudit.avgConfidence}% Vertrauen</span>
|
|
||||||
</div>
|
|
||||||
<div class="stat-item">
|
|
||||||
<span>${processedAudit.stepCount} Schritte</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
${this.options.collapsible ? `
|
|
||||||
<div class="toggle-icon">
|
|
||||||
<svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2">
|
|
||||||
<polyline points="6 9 12 15 18 9"/>
|
|
||||||
</svg>
|
|
||||||
</div>
|
|
||||||
` : ''}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div id="${detailsId}" class="audit-trail-details ${this.options.collapsible && !this.options.defaultExpanded ? 'collapsed' : ''}">
|
|
||||||
${this.renderSummary(processedAudit)}
|
|
||||||
${this.renderProcessFlow(processedAudit)}
|
|
||||||
${this.renderTechnicalDetails(processedAudit)}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
|
|
||||||
console.log('[AUDIT RENDERER] HTML rendered successfully');
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Render audit summary section
|
|
||||||
*/
|
|
||||||
renderSummary(audit) {
|
|
||||||
return `
|
|
||||||
<div class="audit-summary">
|
|
||||||
<div class="summary-header">📊 Analyse-Qualität</div>
|
|
||||||
<div class="summary-grid">
|
|
||||||
<div class="summary-stat">
|
|
||||||
<div class="summary-value success">${audit.highConfidenceSteps}</div>
|
|
||||||
<div class="summary-label">Hohe Sicherheit</div>
|
|
||||||
</div>
|
|
||||||
<div class="summary-stat">
|
|
||||||
<div class="summary-value ${audit.lowConfidenceSteps > 0 ? 'warning' : 'success'}">
|
|
||||||
${audit.lowConfidenceSteps}
|
|
||||||
</div>
|
|
||||||
<div class="summary-label">Unsichere Schritte</div>
|
|
||||||
</div>
|
|
||||||
<div class="summary-stat">
|
|
||||||
<div class="summary-value">${auditService.formatDuration(audit.totalTime)}</div>
|
|
||||||
<div class="summary-label">Verarbeitungszeit</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
${audit.summary.keyInsights && audit.summary.keyInsights.length > 0 ? `
|
|
||||||
<div class="insights-section">
|
|
||||||
<div class="insights-header success">✓ Erkenntnisse:</div>
|
|
||||||
<ul class="insights-list">
|
|
||||||
${audit.summary.keyInsights.map(insight => `<li>${this.escapeHtml(insight)}</li>`).join('')}
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
` : ''}
|
|
||||||
|
|
||||||
${audit.summary.potentialIssues && audit.summary.potentialIssues.length > 0 ? `
|
|
||||||
<div class="insights-section">
|
|
||||||
<div class="insights-header warning">⚠ Hinweise:</div>
|
|
||||||
<ul class="insights-list">
|
|
||||||
${audit.summary.potentialIssues.map(issue => `<li>${this.escapeHtml(issue)}</li>`).join('')}
|
|
||||||
</ul>
|
|
||||||
</div>
|
|
||||||
` : ''}
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Render process flow section
|
|
||||||
*/
|
|
||||||
renderProcessFlow(audit) {
|
|
||||||
if (!audit.phases || audit.phases.length === 0) {
|
|
||||||
return '<div class="audit-process-flow"><p>Keine Phasen verfügbar</p></div>';
|
|
||||||
}
|
|
||||||
|
|
||||||
return `
|
|
||||||
<div class="audit-process-flow">
|
|
||||||
${audit.phases.map((phase, index) => `
|
|
||||||
<div class="phase-group ${index === audit.phases.length - 1 ? 'last-phase' : ''}">
|
|
||||||
<div class="phase-header">
|
|
||||||
<div class="phase-info">
|
|
||||||
<span class="phase-icon">${phase.icon || '📋'}</span>
|
|
||||||
<span class="phase-name">${phase.displayName || phase.name}</span>
|
|
||||||
</div>
|
|
||||||
<div class="phase-divider"></div>
|
|
||||||
<div class="phase-stats">
|
|
||||||
<div class="confidence-bar">
|
|
||||||
<div class="confidence-fill"
|
|
||||||
style="width: ${phase.avgConfidence || 0}%; background-color: ${auditService.getConfidenceColor(phase.avgConfidence || 0)}">
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<span class="confidence-text">${phase.avgConfidence || 0}%</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div class="phase-entries">
|
|
||||||
${(phase.entries || []).map(entry => `
|
|
||||||
<div class="audit-entry">
|
|
||||||
<div class="entry-main">
|
|
||||||
<span class="entry-action">${auditService.getActionDisplayName(entry.action)}</span>
|
|
||||||
<div class="entry-meta">
|
|
||||||
<div class="confidence-indicator"
|
|
||||||
style="background-color: ${auditService.getConfidenceColor(entry.confidence || 0)}">
|
|
||||||
</div>
|
|
||||||
<span class="confidence-value">${entry.confidence || 0}%</span>
|
|
||||||
<span class="processing-time">${entry.processingTimeMs || 0}ms</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
${(entry.inputSummary && entry.inputSummary !== 'null') || (entry.outputSummary && entry.outputSummary !== 'null') ? `
|
|
||||||
<div class="entry-details">
|
|
||||||
${entry.inputSummary && entry.inputSummary !== 'null' ? `
|
|
||||||
<div class="detail-item"><strong>Input:</strong> ${this.escapeHtml(entry.inputSummary)}</div>
|
|
||||||
` : ''}
|
|
||||||
${entry.outputSummary && entry.outputSummary !== 'null' ? `
|
|
||||||
<div class="detail-item"><strong>Output:</strong> ${this.escapeHtml(entry.outputSummary)}</div>
|
|
||||||
` : ''}
|
|
||||||
</div>
|
|
||||||
` : ''}
|
|
||||||
</div>
|
|
||||||
`).join('')}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
`).join('')}
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Render technical details section
|
|
||||||
*/
|
|
||||||
renderTechnicalDetails(audit) {
|
|
||||||
const technicalId = `${this.componentId}-technical`;
|
|
||||||
|
|
||||||
return `
|
|
||||||
<div class="technical-toggle">
|
|
||||||
<button class="technical-toggle-btn" data-target="${technicalId}">
|
|
||||||
🔧 Technische Details anzeigen
|
|
||||||
</button>
|
|
||||||
<div id="${technicalId}" class="technical-details collapsed">
|
|
||||||
${(audit.phases || []).map(phase =>
|
|
||||||
(phase.entries || []).map(entry => `
|
|
||||||
<div class="technical-entry">
|
|
||||||
<div class="technical-header">
|
|
||||||
<span class="technical-phase">${entry.phase}/${entry.action}</span>
|
|
||||||
<span class="technical-time">
|
|
||||||
${new Date(entry.timestamp).toLocaleTimeString('de-DE', {
|
|
||||||
hour: '2-digit',
|
|
||||||
minute: '2-digit',
|
|
||||||
second: '2-digit'
|
|
||||||
})} • ${entry.processingTimeMs || 0}ms
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
<div class="technical-content">
|
|
||||||
<div class="technical-row">
|
|
||||||
<strong>Confidence:</strong> ${entry.confidence || 0}%
|
|
||||||
</div>
|
|
||||||
${entry.metadata && Object.keys(entry.metadata).length > 0 ? `
|
|
||||||
<div class="technical-row">
|
|
||||||
<strong>Metadata:</strong> ${this.escapeHtml(JSON.stringify(entry.metadata))}
|
|
||||||
</div>
|
|
||||||
` : ''}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
`).join('')
|
|
||||||
).join('')}
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Attach event handlers for interactions
|
|
||||||
*/
|
|
||||||
attachEventHandlers() {
|
|
||||||
console.log('[AUDIT RENDERER] Attaching event handlers...');
|
|
||||||
|
|
||||||
// Handle collapsible header
|
|
||||||
if (this.options.collapsible) {
|
|
||||||
const header = document.querySelector(`[data-target="${this.componentId}-details"]`);
|
|
||||||
const details = document.getElementById(`${this.componentId}-details`);
|
|
||||||
const toggleIcon = header?.querySelector('.toggle-icon svg');
|
|
||||||
|
|
||||||
if (header && details && toggleIcon) {
|
|
||||||
// Remove existing listeners
|
|
||||||
header.replaceWith(header.cloneNode(true));
|
|
||||||
const newHeader = document.querySelector(`[data-target="${this.componentId}-details"]`);
|
|
||||||
const newToggleIcon = newHeader?.querySelector('.toggle-icon svg');
|
|
||||||
|
|
||||||
if (newHeader && newToggleIcon) {
|
|
||||||
newHeader.addEventListener('click', () => {
|
|
||||||
const isCollapsed = details.classList.contains('collapsed');
|
|
||||||
|
|
||||||
if (isCollapsed) {
|
|
||||||
details.classList.remove('collapsed');
|
|
||||||
newToggleIcon.style.transform = 'rotate(180deg)';
|
|
||||||
} else {
|
|
||||||
details.classList.add('collapsed');
|
|
||||||
newToggleIcon.style.transform = 'rotate(0deg)';
|
|
||||||
}
|
|
||||||
});
|
|
||||||
console.log('[AUDIT RENDERER] Collapsible header handler attached');
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle technical details toggle
|
|
||||||
const technicalBtn = document.querySelector(`[data-target="${this.componentId}-technical"]`);
|
|
||||||
const technicalDetails = document.getElementById(`${this.componentId}-technical`);
|
|
||||||
|
|
||||||
if (technicalBtn && technicalDetails) {
|
|
||||||
// Remove existing listener
|
|
||||||
technicalBtn.replaceWith(technicalBtn.cloneNode(true));
|
|
||||||
const newTechnicalBtn = document.querySelector(`[data-target="${this.componentId}-technical"]`);
|
|
||||||
|
|
||||||
if (newTechnicalBtn) {
|
|
||||||
newTechnicalBtn.addEventListener('click', () => {
|
|
||||||
const isCollapsed = technicalDetails.classList.contains('collapsed');
|
|
||||||
|
|
||||||
if (isCollapsed) {
|
|
||||||
technicalDetails.classList.remove('collapsed');
|
|
||||||
newTechnicalBtn.textContent = '🔧 Technische Details ausblenden';
|
|
||||||
} else {
|
|
||||||
technicalDetails.classList.add('collapsed');
|
|
||||||
newTechnicalBtn.textContent = '🔧 Technische Details anzeigen';
|
|
||||||
}
|
|
||||||
});
|
|
||||||
console.log('[AUDIT RENDERER] Technical details handler attached');
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Render empty state
|
|
||||||
*/
|
|
||||||
renderEmpty() {
|
|
||||||
const container = document.getElementById(this.containerId);
|
|
||||||
if (container) {
|
|
||||||
container.innerHTML = `
|
|
||||||
<div class="audit-trail-container">
|
|
||||||
<div class="audit-trail-header">
|
|
||||||
<div class="audit-icon">
|
|
||||||
<div class="audit-icon-gradient">ⓘ</div>
|
|
||||||
<h4>Kein Audit-Trail verfügbar</h4>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Render error state
|
|
||||||
*/
|
|
||||||
renderError(error) {
|
|
||||||
const container = document.getElementById(this.containerId);
|
|
||||||
if (container) {
|
|
||||||
container.innerHTML = `
|
|
||||||
<div class="audit-trail-container">
|
|
||||||
<div class="audit-trail-header">
|
|
||||||
<div class="audit-icon">
|
|
||||||
<div class="audit-icon-gradient" style="background: var(--color-error);">✗</div>
|
|
||||||
<h4>Audit-Trail Fehler</h4>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
<div class="audit-summary">
|
|
||||||
<p style="color: var(--color-error);">
|
|
||||||
Fehler beim Laden der Audit-Informationen: ${this.escapeHtml(error.message)}
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
`;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Utility method to escape HTML
|
|
||||||
*/
|
|
||||||
escapeHtml(text) {
|
|
||||||
if (typeof text !== 'string') return String(text);
|
|
||||||
const div = document.createElement('div');
|
|
||||||
div.textContent = text;
|
|
||||||
return div.innerHTML;
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Clear the audit trail display
|
|
||||||
*/
|
|
||||||
clear() {
|
|
||||||
const container = document.getElementById(this.containerId);
|
|
||||||
if (container) {
|
|
||||||
container.innerHTML = '';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* Get container element
|
|
||||||
*/
|
|
||||||
getContainer() {
|
|
||||||
return document.getElementById(this.containerId);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large
Load Diff
@ -354,7 +354,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
let semanticSearchAvailable = false;
|
let semanticSearchAvailable = false;
|
||||||
let lastSemanticResults = null;
|
let lastSemanticResults = null;
|
||||||
|
|
||||||
// Check embeddings availability
|
|
||||||
async function checkEmbeddingsAvailability() {
|
async function checkEmbeddingsAvailability() {
|
||||||
try {
|
try {
|
||||||
const res = await fetch('/api/ai/embeddings-status');
|
const res = await fetch('/api/ai/embeddings-status');
|
||||||
@ -363,15 +362,13 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
|
|
||||||
if (semanticSearchAvailable) {
|
if (semanticSearchAvailable) {
|
||||||
elements.semanticContainer.classList.remove('hidden');
|
elements.semanticContainer.classList.remove('hidden');
|
||||||
elements.semanticCheckbox.disabled = false; // 👈 re-enable
|
elements.semanticCheckbox.disabled = false;
|
||||||
}
|
}
|
||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error('[EMBEDDINGS] Status check failed:', err);
|
console.error('[EMBEDDINGS] Status check failed:', err);
|
||||||
// leave the checkbox disabled
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Semantic search function
|
|
||||||
async function performSemanticSearch(query) {
|
async function performSemanticSearch(query) {
|
||||||
if (!semanticSearchAvailable || !query.trim()) {
|
if (!semanticSearchAvailable || !query.trim()) {
|
||||||
return null;
|
return null;
|
||||||
@ -576,7 +573,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// FIXED: Consolidated filtering logic with semantic search support
|
|
||||||
async function filterTools() {
|
async function filterTools() {
|
||||||
const searchTerm = elements.searchInput.value.trim().toLowerCase();
|
const searchTerm = elements.searchInput.value.trim().toLowerCase();
|
||||||
const selectedDomain = elements.domainSelect.value;
|
const selectedDomain = elements.domainSelect.value;
|
||||||
@ -594,7 +590,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
let filteredTools = window.toolsData;
|
let filteredTools = window.toolsData;
|
||||||
let semanticResults = null;
|
let semanticResults = null;
|
||||||
|
|
||||||
// CONSOLIDATED: Use semantic search if enabled and search term exists
|
|
||||||
if (semanticSearchEnabled && semanticSearchAvailable && searchTerm) {
|
if (semanticSearchEnabled && semanticSearchAvailable && searchTerm) {
|
||||||
semanticResults = await performSemanticSearch(searchTerm);
|
semanticResults = await performSemanticSearch(searchTerm);
|
||||||
lastSemanticResults = semanticResults;
|
lastSemanticResults = semanticResults;
|
||||||
@ -605,7 +600,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
} else {
|
} else {
|
||||||
lastSemanticResults = null;
|
lastSemanticResults = null;
|
||||||
|
|
||||||
// Traditional text-based search
|
|
||||||
if (searchTerm) {
|
if (searchTerm) {
|
||||||
filteredTools = window.toolsData.filter(tool =>
|
filteredTools = window.toolsData.filter(tool =>
|
||||||
tool.name.toLowerCase().includes(searchTerm) ||
|
tool.name.toLowerCase().includes(searchTerm) ||
|
||||||
@ -615,7 +609,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Apply additional filters to the results
|
|
||||||
filteredTools = filteredTools.filter(tool => {
|
filteredTools = filteredTools.filter(tool => {
|
||||||
if (selectedDomain && !(tool.domains || []).includes(selectedDomain)) {
|
if (selectedDomain && !(tool.domains || []).includes(selectedDomain)) {
|
||||||
return false;
|
return false;
|
||||||
@ -666,9 +659,8 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/* existing code continues */
|
|
||||||
const finalResults = semanticSearchEnabled && lastSemanticResults
|
const finalResults = semanticSearchEnabled && lastSemanticResults
|
||||||
? filteredTools // now properly re-sorted
|
? filteredTools
|
||||||
: (searchTerm && window.prioritizeSearchResults
|
: (searchTerm && window.prioritizeSearchResults
|
||||||
? window.prioritizeSearchResults(filteredTools, searchTerm)
|
? window.prioritizeSearchResults(filteredTools, searchTerm)
|
||||||
: filteredTools);
|
: filteredTools);
|
||||||
@ -726,7 +718,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
filterTagCloud();
|
filterTagCloud();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Event listeners
|
|
||||||
elements.searchInput.addEventListener('input', (e) => {
|
elements.searchInput.addEventListener('input', (e) => {
|
||||||
const hasValue = e.target.value.length > 0;
|
const hasValue = e.target.value.length > 0;
|
||||||
elements.clearSearch.classList.toggle('hidden', !hasValue);
|
elements.clearSearch.classList.toggle('hidden', !hasValue);
|
||||||
@ -741,7 +732,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
filterTools();
|
filterTools();
|
||||||
});
|
});
|
||||||
|
|
||||||
// Semantic search checkbox handler
|
|
||||||
if (elements.semanticCheckbox) {
|
if (elements.semanticCheckbox) {
|
||||||
elements.semanticCheckbox.addEventListener('change', (e) => {
|
elements.semanticCheckbox.addEventListener('change', (e) => {
|
||||||
semanticSearchEnabled = e.target.checked;
|
semanticSearchEnabled = e.target.checked;
|
||||||
@ -812,7 +802,6 @@ const sortedTags = Object.entries(tagFrequency)
|
|||||||
window.clearTagFilters = resetTags;
|
window.clearTagFilters = resetTags;
|
||||||
window.clearAllFilters = resetAllFilters;
|
window.clearAllFilters = resetAllFilters;
|
||||||
|
|
||||||
// Initialize
|
|
||||||
checkEmbeddingsAvailability();
|
checkEmbeddingsAvailability();
|
||||||
initializeCollapsible();
|
initializeCollapsible();
|
||||||
initTagCloud();
|
initTagCloud();
|
||||||
|
|||||||
@ -20,7 +20,7 @@ const AI_ANALYZER_API_KEY = getEnv('AI_ANALYZER_API_KEY');
|
|||||||
const AI_ANALYZER_MODEL = getEnv('AI_ANALYZER_MODEL');
|
const AI_ANALYZER_MODEL = getEnv('AI_ANALYZER_MODEL');
|
||||||
|
|
||||||
const rateLimitStore = new Map<string, { count: number; resetTime: number }>();
|
const rateLimitStore = new Map<string, { count: number; resetTime: number }>();
|
||||||
const RATE_LIMIT_WINDOW = 60 * 1000; // 1 minute
|
const RATE_LIMIT_WINDOW = 60 * 1000;
|
||||||
const RATE_LIMIT_MAX = 5;
|
const RATE_LIMIT_MAX = 5;
|
||||||
|
|
||||||
function sanitizeInput(input: string): string {
|
function sanitizeInput(input: string): string {
|
||||||
|
|||||||
@ -198,15 +198,12 @@ const phases = data.phases;
|
|||||||
<script define:vars={{ toolsData: data.tools, phases: data.phases }}>
|
<script define:vars={{ toolsData: data.tools, phases: data.phases }}>
|
||||||
window.toolsData = toolsData;
|
window.toolsData = toolsData;
|
||||||
|
|
||||||
// CONSOLIDATED: Approach selection - Pure navigation aid
|
|
||||||
window.selectApproach = function(approach) {
|
window.selectApproach = function(approach) {
|
||||||
console.log(`Selected approach: ${approach}`);
|
console.log(`Selected approach: ${approach}`);
|
||||||
|
|
||||||
// Clear any existing AI results
|
|
||||||
const aiResults = document.getElementById('ai-results');
|
const aiResults = document.getElementById('ai-results');
|
||||||
if (aiResults) aiResults.style.display = 'none';
|
if (aiResults) aiResults.style.display = 'none';
|
||||||
|
|
||||||
// Update visual selection state
|
|
||||||
document.querySelectorAll('.approach-card').forEach(card => {
|
document.querySelectorAll('.approach-card').forEach(card => {
|
||||||
card.classList.remove('selected');
|
card.classList.remove('selected');
|
||||||
});
|
});
|
||||||
@ -214,14 +211,12 @@ const phases = data.phases;
|
|||||||
const selectedCard = document.querySelector(`.approach-card.${approach}`);
|
const selectedCard = document.querySelector(`.approach-card.${approach}`);
|
||||||
if (selectedCard) selectedCard.classList.add('selected');
|
if (selectedCard) selectedCard.classList.add('selected');
|
||||||
|
|
||||||
// Hide all approach sections first (ensures mutual exclusivity)
|
|
||||||
const methodologySection = document.getElementById('methodology-section');
|
const methodologySection = document.getElementById('methodology-section');
|
||||||
const targetedSection = document.getElementById('targeted-section');
|
const targetedSection = document.getElementById('targeted-section');
|
||||||
|
|
||||||
if (methodologySection) methodologySection.classList.remove('active');
|
if (methodologySection) methodologySection.classList.remove('active');
|
||||||
if (targetedSection) targetedSection.classList.remove('active');
|
if (targetedSection) targetedSection.classList.remove('active');
|
||||||
|
|
||||||
// Show the selected approach section (navigation aid only)
|
|
||||||
if (approach === 'methodology') {
|
if (approach === 'methodology') {
|
||||||
if (methodologySection) {
|
if (methodologySection) {
|
||||||
methodologySection.classList.add('active');
|
methodologySection.classList.add('active');
|
||||||
@ -235,11 +230,9 @@ const phases = data.phases;
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
// CONSOLIDATED: Phase selection - Sets unified filter dropdown
|
|
||||||
window.selectPhase = function(phase) {
|
window.selectPhase = function(phase) {
|
||||||
console.log(`Selected NIST phase: ${phase}`);
|
console.log(`Selected NIST phase: ${phase}`);
|
||||||
|
|
||||||
// Update visual selection of phase cards
|
|
||||||
document.querySelectorAll('.phase-card').forEach(card => {
|
document.querySelectorAll('.phase-card').forEach(card => {
|
||||||
card.classList.remove('active');
|
card.classList.remove('active');
|
||||||
});
|
});
|
||||||
@ -249,23 +242,19 @@ const phases = data.phases;
|
|||||||
selectedCard.classList.add('active');
|
selectedCard.classList.add('active');
|
||||||
}
|
}
|
||||||
|
|
||||||
// CONSOLIDATED: Set the unified phase-select dropdown
|
|
||||||
const phaseSelect = document.getElementById('phase-select');
|
const phaseSelect = document.getElementById('phase-select');
|
||||||
if (phaseSelect) {
|
if (phaseSelect) {
|
||||||
phaseSelect.value = phase;
|
phaseSelect.value = phase;
|
||||||
|
|
||||||
// Trigger the change event to activate unified filtering
|
|
||||||
const changeEvent = new Event('change', { bubbles: true });
|
const changeEvent = new Event('change', { bubbles: true });
|
||||||
phaseSelect.dispatchEvent(changeEvent);
|
phaseSelect.dispatchEvent(changeEvent);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Switch to grid view to show filtered results
|
|
||||||
const gridToggle = document.querySelector('.view-toggle[data-view="grid"]');
|
const gridToggle = document.querySelector('.view-toggle[data-view="grid"]');
|
||||||
if (gridToggle && !gridToggle.classList.contains('active')) {
|
if (gridToggle && !gridToggle.classList.contains('active')) {
|
||||||
gridToggle.click();
|
gridToggle.click();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Scroll to filtered results
|
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
window.scrollToElementById('tools-grid');
|
window.scrollToElementById('tools-grid');
|
||||||
}, 200);
|
}, 200);
|
||||||
@ -309,17 +298,14 @@ const phases = data.phases;
|
|||||||
const filtersSection = document.getElementById('filters-section');
|
const filtersSection = document.getElementById('filters-section');
|
||||||
const noResults = document.getElementById('no-results');
|
const noResults = document.getElementById('no-results');
|
||||||
|
|
||||||
// FIXED: Hide approach sections when switching to ANY view mode
|
|
||||||
const methodologySection = document.getElementById('methodology-section');
|
const methodologySection = document.getElementById('methodology-section');
|
||||||
const targetedSection = document.getElementById('targeted-section');
|
const targetedSection = document.getElementById('targeted-section');
|
||||||
|
|
||||||
// Hide all main content areas
|
|
||||||
if (toolsGrid) toolsGrid.style.display = 'none';
|
if (toolsGrid) toolsGrid.style.display = 'none';
|
||||||
if (matrixContainer) matrixContainer.style.display = 'none';
|
if (matrixContainer) matrixContainer.style.display = 'none';
|
||||||
if (aiInterface) aiInterface.style.display = 'none';
|
if (aiInterface) aiInterface.style.display = 'none';
|
||||||
if (noResults) noResults.style.display = 'none';
|
if (noResults) noResults.style.display = 'none';
|
||||||
|
|
||||||
// FIXED: Hide approach sections when switching to view modes
|
|
||||||
if (methodologySection) methodologySection.classList.remove('active');
|
if (methodologySection) methodologySection.classList.remove('active');
|
||||||
if (targetedSection) targetedSection.classList.remove('active');
|
if (targetedSection) targetedSection.classList.remove('active');
|
||||||
|
|
||||||
@ -335,18 +321,14 @@ const phases = data.phases;
|
|||||||
case 'ai':
|
case 'ai':
|
||||||
if (aiInterface) aiInterface.style.display = 'block';
|
if (aiInterface) aiInterface.style.display = 'block';
|
||||||
|
|
||||||
// FIXED: Show filters but hide everything except view controls
|
|
||||||
if (filtersSection) {
|
if (filtersSection) {
|
||||||
filtersSection.style.display = 'block';
|
filtersSection.style.display = 'block';
|
||||||
|
|
||||||
// Hide all filter sections except the last one (view controls)
|
|
||||||
const filterSections = filtersSection.querySelectorAll('.filter-section');
|
const filterSections = filtersSection.querySelectorAll('.filter-section');
|
||||||
filterSections.forEach((section, index) => {
|
filterSections.forEach((section, index) => {
|
||||||
if (index === filterSections.length - 1) {
|
if (index === filterSections.length - 1) {
|
||||||
// Keep view controls visible
|
|
||||||
section.style.display = 'block';
|
section.style.display = 'block';
|
||||||
} else {
|
} else {
|
||||||
// Hide other filter sections
|
|
||||||
section.style.display = 'none';
|
section.style.display = 'none';
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@ -354,7 +336,6 @@ const phases = data.phases;
|
|||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
||||||
// FIXED: Reset filter sections visibility when not in AI view
|
|
||||||
if (view !== 'ai' && filtersSection) {
|
if (view !== 'ai' && filtersSection) {
|
||||||
const filterSections = filtersSection.querySelectorAll('.filter-section');
|
const filterSections = filtersSection.querySelectorAll('.filter-section');
|
||||||
filterSections.forEach(section => {
|
filterSections.forEach(section => {
|
||||||
@ -363,7 +344,6 @@ const phases = data.phases;
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Navigation functions for AI recommendations (unchanged)
|
|
||||||
window.navigateToGrid = function(toolName) {
|
window.navigateToGrid = function(toolName) {
|
||||||
console.log('Navigating to grid for tool:', toolName);
|
console.log('Navigating to grid for tool:', toolName);
|
||||||
|
|
||||||
@ -485,8 +465,6 @@ const phases = data.phases;
|
|||||||
}, 100);
|
}, 100);
|
||||||
}
|
}
|
||||||
|
|
||||||
// REPLACE the existing toolsFiltered event listener in index.astro with this enhanced version:
|
|
||||||
|
|
||||||
window.addEventListener('toolsFiltered', (event) => {
|
window.addEventListener('toolsFiltered', (event) => {
|
||||||
const { tools: filtered, semanticSearch } = event.detail;
|
const { tools: filtered, semanticSearch } = event.detail;
|
||||||
const currentView = document.querySelector('.view-toggle.active')?.getAttribute('data-view');
|
const currentView = document.querySelector('.view-toggle.active')?.getAttribute('data-view');
|
||||||
@ -504,11 +482,9 @@ const phases = data.phases;
|
|||||||
if (semanticSearch && filtered.length > 0) {
|
if (semanticSearch && filtered.length > 0) {
|
||||||
console.log('[SEMANTIC] Reordering tools by semantic similarity');
|
console.log('[SEMANTIC] Reordering tools by semantic similarity');
|
||||||
|
|
||||||
// FIXED: Create ordered array of cards based on semantic similarity
|
|
||||||
const orderedCards = [];
|
const orderedCards = [];
|
||||||
const remainingCards = [];
|
const remainingCards = [];
|
||||||
|
|
||||||
// First pass: collect cards in semantic order
|
|
||||||
filtered.forEach(tool => {
|
filtered.forEach(tool => {
|
||||||
const toolName = tool.name.toLowerCase();
|
const toolName = tool.name.toLowerCase();
|
||||||
const matchingCard = Array.from(allToolCards).find(card =>
|
const matchingCard = Array.from(allToolCards).find(card =>
|
||||||
@ -520,12 +496,10 @@ const phases = data.phases;
|
|||||||
orderedCards.push(matchingCard);
|
orderedCards.push(matchingCard);
|
||||||
visibleCount++;
|
visibleCount++;
|
||||||
|
|
||||||
// Add semantic indicators if available
|
|
||||||
if (tool._semanticSimilarity) {
|
if (tool._semanticSimilarity) {
|
||||||
matchingCard.setAttribute('data-semantic-similarity', tool._semanticSimilarity.toFixed(3));
|
matchingCard.setAttribute('data-semantic-similarity', tool._semanticSimilarity.toFixed(3));
|
||||||
matchingCard.setAttribute('data-semantic-rank', tool._semanticRank || '');
|
matchingCard.setAttribute('data-semantic-rank', tool._semanticRank || '');
|
||||||
|
|
||||||
// Visual indication of semantic ranking (subtle)
|
|
||||||
const header = matchingCard.querySelector('.tool-card-header h3');
|
const header = matchingCard.querySelector('.tool-card-header h3');
|
||||||
if (header && tool._semanticRank <= 3) {
|
if (header && tool._semanticRank <= 3) {
|
||||||
const existingIndicator = header.querySelector('.semantic-rank-indicator');
|
const existingIndicator = header.querySelector('.semantic-rank-indicator');
|
||||||
@ -551,7 +525,6 @@ const phases = data.phases;
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Second pass: hide non-matching cards and collect them
|
|
||||||
allToolCards.forEach(card => {
|
allToolCards.forEach(card => {
|
||||||
const toolName = card.getAttribute('data-tool-name');
|
const toolName = card.getAttribute('data-tool-name');
|
||||||
if (!filteredNames.has(toolName)) {
|
if (!filteredNames.has(toolName)) {
|
||||||
@ -560,18 +533,15 @@ const phases = data.phases;
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Reorder DOM: semantic results first, then hidden cards
|
|
||||||
const allCards = [...orderedCards, ...remainingCards];
|
const allCards = [...orderedCards, ...remainingCards];
|
||||||
allCards.forEach(card => {
|
allCards.forEach(card => {
|
||||||
toolsContainer.appendChild(card);
|
toolsContainer.appendChild(card);
|
||||||
});
|
});
|
||||||
|
|
||||||
} else {
|
} else {
|
||||||
// FIXED: Standard filtering without semantic ordering
|
|
||||||
allToolCards.forEach(card => {
|
allToolCards.forEach(card => {
|
||||||
const toolName = card.getAttribute('data-tool-name');
|
const toolName = card.getAttribute('data-tool-name');
|
||||||
|
|
||||||
// Clean up any semantic indicators
|
|
||||||
card.removeAttribute('data-semantic-similarity');
|
card.removeAttribute('data-semantic-similarity');
|
||||||
card.removeAttribute('data-semantic-rank');
|
card.removeAttribute('data-semantic-rank');
|
||||||
const semanticIndicator = card.querySelector('.semantic-rank-indicator');
|
const semanticIndicator = card.querySelector('.semantic-rank-indicator');
|
||||||
@ -587,10 +557,8 @@ const phases = data.phases;
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// Restore original order when not using semantic search
|
|
||||||
if (!semanticSearch) {
|
if (!semanticSearch) {
|
||||||
const originalOrder = Array.from(allToolCards).sort((a, b) => {
|
const originalOrder = Array.from(allToolCards).sort((a, b) => {
|
||||||
// Get original indices from data attributes or DOM order
|
|
||||||
const aIndex = Array.from(allToolCards).indexOf(a);
|
const aIndex = Array.from(allToolCards).indexOf(a);
|
||||||
const bIndex = Array.from(allToolCards).indexOf(b);
|
const bIndex = Array.from(allToolCards).indexOf(b);
|
||||||
return aIndex - bIndex;
|
return aIndex - bIndex;
|
||||||
@ -602,14 +570,12 @@ const phases = data.phases;
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Show/hide no results message
|
|
||||||
if (visibleCount === 0) {
|
if (visibleCount === 0) {
|
||||||
noResults.style.display = 'block';
|
noResults.style.display = 'block';
|
||||||
} else {
|
} else {
|
||||||
noResults.style.display = 'none';
|
noResults.style.display = 'none';
|
||||||
}
|
}
|
||||||
|
|
||||||
// Log semantic search info
|
|
||||||
if (semanticSearch) {
|
if (semanticSearch) {
|
||||||
console.log(`[SEMANTIC] Displayed ${visibleCount} tools in semantic order`);
|
console.log(`[SEMANTIC] Displayed ${visibleCount} tools in semantic order`);
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,10 +1,10 @@
|
|||||||
// src/utils/aiPipeline.ts - Enhanced with Proper Confidence Scoring
|
// src/utils/aiPipeline.ts
|
||||||
|
|
||||||
import { getCompressedToolsDataForAI } from './dataService.js';
|
import { getCompressedToolsDataForAI } from './dataService.js';
|
||||||
import { embeddingsService, type EmbeddingData, type SimilarityResult } from './embeddings.js';
|
import { embeddingsService, type EmbeddingData, type SimilarityResult } from './embeddings.js';
|
||||||
import { AI_PROMPTS, getPrompt } from '../config/prompts.js';
|
import { AI_PROMPTS, getPrompt } from '../config/prompts.js';
|
||||||
import { isToolHosted } from './toolHelpers.js';
|
import { isToolHosted } from './toolHelpers.js';
|
||||||
import { auditService } from './auditService.js'; // Add this import
|
import { auditService } from './auditService.js';
|
||||||
|
|
||||||
interface AIConfig {
|
interface AIConfig {
|
||||||
endpoint: string;
|
endpoint: string;
|
||||||
@ -188,7 +188,6 @@ class ImprovedMicroTaskAIPipeline {
|
|||||||
metadata
|
metadata
|
||||||
};
|
};
|
||||||
|
|
||||||
// Add to context audit trail instead of temp storage
|
|
||||||
if (!context.auditTrail) {
|
if (!context.auditTrail) {
|
||||||
context.auditTrail = [];
|
context.auditTrail = [];
|
||||||
}
|
}
|
||||||
@ -209,25 +208,6 @@ class ImprovedMicroTaskAIPipeline {
|
|||||||
|
|
||||||
console.log('[AUDIT] Merged temporary entries into context');
|
console.log('[AUDIT] Merged temporary entries into context');
|
||||||
}
|
}
|
||||||
/**
|
|
||||||
private summarizeForAudit(data: any): any {
|
|
||||||
if (this.auditConfig.detailLevel === 'minimal') {
|
|
||||||
if (typeof data === 'string' && data.length > 100) {
|
|
||||||
return data.slice(0, 100) + '...[truncated]';
|
|
||||||
}
|
|
||||||
if (Array.isArray(data) && data.length > 3) {
|
|
||||||
return [...data.slice(0, 3), `...[${data.length - 3} more items]`];
|
|
||||||
}
|
|
||||||
} else if (this.auditConfig.detailLevel === 'standard') {
|
|
||||||
if (typeof data === 'string' && data.length > 500) {
|
|
||||||
return data.slice(0, 500) + '...[truncated]';
|
|
||||||
}
|
|
||||||
if (Array.isArray(data) && data.length > 10) {
|
|
||||||
return [...data.slice(0, 10), `...[${data.length - 10} more items]`];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return data;
|
|
||||||
}**/
|
|
||||||
|
|
||||||
private calculateSelectionConfidence(result: any, candidateCount: number): number {
|
private calculateSelectionConfidence(result: any, candidateCount: number): number {
|
||||||
if (!result || !result.selectedTools) return 30;
|
if (!result || !result.selectedTools) return 30;
|
||||||
@ -388,7 +368,6 @@ class ImprovedMicroTaskAIPipeline {
|
|||||||
|
|
||||||
context.embeddingsSimilarities = new Map<string, number>();
|
context.embeddingsSimilarities = new Map<string, number>();
|
||||||
|
|
||||||
// Always try to initialize embeddings - let the service decide if it should be enabled
|
|
||||||
try {
|
try {
|
||||||
console.log('[AI PIPELINE] Attempting embeddings initialization...');
|
console.log('[AI PIPELINE] Attempting embeddings initialization...');
|
||||||
await embeddingsService.waitForInitialization();
|
await embeddingsService.waitForInitialization();
|
||||||
@ -1194,7 +1173,6 @@ ${JSON.stringify(conceptsToSend, null, 2)}`;
|
|||||||
})) || []
|
})) || []
|
||||||
};
|
};
|
||||||
|
|
||||||
// Process audit trail before returning
|
|
||||||
const processedAuditTrail = this.auditConfig.enabled && context.auditTrail
|
const processedAuditTrail = this.auditConfig.enabled && context.auditTrail
|
||||||
? context.auditTrail
|
? context.auditTrail
|
||||||
: [];
|
: [];
|
||||||
@ -1237,7 +1215,7 @@ ${JSON.stringify(conceptsToSend, null, 2)}`;
|
|||||||
...base,
|
...base,
|
||||||
recommended_tools: recommendedToolsWithConfidence,
|
recommended_tools: recommendedToolsWithConfidence,
|
||||||
workflow_suggestion: finalContent,
|
workflow_suggestion: finalContent,
|
||||||
auditTrail: processedAuditTrail // Always include audit trail array
|
auditTrail: processedAuditTrail
|
||||||
};
|
};
|
||||||
} else {
|
} else {
|
||||||
const recommendedToolsWithConfidence = context.selectedTools?.map(st => {
|
const recommendedToolsWithConfidence = context.selectedTools?.map(st => {
|
||||||
@ -1279,7 +1257,7 @@ ${JSON.stringify(conceptsToSend, null, 2)}`;
|
|||||||
...base,
|
...base,
|
||||||
recommended_tools: recommendedToolsWithConfidence,
|
recommended_tools: recommendedToolsWithConfidence,
|
||||||
additional_considerations: finalContent,
|
additional_considerations: finalContent,
|
||||||
auditTrail: processedAuditTrail // Always include audit trail array
|
auditTrail: processedAuditTrail
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@ -1,12 +1,10 @@
|
|||||||
// src/utils/auditService.ts - Centralized Audit Trail Management
|
// src/utils/auditService.ts
|
||||||
|
|
||||||
|
|
||||||
function env(key: string, fallback: string | undefined = undefined): string | undefined {
|
function env(key: string, fallback: string | undefined = undefined): string | undefined {
|
||||||
// during dev/server-side rendering
|
|
||||||
if (typeof process !== 'undefined' && process.env?.[key] !== undefined) {
|
if (typeof process !== 'undefined' && process.env?.[key] !== undefined) {
|
||||||
return process.env[key];
|
return process.env[key];
|
||||||
}
|
}
|
||||||
// during client build / browser
|
|
||||||
if (typeof import.meta !== 'undefined' && (import.meta as any).env?.[key] !== undefined) {
|
if (typeof import.meta !== 'undefined' && (import.meta as any).env?.[key] !== undefined) {
|
||||||
return (import.meta as any).env[key];
|
return (import.meta as any).env[key];
|
||||||
}
|
}
|
||||||
@ -67,7 +65,6 @@ class AuditService {
|
|||||||
private config: AuditConfig;
|
private config: AuditConfig;
|
||||||
private tempEntries: AuditEntry[] = [];
|
private tempEntries: AuditEntry[] = [];
|
||||||
|
|
||||||
// Phase configuration with German translations
|
|
||||||
private readonly phaseConfig = {
|
private readonly phaseConfig = {
|
||||||
'initialization': { icon: '🚀', displayName: 'Initialisierung' },
|
'initialization': { icon: '🚀', displayName: 'Initialisierung' },
|
||||||
'retrieval': { icon: '🔍', displayName: 'Datensuche' },
|
'retrieval': { icon: '🔍', displayName: 'Datensuche' },
|
||||||
@ -77,7 +74,6 @@ class AuditService {
|
|||||||
'completion': { icon: '✅', displayName: 'Finalisierung' }
|
'completion': { icon: '✅', displayName: 'Finalisierung' }
|
||||||
};
|
};
|
||||||
|
|
||||||
// Action translations
|
|
||||||
private readonly actionTranslations = {
|
private readonly actionTranslations = {
|
||||||
'pipeline-start': 'Analyse gestartet',
|
'pipeline-start': 'Analyse gestartet',
|
||||||
'embeddings-search': 'Ähnliche Tools gesucht',
|
'embeddings-search': 'Ähnliche Tools gesucht',
|
||||||
@ -95,7 +91,6 @@ class AuditService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private loadConfig(): AuditConfig {
|
private loadConfig(): AuditConfig {
|
||||||
// use the helper if you added it
|
|
||||||
const enabledFlag =
|
const enabledFlag =
|
||||||
(typeof import.meta !== 'undefined' &&
|
(typeof import.meta !== 'undefined' &&
|
||||||
(import.meta as any).env?.PUBLIC_FORENSIC_AUDIT_ENABLED) ?? 'false';
|
(import.meta as any).env?.PUBLIC_FORENSIC_AUDIT_ENABLED) ?? 'false';
|
||||||
@ -172,7 +167,6 @@ class AuditService {
|
|||||||
try {
|
try {
|
||||||
console.log('[AUDIT] Processing', rawAuditTrail.length, 'audit entries');
|
console.log('[AUDIT] Processing', rawAuditTrail.length, 'audit entries');
|
||||||
|
|
||||||
// Calculate summary statistics with safe defaults
|
|
||||||
const totalTime = rawAuditTrail.reduce((sum, entry) => sum + (entry.processingTimeMs || 0), 0);
|
const totalTime = rawAuditTrail.reduce((sum, entry) => sum + (entry.processingTimeMs || 0), 0);
|
||||||
const validConfidenceEntries = rawAuditTrail.filter(entry => typeof entry.confidence === 'number');
|
const validConfidenceEntries = rawAuditTrail.filter(entry => typeof entry.confidence === 'number');
|
||||||
const avgConfidence = validConfidenceEntries.length > 0
|
const avgConfidence = validConfidenceEntries.length > 0
|
||||||
@ -182,7 +176,6 @@ class AuditService {
|
|||||||
const highConfidenceSteps = rawAuditTrail.filter(entry => (entry.confidence || 0) >= 80).length;
|
const highConfidenceSteps = rawAuditTrail.filter(entry => (entry.confidence || 0) >= 80).length;
|
||||||
const lowConfidenceSteps = rawAuditTrail.filter(entry => (entry.confidence || 0) < 60).length;
|
const lowConfidenceSteps = rawAuditTrail.filter(entry => (entry.confidence || 0) < 60).length;
|
||||||
|
|
||||||
// Group entries by phase with safe handling
|
|
||||||
const groupedEntries = rawAuditTrail.reduce((groups, entry) => {
|
const groupedEntries = rawAuditTrail.reduce((groups, entry) => {
|
||||||
const phase = entry.phase || 'unknown';
|
const phase = entry.phase || 'unknown';
|
||||||
if (!groups[phase]) groups[phase] = [];
|
if (!groups[phase]) groups[phase] = [];
|
||||||
@ -190,7 +183,6 @@ class AuditService {
|
|||||||
return groups;
|
return groups;
|
||||||
}, {} as Record<string, AuditEntry[]>);
|
}, {} as Record<string, AuditEntry[]>);
|
||||||
|
|
||||||
// Process phases with error handling
|
|
||||||
const phases = Object.entries(groupedEntries).map(([phase, entries]) => {
|
const phases = Object.entries(groupedEntries).map(([phase, entries]) => {
|
||||||
const phaseConfig = this.phaseConfig[phase] || { icon: '📋', displayName: phase };
|
const phaseConfig = this.phaseConfig[phase] || { icon: '📋', displayName: phase };
|
||||||
const validEntries = entries.filter(entry => entry && typeof entry === 'object');
|
const validEntries = entries.filter(entry => entry && typeof entry === 'object');
|
||||||
@ -211,9 +203,8 @@ class AuditService {
|
|||||||
.map(e => this.compressEntry(e))
|
.map(e => this.compressEntry(e))
|
||||||
.filter((e): e is CompressedAuditEntry => e !== null)
|
.filter((e): e is CompressedAuditEntry => e !== null)
|
||||||
};
|
};
|
||||||
}).filter(phase => phase.entries.length > 0); // Only include phases with valid entries
|
}).filter(phase => phase.entries.length > 0);
|
||||||
|
|
||||||
// Generate analysis summary
|
|
||||||
const summary = this.generateSummary(rawAuditTrail, avgConfidence, lowConfidenceSteps);
|
const summary = this.generateSummary(rawAuditTrail, avgConfidence, lowConfidenceSteps);
|
||||||
|
|
||||||
const result: ProcessedAuditTrail = {
|
const result: ProcessedAuditTrail = {
|
||||||
@ -261,12 +252,9 @@ class AuditService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Compress data based on detail level
|
|
||||||
*/
|
|
||||||
private compressData(data: any): any {
|
private compressData(data: any): any {
|
||||||
if (this.config.detailLevel === 'verbose') {
|
if (this.config.detailLevel === 'verbose') {
|
||||||
return data; // Keep full data
|
return data;
|
||||||
} else if (this.config.detailLevel === 'standard') {
|
} else if (this.config.detailLevel === 'standard') {
|
||||||
return this.summarizeForStorage(data);
|
return this.summarizeForStorage(data);
|
||||||
} else {
|
} else {
|
||||||
@ -274,9 +262,6 @@ class AuditService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Summarize data for display purposes
|
|
||||||
*/
|
|
||||||
private summarizeData(data: any): string {
|
private summarizeData(data: any): string {
|
||||||
if (data === null || data === undefined) return 'null';
|
if (data === null || data === undefined) return 'null';
|
||||||
if (typeof data === 'string') {
|
if (typeof data === 'string') {
|
||||||
@ -327,15 +312,11 @@ class AuditService {
|
|||||||
return data;
|
return data;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Generate analysis summary
|
|
||||||
*/
|
|
||||||
private generateSummary(entries: AuditEntry[], avgConfidence: number, lowConfidenceSteps: number): {
|
private generateSummary(entries: AuditEntry[], avgConfidence: number, lowConfidenceSteps: number): {
|
||||||
analysisQuality: 'excellent' | 'good' | 'fair' | 'poor';
|
analysisQuality: 'excellent' | 'good' | 'fair' | 'poor';
|
||||||
keyInsights: string[];
|
keyInsights: string[];
|
||||||
potentialIssues: string[];
|
potentialIssues: string[];
|
||||||
} {
|
} {
|
||||||
// Determine analysis quality
|
|
||||||
let analysisQuality: 'excellent' | 'good' | 'fair' | 'poor';
|
let analysisQuality: 'excellent' | 'good' | 'fair' | 'poor';
|
||||||
if (avgConfidence >= 85 && lowConfidenceSteps === 0) {
|
if (avgConfidence >= 85 && lowConfidenceSteps === 0) {
|
||||||
analysisQuality = 'excellent';
|
analysisQuality = 'excellent';
|
||||||
@ -347,7 +328,6 @@ class AuditService {
|
|||||||
analysisQuality = 'poor';
|
analysisQuality = 'poor';
|
||||||
}
|
}
|
||||||
|
|
||||||
// Generate key insights
|
|
||||||
const keyInsights: string[] = [];
|
const keyInsights: string[] = [];
|
||||||
const embeddingsUsed = entries.some(e => e.action === 'embeddings-search');
|
const embeddingsUsed = entries.some(e => e.action === 'embeddings-search');
|
||||||
if (embeddingsUsed) {
|
if (embeddingsUsed) {
|
||||||
@ -362,7 +342,6 @@ class AuditService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Identify potential issues
|
|
||||||
const potentialIssues: string[] = [];
|
const potentialIssues: string[] = [];
|
||||||
if (lowConfidenceSteps > 2) {
|
if (lowConfidenceSteps > 2) {
|
||||||
potentialIssues.push(`${lowConfidenceSteps} Analyseschritte mit niedriger Konfidenz`);
|
potentialIssues.push(`${lowConfidenceSteps} Analyseschritte mit niedriger Konfidenz`);
|
||||||
@ -380,16 +359,10 @@ class AuditService {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Get translated action name
|
|
||||||
*/
|
|
||||||
getActionDisplayName(action: string): string {
|
getActionDisplayName(action: string): string {
|
||||||
return this.actionTranslations[action] || action;
|
return this.actionTranslations[action] || action;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Format duration for display
|
|
||||||
*/
|
|
||||||
formatDuration(ms: number): string {
|
formatDuration(ms: number): string {
|
||||||
if (ms < 1000) return '< 1s';
|
if (ms < 1000) return '< 1s';
|
||||||
if (ms < 60000) return `${Math.ceil(ms / 1000)}s`;
|
if (ms < 60000) return `${Math.ceil(ms / 1000)}s`;
|
||||||
@ -398,30 +371,20 @@ class AuditService {
|
|||||||
return seconds > 0 ? `${minutes}m ${seconds}s` : `${minutes}m`;
|
return seconds > 0 ? `${minutes}m ${seconds}s` : `${minutes}m`;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Get confidence color for UI
|
|
||||||
*/
|
|
||||||
getConfidenceColor(confidence: number): string {
|
getConfidenceColor(confidence: number): string {
|
||||||
if (confidence >= 80) return 'var(--color-accent)';
|
if (confidence >= 80) return 'var(--color-accent)';
|
||||||
if (confidence >= 60) return 'var(--color-warning)';
|
if (confidence >= 60) return 'var(--color-warning)';
|
||||||
return 'var(--color-error)';
|
return 'var(--color-error)';
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Check if audit is enabled
|
|
||||||
*/
|
|
||||||
isEnabled(): boolean {
|
isEnabled(): boolean {
|
||||||
return this.config.enabled;
|
return this.config.enabled;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* Get current configuration
|
|
||||||
*/
|
|
||||||
getConfig(): AuditConfig {
|
getConfig(): AuditConfig {
|
||||||
return { ...this.config };
|
return { ...this.config };
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Export singleton instance
|
|
||||||
export const auditService = new AuditService();
|
export const auditService = new AuditService();
|
||||||
export type { ProcessedAuditTrail, CompressedAuditEntry };
|
export type { ProcessedAuditTrail, CompressedAuditEntry };
|
||||||
@ -37,19 +37,17 @@ class EmbeddingsService {
|
|||||||
private readonly embeddingsPath = path.join(process.cwd(), 'data', 'embeddings.json');
|
private readonly embeddingsPath = path.join(process.cwd(), 'data', 'embeddings.json');
|
||||||
private readonly batchSize: number;
|
private readonly batchSize: number;
|
||||||
private readonly batchDelay: number;
|
private readonly batchDelay: number;
|
||||||
private enabled: boolean = false; // Make mutable again
|
private enabled: boolean = false;
|
||||||
|
|
||||||
constructor() {
|
constructor() {
|
||||||
this.batchSize = parseInt(process.env.AI_EMBEDDINGS_BATCH_SIZE || '20', 10);
|
this.batchSize = parseInt(process.env.AI_EMBEDDINGS_BATCH_SIZE || '20', 10);
|
||||||
this.batchDelay = parseInt(process.env.AI_EMBEDDINGS_BATCH_DELAY_MS || '1000', 10);
|
this.batchDelay = parseInt(process.env.AI_EMBEDDINGS_BATCH_DELAY_MS || '1000', 10);
|
||||||
|
|
||||||
// Don't call async method from constructor - handle in initialize() instead
|
this.enabled = true;
|
||||||
this.enabled = true; // Start optimistically enabled for development
|
|
||||||
}
|
}
|
||||||
|
|
||||||
private async checkEnabledStatus(): Promise<void> {
|
private async checkEnabledStatus(): Promise<void> {
|
||||||
try {
|
try {
|
||||||
// Add debugging to see what's actually in process.env
|
|
||||||
console.log('[EMBEDDINGS] Debug env check:', {
|
console.log('[EMBEDDINGS] Debug env check:', {
|
||||||
AI_EMBEDDINGS_ENABLED: process.env.AI_EMBEDDINGS_ENABLED,
|
AI_EMBEDDINGS_ENABLED: process.env.AI_EMBEDDINGS_ENABLED,
|
||||||
envKeys: Object.keys(process.env).filter(k => k.includes('EMBEDDINGS')).length,
|
envKeys: Object.keys(process.env).filter(k => k.includes('EMBEDDINGS')).length,
|
||||||
@ -59,7 +57,6 @@ class EmbeddingsService {
|
|||||||
const envEnabled = process.env.AI_EMBEDDINGS_ENABLED;
|
const envEnabled = process.env.AI_EMBEDDINGS_ENABLED;
|
||||||
|
|
||||||
if (envEnabled === 'true') {
|
if (envEnabled === 'true') {
|
||||||
// Check if we have the required API configuration
|
|
||||||
const endpoint = process.env.AI_EMBEDDINGS_ENDPOINT;
|
const endpoint = process.env.AI_EMBEDDINGS_ENDPOINT;
|
||||||
const model = process.env.AI_EMBEDDINGS_MODEL;
|
const model = process.env.AI_EMBEDDINGS_MODEL;
|
||||||
|
|
||||||
@ -74,7 +71,6 @@ class EmbeddingsService {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Check if embeddings file exists
|
|
||||||
try {
|
try {
|
||||||
await fs.stat(this.embeddingsPath);
|
await fs.stat(this.embeddingsPath);
|
||||||
console.log('[EMBEDDINGS] Existing embeddings file found - enabling');
|
console.log('[EMBEDDINGS] Existing embeddings file found - enabling');
|
||||||
@ -103,7 +99,6 @@ class EmbeddingsService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private async performInitialization(): Promise<void> {
|
private async performInitialization(): Promise<void> {
|
||||||
// 1️⃣ Respect the on/off switch that the newer code introduced
|
|
||||||
await this.checkEnabledStatus();
|
await this.checkEnabledStatus();
|
||||||
if (!this.enabled) {
|
if (!this.enabled) {
|
||||||
console.log('[EMBEDDINGS] Embeddings disabled, skipping initialization');
|
console.log('[EMBEDDINGS] Embeddings disabled, skipping initialization');
|
||||||
@ -114,14 +109,11 @@ class EmbeddingsService {
|
|||||||
try {
|
try {
|
||||||
console.log('[EMBEDDINGS] Initializing embeddings system…');
|
console.log('[EMBEDDINGS] Initializing embeddings system…');
|
||||||
|
|
||||||
// Make sure the data folder exists
|
|
||||||
await fs.mkdir(path.dirname(this.embeddingsPath), { recursive: true });
|
await fs.mkdir(path.dirname(this.embeddingsPath), { recursive: true });
|
||||||
|
|
||||||
// Load current tools / concepts and generate a hash
|
|
||||||
const toolsData = await getCompressedToolsDataForAI();
|
const toolsData = await getCompressedToolsDataForAI();
|
||||||
const currentDataHash = await this.hashToolsFile(); // <- keep the old helper
|
const currentDataHash = await this.hashToolsFile();
|
||||||
|
|
||||||
// Try to read an existing file
|
|
||||||
const existing = await this.loadEmbeddings();
|
const existing = await this.loadEmbeddings();
|
||||||
console.log('[EMBEDDINGS] Current hash:', currentDataHash);
|
console.log('[EMBEDDINGS] Current hash:', currentDataHash);
|
||||||
console.log('[EMBEDDINGS] Existing file version:', existing?.version);
|
console.log('[EMBEDDINGS] Existing file version:', existing?.version);
|
||||||
@ -138,8 +130,7 @@ class EmbeddingsService {
|
|||||||
this.embeddings = existing.embeddings;
|
this.embeddings = existing.embeddings;
|
||||||
} else {
|
} else {
|
||||||
console.log('[EMBEDDINGS] Generating new embeddings…');
|
console.log('[EMBEDDINGS] Generating new embeddings…');
|
||||||
// 2️⃣ Build and persist new vectors
|
await this.generateEmbeddings(toolsData, currentDataHash);
|
||||||
await this.generateEmbeddings(toolsData, currentDataHash); // <- old helper
|
|
||||||
}
|
}
|
||||||
|
|
||||||
this.isInitialized = true;
|
this.isInitialized = true;
|
||||||
@ -147,9 +138,8 @@ class EmbeddingsService {
|
|||||||
} catch (err) {
|
} catch (err) {
|
||||||
console.error('[EMBEDDINGS] Failed to initialize:', err);
|
console.error('[EMBEDDINGS] Failed to initialize:', err);
|
||||||
this.isInitialized = false;
|
this.isInitialized = false;
|
||||||
throw err; // Let the caller know – same behaviour as before
|
throw err;
|
||||||
} finally {
|
} finally {
|
||||||
// 3️⃣ Always clear the promise so subsequent calls don't hang
|
|
||||||
this.initializationPromise = null;
|
this.initializationPromise = null;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -157,7 +147,7 @@ class EmbeddingsService {
|
|||||||
private async hashToolsFile(): Promise<string> {
|
private async hashToolsFile(): Promise<string> {
|
||||||
const file = path.join(process.cwd(), 'src', 'data', 'tools.yaml');
|
const file = path.join(process.cwd(), 'src', 'data', 'tools.yaml');
|
||||||
const raw = await fs.readFile(file, 'utf8');
|
const raw = await fs.readFile(file, 'utf8');
|
||||||
return crypto.createHash('sha256').update(raw).digest('hex'); // 64-char hex
|
return crypto.createHash('sha256').update(raw).digest('hex');
|
||||||
}
|
}
|
||||||
|
|
||||||
private async loadEmbeddings(): Promise<EmbeddingsDatabase | null> {
|
private async loadEmbeddings(): Promise<EmbeddingsDatabase | null> {
|
||||||
@ -298,7 +288,6 @@ class EmbeddingsService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async waitForInitialization(): Promise<void> {
|
async waitForInitialization(): Promise<void> {
|
||||||
// Always re-check environment status first in case variables loaded after initial check
|
|
||||||
await this.checkEnabledStatus();
|
await this.checkEnabledStatus();
|
||||||
|
|
||||||
if (!this.enabled || this.isInitialized) {
|
if (!this.enabled || this.isInitialized) {
|
||||||
@ -313,7 +302,6 @@ class EmbeddingsService {
|
|||||||
return this.initialize();
|
return this.initialize();
|
||||||
}
|
}
|
||||||
|
|
||||||
// Force re-check of environment status (useful for development)
|
|
||||||
async forceRecheckEnvironment(): Promise<void> {
|
async forceRecheckEnvironment(): Promise<void> {
|
||||||
this.enabled = false;
|
this.enabled = false;
|
||||||
this.isInitialized = false;
|
this.isInitialized = false;
|
||||||
@ -342,7 +330,6 @@ class EmbeddingsService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
try {
|
try {
|
||||||
// If we have embeddings data, use it
|
|
||||||
if (this.isInitialized && this.embeddings.length > 0) {
|
if (this.isInitialized && this.embeddings.length > 0) {
|
||||||
console.log(`[EMBEDDINGS] Using embeddings data for similarity search: ${query}`);
|
console.log(`[EMBEDDINGS] Using embeddings data for similarity search: ${query}`);
|
||||||
|
|
||||||
@ -394,7 +381,6 @@ class EmbeddingsService {
|
|||||||
return results;
|
return results;
|
||||||
|
|
||||||
} else {
|
} else {
|
||||||
// Fallback: generate mock similarity results from actual tools data
|
|
||||||
console.log(`[EMBEDDINGS] No embeddings data, using fallback text matching: ${query}`);
|
console.log(`[EMBEDDINGS] No embeddings data, using fallback text matching: ${query}`);
|
||||||
|
|
||||||
const { getToolsData } = await import('./dataService.js');
|
const { getToolsData } = await import('./dataService.js');
|
||||||
@ -407,17 +393,14 @@ class EmbeddingsService {
|
|||||||
.map((tool: any) => {
|
.map((tool: any) => {
|
||||||
let similarity = 0;
|
let similarity = 0;
|
||||||
|
|
||||||
// Name matching
|
|
||||||
if (tool.name.toLowerCase().includes(queryLower)) {
|
if (tool.name.toLowerCase().includes(queryLower)) {
|
||||||
similarity += 0.8;
|
similarity += 0.8;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Description matching
|
|
||||||
if (tool.description && tool.description.toLowerCase().includes(queryLower)) {
|
if (tool.description && tool.description.toLowerCase().includes(queryLower)) {
|
||||||
similarity += 0.6;
|
similarity += 0.6;
|
||||||
}
|
}
|
||||||
|
|
||||||
// Tag matching
|
|
||||||
if (tool.tags && Array.isArray(tool.tags)) {
|
if (tool.tags && Array.isArray(tool.tags)) {
|
||||||
const matchingTags = tool.tags.filter((tag: string) =>
|
const matchingTags = tool.tags.filter((tag: string) =>
|
||||||
tag.toLowerCase().includes(queryLower) || queryLower.includes(tag.toLowerCase())
|
tag.toLowerCase().includes(queryLower) || queryLower.includes(tag.toLowerCase())
|
||||||
@ -427,7 +410,6 @@ class EmbeddingsService {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Word-level matching
|
|
||||||
const toolText = `${tool.name} ${tool.description || ''} ${(tool.tags || []).join(' ')}`.toLowerCase();
|
const toolText = `${tool.name} ${tool.description || ''} ${(tool.tags || []).join(' ')}`.toLowerCase();
|
||||||
const matchingWords = queryWords.filter(word => toolText.includes(word));
|
const matchingWords = queryWords.filter(word => toolText.includes(word));
|
||||||
if (queryWords.length > 0) {
|
if (queryWords.length > 0) {
|
||||||
@ -439,7 +421,7 @@ class EmbeddingsService {
|
|||||||
type: 'tool' as const,
|
type: 'tool' as const,
|
||||||
name: tool.name,
|
name: tool.name,
|
||||||
content: toolText,
|
content: toolText,
|
||||||
embedding: [], // Empty for fallback
|
embedding: [],
|
||||||
metadata: {
|
metadata: {
|
||||||
domains: tool.domains || [],
|
domains: tool.domains || [],
|
||||||
phases: tool.phases || [],
|
phases: tool.phases || [],
|
||||||
@ -465,10 +447,7 @@ class EmbeddingsService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
isEnabled(): boolean {
|
isEnabled(): boolean {
|
||||||
// If not enabled and not initialized, try re-checking environment
|
|
||||||
// This handles the case where environment variables loaded after initial check
|
|
||||||
if (!this.enabled && !this.isInitialized) {
|
if (!this.enabled && !this.isInitialized) {
|
||||||
// Don't await this, just trigger it and return current status
|
|
||||||
this.checkEnabledStatus().catch(console.error);
|
this.checkEnabledStatus().catch(console.error);
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -477,7 +456,7 @@ class EmbeddingsService {
|
|||||||
|
|
||||||
getStats(): { enabled: boolean; initialized: boolean; count: number } {
|
getStats(): { enabled: boolean; initialized: boolean; count: number } {
|
||||||
return {
|
return {
|
||||||
enabled: this.enabled, // Always true during development
|
enabled: this.enabled,
|
||||||
initialized: this.isInitialized,
|
initialized: this.isInitialized,
|
||||||
count: this.embeddings.length
|
count: this.embeddings.length
|
||||||
};
|
};
|
||||||
|
|||||||
@ -338,7 +338,7 @@ export class NextcloudUploader {
|
|||||||
info: {
|
info: {
|
||||||
path: remotePath,
|
path: remotePath,
|
||||||
exists: true,
|
exists: true,
|
||||||
response: text.substring(0, 200) + '...' // Truncated for safety
|
response: text.substring(0, 200) + '...'
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user