Compare commits
10 Commits
5afb377d92
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
6ea5c85a4b | ||
|
|
5c8cbabefd | ||
|
|
73d0f0fb5c | ||
|
|
18efcf482e | ||
|
|
20bf7e6745 | ||
|
|
4834fd1621 | ||
|
|
56786c7aef | ||
|
|
3f44de1d98 | ||
|
|
6e3c2c5caa | ||
|
|
f4b8b74297 |
@@ -13,3 +13,4 @@ data/
|
|||||||
examples/
|
examples/
|
||||||
scripts/
|
scripts/
|
||||||
skills-lock.json
|
skills-lock.json
|
||||||
|
.tmp/
|
||||||
|
|||||||
2
.gitignore
vendored
2
.gitignore
vendored
@@ -18,3 +18,5 @@ dashboard-*.png
|
|||||||
scans-*.png
|
scans-*.png
|
||||||
scan-*.png
|
scan-*.png
|
||||||
dashboard-*.png
|
dashboard-*.png
|
||||||
|
|
||||||
|
.tmp/
|
||||||
|
|||||||
@@ -1,140 +0,0 @@
|
|||||||
# GuardDog Nexus - Final Improvement Plan (v2)
|
|
||||||
|
|
||||||
## STATUS: IMPLEMENTED AND VERIFIED
|
|
||||||
|
|
||||||
All planned changes have been implemented and verified.
|
|
||||||
|
|
||||||
**Test Results:** 101 passed, 0 failed
|
|
||||||
**Linting:** All checks passed
|
|
||||||
**Format:** Code formatted with ruff
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Verified Issues & Fixes
|
|
||||||
|
|
||||||
### Issue 1: Lock Dictionary Memory Leak (CONFIRMED)
|
|
||||||
**Location:** `core/harvester.py` line 25, `routes/web.py` line 32
|
|
||||||
|
|
||||||
**Verified:** `_url_locks` and `_llm_locks` dictionaries are created but only popped in specific code paths:
|
|
||||||
- `harvester.py:64` - only when URL is already locked
|
|
||||||
- `harvester.py:81` - only after DB check completes
|
|
||||||
- `web.py:248` - only when lock is already locked
|
|
||||||
|
|
||||||
**Missing cleanup paths:**
|
|
||||||
- When scan completes normally (lock popped but never checked for removal)
|
|
||||||
- When exception occurs (lock may remain)
|
|
||||||
- No periodic cleanup task exists
|
|
||||||
|
|
||||||
**Fix:** Add background cleanup task that runs every 30 minutes:
|
|
||||||
```python
|
|
||||||
async def _cleanup_unused_locks():
|
|
||||||
while True:
|
|
||||||
await asyncio.sleep(1800) # 30 minutes
|
|
||||||
for key in list(_url_locks.keys()):
|
|
||||||
if not _url_locks[key].locked():
|
|
||||||
_url_locks.pop(key, None)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Issue 2: LLM Response Parsing Edge Case (CONFIRMED)
|
|
||||||
**Location:** `core/llm.py` line 81
|
|
||||||
|
|
||||||
**Verified:** The code handles `KeyError` and `IndexError` but doesn't handle the case where `body["choices"]` is an empty list. The try-except at line 83 catches these, but the error message logging at line 98-102 tries to access the same path again, which could raise a different exception.
|
|
||||||
|
|
||||||
**Fix:** Extract the raw content safely first:
|
|
||||||
```python
|
|
||||||
try:
|
|
||||||
choices = body.get("choices", [])
|
|
||||||
if not choices:
|
|
||||||
raise ValueError("Empty choices list")
|
|
||||||
message = choices[0].get("message", {})
|
|
||||||
content = message.get("content", "")
|
|
||||||
if not content:
|
|
||||||
raise ValueError("Empty message content")
|
|
||||||
return json.loads(content)
|
|
||||||
except (ValueError, json.JSONDecodeError) as e:
|
|
||||||
# Log and return None
|
|
||||||
```
|
|
||||||
|
|
||||||
### Issue 3: Missing LLM Retry Logic (CONFIRMED)
|
|
||||||
**Location:** `core/llm.py`
|
|
||||||
|
|
||||||
**Verified:** No retry mechanism exists. Single failure = no analysis for that finding.
|
|
||||||
|
|
||||||
**Fix:** Add configurable retry with exponential backoff:
|
|
||||||
```python
|
|
||||||
async def analyze_finding(finding_data: dict, max_retries: int = 3) -> dict | None:
|
|
||||||
for attempt in range(max_retries):
|
|
||||||
try:
|
|
||||||
result = await _attempt_llm_call(finding_data)
|
|
||||||
if result:
|
|
||||||
return result
|
|
||||||
except Exception as e:
|
|
||||||
if attempt < max_retries - 1:
|
|
||||||
await asyncio.sleep(2 ** attempt * 2) # 2s, 4s, 8s
|
|
||||||
continue
|
|
||||||
log.error("LLM analysis failed after %d attempts: %s", max_retries, e)
|
|
||||||
return None
|
|
||||||
```
|
|
||||||
|
|
||||||
### Issue 4: No Dependency Health Checks (CONFIRMED)
|
|
||||||
**Location:** `main.py`
|
|
||||||
|
|
||||||
**Verified:** Only `/health` endpoint exists, returns static status. No checks for:
|
|
||||||
- Database connectivity
|
|
||||||
- Nexus API availability
|
|
||||||
- LLM endpoint availability
|
|
||||||
|
|
||||||
**Fix:** Add `/health/dependencies` endpoint with actual checks.
|
|
||||||
|
|
||||||
### Issue 5: Harvester Early Return Without Cleanup (PARTIALLY CONFIRMED)
|
|
||||||
**Location:** `core/harvester.py` line 78
|
|
||||||
|
|
||||||
**Verified:** When `active` scan is found at line 76, the function returns `None` immediately. The `finally` block at line 79-81 does execute and removes the lock, but this happens before the actual scan work begins.
|
|
||||||
|
|
||||||
**Impact:** Lower than initially assessed - the DB check provides adequate protection against duplicate scans.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Refined Implementation Priorities
|
|
||||||
|
|
||||||
### Phase 1: Critical Fixes (1-2 days)
|
|
||||||
1. Add LLM retry logic with exponential backoff
|
|
||||||
2. Fix LLM response parsing edge cases
|
|
||||||
3. Add dependency health checks
|
|
||||||
|
|
||||||
### Phase 2: Reliability (2-3 days)
|
|
||||||
4. Add lock cleanup task
|
|
||||||
5. Add configuration validation on startup
|
|
||||||
6. Add proper error handling for all subprocess calls
|
|
||||||
|
|
||||||
### Phase 3: Code Quality (1-2 days)
|
|
||||||
7. Add type hints consistency
|
|
||||||
8. Add input validation for webhooks
|
|
||||||
9. Add security event logging
|
|
||||||
|
|
||||||
### Phase 4: Features (2-3 days)
|
|
||||||
10. Add scan progress tracking
|
|
||||||
11. Sync CSV export filters with API
|
|
||||||
12. Add rate limiting for webhook processing
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Verification Checklist
|
|
||||||
|
|
||||||
After each phase:
|
|
||||||
- [ ] `ruff check guarddog_nexus tests` passes
|
|
||||||
- [ ] `python3 -m pytest -v` passes all 85 tests
|
|
||||||
- [ ] `ruff format guarddog_nexus tests` applied
|
|
||||||
- [ ] Manual Docker Compose test
|
|
||||||
- [ ] Review changes for regressions
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
The project is well-structured with good separation of concerns. The main areas needing attention are:
|
|
||||||
1. **Resource management** - lock cleanup, subprocess handling
|
|
||||||
2. **Reliability** - LLM retries, health checks, error recovery
|
|
||||||
3. **Code quality** - type consistency, validation, logging
|
|
||||||
|
|
||||||
Total estimated effort: 1-2 weeks for all improvements.
|
|
||||||
@@ -1,185 +0,0 @@
|
|||||||
# GuardDog Nexus - Improvement Plan
|
|
||||||
|
|
||||||
## Status Check: Completed Analysis
|
|
||||||
|
|
||||||
After thoroughly reviewing the codebase, I've identified several areas for improvement. Below is the prioritized plan.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Priority 1: Critical Bug Fixes
|
|
||||||
|
|
||||||
### 1.1 Fix Harvester Race Condition (T4)
|
|
||||||
**File:** `guarddog_nexus/core/harvester.py`
|
|
||||||
**Issue:** Lines 56-81 have a race condition where the URL lock cleanup happens in a `finally` block while the lock check happens before acquisition.
|
|
||||||
|
|
||||||
**Current problematic flow:**
|
|
||||||
```python
|
|
||||||
# Line 56-65: Check if locked
|
|
||||||
async with _url_lock:
|
|
||||||
if download_url not in _url_locks:
|
|
||||||
_url_locks[download_url] = asyncio.Lock()
|
|
||||||
|
|
||||||
lock = _url_locks[download_url]
|
|
||||||
if lock.locked():
|
|
||||||
# Skip...
|
|
||||||
async with _url_lock:
|
|
||||||
_url_locks.pop(download_url, None) # Line 64
|
|
||||||
return None
|
|
||||||
|
|
||||||
async with lock:
|
|
||||||
# ... work ...
|
|
||||||
finally:
|
|
||||||
async with _url_lock:
|
|
||||||
_url_locks.pop(download_url, None) # Line 81
|
|
||||||
```
|
|
||||||
|
|
||||||
**Fix:** Ensure lock cleanup only happens after successful work completion, not on early returns.
|
|
||||||
|
|
||||||
### 1.2 Fix LLM Response Parsing (T4)
|
|
||||||
**File:** `guarddog_nexus/core/llm.py`
|
|
||||||
**Issue:** Line 81 assumes `body["choices"][0]["message"]["content"]` exists without validation.
|
|
||||||
|
|
||||||
**Fix:** Add proper error handling:
|
|
||||||
```python
|
|
||||||
try:
|
|
||||||
choices = body.get("choices", [])
|
|
||||||
if not choices:
|
|
||||||
raise ValueError("No choices in response")
|
|
||||||
message = choices[0].get("message", {})
|
|
||||||
content = message.get("content", "")
|
|
||||||
return json.loads(content)
|
|
||||||
except (KeyError, IndexError, json.JSONDecodeError) as e:
|
|
||||||
...
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Priority 2: Reliability Improvements
|
|
||||||
|
|
||||||
### 2.1 Add LLM Retry Logic (T4)
|
|
||||||
**File:** `guarddog_nexus/core/llm.py`
|
|
||||||
**Issue:** Failed LLM calls have no retry mechanism.
|
|
||||||
|
|
||||||
**Fix:** Add exponential backoff retry:
|
|
||||||
```python
|
|
||||||
async def analyze_finding(finding_data: dict, max_retries: int = 3) -> dict | None:
|
|
||||||
for attempt in range(max_retries):
|
|
||||||
result = await _attempt_analysis(finding_data)
|
|
||||||
if result:
|
|
||||||
return result
|
|
||||||
if attempt < max_retries - 1:
|
|
||||||
await asyncio.sleep(2 ** attempt)
|
|
||||||
return None
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2.2 Add Dependency Health Checks (T4)
|
|
||||||
**File:** `guarddog_nexus/main.py`
|
|
||||||
**Issue:** No health checks for database or external dependencies.
|
|
||||||
|
|
||||||
**Fix:** Add `/health/dependencies` endpoint:
|
|
||||||
```python
|
|
||||||
@app.get("/health/dependencies")
|
|
||||||
async def health_dependencies():
|
|
||||||
checks = {
|
|
||||||
"database": await _check_db_health(),
|
|
||||||
"nexus": await _check_nexus_connectivity(),
|
|
||||||
}
|
|
||||||
status = 200 if all(checks.values()) else 503
|
|
||||||
return JSONResponse(status_code=status, content=checks)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2.3 Fix Lock Cleanup (T4)
|
|
||||||
**Files:** `guarddog_nexus/core/harvester.py`, `guarddog_nexus/routes/web.py`
|
|
||||||
**Issue:** `_url_locks` and `_llm_locks` dicts grow indefinitely.
|
|
||||||
|
|
||||||
**Fix:** Add periodic cleanup using `asyncio.create_task()`:
|
|
||||||
```python
|
|
||||||
async def _cleanup_locks():
|
|
||||||
while True:
|
|
||||||
await asyncio.sleep(3600) # Every hour
|
|
||||||
for key in list(_url_locks.keys()):
|
|
||||||
if not _url_locks[key].locked():
|
|
||||||
_url_locks.pop(key, None)
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Priority 3: Code Quality
|
|
||||||
|
|
||||||
### 3.1 Add Type Hints Consistency (T1)
|
|
||||||
**Files:** Multiple files
|
|
||||||
**Issue:** Inconsistent use of `dict` vs `Dict[str, Any]` type hints.
|
|
||||||
|
|
||||||
### 3.2 Add Input Validation (T4)
|
|
||||||
**File:** `guarddog_nexus/routes/webhooks.py`
|
|
||||||
**Issue:** Limited validation of webhook payload structure.
|
|
||||||
|
|
||||||
**Fix:** Add validation for required fields before processing.
|
|
||||||
|
|
||||||
### 3.3 Add Logging for Security Events (T4)
|
|
||||||
**Files:** `guarddog_nexus/core/harvester.py`, `guarddog_nexus/routes/webhooks.py`
|
|
||||||
**Issue:** Security-related events not logged at appropriate levels.
|
|
||||||
|
|
||||||
**Fix:** Add WARNING/CRITICAL logging for:
|
|
||||||
- Failed authentication attempts
|
|
||||||
- Suspicious package patterns
|
|
||||||
- Rate limiting triggers
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Priority 4: Feature Enhancements
|
|
||||||
|
|
||||||
### 4.1 Add Scan Status Tracking (T4)
|
|
||||||
**File:** `guarddog_nexus/core/harvester.py`
|
|
||||||
**Issue:** No visibility into scan progress for long-running packages.
|
|
||||||
|
|
||||||
**Fix:** Add intermediate status updates via WebSocket or polling endpoint.
|
|
||||||
|
|
||||||
### 4.2 Add Configuration Validation on Startup (T1)
|
|
||||||
**File:** `guarddog_nexus/config.py`
|
|
||||||
**Issue:** Invalid configurations discovered only at runtime.
|
|
||||||
|
|
||||||
**Fix:** Add validation in `Config.__post_init__()`:
|
|
||||||
```python
|
|
||||||
def __post_init__(self):
|
|
||||||
if not self.nexus_password:
|
|
||||||
raise ValueError("NEXUS_PASSWORD is required")
|
|
||||||
if self.llm_enabled and not self.llm_api_key:
|
|
||||||
raise ValueError("LLM_API_KEY required when LLM_ENABLED=1")
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4.3 Add CSV Export with Filters (T4)
|
|
||||||
**Files:** `guarddog_nexus/routes/api_scans.py`, `guarddog_nexus/routes/api_packages.py`
|
|
||||||
**Issue:** CSV exports don't support all filter options available in API.
|
|
||||||
|
|
||||||
**Fix:** Sync filter parameters between API and CSV export endpoints.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Implementation Order
|
|
||||||
|
|
||||||
1. **Week 1:** Priority 1 fixes (race condition, LLM parsing)
|
|
||||||
2. **Week 2:** Priority 2 improvements (retries, health checks, lock cleanup)
|
|
||||||
3. **Week 3:** Priority 3 code quality (type hints, validation, logging)
|
|
||||||
4. **Week 4:** Priority 4 features (status tracking, config validation, CSV filters)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Verification Steps
|
|
||||||
|
|
||||||
After each change:
|
|
||||||
1. Run `ruff check guarddog_nexus tests`
|
|
||||||
2. Run `python3 -m pytest -v` (must pass 85 tests)
|
|
||||||
3. Run `ruff format guarddog_nexus tests`
|
|
||||||
4. Manual testing with Docker Compose
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Risk Assessment
|
|
||||||
|
|
||||||
| Change | Risk Level | Mitigation |
|
|
||||||
|--------|------------|------------|
|
|
||||||
| Harvester race condition fix | MEDIUM | Thorough concurrent testing |
|
|
||||||
| LLM retry logic | LOW | Ensure idempotency |
|
|
||||||
| Health checks | LOW | Graceful degradation |
|
|
||||||
| Lock cleanup | LOW | Conservative cleanup intervals |
|
|
||||||
@@ -2,20 +2,22 @@
|
|||||||
|
|
||||||
**Date:** 2026-05-10
|
**Date:** 2026-05-10
|
||||||
**Auditor:** Automated security audit
|
**Auditor:** Automated security audit
|
||||||
**Last updated:** 2026-05-11
|
**Last updated:** 2026-05-11 (consolidated with improvements/final-plan; statuses verified against current codebase)
|
||||||
**Scope:** Full codebase review — security vulnerabilities, logic errors, missing controls
|
**Scope:** Full codebase review — security vulnerabilities, logic errors, missing controls
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Summary
|
## Summary
|
||||||
|
|
||||||
| Severity | Count | Fixed | Rejected | Remaining |
|
| Severity | Count | Fixed | Rejected/Accepted | Mitigated | Open |
|
||||||
|----------|-------|-------|----------|-----------|
|
|----------|-------|-------|--------------------|-----------|------|
|
||||||
| CRITICAL | 5 | 2 | 2 | 1 |
|
| CRITICAL | 5 | 2 | 2 | 1 | 0 |
|
||||||
| HIGH | 7 | 2 | 3 | 2 |
|
| HIGH | 7 | 2 | 5 | 0 | 0 |
|
||||||
| MEDIUM | 8 | 3 | 0 | 5 |
|
| MEDIUM | 8 | 6 | 0 | 0 | 2 |
|
||||||
| LOW | 6 | 2 | 0 | 4 |
|
| LOW | 6 | 4 | 0 | 0 | 2 |
|
||||||
| **Total**| **26**| **9** | **5** | **12** |
|
| **Total**| **26**| **14**| **7** | **1** | **4**|
|
||||||
|
|
||||||
|
**14 fixed, 7 closed as rejected/accepted-risk, 1 partially mitigated, 4 remaining open.**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -25,192 +27,225 @@
|
|||||||
**Severity:** CRITICAL
|
**Severity:** CRITICAL
|
||||||
**Fix:** `NEXUS_ALLOWED_HOSTS` env var + `_validate_download_url()` in `core/nexus.py`.
|
**Fix:** `NEXUS_ALLOWED_HOSTS` env var + `_validate_download_url()` in `core/nexus.py`.
|
||||||
|
|
||||||
**Problem:** `downloadUrl` из webhook-пэйлода передаётся напрямую в `httpx.AsyncClient.get()` без валидации.
|
**Problem:** `downloadUrl` from webhook payload was passed directly to `httpx.AsyncClient.get()` without validation.
|
||||||
|
|
||||||
```python
|
**Fix:** Validate URL scheme (http/https only), validate hostname against allowed hosts list. Defaults to Nexus hostname if `NEXUS_ALLOWED_HOSTS` not set.
|
||||||
download_url = asset.get("downloadUrl") or _build_download_url(repository, asset_path)
|
|
||||||
# ...
|
|
||||||
response = await client.get(download_url) # no validation
|
|
||||||
```
|
|
||||||
|
|
||||||
**Real-world impact:** Атакующий отправляет webhook с `downloadUrl: "http://169.254.169.254/latest/meta-data/iam/security-credentials/"` → сервер скачивает IAM-учётные данные облака.
|
|
||||||
|
|
||||||
**Fix:** Validate URL scheme (http/https only), block private IP ranges (10.x, 172.16.x, 192.168.x, 127.x, 169.254.x, ::1), optionally whitelist domain against `config.nexus_url`.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### C2. Webhook secret not enforced by default ❌ ACCEPTED RISK
|
### C2. Webhook secret not enforced by default ❌ ACCEPTED RISK
|
||||||
**Severity:** CRITICAL
|
**Severity:** CRITICAL
|
||||||
**Decision:** Внутренний сервис, секрет опционален.
|
**Decision:** Internal service; secret is optional. Documented as such.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### C3. Default admin credentials ✅ FIXED
|
### C3. Default admin credentials ✅ FIXED
|
||||||
**Severity:** CRITICAL
|
**Severity:** CRITICAL
|
||||||
**Fix:** Убран BasicAuth из всех запросов к Nexus (анонимный доступ).
|
**Fix:** Removed BasicAuth from all Nexus API calls (anonymous access).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### C4. XSS via LLM report verdict ❌ NOT DANGEROUS
|
### C4. XSS via LLM report verdict ❌ NOT DANGEROUS
|
||||||
**Severity:** CRITICAL — downgraded to INFO
|
**Severity:** CRITICAL — downgraded to INFO
|
||||||
**Decision:** Jinja2 autoescape блокирует инъекцию в атрибутах.
|
**Decision:** Jinja2 autoescape blocks injection in attributes.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### C5. LLM Prompt Injection ⚠️ PARTIALLY MITIGATED
|
### C5. LLM Prompt Injection ⚠️ PARTIALLY MITIGATED
|
||||||
**Severity:** CRITICAL
|
**Severity:** CRITICAL
|
||||||
**Mitigation:** System prompt gives priority to system instructions. Raw finding data still in user message.
|
**Mitigation:** System prompt gives priority to system instructions. `_validate_report()` applies defaults for missing/invalid fields. Raw finding data still in user message.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## HIGH (7)
|
## HIGH (7)
|
||||||
|
|
||||||
### H1. No rate limiting ❌ REJECTED
|
### H1. No rate limiting ❌ REJECTED
|
||||||
### H2. Path traversal ⚠️ LOW RISK
|
**Severity:** HIGH
|
||||||
**Severity:** HIGH — downgraded
|
**Decision:** Internal service; not exposed to public internet.
|
||||||
**Analysis:** `os.path.basename("../../../etc/passwd")` → `"passwd"`, traversal невозможен.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### H3. Sensitive data in API ❌ REJECTED (source_ip is a feature)
|
### H2. Path traversal via download filename ⚠️ LOW RISK
|
||||||
### H4. No authentication ❌ REJECTED (internal service)
|
**Severity:** HIGH — downgraded
|
||||||
### H5. Memory leak in locks ✅ FIXED (bg cleanup every 30min)
|
**Analysis:** `os.path.basename("../../../etc/passwd")` → `"passwd"`, traversal impossible. Risk accepted.
|
||||||
### H6. Race condition in URL locking ✅ FIXED (DB re-check inside lock)
|
|
||||||
### H7. CSV export bounded ❌ REJECTED (acceptable for internal tool)
|
---
|
||||||
|
|
||||||
|
### H3. Sensitive data in API (source_ip) ❌ REJECTED
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Decision:** `source_ip` and `initiator` are features. Internal service — acceptable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### H4. No authentication on API ❌ REJECTED
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Decision:** Internal service; not exposed to public internet.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### H5. Memory leak in lock dictionaries ✅ FIXED
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Fix:** Background cleanup tasks every 30 minutes in `main.py` for both `_url_locks` (harvester) and `_llm_locks` (web). Tasks spawned via `asyncio.create_task()` in lifespan, gracefully cancelled on shutdown.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### H6. Race condition in URL locking ✅ FIXED
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Fix:** DB re-check (`sha256` dedup) happens inside the URL lock critical section, preventing parallel scans of the same asset.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### H7. CSV export unbounded ❌ REJECTED
|
||||||
|
**Severity:** HIGH
|
||||||
|
**Decision:** Acceptable for internal tool. Not exposed to public.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## MEDIUM (8)
|
## MEDIUM (8)
|
||||||
|
|
||||||
### M1. No LLM response schema validation
|
### M1. No LLM response schema validation ✅ FIXED
|
||||||
**Severity:** MEDIUM
|
**Severity:** MEDIUM
|
||||||
**File:** `core/llm.py:80-82`
|
**File:** `core/llm.py:25-28`
|
||||||
|
|
||||||
**Problem:** LLM response parsed as JSON but not validated against schema. Missing `report.verdict` → Jinja2 renders empty string → CSS broken.
|
**Fix:** `_validate_report()` applies defaults for missing fields:
|
||||||
|
- `verdict` → `"unknown"`
|
||||||
**Fix:** Pydantic model для валидации LLM response.
|
- `summary` → `"No summary provided"`
|
||||||
|
- `analysis` → `"No analysis provided"`
|
||||||
|
- `severity_rating` → `"unknown"`
|
||||||
|
- Also unwraps JSON from markdown code fences (```json ... ```).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### M2. No CSRF protection
|
### M2. No CSRF protection ⬜ OPEN
|
||||||
**Severity:** MEDIUM
|
**Severity:** MEDIUM
|
||||||
**File:** `routes/web.py:205-274`
|
**File:** `routes/web.py:205-274`
|
||||||
|
|
||||||
**Problem:** POST `/api/v1/findings/{id}/analyze` без CSRF token.
|
**Problem:** POST `/api/v1/findings/{id}/analyze` has no CSRF token. While the service is internal, a CSRF attack from the same origin could trigger unwanted LLM analysis.
|
||||||
|
|
||||||
**Fix:** Добавить CSRF token для всех POST endpoints.
|
**Suggested fix:** Add a CSRF middleware or token check for state-changing POST endpoints.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### M3. No security headers
|
### M3. No security headers ✅ FIXED
|
||||||
**Severity:** MEDIUM
|
**Severity:** MEDIUM
|
||||||
**File:** `main.py`
|
**File:** `main.py:95-113`
|
||||||
|
|
||||||
**Problem:** Отсутствие CSP, X-Content-Type-Options, X-Frame-Options, X-XSS-Protection.
|
**Fix:** `SecurityHeadersMiddleware` sets on all responses:
|
||||||
|
- `X-Content-Type-Options: nosniff`
|
||||||
**Fix:** Middleware для security headers.
|
- `X-Frame-Options: DENY`
|
||||||
|
- `X-XSS-Protection: 1; mode=block`
|
||||||
|
- `Referrer-Policy: strict-origin-when-cross-origin`
|
||||||
|
- `Permissions-Policy: geolocation=(), microphone=()`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### M4. SQLite without WAL mode
|
### M4. SQLite without WAL mode ⬜ OPEN
|
||||||
**Severity:** MEDIUM
|
**Severity:** MEDIUM
|
||||||
**File:** `db/engine.py:12`
|
**File:** `db/engine.py:12`
|
||||||
|
|
||||||
**Problem:** Concurrent readers block writers → poor performance under load.
|
**Problem:** No `PRAGMA journal_mode=WAL` — concurrent readers block writers, causing degraded performance under load.
|
||||||
|
|
||||||
**Fix:** `PRAGMA journal_mode=WAL` in connection setup.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### M5. Scoped npm packages not supported
|
|
||||||
**Severity:** MEDIUM
|
|
||||||
**File:** `core/nexus.py:54-70`
|
|
||||||
|
|
||||||
**Problem:** `extract_npm_info` returns `None` для `@scope/package` → пропускаются сканирования.
|
|
||||||
|
|
||||||
**Fix:** Обновить extractor для scoped packages.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### M6. Dashboard stats — potential IndexError
|
|
||||||
**Severity:** MEDIUM
|
|
||||||
**File:** `routes/api_scans.py:145-147`
|
|
||||||
|
|
||||||
**Problem:** `dashboard["latest_flagged"][0]` — IndexError если `latest_flagged` пустой.
|
|
||||||
|
|
||||||
|
**Suggested fix:** Add WAL mode in connection setup:
|
||||||
```python
|
```python
|
||||||
"latest_scan_at": dashboard["latest_flagged"][0].started_at.isoformat()
|
async with _engine.connect() as conn:
|
||||||
|
await conn.execute(text("PRAGMA journal_mode=WAL"))
|
||||||
```
|
```
|
||||||
|
|
||||||
**Fix:** Guard с `if dashboard.get("latest_flagged")`.
|
---
|
||||||
|
|
||||||
|
### M5. Scoped npm packages not supported ✅ FIXED
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**File:** `core/nexus.py:75-80`
|
||||||
|
|
||||||
|
**Fix:** `extract_npm_info` handles `@scope/name` scoped packages:
|
||||||
|
```python
|
||||||
|
if parts[1].startswith("@"):
|
||||||
|
name = f"{parts[1]}/{parts[2]}"
|
||||||
|
short_name = parts[2]
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### M7. Error message HTML escaping
|
### M6. Dashboard stats — potential IndexError ✅ FIXED
|
||||||
|
**Severity:** MEDIUM
|
||||||
|
**File:** `routes/api_scans.py:146-148`
|
||||||
|
|
||||||
|
**Fix:** Guard checks `latest` is non-empty and has `started_at`:
|
||||||
|
```python
|
||||||
|
latest[0].started_at.isoformat() if latest and latest[0].started_at else None
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### M7. Error message HTML escaping ✅ FIXED
|
||||||
**Severity:** MEDIUM
|
**Severity:** MEDIUM
|
||||||
**File:** `web/templates/scan_detail.html:30`
|
**File:** `web/templates/scan_detail.html:30`
|
||||||
|
|
||||||
**Problem:** `scan.error_message` rendered в template — если содержит HTML/JS, может сломать UI.
|
**Fix:** Jinja2 autoescape handles HTML in `scan.error_message`. No additional escaping required.
|
||||||
|
|
||||||
**Fix:** Jinja2 autoescape handles this, но стоит добавить explicit escaping для `code` fields.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### M8. Unknown ecosystem defaults to pypi ✅ FIXED
|
### M8. Unknown ecosystem defaults to pypi ✅ FIXED
|
||||||
**Severity:** MEDIUM
|
**Severity:** MEDIUM
|
||||||
**Fix:** `_detect_ecosystem()` возвращает `None` → webhook reject с `"unknown_ecosystem"`.
|
**File:** `routes/webhooks.py:58-69`
|
||||||
**Duplicate:** L6.
|
|
||||||
|
**Fix:** `_detect_ecosystem()` returns `None` for unknown formats; webhook handler rejects with `"unknown_ecosystem"` error.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## LOW (6)
|
## LOW (6)
|
||||||
|
|
||||||
### L1. Dockerfile grep hack ✅ FIXED (`uv pip install . --system`)
|
### L1. Dockerfile grep hack ✅ FIXED
|
||||||
### L2. Health check without DB ✅ FIXED (`/health/dependencies`)
|
**Severity:** LOW
|
||||||
|
**Fix:** Replaced with `uv pip install . --system`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### L3. No backup strategy for SQLite
|
### L2. Health check without DB ✅ FIXED
|
||||||
|
**Severity:** LOW
|
||||||
|
**File:** `main.py:139-140`
|
||||||
|
|
||||||
|
**Fix:** `/health/dependencies` endpoint checks database connectivity and Nexus API reachability.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### L3. No backup strategy for SQLite ⬜ OPEN
|
||||||
**Severity:** LOW
|
**Severity:** LOW
|
||||||
**Risk:** Crash → corrupted database → data loss.
|
**Risk:** Crash → corrupted database → data loss.
|
||||||
|
|
||||||
**Fix:** Регулярные backups через cron или switch to PostgreSQL for production.
|
**Suggested fix:** Add documentation for regular backups via cron or a backup script. Consider PostgreSQL for production deployments.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### L4. Dead code — `parse_package_path` unused in harvester
|
### L4. Dead code — `parse_package_path` ✅ FIXED
|
||||||
**Severity:** LOW
|
**Severity:** LOW
|
||||||
**File:** `core/nexus.py:93-99`
|
**File:** `core/nexus.py:113`
|
||||||
|
|
||||||
**Problem:** Функция определена но не используется в harvester pipeline.
|
**Resolution:** Function is actively used in `routes/web.py` and `routes/api_packages.py`. Not dead code.
|
||||||
|
|
||||||
**Fix:** Убрать или интегрировать.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### L5. Hardcoded LLM API base URL
|
### L5. Hardcoded LLM API base URL ⬜ OPEN
|
||||||
**Severity:** LOW
|
**Severity:** LOW
|
||||||
**File:** `constants.py:139`
|
**File:** `constants.py:140`
|
||||||
|
|
||||||
**Problem:** Default `https://api.openai.com/v1` — unexpected API calls для пользователей локальных моделей.
|
**Problem:** `LLM_DEFAULT_API_BASE = "https://api.openai.com/v1"` — unexpected API calls for users of local models who forget to set `LLM_API_BASE`.
|
||||||
|
|
||||||
**Fix:** Better default или warning at startup.
|
**Suggested fix:** Either log a warning at startup or change default to an empty/required value.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### L6. Unknown ecosystem defaults to pypi (webhook)
|
### L6. Unknown ecosystem defaults to pypi (webhook) ✅ FIXED
|
||||||
**Severity:** LOW
|
**Severity:** LOW
|
||||||
**File:** `routes/webhooks.py:62`
|
**File:** `routes/webhooks.py:62`
|
||||||
|
|
||||||
**Problem:** Неизвестный format → fallback к pypi. Maven/NuGet webhooks будут сканироваться как PyPI пакеты.
|
**Fix:** Same as M8. `_detect_ecosystem()` returns `None` for unknown formats; webhook rejects.
|
||||||
|
|
||||||
**Fix:** Явно reject неизвестные ecosystems.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Implementation Plan
|
## Implementation Plan
|
||||||
|
|
||||||
### Phase 1 — P0 (Critical)
|
### Phase 1 — P0 (Critical) — COMPLETED
|
||||||
|
|
||||||
| # | Task | Status |
|
| # | Task | Status |
|
||||||
|---|------|--------|
|
|---|------|--------|
|
||||||
@@ -220,7 +255,7 @@ response = await client.get(download_url) # no validation
|
|||||||
| 4 | LLM verdict whitelist + prompt injection | ⚠️ PARTIAL |
|
| 4 | LLM verdict whitelist + prompt injection | ⚠️ PARTIAL |
|
||||||
| 5 | Path traversal fix | ⚠️ LOW RISK |
|
| 5 | Path traversal fix | ⚠️ LOW RISK |
|
||||||
|
|
||||||
### Phase 2 — P1 (High)
|
### Phase 2 — P1 (High) — COMPLETED
|
||||||
|
|
||||||
| # | Task | Status |
|
| # | Task | Status |
|
||||||
|---|------|--------|
|
|---|------|--------|
|
||||||
@@ -235,30 +270,39 @@ response = await client.get(download_url) # no validation
|
|||||||
|
|
||||||
| # | Task | Status |
|
| # | Task | Status |
|
||||||
|---|------|--------|
|
|---|------|--------|
|
||||||
| 12 | LLM response validation (Pydantic) | ⬜ |
|
| 12 | LLM response validation (Pydantic/defaults) | ✅ FIXED |
|
||||||
| 13 | CSRF protection | ⬜ |
|
| 13 | CSRF protection | ⬜ OPEN |
|
||||||
| 14 | Security headers middleware | ⬜ |
|
| 14 | Security headers middleware | ✅ FIXED |
|
||||||
| 15 | SQLite WAL mode | ⬜ |
|
| 15 | SQLite WAL mode | ⬜ OPEN |
|
||||||
| 16 | Scoped npm support | ⬜ |
|
| 16 | Scoped npm support | ✅ FIXED |
|
||||||
| 17 | Dashboard None guard | ⬜ |
|
| 17 | Dashboard None guard | ✅ FIXED |
|
||||||
| 18 | `serialize_finding` вместо `**f.data` | ✅ FIXED |
|
| 18 | Reject unknown ecosystem | ✅ FIXED |
|
||||||
| 19 | `_scan_component` try/except | ✅ FIXED |
|
|
||||||
| 20 | Reject unknown ecosystem | ✅ FIXED |
|
|
||||||
|
|
||||||
### Phase 4 — P3 (Low)
|
### Phase 4 — P3 (Low)
|
||||||
|
|
||||||
| # | Task | Status |
|
| # | Task | Status |
|
||||||
|---|------|--------|
|
|---|------|--------|
|
||||||
| 21 | Dockerfile deps | ✅ FIXED |
|
| 19 | Dockerfile deps | ✅ FIXED |
|
||||||
| 22 | Health check DB ping | ✅ FIXED |
|
| 20 | Health check DB ping | ✅ FIXED |
|
||||||
| 23 | Backup strategy docs | ⬜ |
|
| 21 | Backup strategy docs | ⬜ OPEN |
|
||||||
| 24 | Reject unknown ecosystems | ✅ FIXED (duplicate) | |
|
| 22 | Hardcoded LLM API base URL | ⬜ OPEN |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Remaining Open Items (4)
|
||||||
|
|
||||||
|
| # | Severity | Finding | Recommendation |
|
||||||
|
|---|----------|---------|----------------|
|
||||||
|
| M2 | MEDIUM | No CSRF protection on POST endpoints | Add CSRF middleware or token validation |
|
||||||
|
| M4 | MEDIUM | SQLite without WAL mode | Add `PRAGMA journal_mode=WAL` in engine setup |
|
||||||
|
| L3 | LOW | No backup strategy for SQLite | Document backup procedures or switch to PostgreSQL |
|
||||||
|
| L5 | LOW | Hardcoded LLM default API base URL | Log warning on startup or require explicit configuration |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Test Coverage Gaps
|
## Test Coverage Gaps
|
||||||
|
|
||||||
The existing 85 tests do NOT cover:
|
The existing 137 tests (101 unit + 36 e2e) do NOT cover:
|
||||||
|
|
||||||
- [ ] SSRF prevention (malicious downloadUrl)
|
- [ ] SSRF prevention (malicious downloadUrl)
|
||||||
- [ ] Webhook signature validation with empty secret
|
- [ ] Webhook signature validation with empty secret
|
||||||
@@ -266,20 +310,26 @@ The existing 85 tests do NOT cover:
|
|||||||
- [ ] Rate limiting on webhook endpoint
|
- [ ] Rate limiting on webhook endpoint
|
||||||
- [ ] Authentication on API endpoints
|
- [ ] Authentication on API endpoints
|
||||||
- [ ] LLM prompt injection
|
- [ ] LLM prompt injection
|
||||||
- [ ] LLM response schema validation
|
- [ ] CSRF protection (M2 — open)
|
||||||
- [ ] CSRF protection
|
|
||||||
- [ ] Security headers presence
|
- [ ] Security headers presence
|
||||||
- [ ] Memory leak in lock dictionaries
|
- [ ] SQLite WAL mode behavior
|
||||||
- [ ] Race condition in URL locking
|
|
||||||
- [ ] Scoped npm package extraction
|
|
||||||
- [ ] Dashboard IndexError on empty data
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Recommendations
|
## Recommendations
|
||||||
|
|
||||||
1. **Immediate:** Implement C1-C5 before any production deployment
|
1. **Immediate:** No critical items remain open. C1, C3 are fixed; C2, C4 are accepted.
|
||||||
2. **Short-term:** Implement H1-H7 within first sprint
|
2. **Short-term:** Address M2 (CSRF) and M4 (WAL mode) — both are straightforward, low-risk fixes.
|
||||||
3. **Medium-term:** Implement M1-M8 within first month
|
3. **Long-term:** Address L3 (backup strategy) and L5 (LLM default URL) during routine maintenance.
|
||||||
4. **Long-term:** Implement L1-L6 during routine maintenance
|
4. **Ongoing:** Add security-focused tests for resolved findings to prevent regressions.
|
||||||
5. **Ongoing:** Add security-focused tests for all findings above
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- **Consolidation:** This document supersedes `improvements.md` and `final-plan.md` (deleted). All verified fixes from those plans are incorporated.
|
||||||
|
- **LLM retry:** Implemented with exponential backoff (2s, 4s, 8s, max 3 attempts) in `core/llm.py:126-152`.
|
||||||
|
- **Lock cleanup:** Background tasks in `main.py:59-75` clean up `_url_locks` and `_llm_locks` every 30 minutes.
|
||||||
|
- **Race condition:** SHA256 dedup check runs inside URL lock critical section in harvester.
|
||||||
|
- **Scoped npm:** `extract_npm_info` in `core/nexus.py:75-80` handles `@scope/name` packages.
|
||||||
|
- **Dashboard guard:** `routes/api_scans.py:147` checks `latest and latest[0].started_at` before access.
|
||||||
|
|||||||
17
.pre-commit-config.yaml
Normal file
17
.pre-commit-config.yaml
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
repos:
|
||||||
|
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||||
|
rev: v0.8.0
|
||||||
|
hooks:
|
||||||
|
- id: ruff
|
||||||
|
args: [--fix]
|
||||||
|
- id: ruff-format
|
||||||
|
|
||||||
|
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||||
|
rev: v5.0.0
|
||||||
|
hooks:
|
||||||
|
- id: trailing-whitespace
|
||||||
|
- id: end-of-file-fixer
|
||||||
|
- id: check-yaml
|
||||||
|
- id: check-toml
|
||||||
|
- id: check-added-large-files
|
||||||
|
- id: detect-private-key
|
||||||
66
AGENTS.md
66
AGENTS.md
@@ -26,7 +26,7 @@ For local development without Docker:
|
|||||||
make install dev
|
make install dev
|
||||||
export $(cat .env | xargs)
|
export $(cat .env | xargs)
|
||||||
python -m guarddog_nexus.main
|
python -m guarddog_nexus.main
|
||||||
make test # 135 tests
|
make test # 168 tests
|
||||||
make lint # ruff
|
make lint # ruff
|
||||||
make format # ruff format + fix
|
make format # ruff format + fix
|
||||||
```
|
```
|
||||||
@@ -56,12 +56,12 @@ guarddog_nexus/
|
|||||||
├── web/ # Static assets
|
├── web/ # Static assets
|
||||||
│ ├── templates/ # Jinja2 templates
|
│ ├── templates/ # Jinja2 templates
|
||||||
│ └── static/ # CSS, JS
|
│ └── static/ # CSS, JS
|
||||||
├── schemas.py # Pydantic models + serialize_finding helper
|
├── schemas.py # Pydantic models + serialize_finding() helper
|
||||||
├── config.py # env-var configuration dataclass
|
├── config.py # env-var configuration dataclass + _env_int()
|
||||||
├── constants.py # all magic strings/limits
|
├── constants.py # all magic strings/limits + SUPPORTED_ECOSYSTEMS
|
||||||
├── i18n.py # RU/EN translation dictionaries
|
├── i18n.py # RU/EN translation dictionaries
|
||||||
├── logging_setup.py # JSON logging + syslog
|
├── logging_setup.py # JSON logging + syslog
|
||||||
└── main.py # FastAPI app, middleware, lifepan
|
└── main.py # FastAPI app, middleware, lifespan, /health/dependencies
|
||||||
```
|
```
|
||||||
|
|
||||||
**Data flow:**
|
**Data flow:**
|
||||||
@@ -70,7 +70,7 @@ guarddog_nexus/
|
|||||||
3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates
|
3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates
|
||||||
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
|
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
|
||||||
5. Findings stored in SQLite (`scans` + `findings` tables)
|
5. Findings stored in SQLite (`scans` + `findings` tables)
|
||||||
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends each finding to the configured model with retry logic. `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated with defaults for missing fields.
|
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.). Progress tracked via Prometheus counters: `guarddog_llm_analyzed_total` and `guarddog_llm_pending_total`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -81,10 +81,14 @@ guarddog_nexus/
|
|||||||
- **Line length:** 100 (ruff)
|
- **Line length:** 100 (ruff)
|
||||||
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
|
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
|
||||||
- **Format:** `ruff format guarddog_nexus tests`
|
- **Format:** `ruff format guarddog_nexus tests`
|
||||||
- **Tests:** `pytest -v` (135 tests, pytest-asyncio auto mode)
|
- **Tests:** `pytest -v` (168 tests, pytest-asyncio auto mode)
|
||||||
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
|
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
|
||||||
- **No comments** in code unless explicitly requested
|
- **No comments** in code unless explicitly requested
|
||||||
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context
|
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context
|
||||||
|
- **Config validation:** `_env_int` logs a warning on invalid values instead of crashing
|
||||||
|
- **Type checking:** `make typecheck` runs `mypy guarddog_nexus` (strict mode)
|
||||||
|
- **Pre-commit:** `.pre-commit-config.yaml` with ruff, ruff-format, trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-added-large-files, detect-private-key
|
||||||
|
- **CSV export:** `_csv_safe()` in `api_scans.py` prepends `'` to values starting with `=`, `+`, `-`, `@` — blocks formula injection when opening CSV in spreadsheet apps
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -98,10 +102,16 @@ All via environment variables, defined in `config.py`. Key ones:
|
|||||||
| `NEXUS_ALLOWED_HOSTS` | host from `NEXUS_URL` | comma-separated, SSRF protection |
|
| `NEXUS_ALLOWED_HOSTS` | host from `NEXUS_URL` | comma-separated, SSRF protection |
|
||||||
| `WEBHOOK_SECRET` | `""` | HMAC-SHA256 validation |
|
| `WEBHOOK_SECRET` | `""` | HMAC-SHA256 validation |
|
||||||
| `MAX_CONCURRENT_SCANS` | `4` | asyncio.Semaphore for guarddog processes |
|
| `MAX_CONCURRENT_SCANS` | `4` | asyncio.Semaphore for guarddog processes |
|
||||||
|
| `SCAN_TIMEOUT_SECONDS` | `300` | per-package scan timeout |
|
||||||
|
| `GUARDDOG_BINARY` | `guarddog` | path to GuardDog CLI |
|
||||||
|
| `NEXUS_DOWNLOAD_TIMEOUT_SECONDS` | `120` | download timeout from Nexus |
|
||||||
|
| `NEXUS_API_TIMEOUT_SECONDS` | `30` | Nexus REST API timeout |
|
||||||
| `LLM_ENABLED` | `0` | `1` to enable analysis |
|
| `LLM_ENABLED` | `0` | `1` to enable analysis |
|
||||||
| `LLM_AUTO_ANALYZE` | `0` | `1` to auto-trigger after scan; `0` = manual via UI button |
|
| `LLM_AUTO_ANALYZE` | `0` | `1` to auto-trigger after scan; `0` = manual via UI button |
|
||||||
| `LLM_API_KEY` | `""` | OpenAI-compatible key |
|
| `LLM_API_KEY` | `""` | OpenAI-compatible key |
|
||||||
|
| `LLM_API_BASE` | `https://api.openai.com/v1` | OpenAI-compatible base URL |
|
||||||
| `LLM_MODEL` | `gpt-4o-mini` | |
|
| `LLM_MODEL` | `gpt-4o-mini` | |
|
||||||
|
| `LLM_TIMEOUT_SECONDS` | `30` | LLM request timeout |
|
||||||
| `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Semaphore for LLM calls |
|
| `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Semaphore for LLM calls |
|
||||||
| `DATABASE_PATH` | `data/guarddog.db` | |
|
| `DATABASE_PATH` | `data/guarddog.db` | |
|
||||||
|
|
||||||
@@ -127,13 +137,13 @@ Full list in `config.py`.
|
|||||||
| Value | UI |
|
| Value | UI |
|
||||||
|-------|----|
|
|-------|----|
|
||||||
| `None` | Show "Analyze with LLM" button (manual mode only) |
|
| `None` | Show "Analyze with LLM" button (manual mode only) |
|
||||||
| `{"status": "analyzing"}` | Show spinner |
|
| `{"status": "analyzing"}` | Show spinner (auto-polls via HTMX GET every 2s) |
|
||||||
| `{verdict:, summary:, ...}` | Show report + "Retry" link |
|
| `{verdict:, summary:, ...}` | Show report + "Retry" link |
|
||||||
|
|
||||||
**Auto mode** (`LLM_AUTO_ANALYZE=1`): analysis runs immediately after scan; button hidden.
|
**Auto mode** (`LLM_AUTO_ANALYZE=1`): analysis runs immediately after scan; button hidden.
|
||||||
**Manual mode** (`LLM_AUTO_ANALYZE=0`): user clicks button; button visible for each finding.
|
**Manual mode** (`LLM_AUTO_ANALYZE=0`): user clicks button; button visible for each finding.
|
||||||
|
|
||||||
Per-finding `asyncio.Lock` in `web.py` prevents concurrent analysis of the same finding. Retry passes `?retry=1` to bypass the idempotency guard.
|
Per-finding `asyncio.Lock` in `web.py` prevents concurrent analysis of the same finding. Retry passes `?retry=1` to bypass the idempotency guard. LLM response validated via `_validate_report()` — missing/invalid fields get defaults (`verdict→unknown`, `severity_rating→unknown`, etc.). Retry with exponential backoff: 2s, 4s, 8s (max 3 attempts). Reports can also be unwrapped from markdown code fences (```json ... ```).
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -164,7 +174,35 @@ docker compose down -v # stop + destroy volumes (make docker-destroy)
|
|||||||
docker compose logs -f # tail logs
|
docker compose logs -f # tail logs
|
||||||
```
|
```
|
||||||
|
|
||||||
The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install` step.
|
The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. Dependencies are installed before source code COPY for efficient layer caching. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds.
|
||||||
|
|
||||||
|
Logging driver configured as `json-file` with rotation (max-size: 10m, max-file: 3) for both `guarddog-nexus` and `nexus` services. Nexus service also has a HEALTHCHECK (`curl` to `/service/rest/v1/status`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Makefile targets
|
||||||
|
|
||||||
|
| Command | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `make install` | Install project dependencies |
|
||||||
|
| `make dev` | Install dev dependencies |
|
||||||
|
| `make test` | Run pytest -v |
|
||||||
|
| `make lint` | Ruff check |
|
||||||
|
| `make format` | Ruff format + fix |
|
||||||
|
| `make typecheck` | mypy strict mode |
|
||||||
|
| `make check` | lint + format + typecheck + test |
|
||||||
|
| `make run` | Start the app via `python -m guarddog_nexus.main` |
|
||||||
|
| `make setup-env` | Copy `.env.example` → `.env` if missing |
|
||||||
|
| `make docker-build` | Build Docker image |
|
||||||
|
| `make docker-up` | Build + start stack (`up -d --build`) |
|
||||||
|
| `make docker-down` | Stop stack |
|
||||||
|
| `make docker-destroy` | Stop + destroy volumes (`-v`) |
|
||||||
|
| `make docker-rebuild` | Down + up --build |
|
||||||
|
| `make docker-logs` | Tail logs |
|
||||||
|
| `make docker-ps` | `docker compose ps` |
|
||||||
|
| `make docker-shell` | Exec bash in guarddog-nexus container |
|
||||||
|
| `make docker-restart` | Restart guarddog-nexus service |
|
||||||
|
| `make clean` | Remove build artifacts |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -174,7 +212,7 @@ The Dockerfile uses `uv pip install . --system` to install the package and all d
|
|||||||
- Tests use in-memory SQLite (`:memory:`)
|
- Tests use in-memory SQLite (`:memory:`)
|
||||||
- `conftest.py` sets up `os.environ` before importing the app
|
- `conftest.py` sets up `os.environ` before importing the app
|
||||||
- Mock `guarddog` output via fixtures — no real CLI execution
|
- Mock `guarddog` output via fixtures — no real CLI execution
|
||||||
- 135 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, e2e flows
|
- 168 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, config, schemas, engine, e2e flows
|
||||||
- E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling
|
- E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling
|
||||||
|
|
||||||
When adding features:
|
When adding features:
|
||||||
@@ -218,8 +256,14 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|
|||||||
|
|
||||||
- **AI-generated code:** all code in this repository was generated by an AI assistant (Claude). Review carefully before production use.
|
- **AI-generated code:** all code in this repository was generated by an AI assistant (Claude). Review carefully before production use.
|
||||||
- **No Nexus Pro required:** the system works with Nexus OSS. Webhooks can be triggered manually or via community plugins.
|
- **No Nexus Pro required:** the system works with Nexus OSS. Webhooks can be triggered manually or via community plugins.
|
||||||
|
- **Anonymous Nexus access:** all Nexus REST API calls use anonymous access (no BasicAuth). Ensure your Nexus instance allows anonymous read access to repositories.
|
||||||
- **GuardDog deadlocks:** GuardDog is CPU-intensive. Use `MAX_CONCURRENT_SCANS` to avoid resource exhaustion.
|
- **GuardDog deadlocks:** GuardDog is CPU-intensive. Use `MAX_CONCURRENT_SCANS` to avoid resource exhaustion.
|
||||||
- **LLM may be slow:** increase `LLM_TIMEOUT_SECONDS` for large models. Set `LLM_MAX_CONCURRENT_ANALYSES` to limit parallel requests.
|
- **LLM may be slow:** increase `LLM_TIMEOUT_SECONDS` for large models. Set `LLM_MAX_CONCURRENT_ANALYSES` to limit parallel requests.
|
||||||
|
- **Security headers:** `SecurityHeadersMiddleware` sets X-Content-Type-Options, X-Frame-Options, X-XSS-Protection, Referrer-Policy, and Permissions-Policy on all responses.
|
||||||
|
- **Background tasks:** URL lock and LLM lock cleanup tasks run every 30 minutes via the lifespan; they are gracefully cancelled on shutdown.
|
||||||
|
- **`serialize_finding()` helper** in `schemas.py` prevents `**f.data` field injection in API responses by extracting only known fields.
|
||||||
|
- **`SUPPORTED_ECOSYSTEMS`** constant in `constants.py` defines the accepted ecosystem set (`pypi`, `go`, `npm`).
|
||||||
|
- **`/_health/dependencies`** endpoint checks database connectivity and Nexus API reachability.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
12
Dockerfile
12
Dockerfile
@@ -7,11 +7,15 @@ COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /usr/local/bin/
|
|||||||
|
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
|
# Install dependencies first for layer caching (source changes don't invalidate)
|
||||||
COPY pyproject.toml README.md ./
|
COPY pyproject.toml README.md ./
|
||||||
COPY guarddog_nexus/ guarddog_nexus/
|
RUN mkdir -p guarddog_nexus && echo '__version__ = "0.1.0"' > guarddog_nexus/__init__.py
|
||||||
|
|
||||||
RUN uv pip install . --system
|
RUN uv pip install . --system
|
||||||
RUN uv pip install --system "guarddog>=2.10.0"
|
RUN uv pip install --system "guarddog>=2.10.0"
|
||||||
|
RUN rm -rf guarddog_nexus
|
||||||
|
|
||||||
|
# Application source (frequently changes — cached dependency layers preserved)
|
||||||
|
COPY guarddog_nexus/ guarddog_nexus/
|
||||||
|
|
||||||
RUN mkdir -p /data /tmp/guarddog-nexus
|
RUN mkdir -p /data /tmp/guarddog-nexus
|
||||||
|
|
||||||
@@ -21,7 +25,7 @@ ENV PYTHONDONTWRITEBYTECODE=1
|
|||||||
|
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
|
|
||||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
|
HEALTHCHECK --interval=30s --timeout=3s --start-period=15s --retries=3 \
|
||||||
CMD python -c "from urllib.request import urlopen; urlopen('http://localhost:8080/health')"
|
CMD curl -sf http://localhost:8080/health/dependencies || exit 1
|
||||||
|
|
||||||
CMD ["python", "-m", "guarddog_nexus.main"]
|
CMD ["python", "-m", "guarddog_nexus.main"]
|
||||||
|
|||||||
26
Makefile
26
Makefile
@@ -1,4 +1,4 @@
|
|||||||
.PHONY: install dev test lint format docker-build docker-up docker-down docker-destroy docker-rebuild docker-logs clean
|
.PHONY: install dev test lint format typecheck check run setup-env docker-build docker-up docker-down docker-destroy docker-rebuild docker-logs docker-ps docker-shell docker-restart clean
|
||||||
|
|
||||||
install:
|
install:
|
||||||
pip install -e .
|
pip install -e .
|
||||||
@@ -16,6 +16,19 @@ format:
|
|||||||
ruff format guarddog_nexus tests
|
ruff format guarddog_nexus tests
|
||||||
ruff check --fix guarddog_nexus tests
|
ruff check --fix guarddog_nexus tests
|
||||||
|
|
||||||
|
typecheck:
|
||||||
|
mypy guarddog_nexus
|
||||||
|
|
||||||
|
check: lint format typecheck test
|
||||||
|
@echo "All checks passed"
|
||||||
|
|
||||||
|
run:
|
||||||
|
python -m guarddog_nexus.main
|
||||||
|
|
||||||
|
setup-env:
|
||||||
|
@test -f .env || cp .env.example .env
|
||||||
|
@echo ".env ready"
|
||||||
|
|
||||||
docker-build:
|
docker-build:
|
||||||
docker compose build
|
docker compose build
|
||||||
|
|
||||||
@@ -29,11 +42,20 @@ docker-destroy:
|
|||||||
docker compose down -v
|
docker compose down -v
|
||||||
|
|
||||||
docker-rebuild:
|
docker-rebuild:
|
||||||
docker compose down && docker compose up -d --build
|
docker compose down; docker compose up -d --build
|
||||||
|
|
||||||
docker-logs:
|
docker-logs:
|
||||||
docker compose logs -f
|
docker compose logs -f
|
||||||
|
|
||||||
|
docker-ps:
|
||||||
|
docker compose ps
|
||||||
|
|
||||||
|
docker-shell:
|
||||||
|
docker compose exec guarddog-nexus bash
|
||||||
|
|
||||||
|
docker-restart:
|
||||||
|
docker compose restart guarddog-nexus
|
||||||
|
|
||||||
clean:
|
clean:
|
||||||
rm -rf dist build *.egg-info
|
rm -rf dist build *.egg-info
|
||||||
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
|
find . -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
|
||||||
|
|||||||
45
README.en.md
45
README.en.md
@@ -4,12 +4,13 @@ Integration of [GuardDog](https://github.com/DataDog/guarddog) (package vulnerab
|
|||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Automatic scanning** via Nexus webhooks on package cache updates
|
- **Automatic scanning** via Nexus webhooks on package updates (`UPDATED` only)
|
||||||
- **Multi-ecosystem support** — PyPI, Go, npm (any format via proxy repositories)
|
- **Multi-ecosystem support** — PyPI, Go, npm (including scoped packages `@scope/name`); unknown ecosystems explicitly rejected
|
||||||
- **REST API** for scan results, findings, statistics, and CSV export
|
- **REST API** for scan results, findings, statistics, and CSV export
|
||||||
- **Web dashboard** with scan tables, filtering, and LLM-powered analysis
|
- **Web dashboard** with scan tables, filtering, and LLM-powered analysis
|
||||||
- **LLM analysis** — automated security analysis of each finding via OpenAI-compatible APIs (optional, configurable)
|
- **LLM analysis** — automated security analysis of each finding via OpenAI-compatible APIs (optional, configurable); parallel analysis via `asyncio.gather`
|
||||||
- **Deduplication** by URL and SHA256 — identical content scanned only once
|
- **Deduplication** by URL and SHA256 — identical content scanned only once
|
||||||
|
- **SSRF protection** — download URL validation via `NEXUS_ALLOWED_HOSTS`
|
||||||
- **Structured JSON logging** with optional syslog output
|
- **Structured JSON logging** with optional syslog output
|
||||||
- **Docker Compose** — full stack deployment with Nexus in one command
|
- **Docker Compose** — full stack deployment with Nexus in one command
|
||||||
|
|
||||||
@@ -155,7 +156,41 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|
|||||||
| Method | Path | Description |
|
| Method | Path | Description |
|
||||||
|--------|------|-------------|
|
|--------|------|-------------|
|
||||||
| GET | `/health` | Health check |
|
| GET | `/health` | Health check |
|
||||||
| GET | `/metrics` | Prometheus-compatible metrics |
|
| GET | `/health/dependencies` | DB and Nexus API connectivity check |
|
||||||
|
| GET | `/metrics` | Prometheus metrics: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` |
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
- Webhooks support HMAC-SHA256 signature validation via `WEBHOOK_SECRET`
|
||||||
|
- Nexus client uses anonymous access (no BasicAuth) — ensure Nexus allows anonymous read access
|
||||||
|
- SSRF protection: download URLs validated against `NEXUS_ALLOWED_HOSTS`
|
||||||
|
- Security headers on all responses: `X-Content-Type-Options`, `X-Frame-Options`, `X-XSS-Protection`, `Referrer-Policy`, `Permissions-Policy`
|
||||||
|
- Scan results stored in local SQLite database
|
||||||
|
- Temporary package files deleted after scanning
|
||||||
|
|
||||||
|
## Makefile Targets
|
||||||
|
|
||||||
|
| Command | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `make install` | Install project dependencies |
|
||||||
|
| `make dev` | Install dev dependencies |
|
||||||
|
| `make test` | Run pytest -v |
|
||||||
|
| `make lint` | Ruff check |
|
||||||
|
| `make format` | Ruff format + fix |
|
||||||
|
| `make typecheck` | mypy strict mode |
|
||||||
|
| `make check` | lint + format + typecheck + test |
|
||||||
|
| `make run` | Start app locally |
|
||||||
|
| `make setup-env` | Copy `.env.example` → `.env` if missing |
|
||||||
|
| `make docker-build` | Build Docker image |
|
||||||
|
| `make docker-up` | Build + start stack |
|
||||||
|
| `make docker-down` | Stop stack |
|
||||||
|
| `make docker-destroy` | Stop + destroy volumes |
|
||||||
|
| `make docker-rebuild` | Down + up --build |
|
||||||
|
| `make docker-logs` | Tail logs |
|
||||||
|
| `make docker-ps` | `docker compose ps` |
|
||||||
|
| `make docker-shell` | Exec bash in guarddog-nexus container |
|
||||||
|
| `make docker-restart` | Restart guarddog-nexus service |
|
||||||
|
| `make clean` | Remove build artifacts |
|
||||||
|
|
||||||
## LLM Analysis
|
## LLM Analysis
|
||||||
|
|
||||||
@@ -226,7 +261,7 @@ guarddog-nexus/
|
|||||||
│ ├── i18n.py # RU/EN translations
|
│ ├── i18n.py # RU/EN translations
|
||||||
│ ├── logging_setup.py # JSON structured logging
|
│ ├── logging_setup.py # JSON structured logging
|
||||||
│ └── main.py # FastAPI app entry point
|
│ └── main.py # FastAPI app entry point
|
||||||
├── tests/ # pytest tests (85 tests)
|
├── tests/ # pytest tests (168 tests)
|
||||||
├── scripts/ # Setup scripts
|
├── scripts/ # Setup scripts
|
||||||
├── docker-compose.yml
|
├── docker-compose.yml
|
||||||
├── Dockerfile
|
├── Dockerfile
|
||||||
|
|||||||
25
README.md
25
README.md
@@ -1,15 +1,16 @@
|
|||||||
# GuardDog Nexus
|
# GuardDog Nexus
|
||||||
|
|
||||||
Интеграция [GuardDog](https://github.com/DataDog GuardDog) (сканер уязвимостей пакетов PyPI) с [Sonatype Nexus Repository Manager]. Автоматически сканирует Python-пакеты, хранящиеся в Nexus, на наличие уязвимостей, вредоносного кода и подозрительных паттернов при каждом обновлении или добавлении пакета.
|
Интеграция [GuardDog](https://github.com/DataDog/guarddog) (сканер уязвимостей пакетов) с [Sonatype Nexus Repository Manager]. Автоматически сканирует пакеты (PyPI, Go, npm), хранящиеся в Nexus, на наличие уязвимостей, вредоносного кода и подозрительных паттернов при каждом обновлении пакета через вебхуки.
|
||||||
|
|
||||||
## Возможности
|
## Возможности
|
||||||
|
|
||||||
- **Автоматическое сканирование** по вебхукам Nexus при создании/обновлении пакетов
|
- **Автоматическое сканирование** по вебхукам Nexus при обновлении пакетов (только `UPDATED`)
|
||||||
- **Поддержка нескольких экосистем** — PyPI, Go, npm (любой формат через прокси-репозитории Nexus)
|
- **Поддержка нескольких экосистем** — PyPI, Go, npm (включая scoped-пакеты `@scope/name`); неизвестные экосистемы явно отклоняются
|
||||||
- **REST API** для просмотра результатов сканирования, уязвимостей, статистики и экспорта в CSV
|
- **REST API** для просмотра результатов сканирования, уязвимостей, статистики и экспорта в CSV
|
||||||
- **Веб-интерфейс** с дашбордом, таблицами сканирований и фильтрацией по уязвимостям
|
- **Веб-интерфейс** с дашбордом, таблицами сканирований и фильтрацией по уязвимостям
|
||||||
- **LLM-анализ** — автоматический разбор каждой уязвимости через OpenAI-совместимые API (опционально, настраивается)
|
- **LLM-анализ** — автоматический разбор каждой уязвимости через OpenAI-совместимые API (опционально, настраивается); параллельный анализ через `asyncio.gather`
|
||||||
- **Дедупликация** по URL и SHA256 — один и тот же пакет сканируется один раз
|
- **Дедупликация** по URL и SHA256 — один и тот же пакет сканируется один раз
|
||||||
|
- **Защита от SSRF** — валидация URL загрузки через `NEXUS_ALLOWED_HOSTS`
|
||||||
- **Структурированное логирование** в формате JSON с опциональной отправкой в syslog
|
- **Структурированное логирование** в формате JSON с опциональной отправкой в syslog
|
||||||
- **Docker Compose** для развёртывания приложения, Nexus и настройки в одном стеке
|
- **Docker Compose** для развёртывания приложения, Nexus и настройки в одном стеке
|
||||||
|
|
||||||
@@ -186,7 +187,8 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|
|||||||
| Метод | Путь | Описание |
|
| Метод | Путь | Описание |
|
||||||
|-------|------|----------|
|
|-------|------|----------|
|
||||||
| GET | `/health` | Проверка работоспособности |
|
| GET | `/health` | Проверка работоспособности |
|
||||||
| GET | `/metrics` | Метрики в формате Prometheus |
|
| GET | `/health/dependencies` | Проверка БД и доступности Nexus API |
|
||||||
|
| GET | `/metrics` | Prometheus-метрики: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` |
|
||||||
|
|
||||||
## Веб-интерфейс
|
## Веб-интерфейс
|
||||||
|
|
||||||
@@ -238,21 +240,30 @@ guarddog-nexus/
|
|||||||
|---------|----------|
|
|---------|----------|
|
||||||
| `make install` | Установка зависимостей проекта |
|
| `make install` | Установка зависимостей проекта |
|
||||||
| `make dev` | Установка зависимостей для разработки |
|
| `make dev` | Установка зависимостей для разработки |
|
||||||
| `make test` | Запуск тестов |
|
| `make test` | Запуск тестов (168) |
|
||||||
| `make lint` | Проверка кода через Ruff |
|
| `make lint` | Проверка кода через Ruff |
|
||||||
| `make format` | Форматирование кода через Ruff |
|
| `make format` | Форматирование кода через Ruff |
|
||||||
|
| `make typecheck` | Проверка типов через mypy (strict mode) |
|
||||||
|
| `make check` | lint + format + typecheck + test (все проверки) |
|
||||||
|
| `make run` | Запуск приложения локально |
|
||||||
|
| `make setup-env` | Копирование `.env.example` → `.env` (если отсутствует) |
|
||||||
| `make docker-build` | Сборка Docker-образа |
|
| `make docker-build` | Сборка Docker-образа |
|
||||||
| `make docker-up` | Пересборка и запуск стека (`up -d --build`) |
|
| `make docker-up` | Пересборка и запуск стека (`up -d --build`) |
|
||||||
| `make docker-down` | Остановка стека |
|
| `make docker-down` | Остановка стека |
|
||||||
| `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) |
|
| `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) |
|
||||||
| `make docker-logs` | Просмотр логов стека |
|
| `make docker-logs` | Просмотр логов стека |
|
||||||
| `make docker-rebuild` | Полная пересборка (down + build + up) |
|
| `make docker-rebuild` | Полная пересборка (down + build + up) |
|
||||||
|
| `make docker-ps` | Статус контейнеров (`docker compose ps`) |
|
||||||
|
| `make docker-shell` | Интерактивная оболочка в контейнере |
|
||||||
|
| `make docker-restart` | Перезапуск контейнера guarddog-nexus |
|
||||||
| `make clean` | Очистка артефактов сборки |
|
| `make clean` | Очистка артефактов сборки |
|
||||||
|
|
||||||
## Безопасность
|
## Безопасность
|
||||||
|
|
||||||
- Вебхуки поддерживают HMAC-SHA256 подпись через `WEBHOOK_SECRET`
|
- Вебхуки поддерживают HMAC-SHA256 подпись через `WEBHOOK_SECRET`
|
||||||
- Nexus-клиент использует BasicAuth для аутентификации
|
- Nexus-клиент использует анонимный доступ (без BasicAuth) — убедитесь, что Nexus разрешает анонимное чтение репозиториев
|
||||||
|
- Защита от SSRF: URL загрузки валидируется через `NEXUS_ALLOWED_HOSTS`
|
||||||
|
- Заголовки безопасности на всех ответах: `X-Content-Type-Options`, `X-Frame-Options`, `X-XSS-Protection`, `Referrer-Policy`, `Permissions-Policy`
|
||||||
- Результаты сканирования хранятся в локальной SQLite-базе
|
- Результаты сканирования хранятся в локальной SQLite-базе
|
||||||
- Временные файлы пакетов удаляются после сканирования
|
- Временные файлы пакетов удаляются после сканирования
|
||||||
|
|
||||||
|
|||||||
@@ -24,6 +24,11 @@ services:
|
|||||||
nexus-setup:
|
nexus-setup:
|
||||||
condition: service_completed_successfully
|
condition: service_completed_successfully
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
logging:
|
||||||
|
driver: "json-file"
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
nexus:
|
nexus:
|
||||||
image: sonatype/nexus3:3.79.0
|
image: sonatype/nexus3:3.79.0
|
||||||
@@ -32,6 +37,17 @@ services:
|
|||||||
volumes:
|
volumes:
|
||||||
- nexus-data:/nexus-data
|
- nexus-data:/nexus-data
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
logging:
|
||||||
|
driver: "json-file"
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "curl", "-sf", "http://localhost:8081/service/rest/v1/status"]
|
||||||
|
interval: 15s
|
||||||
|
timeout: 5s
|
||||||
|
start_period: 60s
|
||||||
|
retries: 10
|
||||||
|
|
||||||
nexus-setup:
|
nexus-setup:
|
||||||
image: alpine:3.21
|
image: alpine:3.21
|
||||||
|
|||||||
@@ -76,8 +76,8 @@ async def harvest(
|
|||||||
_url_locks.pop(download_url, None)
|
_url_locks.pop(download_url, None)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
active_found = False
|
||||||
async with lock:
|
async with lock:
|
||||||
try:
|
|
||||||
# Re-check DB in case another task already created and finished a scan
|
# Re-check DB in case another task already created and finished a scan
|
||||||
active = await session.scalar(
|
active = await session.scalar(
|
||||||
select(Scan.id).where(
|
select(Scan.id).where(
|
||||||
@@ -87,11 +87,14 @@ async def harvest(
|
|||||||
)
|
)
|
||||||
if active:
|
if active:
|
||||||
log.info("Already scanning this URL, skipping")
|
log.info("Already scanning this URL, skipping")
|
||||||
return None
|
active_found = True
|
||||||
finally:
|
|
||||||
async with _url_lock:
|
async with _url_lock:
|
||||||
_url_locks.pop(download_url, None)
|
_url_locks.pop(download_url, None)
|
||||||
|
|
||||||
|
if active_found:
|
||||||
|
return None
|
||||||
|
|
||||||
scan = Scan(
|
scan = Scan(
|
||||||
package_name=package_name,
|
package_name=package_name,
|
||||||
package_version=package_version,
|
package_version=package_version,
|
||||||
|
|||||||
@@ -23,12 +23,13 @@ _REPORT_DEFAULTS = {
|
|||||||
|
|
||||||
|
|
||||||
def _validate_report(report: dict) -> dict:
|
def _validate_report(report: dict) -> dict:
|
||||||
|
result = dict(report)
|
||||||
for field, default in _REPORT_DEFAULTS.items():
|
for field, default in _REPORT_DEFAULTS.items():
|
||||||
if not report.get(field):
|
if not result.get(field):
|
||||||
report[field] = default
|
result[field] = default
|
||||||
if report["verdict"] not in ("safe", "suspicious", "malicious", "unknown"):
|
if result["verdict"] not in ("safe", "suspicious", "malicious", "unknown"):
|
||||||
report["verdict"] = "unknown"
|
result["verdict"] = "unknown"
|
||||||
return report
|
return result
|
||||||
|
|
||||||
|
|
||||||
def _build_user_message(finding: dict) -> str:
|
def _build_user_message(finding: dict) -> str:
|
||||||
|
|||||||
@@ -122,7 +122,8 @@ def parse_package_path(path: str) -> tuple[str, str]:
|
|||||||
async def download_asset(download_url: str, dest_dir: str) -> str | None:
|
async def download_asset(download_url: str, dest_dir: str) -> str | None:
|
||||||
"""Download an asset from Nexus using async httpx."""
|
"""Download an asset from Nexus using async httpx."""
|
||||||
if not _validate_download_url(download_url):
|
if not _validate_download_url(download_url):
|
||||||
log.warning("SSRF prevention: blocked download from %s", download_url)
|
parsed = urlparse(download_url)
|
||||||
|
log.warning("SSRF prevention: blocked download from %s", parsed.hostname or "unknown")
|
||||||
return None
|
return None
|
||||||
|
|
||||||
dest_path = os.path.join(dest_dir, os.path.basename(download_url.split("?")[0]))
|
dest_path = os.path.join(dest_dir, os.path.basename(download_url.split("?")[0]))
|
||||||
|
|||||||
@@ -55,19 +55,26 @@ class LangMiddleware(BaseHTTPMiddleware):
|
|||||||
async def lifespan(app: FastAPI):
|
async def lifespan(app: FastAPI):
|
||||||
await init_db()
|
await init_db()
|
||||||
log.info("%s started on %s:%s", APP_NAME, config.host, config.port)
|
log.info("%s started on %s:%s", APP_NAME, config.host, config.port)
|
||||||
# Start background lock cleanup tasks
|
tasks = [
|
||||||
asyncio.create_task(_start_lock_cleanup())
|
asyncio.create_task(_cleanup_url_locks()),
|
||||||
|
asyncio.create_task(_cleanup_llm_locks()),
|
||||||
|
]
|
||||||
yield
|
yield
|
||||||
|
for t in tasks:
|
||||||
|
t.cancel()
|
||||||
log.info("%s shutting down", APP_NAME)
|
log.info("%s shutting down", APP_NAME)
|
||||||
|
|
||||||
|
|
||||||
async def _start_lock_cleanup():
|
async def _cleanup_url_locks():
|
||||||
"""Start background tasks for cleanup of unused locks."""
|
from guarddog_nexus.core.harvester import _cleanup_url_locks as _fn
|
||||||
from guarddog_nexus.core.harvester import _cleanup_url_locks
|
|
||||||
from guarddog_nexus.routes.web import _cleanup_llm_locks
|
|
||||||
|
|
||||||
asyncio.create_task(_cleanup_url_locks())
|
await _fn()
|
||||||
asyncio.create_task(_cleanup_llm_locks())
|
|
||||||
|
|
||||||
|
async def _cleanup_llm_locks():
|
||||||
|
from guarddog_nexus.routes.web import _cleanup_llm_locks as _fn
|
||||||
|
|
||||||
|
await _fn()
|
||||||
|
|
||||||
|
|
||||||
class RequestLoggingMiddleware(BaseHTTPMiddleware):
|
class RequestLoggingMiddleware(BaseHTTPMiddleware):
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ from ..db.engine import get_session
|
|||||||
from ..db.models import Scan
|
from ..db.models import Scan
|
||||||
from ..db.queries import build_package_list_query
|
from ..db.queries import build_package_list_query
|
||||||
from ..schemas import PackageDetailOut, PackageListResponse, serialize_finding
|
from ..schemas import PackageDetailOut, PackageListResponse, serialize_finding
|
||||||
|
from .api_scans import _csv_safe
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/v1/packages", tags=["packages"])
|
router = APIRouter(prefix="/api/v1/packages", tags=["packages"])
|
||||||
|
|
||||||
@@ -102,8 +103,8 @@ async def export_packages_csv(
|
|||||||
for r in rows:
|
for r in rows:
|
||||||
writer.writerow(
|
writer.writerow(
|
||||||
[
|
[
|
||||||
r.pkg_name,
|
_csv_safe(r.pkg_name),
|
||||||
r.pkg_ver,
|
_csv_safe(r.pkg_ver),
|
||||||
r.ecosystem,
|
r.ecosystem,
|
||||||
r.repository,
|
r.repository,
|
||||||
r.last_scan.isoformat() if r.last_scan else "",
|
r.last_scan.isoformat() if r.last_scan else "",
|
||||||
|
|||||||
@@ -21,6 +21,13 @@ from ..db.models import Scan
|
|||||||
from ..db.queries import build_scan_list_query, get_dashboard_stats
|
from ..db.queries import build_scan_list_query, get_dashboard_stats
|
||||||
from ..schemas import ScanDetailOut, ScanListResponse, StatsResponse, serialize_finding
|
from ..schemas import ScanDetailOut, ScanListResponse, StatsResponse, serialize_finding
|
||||||
|
|
||||||
|
|
||||||
|
def _csv_safe(value: str) -> str:
|
||||||
|
if value and value[0] in "=+-@":
|
||||||
|
return "'" + value
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/v1/scans", tags=["scans"])
|
router = APIRouter(prefix="/api/v1/scans", tags=["scans"])
|
||||||
|
|
||||||
|
|
||||||
@@ -112,8 +119,8 @@ async def export_scans_csv(
|
|||||||
writer.writerow(
|
writer.writerow(
|
||||||
[
|
[
|
||||||
s.id,
|
s.id,
|
||||||
s.package_name,
|
_csv_safe(s.package_name),
|
||||||
s.package_version,
|
_csv_safe(s.package_version),
|
||||||
s.ecosystem,
|
s.ecosystem,
|
||||||
s.repository,
|
s.repository,
|
||||||
s.status,
|
s.status,
|
||||||
@@ -121,7 +128,7 @@ async def export_scans_csv(
|
|||||||
s.flagged,
|
s.flagged,
|
||||||
s.started_at.isoformat() if s.started_at else "",
|
s.started_at.isoformat() if s.started_at else "",
|
||||||
s.finished_at.isoformat() if s.finished_at else "",
|
s.finished_at.isoformat() if s.finished_at else "",
|
||||||
s.error_message or "",
|
_csv_safe(s.error_message or ""),
|
||||||
s.sha256 or "",
|
s.sha256 or "",
|
||||||
]
|
]
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -33,6 +33,19 @@ async def metrics(session: AsyncSession = Depends(get_session)) -> Response:
|
|||||||
# Latest scan timestamp
|
# Latest scan timestamp
|
||||||
latest = await session.scalar(select(func.max(Scan.started_at)))
|
latest = await session.scalar(select(func.max(Scan.started_at)))
|
||||||
|
|
||||||
|
# LLM analysis
|
||||||
|
analyzed = (
|
||||||
|
await session.scalar(
|
||||||
|
select(func.count(Finding.id)).where(
|
||||||
|
func.json_extract(Finding.report, "$.verdict").isnot(None)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
pending = (
|
||||||
|
await session.scalar(select(func.count(Finding.id)).where(Finding.report.is_(None))) or 0
|
||||||
|
)
|
||||||
|
|
||||||
lines = [
|
lines = [
|
||||||
"# HELP guarddog_scans_total Total number of package scans.",
|
"# HELP guarddog_scans_total Total number of package scans.",
|
||||||
"# TYPE guarddog_scans_total counter",
|
"# TYPE guarddog_scans_total counter",
|
||||||
@@ -46,6 +59,14 @@ async def metrics(session: AsyncSession = Depends(get_session)) -> Response:
|
|||||||
"# TYPE guarddog_findings_total counter",
|
"# TYPE guarddog_findings_total counter",
|
||||||
f"guarddog_findings_total {findings_total}",
|
f"guarddog_findings_total {findings_total}",
|
||||||
"",
|
"",
|
||||||
|
"# HELP guarddog_llm_analyzed_total Total findings analyzed by LLM.",
|
||||||
|
"# TYPE guarddog_llm_analyzed_total gauge",
|
||||||
|
f"guarddog_llm_analyzed_total {analyzed}",
|
||||||
|
"",
|
||||||
|
"# HELP guarddog_llm_pending_total Total findings pending LLM analysis.",
|
||||||
|
"# TYPE guarddog_llm_pending_total gauge",
|
||||||
|
f"guarddog_llm_pending_total {pending}",
|
||||||
|
"",
|
||||||
"# HELP guarddog_scans_by_status Scans grouped by status.",
|
"# HELP guarddog_scans_by_status Scans grouped by status.",
|
||||||
"# TYPE guarddog_scans_by_status gauge",
|
"# TYPE guarddog_scans_by_status gauge",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -60,7 +60,11 @@ def _render(name: str, **context) -> HTMLResponse:
|
|||||||
|
|
||||||
|
|
||||||
def _parse_flagged(value: str) -> bool | None:
|
def _parse_flagged(value: str) -> bool | None:
|
||||||
return True if value == "1" else None
|
if value == "1":
|
||||||
|
return True
|
||||||
|
if value == "0":
|
||||||
|
return False
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
@router.get("/", response_class=HTMLResponse)
|
@router.get("/", response_class=HTMLResponse)
|
||||||
@@ -262,11 +266,10 @@ async def analyze_finding_htmx(
|
|||||||
return _render("_llm_spinner.html", request=request, finding_id=finding_id)
|
return _render("_llm_spinner.html", request=request, finding_id=finding_id)
|
||||||
|
|
||||||
async with lock:
|
async with lock:
|
||||||
try:
|
|
||||||
finding.report = {"status": "analyzing"}
|
finding.report = {"status": "analyzing"}
|
||||||
await session.commit()
|
await session.commit()
|
||||||
report = await analyze_finding(finding.data)
|
report = await analyze_finding(finding.data)
|
||||||
finally:
|
|
||||||
async with _llm_lock:
|
async with _llm_lock:
|
||||||
_llm_locks.pop(finding_id, None)
|
_llm_locks.pop(finding_id, None)
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,6 @@
|
|||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
import hmac
|
import hmac
|
||||||
import json
|
|
||||||
import re
|
import re
|
||||||
from urllib.parse import urlencode
|
from urllib.parse import urlencode
|
||||||
|
|
||||||
@@ -22,6 +21,7 @@ from ..constants import (
|
|||||||
from ..core.harvester import harvest
|
from ..core.harvester import harvest
|
||||||
from ..db.engine import get_session
|
from ..db.engine import get_session
|
||||||
from ..logging_setup import log
|
from ..logging_setup import log
|
||||||
|
from ..schemas import WebhookPayload
|
||||||
|
|
||||||
router = APIRouter(prefix="/webhooks", tags=["webhooks"])
|
router = APIRouter(prefix="/webhooks", tags=["webhooks"])
|
||||||
|
|
||||||
@@ -41,17 +41,25 @@ def _build_download_url(repo: str, asset_path: str) -> str:
|
|||||||
return f"{base}/repository/{repo}/{asset_path}"
|
return f"{base}/repository/{repo}/{asset_path}"
|
||||||
|
|
||||||
|
|
||||||
def _extract_asset_path(asset: dict) -> str | None:
|
def _extract_asset_path(asset) -> str | None:
|
||||||
|
if isinstance(asset, dict):
|
||||||
for key in ("path", "name"):
|
for key in ("path", "name"):
|
||||||
val = asset.get(key)
|
val = asset.get(key)
|
||||||
if val:
|
if val:
|
||||||
return val
|
return val
|
||||||
return None
|
return None
|
||||||
|
if asset.path:
|
||||||
|
return asset.path
|
||||||
|
if asset.name:
|
||||||
|
return asset.name
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def _detect_ecosystem(source: dict) -> str | None:
|
def _detect_ecosystem(source) -> str | None:
|
||||||
"""Detect ecosystem from asset or component format field."""
|
if isinstance(source, dict):
|
||||||
fmt = source.get("format", "").lower()
|
fmt = source.get("format", "").lower()
|
||||||
|
else:
|
||||||
|
fmt = (source.format or "").lower()
|
||||||
if fmt in ("pypi", "pip", "python"):
|
if fmt in ("pypi", "pip", "python"):
|
||||||
return "pypi"
|
return "pypi"
|
||||||
if fmt in ("go", "golang"):
|
if fmt in ("go", "golang"):
|
||||||
@@ -81,17 +89,17 @@ async def nexus_webhook(
|
|||||||
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid signature")
|
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Invalid signature")
|
||||||
|
|
||||||
try:
|
try:
|
||||||
data = json.loads(payload.decode("utf-8"))
|
data = WebhookPayload.model_validate_json(payload.decode("utf-8"))
|
||||||
except (json.JSONDecodeError, UnicodeDecodeError):
|
except Exception:
|
||||||
log.warning("Webhook received invalid body")
|
log.warning("Webhook received invalid body")
|
||||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid request body")
|
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Invalid request body")
|
||||||
|
|
||||||
action = data.get("action", "").upper()
|
action = data.action.upper()
|
||||||
if action not in RELEVANT_WEBHOOK_ACTIONS:
|
if action not in RELEVANT_WEBHOOK_ACTIONS:
|
||||||
return {"status": WEBHOOK_STATUS_IGNORED, "action": action}
|
return {"status": WEBHOOK_STATUS_IGNORED, "action": action}
|
||||||
|
|
||||||
# Nexus sends initiator as "username/IP" — parse both fields
|
# Nexus sends initiator as "username/IP" — parse both fields
|
||||||
raw_initiator = data.get("initiator", "")
|
raw_initiator = data.initiator or ""
|
||||||
initiator = None
|
initiator = None
|
||||||
source_ip = None
|
source_ip = None
|
||||||
if raw_initiator and "/" in raw_initiator:
|
if raw_initiator and "/" in raw_initiator:
|
||||||
@@ -104,21 +112,21 @@ async def nexus_webhook(
|
|||||||
|
|
||||||
log.info("Webhook: action=%s initiator=%s source_ip=%s", action, initiator, source_ip)
|
log.info("Webhook: action=%s initiator=%s source_ip=%s", action, initiator, source_ip)
|
||||||
|
|
||||||
repository = data.get("repositoryName", "")
|
repository = data.repositoryName
|
||||||
if not repository:
|
if not repository:
|
||||||
log.warning("Webhook rejected: missing repositoryName")
|
log.warning("Webhook rejected: missing repositoryName")
|
||||||
raise HTTPException(
|
raise HTTPException(
|
||||||
status_code=status.HTTP_400_BAD_REQUEST, detail="Missing repository name"
|
status_code=status.HTTP_400_BAD_REQUEST, detail="Missing repository name"
|
||||||
)
|
)
|
||||||
asset = data.get("asset")
|
asset = data.asset
|
||||||
component = data.get("component")
|
component = data.component
|
||||||
|
|
||||||
if asset:
|
if asset:
|
||||||
asset_path = _extract_asset_path(asset)
|
asset_path = _extract_asset_path(asset)
|
||||||
if not asset_path or not _is_package_asset(asset_path):
|
if not asset_path or not _is_package_asset(asset_path):
|
||||||
return {"status": WEBHOOK_STATUS_IGNORED, "reason": WEBHOOK_IGNORE_NON_PACKAGE}
|
return {"status": WEBHOOK_STATUS_IGNORED, "reason": WEBHOOK_IGNORE_NON_PACKAGE}
|
||||||
|
|
||||||
download_url = asset.get("downloadUrl") or _build_download_url(repository, asset_path)
|
download_url = asset.downloadUrl or _build_download_url(repository, asset_path)
|
||||||
ecosystem = _detect_ecosystem(asset)
|
ecosystem = _detect_ecosystem(asset)
|
||||||
if ecosystem is None:
|
if ecosystem is None:
|
||||||
return {"status": WEBHOOK_STATUS_IGNORED, "reason": "unknown_ecosystem"}
|
return {"status": WEBHOOK_STATUS_IGNORED, "reason": "unknown_ecosystem"}
|
||||||
@@ -137,8 +145,8 @@ async def nexus_webhook(
|
|||||||
return {"status": WEBHOOK_STATUS_ACCEPTED, "asset": asset_path, "action": action}
|
return {"status": WEBHOOK_STATUS_ACCEPTED, "asset": asset_path, "action": action}
|
||||||
|
|
||||||
if component:
|
if component:
|
||||||
name = component.get("name", "")
|
name = component.name
|
||||||
version = component.get("version", "")
|
version = component.version
|
||||||
if not name or not version:
|
if not name or not version:
|
||||||
return {
|
return {
|
||||||
"status": WEBHOOK_STATUS_IGNORED,
|
"status": WEBHOOK_STATUS_IGNORED,
|
||||||
|
|||||||
@@ -102,6 +102,30 @@ class StatsResponse(BaseModel):
|
|||||||
latest_scan_at: datetime | None = None
|
latest_scan_at: datetime | None = None
|
||||||
|
|
||||||
|
|
||||||
|
# Webhook payload models
|
||||||
|
class WebhookAsset(BaseModel):
|
||||||
|
id: str | None = None
|
||||||
|
format: str = ""
|
||||||
|
path: str | None = None
|
||||||
|
name: str | None = None
|
||||||
|
downloadUrl: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class WebhookComponent(BaseModel):
|
||||||
|
id: str | None = None
|
||||||
|
format: str = ""
|
||||||
|
name: str = ""
|
||||||
|
version: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
class WebhookPayload(BaseModel):
|
||||||
|
action: str = ""
|
||||||
|
repositoryName: str = ""
|
||||||
|
initiator: str | None = None
|
||||||
|
asset: WebhookAsset | None = None
|
||||||
|
component: WebhookComponent | None = None
|
||||||
|
|
||||||
|
|
||||||
# Finding data known fields (prevents **f.data from overwriting id/scan_id)
|
# Finding data known fields (prevents **f.data from overwriting id/scan_id)
|
||||||
_FINDING_DATA_FIELDS = ("rule", "severity", "message", "location", "code")
|
_FINDING_DATA_FIELDS = ("rule", "severity", "message", "location", "code")
|
||||||
|
|
||||||
@@ -115,5 +139,5 @@ def serialize_finding(finding) -> dict:
|
|||||||
"created_at": finding.created_at.isoformat() if finding.created_at else None,
|
"created_at": finding.created_at.isoformat() if finding.created_at else None,
|
||||||
}
|
}
|
||||||
for field in _FINDING_DATA_FIELDS:
|
for field in _FINDING_DATA_FIELDS:
|
||||||
result[field] = finding.data.get(field, "")
|
result[field] = finding.data.get(field) or ""
|
||||||
return result
|
return result
|
||||||
|
|||||||
@@ -231,7 +231,9 @@ table td:first-child { font-variant-numeric: tabular-nums; }
|
|||||||
.copy-btn:hover { background: #202632; }
|
.copy-btn:hover { background: #202632; }
|
||||||
.copy-btn.copied { color: #4caf50; border-color: #4caf50; }
|
.copy-btn.copied { color: #4caf50; border-color: #4caf50; }
|
||||||
|
|
||||||
.htmx-indicator { display: inline; }
|
.htmx-indicator { display: none; }
|
||||||
|
.htmx-indicator.htmx-request,
|
||||||
|
.htmx-request .htmx-indicator { display: inline; }
|
||||||
|
|
||||||
/* ------------------------------------------------------------------ */
|
/* ------------------------------------------------------------------ */
|
||||||
/* Nav / breadcrumbs / empty state */
|
/* Nav / breadcrumbs / empty state */
|
||||||
|
|||||||
@@ -10,6 +10,9 @@
|
|||||||
hx-target="closest .llm-report"
|
hx-target="closest .llm-report"
|
||||||
hx-swap="outerHTML"
|
hx-swap="outerHTML"
|
||||||
hx-indicator="closest .llm-report">{{ t('llm_retry', request.state.lang) }}</button>
|
hx-indicator="closest .llm-report">{{ t('llm_retry', request.state.lang) }}</button>
|
||||||
|
<span class="htmx-indicator llm-retry-spinner">
|
||||||
|
<span class="spinner"></span> {{ t('llm_analyzing', request.state.lang) }}
|
||||||
|
</span>
|
||||||
{% endif %}
|
{% endif %}
|
||||||
</div>
|
</div>
|
||||||
<p class="llm-summary">{{ report.summary }}</p>
|
<p class="llm-summary">{{ report.summary }}</p>
|
||||||
|
|||||||
@@ -55,7 +55,9 @@
|
|||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
{% if f.report and f.report.status == "analyzing" %}
|
{% if f.report and f.report.status == "analyzing" %}
|
||||||
|
{% with finding_id=f.id %}
|
||||||
{% include "_llm_spinner.html" %}
|
{% include "_llm_spinner.html" %}
|
||||||
|
{% endwith %}
|
||||||
{% elif f.report and f.report.verdict %}
|
{% elif f.report and f.report.verdict %}
|
||||||
{% with report=f.report, finding_id=f.id %}
|
{% with report=f.report, finding_id=f.id %}
|
||||||
{% include "_llm_report_fragment.html" %}
|
{% include "_llm_report_fragment.html" %}
|
||||||
@@ -67,7 +69,7 @@
|
|||||||
hx-target="#llm-{{ f.id }}"
|
hx-target="#llm-{{ f.id }}"
|
||||||
hx-swap="outerHTML"
|
hx-swap="outerHTML"
|
||||||
hx-indicator="#llm-spinner-{{ f.id }}">
|
hx-indicator="#llm-spinner-{{ f.id }}">
|
||||||
<span id="llm-spinner-{{ f.id }}" class="htmx-indicator" style="display:none;">
|
<span id="llm-spinner-{{ f.id }}" class="htmx-indicator">
|
||||||
<span class="spinner"></span>
|
<span class="spinner"></span>
|
||||||
</span>
|
</span>
|
||||||
{{ t('btn_analyze_llm', request.state.lang) }}
|
{{ t('btn_analyze_llm', request.state.lang) }}
|
||||||
|
|||||||
@@ -51,7 +51,9 @@
|
|||||||
{% endif %}
|
{% endif %}
|
||||||
|
|
||||||
{% if f.report and f.report.status == "analyzing" %}
|
{% if f.report and f.report.status == "analyzing" %}
|
||||||
|
{% with finding_id=f.id %}
|
||||||
{% include "_llm_spinner.html" %}
|
{% include "_llm_spinner.html" %}
|
||||||
|
{% endwith %}
|
||||||
{% elif f.report and f.report.verdict %}
|
{% elif f.report and f.report.verdict %}
|
||||||
{% with report=f.report, finding_id=f.id %}
|
{% with report=f.report, finding_id=f.id %}
|
||||||
{% include "_llm_report_fragment.html" %}
|
{% include "_llm_report_fragment.html" %}
|
||||||
@@ -63,7 +65,7 @@
|
|||||||
hx-target="#llm-{{ f.id }}"
|
hx-target="#llm-{{ f.id }}"
|
||||||
hx-swap="outerHTML"
|
hx-swap="outerHTML"
|
||||||
hx-indicator="#llm-spinner-{{ f.id }}">
|
hx-indicator="#llm-spinner-{{ f.id }}">
|
||||||
<span id="llm-spinner-{{ f.id }}" class="htmx-indicator" style="display:none;">
|
<span id="llm-spinner-{{ f.id }}" class="htmx-indicator">
|
||||||
<span class="spinner"></span>
|
<span class="spinner"></span>
|
||||||
</span>
|
</span>
|
||||||
{{ t('btn_analyze_llm', request.state.lang) }}
|
{{ t('btn_analyze_llm', request.state.lang) }}
|
||||||
|
|||||||
@@ -247,3 +247,52 @@ async def test_health_no_db_leak(client):
|
|||||||
for _ in range(5):
|
for _ in range(5):
|
||||||
resp = await client.get("/health")
|
resp = await client.get("/health")
|
||||||
assert resp.status_code == 200
|
assert resp.status_code == 200
|
||||||
|
|
||||||
|
|
||||||
|
# --- CSV formula injection ---
|
||||||
|
|
||||||
|
|
||||||
|
class TestCsvSafe:
|
||||||
|
def test_formula_prefixes_escaped(self):
|
||||||
|
from guarddog_nexus.routes.api_scans import _csv_safe
|
||||||
|
|
||||||
|
assert _csv_safe("=cmd|'calc'!A0") == "'=cmd|'calc'!A0"
|
||||||
|
assert _csv_safe("+SUM(1,2)") == "'+SUM(1,2)"
|
||||||
|
assert _csv_safe("-3+4") == "'-3+4"
|
||||||
|
assert _csv_safe("@REF(A1)") == "'@REF(A1)"
|
||||||
|
|
||||||
|
def test_normal_values_unchanged(self):
|
||||||
|
from guarddog_nexus.routes.api_scans import _csv_safe
|
||||||
|
|
||||||
|
assert _csv_safe("requests") == "requests"
|
||||||
|
assert _csv_safe("2.0.0") == "2.0.0"
|
||||||
|
|
||||||
|
def test_empty_string(self):
|
||||||
|
from guarddog_nexus.routes.api_scans import _csv_safe
|
||||||
|
|
||||||
|
assert _csv_safe("") == ""
|
||||||
|
|
||||||
|
def test_none_passes_through(self):
|
||||||
|
from guarddog_nexus.routes.api_scans import _csv_safe
|
||||||
|
|
||||||
|
assert _csv_safe(None) is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_csv_export_escapes_formula_injection(client, db_session):
|
||||||
|
from guarddog_nexus.db.models import Scan, ScanStatus
|
||||||
|
|
||||||
|
scan = Scan(
|
||||||
|
package_name="=cmd|'calc'!A0",
|
||||||
|
package_version="1.0",
|
||||||
|
ecosystem="pypi",
|
||||||
|
repository="pypi-proxy",
|
||||||
|
nexus_asset_url="http://nexus:8081/repo/evil-1.0.tar.gz",
|
||||||
|
status=ScanStatus.COMPLETED.value,
|
||||||
|
)
|
||||||
|
db_session.add(scan)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
resp = await client.get("/api/v1/scans/export")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert "'=cmd" in resp.text
|
||||||
|
|||||||
34
tests/test_config.py
Normal file
34
tests/test_config.py
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
"""Tests for config module — _env_int error path."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
|
||||||
|
def test_env_int_invalid_value_warns_and_returns_default():
|
||||||
|
from guarddog_nexus.config import _env_int
|
||||||
|
|
||||||
|
os.environ["TEST_PORT"] = "notanumber"
|
||||||
|
|
||||||
|
with patch("logging.getLogger") as mock_logger:
|
||||||
|
result = _env_int("TEST_PORT", 42)
|
||||||
|
|
||||||
|
assert result == 42
|
||||||
|
mock_logger.return_value.warning.assert_called_once()
|
||||||
|
|
||||||
|
del os.environ["TEST_PORT"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_env_int_missing_returns_default():
|
||||||
|
from guarddog_nexus.config import _env_int
|
||||||
|
|
||||||
|
os.environ.pop("TEST_MISSING", None)
|
||||||
|
|
||||||
|
assert _env_int("TEST_MISSING", 99) == 99
|
||||||
|
|
||||||
|
|
||||||
|
def test_env_int_valid_returns_parsed():
|
||||||
|
from guarddog_nexus.config import _env_int
|
||||||
|
|
||||||
|
os.environ["TEST_VALID"] = "8080"
|
||||||
|
assert _env_int("TEST_VALID", 42) == 8080
|
||||||
|
del os.environ["TEST_VALID"]
|
||||||
59
tests/test_engine.py
Normal file
59
tests/test_engine.py
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
"""Tests for database engine — reaping and migrations."""
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
from sqlalchemy import text
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_reap_stale_analysis_resets_stuck_findings(db_session):
|
||||||
|
from guarddog_nexus.db.models import Finding
|
||||||
|
|
||||||
|
stuck = Finding(
|
||||||
|
scan_id=1,
|
||||||
|
data={"rule": "test", "severity": "WARNING", "message": "test"},
|
||||||
|
report={"status": "analyzing"},
|
||||||
|
)
|
||||||
|
db_session.add(stuck)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
from guarddog_nexus.db.engine import _engine
|
||||||
|
|
||||||
|
async with _engine.begin() as conn:
|
||||||
|
pass # ensure tables exist in _engine too
|
||||||
|
|
||||||
|
await db_session.execute(
|
||||||
|
text(
|
||||||
|
"UPDATE findings SET report = NULL "
|
||||||
|
"WHERE report IS NOT NULL "
|
||||||
|
"AND json_extract(report, '$.status') = 'analyzing'"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
await db_session.refresh(stuck)
|
||||||
|
assert stuck.report is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_reap_stale_analysis_spares_completed_reports(db_session):
|
||||||
|
from guarddog_nexus.db.models import Finding
|
||||||
|
|
||||||
|
valid = Finding(
|
||||||
|
scan_id=1,
|
||||||
|
data={"rule": "test", "severity": "WARNING", "message": "test"},
|
||||||
|
report={"verdict": "safe", "summary": "ok"},
|
||||||
|
)
|
||||||
|
db_session.add(valid)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
await db_session.execute(
|
||||||
|
text(
|
||||||
|
"UPDATE findings SET report = NULL "
|
||||||
|
"WHERE report IS NOT NULL "
|
||||||
|
"AND json_extract(report, '$.status') = 'analyzing'"
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
await db_session.refresh(valid)
|
||||||
|
assert valid.report == {"verdict": "safe", "summary": "ok"}
|
||||||
@@ -231,3 +231,29 @@ async def test_harvest_skips_non_package_asset(db_session):
|
|||||||
db_session,
|
db_session,
|
||||||
)
|
)
|
||||||
assert scan is None
|
assert scan is None
|
||||||
|
|
||||||
|
|
||||||
|
# --- Lock cleanup ---
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_cleanup_url_locks_removes_unlocked():
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from guarddog_nexus.core.harvester import _url_lock, _url_locks
|
||||||
|
|
||||||
|
async with _url_lock:
|
||||||
|
_url_locks["locked"] = asyncio.Lock()
|
||||||
|
_url_locks["unlocked"] = asyncio.Lock()
|
||||||
|
|
||||||
|
await _url_locks["locked"].acquire()
|
||||||
|
|
||||||
|
for key in list(_url_locks.keys()):
|
||||||
|
if not _url_locks[key].locked():
|
||||||
|
_url_locks.pop(key, None)
|
||||||
|
|
||||||
|
assert "locked" in _url_locks
|
||||||
|
assert "unlocked" not in _url_locks
|
||||||
|
|
||||||
|
_url_locks["locked"].release()
|
||||||
|
_url_locks.clear()
|
||||||
|
|||||||
@@ -230,3 +230,97 @@ async def test_analyze_endpoint_failure(client, sample_finding):
|
|||||||
assert "failed" in resp.text.lower()
|
assert "failed" in resp.text.lower()
|
||||||
|
|
||||||
guarddog_nexus.config.config.llm_enabled = False
|
guarddog_nexus.config.config.llm_enabled = False
|
||||||
|
|
||||||
|
|
||||||
|
# --- GET /analyze polling endpoint ---
|
||||||
|
|
||||||
|
|
||||||
|
class TestAnalyzeStatusEndpoint:
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_status_finding_not_found(self, client):
|
||||||
|
resp = await client.get("/api/v1/findings/99999/analyze")
|
||||||
|
assert resp.status_code == 404
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_status_returns_report_when_complete(self, client, sample_finding_with_report):
|
||||||
|
import guarddog_nexus.config
|
||||||
|
|
||||||
|
guarddog_nexus.config.config.llm_enabled = True
|
||||||
|
|
||||||
|
resp = await client.get(f"/api/v1/findings/{sample_finding_with_report.id}/analyze")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert "safe" in resp.text
|
||||||
|
|
||||||
|
guarddog_nexus.config.config.llm_enabled = False
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_status_returns_spinner_when_no_report(self, client, sample_finding):
|
||||||
|
import guarddog_nexus.config
|
||||||
|
|
||||||
|
guarddog_nexus.config.config.llm_enabled = True
|
||||||
|
|
||||||
|
resp = await client.get(f"/api/v1/findings/{sample_finding.id}/analyze")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert "hx-get" in resp.text.lower()
|
||||||
|
|
||||||
|
guarddog_nexus.config.config.llm_enabled = False
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_status_returns_spinner_when_analyzing(self, client, db_session, sample_finding):
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from guarddog_nexus.db.models import Finding
|
||||||
|
|
||||||
|
finding = await db_session.scalar(select(Finding).where(Finding.id == sample_finding.id))
|
||||||
|
finding.report = {"status": "analyzing"}
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
resp = await client.get(f"/api/v1/findings/{sample_finding.id}/analyze")
|
||||||
|
assert resp.status_code == 200
|
||||||
|
assert "hx-get" in resp.text.lower()
|
||||||
|
|
||||||
|
|
||||||
|
# --- LLM retry exhaustion ---
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_analyze_finding_exhausts_all_retries():
|
||||||
|
import guarddog_nexus.config
|
||||||
|
from guarddog_nexus.core.llm import analyze_finding
|
||||||
|
|
||||||
|
guarddog_nexus.config.config.llm_api_key = "sk-test"
|
||||||
|
|
||||||
|
with patch("guarddog_nexus.core.llm._attempt_llm_call", return_value=None):
|
||||||
|
with patch("guarddog_nexus.core.llm.asyncio.sleep") as mock_sleep:
|
||||||
|
result = await analyze_finding({"rule": "test-rule"}, max_retries=2)
|
||||||
|
|
||||||
|
assert result is None
|
||||||
|
assert mock_sleep.call_count == 1
|
||||||
|
|
||||||
|
guarddog_nexus.config.config.llm_api_key = ""
|
||||||
|
|
||||||
|
|
||||||
|
# --- LLM lock cleanup ---
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_cleanup_llm_locks_removes_unlocked():
|
||||||
|
import asyncio
|
||||||
|
|
||||||
|
from guarddog_nexus.routes.web import _llm_lock, _llm_locks
|
||||||
|
|
||||||
|
async with _llm_lock:
|
||||||
|
_llm_locks[100] = asyncio.Lock()
|
||||||
|
_llm_locks[200] = asyncio.Lock()
|
||||||
|
|
||||||
|
await _llm_locks[100].acquire()
|
||||||
|
|
||||||
|
for key in list(_llm_locks.keys()):
|
||||||
|
if not _llm_locks[key].locked():
|
||||||
|
_llm_locks.pop(key, None)
|
||||||
|
|
||||||
|
assert 100 in _llm_locks
|
||||||
|
assert 200 not in _llm_locks
|
||||||
|
|
||||||
|
_llm_locks[100].release()
|
||||||
|
_llm_locks.clear()
|
||||||
|
|||||||
@@ -95,3 +95,57 @@ class TestDispatchExtractor:
|
|||||||
|
|
||||||
def test_unknown_ecosystem(self):
|
def test_unknown_ecosystem(self):
|
||||||
assert extract_package_info("/packages/pkg/1.0/file.tar.gz", "unknown") == ("pkg", "1.0")
|
assert extract_package_info("/packages/pkg/1.0/file.tar.gz", "unknown") == ("pkg", "1.0")
|
||||||
|
|
||||||
|
|
||||||
|
class TestValidateDownloadUrl:
|
||||||
|
def test_allowed_hostname_passes(self):
|
||||||
|
from guarddog_nexus.core.nexus import _validate_download_url
|
||||||
|
|
||||||
|
assert _validate_download_url("http://nexus:8081/repository/pkg/foo.tar.gz") is True
|
||||||
|
assert _validate_download_url("https://nexus:8081/repository/bar") is True
|
||||||
|
|
||||||
|
def test_blocked_hostname(self):
|
||||||
|
from guarddog_nexus.core.nexus import _validate_download_url
|
||||||
|
|
||||||
|
assert _validate_download_url("http://evil.com/malware.tar.gz") is False
|
||||||
|
assert _validate_download_url("https://169.254.169.254/latest/meta-data") is False
|
||||||
|
|
||||||
|
def test_non_http_scheme_blocked(self):
|
||||||
|
from guarddog_nexus.core.nexus import _validate_download_url
|
||||||
|
|
||||||
|
assert _validate_download_url("file:///etc/passwd") is False
|
||||||
|
assert _validate_download_url("ftp://nexus:8081/foo") is False
|
||||||
|
|
||||||
|
def test_empty_or_invalid_url_blocked(self):
|
||||||
|
from guarddog_nexus.core.nexus import _validate_download_url
|
||||||
|
|
||||||
|
assert _validate_download_url("") is False
|
||||||
|
assert _validate_download_url("not-a-valid-url") is False
|
||||||
|
|
||||||
|
|
||||||
|
class TestNpmScopedEdgeCases:
|
||||||
|
def test_scoped_too_short(self):
|
||||||
|
from guarddog_nexus.core.nexus import extract_npm_info
|
||||||
|
|
||||||
|
assert extract_npm_info("packages/@scope") is None
|
||||||
|
|
||||||
|
def test_scoped_no_filename_match(self):
|
||||||
|
from guarddog_nexus.core.nexus import extract_npm_info
|
||||||
|
|
||||||
|
assert extract_npm_info("packages/@scope/name/-/otherfile.tgz") is None
|
||||||
|
|
||||||
|
def test_scoped_version_with_hyphens(self):
|
||||||
|
from guarddog_nexus.core.nexus import extract_npm_info
|
||||||
|
|
||||||
|
assert extract_npm_info("packages/@scope/name/-/name-1.0.0-beta.1.tgz") == (
|
||||||
|
"@scope/name",
|
||||||
|
"1.0.0-beta.1",
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_scoped_tar_gz_extension(self):
|
||||||
|
from guarddog_nexus.core.nexus import extract_npm_info
|
||||||
|
|
||||||
|
assert extract_npm_info("packages/@scope/name/-/name-1.0.0.tar.gz") == (
|
||||||
|
"@scope/name",
|
||||||
|
"1.0.0",
|
||||||
|
)
|
||||||
|
|||||||
78
tests/test_schemas.py
Normal file
78
tests/test_schemas.py
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
"""Tests for schemas and serialize_finding."""
|
||||||
|
|
||||||
|
import datetime
|
||||||
|
from unittest.mock import MagicMock
|
||||||
|
|
||||||
|
|
||||||
|
class TestSerializeFinding:
|
||||||
|
def test_normal_finding(self):
|
||||||
|
from guarddog_nexus.schemas import serialize_finding
|
||||||
|
|
||||||
|
finding = MagicMock()
|
||||||
|
finding.id = 42
|
||||||
|
finding.scan_id = 7
|
||||||
|
finding.report = {"verdict": "safe"}
|
||||||
|
finding.created_at = datetime.datetime(2026, 1, 1, tzinfo=datetime.timezone.utc)
|
||||||
|
finding.data = {
|
||||||
|
"rule": "shady-links",
|
||||||
|
"severity": "WARNING",
|
||||||
|
"message": "Suspicious URL",
|
||||||
|
"location": "setup.py:15",
|
||||||
|
"code": "url = 'http://evil.com'",
|
||||||
|
}
|
||||||
|
|
||||||
|
result = serialize_finding(finding)
|
||||||
|
|
||||||
|
assert result["id"] == 42
|
||||||
|
assert result["scan_id"] == 7
|
||||||
|
assert result["rule"] == "shady-links"
|
||||||
|
assert result["severity"] == "WARNING"
|
||||||
|
assert result["report"] == {"verdict": "safe"}
|
||||||
|
assert result["created_at"] == "2026-01-01T00:00:00+00:00"
|
||||||
|
|
||||||
|
def test_created_at_none(self):
|
||||||
|
from guarddog_nexus.schemas import serialize_finding
|
||||||
|
|
||||||
|
finding = MagicMock()
|
||||||
|
finding.id = 1
|
||||||
|
finding.scan_id = 1
|
||||||
|
finding.report = None
|
||||||
|
finding.created_at = None
|
||||||
|
finding.data = {"rule": "test", "message": "msg"}
|
||||||
|
|
||||||
|
result = serialize_finding(finding)
|
||||||
|
|
||||||
|
assert result["created_at"] is None
|
||||||
|
assert result["report"] is None
|
||||||
|
|
||||||
|
def test_missing_data_fields_default_to_empty_string(self):
|
||||||
|
from guarddog_nexus.schemas import serialize_finding
|
||||||
|
|
||||||
|
finding = MagicMock()
|
||||||
|
finding.id = 1
|
||||||
|
finding.scan_id = 1
|
||||||
|
finding.report = None
|
||||||
|
finding.created_at = None
|
||||||
|
finding.data = {"rule": "only-rule"}
|
||||||
|
|
||||||
|
result = serialize_finding(finding)
|
||||||
|
|
||||||
|
assert result["rule"] == "only-rule"
|
||||||
|
assert result["severity"] == ""
|
||||||
|
assert result["message"] == ""
|
||||||
|
|
||||||
|
def test_data_values_none_become_empty_strings(self):
|
||||||
|
from guarddog_nexus.schemas import serialize_finding
|
||||||
|
|
||||||
|
finding = MagicMock()
|
||||||
|
finding.id = 1
|
||||||
|
finding.scan_id = 1
|
||||||
|
finding.report = None
|
||||||
|
finding.created_at = None
|
||||||
|
finding.data = {"rule": None, "severity": None, "message": None}
|
||||||
|
|
||||||
|
result = serialize_finding(finding)
|
||||||
|
|
||||||
|
assert result["rule"] == ""
|
||||||
|
assert result["severity"] == ""
|
||||||
|
assert result["message"] == ""
|
||||||
@@ -164,3 +164,42 @@ async def test_webhook_invalid_signature(client, sample_nexus_webhook):
|
|||||||
assert resp.status_code == 403
|
assert resp.status_code == 403
|
||||||
|
|
||||||
guarddog_nexus.config.config.webhook_secret = ""
|
guarddog_nexus.config.config.webhook_secret = ""
|
||||||
|
|
||||||
|
|
||||||
|
# --- Unknown ecosystem rejection ---
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_webhook_rejects_unknown_ecosystem_asset(client):
|
||||||
|
resp = await client.post(
|
||||||
|
"/webhooks/nexus",
|
||||||
|
json={
|
||||||
|
"action": "UPDATED",
|
||||||
|
"repositoryName": "test-repo",
|
||||||
|
"asset": {
|
||||||
|
"format": "maven",
|
||||||
|
"name": "/packages/test/1.0/test-1.0.tar.gz",
|
||||||
|
"downloadUrl": "http://nexus:8081/repo/test/1.0/test-1.0.tar.gz",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["status"] == "ignored"
|
||||||
|
assert data["reason"] == "unknown_ecosystem"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_webhook_rejects_unknown_ecosystem_component(client):
|
||||||
|
resp = await client.post(
|
||||||
|
"/webhooks/nexus",
|
||||||
|
json={
|
||||||
|
"action": "UPDATED",
|
||||||
|
"repositoryName": "test-repo",
|
||||||
|
"component": {"format": "maven", "name": "test", "version": "1.0"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert resp.status_code == 200
|
||||||
|
data = resp.json()
|
||||||
|
assert data["status"] == "ignored"
|
||||||
|
assert data["reason"] == "unknown_ecosystem"
|
||||||
|
|||||||
Reference in New Issue
Block a user