242 lines
9.2 KiB
Markdown
242 lines
9.2 KiB
Markdown
# GuardDog Nexus
|
||
|
||
Integration of [GuardDog](https://github.com/DataDog/guarddog) (package vulnerability scanner) with [Sonatype Nexus Repository Manager](https://www.sonatype.com/products/nexus-repository). Automatically scans packages stored in Nexus for vulnerabilities, malware, and suspicious patterns via webhooks.
|
||
|
||
## Features
|
||
|
||
- **Automatic scanning** via Nexus webhooks on package cache updates
|
||
- **Multi-ecosystem support** — PyPI, Go, npm (any format via proxy repositories)
|
||
- **REST API** for scan results, findings, statistics, and CSV export
|
||
- **Web dashboard** with scan tables, filtering, and LLM-powered analysis
|
||
- **LLM analysis** — automated security analysis of each finding via OpenAI-compatible APIs (optional, configurable)
|
||
- **Deduplication** by URL and SHA256 — identical content scanned only once
|
||
- **Structured JSON logging** with optional syslog output
|
||
- **Docker Compose** — full stack deployment with Nexus in one command
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Nexus ──(webhook)──> GuardDog Nexus ──(REST API)──> Web UI
|
||
│
|
||
├──> GuardDog CLI (scanning)
|
||
├──> LLM API (finding analysis)
|
||
├──> SQLite (result storage)
|
||
└──> REST API (data for UI + CSV export)
|
||
```
|
||
|
||
## Quick Start
|
||
|
||
### Prerequisites
|
||
|
||
- Docker and Docker Compose
|
||
- Python 3.10+ (for local development)
|
||
|
||
### Docker Deployment
|
||
|
||
```bash
|
||
cp .env.example .env
|
||
# edit .env: optionally LLM_* vars
|
||
|
||
make docker-up
|
||
```
|
||
|
||
After startup:
|
||
|
||
| Service | URL | Port |
|
||
|---------|-----|------|
|
||
| GuardDog Nexus | http://localhost:8080 | 8080 |
|
||
| Sonatype Nexus | http://localhost:8081 | 8081 |
|
||
|
||
## Environment Variables
|
||
|
||
| Variable | Default | Description |
|
||
|----------|---------|-------------|
|
||
| `NEXUS_URL` | `http://localhost:8081` | Sonatype Nexus URL |
|
||
| `DATABASE_PATH` | `data/guarddog.db` | SQLite database path |
|
||
| `HOST` | `0.0.0.0` | Listen host |
|
||
| `PORT` | `8080` | Listen port |
|
||
| `LOG_LEVEL` | `INFO` | Logging level |
|
||
| `WEBHOOK_SECRET` | _(empty)_ | HMAC-SHA256 webhook validation secret |
|
||
| `SCAN_TIMEOUT_SECONDS` | `300` | Scan timeout per package |
|
||
| `TEMP_DIR` | `/tmp/guarddog-nexus` | Temporary download directory |
|
||
| `MAX_CONCURRENT_SCANS` | `4` | Maximum simultaneous GuardDog processes |
|
||
| `LLM_ENABLED` | `0` | Set to `1` to enable LLM analysis |
|
||
| `LLM_AUTO_ANALYZE` | `0` | Set to `1` to auto-analyze after scan; `0` = manual mode via UI button |
|
||
| `LLM_API_KEY` | _(empty)_ | API key (OpenAI / Groq / Ollama / etc.) |
|
||
| `LLM_API_BASE` | `https://api.openai.com/v1` | OpenAI-compatible base URL |
|
||
| `LLM_MODEL` | `gpt-4o-mini` | Model name |
|
||
| `LLM_TIMEOUT_SECONDS` | `30` | LLM request timeout |
|
||
| `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Max parallel LLM analysis calls |
|
||
|
||
## Nexus Configuration
|
||
|
||
### Proxy Repositories
|
||
|
||
The setup script (`scripts/setup-nexus.sh`) creates three proxy repositories:
|
||
|
||
- `pypi-proxy` → `https://pypi.org`
|
||
- `go-proxy` → `https://proxy.golang.org`
|
||
- `npm-proxy` → `https://registry.npmjs.org`
|
||
|
||
### Webhooks
|
||
|
||
GuardDog Nexus accepts `UPDATED` webhook events from Nexus.
|
||
|
||
1. In Nexus admin panel: **System → Capabilities**
|
||
2. Create **Webhook: Repository** capability
|
||
3. URL: `http://guarddog-nexus:8080/webhooks/nexus`
|
||
4. Event type: **Repository → Asset → Updated**
|
||
5. Repository filter: `pypi-proxy`, `go-proxy`, `npm-proxy`
|
||
6. Set `WEBHOOK_SECRET` in `.env` for HMAC-SHA256 signature validation
|
||
|
||
> For local testing without real webhooks, POST manually:
|
||
> ```bash
|
||
> curl -X POST http://localhost:8080/webhooks/nexus \
|
||
> -H "Content-Type: application/json" \
|
||
> -d '{"action":"UPDATED","repositoryName":"pypi-proxy","asset":{"format":"pypi","name":"/packages/pkg/1.0/pkg-1.0.tar.gz","downloadUrl":"http://nexus:8081/repository/pypi-proxy/packages/pkg/1.0/pkg-1.0.tar.gz"}}'
|
||
> ```
|
||
|
||
### E2E Testing
|
||
|
||
The `examples/` directory contains sample malicious packages for PyPI, npm, and Go with patterns that GuardDog reliably detects (`exec-base64`, `shady-links`, `code-execution`, `npm-api-obfuscation`, `go-exec-base64`).
|
||
|
||
Run the E2E test (requires running Docker stack):
|
||
|
||
```bash
|
||
./examples/trigger-scans.sh
|
||
```
|
||
|
||
The script builds `.tar.gz` / `.tgz` / `.zip` archives, copies them into the container, and sends webhooks. After ~15 seconds it checks results: all three packages should be flagged with findings.
|
||
|
||
For manual testing of individual ecosystems:
|
||
|
||
```bash
|
||
# PyPI: exec-base64 + shady-links + code-execution
|
||
curl -X POST http://localhost:8080/webhooks/nexus \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"action":"UPDATED","repositoryName":"pypi-proxy","asset":{"format":"pypi","name":"/packages/evil-pypi/0.1.0/evil-pypi-0.1.0.tar.gz","downloadUrl":"http://nexus:8081/repository/pypi-proxy/packages/evil-pypi/0.1.0/evil-pypi-0.1.0.tar.gz"}}'
|
||
|
||
# npm: eval + Buffer(base64) + shady-links
|
||
curl -X POST http://localhost:8080/webhooks/nexus \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"action":"UPDATED","repositoryName":"npm-proxy","asset":{"format":"npm","name":"/packages/evil-npm/-/evil-npm-1.0.0.tgz","downloadUrl":"http://nexus:8081/repository/npm-proxy/evil-npm/-/evil-npm-1.0.0.tgz"}}'
|
||
|
||
# Go: exec + base64 + suspicious HTTP call
|
||
curl -X POST http://localhost:8080/webhooks/nexus \
|
||
-H "Content-Type: application/json" \
|
||
-d '{"action":"UPDATED","repositoryName":"go-proxy","asset":{"format":"go","name":"/packages/github.com/evil/evil-go/@v/v0.1.0.zip","downloadUrl":"http://nexus:8081/repository/go-proxy/github.com/evil/evil-go/@v/v0.1.0.zip"}}'
|
||
```
|
||
|
||
## REST API
|
||
|
||
### Scans
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| GET | `/api/v1/scans` | List scans (pagination, filters) |
|
||
| GET | `/api/v1/scans/stats` | Statistics |
|
||
| GET | `/api/v1/scans/{id}` | Scan detail with findings |
|
||
| GET | `/api/v1/scans/export` | Export scans to CSV |
|
||
|
||
### Packages
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| GET | `/api/v1/packages` | List aggregated packages |
|
||
| GET | `/api/v1/packages/{name}/{version}` | Package detail with scans and findings |
|
||
| GET | `/api/v1/packages/export` | Export packages to CSV |
|
||
|
||
### Findings
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| GET | `/api/v1/findings` | List findings (filter by rule, severity, scan_id) |
|
||
| POST | `/api/v1/findings/{id}/analyze` | Trigger LLM analysis for a finding |
|
||
|
||
### Other
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| GET | `/health` | Health check |
|
||
| GET | `/metrics` | Prometheus-compatible metrics |
|
||
|
||
## LLM Analysis
|
||
|
||
GuardDog Nexus can analyze findings through an LLM. When enabled (`LLM_ENABLED=1`), flagged findings receive an AI breakdown: threat assessment, code analysis, and recommendations.
|
||
|
||
### Operating Modes
|
||
|
||
The `LLM_AUTO_ANALYZE` variable controls the analysis mode:
|
||
|
||
- **`LLM_AUTO_ANALYZE=1` (automatic):** each finding is automatically sent to the LLM after a scan completes. Reports are saved to the database and included in JSON log output. No "Analyze" button is shown in the UI.
|
||
- **`LLM_AUTO_ANALYZE=0` (manual, default):** an "Analyze with LLM" button is shown next to each finding in the web UI. The user clicks to trigger analysis and see the inline verdict.
|
||
|
||
### finding.report State Machine
|
||
|
||
The `finding.report` field transitions through these states:
|
||
|
||
| Value | UI |
|
||
|-------|----|
|
||
| `None` | "Analyze with LLM" button (manual mode only) |
|
||
| `{"status": "analyzing"}` | Spinner |
|
||
| `{verdict:, summary:, ...}` | Report + "Retry" link |
|
||
|
||
### Supported Providers
|
||
|
||
Any OpenAI-compatible API. Configuration examples:
|
||
|
||
```bash
|
||
# OpenAI (manual mode)
|
||
LLM_ENABLED=1
|
||
LLM_AUTO_ANALYZE=0
|
||
LLM_API_KEY=sk-...
|
||
LLM_API_BASE=https://api.openai.com/v1
|
||
LLM_MODEL=gpt-4o-mini
|
||
|
||
# Groq with auto-analysis (faster, free tier)
|
||
LLM_ENABLED=1
|
||
LLM_AUTO_ANALYZE=1
|
||
LLM_API_KEY=gsk_...
|
||
LLM_API_BASE=https://api.groq.com/openai/v1
|
||
LLM_MODEL=llama-3.3-70b-versatile
|
||
|
||
# Local Ollama
|
||
LLM_ENABLED=1
|
||
LLM_API_KEY=ollama
|
||
LLM_API_BASE=http://host.docker.internal:11434/v1
|
||
LLM_MODEL=llama3.2
|
||
```
|
||
|
||
### Response Format
|
||
|
||
LLM returns JSON with fields:
|
||
- `verdict` — `safe` / `suspicious` / `malicious`
|
||
- `summary` — one-line verdict
|
||
- `analysis` — detailed breakdown (2–3 paragraphs)
|
||
- `severity_rating` — `low` / `medium` / `high` / `critical`
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
guarddog-nexus/
|
||
├── guarddog_nexus/ # Main package
|
||
│ ├── core/ # Business logic (scanner, harvester, nexus, llm)
|
||
│ ├── db/ # Database (engine, models, queries)
|
||
│ ├── routes/ # HTTP layer (webhooks, api, web, metrics)
|
||
│ ├── web/ # Static assets (templates, CSS, JS)
|
||
│ ├── config.py # Environment configuration
|
||
│ ├── constants.py # Centralized constants
|
||
│ ├── i18n.py # RU/EN translations
|
||
│ ├── logging_setup.py # JSON structured logging
|
||
│ └── main.py # FastAPI app entry point
|
||
├── tests/ # pytest tests (85 tests)
|
||
├── scripts/ # Setup scripts
|
||
├── docker-compose.yml
|
||
├── Dockerfile
|
||
└── pyproject.toml
|
||
```
|
||
|
||
## License
|
||
|
||
MIT
|
||
|
||
---
|
||
|
||
> **⚠ All code in this repository was generated by an AI assistant (Claude).** Review carefully before using in production.
|