# AGENTS.md — GuardDog Nexus

## Project overview

GuardDog Nexus integrates [GuardDog](https://github.com/DataDog/guarddog) with [Sonatype Nexus Repository Manager](https://www.sonatype.com/products/nexus-repository). It receives webhooks from Nexus, downloads packages from proxied repositories, scans them with GuardDog CLI, and stores results in SQLite. A web dashboard built with FastAPI + Jinja2 + htmx displays findings with optional LLM-based analysis.

**Stack:** Python 3.12, FastAPI, SQLAlchemy (async), aiosqlite, Jinja2, htmx, Docker Compose.

**Package ecosystems:** PyPI, Go (proxy.golang.org), npm (registry.npmjs.org).

**GuardDog binary:** installed inside the Docker image via `uv pip install --system guarddog`. Supports `pypi`, `go`, `npm` subcommands.

---

## Quick start

```bash
cp .env.example .env
# edit .env to set NEXUS_PASSWORD, optionally LLM vars
make docker-up
# → guarddog-nexus :8080, Nexus :8081
```

For local development without Docker:
```bash
make install dev
export $(cat .env | xargs)
python -m guarddog_nexus.main
make test     # 85 tests
make lint     # ruff
make format   # ruff format + fix
```

---

## Architecture

```
guarddog_nexus/
├── core/              # Business logic
│   ├── scanner.py     # subprocess guarddog CLI
│   ├── harvester.py   # download → sha256 → scan → store
│   ├── nexus.py       # httpx client + pypi/go/npm path extractors
│   └── llm.py         # OpenAI-compatible analysis client
├── db/                # Persistence
│   ├── engine.py      # async SQLAlchemy + auto-migration
│   ├── models.py      # Scan, Finding ORM
│   └── queries.py     # shared query builders
├── routes/            # HTTP layer
│   ├── webhooks.py    # POST /webhooks/nexus
│   ├── api_scans.py   # REST /api/v1/scans
│   ├── api_packages.py
│   ├── api_findings.py
│   ├── metrics.py     # GET /metrics (Prometheus)
│   └── web.py         # HTML UI (Jinja2 + htmx)
├── web/               # Static assets
│   ├── templates/     # Jinja2 templates
│   └── static/        # CSS, JS
├── config.py          # env-var configuration dataclass
├── constants.py       # all magic strings/limits
├── i18n.py            # RU/EN translation dictionaries
├── logging_setup.py   # JSON logging + syslog
└── main.py            # FastAPI app, middleware, lifepan
```

**Data flow:**
1. Nexus sends `UPDATED` webhook → `POST /webhooks/nexus`
2. `webhooks.py` validates signature, extracts asset info, spawns background task
3. `harvester.py` downloads file (async via `asyncio.to_thread`), computes SHA256, deduplicates
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
5. Findings stored in SQLite (`scans` + `findings` tables)
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends each finding to the configured model. `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure.

---

## Key conventions

- **Python ≥ 3.10** — type hints with `| None` syntax, no `Optional`
- **Imports:** absolute from `guarddog_nexus.<module>`; relative (`..`) within subpackages
- **Line length:** 100 (ruff)
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
- **Format:** `ruff format guarddog_nexus tests`
- **Tests:** `pytest -v` (85 tests, pytest-asyncio auto mode)
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
- **No comments** in code unless explicitly requested
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context

---

## Configuration

All via environment variables, defined in `config.py`. Key ones:

| Variable | Default | Notes |
|----------|---------|-------|
| `NEXUS_URL` | `http://localhost:8081` | |
| `NEXUS_PASSWORD` | — | Required |
| `WEBHOOK_SECRET` | `""` | HMAC-SHA256 validation |
| `MAX_CONCURRENT_SCANS` | `4` | asyncio.Semaphore for guarddog processes |
| `LLM_ENABLED` | `0` | `1` to enable analysis |
| `LLM_AUTO_ANALYZE` | `0` | `1` to auto-trigger after scan; `0` = manual via UI button |
| `LLM_API_KEY` | `""` | OpenAI-compatible key |
| `LLM_MODEL` | `gpt-4o-mini` | |
| `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Semaphore for LLM calls |
| `DATABASE_PATH` | `data/guarddog.db` | |

Full list in `config.py`.

---

## Database

- **SQLite** via `aiosqlite` async driver
- Tables: `scans`, `findings`
- Auto-migration in `db/engine.py` — `_migrate()` adds missing columns on startup
- Indexes created in `_ensure_indexes()`: `scans.status`, `scans.sha256`, `scans.package_name`, `scans.package_version`, `scans.flagged`, `scans.nexus_asset_url`, `findings.scan_id`
- `Scan` fields: id, package_name, package_version, ecosystem, repository, nexus_asset_url, sha256, status, total_findings, flagged, started_at, finished_at, error_message, initiator, source_ip
- `Finding` fields: id, scan_id, data (JSON), report (JSON, nullable), created_at

---

## LLM analysis

`finding.report` drives UI state:

| Value | UI |
|-------|----|
| `None` | Show "Analyze with LLM" button (manual mode only) |
| `{"status": "analyzing"}` | Show spinner |
| `{verdict:, summary:, ...}` | Show report + "Retry" link |

**Auto mode** (`LLM_AUTO_ANALYZE=1`): analysis runs immediately after scan; button hidden.
**Manual mode** (`LLM_AUTO_ANALYZE=0`): user clicks button; button visible for each finding.

Per-finding `asyncio.Lock` in `web.py` prevents concurrent analysis of the same finding. Retry passes `?retry=1` to bypass the idempotency guard.

---

## Webhooks

Only `UPDATED` action is accepted (not `CREATED`). Format field in asset data determines ecosystem: `pypi`, `go`, `npm`.

Per-URL locking (asyncio.Lock) prevents parallel scans of the same asset. SHA256 dedup prevents re-scanning identical file content.

---

## Templates (htmx)

- Fragment templates prefixed with `_` are returned for `HX-Request` requests
- Filter-bar lives outside htmx target — never replaced
- Sortable columns use `hx-get` with all filter params in URL
- Language persists via cookie `lang`, set by middleware
- Shared includes: `_status_badge.html` (scan status), `_pagination.html` (page nav), `_llm_spinner.html` (LLM progress)

---

## Docker

```
docker compose up -d --build   # build + start
docker compose down             # stop
docker compose down -v          # stop + destroy volumes (make docker-destroy)
docker compose logs -f          # tail logs
```

The Dockerfile parses `pyproject.toml` for dependency list (single source of truth). GuardDog is installed as a separate `uv pip install` step.

---

## Testing

- `make test` runs `pytest -v`
- Tests use in-memory SQLite (`:memory:`)
- `conftest.py` sets up `os.environ` before importing the app
- Mock `guarddog` output via fixtures — no real CLI execution
- 85 tests covering: API, webhooks, harvester, scanner, web UI

When adding features:
- Always `python3 -m pytest -v` before committing
- Always `ruff check guarddog_nexus tests`
- Add tests for new extractors, endpoints, edge cases

---

## Common tasks

**Add a new ecosystem:**
1. Add extractor in `core/nexus.py` → `EXTRACTORS` dict
2. Add format handler in `routes/webhooks.py` → `_detect_ecosystem()`
3. Create proxy repo in `scripts/setup-nexus.sh`
4. Add test fixture in `tests/conftest.py`
5. Add tests in `tests/test_nexus.py`

**Add a new env var:**
1. `config.py` → field in Config dataclass
2. `.env.example` → add with default
3. `docker-compose.yml` → add to environment if needed in Docker
4. `README.md` → add to env table

**Add a UI string (i18n):**
1. `i18n.py` → add to `_STRINGS` dict
2. Template → `{{ t('key', request.state.lang) }}`

**Run a manual webhook test:**
```bash
curl -X POST http://localhost:8080/webhooks/nexus \
  -H "Content-Type: application/json" \
  -d '{"action":"UPDATED","repositoryName":"pypi-proxy",
       "asset":{"format":"pypi","name":"/packages/pkg/ver/pkg-ver.tar.gz",
       "downloadUrl":"http://nexus:8081/repository/pypi-proxy/packages/pkg/ver/pkg-ver.tar.gz"}}'
```

---

## Notes

- **AI-generated code:** all code in this repository was generated by an AI assistant (Claude). Review carefully before production use.
- **No Nexus Pro required:** the system works with Nexus OSS. Webhooks can be triggered manually or via community plugins.
- **GuardDog deadlocks:** GuardDog is CPU-intensive. Use `MAX_CONCURRENT_SCANS` to avoid resource exhaustion.
- **LLM may be slow:** increase `LLM_TIMEOUT_SECONDS` for large models. Set `LLM_MAX_CONCURRENT_ANALYSES` to limit parallel requests.

---

## Workflow

**After every change** — follow these steps in order:

1. **Document** — update `AGENTS.md` if the change introduces a new concept, env var, endpoint, or workflow.
2. **Lint** — `ruff check guarddog_nexus && ruff format guarddog_nexus`
3. **Test** — `python3 -m pytest -v` (must pass 100%)
4. **Commit** — use the existing commit prefix convention (`feat:`, `fix:`, `refactor:`, `docs:`, `ui:`).
5. **Rebuild** — `docker compose up -d --build` to deploy changes.