From d483a8b21d3db9bf33501a076343a55b06453472 Mon Sep 17 00:00:00 2001 From: Marker689 Date: Sun, 10 May 2026 09:58:34 +0300 Subject: [PATCH] =?UTF-8?q?docs:=20=D0=BE=D0=B1=D0=BD=D0=BE=D0=B2=D0=BB?= =?UTF-8?q?=D1=91=D0=BD=20AGENTS.md=20=E2=80=94=2085=20=D1=82=D0=B5=D1=81?= =?UTF-8?q?=D1=82=D0=BE=D0=B2,=20LLM-=D1=81=D1=82=D0=B5=D0=B9=D1=82-=D0=BC?= =?UTF-8?q?=D0=B0=D1=88=D0=B8=D0=BD=D0=B0,=20=5Fensure=5Findexes,=20async?= =?UTF-8?q?=20I/O?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- AGENTS.md | 31 ++++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index f56960f..bc62a43 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,7 +26,7 @@ For local development without Docker: make install dev export $(cat .env | xargs) python -m guarddog_nexus.main -make test # 50 tests +make test # 85 tests make lint # ruff make format # ruff format + fix ``` @@ -66,10 +66,10 @@ guarddog_nexus/ **Data flow:** 1. Nexus sends `UPDATED` webhook → `POST /webhooks/nexus` 2. `webhooks.py` validates signature, extracts asset info, spawns background task -3. `harvester.py` downloads file, computes SHA256, deduplicates +3. `harvester.py` downloads file (async via `asyncio.to_thread`), computes SHA256, deduplicates 4. `scanner.py` runs `guarddog scan --output-format json` 5. Findings stored in SQLite (`scans` + `findings` tables) -6. If `LLM_ENABLED=1`, `llm.py` sends each finding to the configured model, stores report in `findings.report` +6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends each finding to the configured model. `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. --- @@ -80,9 +80,10 @@ guarddog_nexus/ - **Line length:** 100 (ruff) - **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules) - **Format:** `ruff format guarddog_nexus tests` -- **Tests:** `pytest -v` (50 tests, pytest-asyncio auto mode) +- **Tests:** `pytest -v` (85 tests, pytest-asyncio auto mode) - **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:` - **No comments** in code unless explicitly requested +- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context --- @@ -97,6 +98,7 @@ All via environment variables, defined in `config.py`. Key ones: | `WEBHOOK_SECRET` | `""` | HMAC-SHA256 validation | | `MAX_CONCURRENT_SCANS` | `4` | asyncio.Semaphore for guarddog processes | | `LLM_ENABLED` | `0` | `1` to enable analysis | +| `LLM_AUTO_ANALYZE` | `0` | `1` to auto-trigger after scan; `0` = manual via UI button | | `LLM_API_KEY` | `""` | OpenAI-compatible key | | `LLM_MODEL` | `gpt-4o-mini` | | | `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Semaphore for LLM calls | @@ -111,11 +113,29 @@ Full list in `config.py`. - **SQLite** via `aiosqlite` async driver - Tables: `scans`, `findings` - Auto-migration in `db/engine.py` — `_migrate()` adds missing columns on startup +- Indexes created in `_ensure_indexes()`: `scans.status`, `scans.sha256`, `scans.package_name`, `scans.package_version`, `scans.flagged`, `scans.nexus_asset_url`, `findings.scan_id` - `Scan` fields: id, package_name, package_version, ecosystem, repository, nexus_asset_url, sha256, status, total_findings, flagged, started_at, finished_at, error_message, initiator, source_ip - `Finding` fields: id, scan_id, data (JSON), report (JSON, nullable), created_at --- +## LLM analysis + +`finding.report` drives UI state: + +| Value | UI | +|-------|----| +| `None` | Show "Analyze with LLM" button (manual mode only) | +| `{"status": "analyzing"}` | Show spinner | +| `{verdict:, summary:, ...}` | Show report + "Retry" link | + +**Auto mode** (`LLM_AUTO_ANALYZE=1`): analysis runs immediately after scan; button hidden. +**Manual mode** (`LLM_AUTO_ANALYZE=0`): user clicks button; button visible for each finding. + +Per-finding `asyncio.Lock` in `web.py` prevents concurrent analysis of the same finding. Retry passes `?retry=1` to bypass the idempotency guard. + +--- + ## Webhooks Only `UPDATED` action is accepted (not `CREATED`). Format field in asset data determines ecosystem: `pypi`, `go`, `npm`. @@ -130,6 +150,7 @@ Per-URL locking (asyncio.Lock) prevents parallel scans of the same asset. SHA256 - Filter-bar lives outside htmx target — never replaced - Sortable columns use `hx-get` with all filter params in URL - Language persists via cookie `lang`, set by middleware +- Shared includes: `_status_badge.html` (scan status), `_pagination.html` (page nav), `_llm_spinner.html` (LLM progress) --- @@ -152,7 +173,7 @@ The Dockerfile parses `pyproject.toml` for dependency list (single source of tru - Tests use in-memory SQLite (`:memory:`) - `conftest.py` sets up `os.environ` before importing the app - Mock `guarddog` output via fixtures — no real CLI execution -- 50 tests covering: API, webhooks, harvester, scanner, web UI +- 85 tests covering: API, webhooks, harvester, scanner, web UI When adding features: - Always `python3 -m pytest -v` before committing