docs: обновлён AGENTS.md — 85 тестов, LLM-стейт-машина, _ensure_indexes, async I/O
This commit is contained in:
31
AGENTS.md
31
AGENTS.md
@@ -26,7 +26,7 @@ For local development without Docker:
|
|||||||
make install dev
|
make install dev
|
||||||
export $(cat .env | xargs)
|
export $(cat .env | xargs)
|
||||||
python -m guarddog_nexus.main
|
python -m guarddog_nexus.main
|
||||||
make test # 50 tests
|
make test # 85 tests
|
||||||
make lint # ruff
|
make lint # ruff
|
||||||
make format # ruff format + fix
|
make format # ruff format + fix
|
||||||
```
|
```
|
||||||
@@ -66,10 +66,10 @@ guarddog_nexus/
|
|||||||
**Data flow:**
|
**Data flow:**
|
||||||
1. Nexus sends `UPDATED` webhook → `POST /webhooks/nexus`
|
1. Nexus sends `UPDATED` webhook → `POST /webhooks/nexus`
|
||||||
2. `webhooks.py` validates signature, extracts asset info, spawns background task
|
2. `webhooks.py` validates signature, extracts asset info, spawns background task
|
||||||
3. `harvester.py` downloads file, computes SHA256, deduplicates
|
3. `harvester.py` downloads file (async via `asyncio.to_thread`), computes SHA256, deduplicates
|
||||||
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
|
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
|
||||||
5. Findings stored in SQLite (`scans` + `findings` tables)
|
5. Findings stored in SQLite (`scans` + `findings` tables)
|
||||||
6. If `LLM_ENABLED=1`, `llm.py` sends each finding to the configured model, stores report in `findings.report`
|
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends each finding to the configured model. `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -80,9 +80,10 @@ guarddog_nexus/
|
|||||||
- **Line length:** 100 (ruff)
|
- **Line length:** 100 (ruff)
|
||||||
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
|
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
|
||||||
- **Format:** `ruff format guarddog_nexus tests`
|
- **Format:** `ruff format guarddog_nexus tests`
|
||||||
- **Tests:** `pytest -v` (50 tests, pytest-asyncio auto mode)
|
- **Tests:** `pytest -v` (85 tests, pytest-asyncio auto mode)
|
||||||
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
|
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
|
||||||
- **No comments** in code unless explicitly requested
|
- **No comments** in code unless explicitly requested
|
||||||
|
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -97,6 +98,7 @@ All via environment variables, defined in `config.py`. Key ones:
|
|||||||
| `WEBHOOK_SECRET` | `""` | HMAC-SHA256 validation |
|
| `WEBHOOK_SECRET` | `""` | HMAC-SHA256 validation |
|
||||||
| `MAX_CONCURRENT_SCANS` | `4` | asyncio.Semaphore for guarddog processes |
|
| `MAX_CONCURRENT_SCANS` | `4` | asyncio.Semaphore for guarddog processes |
|
||||||
| `LLM_ENABLED` | `0` | `1` to enable analysis |
|
| `LLM_ENABLED` | `0` | `1` to enable analysis |
|
||||||
|
| `LLM_AUTO_ANALYZE` | `0` | `1` to auto-trigger after scan; `0` = manual via UI button |
|
||||||
| `LLM_API_KEY` | `""` | OpenAI-compatible key |
|
| `LLM_API_KEY` | `""` | OpenAI-compatible key |
|
||||||
| `LLM_MODEL` | `gpt-4o-mini` | |
|
| `LLM_MODEL` | `gpt-4o-mini` | |
|
||||||
| `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Semaphore for LLM calls |
|
| `LLM_MAX_CONCURRENT_ANALYSES` | `2` | Semaphore for LLM calls |
|
||||||
@@ -111,11 +113,29 @@ Full list in `config.py`.
|
|||||||
- **SQLite** via `aiosqlite` async driver
|
- **SQLite** via `aiosqlite` async driver
|
||||||
- Tables: `scans`, `findings`
|
- Tables: `scans`, `findings`
|
||||||
- Auto-migration in `db/engine.py` — `_migrate()` adds missing columns on startup
|
- Auto-migration in `db/engine.py` — `_migrate()` adds missing columns on startup
|
||||||
|
- Indexes created in `_ensure_indexes()`: `scans.status`, `scans.sha256`, `scans.package_name`, `scans.package_version`, `scans.flagged`, `scans.nexus_asset_url`, `findings.scan_id`
|
||||||
- `Scan` fields: id, package_name, package_version, ecosystem, repository, nexus_asset_url, sha256, status, total_findings, flagged, started_at, finished_at, error_message, initiator, source_ip
|
- `Scan` fields: id, package_name, package_version, ecosystem, repository, nexus_asset_url, sha256, status, total_findings, flagged, started_at, finished_at, error_message, initiator, source_ip
|
||||||
- `Finding` fields: id, scan_id, data (JSON), report (JSON, nullable), created_at
|
- `Finding` fields: id, scan_id, data (JSON), report (JSON, nullable), created_at
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## LLM analysis
|
||||||
|
|
||||||
|
`finding.report` drives UI state:
|
||||||
|
|
||||||
|
| Value | UI |
|
||||||
|
|-------|----|
|
||||||
|
| `None` | Show "Analyze with LLM" button (manual mode only) |
|
||||||
|
| `{"status": "analyzing"}` | Show spinner |
|
||||||
|
| `{verdict:, summary:, ...}` | Show report + "Retry" link |
|
||||||
|
|
||||||
|
**Auto mode** (`LLM_AUTO_ANALYZE=1`): analysis runs immediately after scan; button hidden.
|
||||||
|
**Manual mode** (`LLM_AUTO_ANALYZE=0`): user clicks button; button visible for each finding.
|
||||||
|
|
||||||
|
Per-finding `asyncio.Lock` in `web.py` prevents concurrent analysis of the same finding. Retry passes `?retry=1` to bypass the idempotency guard.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Webhooks
|
## Webhooks
|
||||||
|
|
||||||
Only `UPDATED` action is accepted (not `CREATED`). Format field in asset data determines ecosystem: `pypi`, `go`, `npm`.
|
Only `UPDATED` action is accepted (not `CREATED`). Format field in asset data determines ecosystem: `pypi`, `go`, `npm`.
|
||||||
@@ -130,6 +150,7 @@ Per-URL locking (asyncio.Lock) prevents parallel scans of the same asset. SHA256
|
|||||||
- Filter-bar lives outside htmx target — never replaced
|
- Filter-bar lives outside htmx target — never replaced
|
||||||
- Sortable columns use `hx-get` with all filter params in URL
|
- Sortable columns use `hx-get` with all filter params in URL
|
||||||
- Language persists via cookie `lang`, set by middleware
|
- Language persists via cookie `lang`, set by middleware
|
||||||
|
- Shared includes: `_status_badge.html` (scan status), `_pagination.html` (page nav), `_llm_spinner.html` (LLM progress)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -152,7 +173,7 @@ The Dockerfile parses `pyproject.toml` for dependency list (single source of tru
|
|||||||
- Tests use in-memory SQLite (`:memory:`)
|
- Tests use in-memory SQLite (`:memory:`)
|
||||||
- `conftest.py` sets up `os.environ` before importing the app
|
- `conftest.py` sets up `os.environ` before importing the app
|
||||||
- Mock `guarddog` output via fixtures — no real CLI execution
|
- Mock `guarddog` output via fixtures — no real CLI execution
|
||||||
- 50 tests covering: API, webhooks, harvester, scanner, web UI
|
- 85 tests covering: API, webhooks, harvester, scanner, web UI
|
||||||
|
|
||||||
When adding features:
|
When adding features:
|
||||||
- Always `python3 -m pytest -v` before committing
|
- Always `python3 -m pytest -v` before committing
|
||||||
|
|||||||
Reference in New Issue
Block a user