From 6ea5c85a4be58401e8798cd1d3059a637a385fc7 Mon Sep 17 00:00:00 2001 From: Marker689 Date: Mon, 11 May 2026 23:21:11 +0300 Subject: [PATCH] docs: sync documentation with latest changes (168 tests, metrics, Makefile, pre-commit) --- AGENTS.md | 41 ++++++++++++++++++++++++++++++++++++----- README.en.md | 28 ++++++++++++++++++++++++++-- README.md | 11 +++++++++-- 3 files changed, 71 insertions(+), 9 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 3d2c762..56eef56 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -26,7 +26,7 @@ For local development without Docker: make install dev export $(cat .env | xargs) python -m guarddog_nexus.main -make test # 137 tests (101 unit + 36 e2e) +make test # 168 tests make lint # ruff make format # ruff format + fix ``` @@ -70,7 +70,7 @@ guarddog_nexus/ 3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates 4. `scanner.py` runs `guarddog scan --output-format json` 5. Findings stored in SQLite (`scans` + `findings` tables) -6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.). +6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.). Progress tracked via Prometheus counters: `guarddog_llm_analyzed_total` and `guarddog_llm_pending_total`. --- @@ -81,11 +81,14 @@ guarddog_nexus/ - **Line length:** 100 (ruff) - **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules) - **Format:** `ruff format guarddog_nexus tests` -- **Tests:** `pytest -v` (137 tests: 101 unit + 36 e2e, pytest-asyncio auto mode) +- **Tests:** `pytest -v` (168 tests, pytest-asyncio auto mode) - **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:` - **No comments** in code unless explicitly requested - **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context - **Config validation:** `_env_int` logs a warning on invalid values instead of crashing +- **Type checking:** `make typecheck` runs `mypy guarddog_nexus` (strict mode) +- **Pre-commit:** `.pre-commit-config.yaml` with ruff, ruff-format, trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-added-large-files, detect-private-key +- **CSV export:** `_csv_safe()` in `api_scans.py` prepends `'` to values starting with `=`, `+`, `-`, `@` — blocks formula injection when opening CSV in spreadsheet apps --- @@ -171,7 +174,35 @@ docker compose down -v # stop + destroy volumes (make docker-destroy) docker compose logs -f # tail logs ``` -The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds. +The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. Dependencies are installed before source code COPY for efficient layer caching. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds. + +Logging driver configured as `json-file` with rotation (max-size: 10m, max-file: 3) for both `guarddog-nexus` and `nexus` services. Nexus service also has a HEALTHCHECK (`curl` to `/service/rest/v1/status`). + +--- + +## Makefile targets + +| Command | Description | +|---------|-------------| +| `make install` | Install project dependencies | +| `make dev` | Install dev dependencies | +| `make test` | Run pytest -v | +| `make lint` | Ruff check | +| `make format` | Ruff format + fix | +| `make typecheck` | mypy strict mode | +| `make check` | lint + format + typecheck + test | +| `make run` | Start the app via `python -m guarddog_nexus.main` | +| `make setup-env` | Copy `.env.example` → `.env` if missing | +| `make docker-build` | Build Docker image | +| `make docker-up` | Build + start stack (`up -d --build`) | +| `make docker-down` | Stop stack | +| `make docker-destroy` | Stop + destroy volumes (`-v`) | +| `make docker-rebuild` | Down + up --build | +| `make docker-logs` | Tail logs | +| `make docker-ps` | `docker compose ps` | +| `make docker-shell` | Exec bash in guarddog-nexus container | +| `make docker-restart` | Restart guarddog-nexus service | +| `make clean` | Remove build artifacts | --- @@ -181,7 +212,7 @@ The Dockerfile uses `uv pip install . --system` to install the package and all d - Tests use in-memory SQLite (`:memory:`) - `conftest.py` sets up `os.environ` before importing the app - Mock `guarddog` output via fixtures — no real CLI execution -- 137 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, e2e flows +- 168 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, config, schemas, engine, e2e flows - E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling When adding features: diff --git a/README.en.md b/README.en.md index a8b8017..d60ee56 100644 --- a/README.en.md +++ b/README.en.md @@ -157,7 +157,7 @@ curl -X POST http://localhost:8080/webhooks/nexus \ |--------|------|-------------| | GET | `/health` | Health check | | GET | `/health/dependencies` | DB and Nexus API connectivity check | -| GET | `/metrics` | Prometheus-compatible metrics | +| GET | `/metrics` | Prometheus metrics: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` | ## Security @@ -168,6 +168,30 @@ curl -X POST http://localhost:8080/webhooks/nexus \ - Scan results stored in local SQLite database - Temporary package files deleted after scanning +## Makefile Targets + +| Command | Description | +|---------|-------------| +| `make install` | Install project dependencies | +| `make dev` | Install dev dependencies | +| `make test` | Run pytest -v | +| `make lint` | Ruff check | +| `make format` | Ruff format + fix | +| `make typecheck` | mypy strict mode | +| `make check` | lint + format + typecheck + test | +| `make run` | Start app locally | +| `make setup-env` | Copy `.env.example` → `.env` if missing | +| `make docker-build` | Build Docker image | +| `make docker-up` | Build + start stack | +| `make docker-down` | Stop stack | +| `make docker-destroy` | Stop + destroy volumes | +| `make docker-rebuild` | Down + up --build | +| `make docker-logs` | Tail logs | +| `make docker-ps` | `docker compose ps` | +| `make docker-shell` | Exec bash in guarddog-nexus container | +| `make docker-restart` | Restart guarddog-nexus service | +| `make clean` | Remove build artifacts | + ## LLM Analysis GuardDog Nexus can analyze findings through an LLM. When enabled (`LLM_ENABLED=1`), flagged findings receive an AI breakdown: threat assessment, code analysis, and recommendations. @@ -237,7 +261,7 @@ guarddog-nexus/ │ ├── i18n.py # RU/EN translations │ ├── logging_setup.py # JSON structured logging │ └── main.py # FastAPI app entry point -├── tests/ # pytest tests (137 tests: 101 unit + 36 e2e) +├── tests/ # pytest tests (168 tests) ├── scripts/ # Setup scripts ├── docker-compose.yml ├── Dockerfile diff --git a/README.md b/README.md index 12296bd..39be4bf 100644 --- a/README.md +++ b/README.md @@ -188,7 +188,7 @@ curl -X POST http://localhost:8080/webhooks/nexus \ |-------|------|----------| | GET | `/health` | Проверка работоспособности | | GET | `/health/dependencies` | Проверка БД и доступности Nexus API | -| GET | `/metrics` | Метрики в формате Prometheus | +| GET | `/metrics` | Prometheus-метрики: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` | ## Веб-интерфейс @@ -240,15 +240,22 @@ guarddog-nexus/ |---------|----------| | `make install` | Установка зависимостей проекта | | `make dev` | Установка зависимостей для разработки | -| `make test` | Запуск тестов | +| `make test` | Запуск тестов (168) | | `make lint` | Проверка кода через Ruff | | `make format` | Форматирование кода через Ruff | +| `make typecheck` | Проверка типов через mypy (strict mode) | +| `make check` | lint + format + typecheck + test (все проверки) | +| `make run` | Запуск приложения локально | +| `make setup-env` | Копирование `.env.example` → `.env` (если отсутствует) | | `make docker-build` | Сборка Docker-образа | | `make docker-up` | Пересборка и запуск стека (`up -d --build`) | | `make docker-down` | Остановка стека | | `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) | | `make docker-logs` | Просмотр логов стека | | `make docker-rebuild` | Полная пересборка (down + build + up) | +| `make docker-ps` | Статус контейнеров (`docker compose ps`) | +| `make docker-shell` | Интерактивная оболочка в контейнере | +| `make docker-restart` | Перезапуск контейнера guarddog-nexus | | `make clean` | Очистка артефактов сборки | ## Безопасность