docs: sync documentation with latest changes (168 tests, metrics, Makefile, pre-commit)

This commit is contained in:
Marker689
2026-05-11 23:21:11 +03:00
parent 5c8cbabefd
commit 6ea5c85a4b
3 changed files with 71 additions and 9 deletions

View File

@@ -26,7 +26,7 @@ For local development without Docker:
make install dev make install dev
export $(cat .env | xargs) export $(cat .env | xargs)
python -m guarddog_nexus.main python -m guarddog_nexus.main
make test # 137 tests (101 unit + 36 e2e) make test # 168 tests
make lint # ruff make lint # ruff
make format # ruff format + fix make format # ruff format + fix
``` ```
@@ -70,7 +70,7 @@ guarddog_nexus/
3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates 3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json` 4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
5. Findings stored in SQLite (`scans` + `findings` tables) 5. Findings stored in SQLite (`scans` + `findings` tables)
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None``{"status": "analyzing"}``{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.). 6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None``{"status": "analyzing"}``{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.). Progress tracked via Prometheus counters: `guarddog_llm_analyzed_total` and `guarddog_llm_pending_total`.
--- ---
@@ -81,11 +81,14 @@ guarddog_nexus/
- **Line length:** 100 (ruff) - **Line length:** 100 (ruff)
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules) - **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
- **Format:** `ruff format guarddog_nexus tests` - **Format:** `ruff format guarddog_nexus tests`
- **Tests:** `pytest -v` (137 tests: 101 unit + 36 e2e, pytest-asyncio auto mode) - **Tests:** `pytest -v` (168 tests, pytest-asyncio auto mode)
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:` - **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
- **No comments** in code unless explicitly requested - **No comments** in code unless explicitly requested
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context - **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context
- **Config validation:** `_env_int` logs a warning on invalid values instead of crashing - **Config validation:** `_env_int` logs a warning on invalid values instead of crashing
- **Type checking:** `make typecheck` runs `mypy guarddog_nexus` (strict mode)
- **Pre-commit:** `.pre-commit-config.yaml` with ruff, ruff-format, trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-added-large-files, detect-private-key
- **CSV export:** `_csv_safe()` in `api_scans.py` prepends `'` to values starting with `=`, `+`, `-`, `@` — blocks formula injection when opening CSV in spreadsheet apps
--- ---
@@ -171,7 +174,35 @@ docker compose down -v # stop + destroy volumes (make docker-destroy)
docker compose logs -f # tail logs docker compose logs -f # tail logs
``` ```
The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds. The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. Dependencies are installed before source code COPY for efficient layer caching. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds.
Logging driver configured as `json-file` with rotation (max-size: 10m, max-file: 3) for both `guarddog-nexus` and `nexus` services. Nexus service also has a HEALTHCHECK (`curl` to `/service/rest/v1/status`).
---
## Makefile targets
| Command | Description |
|---------|-------------|
| `make install` | Install project dependencies |
| `make dev` | Install dev dependencies |
| `make test` | Run pytest -v |
| `make lint` | Ruff check |
| `make format` | Ruff format + fix |
| `make typecheck` | mypy strict mode |
| `make check` | lint + format + typecheck + test |
| `make run` | Start the app via `python -m guarddog_nexus.main` |
| `make setup-env` | Copy `.env.example` → `.env` if missing |
| `make docker-build` | Build Docker image |
| `make docker-up` | Build + start stack (`up -d --build`) |
| `make docker-down` | Stop stack |
| `make docker-destroy` | Stop + destroy volumes (`-v`) |
| `make docker-rebuild` | Down + up --build |
| `make docker-logs` | Tail logs |
| `make docker-ps` | `docker compose ps` |
| `make docker-shell` | Exec bash in guarddog-nexus container |
| `make docker-restart` | Restart guarddog-nexus service |
| `make clean` | Remove build artifacts |
--- ---
@@ -181,7 +212,7 @@ The Dockerfile uses `uv pip install . --system` to install the package and all d
- Tests use in-memory SQLite (`:memory:`) - Tests use in-memory SQLite (`:memory:`)
- `conftest.py` sets up `os.environ` before importing the app - `conftest.py` sets up `os.environ` before importing the app
- Mock `guarddog` output via fixtures — no real CLI execution - Mock `guarddog` output via fixtures — no real CLI execution
- 137 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, e2e flows - 168 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, config, schemas, engine, e2e flows
- E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling - E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling
When adding features: When adding features:

View File

@@ -157,7 +157,7 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|--------|------|-------------| |--------|------|-------------|
| GET | `/health` | Health check | | GET | `/health` | Health check |
| GET | `/health/dependencies` | DB and Nexus API connectivity check | | GET | `/health/dependencies` | DB and Nexus API connectivity check |
| GET | `/metrics` | Prometheus-compatible metrics | | GET | `/metrics` | Prometheus metrics: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` |
## Security ## Security
@@ -168,6 +168,30 @@ curl -X POST http://localhost:8080/webhooks/nexus \
- Scan results stored in local SQLite database - Scan results stored in local SQLite database
- Temporary package files deleted after scanning - Temporary package files deleted after scanning
## Makefile Targets
| Command | Description |
|---------|-------------|
| `make install` | Install project dependencies |
| `make dev` | Install dev dependencies |
| `make test` | Run pytest -v |
| `make lint` | Ruff check |
| `make format` | Ruff format + fix |
| `make typecheck` | mypy strict mode |
| `make check` | lint + format + typecheck + test |
| `make run` | Start app locally |
| `make setup-env` | Copy `.env.example``.env` if missing |
| `make docker-build` | Build Docker image |
| `make docker-up` | Build + start stack |
| `make docker-down` | Stop stack |
| `make docker-destroy` | Stop + destroy volumes |
| `make docker-rebuild` | Down + up --build |
| `make docker-logs` | Tail logs |
| `make docker-ps` | `docker compose ps` |
| `make docker-shell` | Exec bash in guarddog-nexus container |
| `make docker-restart` | Restart guarddog-nexus service |
| `make clean` | Remove build artifacts |
## LLM Analysis ## LLM Analysis
GuardDog Nexus can analyze findings through an LLM. When enabled (`LLM_ENABLED=1`), flagged findings receive an AI breakdown: threat assessment, code analysis, and recommendations. GuardDog Nexus can analyze findings through an LLM. When enabled (`LLM_ENABLED=1`), flagged findings receive an AI breakdown: threat assessment, code analysis, and recommendations.
@@ -237,7 +261,7 @@ guarddog-nexus/
│ ├── i18n.py # RU/EN translations │ ├── i18n.py # RU/EN translations
│ ├── logging_setup.py # JSON structured logging │ ├── logging_setup.py # JSON structured logging
│ └── main.py # FastAPI app entry point │ └── main.py # FastAPI app entry point
├── tests/ # pytest tests (137 tests: 101 unit + 36 e2e) ├── tests/ # pytest tests (168 tests)
├── scripts/ # Setup scripts ├── scripts/ # Setup scripts
├── docker-compose.yml ├── docker-compose.yml
├── Dockerfile ├── Dockerfile

View File

@@ -188,7 +188,7 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|-------|------|----------| |-------|------|----------|
| GET | `/health` | Проверка работоспособности | | GET | `/health` | Проверка работоспособности |
| GET | `/health/dependencies` | Проверка БД и доступности Nexus API | | GET | `/health/dependencies` | Проверка БД и доступности Nexus API |
| GET | `/metrics` | Метрики в формате Prometheus | | GET | `/metrics` | Prometheus-метрики: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` |
## Веб-интерфейс ## Веб-интерфейс
@@ -240,15 +240,22 @@ guarddog-nexus/
|---------|----------| |---------|----------|
| `make install` | Установка зависимостей проекта | | `make install` | Установка зависимостей проекта |
| `make dev` | Установка зависимостей для разработки | | `make dev` | Установка зависимостей для разработки |
| `make test` | Запуск тестов | | `make test` | Запуск тестов (168) |
| `make lint` | Проверка кода через Ruff | | `make lint` | Проверка кода через Ruff |
| `make format` | Форматирование кода через Ruff | | `make format` | Форматирование кода через Ruff |
| `make typecheck` | Проверка типов через mypy (strict mode) |
| `make check` | lint + format + typecheck + test (все проверки) |
| `make run` | Запуск приложения локально |
| `make setup-env` | Копирование `.env.example``.env` (если отсутствует) |
| `make docker-build` | Сборка Docker-образа | | `make docker-build` | Сборка Docker-образа |
| `make docker-up` | Пересборка и запуск стека (`up -d --build`) | | `make docker-up` | Пересборка и запуск стека (`up -d --build`) |
| `make docker-down` | Остановка стека | | `make docker-down` | Остановка стека |
| `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) | | `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) |
| `make docker-logs` | Просмотр логов стека | | `make docker-logs` | Просмотр логов стека |
| `make docker-rebuild` | Полная пересборка (down + build + up) | | `make docker-rebuild` | Полная пересборка (down + build + up) |
| `make docker-ps` | Статус контейнеров (`docker compose ps`) |
| `make docker-shell` | Интерактивная оболочка в контейнере |
| `make docker-restart` | Перезапуск контейнера guarddog-nexus |
| `make clean` | Очистка артефактов сборки | | `make clean` | Очистка артефактов сборки |
## Безопасность ## Безопасность