docs: sync documentation with latest changes (168 tests, metrics, Makefile, pre-commit)
This commit is contained in:
41
AGENTS.md
41
AGENTS.md
@@ -26,7 +26,7 @@ For local development without Docker:
|
|||||||
make install dev
|
make install dev
|
||||||
export $(cat .env | xargs)
|
export $(cat .env | xargs)
|
||||||
python -m guarddog_nexus.main
|
python -m guarddog_nexus.main
|
||||||
make test # 137 tests (101 unit + 36 e2e)
|
make test # 168 tests
|
||||||
make lint # ruff
|
make lint # ruff
|
||||||
make format # ruff format + fix
|
make format # ruff format + fix
|
||||||
```
|
```
|
||||||
@@ -70,7 +70,7 @@ guarddog_nexus/
|
|||||||
3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates
|
3. `harvester.py` downloads file (async via `asyncio.to_thread`), validates URL against `NEXUS_ALLOWED_HOSTS` (SSRF protection), computes SHA256, deduplicates
|
||||||
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
|
4. `scanner.py` runs `guarddog <ecosystem> scan <file> --output-format json`
|
||||||
5. Findings stored in SQLite (`scans` + `findings` tables)
|
5. Findings stored in SQLite (`scans` + `findings` tables)
|
||||||
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.).
|
6. If `LLM_ENABLED=1` and `LLM_AUTO_ANALYZE=1`, `llm.py` sends findings to the configured model in parallel via `asyncio.gather` (respects `LLM_MAX_CONCURRENT_ANALYSES`). Retry logic with exponential backoff (2s, 4s, 8s, max 3 attempts). `finding.report` state machine: `None` → `{"status": "analyzing"}` → `{verdict, summary, analysis, severity_rating}` or `None` on failure. LLM response validated via `_validate_report()` which applies defaults for missing fields (`verdict→unknown`, `severity_rating→unknown`, etc.). Progress tracked via Prometheus counters: `guarddog_llm_analyzed_total` and `guarddog_llm_pending_total`.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -81,11 +81,14 @@ guarddog_nexus/
|
|||||||
- **Line length:** 100 (ruff)
|
- **Line length:** 100 (ruff)
|
||||||
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
|
- **Lint:** `ruff check guarddog_nexus tests` (E/F/I/W rules)
|
||||||
- **Format:** `ruff format guarddog_nexus tests`
|
- **Format:** `ruff format guarddog_nexus tests`
|
||||||
- **Tests:** `pytest -v` (137 tests: 101 unit + 36 e2e, pytest-asyncio auto mode)
|
- **Tests:** `pytest -v` (168 tests, pytest-asyncio auto mode)
|
||||||
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
|
- **Commits:** Russian descriptions, prefix convention: `feat:`, `fix:`, `refactor:`, `docs:`, `ui:`
|
||||||
- **No comments** in code unless explicitly requested
|
- **No comments** in code unless explicitly requested
|
||||||
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context
|
- **Async I/O:** file reads/writes wrapped in `asyncio.to_thread()` — never raw `open()` in async context
|
||||||
- **Config validation:** `_env_int` logs a warning on invalid values instead of crashing
|
- **Config validation:** `_env_int` logs a warning on invalid values instead of crashing
|
||||||
|
- **Type checking:** `make typecheck` runs `mypy guarddog_nexus` (strict mode)
|
||||||
|
- **Pre-commit:** `.pre-commit-config.yaml` with ruff, ruff-format, trailing-whitespace, end-of-file-fixer, check-yaml, check-toml, check-added-large-files, detect-private-key
|
||||||
|
- **CSV export:** `_csv_safe()` in `api_scans.py` prepends `'` to values starting with `=`, `+`, `-`, `@` — blocks formula injection when opening CSV in spreadsheet apps
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -171,7 +174,35 @@ docker compose down -v # stop + destroy volumes (make docker-destroy)
|
|||||||
docker compose logs -f # tail logs
|
docker compose logs -f # tail logs
|
||||||
```
|
```
|
||||||
|
|
||||||
The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds.
|
The Dockerfile uses `uv pip install . --system` to install the package and all dependencies from `pyproject.toml`. GuardDog is installed as a separate `uv pip install --system "guarddog>=2.10.0"` step. Dependencies are installed before source code COPY for efficient layer caching. A `.dockerignore` excludes cache dirs, tests, and examples. Docker HEALTHCHECK at `/health` runs every 30 seconds.
|
||||||
|
|
||||||
|
Logging driver configured as `json-file` with rotation (max-size: 10m, max-file: 3) for both `guarddog-nexus` and `nexus` services. Nexus service also has a HEALTHCHECK (`curl` to `/service/rest/v1/status`).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Makefile targets
|
||||||
|
|
||||||
|
| Command | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `make install` | Install project dependencies |
|
||||||
|
| `make dev` | Install dev dependencies |
|
||||||
|
| `make test` | Run pytest -v |
|
||||||
|
| `make lint` | Ruff check |
|
||||||
|
| `make format` | Ruff format + fix |
|
||||||
|
| `make typecheck` | mypy strict mode |
|
||||||
|
| `make check` | lint + format + typecheck + test |
|
||||||
|
| `make run` | Start the app via `python -m guarddog_nexus.main` |
|
||||||
|
| `make setup-env` | Copy `.env.example` → `.env` if missing |
|
||||||
|
| `make docker-build` | Build Docker image |
|
||||||
|
| `make docker-up` | Build + start stack (`up -d --build`) |
|
||||||
|
| `make docker-down` | Stop stack |
|
||||||
|
| `make docker-destroy` | Stop + destroy volumes (`-v`) |
|
||||||
|
| `make docker-rebuild` | Down + up --build |
|
||||||
|
| `make docker-logs` | Tail logs |
|
||||||
|
| `make docker-ps` | `docker compose ps` |
|
||||||
|
| `make docker-shell` | Exec bash in guarddog-nexus container |
|
||||||
|
| `make docker-restart` | Restart guarddog-nexus service |
|
||||||
|
| `make clean` | Remove build artifacts |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -181,7 +212,7 @@ The Dockerfile uses `uv pip install . --system` to install the package and all d
|
|||||||
- Tests use in-memory SQLite (`:memory:`)
|
- Tests use in-memory SQLite (`:memory:`)
|
||||||
- `conftest.py` sets up `os.environ` before importing the app
|
- `conftest.py` sets up `os.environ` before importing the app
|
||||||
- Mock `guarddog` output via fixtures — no real CLI execution
|
- Mock `guarddog` output via fixtures — no real CLI execution
|
||||||
- 137 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, e2e flows
|
- 168 tests covering: API, webhooks, harvester, scanner, web UI, i18n, metrics, LLM analysis, config, schemas, engine, e2e flows
|
||||||
- E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling
|
- E2E tests in `tests/e2e/` cover full webhook-to-scan pipeline, API filtering/pagination, LLM analysis, and error handling
|
||||||
|
|
||||||
When adding features:
|
When adding features:
|
||||||
|
|||||||
28
README.en.md
28
README.en.md
@@ -157,7 +157,7 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|
|||||||
|--------|------|-------------|
|
|--------|------|-------------|
|
||||||
| GET | `/health` | Health check |
|
| GET | `/health` | Health check |
|
||||||
| GET | `/health/dependencies` | DB and Nexus API connectivity check |
|
| GET | `/health/dependencies` | DB and Nexus API connectivity check |
|
||||||
| GET | `/metrics` | Prometheus-compatible metrics |
|
| GET | `/metrics` | Prometheus metrics: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` |
|
||||||
|
|
||||||
## Security
|
## Security
|
||||||
|
|
||||||
@@ -168,6 +168,30 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|
|||||||
- Scan results stored in local SQLite database
|
- Scan results stored in local SQLite database
|
||||||
- Temporary package files deleted after scanning
|
- Temporary package files deleted after scanning
|
||||||
|
|
||||||
|
## Makefile Targets
|
||||||
|
|
||||||
|
| Command | Description |
|
||||||
|
|---------|-------------|
|
||||||
|
| `make install` | Install project dependencies |
|
||||||
|
| `make dev` | Install dev dependencies |
|
||||||
|
| `make test` | Run pytest -v |
|
||||||
|
| `make lint` | Ruff check |
|
||||||
|
| `make format` | Ruff format + fix |
|
||||||
|
| `make typecheck` | mypy strict mode |
|
||||||
|
| `make check` | lint + format + typecheck + test |
|
||||||
|
| `make run` | Start app locally |
|
||||||
|
| `make setup-env` | Copy `.env.example` → `.env` if missing |
|
||||||
|
| `make docker-build` | Build Docker image |
|
||||||
|
| `make docker-up` | Build + start stack |
|
||||||
|
| `make docker-down` | Stop stack |
|
||||||
|
| `make docker-destroy` | Stop + destroy volumes |
|
||||||
|
| `make docker-rebuild` | Down + up --build |
|
||||||
|
| `make docker-logs` | Tail logs |
|
||||||
|
| `make docker-ps` | `docker compose ps` |
|
||||||
|
| `make docker-shell` | Exec bash in guarddog-nexus container |
|
||||||
|
| `make docker-restart` | Restart guarddog-nexus service |
|
||||||
|
| `make clean` | Remove build artifacts |
|
||||||
|
|
||||||
## LLM Analysis
|
## LLM Analysis
|
||||||
|
|
||||||
GuardDog Nexus can analyze findings through an LLM. When enabled (`LLM_ENABLED=1`), flagged findings receive an AI breakdown: threat assessment, code analysis, and recommendations.
|
GuardDog Nexus can analyze findings through an LLM. When enabled (`LLM_ENABLED=1`), flagged findings receive an AI breakdown: threat assessment, code analysis, and recommendations.
|
||||||
@@ -237,7 +261,7 @@ guarddog-nexus/
|
|||||||
│ ├── i18n.py # RU/EN translations
|
│ ├── i18n.py # RU/EN translations
|
||||||
│ ├── logging_setup.py # JSON structured logging
|
│ ├── logging_setup.py # JSON structured logging
|
||||||
│ └── main.py # FastAPI app entry point
|
│ └── main.py # FastAPI app entry point
|
||||||
├── tests/ # pytest tests (137 tests: 101 unit + 36 e2e)
|
├── tests/ # pytest tests (168 tests)
|
||||||
├── scripts/ # Setup scripts
|
├── scripts/ # Setup scripts
|
||||||
├── docker-compose.yml
|
├── docker-compose.yml
|
||||||
├── Dockerfile
|
├── Dockerfile
|
||||||
|
|||||||
11
README.md
11
README.md
@@ -188,7 +188,7 @@ curl -X POST http://localhost:8080/webhooks/nexus \
|
|||||||
|-------|------|----------|
|
|-------|------|----------|
|
||||||
| GET | `/health` | Проверка работоспособности |
|
| GET | `/health` | Проверка работоспособности |
|
||||||
| GET | `/health/dependencies` | Проверка БД и доступности Nexus API |
|
| GET | `/health/dependencies` | Проверка БД и доступности Nexus API |
|
||||||
| GET | `/metrics` | Метрики в формате Prometheus |
|
| GET | `/metrics` | Prometheus-метрики: `guarddog_scans_total`, `guarddog_scans_flagged_total`, `guarddog_findings_total`, `guarddog_llm_analyzed_total`, `guarddog_llm_pending_total`, `guarddog_scans_by_status`, `guarddog_scans_by_ecosystem`, `guarddog_last_scan_timestamp_seconds` |
|
||||||
|
|
||||||
## Веб-интерфейс
|
## Веб-интерфейс
|
||||||
|
|
||||||
@@ -240,15 +240,22 @@ guarddog-nexus/
|
|||||||
|---------|----------|
|
|---------|----------|
|
||||||
| `make install` | Установка зависимостей проекта |
|
| `make install` | Установка зависимостей проекта |
|
||||||
| `make dev` | Установка зависимостей для разработки |
|
| `make dev` | Установка зависимостей для разработки |
|
||||||
| `make test` | Запуск тестов |
|
| `make test` | Запуск тестов (168) |
|
||||||
| `make lint` | Проверка кода через Ruff |
|
| `make lint` | Проверка кода через Ruff |
|
||||||
| `make format` | Форматирование кода через Ruff |
|
| `make format` | Форматирование кода через Ruff |
|
||||||
|
| `make typecheck` | Проверка типов через mypy (strict mode) |
|
||||||
|
| `make check` | lint + format + typecheck + test (все проверки) |
|
||||||
|
| `make run` | Запуск приложения локально |
|
||||||
|
| `make setup-env` | Копирование `.env.example` → `.env` (если отсутствует) |
|
||||||
| `make docker-build` | Сборка Docker-образа |
|
| `make docker-build` | Сборка Docker-образа |
|
||||||
| `make docker-up` | Пересборка и запуск стека (`up -d --build`) |
|
| `make docker-up` | Пересборка и запуск стека (`up -d --build`) |
|
||||||
| `make docker-down` | Остановка стека |
|
| `make docker-down` | Остановка стека |
|
||||||
| `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) |
|
| `make docker-destroy` | Остановка стека с удалением всех данных (`-v`) |
|
||||||
| `make docker-logs` | Просмотр логов стека |
|
| `make docker-logs` | Просмотр логов стека |
|
||||||
| `make docker-rebuild` | Полная пересборка (down + build + up) |
|
| `make docker-rebuild` | Полная пересборка (down + build + up) |
|
||||||
|
| `make docker-ps` | Статус контейнеров (`docker compose ps`) |
|
||||||
|
| `make docker-shell` | Интерактивная оболочка в контейнере |
|
||||||
|
| `make docker-restart` | Перезапуск контейнера guarddog-nexus |
|
||||||
| `make clean` | Очистка артефактов сборки |
|
| `make clean` | Очистка артефактов сборки |
|
||||||
|
|
||||||
## Безопасность
|
## Безопасность
|
||||||
|
|||||||
Reference in New Issue
Block a user