16 Commits

Author SHA1 Message Date
16b2330cd0 Merge pull request '(feature) Add manual storage metrics refresh' (#102) from issue-98-storage-metrics-refresh into master
Reviewed-on: #102
2026-06-08 23:39:22 +02:00
89397b0cda (feature) Add manual storage metrics refresh
Add a storage metrics refresh service and management command that update cached snapshot storage metadata outside web requests. The command supports host/kind filters, bounded scans, and dry-run output for operational refreshes.

Refs #98
2026-06-08 23:31:32 +02:00
aac424e7ac Merge pull request '(bugfix) Bound snapshot storage metadata scans' (#101) from issue-100-bound-storage-metadata-scans into master
Reviewed-on: #101
2026-06-08 23:26:23 +02:00
9ece39b72e (bugfix) Bound snapshot storage metadata scans
Limit snapshot storage scans recorded by backup workers so very large backup targets cannot make run finalization walk unbounded file trees. Limited scans now record scan_limited, entries_scanned, and max_entries in snapshot storage metadata.

Closes #100
2026-06-08 23:18:50 +02:00
42b3430274 Merge pull request '(bugfix) Avoid live backup data scans in web views' (#99) from issue-97-avoid-live-storage-scans into master
Reviewed-on: #99
2026-06-08 23:11:54 +02:00
f886a7b620 (bugfix) Avoid live backup data scans in web views
Use stored snapshot storage metadata for dashboard and host backup data summaries instead of walking snapshot directories during request rendering. Snapshots without recorded storage metadata are counted as not measured so large backup targets cannot trigger unbounded filesystem scans from live-refresh views.

Closes #97
2026-06-08 22:48:22 +02:00
522ae98bb1 Merge pull request '(bugfix) Allow blank notification webhook headers' (#96) from issue-95-notification-500 into master
Reviewed-on: #96
2026-06-08 22:25:16 +02:00
Codex
ac0cdb59d6 (bugfix) Allow blank notification webhook headers
Normalize blank notification webhook headers to an empty JSON object so creating email targets from the browser does not try to store NULL in the JSON field.

Closes #95
2026-06-08 22:23:03 +02:00
51142081c9 Merge pull request '(release) Prepare 1.2.0' (#74) from release-1.2 into master
Reviewed-on: #74
2026-05-28 22:19:23 +02:00
02616eebbc (release) Prepare 1.2.0
Bump the package version to 1.2.0 and document the operations-focused release
with updater, readonly access, notifications, bandwidth controls, live progress,
and more robust retention cleanup.

Make the CLI version test assert against the package version so future release
bumps do not require changing a hardcoded expected value.
2026-05-28 22:19:03 +02:00
a61e3d8302 Merge pull request '(feature) Add staff updater page' (#73) from issue-39-updater-ui into master
Reviewed-on: #73
2026-05-28 22:11:07 +02:00
0450f8bdb0 (feature) Add staff updater page
Add a Django updater view for checking configured Gitea releases, inspecting
the installed git checkout, fetching tags, pulling the current branch, and
running the configured native systemd update command.

Document the updater environment settings and keep the page staff-only so
readonly status users cannot trigger deployment actions.
2026-05-28 22:10:45 +02:00
b4833560b5 Merge pull request '(feature) Add read-only access level to control panel' (#72) from issue-40-access-levels into master
Reviewed-on: #72
2026-05-28 22:00:45 +02:00
81ee848f5f (feature) Add read-only access level to control panel
Introduce a central access policy that lets authenticated non-staff users view
backup status pages while keeping credentials, logs, configs, and mutating
actions staff-only.

Hide sensitive navigation and host controls for read-only users, expose only
the status API to authenticated viewers, and document the two access levels.
2026-05-28 22:00:16 +02:00
7f2bbe4d20 Merge pull request '(bugfix) Enable rsync progress output for live real runs' (#71) from issue-64-rsync-progress-live-runs into master
Reviewed-on: #71
2026-05-28 21:43:02 +02:00
29f455a153 (bugfix) Enable rsync progress output for live real runs
Default queued and management-command backups to verbose rsync output so live
run views show progress for long-running real backups, matching dry-run
visibility.

Add a quiet-rsync escape hatch for operators who intentionally want less noisy
real-run logs.
2026-05-28 21:42:40 +02:00
40 changed files with 1520 additions and 327 deletions

View File

@@ -1,5 +1,34 @@
# Changelog # Changelog
## 1.2.0 - 2026-05-28
Operations-focused release for more reliable production backups and maintenance.
### Added
- Staff-only updater page for checking configured Gitea releases, inspecting the installed git checkout, fetching tags, pulling the current branch, and running the native systemd updater.
- Read-only control panel access level for authenticated non-staff users, with status pages visible and credentials, logs, configs, retention, and mutating actions kept staff-only.
- Run completion notifications for email and webhooks, including recorded delivery history per run and target.
- Dedicated hosts page with host cards, enabled/disabled filtering, and quick host/schedule/retention state controls.
- Per-host rsync bandwidth limit overrides with inherit, unlimited, and explicit limit semantics.
- Backup data totals by snapshot kind on dashboard and host detail pages, including unique/non-hardlinked data totals.
### Changed
- Real backup runs now default to verbose rsync progress output so the live run view behaves consistently with dry-runs.
- Run progress panels are shared between dry-runs and real runs for more consistent status, timing, cancellation, and log display.
- Incomplete snapshot cleanup now requires operator review before deletion.
- Incomplete snapshot size reporting now prefers on-disk measurement when metadata is stale or missing.
- Installer and environment examples now include optional updater configuration.
### Fixed
- Remote preflight shell commands are now quoted correctly, including roots such as `/`.
- Worker reconciliation now detects real rsync failures and stale/running process state more reliably.
- Retention pruning and incomplete cleanup can delete snapshots containing restrictive directory modes preserved by rsync archive mode.
- Snapshot data summaries no longer count incomplete metadata/log files as backup data when measuring from disk.
- Filesystem SSH credential tests use writable test state without changing production defaults.
## 1.1.0 - 2026-05-21 ## 1.1.0 - 2026-05-21
UI-focused release for the Django control panel. UI-focused release for the Django control panel.

View File

@@ -131,6 +131,11 @@ Create a superuser if needed:
sudo -u pobsync pobsync-manage createsuperuser sudo -u pobsync pobsync-manage createsuperuser
``` ```
The control panel supports two access levels. Django staff users can manage hosts, SSH keys, configs, retention,
notifications, logs, and administrative actions. Normal authenticated users can view backup status pages such as the
dashboard, hosts, runs, snapshots, schedules, purged history, changelog, and `/api/status/`, but cannot see SSH
credentials or run mutating actions.
For other Django management commands on native installs, use `pobsync-manage` so the production environment file is For other Django management commands on native installs, use `pobsync-manage` so the production environment file is
loaded before Django starts: loaded before Django starts:
@@ -153,6 +158,7 @@ The UI includes:
- Django-managed SSH keys - Django-managed SSH keys
- `/self-check/` for runtime checks - `/self-check/` for runtime checks
- `/logs/` for filtered pobsync service logs - `/logs/` for filtered pobsync service logs
- `/updater/` for checking Gitea releases, pulling the git checkout, and running the native updater
## Bandwidth Limits ## Bandwidth Limits
@@ -238,6 +244,21 @@ The updater is a thin wrapper around the installer for normal production deploys
Python dependencies, runs migrations, collects static files, and restarts the systemd services so new Django code is Python dependencies, runs migrations, collects static files, and restarts the systemd services so new Django code is
loaded. loaded.
The Django control panel also exposes an `/updater/` page for staff users. It can check a Gitea releases endpoint, run
`git fetch`, run a fast-forward-only pull for the installed branch, and invoke the configured native update command.
Configure these optional environment variables in `/etc/pobsync/pobsync.env`:
```
POBSYNC_UPDATE_RELEASES_URL=https://code.example.test/api/v1/repos/owner/pobsync/releases
POBSYNC_UPDATE_RELEASES_TOKEN=
POBSYNC_UPDATE_GIT_REMOTE=origin
POBSYNC_UPDATE_COMMAND=sudo -n scripts/update-systemd
```
If the web service runs as the `pobsync` user, `POBSYNC_UPDATE_COMMAND` needs a matching sudoers rule or a different
operator-approved command. Without that, the page still shows update status and command output, but the native update
action will fail with a permission error instead of silently doing the wrong thing.
Use the full installer again when you intentionally want to change install-time settings, install OS packages, enable Use the full installer again when you intentionally want to change install-time settings, install OS packages, enable
nginx, or rewrite the environment file: nginx, or rewrite the environment file:

View File

@@ -16,3 +16,11 @@ POBSYNC_GUNICORN_WORKERS=2
POBSYNC_GUNICORN_TIMEOUT=120 POBSYNC_GUNICORN_TIMEOUT=120
POBSYNC_WORKER_INTERVAL=15 POBSYNC_WORKER_INTERVAL=15
POBSYNC_SCHEDULER_INTERVAL=60 POBSYNC_SCHEDULER_INTERVAL=60
# Optional UI updater integration.
# Point this at the Gitea releases API endpoint, for example:
# https://code.example.test/api/v1/repos/owner/pobsync/releases
POBSYNC_UPDATE_RELEASES_URL=
POBSYNC_UPDATE_RELEASES_TOKEN=
POBSYNC_UPDATE_GIT_REMOTE=origin
POBSYNC_UPDATE_COMMAND=sudo -n scripts/update-systemd

View File

@@ -73,8 +73,13 @@ One-off maintenance commands are still available when the UI is not the right to
pobsync backup <host> --dry-run pobsync backup <host> --dry-run
pobsync discover-snapshots --host <host> pobsync discover-snapshots --host <host>
pobsync retention <host> pobsync retention <host>
pobsync django refresh_pobsync_storage_metrics --host <host>
``` ```
`refresh_pobsync_storage_metrics` refreshes cached snapshot storage metadata outside web requests. Use `--kind` to limit
the scan to `scheduled`, `manual`, or `incomplete`, `--max-entries` to bound large scans, and `--dry-run` to inspect
candidate counts without writing metadata.
For scripted configuration changes, call the Django management command explicitly so it is clear that this is an For scripted configuration changes, call the Django management command explicitly so it is clear that this is an
automation/debugging path rather than the normal UI workflow: automation/debugging path rather than the normal UI workflow:

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "pobsync" name = "pobsync"
version = "1.1.0" version = "1.2.0"
description = "Pull-based rsync backup tool with hardlinked snapshots" description = "Pull-based rsync backup tool with hardlinked snapshots"
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [ dependencies = [

View File

@@ -472,6 +472,10 @@ POBSYNC_GUNICORN_WORKERS=2
POBSYNC_GUNICORN_TIMEOUT=120 POBSYNC_GUNICORN_TIMEOUT=120
POBSYNC_WORKER_INTERVAL=15 POBSYNC_WORKER_INTERVAL=15
POBSYNC_SCHEDULER_INTERVAL=60 POBSYNC_SCHEDULER_INTERVAL=60
POBSYNC_UPDATE_RELEASES_URL=
POBSYNC_UPDATE_RELEASES_TOKEN=
POBSYNC_UPDATE_GIT_REMOTE=origin
POBSYNC_UPDATE_COMMAND=sudo -n scripts/update-systemd
EOF EOF
chmod 0640 "$ENV_FILE" chmod 0640 "$ENV_FILE"
chown "root:$SERVICE_GROUP" "$ENV_FILE" chown "root:$SERVICE_GROUP" "$ENV_FILE"

View File

@@ -1,2 +1,2 @@
__all__ = ["__version__"] __all__ = ["__version__"]
__version__ = "1.1.0" __version__ = "1.2.0"

View File

@@ -24,6 +24,7 @@ from ..util import ensure_dir, realpath_startswith, sanitize_host, write_yaml_at
DEFAULT_DRY_RUN_TIMEOUT_SECONDS = 900 DEFAULT_DRY_RUN_TIMEOUT_SECONDS = 900
RSYNC_PARTIAL_VANISHED_EXIT_CODE = 24 RSYNC_PARTIAL_VANISHED_EXIT_CODE = 24
SNAPSHOT_STORAGE_SCAN_MAX_ENTRIES = 200_000
def dry_run_log_path(host: str, run_id: int | None = None) -> Path: def dry_run_log_path(host: str, run_id: int | None = None) -> Path:
@@ -100,7 +101,11 @@ def _collect_run_stats(
) -> dict[str, Any]: ) -> dict[str, Any]:
stats: dict[str, Any] = { stats: dict[str, Any] = {
"rsync": read_rsync_stats(log_path), "rsync": read_rsync_stats(log_path),
"storage": collect_storage_stats(backup_root=backup_root, snapshot_dir=snapshot_dir), "storage": collect_storage_stats(
backup_root=backup_root,
snapshot_dir=snapshot_dir,
snapshot_max_entries=SNAPSHOT_STORAGE_SCAN_MAX_ENTRIES if snapshot_dir is not None else None,
),
} }
if duration_seconds is not None: if duration_seconds is not None:
stats["duration_seconds"] = int(duration_seconds) stats["duration_seconds"] = int(duration_seconds)

View File

@@ -62,7 +62,12 @@ def read_rsync_stats(log_path: Path) -> dict[str, Any]:
return parse_rsync_stats(text) return parse_rsync_stats(text)
def collect_storage_stats(*, backup_root: Path, snapshot_dir: Path | None = None) -> dict[str, Any]: def collect_storage_stats(
*,
backup_root: Path,
snapshot_dir: Path | None = None,
snapshot_max_entries: int | None = None,
) -> dict[str, Any]:
stats: dict[str, Any] = { stats: dict[str, Any] = {
"backup_root": str(backup_root), "backup_root": str(backup_root),
} }
@@ -71,7 +76,7 @@ def collect_storage_stats(*, backup_root: Path, snapshot_dir: Path | None = None
stats["capacity"] = capacity stats["capacity"] = capacity
if snapshot_dir is not None: if snapshot_dir is not None:
snapshot_usage = tree_usage(snapshot_dir) snapshot_usage = tree_usage(snapshot_dir, max_entries=snapshot_max_entries)
if snapshot_usage: if snapshot_usage:
stats["snapshot"] = { stats["snapshot"] = {
"path": str(snapshot_dir), "path": str(snapshot_dir),
@@ -103,13 +108,15 @@ def filesystem_capacity(path: Path) -> dict[str, Any]:
} }
def tree_usage(path: Path) -> dict[str, Any]: def tree_usage(path: Path, *, max_entries: int | None = None) -> dict[str, Any]:
apparent_size = 0 apparent_size = 0
allocated_size = 0 allocated_size = 0
files = 0 files = 0
directories = 0 directories = 0
hardlinked_files = 0 hardlinked_files = 0
hardlinked_apparent_size = 0 hardlinked_apparent_size = 0
entries_scanned = 0
scan_limited = False
seen_allocated_inodes: set[tuple[int, int]] = set() seen_allocated_inodes: set[tuple[int, int]] = set()
try: try:
@@ -119,6 +126,7 @@ def tree_usage(path: Path) -> dict[str, Any]:
if path.is_file(): if path.is_file():
files = 1 files = 1
entries_scanned = 1
apparent_size = root_stat.st_size apparent_size = root_stat.st_size
allocated_size = int(getattr(root_stat, "st_blocks", 0) * 512) allocated_size = int(getattr(root_stat, "st_blocks", 0) * 512)
if root_stat.st_nlink > 1: if root_stat.st_nlink > 1:
@@ -126,14 +134,25 @@ def tree_usage(path: Path) -> dict[str, Any]:
hardlinked_apparent_size = root_stat.st_size hardlinked_apparent_size = root_stat.st_size
else: else:
for current_root, dirnames, filenames in path.walk(): for current_root, dirnames, filenames in path.walk():
directories += len(dirnames) for _dirname in dirnames:
if _scan_limit_reached(entries_scanned, max_entries):
scan_limited = True
break
directories += 1
entries_scanned += 1
if scan_limited:
break
for filename in filenames: for filename in filenames:
if _scan_limit_reached(entries_scanned, max_entries):
scan_limited = True
break
file_path = current_root / filename file_path = current_root / filename
try: try:
file_stat = file_path.lstat() file_stat = file_path.lstat()
except OSError: except OSError:
continue continue
files += 1 files += 1
entries_scanned += 1
apparent_size += file_stat.st_size apparent_size += file_stat.st_size
inode_key = (file_stat.st_dev, file_stat.st_ino) inode_key = (file_stat.st_dev, file_stat.st_ino)
if inode_key not in seen_allocated_inodes: if inode_key not in seen_allocated_inodes:
@@ -142,6 +161,8 @@ def tree_usage(path: Path) -> dict[str, Any]:
if file_stat.st_nlink > 1: if file_stat.st_nlink > 1:
hardlinked_files += 1 hardlinked_files += 1
hardlinked_apparent_size += file_stat.st_size hardlinked_apparent_size += file_stat.st_size
if scan_limited:
break
return { return {
"path": str(path), "path": str(path),
@@ -149,12 +170,19 @@ def tree_usage(path: Path) -> dict[str, Any]:
"allocated_size_bytes": int(allocated_size), "allocated_size_bytes": int(allocated_size),
"files": files, "files": files,
"directories": directories, "directories": directories,
"entries_scanned": entries_scanned,
"scan_limited": scan_limited,
"max_entries": max_entries,
"hardlinked_files": hardlinked_files, "hardlinked_files": hardlinked_files,
"hardlinked_apparent_size_bytes": int(hardlinked_apparent_size), "hardlinked_apparent_size_bytes": int(hardlinked_apparent_size),
"hardlink_apparent_ratio": round(hardlinked_apparent_size / apparent_size, 4) if apparent_size else 0.0, "hardlink_apparent_ratio": round(hardlinked_apparent_size / apparent_size, 4) if apparent_size else 0.0,
} }
def _scan_limit_reached(entries_scanned: int, max_entries: int | None) -> bool:
return max_entries is not None and max_entries >= 0 and entries_scanned >= max_entries
def _parse_colon_stat(line: str, stats: dict[str, Any]) -> None: def _parse_colon_stat(line: str, stats: dict[str, Any]) -> None:
if ":" not in line: if ":" not in line:
return return

View File

@@ -0,0 +1,39 @@
from __future__ import annotations
from collections.abc import Callable
from functools import wraps
from typing import Any
from django.contrib.auth.decorators import login_required
from django.core.exceptions import PermissionDenied
from django.http import HttpRequest, HttpResponse
def can_view_status(user) -> bool:
return bool(user.is_authenticated)
def can_manage_control_panel(user) -> bool:
return bool(user.is_authenticated and user.is_staff)
def status_view_required(view_func: Callable[..., HttpResponse]) -> Callable[..., HttpResponse]:
return login_required(view_func)
def control_panel_admin_required(view_func: Callable[..., HttpResponse]) -> Callable[..., HttpResponse]:
@login_required
@wraps(view_func)
def wrapper(request: HttpRequest, *args: Any, **kwargs: Any) -> HttpResponse:
if not can_manage_control_panel(request.user):
raise PermissionDenied
return view_func(request, *args, **kwargs)
return wrapper
def access_context(request: HttpRequest) -> dict[str, Any]:
return {
"can_view_status": can_view_status(request.user),
"can_manage_control_panel": can_manage_control_panel(request.user),
}

View File

@@ -2,16 +2,16 @@ from __future__ import annotations
from typing import Any from typing import Any
from django.contrib.admin.views.decorators import staff_member_required
from django.db import connection from django.db import connection
from django.db.models import Count from django.db.models import Count
from django.http import JsonResponse from django.http import JsonResponse
from django.utils import timezone from django.utils import timezone
from .models import BackupRun, HostConfig, ScheduleConfig, SnapshotRecord from .models import BackupRun, HostConfig, ScheduleConfig, SnapshotRecord
from .access import control_panel_admin_required, status_view_required
@staff_member_required @control_panel_admin_required
def api_index(request) -> JsonResponse: def api_index(request) -> JsonResponse:
return JsonResponse( return JsonResponse(
{ {
@@ -26,7 +26,7 @@ def api_index(request) -> JsonResponse:
) )
@staff_member_required @status_view_required
def status(request) -> JsonResponse: def status(request) -> JsonResponse:
latest_run = BackupRun.objects.select_related("host", "snapshot").order_by("-created_at").first() latest_run = BackupRun.objects.select_related("host", "snapshot").order_by("-created_at").first()
latest_schedule = ScheduleConfig.objects.select_related("host").order_by("-last_started_at", "-updated_at").first() latest_schedule = ScheduleConfig.objects.select_related("host").order_by("-last_started_at", "-updated_at").first()
@@ -55,7 +55,7 @@ def status(request) -> JsonResponse:
) )
@staff_member_required @control_panel_admin_required
def hosts(request) -> JsonResponse: def hosts(request) -> JsonResponse:
host_qs = ( host_qs = (
HostConfig.objects.annotate(snapshot_count=Count("snapshots", distinct=True), run_count=Count("runs", distinct=True)) HostConfig.objects.annotate(snapshot_count=Count("snapshots", distinct=True), run_count=Count("runs", distinct=True))
@@ -65,7 +65,7 @@ def hosts(request) -> JsonResponse:
return JsonResponse({"ok": True, "hosts": [_host_payload(host) for host in host_qs]}) return JsonResponse({"ok": True, "hosts": [_host_payload(host) for host in host_qs]})
@staff_member_required @control_panel_admin_required
def snapshots(request) -> JsonResponse: def snapshots(request) -> JsonResponse:
snapshot_qs = SnapshotRecord.objects.select_related("host", "base").order_by("host__host", "-started_at", "dirname") snapshot_qs = SnapshotRecord.objects.select_related("host", "base").order_by("host__host", "-started_at", "dirname")
host_filter = request.GET.get("host") host_filter = request.GET.get("host")
@@ -78,7 +78,7 @@ def snapshots(request) -> JsonResponse:
return JsonResponse({"ok": True, "snapshots": [_snapshot_payload(snapshot) for snapshot in snapshot_qs[:limit]]}) return JsonResponse({"ok": True, "snapshots": [_snapshot_payload(snapshot) for snapshot in snapshot_qs[:limit]]})
@staff_member_required @control_panel_admin_required
def runs(request) -> JsonResponse: def runs(request) -> JsonResponse:
run_qs = BackupRun.objects.select_related("host", "snapshot").order_by("-created_at") run_qs = BackupRun.objects.select_related("host", "snapshot").order_by("-created_at")
host_filter = request.GET.get("host") host_filter = request.GET.get("host")

View File

@@ -27,7 +27,7 @@ def queue_backup_run(
host: HostConfig, host: HostConfig,
run_type: str = BackupRun.RunType.MANUAL, run_type: str = BackupRun.RunType.MANUAL,
dry_run: bool = False, dry_run: bool = False,
verbose_output: bool = False, verbose_output: bool = True,
prune: bool = False, prune: bool = False,
prune_max_delete: int = 10, prune_max_delete: int = 10,
prune_protect_bases: bool = False, prune_protect_bases: bool = False,

View File

@@ -0,0 +1,9 @@
from __future__ import annotations
from django.http import HttpRequest
from .access import access_context
def pobsync_access(request: HttpRequest) -> dict[str, object]:
return access_context(request)

View File

@@ -208,6 +208,10 @@ class NotificationTargetForm(forms.ModelForm):
recipients = [line.strip() for line in value.replace(",", "\n").splitlines() if line.strip()] recipients = [line.strip() for line in value.replace(",", "\n").splitlines() if line.strip()]
return "\n".join(recipients) return "\n".join(recipients)
def clean_webhook_headers(self) -> dict[str, object]:
value = self.cleaned_data.get("webhook_headers")
return value or {}
class SshCredentialForm(forms.ModelForm): class SshCredentialForm(forms.ModelForm):
private_key_file = forms.FileField( private_key_file = forms.FileField(

View File

@@ -0,0 +1,50 @@
from __future__ import annotations
import json
from typing import Any
from django.core.management.base import BaseCommand, CommandError
from pobsync.snapshot_meta import normalize_kind
from pobsync_backend.models import HostConfig
from pobsync_backend.storage_metrics import DEFAULT_STORAGE_SCAN_MAX_ENTRIES, refresh_snapshot_storage_metrics
class Command(BaseCommand):
help = "Refresh cached snapshot storage metrics outside web requests."
def add_arguments(self, parser) -> None:
parser.add_argument("--host", default=None)
parser.add_argument("--kind", default="all", help="scheduled|manual|incomplete|all")
parser.add_argument("--max-entries", type=int, default=DEFAULT_STORAGE_SCAN_MAX_ENTRIES)
parser.add_argument("--dry-run", action="store_true", help="Measure candidates without writing metadata")
def handle(self, *args: Any, **options: Any) -> None:
host = None
if options["host"]:
try:
host = HostConfig.objects.get(host=options["host"])
except HostConfig.DoesNotExist as exc:
raise CommandError(f"Missing host {options['host']!r}") from exc
kind = normalize_kind(options["kind"])
result = refresh_snapshot_storage_metrics(
host=host,
kind=None if kind == "all" else kind,
max_entries=int(options["max_entries"]),
dry_run=bool(options["dry_run"]),
)
self.stdout.write(
json.dumps(
{
"scanned": result.scanned,
"updated": result.updated,
"skipped": result.skipped,
"missing": result.missing,
"errors": result.errors,
"dry_run": bool(options["dry_run"]),
},
indent=2,
sort_keys=True,
)
)

View File

@@ -19,6 +19,7 @@ class Command(BaseCommand):
parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Runtime state root") parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Runtime state root")
parser.add_argument("--dry-run", action="store_true", help="Run rsync --dry-run") parser.add_argument("--dry-run", action="store_true", help="Run rsync --dry-run")
parser.add_argument("--verbose-rsync", action="store_true", help="Write itemized rsync output to the run log") parser.add_argument("--verbose-rsync", action="store_true", help="Write itemized rsync output to the run log")
parser.add_argument("--quiet-rsync", action="store_true", help="Skip default rsync progress output for real runs")
parser.add_argument("--prune", action="store_true", help="Apply retention after a successful run") parser.add_argument("--prune", action="store_true", help="Apply retention after a successful run")
parser.add_argument("--prune-max-delete", type=int, default=10) parser.add_argument("--prune-max-delete", type=int, default=10)
parser.add_argument("--prune-protect-bases", action="store_true") parser.add_argument("--prune-protect-bases", action="store_true")
@@ -32,6 +33,7 @@ class Command(BaseCommand):
except HostConfig.DoesNotExist as exc: except HostConfig.DoesNotExist as exc:
raise CommandError(f"Missing enabled host {host_name!r}") from exc raise CommandError(f"Missing enabled host {host_name!r}") from exc
verbose_output = bool(options["dry_run"] or options["verbose_rsync"] or not options["quiet_rsync"])
run = BackupRun.objects.create( run = BackupRun.objects.create(
host=host, host=host,
run_type=BackupRun.RunType.MANUAL if options["manual"] else BackupRun.RunType.SCHEDULED, run_type=BackupRun.RunType.MANUAL if options["manual"] else BackupRun.RunType.SCHEDULED,
@@ -39,7 +41,7 @@ class Command(BaseCommand):
result={ result={
"requested": { "requested": {
"dry_run": bool(options["dry_run"]), "dry_run": bool(options["dry_run"]),
"verbose_output": bool(options["dry_run"] or options["verbose_rsync"]), "verbose_output": verbose_output,
"prune": bool(options["prune"]), "prune": bool(options["prune"]),
"prune_max_delete": int(options["prune_max_delete"]), "prune_max_delete": int(options["prune_max_delete"]),
"prune_protect_bases": bool(options["prune_protect_bases"]), "prune_protect_bases": bool(options["prune_protect_bases"]),
@@ -50,7 +52,7 @@ class Command(BaseCommand):
run=run, run=run,
prefix=paths.home, prefix=paths.home,
dry_run=bool(options["dry_run"]), dry_run=bool(options["dry_run"]),
verbose_output=bool(options["dry_run"] or options["verbose_rsync"]), verbose_output=verbose_output,
prune=bool(options["prune"]), prune=bool(options["prune"]),
prune_max_delete=int(options["prune_max_delete"]), prune_max_delete=int(options["prune_max_delete"]),
prune_protect_bases=bool(options["prune_protect_bases"]), prune_protect_bases=bool(options["prune_protect_bases"]),

View File

@@ -5,7 +5,7 @@ from typing import Any, Iterable
from django.utils import timezone from django.utils import timezone
from pobsync.run_stats import filesystem_capacity, tree_usage from pobsync.run_stats import filesystem_capacity
from .models import BackupRun, GlobalConfig, HostConfig, SnapshotRecord from .models import BackupRun, GlobalConfig, HostConfig, SnapshotRecord
@@ -118,14 +118,26 @@ def _backup_data_by_kind(host: HostConfig) -> dict[str, Any]:
for snapshot in host.snapshots.all(): for snapshot in host.snapshots.all():
summary = _snapshot_summary(snapshot) summary = _snapshot_summary(snapshot)
row = rows.setdefault(snapshot.kind, _empty_snapshot_data_row()) row = rows.setdefault(snapshot.kind, _empty_snapshot_data_row())
allocated = summary.get("allocated_size_bytes") or summary.get("apparent_size_bytes") or 0
apparent = summary.get("apparent_size_bytes") or 0
unique_apparent = summary.get("unique_apparent_size_bytes") or 0
row["count"] += 1 row["count"] += 1
total["count"] += 1
if not summary.get("storage_measured"):
row["unknown_count"] += 1
total["unknown_count"] += 1
continue
allocated = summary.get("allocated_size_bytes")
if allocated is None:
allocated = summary.get("apparent_size_bytes")
apparent = summary.get("apparent_size_bytes")
unique_apparent = summary.get("unique_apparent_size_bytes")
allocated = int(allocated or 0)
apparent = int(apparent or 0)
unique_apparent = int(unique_apparent or 0)
row["measured_count"] += 1
row["allocated_size_bytes"] += int(allocated) row["allocated_size_bytes"] += int(allocated)
row["apparent_size_bytes"] += int(apparent) row["apparent_size_bytes"] += int(apparent)
row["unique_apparent_size_bytes"] += int(unique_apparent) row["unique_apparent_size_bytes"] += int(unique_apparent)
total["count"] += 1 total["measured_count"] += 1
total["allocated_size_bytes"] += int(allocated) total["allocated_size_bytes"] += int(allocated)
total["apparent_size_bytes"] += int(apparent) total["apparent_size_bytes"] += int(apparent)
total["unique_apparent_size_bytes"] += int(unique_apparent) total["unique_apparent_size_bytes"] += int(unique_apparent)
@@ -141,6 +153,8 @@ def _backup_data_by_kind(host: HostConfig) -> dict[str, Any]:
def _empty_snapshot_data_row() -> dict[str, int]: def _empty_snapshot_data_row() -> dict[str, int]:
return { return {
"count": 0, "count": 0,
"measured_count": 0,
"unknown_count": 0,
"allocated_size_bytes": 0, "allocated_size_bytes": 0,
"apparent_size_bytes": 0, "apparent_size_bytes": 0,
"unique_apparent_size_bytes": 0, "unique_apparent_size_bytes": 0,
@@ -159,6 +173,8 @@ def _sum_backup_data_by_kind(rows: Iterable[dict[str, dict[str, int]]]) -> dict[
for kind, values in row.items(): for kind, values in row.items():
total_row = total_rows.setdefault(kind, _empty_snapshot_data_row()) total_row = total_rows.setdefault(kind, _empty_snapshot_data_row())
total_row["count"] += values.get("count", 0) total_row["count"] += values.get("count", 0)
total_row["measured_count"] += values.get("measured_count", 0)
total_row["unknown_count"] += values.get("unknown_count", 0)
total_row["allocated_size_bytes"] += values.get("allocated_size_bytes", 0) total_row["allocated_size_bytes"] += values.get("allocated_size_bytes", 0)
total_row["apparent_size_bytes"] += values.get("apparent_size_bytes", 0) total_row["apparent_size_bytes"] += values.get("apparent_size_bytes", 0)
total_row["unique_apparent_size_bytes"] += values.get("unique_apparent_size_bytes", 0) total_row["unique_apparent_size_bytes"] += values.get("unique_apparent_size_bytes", 0)
@@ -173,15 +189,7 @@ def _snapshot_summary(snapshot: SnapshotRecord | None) -> dict[str, Any]:
stats = metadata.get("stats") if isinstance(metadata.get("stats"), dict) else {} stats = metadata.get("stats") if isinstance(metadata.get("stats"), dict) else {}
storage = stats.get("storage") if isinstance(stats.get("storage"), dict) else {} storage = stats.get("storage") if isinstance(stats.get("storage"), dict) else {}
snapshot_storage = storage.get("snapshot") if isinstance(storage.get("snapshot"), dict) else {} snapshot_storage = storage.get("snapshot") if isinstance(storage.get("snapshot"), dict) else {}
if snapshot.kind == SnapshotRecord.Kind.INCOMPLETE: storage_measured = _has_recorded_snapshot_storage(snapshot_storage)
snapshot_storage = _snapshot_storage_from_filesystem(snapshot)
else:
has_recorded_size = (
_int_at(snapshot_storage, "allocated_size_bytes") is not None
or _int_at(snapshot_storage, "apparent_size_bytes") is not None
)
if not has_recorded_size:
snapshot_storage = _snapshot_storage_from_filesystem(snapshot)
apparent_size = _int_at(snapshot_storage, "apparent_size_bytes") apparent_size = _int_at(snapshot_storage, "apparent_size_bytes")
hardlinked_apparent = _int_at(snapshot_storage, "hardlinked_apparent_size_bytes") or 0 hardlinked_apparent = _int_at(snapshot_storage, "hardlinked_apparent_size_bytes") or 0
return { return {
@@ -195,19 +203,18 @@ def _snapshot_summary(snapshot: SnapshotRecord | None) -> dict[str, Any]:
"hardlinked_files": _int_at(snapshot_storage, "hardlinked_files"), "hardlinked_files": _int_at(snapshot_storage, "hardlinked_files"),
"hardlinked_apparent_size_bytes": hardlinked_apparent, "hardlinked_apparent_size_bytes": hardlinked_apparent,
"unique_apparent_size_bytes": max((apparent_size or 0) - hardlinked_apparent, 0), "unique_apparent_size_bytes": max((apparent_size or 0) - hardlinked_apparent, 0),
"storage_measured": storage_measured,
} }
def _snapshot_storage_from_filesystem(snapshot: SnapshotRecord) -> dict[str, Any]: def _has_recorded_snapshot_storage(snapshot_storage: dict[str, Any]) -> bool:
if not snapshot.path: return any(
return {} _int_at(snapshot_storage, key) is not None
snapshot_path = Path(snapshot.path) for key in (
data_path = snapshot_path / "data" "allocated_size_bytes",
if snapshot_path.name == "data": "apparent_size_bytes",
return tree_usage(snapshot_path) )
if data_path.exists(): )
return tree_usage(data_path)
return tree_usage(snapshot_path)
def _is_real_run(run: BackupRun) -> bool: def _is_real_run(run: BackupRun) -> bool:

View File

@@ -0,0 +1,133 @@
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Iterable
from django.utils import timezone
from pobsync.run_stats import tree_usage
from .models import HostConfig, SnapshotRecord
DEFAULT_STORAGE_SCAN_MAX_ENTRIES = 200_000
@dataclass(frozen=True)
class StorageMetricsRefreshResult:
scanned: int
updated: int
skipped: int
missing: int
errors: int
def refresh_snapshot_storage_metrics(
*,
host: HostConfig | None = None,
kind: str | None = None,
max_entries: int = DEFAULT_STORAGE_SCAN_MAX_ENTRIES,
dry_run: bool = False,
) -> StorageMetricsRefreshResult:
result = {
"scanned": 0,
"updated": 0,
"skipped": 0,
"missing": 0,
"errors": 0,
}
for snapshot in _snapshot_queryset(host=host, kind=kind):
refresh = refresh_snapshot_storage_metric(snapshot, max_entries=max_entries, dry_run=dry_run)
result["scanned"] += 1
result[refresh["status"]] += 1
return StorageMetricsRefreshResult(**result)
def refresh_snapshot_storage_metric(
snapshot: SnapshotRecord,
*,
max_entries: int = DEFAULT_STORAGE_SCAN_MAX_ENTRIES,
dry_run: bool = False,
) -> dict[str, Any]:
data_path = _snapshot_data_path(snapshot)
if data_path is None or not data_path.exists():
_record_storage_measurement_error(snapshot, reason="missing_path", dry_run=dry_run)
return {"status": "missing", "snapshot": snapshot, "path": str(data_path) if data_path else ""}
try:
usage = tree_usage(data_path, max_entries=max_entries)
except OSError as exc:
_record_storage_measurement_error(snapshot, reason=type(exc).__name__, message=str(exc), dry_run=dry_run)
return {"status": "errors", "snapshot": snapshot, "path": str(data_path), "error": str(exc)}
if not usage:
_record_storage_measurement_error(snapshot, reason="unreadable", dry_run=dry_run)
return {"status": "errors", "snapshot": snapshot, "path": str(data_path)}
metadata = _metadata_with_storage(snapshot.metadata, usage=usage, source="manual_refresh")
if dry_run:
return {"status": "skipped", "snapshot": snapshot, "path": str(data_path), "usage": usage}
snapshot.metadata = metadata
snapshot.save(update_fields=["metadata"])
return {"status": "updated", "snapshot": snapshot, "path": str(data_path), "usage": usage}
def _snapshot_queryset(*, host: HostConfig | None, kind: str | None) -> Iterable[SnapshotRecord]:
snapshots = SnapshotRecord.objects.select_related("host").order_by("host__host", "kind", "dirname")
if host is not None:
snapshots = snapshots.filter(host=host)
if kind:
snapshots = snapshots.filter(kind=kind)
return snapshots
def _snapshot_data_path(snapshot: SnapshotRecord) -> Path | None:
if not snapshot.path:
return None
snapshot_path = Path(snapshot.path)
data_path = snapshot_path / "data"
if snapshot_path.name == "data":
return snapshot_path
if data_path.exists():
return data_path
return snapshot_path
def _metadata_with_storage(metadata: object, *, usage: dict[str, Any], source: str) -> dict[str, Any]:
metadata_dict = dict(metadata) if isinstance(metadata, dict) else {}
stats = dict(metadata_dict.get("stats")) if isinstance(metadata_dict.get("stats"), dict) else {}
storage = dict(stats.get("storage")) if isinstance(stats.get("storage"), dict) else {}
storage["snapshot"] = {
**usage,
"measured_at": timezone.now().isoformat(),
"measurement_source": source,
}
stats["storage"] = storage
metadata_dict["stats"] = stats
return metadata_dict
def _record_storage_measurement_error(
snapshot: SnapshotRecord,
*,
reason: str,
message: str = "",
dry_run: bool,
) -> None:
if dry_run:
return
metadata = dict(snapshot.metadata) if isinstance(snapshot.metadata, dict) else {}
stats = dict(metadata.get("stats")) if isinstance(metadata.get("stats"), dict) else {}
storage = dict(stats.get("storage")) if isinstance(stats.get("storage"), dict) else {}
storage["snapshot_measurement_error"] = {
"reason": reason,
"message": message,
"measured_at": timezone.now().isoformat(),
"measurement_source": "manual_refresh",
}
stats["storage"] = storage
metadata["stats"] = stats
snapshot.metadata = metadata
snapshot.save(update_fields=["metadata"])

View File

@@ -275,6 +275,23 @@
.status.skipped { color: var(--muted); background: #f7f9fb; } .status.skipped { color: var(--muted); background: #f7f9fb; }
.stack { display: grid; gap: 5px; } .stack { display: grid; gap: 5px; }
.stack.spaced { margin-bottom: 14px; } .stack.spaced { margin-bottom: 14px; }
.detail-list {
display: grid;
gap: 8px 14px;
grid-template-columns: minmax(120px, max-content) minmax(0, 1fr);
margin: 0 0 12px;
}
.detail-list dt {
color: var(--muted);
font-size: 12px;
font-weight: 750;
text-transform: uppercase;
}
.detail-list dd {
margin: 0;
min-width: 0;
overflow-wrap: anywhere;
}
.two-col { display: grid; gap: 18px; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); } .two-col { display: grid; gap: 18px; grid-template-columns: minmax(0, 1fr) minmax(0, 1fr); }
.panel-grid { .panel-grid {
display: grid; display: grid;
@@ -919,17 +936,24 @@
<span class="nav-primary" aria-label="Primary navigation"> <span class="nav-primary" aria-label="Primary navigation">
<a href="{% url 'dashboard' %}" {% if request.resolver_match.url_name == "dashboard" %}aria-current="page"{% endif %}>Dashboard</a> <a href="{% url 'dashboard' %}" {% if request.resolver_match.url_name == "dashboard" %}aria-current="page"{% endif %}>Dashboard</a>
<a href="{% url 'hosts_list' %}" {% if request.resolver_match.url_name == "hosts_list" or request.resolver_match.url_name == "host_detail" or request.resolver_match.url_name == "create_host_config" or request.resolver_match.url_name == "edit_host_config" or request.resolver_match.url_name == "edit_host_schedule" %}aria-current="page"{% endif %}>Hosts</a> <a href="{% url 'hosts_list' %}" {% if request.resolver_match.url_name == "hosts_list" or request.resolver_match.url_name == "host_detail" or request.resolver_match.url_name == "create_host_config" or request.resolver_match.url_name == "edit_host_config" or request.resolver_match.url_name == "edit_host_schedule" %}aria-current="page"{% endif %}>Hosts</a>
<a href="{% url 'ssh_credentials' %}" {% if request.resolver_match.url_name == "ssh_credentials" or request.resolver_match.url_name == "create_ssh_credential" or request.resolver_match.url_name == "generate_ssh_credential" or request.resolver_match.url_name == "edit_ssh_credential" %}aria-current="page"{% endif %}>SSH Keys</a> {% if can_manage_control_panel %}
<a href="{% url 'notification_targets' %}" {% if request.resolver_match.url_name == "notification_targets" or request.resolver_match.url_name == "create_notification_target" or request.resolver_match.url_name == "edit_notification_target" %}aria-current="page"{% endif %}>Notifications</a> <a href="{% url 'ssh_credentials' %}" {% if request.resolver_match.url_name == "ssh_credentials" or request.resolver_match.url_name == "create_ssh_credential" or request.resolver_match.url_name == "generate_ssh_credential" or request.resolver_match.url_name == "edit_ssh_credential" %}aria-current="page"{% endif %}>SSH Keys</a>
<a href="{% url 'logs' %}" {% if request.resolver_match.url_name == "logs" %}aria-current="page"{% endif %}>Logs</a> <a href="{% url 'notification_targets' %}" {% if request.resolver_match.url_name == "notification_targets" or request.resolver_match.url_name == "create_notification_target" or request.resolver_match.url_name == "edit_notification_target" %}aria-current="page"{% endif %}>Notifications</a>
<a href="{% url 'logs' %}" {% if request.resolver_match.url_name == "logs" %}aria-current="page"{% endif %}>Logs</a>
<a href="{% url 'updater' %}" {% if request.resolver_match.url_name == "updater" %}aria-current="page"{% endif %}>Updater</a>
{% endif %}
<a href="{% url 'purged_snapshots' %}" {% if request.resolver_match.url_name == "purged_snapshots" %}aria-current="page"{% endif %}>Purged</a> <a href="{% url 'purged_snapshots' %}" {% if request.resolver_match.url_name == "purged_snapshots" %}aria-current="page"{% endif %}>Purged</a>
</span> </span>
<span class="spacer"></span> <span class="spacer"></span>
<span class="nav-secondary" aria-label="System navigation"> <span class="nav-secondary" aria-label="System navigation">
<a href="{% url 'self_check' %}" {% if request.resolver_match.url_name == "self_check" %}aria-current="page"{% endif %}>Self Check</a> {% if can_manage_control_panel %}
<a href="{% url 'self_check' %}" {% if request.resolver_match.url_name == "self_check" %}aria-current="page"{% endif %}>Self Check</a>
{% endif %}
<a href="{% url 'changelog' %}" {% if request.resolver_match.url_name == "changelog" %}aria-current="page"{% endif %}>Changelog</a> <a href="{% url 'changelog' %}" {% if request.resolver_match.url_name == "changelog" %}aria-current="page"{% endif %}>Changelog</a>
<a href="/api/status/">Status API</a> <a href="/api/status/">Status API</a>
<a href="{% url 'admin:index' %}">Admin</a> {% if can_manage_control_panel %}
<a href="{% url 'admin:index' %}">Admin</a>
{% endif %}
</span> </span>
<span class="muted nav-user">{{ request.user.username }}</span> <span class="muted nav-user">{{ request.user.username }}</span>
</nav> </nav>

View File

@@ -9,27 +9,31 @@
<h1>Dashboard</h1> <h1>Dashboard</h1>
<div class="page-subtitle">Backup health, required action, storage pressure, and recent activity in one place.</div> <div class="page-subtitle">Backup health, required action, storage pressure, and recent activity in one place.</div>
</div> </div>
<section class="actions" aria-label="Dashboard actions"> {% if can_manage_control_panel %}
<a class="button-link" href="{% url 'create_host_config' %}">New host</a> <section class="actions" aria-label="Dashboard actions">
<a class="button-link secondary" href="{% url 'edit_global_config' %}">{% if global_config %}Edit global config{% else %}Create global config{% endif %}</a> <a class="button-link" href="{% url 'create_host_config' %}">New host</a>
</section> <a class="button-link secondary" href="{% url 'edit_global_config' %}">{% if global_config %}Edit global config{% else %}Create global config{% endif %}</a>
</section>
{% endif %}
</header> </header>
{% if not global_config or not counts.hosts %} {% if can_manage_control_panel %}
<section class="panel"> {% if not global_config or not counts.hosts %}
<h2>Setup</h2> <section class="panel">
{% if not global_config %} <h2>Setup</h2>
<p class="muted">No default global config exists yet. Create it first so discovery, backups, and retention know where pobsync stores snapshots.</p> {% if not global_config %}
<div class="actions inline"> <p class="muted">No default global config exists yet. Create it first so discovery, backups, and retention know where pobsync stores snapshots.</p>
<a class="button-link" href="{% url 'edit_global_config' %}">Create global config</a> <div class="actions inline">
</div> <a class="button-link" href="{% url 'edit_global_config' %}">Create global config</a>
{% elif not counts.hosts %} </div>
<p class="muted">Global config is ready. Add the first host to make this dashboard useful.</p> {% elif not counts.hosts %}
<div class="actions inline"> <p class="muted">Global config is ready. Add the first host to make this dashboard useful.</p>
<a class="button-link" href="{% url 'create_host_config' %}">Add first host</a> <div class="actions inline">
</div> <a class="button-link" href="{% url 'create_host_config' %}">Add first host</a>
{% endif %} </div>
</section> {% endif %}
</section>
{% endif %}
{% endif %} {% endif %}
<div <div

View File

@@ -27,10 +27,12 @@
{{ retention_warning.incomplete_count }} incomplete snapshot(s) exist. Retention does not delete incomplete {{ retention_warning.incomplete_count }} incomplete snapshot(s) exist. Retention does not delete incomplete
snapshots automatically; inspect them before cleanup. snapshots automatically; inspect them before cleanup.
</div> </div>
<form method="post" action="{% url 'resolve_host_incomplete_reviews' host.host %}"> {% if can_manage_control_panel %}
{% csrf_token %} <form method="post" action="{% url 'resolve_host_incomplete_reviews' host.host %}">
<button type="submit" class="secondary">Mark incomplete reviewed</button> {% csrf_token %}
</form> <button type="submit" class="secondary">Mark incomplete reviewed</button>
</form>
{% endif %}
{% endif %} {% endif %}
{% if retention_warning.error %} {% if retention_warning.error %}
<div>{{ retention_warning.error }}</div> <div>{{ retention_warning.error }}</div>
@@ -80,8 +82,9 @@
</div> </div>
</article> </article>
<article class="panel host-control-panel"> {% if can_manage_control_panel %}
<h2>Backup Control</h2> <article class="panel host-control-panel">
<h2>Backup Control</h2>
<div class="operator-state"> <div class="operator-state">
{% if active_run %} {% if active_run %}
<span class="status {{ active_run.status }}">{{ active_run.status }}</span> <span class="status {{ active_run.status }}">{{ active_run.status }}</span>
@@ -105,6 +108,7 @@
</form> </form>
<form class="inline-form" method="post" action="{% url 'queue_manual_backup' host.host %}"> <form class="inline-form" method="post" action="{% url 'queue_manual_backup' host.host %}">
{% csrf_token %} {% csrf_token %}
<input type="hidden" name="verbose_output" value="on">
<input type="hidden" name="prune_max_delete" value="10"> <input type="hidden" name="prune_max_delete" value="10">
<button type="submit" {% if not can_queue_real_backup %}disabled{% endif %}>Queue backup</button> <button type="submit" {% if not can_queue_real_backup %}disabled{% endif %}>Queue backup</button>
</form> </form>
@@ -120,10 +124,16 @@
<p class="muted">Real backups are blocked by failed preflight checks. Dry-runs may still be available when storage-only checks fail.</p> <p class="muted">Real backups are blocked by failed preflight checks. Dry-runs may still be available when storage-only checks fail.</p>
{% endif %} {% endif %}
{% endif %} {% endif %}
</article> </article>
{% endif %}
<article class="panel host-control-panel"> <article class="panel host-control-panel">
<h2>Schedule <a class="button-link secondary compact" href="{% url 'edit_host_schedule' host.host %}">Edit schedule</a></h2> <h2>
Schedule
{% if can_manage_control_panel %}
<a class="button-link secondary compact" href="{% url 'edit_host_schedule' host.host %}">Edit schedule</a>
{% endif %}
</h2>
{% if schedule %} {% if schedule %}
<div class="host-control-meta"> <div class="host-control-meta">
<div><span class="label">Schedule expression</span><strong>{{ schedule.cron_expr }}</strong></div> <div><span class="label">Schedule expression</span><strong>{{ schedule.cron_expr }}</strong></div>
@@ -135,7 +145,9 @@
<p class="muted">Evaluated by the pobsync scheduler service.</p> <p class="muted">Evaluated by the pobsync scheduler service.</p>
{% else %} {% else %}
<p class="muted">No schedule configured.</p> <p class="muted">No schedule configured.</p>
<a class="button-link secondary compact" href="{% url 'edit_host_schedule' host.host %}">Add schedule</a> {% if can_manage_control_panel %}
<a class="button-link secondary compact" href="{% url 'edit_host_schedule' host.host %}">Add schedule</a>
{% endif %}
{% endif %} {% endif %}
</article> </article>
@@ -178,27 +190,35 @@
<div class="metric"> <div class="metric">
<div class="label">Scheduled</div> <div class="label">Scheduled</div>
<div class="value">{{ stats_summary.backup_data.scheduled.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ stats_summary.backup_data.scheduled.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">unique {{ stats_summary.backup_data.scheduled.unique_apparent_size_bytes|filesizeformat }}</div> <div class="muted">
unique {{ stats_summary.backup_data.scheduled.unique_apparent_size_bytes|filesizeformat }}{% if stats_summary.backup_data.scheduled.unknown_count %}; {{ stats_summary.backup_data.scheduled.unknown_count }} not measured{% endif %}
</div>
</div> </div>
<div class="metric"> <div class="metric">
<div class="label">Manual</div> <div class="label">Manual</div>
<div class="value">{{ stats_summary.backup_data.manual.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ stats_summary.backup_data.manual.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">unique {{ stats_summary.backup_data.manual.unique_apparent_size_bytes|filesizeformat }}</div> <div class="muted">
unique {{ stats_summary.backup_data.manual.unique_apparent_size_bytes|filesizeformat }}{% if stats_summary.backup_data.manual.unknown_count %}; {{ stats_summary.backup_data.manual.unknown_count }} not measured{% endif %}
</div>
</div> </div>
<div class="metric"> <div class="metric">
<div class="label">Incomplete</div> <div class="label">Incomplete</div>
<div class="value">{{ stats_summary.backup_data.incomplete.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ stats_summary.backup_data.incomplete.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">measured from disk</div> <div class="muted">
stored metadata{% if stats_summary.backup_data.incomplete.unknown_count %}; {{ stats_summary.backup_data.incomplete.unknown_count }} not measured{% endif %}
</div>
</div> </div>
<div class="metric"> <div class="metric">
<div class="label">Total</div> <div class="label">Total</div>
<div class="value">{{ stats_summary.backup_data.total.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ stats_summary.backup_data.total.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">unique {{ stats_summary.backup_data.total.unique_apparent_size_bytes|filesizeformat }}</div> <div class="muted">
unique {{ stats_summary.backup_data.total.unique_apparent_size_bytes|filesizeformat }}{% if stats_summary.backup_data.total.unknown_count %}; {{ stats_summary.backup_data.total.unknown_count }} not measured{% endif %}
</div>
</div> </div>
</section> </section>
<p class="muted"> <p class="muted">
Main totals use allocated snapshot size. Unique values estimate non-hardlinked visible data; incomplete Main totals use stored snapshot metadata. Unique values estimate non-hardlinked visible data; snapshots without
snapshots are measured from disk because their metadata can be stale. recorded storage metadata are shown as not measured until a backup or metrics refresh records them.
</p> </p>
</section> </section>
@@ -251,8 +271,9 @@
</section> </section>
{% endif %} {% endif %}
<section class="panel"> {% if can_manage_control_panel %}
<h2>Host Check</h2> <section class="panel">
<h2>Host Check</h2>
<section class="grid" aria-label="Host check summary"> <section class="grid" aria-label="Host check summary">
<div class="metric"><div class="label">OK</div><div class="value">{{ host_check_summary.ok }}</div></div> <div class="metric"><div class="label">OK</div><div class="value">{{ host_check_summary.ok }}</div></div>
<div class="metric"><div class="label">Warnings</div><div class="value">{{ host_check_summary.warning }}</div></div> <div class="metric"><div class="label">Warnings</div><div class="value">{{ host_check_summary.warning }}</div></div>
@@ -278,26 +299,32 @@
</article> </article>
{% endfor %} {% endfor %}
</div> </div>
</section> </section>
{% endif %}
<div class="panel-grid"> <div class="panel-grid">
<section class="panel"> <section class="panel">
<h2>Configuration</h2> <h2>Configuration</h2>
<div class="host-control-meta"> <div class="host-control-meta">
<div><span class="label">Address</span><strong>{{ host.address }}</strong></div> <div><span class="label">Address</span><strong>{{ host.address }}</strong></div>
<div><span class="label">SSH key</span><strong>{{ host.ssh_credential|default:"global default" }}</strong></div> {% if can_manage_control_panel %}
<div><span class="label">SSH</span><strong>{{ host.ssh_user|default:"global" }}{% if host.ssh_port %}:{{ host.ssh_port }}{% endif %}</strong></div> <div><span class="label">SSH key</span><strong>{{ host.ssh_credential|default:"global default" }}</strong></div>
<div><span class="label">SSH</span><strong>{{ host.ssh_user|default:"global" }}{% if host.ssh_port %}:{{ host.ssh_port }}{% endif %}</strong></div>
{% endif %}
<div><span class="label">Backup source</span><strong>{{ host.source_root|default:"global default" }}</strong></div> <div><span class="label">Backup source</span><strong>{{ host.source_root|default:"global default" }}</strong></div>
<div><span class="label">Retention</span><strong>d{{ host.retention_daily }} w{{ host.retention_weekly }} m{{ host.retention_monthly }} y{{ host.retention_yearly }}</strong></div> <div><span class="label">Retention</span><strong>d{{ host.retention_daily }} w{{ host.retention_weekly }} m{{ host.retention_monthly }} y{{ host.retention_yearly }}</strong></div>
</div> </div>
<div class="actions inline"> {% if can_manage_control_panel %}
<a class="button-link secondary compact" href="{% url 'edit_host_config' host.host %}">Edit config</a> <div class="actions inline">
<a class="button-link secondary compact" href="{% url 'host_retention_plan' host.host %}">Plan retention</a> <a class="button-link secondary compact" href="{% url 'edit_host_config' host.host %}">Edit config</a>
</div> <a class="button-link secondary compact" href="{% url 'host_retention_plan' host.host %}">Plan retention</a>
</div>
{% endif %}
</section> </section>
<section class="panel"> {% if can_manage_control_panel %}
<h2>Connection Preflight &amp; SSH</h2> <section class="panel">
<h2>Connection Preflight &amp; SSH</h2>
{% if last_preflight %} {% if last_preflight %}
<div class="host-control-meta"> <div class="host-control-meta">
<div> <div>
@@ -340,7 +367,8 @@
{% endfor %} {% endfor %}
</div> </div>
{% endif %} {% endif %}
</section> </section>
{% endif %}
<section class="panel"> <section class="panel">
<h2>Snapshot Storage</h2> <h2>Snapshot Storage</h2>
@@ -359,16 +387,18 @@
</div> </div>
{% endif %} {% endif %}
</div> </div>
<div class="actions inline"> {% if can_manage_control_panel %}
<form method="post" action="{% url 'discover_host_snapshots' host.host %}"> <div class="actions inline">
{% csrf_token %} <form method="post" action="{% url 'discover_host_snapshots' host.host %}">
<button type="submit" class="secondary compact">Discover snapshots</button> {% csrf_token %}
</form> <button type="submit" class="secondary compact">Discover snapshots</button>
<form method="post" action="{% url 'prepare_host_directories' host.host %}"> </form>
{% csrf_token %} <form method="post" action="{% url 'prepare_host_directories' host.host %}">
<button type="submit" class="secondary compact">Prepare directories</button> {% csrf_token %}
</form> <button type="submit" class="secondary compact">Prepare directories</button>
</div> </form>
</div>
{% endif %}
</section> </section>
</div> </div>
@@ -450,8 +480,9 @@
</section> </section>
{% endif %} {% endif %}
<section class="panel"> {% if can_manage_control_panel %}
<h2>Backup Options</h2> <section class="panel">
<h2>Backup Options</h2>
<p class="muted">Use this when the quick actions above need a custom label, include/exclude override, or prune limit.</p> <p class="muted">Use this when the quick actions above need a custom label, include/exclude override, or prune limit.</p>
<form method="post" action="{% url 'queue_manual_backup' host.host %}" class="form-grid"> <form method="post" action="{% url 'queue_manual_backup' host.host %}" class="form-grid">
{% csrf_token %} {% csrf_token %}
@@ -470,7 +501,8 @@
<button type="submit" {% if not can_queue_dry_run and not can_queue_real_backup %}disabled{% endif %}>Queue with options</button> <button type="submit" {% if not can_queue_dry_run and not can_queue_real_backup %}disabled{% endif %}>Queue with options</button>
</div> </div>
</form> </form>
</section> </section>
{% endif %}
<section class="panel"> <section class="panel">
<h2>Latest Runs <a class="button-link secondary compact" href="{% url 'runs_list' %}?host={{ host.host }}">View all</a></h2> <h2>Latest Runs <a class="button-link secondary compact" href="{% url 'runs_list' %}?host={{ host.host }}">View all</a></h2>

View File

@@ -9,9 +9,11 @@
<h1>Hosts</h1> <h1>Hosts</h1>
<div class="page-subtitle">Configured backup targets, schedules, retention state, and host-level controls.</div> <div class="page-subtitle">Configured backup targets, schedules, retention state, and host-level controls.</div>
</div> </div>
<section class="actions" aria-label="Host actions"> {% if can_manage_control_panel %}
<a class="button-link" href="{% url 'create_host_config' %}">New host</a> <section class="actions" aria-label="Host actions">
</section> <a class="button-link" href="{% url 'create_host_config' %}">New host</a>
</section>
{% endif %}
</header> </header>
<section class="grid dashboard-summary-grid" aria-label="Host summary"> <section class="grid dashboard-summary-grid" aria-label="Host summary">

View File

@@ -105,22 +105,30 @@
<div class="host-card-stat"> <div class="host-card-stat">
<div class="label">Scheduled data</div> <div class="label">Scheduled data</div>
<div class="value">{{ host.stats_summary.backup_data.scheduled.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ host.stats_summary.backup_data.scheduled.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">unique {{ host.stats_summary.backup_data.scheduled.unique_apparent_size_bytes|filesizeformat }}</div> <div class="muted">
unique {{ host.stats_summary.backup_data.scheduled.unique_apparent_size_bytes|filesizeformat }}{% if host.stats_summary.backup_data.scheduled.unknown_count %}; {{ host.stats_summary.backup_data.scheduled.unknown_count }} not measured{% endif %}
</div>
</div> </div>
<div class="host-card-stat"> <div class="host-card-stat">
<div class="label">Manual data</div> <div class="label">Manual data</div>
<div class="value">{{ host.stats_summary.backup_data.manual.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ host.stats_summary.backup_data.manual.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">unique {{ host.stats_summary.backup_data.manual.unique_apparent_size_bytes|filesizeformat }}</div> <div class="muted">
unique {{ host.stats_summary.backup_data.manual.unique_apparent_size_bytes|filesizeformat }}{% if host.stats_summary.backup_data.manual.unknown_count %}; {{ host.stats_summary.backup_data.manual.unknown_count }} not measured{% endif %}
</div>
</div> </div>
<div class="host-card-stat"> <div class="host-card-stat">
<div class="label">Incomplete data</div> <div class="label">Incomplete data</div>
<div class="value">{{ host.stats_summary.backup_data.incomplete.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ host.stats_summary.backup_data.incomplete.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">measured from disk</div> <div class="muted">
stored metadata{% if host.stats_summary.backup_data.incomplete.unknown_count %}; {{ host.stats_summary.backup_data.incomplete.unknown_count }} not measured{% endif %}
</div>
</div> </div>
<div class="host-card-stat"> <div class="host-card-stat">
<div class="label">Total data</div> <div class="label">Total data</div>
<div class="value">{{ host.stats_summary.backup_data.total.allocated_size_bytes|filesizeformat }}</div> <div class="value">{{ host.stats_summary.backup_data.total.allocated_size_bytes|filesizeformat }}</div>
<div class="muted">unique {{ host.stats_summary.backup_data.total.unique_apparent_size_bytes|filesizeformat }}</div> <div class="muted">
unique {{ host.stats_summary.backup_data.total.unique_apparent_size_bytes|filesizeformat }}{% if host.stats_summary.backup_data.total.unknown_count %}; {{ host.stats_summary.backup_data.total.unknown_count }} not measured{% endif %}
</div>
</div> </div>
</div> </div>
</div> </div>

View File

@@ -130,22 +130,30 @@
<div> <div>
<span class="label">Scheduled data</span> <span class="label">Scheduled data</span>
<strong>{{ stats_summary.backup_data.scheduled.allocated_size_bytes|filesizeformat }}</strong> <strong>{{ stats_summary.backup_data.scheduled.allocated_size_bytes|filesizeformat }}</strong>
<span class="muted">unique {{ stats_summary.backup_data.scheduled.unique_apparent_size_bytes|filesizeformat }}</span> <span class="muted">
unique {{ stats_summary.backup_data.scheduled.unique_apparent_size_bytes|filesizeformat }}{% if stats_summary.backup_data.scheduled.unknown_count %}; {{ stats_summary.backup_data.scheduled.unknown_count }} not measured{% endif %}
</span>
</div> </div>
<div> <div>
<span class="label">Manual data</span> <span class="label">Manual data</span>
<strong>{{ stats_summary.backup_data.manual.allocated_size_bytes|filesizeformat }}</strong> <strong>{{ stats_summary.backup_data.manual.allocated_size_bytes|filesizeformat }}</strong>
<span class="muted">unique {{ stats_summary.backup_data.manual.unique_apparent_size_bytes|filesizeformat }}</span> <span class="muted">
unique {{ stats_summary.backup_data.manual.unique_apparent_size_bytes|filesizeformat }}{% if stats_summary.backup_data.manual.unknown_count %}; {{ stats_summary.backup_data.manual.unknown_count }} not measured{% endif %}
</span>
</div> </div>
<div> <div>
<span class="label">Incomplete data</span> <span class="label">Incomplete data</span>
<strong>{{ stats_summary.backup_data.incomplete.allocated_size_bytes|filesizeformat }}</strong> <strong>{{ stats_summary.backup_data.incomplete.allocated_size_bytes|filesizeformat }}</strong>
<span class="muted">measured from disk</span> <span class="muted">
stored metadata{% if stats_summary.backup_data.incomplete.unknown_count %}; {{ stats_summary.backup_data.incomplete.unknown_count }} not measured{% endif %}
</span>
</div> </div>
<div> <div>
<span class="label">Total snapshot data</span> <span class="label">Total snapshot data</span>
<strong>{{ stats_summary.backup_data.total.allocated_size_bytes|filesizeformat }}</strong> <strong>{{ stats_summary.backup_data.total.allocated_size_bytes|filesizeformat }}</strong>
<span class="muted">unique {{ stats_summary.backup_data.total.unique_apparent_size_bytes|filesizeformat }}</span> <span class="muted">
unique {{ stats_summary.backup_data.total.unique_apparent_size_bytes|filesizeformat }}{% if stats_summary.backup_data.total.unknown_count %}; {{ stats_summary.backup_data.total.unknown_count }} not measured{% endif %}
</span>
</div> </div>
</div> </div>
</article> </article>

View File

@@ -32,7 +32,7 @@
</section> </section>
{% endif %} {% endif %}
{% if run.status == "failed" or run.status == "warning" %} {% if can_manage_control_panel and run.status == "failed" or can_manage_control_panel and run.status == "warning" %}
{% if not run.reviewed_at %} {% if not run.reviewed_at %}
<section class="panel highlight warning"> <section class="panel highlight warning">
<h2>Review Required</h2> <h2>Review Required</h2>
@@ -78,7 +78,7 @@
{% endif %} {% endif %}
<div> <div>
<strong>Log:</strong> <strong>Log:</strong>
{% if dry_run_summary.log_available %} {% if dry_run_summary.log_available and can_manage_control_panel %}
<a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a> <a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a>
{% elif rsync_log_path %} {% elif rsync_log_path %}
<span class="muted">{{ rsync_log_path }} (missing)</span> <span class="muted">{{ rsync_log_path }} (missing)</span>
@@ -155,7 +155,7 @@
{% if live_progress.log.path %} {% if live_progress.log.path %}
<div> <div>
<strong>Log:</strong> <strong>Log:</strong>
{% if live_progress.log.exists %} {% if live_progress.log.exists and can_manage_control_panel %}
<a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a> <a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a>
{% else %} {% else %}
<span class="muted">{{ live_progress.log.path }} (missing)</span> <span class="muted">{{ live_progress.log.path }} (missing)</span>
@@ -189,7 +189,7 @@
<div><strong>Base:</strong> {{ run.base_path|default:"" }}</div> <div><strong>Base:</strong> {{ run.base_path|default:"" }}</div>
<div> <div>
<strong>Rsync log:</strong> <strong>Rsync log:</strong>
{% if rsync_log_exists %} {% if rsync_log_exists and can_manage_control_panel %}
<a href="{% url 'run_rsync_log' run.id %}">{{ rsync_log_path }}</a> <a href="{% url 'run_rsync_log' run.id %}">{{ rsync_log_path }}</a>
{% elif rsync_log_path %} {% elif rsync_log_path %}
<span class="muted">{{ rsync_log_path }} (missing)</span> <span class="muted">{{ rsync_log_path }} (missing)</span>
@@ -204,7 +204,7 @@
<section class="panel"> <section class="panel">
<h2>Rsync Log</h2> <h2>Rsync Log</h2>
<div class="stack spaced"> <div class="stack spaced">
{% if rsync_log_exists %} {% if rsync_log_exists and can_manage_control_panel %}
<div><a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a></div> <div><a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a></div>
<div class="muted">{{ rsync_log_path }}</div> <div class="muted">{{ rsync_log_path }}</div>
{% elif rsync_log_path %} {% elif rsync_log_path %}

View File

@@ -0,0 +1,122 @@
{% extends "pobsync_backend/base.html" %}
{% block title %}Updater | pobsync{% endblock %}
{% block content %}
<header class="page-header">
<div class="page-title">
<div class="page-kicker">Operations</div>
<h1>Updater</h1>
<div class="page-subtitle">Check Gitea releases, pull the installed git checkout, and run the native systemd updater.</div>
</div>
<section class="actions" aria-label="Updater actions">
<a class="button-link secondary" href="{% url 'dashboard' %}">Back to dashboard</a>
</section>
</header>
<section class="panel-grid">
<section class="panel">
<h2>Installed App</h2>
<dl class="detail-list">
<dt>Version</dt>
<dd>{{ status.installed_version }}</dd>
<dt>Git branch</dt>
<dd>{{ status.git.branch|default:"unknown" }}</dd>
<dt>Git commit</dt>
<dd>{{ status.git.commit|default:"unknown" }}</dd>
<dt>Git describe</dt>
<dd>{{ status.git.describe|default:"unknown" }}</dd>
<dt>App directory</dt>
<dd>{{ status.app_dir }}</dd>
</dl>
</section>
<section class="panel">
<h2>Release Check</h2>
<dl class="detail-list">
<dt>Status</dt>
<dd>
{% if status.update_available == True %}
<span class="status warning">update available</span>
{% elif status.update_available == False %}
<span class="status ok">up to date</span>
{% elif status.release_check_configured %}
<span class="status skipped">not checked</span>
{% else %}
<span class="status skipped">not configured</span>
{% endif %}
</dd>
<dt>Latest release</dt>
<dd>
{% if status.latest_release %}
{% if status.latest_release.html_url %}
<a href="{{ status.latest_release.html_url }}">
{% if status.latest_release.tag_name %}{{ status.latest_release.tag_name }}{% else %}{{ status.latest_release.name }}{% endif %}
</a>
{% else %}
{% if status.latest_release.tag_name %}{{ status.latest_release.tag_name }}{% else %}{{ status.latest_release.name }}{% endif %}
{% endif %}
{% else %}
none
{% endif %}
</dd>
<dt>Release endpoint</dt>
<dd>{% if status.release_check_configured %}configured{% else %}set POBSYNC_UPDATE_RELEASES_URL{% endif %}</dd>
</dl>
{% if status.release_error %}
<p class="status failed">{{ status.release_error }}</p>
{% endif %}
<form method="post" class="actions inline">
{% csrf_token %}
<button type="submit" name="action" value="check_release">Check releases</button>
</form>
</section>
</section>
<section class="panel">
<h2>Update Actions</h2>
<p class="muted">Run these from the installed checkout. The native updater may require a sudoers rule for the pobsync service user.</p>
<dl class="detail-list">
<dt>Git remote</dt>
<dd>{{ status.git_remote }}</dd>
<dt>Update command</dt>
<dd><code>{{ status.update_command }}</code></dd>
</dl>
<div class="actions">
<form method="post" class="inline-form">
{% csrf_token %}
<button class="secondary" type="submit" name="action" value="git_fetch">Fetch releases</button>
</form>
<form method="post" class="inline-form">
{% csrf_token %}
<button class="secondary" type="submit" name="action" value="git_pull">Pull current branch</button>
</form>
<form method="post" class="inline-form">
{% csrf_token %}
<button type="submit" name="action" value="run_update">Run native updater</button>
</form>
</div>
</section>
{% if action_result %}
<section class="panel">
<h2>Last Action Result</h2>
<dl class="detail-list">
<dt>Status</dt>
<dd><span class="status {% if action_result.ok %}ok{% else %}failed{% endif %}">{% if action_result.ok %}ok{% else %}failed{% endif %}</span></dd>
<dt>Exit code</dt>
<dd>{{ action_result.exit_code }}</dd>
<dt>Command</dt>
<dd><code>{{ action_result.command|join:" " }}</code></dd>
</dl>
{% if action_result.stdout %}
<h3>Stdout</h3>
<pre>{{ action_result.stdout }}</pre>
{% endif %}
{% if action_result.stderr %}
<h3>Stderr</h3>
<pre>{{ action_result.stderr }}</pre>
{% endif %}
</section>
{% endif %}
{% endblock %}

View File

@@ -18,6 +18,12 @@ class ApiTests(TestCase):
is_staff=True, is_staff=True,
is_superuser=True, is_superuser=True,
) )
self.readonly_user = user_model.objects.create_user(
username="viewer",
password="secret",
is_staff=False,
is_superuser=False,
)
def test_api_requires_staff_login(self) -> None: def test_api_requires_staff_login(self) -> None:
response = self.client.get("/api/hosts/") response = self.client.get("/api/hosts/")
@@ -25,6 +31,15 @@ class ApiTests(TestCase):
self.assertEqual(response.status_code, 302) self.assertEqual(response.status_code, 302)
self.assertIn("/admin/login/", response["Location"]) self.assertIn("/admin/login/", response["Location"])
def test_readonly_user_can_access_status_endpoint_only(self) -> None:
self.client.force_login(self.readonly_user)
status_response = self.client.get("/api/status/")
hosts_response = self.client.get("/api/hosts/")
self.assertEqual(status_response.status_code, 200)
self.assertEqual(hosts_response.status_code, 403)
def test_hosts_endpoint_returns_counts_and_schedule(self) -> None: def test_hosts_endpoint_returns_counts_and_schedule(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")

View File

@@ -39,13 +39,20 @@ class BackupWorkerTests(TestCase):
}, },
) )
def test_queue_backup_run_can_request_verbose_output(self) -> None: def test_queue_backup_run_enables_verbose_output_by_default(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
run = queue_backup_run(host=host, verbose_output=True) run = queue_backup_run(host=host)
self.assertTrue(run.result["requested"]["verbose_output"]) self.assertTrue(run.result["requested"]["verbose_output"])
def test_queue_backup_run_can_disable_verbose_output(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
run = queue_backup_run(host=host, verbose_output=False)
self.assertFalse(run.result["requested"]["verbose_output"])
def test_worker_executes_next_queued_run(self) -> None: def test_worker_executes_next_queued_run(self) -> None:
with TemporaryDirectory() as tmp: with TemporaryDirectory() as tmp:
backup_root = Path(tmp) / "backups" backup_root = Path(tmp) / "backups"

View File

@@ -5,6 +5,7 @@ from unittest.mock import patch
from django.test import SimpleTestCase from django.test import SimpleTestCase
from pobsync import __version__
from pobsync.cli import main from pobsync.cli import main
@@ -15,7 +16,7 @@ class ConsoleEntrypointTests(SimpleTestCase):
exit_code = main(["--version"]) exit_code = main(["--version"])
self.assertEqual(exit_code, 0) self.assertEqual(exit_code, 0)
self.assertEqual(stdout.getvalue().strip(), "pobsync 1.1.0") self.assertEqual(stdout.getvalue().strip(), f"pobsync {__version__}")
def test_maps_backup_alias_to_django_command(self) -> None: def test_maps_backup_alias_to_django_command(self) -> None:
with patch("pobsync.cli.execute_from_command_line") as execute: with patch("pobsync.cli.execute_from_command_line") as execute:

View File

@@ -39,12 +39,16 @@ class RunBackupRecordsSnapshotTests(TestCase):
"host": host.host, "host": host.host,
"snapshot": str(snapshot_dir), "snapshot": str(snapshot_dir),
"base": None, "base": None,
"verbose_output": True,
"rsync": {"exit_code": 0}, "rsync": {"exit_code": 0},
} }
call_command("run_pobsync_backup", host.host, prefix=str(Path(tmp) / "home"), stdout=StringIO()) call_command("run_pobsync_backup", host.host, prefix=str(Path(tmp) / "home"), stdout=StringIO())
run_scheduled.assert_called_once()
self.assertTrue(run_scheduled.call_args.kwargs["verbose_output"])
self.assertEqual(BackupRun.objects.count(), 1) self.assertEqual(BackupRun.objects.count(), 1)
run = BackupRun.objects.get() run = BackupRun.objects.get()
self.assertTrue(run.result["verbose_output"])
self.assertEqual(SnapshotRecord.objects.count(), 1) self.assertEqual(SnapshotRecord.objects.count(), 1)
record = SnapshotRecord.objects.get() record = SnapshotRecord.objects.get()
self.assertEqual(run.snapshot, record) self.assertEqual(run.snapshot, record)
@@ -52,6 +56,45 @@ class RunBackupRecordsSnapshotTests(TestCase):
self.assertEqual(record.kind, "scheduled") self.assertEqual(record.kind, "scheduled")
self.assertEqual(record.status, "success") self.assertEqual(record.status, "success")
def test_backup_command_can_skip_default_verbose_rsync_output(self) -> None:
with TemporaryDirectory() as tmp:
backup_root = Path(tmp) / "backups"
GlobalConfig.objects.create(name="default", backup_root=str(backup_root))
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
snapshot_dir = backup_root / host.host / "scheduled" / "20260519-021500Z__ABCDEFGH"
meta_dir = snapshot_dir / "meta"
meta_dir.mkdir(parents=True)
write_yaml_atomic(
meta_dir / "meta.yaml",
{
"status": "success",
"started_at": "2026-05-19T02:15:00Z",
"ended_at": "2026-05-19T02:16:00Z",
},
)
with patch("pobsync_backend.backup_runner.run_scheduled") as run_scheduled:
run_scheduled.return_value = {
"ok": True,
"dry_run": False,
"host": host.host,
"snapshot": str(snapshot_dir),
"base": None,
"verbose_output": False,
"rsync": {"exit_code": 0},
}
call_command(
"run_pobsync_backup",
host.host,
prefix=str(Path(tmp) / "home"),
quiet_rsync=True,
stdout=StringIO(),
)
run_scheduled.assert_called_once()
self.assertFalse(run_scheduled.call_args.kwargs["verbose_output"])
self.assertFalse(BackupRun.objects.get().result["verbose_output"])
def test_prune_uses_sql_retention_after_snapshot_record_is_created(self) -> None: def test_prune_uses_sql_retention_after_snapshot_record_is_created(self) -> None:
with TemporaryDirectory() as tmp: with TemporaryDirectory() as tmp:
backup_root = Path(tmp) / "backups" backup_root = Path(tmp) / "backups"

View File

@@ -6,7 +6,7 @@ from unittest.mock import patch
from django.test import SimpleTestCase from django.test import SimpleTestCase
from pobsync.commands.run_scheduled import run_scheduled from pobsync.commands.run_scheduled import SNAPSHOT_STORAGE_SCAN_MAX_ENTRIES, run_scheduled
from pobsync.errors import ConfigError from pobsync.errors import ConfigError
from pobsync.rsync import RsyncResult from pobsync.rsync import RsyncResult
@@ -270,9 +270,12 @@ class RunScheduledConfigSourceTests(SimpleTestCase):
self.assertEqual(result["stats"]["rsync"]["files_transferred"], 2) self.assertEqual(result["stats"]["rsync"]["files_transferred"], 2)
self.assertEqual(result["stats"]["rsync"]["link_dest_estimated_savings_bytes"], 1500) self.assertEqual(result["stats"]["rsync"]["link_dest_estimated_savings_bytes"], 1500)
self.assertIn("snapshot", result["stats"]["storage"]) self.assertIn("snapshot", result["stats"]["storage"])
self.assertEqual(result["stats"]["storage"]["snapshot"]["max_entries"], SNAPSHOT_STORAGE_SCAN_MAX_ENTRIES)
self.assertFalse(result["stats"]["storage"]["snapshot"]["scan_limited"])
self.assertIn("capacity", result["stats"]["storage"]) self.assertIn("capacity", result["stats"]["storage"])
self.assertIn("stats:", meta_text) self.assertIn("stats:", meta_text)
self.assertIn("files_total: 10", meta_text) self.assertIn("files_total: 10", meta_text)
self.assertIn(f"max_entries: {SNAPSHOT_STORAGE_SCAN_MAX_ENTRIES}", meta_text)
def test_real_run_reports_running_state_callback_before_rsync_returns(self) -> None: def test_real_run_reports_running_state_callback_before_rsync_returns(self) -> None:
states = [] states = []

View File

@@ -6,7 +6,7 @@ from tempfile import TemporaryDirectory
from django.test import SimpleTestCase from django.test import SimpleTestCase
from pobsync.run_stats import parse_rsync_stats, tree_usage from pobsync.run_stats import collect_storage_stats, parse_rsync_stats, tree_usage
class RunStatsTests(SimpleTestCase): class RunStatsTests(SimpleTestCase):
@@ -58,3 +58,37 @@ total size is 1.50M speedup is 125.00
self.assertEqual(stats["hardlinked_files"], 2) self.assertEqual(stats["hardlinked_files"], 2)
self.assertEqual(stats["hardlinked_apparent_size_bytes"], 6) self.assertEqual(stats["hardlinked_apparent_size_bytes"], 6)
self.assertEqual(stats["hardlink_apparent_ratio"], 1.0) self.assertEqual(stats["hardlink_apparent_ratio"], 1.0)
self.assertFalse(stats["scan_limited"])
def test_tree_usage_can_limit_large_scans(self) -> None:
with TemporaryDirectory() as tmp:
root = Path(tmp)
for index in range(5):
(root / f"file-{index}").write_bytes(b"x")
stats = tree_usage(root, max_entries=2)
self.assertEqual(stats["files"], 2)
self.assertEqual(stats["entries_scanned"], 2)
self.assertEqual(stats["max_entries"], 2)
self.assertTrue(stats["scan_limited"])
self.assertEqual(stats["apparent_size_bytes"], 2)
def test_collect_storage_stats_marks_limited_snapshot_scan(self) -> None:
with TemporaryDirectory() as tmp:
root = Path(tmp)
snapshot = root / "snapshot"
snapshot.mkdir()
for index in range(4):
(snapshot / f"file-{index}").write_bytes(b"x")
stats = collect_storage_stats(
backup_root=root,
snapshot_dir=snapshot,
snapshot_max_entries=1,
)
self.assertEqual(stats["snapshot"]["files"], 1)
self.assertEqual(stats["snapshot"]["entries_scanned"], 1)
self.assertEqual(stats["snapshot"]["max_entries"], 1)
self.assertTrue(stats["snapshot"]["scan_limited"])

View File

@@ -1,12 +1,10 @@
from __future__ import annotations from __future__ import annotations
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from unittest.mock import patch
from tempfile import TemporaryDirectory
from django.test import TestCase from django.test import TestCase
from pobsync.run_stats import tree_usage
from pobsync_backend.models import HostConfig, SnapshotRecord from pobsync_backend.models import HostConfig, SnapshotRecord
from pobsync_backend.stats_summary import collect_dashboard_stats, collect_host_stats from pobsync_backend.stats_summary import collect_dashboard_stats, collect_host_stats
@@ -18,114 +16,109 @@ class StatsSummaryTests(TestCase):
self._snapshot(web, "20260519-021500Z__SCHED01", SnapshotRecord.Kind.SCHEDULED, allocated=100) self._snapshot(web, "20260519-021500Z__SCHED01", SnapshotRecord.Kind.SCHEDULED, allocated=100)
self._snapshot(web, "20260519-031500Z__MANUAL1", SnapshotRecord.Kind.MANUAL, allocated=200) self._snapshot(web, "20260519-031500Z__MANUAL1", SnapshotRecord.Kind.MANUAL, allocated=200)
self._snapshot(db, "20260519-041500Z__SCHED02", SnapshotRecord.Kind.SCHEDULED, allocated=300) self._snapshot(db, "20260519-041500Z__SCHED02", SnapshotRecord.Kind.SCHEDULED, allocated=300)
with TemporaryDirectory() as tmp: self._snapshot(db, "20260519-051500Z__BROKEN1", SnapshotRecord.Kind.INCOMPLETE, allocated=400)
incomplete_usage = self._incomplete_snapshot_on_disk(
db,
Path(tmp),
"20260519-051500Z__BROKEN1",
)
stats = collect_dashboard_stats(hosts=[web, db], global_config=None) stats = collect_dashboard_stats(hosts=[web, db], global_config=None)
self.assertEqual(stats["backup_data"]["scheduled"]["count"], 2) self.assertEqual(stats["backup_data"]["scheduled"]["count"], 2)
self.assertEqual(stats["backup_data"]["scheduled"]["allocated_size_bytes"], 400) self.assertEqual(stats["backup_data"]["scheduled"]["allocated_size_bytes"], 400)
self.assertEqual(stats["backup_data"]["manual"]["allocated_size_bytes"], 200) self.assertEqual(stats["backup_data"]["manual"]["allocated_size_bytes"], 200)
self.assertEqual(stats["backup_data"]["incomplete"]["allocated_size_bytes"], incomplete_usage["allocated_size_bytes"]) self.assertEqual(stats["backup_data"]["incomplete"]["allocated_size_bytes"], 400)
self.assertEqual(stats["backup_data"]["total"]["count"], 4) self.assertEqual(stats["backup_data"]["total"]["count"], 4)
self.assertEqual(stats["backup_data"]["total"]["allocated_size_bytes"], 600 + incomplete_usage["allocated_size_bytes"]) self.assertEqual(stats["backup_data"]["total"]["measured_count"], 4)
self.assertEqual(stats["backup_data"]["total"]["unique_apparent_size_bytes"], 1200 + incomplete_usage["apparent_size_bytes"]) self.assertEqual(stats["backup_data"]["total"]["unknown_count"], 0)
self.assertEqual(stats["backup_data"]["total"]["allocated_size_bytes"], 1000)
self.assertEqual(stats["backup_data"]["total"]["unique_apparent_size_bytes"], 2000)
def test_collect_host_stats_sums_backup_data_by_snapshot_kind(self) -> None: def test_collect_host_stats_sums_backup_data_by_snapshot_kind(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
self._snapshot(host, "20260519-021500Z__SCHED01", SnapshotRecord.Kind.SCHEDULED, allocated=100) self._snapshot(host, "20260519-021500Z__SCHED01", SnapshotRecord.Kind.SCHEDULED, allocated=100)
self._snapshot(host, "20260519-031500Z__SCHED02", SnapshotRecord.Kind.SCHEDULED, allocated=200) self._snapshot(host, "20260519-031500Z__SCHED02", SnapshotRecord.Kind.SCHEDULED, allocated=200)
self._snapshot(host, "20260519-041500Z__MANUAL1", SnapshotRecord.Kind.MANUAL, allocated=300) self._snapshot(host, "20260519-041500Z__MANUAL1", SnapshotRecord.Kind.MANUAL, allocated=300)
with TemporaryDirectory() as tmp: self._snapshot(host, "20260519-051500Z__BROKEN1", SnapshotRecord.Kind.INCOMPLETE, allocated=400)
incomplete_usage = self._incomplete_snapshot_on_disk(
host,
Path(tmp),
"20260519-051500Z__BROKEN1",
)
stats = collect_host_stats(host=host) stats = collect_host_stats(host=host)
self.assertEqual(stats["backup_data"]["scheduled"]["count"], 2) self.assertEqual(stats["backup_data"]["scheduled"]["count"], 2)
self.assertEqual(stats["backup_data"]["scheduled"]["allocated_size_bytes"], 300) self.assertEqual(stats["backup_data"]["scheduled"]["allocated_size_bytes"], 300)
self.assertEqual(stats["backup_data"]["manual"]["allocated_size_bytes"], 300) self.assertEqual(stats["backup_data"]["manual"]["allocated_size_bytes"], 300)
self.assertEqual(stats["backup_data"]["incomplete"]["allocated_size_bytes"], incomplete_usage["allocated_size_bytes"]) self.assertEqual(stats["backup_data"]["incomplete"]["allocated_size_bytes"], 400)
self.assertEqual(stats["backup_data"]["total"]["count"], 4) self.assertEqual(stats["backup_data"]["total"]["count"], 4)
self.assertEqual(stats["backup_data"]["total"]["allocated_size_bytes"], 600 + incomplete_usage["allocated_size_bytes"]) self.assertEqual(stats["backup_data"]["total"]["measured_count"], 4)
self.assertEqual(stats["backup_data"]["total"]["unique_apparent_size_bytes"], 1200 + incomplete_usage["apparent_size_bytes"]) self.assertEqual(stats["backup_data"]["total"]["unknown_count"], 0)
self.assertEqual(stats["backup_data"]["total"]["allocated_size_bytes"], 1000)
self.assertEqual(stats["backup_data"]["total"]["unique_apparent_size_bytes"], 2000)
def test_collect_host_stats_falls_back_to_filesystem_usage_for_snapshots_without_metadata(self) -> None: def test_collect_host_stats_marks_snapshots_without_storage_metadata_unknown(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
with TemporaryDirectory() as tmp: SnapshotRecord.objects.create(
incomplete_dir = Path(tmp) / host.host / ".incomplete" / "20260519-051500Z__BROKEN1" host=host,
data_dir = incomplete_dir / "data" kind=SnapshotRecord.Kind.INCOMPLETE,
meta_dir = incomplete_dir / "meta" dirname="20260519-051500Z__BROKEN1",
data_dir.mkdir(parents=True) path="/backups/web-01/.incomplete/20260519-051500Z__BROKEN1",
meta_dir.mkdir() status="failed",
data_dir.joinpath("partial-file").write_text("interrupted backup data\n", encoding="utf-8") metadata={},
meta_dir.joinpath("rsync.log").write_text("not part of the backup data total\n", encoding="utf-8") )
expected_usage = tree_usage(data_dir)
SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=incomplete_dir.name,
path=str(incomplete_dir),
status="failed",
metadata={},
)
with patch("pobsync_backend.stats_summary.tree_usage", create=True) as tree_usage:
stats = collect_host_stats(host=host) stats = collect_host_stats(host=host)
tree_usage.assert_not_called()
self.assertEqual(stats["backup_data"]["incomplete"]["count"], 1) self.assertEqual(stats["backup_data"]["incomplete"]["count"], 1)
self.assertEqual( self.assertEqual(stats["backup_data"]["incomplete"]["measured_count"], 0)
stats["backup_data"]["incomplete"]["allocated_size_bytes"], self.assertEqual(stats["backup_data"]["incomplete"]["unknown_count"], 1)
expected_usage["allocated_size_bytes"], self.assertEqual(stats["backup_data"]["incomplete"]["allocated_size_bytes"], 0)
) self.assertEqual(stats["backup_data"]["incomplete"]["apparent_size_bytes"], 0)
self.assertEqual( self.assertEqual(stats["backup_data"]["total"]["unknown_count"], 1)
stats["backup_data"]["incomplete"]["apparent_size_bytes"],
expected_usage["apparent_size_bytes"],
)
self.assertEqual(
stats["backup_data"]["total"]["allocated_size_bytes"],
expected_usage["allocated_size_bytes"],
)
def test_collect_host_stats_measures_incomplete_data_from_disk_even_with_stale_metadata(self) -> None: def test_collect_host_stats_uses_recorded_zero_storage_without_rescanning(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
with TemporaryDirectory() as tmp: SnapshotRecord.objects.create(
incomplete_dir = Path(tmp) / host.host / ".incomplete" / "20260519-051500Z__BROKEN1" host=host,
data_dir = incomplete_dir / "data" kind=SnapshotRecord.Kind.INCOMPLETE,
data_dir.mkdir(parents=True) dirname="20260519-051500Z__BROKEN1",
data_dir.joinpath("partial-file").write_text("interrupted backup data\n", encoding="utf-8") path="/backups/web-01/.incomplete/20260519-051500Z__BROKEN1",
expected_usage = tree_usage(data_dir) status="failed",
SnapshotRecord.objects.create( metadata={
host=host, "stats": {
kind=SnapshotRecord.Kind.INCOMPLETE, "storage": {
dirname=incomplete_dir.name, "snapshot": {
path=str(incomplete_dir), "apparent_size_bytes": 0,
status="failed", "allocated_size_bytes": 0,
metadata={
"stats": {
"storage": {
"snapshot": {
"apparent_size_bytes": 0,
"allocated_size_bytes": 0,
}
} }
} }
}, }
) },
)
with patch("pobsync_backend.stats_summary.tree_usage", create=True) as tree_usage:
stats = collect_host_stats(host=host) stats = collect_host_stats(host=host)
self.assertEqual( tree_usage.assert_not_called()
stats["backup_data"]["incomplete"]["allocated_size_bytes"], self.assertEqual(stats["backup_data"]["incomplete"]["measured_count"], 1)
expected_usage["allocated_size_bytes"], self.assertEqual(stats["backup_data"]["incomplete"]["unknown_count"], 0)
self.assertEqual(stats["backup_data"]["incomplete"]["allocated_size_bytes"], 0)
self.assertEqual(stats["backup_data"]["incomplete"]["apparent_size_bytes"], 0)
def test_collect_dashboard_stats_does_not_scan_filesystem_for_missing_snapshot_metadata(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.SCHEDULED,
dirname="20260519-051500Z__SCHED01",
path="/backups/web-01/scheduled/20260519-051500Z__SCHED01",
status="success",
metadata={},
) )
self.assertGreater(stats["backup_data"]["incomplete"]["apparent_size_bytes"], 0)
with patch("pobsync_backend.stats_summary.tree_usage", create=True) as tree_usage:
stats = collect_dashboard_stats(hosts=[host], global_config=None)
tree_usage.assert_not_called()
self.assertEqual(stats["backup_data"]["scheduled"]["count"], 1)
self.assertEqual(stats["backup_data"]["scheduled"]["measured_count"], 0)
self.assertEqual(stats["backup_data"]["scheduled"]["unknown_count"], 1)
self.assertEqual(stats["backup_data"]["scheduled"]["allocated_size_bytes"], 0)
def test_collect_host_stats_reports_non_hardlinked_snapshot_data(self) -> None: def test_collect_host_stats_reports_non_hardlinked_snapshot_data(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
@@ -147,21 +140,6 @@ class StatsSummaryTests(TestCase):
def _snapshot(self, host: HostConfig, dirname: str, kind: str, *, allocated: int) -> SnapshotRecord: def _snapshot(self, host: HostConfig, dirname: str, kind: str, *, allocated: int) -> SnapshotRecord:
return self._snapshot_with_sizes(host, dirname, kind, allocated=allocated) return self._snapshot_with_sizes(host, dirname, kind, allocated=allocated)
def _incomplete_snapshot_on_disk(self, host: HostConfig, root: Path, dirname: str) -> dict:
incomplete_dir = root / host.host / ".incomplete" / dirname
data_dir = incomplete_dir / "data"
data_dir.mkdir(parents=True)
data_dir.joinpath("partial-file").write_text("interrupted backup data\n", encoding="utf-8")
usage = tree_usage(data_dir)
SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=dirname,
path=str(incomplete_dir),
status="failed",
)
return usage
def _snapshot_with_sizes( def _snapshot_with_sizes(
self, self,
host: HostConfig, host: HostConfig,

View File

@@ -0,0 +1,127 @@
from __future__ import annotations
import json
from io import StringIO
from pathlib import Path
from tempfile import TemporaryDirectory
from django.core.management import call_command
from django.test import TestCase
from pobsync_backend.models import HostConfig, SnapshotRecord
from pobsync_backend.storage_metrics import refresh_snapshot_storage_metric, refresh_snapshot_storage_metrics
class StorageMetricsTests(TestCase):
def test_refresh_snapshot_storage_metric_updates_snapshot_metadata(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
with TemporaryDirectory() as tmp:
snapshot = self._snapshot_with_file(host, Path(tmp), "scheduled", "payload.txt", b"payload")
result = refresh_snapshot_storage_metric(snapshot, max_entries=100)
snapshot.refresh_from_db()
storage = snapshot.metadata["stats"]["storage"]["snapshot"]
self.assertEqual(result["status"], "updated")
self.assertEqual(storage["files"], 1)
self.assertEqual(storage["apparent_size_bytes"], 7)
self.assertEqual(storage["max_entries"], 100)
self.assertFalse(storage["scan_limited"])
self.assertEqual(storage["measurement_source"], "manual_refresh")
self.assertIn("measured_at", storage)
def test_refresh_snapshot_storage_metric_dry_run_does_not_write_metadata(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
with TemporaryDirectory() as tmp:
snapshot = self._snapshot_with_file(host, Path(tmp), "scheduled", "payload.txt", b"payload")
result = refresh_snapshot_storage_metric(snapshot, max_entries=100, dry_run=True)
snapshot.refresh_from_db()
self.assertEqual(result["status"], "skipped")
self.assertEqual(snapshot.metadata, {})
def test_refresh_snapshot_storage_metrics_filters_by_host_and_kind(self) -> None:
web = HostConfig.objects.create(host="web-01", address="web-01.example.test")
db = HostConfig.objects.create(host="db-01", address="db-01.example.test")
with TemporaryDirectory() as tmp:
root = Path(tmp)
target = self._snapshot_with_file(web, root, "scheduled", "target.txt", b"target")
other_kind = self._snapshot_with_file(web, root, "manual", "manual.txt", b"manual")
other_host = self._snapshot_with_file(db, root, "scheduled", "db.txt", b"db")
result = refresh_snapshot_storage_metrics(host=web, kind=SnapshotRecord.Kind.SCHEDULED, max_entries=100)
self.assertEqual(result.scanned, 1)
self.assertEqual(result.updated, 1)
target.refresh_from_db()
other_kind.refresh_from_db()
other_host.refresh_from_db()
self.assertIn("stats", target.metadata)
self.assertEqual(other_kind.metadata, {})
self.assertEqual(other_host.metadata, {})
def test_refresh_snapshot_storage_metrics_records_missing_paths(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
snapshot = SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.SCHEDULED,
dirname="20260608-100000Z__MISSING",
path="/missing/pobsync/snapshot",
status="success",
metadata={},
)
result = refresh_snapshot_storage_metrics(host=host, max_entries=100)
snapshot.refresh_from_db()
self.assertEqual(result.scanned, 1)
self.assertEqual(result.missing, 1)
error = snapshot.metadata["stats"]["storage"]["snapshot_measurement_error"]
self.assertEqual(error["reason"], "missing_path")
self.assertEqual(error["measurement_source"], "manual_refresh")
def test_refresh_command_outputs_counts(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
with TemporaryDirectory() as tmp:
self._snapshot_with_file(host, Path(tmp), "scheduled", "payload.txt", b"payload")
output = StringIO()
call_command(
"refresh_pobsync_storage_metrics",
"--host",
host.host,
"--kind",
"scheduled",
"--max-entries",
"100",
stdout=output,
)
result = json.loads(output.getvalue())
self.assertEqual(result["scanned"], 1)
self.assertEqual(result["updated"], 1)
self.assertFalse(result["dry_run"])
def _snapshot_with_file(
self,
host: HostConfig,
root: Path,
kind: str,
filename: str,
content: bytes,
) -> SnapshotRecord:
dirname = f"20260608-100000Z__{host.host.replace('-', '').upper()}{kind[:3].upper()}"
parent = ".incomplete" if kind == SnapshotRecord.Kind.INCOMPLETE else kind
snapshot_dir = root / host.host / parent / dirname
data_dir = snapshot_dir / "data"
data_dir.mkdir(parents=True)
(data_dir / filename).write_bytes(content)
return SnapshotRecord.objects.create(
host=host,
kind=kind,
dirname=dirname,
path=str(snapshot_dir),
status="success",
metadata={},
)

View File

@@ -0,0 +1,72 @@
from __future__ import annotations
import json
from unittest.mock import MagicMock, patch
from django.test import SimpleTestCase, override_settings
from pobsync_backend import updater
class UpdaterTests(SimpleTestCase):
@override_settings(
POBSYNC_UPDATE_RELEASES_URL="https://code.example.test/api/v1/repos/owner/pobsync/releases",
POBSYNC_UPDATE_RELEASES_TOKEN="secret",
)
def test_fetch_latest_release_reads_first_gitea_release(self) -> None:
response = MagicMock()
response.__enter__.return_value.read.return_value = json.dumps(
[
{
"tag_name": "v1.2.0",
"name": "1.2.0",
"html_url": "https://code.example.test/releases/v1.2.0",
}
]
).encode("utf-8")
with patch("pobsync_backend.updater.urlopen", return_value=response) as urlopen:
release = updater.fetch_latest_release()
self.assertEqual(release["tag_name"], "v1.2.0")
request = urlopen.call_args.args[0]
self.assertEqual(request.full_url, "https://code.example.test/api/v1/repos/owner/pobsync/releases")
self.assertEqual(request.headers["Authorization"], "token secret")
@override_settings(POBSYNC_UPDATE_RELEASES_URL="")
def test_collect_update_status_reports_unconfigured_release_check(self) -> None:
with patch("pobsync_backend.updater._git_status", return_value={"branch": "master"}):
status = updater.collect_update_status(check_release=True)
self.assertFalse(status["release_check_configured"])
self.assertEqual(status["release_error"], "POBSYNC_UPDATE_RELEASES_URL is not configured.")
self.assertIsNone(status["update_available"])
@override_settings(POBSYNC_UPDATE_GIT_REMOTE="upstream")
def test_run_git_fetch_uses_configured_remote(self) -> None:
completed = MagicMock(returncode=0, stdout="ok", stderr="")
with patch("pobsync_backend.updater.subprocess.run", return_value=completed) as run:
result = updater.run_git_fetch()
self.assertTrue(result.ok)
self.assertEqual(result.command, ["git", "fetch", "--tags", "--prune", "upstream"])
run.assert_called_once()
@override_settings(POBSYNC_UPDATE_GIT_REMOTE="origin")
def test_run_git_pull_rejects_detached_checkout(self) -> None:
with patch("pobsync_backend.updater._git_current_branch", return_value=""):
result = updater.run_git_pull()
self.assertFalse(result.ok)
self.assertEqual(result.exit_code, 2)
self.assertIn("not on a branch", result.stderr)
@override_settings(POBSYNC_UPDATE_COMMAND="sudo -n scripts/update-systemd --verbose")
def test_run_native_update_splits_configured_command(self) -> None:
completed = MagicMock(returncode=1, stdout="", stderr="sudo failed")
with patch("pobsync_backend.updater.subprocess.run", return_value=completed):
result = updater.run_native_update()
self.assertFalse(result.ok)
self.assertEqual(result.command, ["sudo", "-n", "scripts/update-systemd", "--verbose"])
self.assertEqual(result.stderr, "sudo failed")

View File

@@ -8,11 +8,9 @@ from unittest.mock import patch
from django.contrib.auth import get_user_model from django.contrib.auth import get_user_model
from django.core.files.uploadedfile import SimpleUploadedFile from django.core.files.uploadedfile import SimpleUploadedFile
from django.template.defaultfilters import filesizeformat
from django.test import TestCase, override_settings from django.test import TestCase, override_settings
from django.urls import reverse from django.urls import reverse
from pobsync.run_stats import tree_usage
from pobsync.util import write_yaml_atomic from pobsync.util import write_yaml_atomic
from pobsync_backend.models import ( from pobsync_backend.models import (
BackupRun, BackupRun,
@@ -36,6 +34,12 @@ class ViewTests(TestCase):
is_staff=True, is_staff=True,
is_superuser=True, is_superuser=True,
) )
self.readonly_user = user_model.objects.create_user(
username="viewer",
password="secret",
is_staff=False,
is_superuser=False,
)
def test_dashboard_requires_staff_login(self) -> None: def test_dashboard_requires_staff_login(self) -> None:
response = self.client.get(reverse("dashboard")) response = self.client.get(reverse("dashboard"))
@@ -56,6 +60,7 @@ class ViewTests(TestCase):
self.assertContains(response, reverse("ssh_credentials")) self.assertContains(response, reverse("ssh_credentials"))
self.assertContains(response, reverse("notification_targets")) self.assertContains(response, reverse("notification_targets"))
self.assertContains(response, reverse("logs")) self.assertContains(response, reverse("logs"))
self.assertContains(response, reverse("updater"))
self.assertContains(response, reverse("purged_snapshots")) self.assertContains(response, reverse("purged_snapshots"))
self.assertContains(response, reverse("self_check")) self.assertContains(response, reverse("self_check"))
self.assertContains(response, reverse("changelog")) self.assertContains(response, reverse("changelog"))
@@ -63,6 +68,23 @@ class ViewTests(TestCase):
self.assertContains(response, reverse("admin:index")) self.assertContains(response, reverse("admin:index"))
self.assertContains(response, '<a href="/" aria-current="page">Dashboard</a>', html=False) self.assertContains(response, '<a href="/" aria-current="page">Dashboard</a>', html=False)
def test_readonly_navigation_hides_admin_and_sensitive_links(self) -> None:
self.client.force_login(self.readonly_user)
response = self.client.get(reverse("dashboard"))
self.assertEqual(response.status_code, 200)
self.assertContains(response, reverse("dashboard"))
self.assertContains(response, reverse("hosts_list"))
self.assertContains(response, reverse("changelog"))
self.assertContains(response, "/api/status/")
self.assertNotContains(response, reverse("ssh_credentials"))
self.assertNotContains(response, reverse("notification_targets"))
self.assertNotContains(response, reverse("logs"))
self.assertNotContains(response, reverse("updater"))
self.assertNotContains(response, reverse("self_check"))
self.assertNotContains(response, reverse("admin:index"))
def test_base_navigation_marks_current_secondary_page(self) -> None: def test_base_navigation_marks_current_secondary_page(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
@@ -71,12 +93,166 @@ class ViewTests(TestCase):
self.assertEqual(response.status_code, 200) self.assertEqual(response.status_code, 200)
self.assertContains(response, f'<a href="{reverse("self_check")}" aria-current="page">Self Check</a>', html=False) self.assertContains(response, f'<a href="{reverse("self_check")}" aria-current="page">Self Check</a>', html=False)
def test_changelog_requires_staff_login(self) -> None: def test_changelog_requires_login(self) -> None:
response = self.client.get(reverse("changelog")) response = self.client.get(reverse("changelog"))
self.assertEqual(response.status_code, 302) self.assertEqual(response.status_code, 302)
self.assertIn("/admin/login/", response["Location"]) self.assertIn("/admin/login/", response["Location"])
def test_readonly_user_can_view_status_pages(self) -> None:
self.client.force_login(self.readonly_user)
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
snapshot = SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.SCHEDULED,
dirname="20260519-021500Z__ABCDEFGH",
path="/backups/web-01/scheduled/20260519-021500Z__ABCDEFGH",
status="success",
)
run = BackupRun.objects.create(host=host, status=BackupRun.Status.SUCCESS, snapshot=snapshot)
urls = [
reverse("dashboard"),
reverse("hosts_list"),
reverse("host_detail", args=[host.host]),
reverse("runs_list"),
reverse("run_detail", args=[run.id]),
reverse("snapshots_list"),
reverse("snapshot_detail", args=[snapshot.id]),
reverse("schedules_list"),
reverse("purged_snapshots"),
]
for url in urls:
with self.subTest(url=url):
response = self.client.get(url)
self.assertEqual(response.status_code, 200)
def test_readonly_user_cannot_access_sensitive_or_mutating_views(self) -> None:
self.client.force_login(self.readonly_user)
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
blocked_urls = [
reverse("ssh_credentials"),
reverse("logs"),
reverse("updater"),
reverse("self_check"),
reverse("edit_global_config"),
reverse("create_host_config"),
reverse("edit_host_config", args=[host.host]),
reverse("edit_host_schedule", args=[host.host]),
]
for url in blocked_urls:
with self.subTest(url=url):
response = self.client.get(url)
self.assertEqual(response.status_code, 403)
def test_updater_view_renders_status(self) -> None:
self.client.force_login(self.staff_user)
with patch("pobsync_backend.views.collect_update_status") as collect_update_status:
collect_update_status.return_value = {
"app_dir": "/opt/pobsync/app",
"installed_version": "1.1.0",
"release_check_configured": True,
"update_command": "sudo -n scripts/update-systemd",
"git_remote": "origin",
"git": {"branch": "master", "commit": "abc1234", "describe": "v1.1.0"},
"latest_release": None,
"release_error": "",
"update_available": None,
}
response = self.client.get(reverse("updater"))
self.assertEqual(response.status_code, 200)
self.assertContains(response, "Updater")
self.assertContains(response, "1.1.0")
self.assertContains(response, "master")
collect_update_status.assert_called_once_with(check_release=False)
def test_updater_check_release_requests_release_status(self) -> None:
self.client.force_login(self.staff_user)
with patch("pobsync_backend.views.collect_update_status") as collect_update_status:
collect_update_status.return_value = {
"app_dir": "/opt/pobsync/app",
"installed_version": "1.1.0",
"release_check_configured": True,
"update_command": "sudo -n scripts/update-systemd",
"git_remote": "origin",
"git": {"branch": "master", "commit": "abc1234", "describe": "v1.1.0"},
"latest_release": {"tag_name": "v1.2.0", "html_url": "https://code.example.test/releases/v1.2.0"},
"release_error": "",
"update_available": True,
}
response = self.client.post(reverse("updater"), {"action": "check_release"})
self.assertEqual(response.status_code, 200)
self.assertContains(response, "update available")
self.assertContains(response, "v1.2.0")
collect_update_status.assert_called_once_with(check_release=True)
def test_updater_git_fetch_runs_action_and_renders_output(self) -> None:
self.client.force_login(self.staff_user)
result = type(
"Result",
(),
{
"ok": True,
"exit_code": 0,
"command": ["git", "fetch", "--tags", "--prune", "origin"],
"stdout": "fetched",
"stderr": "",
},
)()
with patch("pobsync_backend.views.run_git_fetch", return_value=result) as run_git_fetch, patch(
"pobsync_backend.views.collect_update_status"
) as collect_update_status:
collect_update_status.return_value = {
"app_dir": "/opt/pobsync/app",
"installed_version": "1.1.0",
"release_check_configured": False,
"update_command": "sudo -n scripts/update-systemd",
"git_remote": "origin",
"git": {"branch": "master", "commit": "abc1234", "describe": "v1.1.0"},
"latest_release": None,
"release_error": "",
"update_available": None,
}
response = self.client.post(reverse("updater"), {"action": "git_fetch"})
self.assertEqual(response.status_code, 200)
self.assertContains(response, "fetched")
self.assertContains(response, "Updater action completed successfully.")
run_git_fetch.assert_called_once_with()
collect_update_status.assert_called_once_with(check_release=True)
def test_readonly_host_detail_hides_backup_controls_and_sensitive_config(self) -> None:
self.client.force_login(self.readonly_user)
GlobalConfig.objects.create(name="default", backup_root="/backups")
credential = SshCredential.objects.create(name="root-key", key_path="/var/lib/pobsync/state/root-key")
host = HostConfig.objects.create(
host="web-01",
address="web-01.example.test",
ssh_credential=credential,
ssh_user="root",
)
response = self.client.get(reverse("host_detail", args=[host.host]))
self.assertEqual(response.status_code, 200)
self.assertContains(response, "Host Status")
self.assertNotContains(response, "Backup Control")
self.assertNotContains(response, "Backup Options")
self.assertNotContains(response, "Connection Preflight")
self.assertNotContains(response, "root-key")
self.assertNotContains(response, reverse("queue_manual_backup", args=[host.host]))
def test_changelog_renders_repository_changelog(self) -> None: def test_changelog_renders_repository_changelog(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
with TemporaryDirectory() as tmp: with TemporaryDirectory() as tmp:
@@ -140,6 +316,27 @@ class ViewTests(TestCase):
self.assertEqual(target.channel, NotificationTarget.Channel.EMAIL) self.assertEqual(target.channel, NotificationTarget.Channel.EMAIL)
self.assertEqual(target.statuses, [BackupRun.Status.FAILED, BackupRun.Status.WARNING]) self.assertEqual(target.statuses, [BackupRun.Status.FAILED, BackupRun.Status.WARNING])
self.assertEqual(target.email_to, "ops@example.test\nbackup@example.test") self.assertEqual(target.email_to, "ops@example.test\nbackup@example.test")
self.assertEqual(target.webhook_headers, {})
def test_notification_target_form_allows_blank_webhook_headers(self) -> None:
self.client.force_login(self.staff_user)
response = self.client.post(
reverse("create_notification_target"),
{
"name": "ops",
"enabled": "on",
"channel": NotificationTarget.Channel.EMAIL,
"statuses": [BackupRun.Status.FAILED],
"email_to": "ops@example.test",
"webhook_headers": "",
},
follow=True,
)
self.assertRedirects(response, reverse("notification_targets"))
target = NotificationTarget.objects.get(name="ops")
self.assertEqual(target.webhook_headers, {})
def test_notification_target_form_requires_channel_destination(self) -> None: def test_notification_target_form_requires_channel_destination(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
@@ -265,23 +462,12 @@ class ViewTests(TestCase):
db = HostConfig.objects.create(host="db-01", address="db-01.example.test") db = HostConfig.objects.create(host="db-01", address="db-01.example.test")
scheduled = self._snapshot(web, "20260519-021500Z__SCHED01", kind=SnapshotRecord.Kind.SCHEDULED) scheduled = self._snapshot(web, "20260519-021500Z__SCHED01", kind=SnapshotRecord.Kind.SCHEDULED)
manual = self._snapshot(web, "20260519-031500Z__MANUAL1", kind=SnapshotRecord.Kind.MANUAL) manual = self._snapshot(web, "20260519-031500Z__MANUAL1", kind=SnapshotRecord.Kind.MANUAL)
incomplete = self._snapshot(db, "20260519-041500Z__BROKEN1", kind=SnapshotRecord.Kind.INCOMPLETE)
self._set_snapshot_storage(scheduled, allocated=100) self._set_snapshot_storage(scheduled, allocated=100)
self._set_snapshot_storage(manual, allocated=200) self._set_snapshot_storage(manual, allocated=200)
with TemporaryDirectory() as tmp: self._set_snapshot_storage(incomplete, allocated=400)
incomplete_dir = Path(tmp) / db.host / ".incomplete" / "20260519-041500Z__BROKEN1"
data_dir = incomplete_dir / "data"
data_dir.mkdir(parents=True)
data_dir.joinpath("partial-file").write_text("interrupted backup data\n", encoding="utf-8")
expected_usage = tree_usage(data_dir)
SnapshotRecord.objects.create(
host=db,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=incomplete_dir.name,
path=str(incomplete_dir),
status="failed",
)
response = self.client.get(reverse("dashboard_priority_live")) response = self.client.get(reverse("dashboard_priority_live"))
self.assertEqual(response.status_code, 200) self.assertEqual(response.status_code, 200)
self.assertContains(response, "Scheduled data") self.assertContains(response, "Scheduled data")
@@ -290,8 +476,8 @@ class ViewTests(TestCase):
self.assertContains(response, "Total snapshot data") self.assertContains(response, "Total snapshot data")
self.assertContains(response, "100&nbsp;bytes", html=True) self.assertContains(response, "100&nbsp;bytes", html=True)
self.assertContains(response, "200&nbsp;bytes", html=True) self.assertContains(response, "200&nbsp;bytes", html=True)
self.assertContains(response, filesizeformat(expected_usage["allocated_size_bytes"])) self.assertContains(response, "400&nbsp;bytes", html=True)
self.assertContains(response, filesizeformat(300 + expected_usage["allocated_size_bytes"])) self.assertContains(response, "700&nbsp;bytes", html=True)
def test_dashboard_hosts_live_returns_hosts_partial(self) -> None: def test_dashboard_hosts_live_returns_hosts_partial(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
@@ -311,23 +497,12 @@ class ViewTests(TestCase):
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
scheduled = self._snapshot(host, "20260519-021500Z__SCHED01", kind=SnapshotRecord.Kind.SCHEDULED) scheduled = self._snapshot(host, "20260519-021500Z__SCHED01", kind=SnapshotRecord.Kind.SCHEDULED)
manual = self._snapshot(host, "20260519-031500Z__MANUAL1", kind=SnapshotRecord.Kind.MANUAL) manual = self._snapshot(host, "20260519-031500Z__MANUAL1", kind=SnapshotRecord.Kind.MANUAL)
incomplete = self._snapshot(host, "20260519-041500Z__BROKEN1", kind=SnapshotRecord.Kind.INCOMPLETE)
self._set_snapshot_storage(scheduled, allocated=100) self._set_snapshot_storage(scheduled, allocated=100)
self._set_snapshot_storage(manual, allocated=200) self._set_snapshot_storage(manual, allocated=200)
with TemporaryDirectory() as tmp: self._set_snapshot_storage(incomplete, allocated=400)
incomplete_dir = Path(tmp) / host.host / ".incomplete" / "20260519-041500Z__BROKEN1"
data_dir = incomplete_dir / "data"
data_dir.mkdir(parents=True)
data_dir.joinpath("partial-file").write_text("interrupted backup data\n", encoding="utf-8")
expected_usage = tree_usage(data_dir)
SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=incomplete_dir.name,
path=str(incomplete_dir),
status="failed",
)
response = self.client.get(reverse("dashboard_hosts_live")) response = self.client.get(reverse("dashboard_hosts_live"))
self.assertEqual(response.status_code, 200) self.assertEqual(response.status_code, 200)
self.assertContains(response, "Scheduled data") self.assertContains(response, "Scheduled data")
@@ -336,32 +511,27 @@ class ViewTests(TestCase):
self.assertContains(response, "Total data") self.assertContains(response, "Total data")
self.assertContains(response, "100&nbsp;bytes", html=True) self.assertContains(response, "100&nbsp;bytes", html=True)
self.assertContains(response, "200&nbsp;bytes", html=True) self.assertContains(response, "200&nbsp;bytes", html=True)
self.assertContains(response, filesizeformat(expected_usage["allocated_size_bytes"])) self.assertContains(response, "400&nbsp;bytes", html=True)
self.assertContains(response, filesizeformat(300 + expected_usage["allocated_size_bytes"])) self.assertContains(response, "700&nbsp;bytes", html=True)
def test_dashboard_host_cards_measure_incomplete_data_without_snapshot_metadata(self) -> None: def test_dashboard_host_cards_mark_incomplete_data_without_snapshot_metadata_unmeasured(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
host = HostConfig.objects.create(host="web-01", address="web-01.example.test") host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
with TemporaryDirectory() as tmp: SnapshotRecord.objects.create(
incomplete_dir = Path(tmp) / host.host / ".incomplete" / "20260519-041500Z__BROKEN1" host=host,
data_dir = incomplete_dir / "data" kind=SnapshotRecord.Kind.INCOMPLETE,
data_dir.mkdir(parents=True) dirname="20260519-041500Z__BROKEN1",
data_dir.joinpath("partial-file").write_text("interrupted backup data\n", encoding="utf-8") path="/backups/web-01/.incomplete/20260519-041500Z__BROKEN1",
expected_usage = tree_usage(data_dir) status="failed",
SnapshotRecord.objects.create( metadata={},
host=host, )
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=incomplete_dir.name,
path=str(incomplete_dir),
status="failed",
metadata={},
)
response = self.client.get(reverse("dashboard_hosts_live")) response = self.client.get(reverse("dashboard_hosts_live"))
self.assertEqual(response.status_code, 200) self.assertEqual(response.status_code, 200)
self.assertContains(response, "Incomplete data") self.assertContains(response, "Incomplete data")
self.assertContains(response, filesizeformat(expected_usage["allocated_size_bytes"])) self.assertContains(response, "0&nbsp;bytes", html=True)
self.assertContains(response, "1 not measured")
def test_hosts_list_renders_host_cards_and_controls(self) -> None: def test_hosts_list_renders_host_cards_and_controls(self) -> None:
self.client.force_login(self.staff_user) self.client.force_login(self.staff_user)
@@ -1372,7 +1542,7 @@ class ViewTests(TestCase):
response = self.client.post( response = self.client.post(
reverse("queue_manual_backup", args=[host.host]), reverse("queue_manual_backup", args=[host.host]),
{"prune_max_delete": "10"}, {"verbose_output": "on", "prune_max_delete": "10"},
follow=True, follow=True,
) )
@@ -1610,7 +1780,7 @@ class ViewTests(TestCase):
response = self.client.post( response = self.client.post(
reverse("queue_manual_backup", args=[host.host]), reverse("queue_manual_backup", args=[host.host]),
{"prune_max_delete": "10"}, {"verbose_output": "on", "prune_max_delete": "10"},
follow=True, follow=True,
) )
@@ -1620,7 +1790,7 @@ class ViewTests(TestCase):
run.result["requested"], run.result["requested"],
{ {
"dry_run": False, "dry_run": False,
"verbose_output": False, "verbose_output": True,
"prune": False, "prune": False,
"prune_max_delete": 10, "prune_max_delete": 10,
"prune_protect_bases": False, "prune_protect_bases": False,

View File

@@ -0,0 +1,149 @@
from __future__ import annotations
import json
import shlex
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen
from django.conf import settings
from pobsync import __version__
@dataclass(frozen=True)
class CommandResult:
command: list[str]
exit_code: int
stdout: str
stderr: str
@property
def ok(self) -> bool:
return self.exit_code == 0
def collect_update_status(*, check_release: bool = False) -> dict[str, Any]:
app_dir = Path(settings.BASE_DIR)
status: dict[str, Any] = {
"app_dir": app_dir,
"installed_version": __version__,
"release_check_configured": bool(settings.POBSYNC_UPDATE_RELEASES_URL),
"update_command": settings.POBSYNC_UPDATE_COMMAND,
"git_remote": settings.POBSYNC_UPDATE_GIT_REMOTE,
"git": _git_status(app_dir),
"latest_release": None,
"release_error": "",
"update_available": None,
}
if check_release:
if not settings.POBSYNC_UPDATE_RELEASES_URL:
status["release_error"] = "POBSYNC_UPDATE_RELEASES_URL is not configured."
else:
try:
latest_release = fetch_latest_release()
status["latest_release"] = latest_release
status["update_available"] = _version_key(latest_release.get("tag_name", "")) != _version_key(__version__)
except (HTTPError, URLError, TimeoutError, json.JSONDecodeError, ValueError) as exc:
status["release_error"] = str(exc)
return status
def fetch_latest_release() -> dict[str, Any]:
request = Request(settings.POBSYNC_UPDATE_RELEASES_URL, headers={"Accept": "application/json"})
if settings.POBSYNC_UPDATE_RELEASES_TOKEN:
request.add_header("Authorization", f"token {settings.POBSYNC_UPDATE_RELEASES_TOKEN}")
with urlopen(request, timeout=10) as response:
payload = json.loads(response.read().decode("utf-8"))
if isinstance(payload, list):
if not payload:
raise ValueError("No releases were returned.")
release = payload[0]
elif isinstance(payload, dict):
release = payload
else:
raise ValueError("Release endpoint returned an unexpected payload.")
if not isinstance(release, dict):
raise ValueError("Release endpoint returned an unexpected release entry.")
return release
def run_git_fetch() -> CommandResult:
remote = settings.POBSYNC_UPDATE_GIT_REMOTE
return _run_command(["git", "fetch", "--tags", "--prune", remote])
def run_git_pull() -> CommandResult:
remote = settings.POBSYNC_UPDATE_GIT_REMOTE
branch = _git_current_branch(Path(settings.BASE_DIR))
if not branch:
return CommandResult(
command=["git", "pull", "--ff-only", remote],
exit_code=2,
stdout="",
stderr="Cannot pull automatically because the installed checkout is not on a branch.",
)
return _run_command(["git", "pull", "--ff-only", remote, branch])
def run_native_update() -> CommandResult:
return _run_command(shlex.split(settings.POBSYNC_UPDATE_COMMAND))
def _run_command(command: list[str]) -> CommandResult:
completed = subprocess.run(
command,
cwd=settings.BASE_DIR,
capture_output=True,
check=False,
text=True,
timeout=600,
)
return CommandResult(
command=command,
exit_code=completed.returncode,
stdout=completed.stdout[-6000:],
stderr=completed.stderr[-6000:],
)
def _git_status(app_dir: Path) -> dict[str, str]:
return {
"branch": _git_current_branch(app_dir),
"commit": _git_output(app_dir, ["git", "rev-parse", "--short", "HEAD"]),
"describe": _git_output(app_dir, ["git", "describe", "--tags", "--always", "--dirty"]),
}
def _git_current_branch(app_dir: Path) -> str:
branch = _git_output(app_dir, ["git", "branch", "--show-current"])
return branch or _git_output(app_dir, ["git", "rev-parse", "--abbrev-ref", "HEAD"])
def _git_output(app_dir: Path, command: list[str]) -> str:
try:
completed = subprocess.run(
command,
cwd=app_dir,
capture_output=True,
check=False,
text=True,
timeout=5,
)
except (FileNotFoundError, subprocess.TimeoutExpired):
return ""
if completed.returncode != 0:
return ""
return completed.stdout.strip()
def _version_key(value: str) -> str:
return value.strip().removeprefix("v")

View File

@@ -10,7 +10,6 @@ from pathlib import Path
from urllib.parse import urlencode from urllib.parse import urlencode
from django.contrib import messages from django.contrib import messages
from django.contrib.admin.views.decorators import staff_member_required
from django.conf import settings from django.conf import settings
from django.http import FileResponse, Http404 from django.http import FileResponse, Http404
from django.db.models import Count, Q from django.db.models import Count, Q
@@ -24,6 +23,7 @@ from pobsync import __version__
from pobsync.errors import PobsyncError from pobsync.errors import PobsyncError
from pobsync.config.defaults import DEFAULT_EXCLUDES, DEFAULT_RSYNC_ARGS from pobsync.config.defaults import DEFAULT_EXCLUDES, DEFAULT_RSYNC_ARGS
from .access import access_context, control_panel_admin_required, status_view_required
from .backup_runner import queue_backup_run from .backup_runner import queue_backup_run
from .config_checks import collect_effective_host_config_checks, collect_global_config_checks from .config_checks import collect_effective_host_config_checks, collect_global_config_checks
from .forms import ( from .forms import (
@@ -57,21 +57,22 @@ from .scheduler import next_due_after
from .snapshot_discovery import discover_snapshots, inspect_snapshot_discovery from .snapshot_discovery import discover_snapshots, inspect_snapshot_discovery
from .ssh_keys import SshKeyError, delete_generated_key_files, generate_ssh_key, merge_known_hosts, scan_known_host from .ssh_keys import SshKeyError, delete_generated_key_files, generate_ssh_key, merge_known_hosts, scan_known_host
from .stats_summary import collect_dashboard_stats, collect_host_stats from .stats_summary import collect_dashboard_stats, collect_host_stats
from .updater import collect_update_status, run_git_fetch, run_git_pull, run_native_update
@staff_member_required @status_view_required
def dashboard(request): def dashboard(request):
return render(request, "pobsync_backend/dashboard.html", _dashboard_context()) return render(request, "pobsync_backend/dashboard.html", _dashboard_context(request))
@staff_member_required @status_view_required
def dashboard_priority_live(request): def dashboard_priority_live(request):
return render(request, "pobsync_backend/partials/dashboard_priority.html", _dashboard_context()) return render(request, "pobsync_backend/partials/dashboard_priority.html", _dashboard_context(request))
@staff_member_required @status_view_required
def dashboard_hosts_live(request): def dashboard_hosts_live(request):
return render(request, "pobsync_backend/partials/dashboard_hosts.html", _dashboard_context()) return render(request, "pobsync_backend/partials/dashboard_hosts.html", _dashboard_context(request))
def _host_cards_context(*, enabled: str = "") -> dict[str, object]: def _host_cards_context(*, enabled: str = "") -> dict[str, object]:
@@ -120,7 +121,7 @@ def _host_cards_context(*, enabled: str = "") -> dict[str, object]:
} }
def _dashboard_context() -> dict[str, object]: def _dashboard_context(request) -> dict[str, object]:
global_config = GlobalConfig.objects.filter(name="default").first() global_config = GlobalConfig.objects.filter(name="default").first()
host_context = _host_cards_context() host_context = _host_cards_context()
hosts = host_context["hosts"] hosts = host_context["hosts"]
@@ -129,6 +130,7 @@ def _dashboard_context() -> dict[str, object]:
recent_runs = BackupRun.objects.select_related("host", "snapshot").order_by("-created_at", "-id")[:6] recent_runs = BackupRun.objects.select_related("host", "snapshot").order_by("-created_at", "-id")[:6]
stats_summary = collect_dashboard_stats(hosts=hosts, global_config=global_config) stats_summary = collect_dashboard_stats(hosts=hosts, global_config=global_config)
context = { context = {
**access_context(request),
"hosts": hosts, "hosts": hosts,
"global_config": global_config, "global_config": global_config,
"stats_summary": stats_summary, "stats_summary": stats_summary,
@@ -159,7 +161,7 @@ def _dashboard_context() -> dict[str, object]:
return context return context
@staff_member_required @status_view_required
def hosts_list(request): def hosts_list(request):
enabled = request.GET.get("enabled", "").strip() enabled = request.GET.get("enabled", "").strip()
if enabled not in {"", "yes", "no"}: if enabled not in {"", "yes", "no"}:
@@ -171,15 +173,16 @@ def hosts_list(request):
request, request,
"pobsync_backend/hosts_list.html", "pobsync_backend/hosts_list.html",
{ {
**access_context(request),
**context, **context,
"global_config": global_config, "global_config": global_config,
"show_host_controls": True, "show_host_controls": request.user.is_staff,
"total_count": HostConfig.objects.count(), "total_count": HostConfig.objects.count(),
}, },
) )
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def update_host_state(request, host: str): def update_host_state(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -290,7 +293,7 @@ def _retention_warning_summary(retention_warning) -> str:
return " ".join(parts) return " ".join(parts)
@staff_member_required @status_view_required
def changelog(request): def changelog(request):
changelog_path = Path(settings.BASE_DIR) / "CHANGELOG.md" changelog_path = Path(settings.BASE_DIR) / "CHANGELOG.md"
try: try:
@@ -312,7 +315,7 @@ def changelog(request):
) )
@staff_member_required @control_panel_admin_required
def self_check(request): def self_check(request):
checks = collect_self_checks() checks = collect_self_checks()
return render( return render(
@@ -325,13 +328,47 @@ def self_check(request):
) )
@staff_member_required @control_panel_admin_required
def logs(request): def logs(request):
context = _log_context(request) context = _log_context(request)
return render(request, "pobsync_backend/logs.html", context) return render(request, "pobsync_backend/logs.html", context)
@staff_member_required @control_panel_admin_required
def updater(request):
action_result = None
check_release = request.method == "POST" and request.POST.get("action") == "check_release"
if request.method == "POST":
action = request.POST.get("action")
if action == "git_fetch":
action_result = run_git_fetch()
check_release = True
elif action == "git_pull":
action_result = run_git_pull()
check_release = True
elif action == "run_update":
action_result = run_native_update()
check_release = True
elif action != "check_release":
messages.error(request, "Unknown updater action.")
if action_result is not None:
if action_result.ok:
messages.success(request, "Updater action completed successfully.")
else:
messages.error(request, f"Updater action failed with exit code {action_result.exit_code}.")
return render(
request,
"pobsync_backend/updater.html",
{
"status": collect_update_status(check_release=check_release),
"action_result": action_result,
},
)
@control_panel_admin_required
def notification_targets(request): def notification_targets(request):
targets = NotificationTarget.objects.order_by("name") targets = NotificationTarget.objects.order_by("name")
deliveries = NotificationDelivery.objects.select_related("target", "run", "run__host").order_by("-created_at")[:12] deliveries = NotificationDelivery.objects.select_related("target", "run", "run__host").order_by("-created_at")[:12]
@@ -345,7 +382,7 @@ def notification_targets(request):
) )
@staff_member_required @control_panel_admin_required
def create_notification_target(request): def create_notification_target(request):
if request.method == "POST": if request.method == "POST":
form = NotificationTargetForm(request.POST) form = NotificationTargetForm(request.POST)
@@ -367,7 +404,7 @@ def create_notification_target(request):
) )
@staff_member_required @control_panel_admin_required
def edit_notification_target(request, target_id: int): def edit_notification_target(request, target_id: int):
target = get_object_or_404(NotificationTarget, id=target_id) target = get_object_or_404(NotificationTarget, id=target_id)
if request.method == "POST": if request.method == "POST":
@@ -390,7 +427,7 @@ def edit_notification_target(request, target_id: int):
) )
@staff_member_required @status_view_required
def runs_list(request): def runs_list(request):
status = request.GET.get("status", "").strip() status = request.GET.get("status", "").strip()
run_type = request.GET.get("type", "").strip() run_type = request.GET.get("type", "").strip()
@@ -422,7 +459,7 @@ def runs_list(request):
return render(request, "pobsync_backend/runs_list.html", context) return render(request, "pobsync_backend/runs_list.html", context)
@staff_member_required @status_view_required
def snapshots_list(request): def snapshots_list(request):
kind = request.GET.get("kind", "").strip() kind = request.GET.get("kind", "").strip()
status = request.GET.get("status", "").strip() status = request.GET.get("status", "").strip()
@@ -448,7 +485,7 @@ def snapshots_list(request):
return render(request, "pobsync_backend/snapshots_list.html", context) return render(request, "pobsync_backend/snapshots_list.html", context)
@staff_member_required @status_view_required
def schedules_list(request): def schedules_list(request):
enabled = request.GET.get("enabled", "").strip() enabled = request.GET.get("enabled", "").strip()
prune = request.GET.get("prune", "").strip() prune = request.GET.get("prune", "").strip()
@@ -486,7 +523,7 @@ def schedules_list(request):
return render(request, "pobsync_backend/schedules_list.html", context) return render(request, "pobsync_backend/schedules_list.html", context)
@staff_member_required @status_view_required
def purged_snapshots(request): def purged_snapshots(request):
host = request.GET.get("host", "").strip() host = request.GET.get("host", "").strip()
action = request.GET.get("action", "").strip() action = request.GET.get("action", "").strip()
@@ -507,7 +544,7 @@ def purged_snapshots(request):
return render(request, "pobsync_backend/purged_snapshots.html", context) return render(request, "pobsync_backend/purged_snapshots.html", context)
@staff_member_required @control_panel_admin_required
def ssh_credentials(request): def ssh_credentials(request):
context = { context = {
"credentials": SshCredential.objects.order_by("name"), "credentials": SshCredential.objects.order_by("name"),
@@ -515,7 +552,7 @@ def ssh_credentials(request):
return render(request, "pobsync_backend/ssh_credentials.html", context) return render(request, "pobsync_backend/ssh_credentials.html", context)
@staff_member_required @control_panel_admin_required
def create_ssh_credential(request): def create_ssh_credential(request):
if request.method == "POST": if request.method == "POST":
form = SshCredentialForm(request.POST, request.FILES) form = SshCredentialForm(request.POST, request.FILES)
@@ -536,7 +573,7 @@ def create_ssh_credential(request):
) )
@staff_member_required @control_panel_admin_required
def generate_ssh_credential(request): def generate_ssh_credential(request):
if request.method == "POST": if request.method == "POST":
form = SshCredentialGenerateForm(request.POST) form = SshCredentialGenerateForm(request.POST)
@@ -572,7 +609,7 @@ def generate_ssh_credential(request):
) )
@staff_member_required @control_panel_admin_required
def edit_ssh_credential(request, credential_id: int): def edit_ssh_credential(request, credential_id: int):
credential = get_object_or_404(SshCredential, id=credential_id) credential = get_object_or_404(SshCredential, id=credential_id)
if request.method == "POST": if request.method == "POST":
@@ -594,7 +631,7 @@ def edit_ssh_credential(request, credential_id: int):
) )
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def delete_ssh_credential(request, credential_id: int): def delete_ssh_credential(request, credential_id: int):
credential = get_object_or_404(SshCredential, id=credential_id) credential = get_object_or_404(SshCredential, id=credential_id)
@@ -618,7 +655,7 @@ def delete_ssh_credential(request, credential_id: int):
return redirect("ssh_credentials") return redirect("ssh_credentials")
@staff_member_required @control_panel_admin_required
def edit_global_config(request): def edit_global_config(request):
global_config = GlobalConfig.objects.filter(name="default").first() global_config = GlobalConfig.objects.filter(name="default").first()
if request.method == "POST": if request.method == "POST":
@@ -644,7 +681,7 @@ def edit_global_config(request):
) )
@staff_member_required @control_panel_admin_required
def create_host_config(request): def create_host_config(request):
if request.method == "POST": if request.method == "POST":
form = CreateHostConfigForm(request.POST) form = CreateHostConfigForm(request.POST)
@@ -672,7 +709,7 @@ def create_host_config(request):
) )
@staff_member_required @status_view_required
def host_detail(request, host: str): def host_detail(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
global_config = GlobalConfig.objects.filter(name="default").first() global_config = GlobalConfig.objects.filter(name="default").first()
@@ -685,7 +722,9 @@ def host_detail(request, host: str):
has_global_config = global_config is not None has_global_config = global_config is not None
backup_gate = collect_backup_gate(host_config, global_config) backup_gate = collect_backup_gate(host_config, global_config)
stats_summary = collect_host_stats(host=host_config, limit=10) stats_summary = collect_host_stats(host=host_config, limit=10)
can_manage = request.user.is_staff
context = { context = {
**access_context(request),
"host": host_config, "host": host_config,
"schedule": schedule, "schedule": schedule,
"retention_warning": _retention_warning_for_host(host_config, schedule), "retention_warning": _retention_warning_for_host(host_config, schedule),
@@ -696,11 +735,11 @@ def host_detail(request, host: str):
"host_check_summary": summarize_self_checks(backup_gate.checks), "host_check_summary": summarize_self_checks(backup_gate.checks),
"backup_gate": backup_gate, "backup_gate": backup_gate,
"last_preflight": (host_config.config or {}).get("last_preflight") if isinstance(host_config.config, dict) else {}, "last_preflight": (host_config.config or {}).get("last_preflight") if isinstance(host_config.config, dict) else {},
"effective_config": effective_host_config_preview(host_config, global_config) if global_config else {}, "effective_config": effective_host_config_preview(host_config, global_config) if global_config and can_manage else {},
"stats_summary": stats_summary, "stats_summary": stats_summary,
"manual_backup_form": ManualBackupForm(initial=_default_manual_backup_initial(host_config)), "manual_backup_form": ManualBackupForm(initial=_default_manual_backup_initial(host_config)),
"can_queue_dry_run": host_config.enabled and has_global_config and backup_gate.can_queue_dry_run and active_run is None, "can_queue_dry_run": can_manage and host_config.enabled and has_global_config and backup_gate.can_queue_dry_run and active_run is None,
"can_queue_real_backup": host_config.enabled and has_global_config and backup_gate.can_queue_real and active_run is None, "can_queue_real_backup": can_manage and host_config.enabled and has_global_config and backup_gate.can_queue_real and active_run is None,
"has_global_config": has_global_config, "has_global_config": has_global_config,
"active_run": active_run, "active_run": active_run,
"latest_runs": host_config.runs.select_related("snapshot").order_by("-created_at")[:10], "latest_runs": host_config.runs.select_related("snapshot").order_by("-created_at")[:10],
@@ -720,7 +759,7 @@ def host_detail(request, host: str):
return render(request, "pobsync_backend/host_detail.html", context) return render(request, "pobsync_backend/host_detail.html", context)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def prepare_host_directories(request, host: str): def prepare_host_directories(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -733,7 +772,7 @@ def prepare_host_directories(request, host: str):
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def scan_host_known_key(request, host: str): def scan_host_known_key(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -755,7 +794,7 @@ def scan_host_known_key(request, host: str):
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def run_host_preflight(request, host: str): def run_host_preflight(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -783,7 +822,7 @@ def run_host_preflight(request, host: str):
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def queue_manual_backup(request, host: str): def queue_manual_backup(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -823,22 +862,22 @@ def queue_manual_backup(request, host: str):
return redirect("run_detail", run_id=run.id) return redirect("run_detail", run_id=run.id)
@staff_member_required @status_view_required
def run_detail(request, run_id: int): def run_detail(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id) run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id)
return render(request, "pobsync_backend/run_detail.html", _run_detail_context(run)) return render(request, "pobsync_backend/run_detail.html", _run_detail_context(run, request=request))
@staff_member_required @status_view_required
def run_detail_live(request, run_id: int): def run_detail_live(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id) run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id)
context = _run_detail_context(run) context = _run_detail_context(run, request=request)
response = render(request, "pobsync_backend/partials/run_detail_live.html", context) response = render(request, "pobsync_backend/partials/run_detail_live.html", context)
response["X-Pobsync-Refresh-Active"] = "true" if context["can_auto_refresh"] else "false" response["X-Pobsync-Refresh-Active"] = "true" if context["can_auto_refresh"] else "false"
return response return response
def _run_detail_context(run: BackupRun) -> dict[str, object]: def _run_detail_context(run: BackupRun, *, request=None) -> dict[str, object]:
result = run.result if isinstance(run.result, dict) else {} result = run.result if isinstance(run.result, dict) else {}
run_stats = result.get("stats") if isinstance(result.get("stats"), dict) else {} run_stats = result.get("stats") if isinstance(result.get("stats"), dict) else {}
rsync_result = result.get("rsync") if isinstance(result.get("rsync"), dict) else {} rsync_result = result.get("rsync") if isinstance(result.get("rsync"), dict) else {}
@@ -848,8 +887,10 @@ def _run_detail_context(run: BackupRun) -> dict[str, object]:
rsync_log_path = _run_rsync_log_path(run) rsync_log_path = _run_rsync_log_path(run)
rsync_log_tail = _run_rsync_log_tail(rsync_result, rsync_log_path) rsync_log_tail = _run_rsync_log_tail(rsync_result, rsync_log_path)
requested = result.get("requested") if isinstance(result.get("requested"), dict) else {} requested = result.get("requested") if isinstance(result.get("requested"), dict) else {}
can_cancel = run.status in {BackupRun.Status.QUEUED, BackupRun.Status.RUNNING} can_manage = bool(request and request.user.is_staff)
can_cancel = can_manage and run.status in {BackupRun.Status.QUEUED, BackupRun.Status.RUNNING}
return { return {
**(access_context(request) if request is not None else {}),
"run": run, "run": run,
"can_cancel": can_cancel, "can_cancel": can_cancel,
"can_auto_refresh": can_cancel, "can_auto_refresh": can_cancel,
@@ -880,7 +921,7 @@ def _run_detail_context(run: BackupRun) -> dict[str, object]:
} }
@staff_member_required @control_panel_admin_required
def run_rsync_log(request, run_id: int): def run_rsync_log(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id) run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id)
log_path = _run_rsync_log_path(run) log_path = _run_rsync_log_path(run)
@@ -889,7 +930,7 @@ def run_rsync_log(request, run_id: int):
return FileResponse(log_path.open("rb"), content_type="text/plain; charset=utf-8") return FileResponse(log_path.open("rb"), content_type="text/plain; charset=utf-8")
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def cancel_run(request, run_id: int): def cancel_run(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host"), id=run_id) run = get_object_or_404(BackupRun.objects.select_related("host"), id=run_id)
@@ -913,7 +954,7 @@ def cancel_run(request, run_id: int):
return redirect("run_detail", run_id=run.id) return redirect("run_detail", run_id=run.id)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def resolve_run_review(request, run_id: int): def resolve_run_review(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host"), id=run_id) run = get_object_or_404(BackupRun.objects.select_related("host"), id=run_id)
@@ -931,7 +972,7 @@ def resolve_run_review(request, run_id: int):
return _redirect_after_run_review(request, run) return _redirect_after_run_review(request, run)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def resolve_host_incomplete_reviews(request, host: str): def resolve_host_incomplete_reviews(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -947,7 +988,7 @@ def resolve_host_incomplete_reviews(request, host: str):
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@staff_member_required @status_view_required
def snapshot_detail(request, snapshot_id: int): def snapshot_detail(request, snapshot_id: int):
snapshot = get_object_or_404( snapshot = get_object_or_404(
SnapshotRecord.objects.select_related("host", "base").prefetch_related("derived_snapshots", "backup_runs"), SnapshotRecord.objects.select_related("host", "base").prefetch_related("derived_snapshots", "backup_runs"),
@@ -965,7 +1006,7 @@ def snapshot_detail(request, snapshot_id: int):
return render(request, "pobsync_backend/snapshot_detail.html", context) return render(request, "pobsync_backend/snapshot_detail.html", context)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def discover_host_snapshots(request, host: str): def discover_host_snapshots(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -986,7 +1027,7 @@ def discover_host_snapshots(request, host: str):
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@staff_member_required @control_panel_admin_required
def host_retention_plan(request, host: str): def host_retention_plan(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
kind = request.GET.get("kind", "scheduled") kind = request.GET.get("kind", "scheduled")
@@ -1037,7 +1078,7 @@ def host_retention_plan(request, host: str):
return render(request, "pobsync_backend/retention_plan.html", context) return render(request, "pobsync_backend/retention_plan.html", context)
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def apply_host_retention(request, host: str): def apply_host_retention(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -1091,7 +1132,7 @@ def apply_host_retention(request, host: str):
return target return target
@staff_member_required @control_panel_admin_required
@require_POST @require_POST
def cleanup_host_incomplete_snapshots(request, host: str): def cleanup_host_incomplete_snapshots(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -1126,7 +1167,7 @@ def cleanup_host_incomplete_snapshots(request, host: str):
return redirect("host_retention_plan", host=host_config.host) return redirect("host_retention_plan", host=host_config.host)
@staff_member_required @control_panel_admin_required
def edit_host_config(request, host: str): def edit_host_config(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
global_config = GlobalConfig.objects.filter(name="default").first() global_config = GlobalConfig.objects.filter(name="default").first()
@@ -1152,7 +1193,7 @@ def edit_host_config(request, host: str):
) )
@staff_member_required @control_panel_admin_required
def edit_host_schedule(request, host: str): def edit_host_schedule(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
schedule = _schedule_for_host(host_config) schedule = _schedule_for_host(host_config)
@@ -1301,6 +1342,7 @@ def _default_manual_backup_initial(host_config: HostConfig) -> dict[str, object]
schedule = _schedule_for_host(host_config) schedule = _schedule_for_host(host_config)
return { return {
"dry_run": True, "dry_run": True,
"verbose_output": True,
"prune": bool(schedule.prune) if schedule else False, "prune": bool(schedule.prune) if schedule else False,
"prune_max_delete": schedule.prune_max_delete if schedule else 10, "prune_max_delete": schedule.prune_max_delete if schedule else 10,
"prune_protect_bases": bool(schedule.prune_protect_bases) if schedule else False, "prune_protect_bases": bool(schedule.prune_protect_bases) if schedule else False,

View File

@@ -48,6 +48,7 @@ TEMPLATES = [
"django.template.context_processors.request", "django.template.context_processors.request",
"django.contrib.auth.context_processors.auth", "django.contrib.auth.context_processors.auth",
"django.contrib.messages.context_processors.messages", "django.contrib.messages.context_processors.messages",
"pobsync_backend.context_processors.pobsync_access",
], ],
}, },
}, },
@@ -55,6 +56,8 @@ TEMPLATES = [
WSGI_APPLICATION = "pobsync_server.wsgi.application" WSGI_APPLICATION = "pobsync_server.wsgi.application"
LOGIN_URL = "/admin/login/"
def _database_config() -> dict[str, object]: def _database_config() -> dict[str, object]:
engine = os.getenv("POBSYNC_DB_ENGINE", "sqlite").strip().lower() engine = os.getenv("POBSYNC_DB_ENGINE", "sqlite").strip().lower()
@@ -102,3 +105,7 @@ POBSYNC_BACKUP_ROOT = os.getenv("POBSYNC_BACKUP_ROOT", "/backups")
POBSYNC_ENV_FILE = os.getenv("POBSYNC_ENV_FILE", "/etc/pobsync/pobsync.env") POBSYNC_ENV_FILE = os.getenv("POBSYNC_ENV_FILE", "/etc/pobsync/pobsync.env")
POBSYNC_SERVICE_USER = os.getenv("POBSYNC_SERVICE_USER", "pobsync") POBSYNC_SERVICE_USER = os.getenv("POBSYNC_SERVICE_USER", "pobsync")
POBSYNC_SERVICE_GROUP = os.getenv("POBSYNC_SERVICE_GROUP", "pobsync") POBSYNC_SERVICE_GROUP = os.getenv("POBSYNC_SERVICE_GROUP", "pobsync")
POBSYNC_UPDATE_RELEASES_URL = os.getenv("POBSYNC_UPDATE_RELEASES_URL", "")
POBSYNC_UPDATE_RELEASES_TOKEN = os.getenv("POBSYNC_UPDATE_RELEASES_TOKEN", "")
POBSYNC_UPDATE_GIT_REMOTE = os.getenv("POBSYNC_UPDATE_GIT_REMOTE", "origin")
POBSYNC_UPDATE_COMMAND = os.getenv("POBSYNC_UPDATE_COMMAND", "sudo -n scripts/update-systemd")

View File

@@ -13,6 +13,7 @@ urlpatterns = [
path("changelog/", views.changelog, name="changelog"), path("changelog/", views.changelog, name="changelog"),
path("self-check/", views.self_check, name="self_check"), path("self-check/", views.self_check, name="self_check"),
path("logs/", views.logs, name="logs"), path("logs/", views.logs, name="logs"),
path("updater/", views.updater, name="updater"),
path("notifications/", views.notification_targets, name="notification_targets"), path("notifications/", views.notification_targets, name="notification_targets"),
path("notifications/new/", views.create_notification_target, name="create_notification_target"), path("notifications/new/", views.create_notification_target, name="create_notification_target"),
path("notifications/<int:target_id>/", views.edit_notification_target, name="edit_notification_target"), path("notifications/<int:target_id>/", views.edit_notification_target, name="edit_notification_target"),