54 Commits

Author SHA1 Message Date
00d4f2a70b Merge pull request 'release-hardening_1.0' (#21) from issue-8-release-hardening into master
Reviewed-on: #21
2026-05-21 03:56:24 +02:00
f8215a0c9a (release) Update 1.0 changelog for hardening work
Bring the 1.0.0 release notes up to date with the release-hardening work
completed after the initial metadata pass: worker heartbeat tracking,
incomplete snapshot cleanup, review resolution, SSH key management hardening,
purged snapshot audit history, and the in-app changelog page.

Refs #8
Refs #10
Refs #11
Refs #16
Refs #19
Refs #20
2026-05-21 03:51:27 +02:00
ea9e3e41e3 (release) Add purged snapshot audit overview
Record snapshot purge history whenever retention or incomplete cleanup removes
snapshot directories and SQL records. Store the purge reason, original kind,
path, action source, and triggering operator so manual, scheduled, CLI, and
incomplete cleanup actions remain auditable after the original snapshot record
is deleted.

Add a staff-only Purged Snapshots page with host/action filters and register
the audit model in Django admin.

Refs #16
Refs #8
2026-05-21 03:46:38 +02:00
5b5a5bc637 (release) Harden SSH key edit and delete flow
Make SSH credential management more explicit by adding an edit action in the
key overview and requiring name confirmation before deletion. Keep deletion
blocked while a key is still selected by hosts or global config, and cover
rename, delete confirmation, and in-use protection in view tests.

Refs #20
Refs #8
2026-05-21 03:38:55 +02:00
c2e5a534aa (release) Add review resolution for operational tasks
Add reviewed state for failed/warning runs and incomplete snapshot records,
then use it to clear dashboard and host “need review” tasks after an operator
has acknowledged them.

Expose Mark reviewed actions on run detail and host retention warnings, keep
reviewed records available for audit/debug, and exclude reviewed problem runs
from operational counts and latest issue summaries.

Refs #19
Refs #8
2026-05-21 03:34:41 +02:00
d0c23deb72 (release) Add explicit incomplete snapshot cleanup
Add a dedicated cleanup path for incomplete snapshots instead of letting
retention prune them implicitly. The retention plan now exposes a guarded
form that requires host and delete-count confirmation before removing
.incomplete snapshot directories and their SQL records.

Keep scheduled/manual retention behavior unchanged, add path safety checks,
and cover cleanup success, confirmation failures, max-delete limits, and
unexpected paths in tests.

Refs #10
2026-05-21 03:26:21 +02:00
4c8ed24561 (release) Track worker heartbeat for running jobs
Record worker pid, host, claim time, and heartbeat metadata on running
backup jobs so operators can see which worker owns a run.

Refresh the heartbeat while rsync is active and reconcile stale running
runs when the worker heartbeat stops. Add a worker option to tune or
disable stale-run reconciliation.

Refs #11
2026-05-21 03:16:38 +02:00
404b7f7500 (release) Add Django changelog page
Expose the repository CHANGELOG.md through a staff-only Django view and
link it from the main navigation.

Render a small safe subset of Markdown without adding a runtime dependency,
copy the changelog into the Docker image, and cover the page with view tests.
2026-05-21 03:10:31 +02:00
beca073ddc (release) Prepare 1.0.0 release metadata
Add the initial 1.0.0 changelog, bump the package/application version,
and expose the release version through `pobsync --version`.

Cover the version output in the console entrypoint tests.
2026-05-21 03:04:59 +02:00
362a9dde62 Merge pull request 'issue-7-config-cleanup-legacy-removal' (#18) from issue-7-config-cleanup-legacy-removal into master
Reviewed-on: #18
2026-05-21 02:57:38 +02:00
a73d34ac9f (refactor) Use operator-facing config errors
Replace remaining model-name based configuration errors with labels that
match the Django-first operating model.

Add coverage for missing global config and host configuration errors so
operator-facing messages stay readable.
2026-05-21 02:56:00 +02:00
1c8cbd96ca (refactor) Normalize maintainer command labels
Prefer --schedule-expression for scripted schedule updates while keeping
--cron as a compatibility alias.

Clean up management command help, errors, and output so operator-facing
text talks about hosts, global config, and Django backup configuration
instead of model names or old SQL-backed pobsync wording.
2026-05-21 02:52:42 +02:00
86873bd035 (refactor) Remove obsolete global config JSON storage
Drop the unused GlobalConfig.data field and remove the remaining YAML
config path helpers from PobsyncPaths.

Keep HostConfig.config as runtime state for preflight data, and relabel it
in the admin so it no longer reads as legacy compatibility storage.
2026-05-21 02:46:09 +02:00
2642f14e49 (refactor) Remove pobsync_home from global config
Drop the obsolete pobsync_home field from GlobalConfig and remove it from
runtime config generation, form saves, and configuration commands.

The runtime state root now comes exclusively from POBSYNC_HOME/settings,
which keeps the Django model focused on backup behavior instead of install
layout.
2026-05-21 02:41:02 +02:00
bb62382e18 (refactor) Remove YAML config import and export path
Drop the pre-Django YAML import/export management commands and remove the
file-based config loader fallback from the backup and retention engines.

Keep the runtime config bridge backed by Django models, and add tests that
ensure engine operations require an explicit Django config source.
2026-05-21 02:34:09 +02:00
c5865a5379 (refactor) Normalize runtime config labels
Hide the old pobsync_home field from the Django admin and replace legacy
operator-facing labels with runtime state root and backup root terminology.

Rename admin compatibility fieldsets, update self-check/config-check text,
and refresh management command help so Django/systemd stays the primary
mental model.
2026-05-21 02:24:55 +02:00
58d567f9bc (refactor) Retire configuration command aliases
Remove the short pobsync aliases for global config, host config, and
schedule changes so the public CLI no longer points operators toward the
old configuration workflow.

Keep operational aliases for backup, discovery, retention, worker, and
scheduler debugging, and document explicit Django management commands for
automation use.
2026-05-21 02:14:16 +02:00
2d9f453767 Merge pull request 'issue-6-restore-story' (#17) from issue-6-restore-story into master
Reviewed-on: #17
2026-05-21 02:06:22 +02:00
20a9f93378 (docs) Add targeted restore examples
Extend the restore guidance with directory and single-file dry-run
examples so operators can restore a focused path without copying an
entire snapshot.

Render the examples on snapshot detail pages using the selected
snapshot's data path and the host-specific staging destination.
2026-05-21 02:05:19 +02:00
b78f102e9d (docs) Add manual restore guidance for snapshots
Document the manual restore workflow in the README and surface snapshot-
specific restore commands on the snapshot detail page.

The guidance keeps restores intentionally manual for now: inspect the
snapshot data directory, run rsync with --dry-run, restore to staging
first, and treat hardlinked snapshot files as read-only.
2026-05-21 02:01:40 +02:00
8858e049ee Merge pull request '(ui) Add dashboard operational status summary' (#15) from issue-5-dashboard-1-0 into master
Reviewed-on: #15

This closes issue #5
2026-05-21 01:54:48 +02:00
a75b97c4c0 (ui) Add dashboard operational status summary
Make queued, running, warning, and failed run states more visible at the
top of the dashboard with contextual status summaries and highlighted
summary metrics.

Also show an all-clear message when configured hosts have no active or
problematic runs.
2026-05-21 01:51:13 +02:00
b4fc5a14b2 (ui) Clarify dashboard backup growth trends
Replace the dashboard trend metric grid with an operational summary that
explains storage usage, runway, average new data, link-dest savings, and
average duration in a more readable way.

Also add an empty state for fresh installs before completed backup stats
exist.
2026-05-21 01:46:49 +02:00
a0fd33fcb8 (ui) Improve dashboard host card scanability
Add per-host status chips for queued, running, warning, and failed runs so
the dashboard shows operational pressure without needing to open each host.

Restructure host cards into clearer backup activity and snapshot health
sections, with less visual clutter and better mobile wrapping.
2026-05-21 01:41:45 +02:00
ef1761385e (ui) Separate healthy and problematic dashboard runs
Split dashboard host cards into last successful backup and latest warning
or failed run so operators can quickly see whether a host is protected even
when recent activity produced an issue.

Also add queued and warning run counts to the dashboard summary metrics.
2026-05-21 01:34:38 +02:00
17215fd191 Merge pull request '(feature) Improve retention UX and prune safety' (#14) from issue-4-retention-ux-and-safety into master
Reviewed-on: #14
2026-05-21 01:27:19 +02:00
97753c3d3c (ui) Show retention apply details on run detail
Record planned delete counts, max-delete settings, base protection, and
ignored incomplete snapshots in retention apply results.

Surface those details on run detail pages so scheduled and manual prune
outcomes are understandable without reading the raw JSON payload.
2026-05-21 01:25:40 +02:00
994f7f66c4 (bugfix) Preserve scheduled backup warning status
Update the scheduler to reflect the actual scheduled BackupRun status after
a run completes, so prune warnings are shown as schedule warnings instead
of being reported as successful schedule executions.
2026-05-21 01:22:06 +02:00
f76b6cad14 (feature) Require delete count confirmation for retention apply
Make manual retention application more explicit by requiring operators to
confirm both the host name and the current number of planned deletions.

This reduces the risk of applying a stale or misunderstood retention plan
when the delete set changes between review and confirmation.
2026-05-21 01:19:08 +02:00
90e293facd (ui) Surface retention warnings on run detail
Show host-level retention warnings on run detail pages so successful or
warning runs still expose scheduled prune limit issues and incomplete
snapshots that need operator attention.
2026-05-21 01:13:44 +02:00
50eb7cf2f3 (ui) Make retention planning warnings explicit
Show keep/delete reasons in the retention plan, surface scheduled prune
limit warnings, and explain base snapshot protection before retention is
applied.

Also surface incomplete snapshots from the retention views without deleting
them automatically, so interrupted backups are visible on the dashboard,
host detail, and retention plan.
2026-05-21 01:10:45 +02:00
26265be440 Merge pull request '(feature) Add backup safety and preflight validation' (#13) from issue-3-backup-safety-preflight-validation into master
Reviewed-on: #13
2026-05-21 00:58:23 +02:00
5faef1492d (ui) Add readable dry-run summaries
Surface dry-run status, transfer estimates, file counts, warnings, and the full
rsync log link directly on the run detail page.

Keep raw rsync output and JSON available, but make the common review path easier
to scan before starting a real backup.
2026-05-21 00:55:19 +02:00
3045093dcf (feature) Add remote host connection preflight
Add an on-demand host preflight action that verifies SSH reachability,
remote rsync availability, and remote source root access.

Persist the latest preflight result on the host config, render it in Django,
and block real backups when the last remote preflight failed.
2026-05-21 00:50:05 +02:00
64a0ff8322 (feature) Add host backup preflight gates
Introduce a host preflight layer that separates dry-run blockers from real backup blockers.
Show the effective per-host backup configuration in Django before queueing a run.

Block real backup queueing when failed host checks remain, while still allowing dry-runs
when only local storage preparation is missing.
2026-05-21 00:41:45 +02:00
155ff63a73 Merge pull request '(feature) Improve run debugging and log filtering' (#12) from issue-2-run-detail-logging-polish into master
Reviewed-on: #12
2026-05-21 00:26:45 +02:00
a9e40df44b (feature) Add focused filtering to the service logs view
Extend the Django logs view with filters for service unit, severity, time
window, host, run id, and message text. Pass severity and time window directly
to journalctl, then apply host/run/message filtering to the returned pobsync
journal lines.

This makes failed or slow backups easier to investigate from the control panel
without needing shell access.
2026-05-21 00:24:07 +02:00
98695f9888 (ui) Surface run debugging details in Django
Restructure the run detail page into clearer sections for summary, failure
classification, requested options, rsync command, rsync log output, stats,
retention, and raw result data.

Show recent rsync log output inline with a link to the full log, and promote
failure and retention warning details out of the JSON payload so failed or slow
runs are easier to debug from the control panel.
2026-05-21 00:17:39 +02:00
d8c0ee5d1e Merge pull request '(ops) Complete native production install and update flow' (#9) from issue-1-production-install-update into master
Reviewed-on: #9
2026-05-20 01:50:23 +02:00
851f967f12 (ops) Expand native install self checks and recovery docs
Extend the runtime self check with native install diagnostics for the
environment file, service user, backup root ownership, and SQLite database
path. Export install metadata from the systemd units and pobsync-manage wrapper
so custom env files and service users are visible to Django checks.

Document restart, journal log inspection, and rollback steps in the README so
production updates have a clear recovery path.
2026-05-20 01:44:51 +02:00
c97c595253 (ops) Add terminal self-check command for native installs
Add check_pobsync_install so native deployments can run the same runtime
diagnostics from the terminal that are available in the Django Self Check view.

The command prints every check with status, returns a failing exit code when
install-critical checks fail, supports fail-on-warning for stricter automation,
and is documented in the installer output and README update flow.
2026-05-20 01:37:07 +02:00
f5acdf2fff (refactor) Remove obsolete source-copy deploy script
Delete the old scripts/deploy path that installed pobsync into /opt/pobsync/lib
with a standalone /opt/pobsync/bin wrapper.

Production deployment is now owned by the native systemd installer and updater,
so keeping the legacy deploy script would make the supported install story less
clear and easier to misuse.
2026-05-20 01:33:07 +02:00
e6ed7954de (ops) Add native update wrapper for production deploys
Add scripts/update-systemd as a safer routine deploy entrypoint for native
systemd installations. The wrapper keeps updates non-interactive, preserves the
existing environment file, skips OS package installation, and avoids superuser
creation prompts while still reusing the installer refresh flow.

Document the update path in the README and capture the maintenance expectation
in the development notes.
2026-05-20 01:30:02 +02:00
73e6bb7285 (ops) Install pobsync-manage for native management commands
Add a pobsync-manage wrapper that loads the native environment file before
running Django management commands, so production commands use the same
database and runtime settings as the systemd services.

Install the wrapper from the systemd installer, use it for migrations,
static collection, SSH key setup, and superuser creation, and document it
in the README for operational commands.
2026-05-20 01:27:08 +02:00
d451d01fe2 (docs) Move 1.0 roadmap tracking to Gitea
Create the 1.0 milestone and roadmap issues in Gitea, making Gitea the
leading tracker for release work.

Remove the repository roadmap document and README reference now that the
roadmap is tracked as milestone issues.
2026-05-20 01:17:35 +02:00
bfe17969e6 (docs) Add pobsync 1.0 roadmap
Document the remaining 1.0 work as ordered, reviewable chunks with
checkboxes for install/update flow, logging, preflight validation,
retention UX, dashboard polish, restore docs, config cleanup, and release
hardening.

Link the roadmap from the README and note how it can later be mirrored into
Gitea milestones and issues.
2026-05-20 01:07:17 +02:00
7cd87bc8a8 (ui) Remove global config count from dashboard summary
Drop the Global Configs metric from the dashboard because pobsync only uses
one default global configuration.

Keep global config management available through the existing dashboard
action and setup prompt.
2026-05-20 00:29:43 +02:00
0babc57f57 (feature) Link rsync logs from backup run detail
Record the final rsync log path for successful real backup runs, matching
the existing dry-run and failure result payloads.

Add a staff-only run log endpoint and surface the link on run detail pages,
including fallback log discovery for older runs based on snapshot_path.

Cover direct log links and inferred scheduled backup logs with view tests.
2026-05-20 00:09:59 +02:00
f41e59e695 (ui) Relax dashboard host card layout
Split each dashboard host card into a timeline area and a compact stats area
so snapshot, run, and schedule details have more room to scan.

Keep the same operational information visible while reducing cramped
column-like wrapping on desktop and mobile layouts.
2026-05-19 23:52:54 +02:00
f3dcb8a3b9 (ui) Replace dashboard host table with host cards
Restructure the dashboard Hosts section into a card per host so snapshot,
run, schedule, retention, and data metrics have room to breathe.

Add reusable host-card styling and keep the same links and operational data
available without forcing everything into a wide table.
2026-05-19 23:46:52 +02:00
c2d7342a47 (refactor) Remove unused schedule user field
Drop the legacy schedule user setting from the Django model, form, defaults,
and configure command.

Schedules are executed by the pobsync scheduler service under the configured
systemd service user, while remote SSH login users are configured separately
on global or host backup config.

Add a migration to remove the unused database column and update schedule
view tests around the simplified form.
2026-05-19 23:41:55 +02:00
5ca2733ea9 (bugfix) Prune snapshots with preserved read-only permissions
Make SQL retention delete the snapshot root when records point at the
snapshot data directory, matching how backup metadata is stored on disk.

Before removing a snapshot tree, temporarily add user write permission to
directories inside that snapshot so rsync-preserved source permissions do
not block cleanup.

Add a regression test for pruning snapshots whose data directory mirrors a
read-only remote root.
2026-05-19 23:29:36 +02:00
8bff241b12 (bugfix) Mark retention failures as backup warnings
Add a warning status for BackupRun records so successful snapshots are not
reported as failed when post-run SQL retention fails.

Keep the prune error in the run result, link the successful snapshot, and
let the management command complete with a warning instead of raising a
backup failure.

Include warning runs in backup trend summaries and add a regression test
for successful backups with failed retention cleanup.
2026-05-19 23:20:52 +02:00
1e04da9de8 (bugfix) Preserve existing schedule values in the edit form
Only apply default schedule initial values when creating a new schedule.

Avoid passing default initial data while editing an existing ScheduleConfig,
so the form renders the active cron-style expression, user, and retention
settings from the database.

Add a regression test that reopens an existing schedule and verifies the
stored values are shown instead of defaults.
2026-05-19 23:13:53 +02:00
75 changed files with 4140 additions and 704 deletions

37
CHANGELOG.md Normal file
View File

@@ -0,0 +1,37 @@
# Changelog
## 1.0.0 - 2026-05-21
Initial stable release of the Django-first pobsync control panel.
### Added
- Django control panel for hosts, global settings, schedules, SSH credentials, snapshots, runs, self-checks, and logs.
- Native systemd installer and updater for production backup servers.
- SQLite by default, with optional MariaDB support.
- Scheduler and worker services for queued manual backups and scheduled backups.
- Manual backup, dry-run, cancellation, verbose rsync logging, and run detail views.
- Snapshot discovery for existing backup directories and SQL-backed snapshot records.
- SQL retention planning and apply flow with base snapshot protection and incomplete snapshot visibility.
- Explicit cleanup flow for incomplete snapshots, separate from normal retention pruning.
- Purged snapshot audit overview with reason, action source, operator, host, kind, path, and timestamp.
- Dashboard and host pages with backup health, latest run/snapshot, next run, and storage/stat summaries.
- Review resolution for failed/warning runs and incomplete snapshot tasks so operational warnings can be acknowledged.
- Worker heartbeat metadata and stale running-run reconciliation for queued backup workers.
- SSH key generation, upload, edit, guarded delete, known_hosts management, and per-host key selection.
- In-app changelog page sourced from this changelog.
- Restore guidance on snapshot detail pages.
### Changed
- Django and the database are now the source of truth for configuration.
- Docker Compose is documented as development and disposable test tooling rather than the primary production path.
- The `pobsync` console entrypoint is now a maintainer layer around Django management commands.
- Scheduled pruning is evaluated by the pobsync scheduler service and recorded through Django, not host cron.
- Retention and incomplete cleanup now preserve audit history even after source snapshot records are removed.
### Removed
- Legacy YAML config import/export workflow.
- Public short aliases for configuration commands.
- Obsolete global config storage fields.

View File

@@ -10,7 +10,7 @@ RUN apt-get update \
WORKDIR /app WORKDIR /app
COPY pyproject.toml README.md ./ COPY pyproject.toml README.md CHANGELOG.md ./
COPY src ./src COPY src ./src
COPY manage.py ./ COPY manage.py ./
COPY scripts/docker-entrypoint ./scripts/docker-entrypoint COPY scripts/docker-entrypoint ./scripts/docker-entrypoint

View File

@@ -43,6 +43,7 @@ The installer will, by default:
- copy the checkout to `/opt/pobsync/app` - copy the checkout to `/opt/pobsync/app`
- create `/opt/pobsync/venv` - create `/opt/pobsync/venv`
- write `/etc/pobsync/pobsync.env` if it does not exist - write `/etc/pobsync/pobsync.env` if it does not exist
- install `pobsync-manage`, a Django management wrapper that loads `/etc/pobsync/pobsync.env`
- create `/var/lib/pobsync`, `/var/log/pobsync`, and the backup root - create `/var/lib/pobsync`, `/var/log/pobsync`, and the backup root
- install Python dependencies - install Python dependencies
- run migrations and collect static files - run migrations and collect static files
@@ -127,7 +128,16 @@ http://127.0.0.1:8010/
Create a superuser if needed: Create a superuser if needed:
``` ```
sudo -u pobsync /opt/pobsync/venv/bin/python /opt/pobsync/app/manage.py createsuperuser sudo -u pobsync pobsync-manage createsuperuser
```
For other Django management commands on native installs, use `pobsync-manage` so the production environment file is
loaded before Django starts:
```
sudo -u pobsync pobsync-manage showmigrations pobsync_backend
sudo -u pobsync pobsync-manage check
sudo -u pobsync pobsync-manage check_pobsync_install
``` ```
The UI includes: The UI includes:
@@ -144,11 +154,50 @@ The UI includes:
- `/self-check/` for runtime checks - `/self-check/` for runtime checks
- `/logs/` for filtered pobsync service logs - `/logs/` for filtered pobsync service logs
## Restoring Data
pobsync 1.0 treats restores as an explicit manual operation. The control panel shows restore guidance on each snapshot
detail page, but it does not run restore commands for you yet. That is deliberate: restores should be inspected and
tested before data is copied back into a live system.
Each snapshot directory contains:
```
<snapshot>/data/ # backed-up filesystem contents
<snapshot>/meta/ # metadata and rsync logs
```
Use the `data/` directory as the rsync source. Start with a dry run and restore to a staging path first:
```
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/ /restore/example.org/
rsync -aHAX --numeric-ids --info=progress2 /backups/example.org/scheduled/<snapshot>/data/ /restore/example.org/
```
After validating the staged files, copy the specific files or directories back to the target machine. For a full-host
restore, use another dry run before writing to the remote root:
```
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/ root@example.org:/
```
For most incidents, prefer a targeted restore instead of copying the whole snapshot. Keep paths relative to the
snapshot's `data/` directory:
```
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/etc/nginx/ /restore/example.org/etc/nginx/
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/home/example/site/public_html/index.php /restore/example.org/home/example/site/public_html/index.php
```
Snapshots may use hardlinks for files that are unchanged between backups. That saves disk space and is safe for normal
restore copies, but do not edit files inside snapshot directories. Treat snapshots as read-only and copy data out with
rsync.
## SSH Keys ## SSH Keys
SSH keys can be managed from `/ssh-credentials/`. The recommended flow is to generate keys from Django or during the SSH keys can be managed from `/ssh-credentials/`. The recommended flow is to generate keys from Django or during the
installer. pobsync stores the private key on disk under `POBSYNC_HOME`, keeps the public key visible in the UI, and lets installer. pobsync stores the private key on disk under the runtime state root (`POBSYNC_HOME`), keeps the public key
you select a credential either as the global default or as a per-host override. visible in the UI, and lets you select a credential either as the global default or as a per-host override.
Generated private keys are stored at: Generated private keys are stored at:
@@ -168,16 +217,52 @@ From a fresh checkout or the existing app directory:
``` ```
git pull git pull
sudo scripts/install-systemd --non-interactive sudo scripts/update-systemd
``` ```
The installer preserves an existing `/etc/pobsync/pobsync.env` unless you pass `--force-env`. It refreshes the installed The updater is a thin wrapper around the installer for normal production deploys. It preserves the existing
app, Python dependencies, migrations, static files, and restarts the systemd services so new Django code is loaded. `/etc/pobsync/pobsync.env`, skips OS package installation, skips superuser creation, refreshes the installed app, updates
Python dependencies, runs migrations, collects static files, and restarts the systemd services so new Django code is
loaded.
Use the full installer again when you intentionally want to change install-time settings, install OS packages, enable
nginx, or rewrite the environment file:
```
sudo scripts/install-systemd --non-interactive
sudo scripts/install-systemd --force-env
```
Then check: Then check:
``` ```
systemctl status pobsync-web pobsync-worker pobsync-scheduler systemctl status pobsync-web pobsync-worker pobsync-scheduler
sudo -u pobsync pobsync-manage check
sudo -u pobsync pobsync-manage check_pobsync_install
```
Restart services manually after environment or reverse proxy changes:
```
sudo systemctl restart pobsync-web pobsync-worker pobsync-scheduler
```
Inspect service logs with:
```
journalctl -u pobsync-web -n 100 --no-pager
journalctl -u pobsync-worker -f
journalctl -u pobsync-scheduler -n 100 --no-pager
```
Rollback to a previous revision by checking out the known-good commit or tag, then running the updater again:
```
git switch master
git pull
git checkout <known-good-commit-or-tag>
sudo scripts/update-systemd
sudo -u pobsync pobsync-manage check_pobsync_install
``` ```
## Development ## Development

24
deploy/bin/pobsync-manage Normal file
View File

@@ -0,0 +1,24 @@
#!/bin/sh
set -eu
APP_DIR="@POBSYNC_APP_DIR@"
VENV_DIR="@POBSYNC_VENV_DIR@"
ENV_FILE="@POBSYNC_ENV_FILE@"
SERVICE_USER="@POBSYNC_USER@"
SERVICE_GROUP="@POBSYNC_GROUP@"
if [ ! -f "$ENV_FILE" ]; then
echo "pobsync environment file not found: $ENV_FILE" >&2
exit 1
fi
set -a
# shellcheck disable=SC1090
. "$ENV_FILE"
set +a
export POBSYNC_ENV_FILE="$ENV_FILE"
export POBSYNC_SERVICE_USER="$SERVICE_USER"
export POBSYNC_SERVICE_GROUP="$SERVICE_GROUP"
cd "$APP_DIR"
exec "$VENV_DIR/bin/python" "$APP_DIR/manage.py" "$@"

View File

@@ -7,6 +7,9 @@ POBSYNC_HOME=/var/lib/pobsync
POBSYNC_BACKUP_ROOT=/backups POBSYNC_BACKUP_ROOT=/backups
POBSYNC_SQLITE_PATH=/var/lib/pobsync/pobsync.sqlite3 POBSYNC_SQLITE_PATH=/var/lib/pobsync/pobsync.sqlite3
POBSYNC_STATIC_ROOT=/var/lib/pobsync/static POBSYNC_STATIC_ROOT=/var/lib/pobsync/static
POBSYNC_ENV_FILE=/etc/pobsync/pobsync.env
POBSYNC_SERVICE_USER=pobsync
POBSYNC_SERVICE_GROUP=pobsync
POBSYNC_WEB_BIND=127.0.0.1:8010 POBSYNC_WEB_BIND=127.0.0.1:8010
POBSYNC_GUNICORN_WORKERS=2 POBSYNC_GUNICORN_WORKERS=2

View File

@@ -9,6 +9,9 @@ User=@POBSYNC_USER@
Group=@POBSYNC_GROUP@ Group=@POBSYNC_GROUP@
WorkingDirectory=@POBSYNC_APP_DIR@ WorkingDirectory=@POBSYNC_APP_DIR@
EnvironmentFile=@POBSYNC_ENV_FILE@ EnvironmentFile=@POBSYNC_ENV_FILE@
Environment=POBSYNC_ENV_FILE=@POBSYNC_ENV_FILE@
Environment=POBSYNC_SERVICE_USER=@POBSYNC_USER@
Environment=POBSYNC_SERVICE_GROUP=@POBSYNC_GROUP@
ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/python manage.py run_pobsync_scheduler --loop --interval "${POBSYNC_SCHEDULER_INTERVAL:-60}"' ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/python manage.py run_pobsync_scheduler --loop --interval "${POBSYNC_SCHEDULER_INTERVAL:-60}"'
Restart=on-failure Restart=on-failure
RestartSec=5 RestartSec=5

View File

@@ -9,6 +9,9 @@ User=@POBSYNC_USER@
Group=@POBSYNC_GROUP@ Group=@POBSYNC_GROUP@
WorkingDirectory=@POBSYNC_APP_DIR@ WorkingDirectory=@POBSYNC_APP_DIR@
EnvironmentFile=@POBSYNC_ENV_FILE@ EnvironmentFile=@POBSYNC_ENV_FILE@
Environment=POBSYNC_ENV_FILE=@POBSYNC_ENV_FILE@
Environment=POBSYNC_SERVICE_USER=@POBSYNC_USER@
Environment=POBSYNC_SERVICE_GROUP=@POBSYNC_GROUP@
ExecStartPre=@POBSYNC_VENV_DIR@/bin/python manage.py migrate --noinput ExecStartPre=@POBSYNC_VENV_DIR@/bin/python manage.py migrate --noinput
ExecStartPre=@POBSYNC_VENV_DIR@/bin/python manage.py collectstatic --noinput --clear ExecStartPre=@POBSYNC_VENV_DIR@/bin/python manage.py collectstatic --noinput --clear
ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/gunicorn pobsync_server.wsgi:application --bind "${POBSYNC_WEB_BIND:-127.0.0.1:8010}" --workers "${POBSYNC_GUNICORN_WORKERS:-2}" --timeout "${POBSYNC_GUNICORN_TIMEOUT:-120}"' ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/gunicorn pobsync_server.wsgi:application --bind "${POBSYNC_WEB_BIND:-127.0.0.1:8010}" --workers "${POBSYNC_GUNICORN_WORKERS:-2}" --timeout "${POBSYNC_GUNICORN_TIMEOUT:-120}"'

View File

@@ -9,6 +9,9 @@ User=@POBSYNC_USER@
Group=@POBSYNC_GROUP@ Group=@POBSYNC_GROUP@
WorkingDirectory=@POBSYNC_APP_DIR@ WorkingDirectory=@POBSYNC_APP_DIR@
EnvironmentFile=@POBSYNC_ENV_FILE@ EnvironmentFile=@POBSYNC_ENV_FILE@
Environment=POBSYNC_ENV_FILE=@POBSYNC_ENV_FILE@
Environment=POBSYNC_SERVICE_USER=@POBSYNC_USER@
Environment=POBSYNC_SERVICE_GROUP=@POBSYNC_GROUP@
ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/python manage.py run_pobsync_worker --loop --interval "${POBSYNC_WORKER_INTERVAL:-15}"' ExecStart=/bin/sh -c 'exec @POBSYNC_VENV_DIR@/bin/python manage.py run_pobsync_worker --loop --interval "${POBSYNC_WORKER_INTERVAL:-15}"'
Restart=on-failure Restart=on-failure
RestartSec=5 RestartSec=5

View File

@@ -47,6 +47,9 @@ pobsync django check
python3 manage.py showmigrations pobsync_backend python3 manage.py showmigrations pobsync_backend
``` ```
The short `pobsync` aliases are limited to operational actions that are useful while debugging a running install.
Configuration aliases are intentionally not public commands; use the Django UI or explicit management commands instead.
Worker and scheduler commands are normally run by systemd services: Worker and scheduler commands are normally run by systemd services:
``` ```
@@ -62,6 +65,14 @@ pobsync discover-snapshots --host <host>
pobsync retention <host> pobsync retention <host>
``` ```
For scripted configuration changes, call the Django management command explicitly so it is clear that this is an
automation/debugging path rather than the normal UI workflow:
```
pobsync django configure_pobsync_host <host> --address <host.example>
pobsync django configure_pobsync_schedule <host> --schedule-expression "15 2 * * *"
```
## Installer Development ## Installer Development
The native installer is interactive by default when stdin is a terminal. It should keep every prompt backed by a command The native installer is interactive by default when stdin is a terminal. It should keep every prompt backed by a command
@@ -74,28 +85,16 @@ sudo scripts/install-systemd
sudo scripts/install-systemd --non-interactive sudo scripts/install-systemd --non-interactive
sudo scripts/install-systemd --verbose sudo scripts/install-systemd --verbose
sudo scripts/install-systemd --create-superuser --superuser-username admin sudo scripts/install-systemd --create-superuser --superuser-username admin
sudo scripts/update-systemd
``` ```
The installer should print a short completion summary with the control panel URL, Self Check reminder, and service log The installer should print a short completion summary with the control panel URL, Self Check reminder, and service log
commands. Keep normal output user-facing: pobsync step names with OK, FAILED, or SKIPPED. Full apt, pip, Django, and commands. Keep normal output user-facing: pobsync step names with OK, FAILED, or SKIPPED. Full apt, pip, Django, and
systemd output belongs behind `--verbose` or in the failed step output. systemd output belongs behind `--verbose` or in the failed step output.
## Migration Helpers The updater is intentionally a small wrapper around the installer for routine production deploys. It should stay
non-interactive, preserve the existing environment file, skip OS package installation, skip superuser creation, and still
Import existing legacy YAML configs: run the Django/runtime refresh steps needed after a code update.
```
python3 manage.py import_pobsync_configs --prefix /opt/pobsync
```
Export SQL config to legacy runtime YAML for inspection or one-off compatibility:
```
python3 manage.py export_pobsync_configs --prefix /opt/pobsync
```
These commands are migration helpers, not the normal operating model. After import, review and continue operating from
the Django control panel.
## Docker With SQLite ## Docker With SQLite
@@ -176,4 +175,3 @@ Next refactor targets:
- Move more snapshot lifecycle details into typed domain objects. - Move more snapshot lifecycle details into typed domain objects.
- Replace remaining dictionary-shaped config at engine boundaries. - Replace remaining dictionary-shaped config at engine boundaries.
- Remove legacy YAML import/export once production migration no longer needs it.

View File

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
[project] [project]
name = "pobsync" name = "pobsync"
version = "0.1.0" version = "1.0.0"
description = "Pull-based rsync backup tool with hardlinked snapshots" description = "Pull-based rsync backup tool with hardlinked snapshots"
requires-python = ">=3.11" requires-python = ">=3.11"
dependencies = [ dependencies = [

View File

@@ -1,97 +0,0 @@
#!/bin/sh
# Deploy pobsync runtime into /opt/pobsync without pip/venv.
# Copies python package sources into /opt/pobsync/lib and installs a stable entrypoint in /opt/pobsync/bin.
set -eu
PREFIX="/opt/pobsync"
usage() {
echo "Usage: $0 [--prefix /opt/pobsync]" >&2
exit 2
}
while [ $# -gt 0 ]; do
case "$1" in
--prefix)
[ $# -ge 2 ] || usage
PREFIX="$2"
shift 2
;;
-h|--help)
usage
;;
*)
echo "Unknown arg: $1" >&2
usage
;;
esac
done
# Determine repo root from this script location
SCRIPT_DIR="$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)"
REPO_ROOT="$(CDPATH= cd -- "${SCRIPT_DIR}/.." && pwd)"
SRC_PKG="${REPO_ROOT}/src/pobsync"
if [ ! -d "${SRC_PKG}" ]; then
echo "ERROR: expected python package at ${SRC_PKG}" >&2
exit 1
fi
BIN_DIR="${PREFIX}/bin"
LIB_DIR="${PREFIX}/lib"
DST_PKG="${LIB_DIR}/pobsync"
BUILD_FILE="${DST_PKG}/_build.txt"
mkdir -p "${BIN_DIR}" "${LIB_DIR}"
# Copy code into /opt/pobsync/lib/pobsync
# We use rsync if available (clean updates with --delete), otherwise fall back to cp -a.
if command -v rsync >/dev/null 2>&1; then
rsync -a --delete \
--exclude '__pycache__/' \
--exclude '*.pyc' \
--exclude '*.pyo' \
--exclude '*.pyd' \
"${SRC_PKG}/" "${DST_PKG}/"
else
# Fallback: wipe + copy
rm -rf "${DST_PKG}"
mkdir -p "${DST_PKG}"
cp -a "${SRC_PKG}/." "${DST_PKG}/"
fi
# Write build info (best-effort)
GIT_SHA="unknown"
if command -v git >/dev/null 2>&1 && [ -d "${REPO_ROOT}/.git" ]; then
GIT_SHA="$(cd "${REPO_ROOT}" && git rev-parse HEAD 2>/dev/null || echo unknown)"
fi
NOW_UTC="$(date -u +"%Y-%m-%dT%H:%M:%SZ" 2>/dev/null || echo unknown)"
{
echo "deployed_at_utc=${NOW_UTC}"
echo "git_sha=${GIT_SHA}"
echo "repo_root=${REPO_ROOT}"
} > "${BUILD_FILE}"
# Install stable entrypoint that always runs code from /opt/pobsync/lib
WRAPPER="${BIN_DIR}/pobsync"
cat > "${WRAPPER}" <<EOF
#!/bin/sh
# managed-by=pobsync deploy
set -eu
PREFIX="${PREFIX}"
export PYTHONPATH="\${PREFIX}/lib"
export PYTHONUNBUFFERED=1
exec /usr/bin/python3 -m pobsync "\$@"
EOF
chmod 0755 "${WRAPPER}"
echo "OK"
echo "- deployed package to ${DST_PKG}"
echo "- wrote build info ${BUILD_FILE}"
echo "- installed entrypoint ${WRAPPER}"

View File

@@ -463,6 +463,9 @@ POBSYNC_BACKUP_ROOT=$BACKUP_ROOT
POBSYNC_TIME_ZONE=$TIME_ZONE POBSYNC_TIME_ZONE=$TIME_ZONE
POBSYNC_SQLITE_PATH=/var/lib/pobsync/pobsync.sqlite3 POBSYNC_SQLITE_PATH=/var/lib/pobsync/pobsync.sqlite3
POBSYNC_STATIC_ROOT=/var/lib/pobsync/static POBSYNC_STATIC_ROOT=/var/lib/pobsync/static
POBSYNC_ENV_FILE=$ENV_FILE
POBSYNC_SERVICE_USER=$SERVICE_USER
POBSYNC_SERVICE_GROUP=$SERVICE_GROUP
POBSYNC_WEB_BIND=$WEB_BIND POBSYNC_WEB_BIND=$WEB_BIND
POBSYNC_GUNICORN_WORKERS=2 POBSYNC_GUNICORN_WORKERS=2
@@ -504,10 +507,23 @@ install_units() {
run_step "Install systemd units" install_units run_step "Install systemd units" install_units
install_manage_wrapper() {
sed \
-e "s|@POBSYNC_APP_DIR@|$APP_DIR|g" \
-e "s|@POBSYNC_VENV_DIR@|$VENV_DIR|g" \
-e "s|@POBSYNC_ENV_FILE@|$ENV_FILE|g" \
-e "s|@POBSYNC_USER@|$SERVICE_USER|g" \
-e "s|@POBSYNC_GROUP@|$SERVICE_GROUP|g" \
"$APP_DIR/deploy/bin/pobsync-manage" > /usr/local/bin/pobsync-manage
chmod 0755 /usr/local/bin/pobsync-manage
}
run_step "Install manage wrapper" install_manage_wrapper
run_step "Reload systemd" systemctl daemon-reload run_step "Reload systemd" systemctl daemon-reload
run_step "Run database migrations" "$VENV_DIR/bin/python" "$APP_DIR/manage.py" migrate --noinput run_step "Run database migrations" /usr/local/bin/pobsync-manage migrate --noinput
run_step "Ensure default SSH key" "$VENV_DIR/bin/python" "$APP_DIR/manage.py" ensure_pobsync_ssh_key --name default --set-global-default run_step "Ensure default SSH key" /usr/local/bin/pobsync-manage ensure_pobsync_ssh_key --name default --set-global-default
run_step "Collect static files" "$VENV_DIR/bin/python" "$APP_DIR/manage.py" collectstatic --noinput --clear run_step "Collect static files" /usr/local/bin/pobsync-manage collectstatic --noinput --clear
run_step "Finalize state permissions" chown -R "$SERVICE_USER:$SERVICE_GROUP" /var/lib/pobsync /var/log/pobsync run_step "Finalize state permissions" chown -R "$SERVICE_USER:$SERVICE_GROUP" /var/lib/pobsync /var/log/pobsync
superuser_exists=$("$VENV_DIR/bin/python" -c "import os; os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'pobsync_server.settings'); import django; django.setup(); from django.contrib.auth import get_user_model; print('yes' if get_user_model().objects.filter(is_superuser=True).exists() else 'no')") superuser_exists=$("$VENV_DIR/bin/python" -c "import os; os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'pobsync_server.settings'); import django; django.setup(); from django.contrib.auth import get_user_model; print('yes' if get_user_model().objects.filter(is_superuser=True).exists() else 'no')")
@@ -519,17 +535,17 @@ if [ "$CREATE_SUPERUSER" -eq 1 ]; then
DJANGO_SUPERUSER_USERNAME="$SUPERUSER_USERNAME" \ DJANGO_SUPERUSER_USERNAME="$SUPERUSER_USERNAME" \
DJANGO_SUPERUSER_EMAIL="$SUPERUSER_EMAIL" \ DJANGO_SUPERUSER_EMAIL="$SUPERUSER_EMAIL" \
DJANGO_SUPERUSER_PASSWORD="$SUPERUSER_PASSWORD" \ DJANGO_SUPERUSER_PASSWORD="$SUPERUSER_PASSWORD" \
"$VENV_DIR/bin/python" "$APP_DIR/manage.py" createsuperuser --noinput /usr/local/bin/pobsync-manage createsuperuser --noinput
run_step "Finalize superuser permissions" chown -R "$SERVICE_USER:$SERVICE_GROUP" /var/lib/pobsync /var/log/pobsync run_step "Finalize superuser permissions" chown -R "$SERVICE_USER:$SERVICE_GROUP" /var/lib/pobsync /var/log/pobsync
else else
note_step "Create Django superuser" "SKIPPED" note_step "Create Django superuser" "SKIPPED"
echo "No superuser password was provided; create one later with:" echo "No superuser password was provided; create one later with:"
echo " sudo -u $SERVICE_USER $VENV_DIR/bin/python $APP_DIR/manage.py createsuperuser" echo " sudo -u $SERVICE_USER pobsync-manage createsuperuser"
fi fi
elif [ "$superuser_exists" != "yes" ]; then elif [ "$superuser_exists" != "yes" ]; then
note_step "Create Django superuser" "SKIPPED" note_step "Create Django superuser" "SKIPPED"
echo "No Django superuser exists yet. Create one with:" echo "No Django superuser exists yet. Create one with:"
echo " sudo -u $SERVICE_USER $VENV_DIR/bin/python $APP_DIR/manage.py createsuperuser" echo " sudo -u $SERVICE_USER pobsync-manage createsuperuser"
else else
note_step "Create Django superuser" "SKIPPED" note_step "Create Django superuser" "SKIPPED"
fi fi
@@ -574,3 +590,5 @@ echo
echo "Useful commands:" echo "Useful commands:"
echo " systemctl status pobsync-web pobsync-worker pobsync-scheduler" echo " systemctl status pobsync-web pobsync-worker pobsync-scheduler"
echo " journalctl -u pobsync-worker -f" echo " journalctl -u pobsync-worker -f"
echo " sudo -u $SERVICE_USER pobsync-manage check"
echo " sudo -u $SERVICE_USER pobsync-manage check_pobsync_install"

41
scripts/update-systemd Executable file
View File

@@ -0,0 +1,41 @@
#!/bin/sh
set -eu
SCRIPT_DIR=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
show_help() {
cat <<'EOF'
Usage: sudo scripts/update-systemd [options]
Refresh an existing native pobsync systemd install from the current checkout.
This is a thin, safer update wrapper around scripts/install-systemd. It keeps
the install non-interactive, preserves the existing environment file, skips
superuser creation, and skips OS package installation by default.
Common options are forwarded to install-systemd, for example:
--source-dir PATH
--app-dir PATH
--venv-dir PATH
--env-file PATH
--service-user USER
--service-group GROUP
--install-extras mariadb
--verbose
If OS packages need to be refreshed, run scripts/install-systemd directly.
EOF
}
case "${1:-}" in
-h|--help)
show_help
exit 0
;;
esac
exec "$SCRIPT_DIR/install-systemd" \
--non-interactive \
--no-install-os-packages \
--no-create-superuser \
"$@"

View File

@@ -1,3 +1,2 @@
__all__ = ["__version__"] __all__ = ["__version__"]
__version__ = "0.1.0" __version__ = "1.0.0"

View File

@@ -6,11 +6,10 @@ from typing import Sequence
from django.core.management import execute_from_command_line from django.core.management import execute_from_command_line
from pobsync import __version__
COMMAND_ALIASES = { COMMAND_ALIASES = {
"configure-global": "configure_pobsync_global",
"configure-host": "configure_pobsync_host",
"schedule": "configure_pobsync_schedule",
"backup": "run_pobsync_backup", "backup": "run_pobsync_backup",
"retention": "run_pobsync_retention", "retention": "run_pobsync_retention",
"discover-snapshots": "discover_pobsync_snapshots", "discover-snapshots": "discover_pobsync_snapshots",
@@ -29,11 +28,17 @@ Usage:
Commands: Commands:
{commands} {commands}
Configuration is managed from the Django control panel. Use
`pobsync django <management-command>` for automation or debugging.
""" """
def main(argv: Sequence[str] | None = None) -> int: def main(argv: Sequence[str] | None = None) -> int:
args = list(sys.argv[1:] if argv is None else argv) args = list(sys.argv[1:] if argv is None else argv)
if args and args[0] in {"--version", "version"}:
print(f"pobsync {__version__}")
return 0
if not args or args[0] in {"-h", "--help", "help"}: if not args or args[0] in {"-h", "--help", "help"}:
print(_usage()) print(_usage())
return 0 return 0

View File

@@ -4,9 +4,8 @@ from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from typing import Any, List from typing import Any, List
from ..config.source import ConfigSource, FileConfigSource from ..config.source import ConfigSource
from ..errors import ConfigError from ..errors import ConfigError
from ..paths import PobsyncPaths
from ..retention import Snapshot, apply_base_protection, build_retention_plan from ..retention import Snapshot, apply_base_protection, build_retention_plan
from ..snapshot_meta import iter_snapshot_dirs, read_snapshot_meta, resolve_host_root from ..snapshot_meta import iter_snapshot_dirs, read_snapshot_meta, resolve_host_root
from ..util import sanitize_host from ..util import sanitize_host
@@ -40,10 +39,9 @@ def run_retention_plan(
if kind not in {"scheduled", "manual", "all"}: if kind not in {"scheduled", "manual", "all"}:
raise ConfigError("kind must be scheduled, manual, or all") raise ConfigError("kind must be scheduled, manual, or all")
paths = PobsyncPaths(home=prefix) if config_source is None:
raise ConfigError("A Django config source is required.")
source = config_source or FileConfigSource(prefix=paths.home) cfg = config_source.effective_config_for_host(host)
cfg = source.effective_config_for_host(host)
retention = cfg.get("retention") retention = cfg.get("retention")
if not isinstance(retention, dict): if not isinstance(retention, dict):

View File

@@ -3,7 +3,7 @@ from __future__ import annotations
from pathlib import Path from pathlib import Path
from typing import Any, Callable from typing import Any, Callable
from ..config.source import ConfigSource, FileConfigSource from ..config.source import ConfigSource
from ..errors import ConfigError from ..errors import ConfigError
from ..lock import acquire_host_lock from ..lock import acquire_host_lock
from ..paths import PobsyncPaths from ..paths import PobsyncPaths
@@ -163,8 +163,9 @@ def run_scheduled(
host = sanitize_host(host) host = sanitize_host(host)
paths = PobsyncPaths(home=prefix) paths = PobsyncPaths(home=prefix)
source = config_source or FileConfigSource(prefix=paths.home) if config_source is None:
cfg = source.effective_config_for_host(host) raise ConfigError("A Django config source is required.")
cfg = config_source.effective_config_for_host(host)
backup_root = cfg.get("backup_root") backup_root = cfg.get("backup_root")
if not isinstance(backup_root, str) or not backup_root.startswith("/"): if not isinstance(backup_root, str) or not backup_root.startswith("/"):
@@ -316,7 +317,6 @@ def run_scheduled(
"duration_seconds": None, "duration_seconds": None,
"base": _base_meta_from_path(base_dir, link_dest), "base": _base_meta_from_path(base_dir, link_dest),
"rsync": {"exit_code": None, "command": cmd, "stats": {}}, "rsync": {"exit_code": None, "command": cmd, "stats": {}},
# Keep existing fields for future expansion / compatibility with current structure.
"overrides": {"includes": [], "excludes": [], "base": None}, "overrides": {"includes": [], "excludes": [], "base": None},
} }
@@ -403,6 +403,7 @@ def run_scheduled(
"host": host, "host": host,
"snapshot": str(final_dir), "snapshot": str(final_dir),
"base": str(base_dir) if base_dir else None, "base": str(base_dir) if base_dir else None,
"log": str(final_log_path),
"rsync": {"exit_code": result.exit_code}, "rsync": {"exit_code": result.exit_code},
"verbose_output": bool(verbose_output), "verbose_output": bool(verbose_output),
"duration_seconds": meta["duration_seconds"], "duration_seconds": meta["duration_seconds"],

View File

@@ -1,53 +0,0 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
import yaml
from ..errors import ConfigError, ValidationError
from ..validate import validate_dict
from .schemas import GLOBAL_SCHEMA, HOST_SCHEMA
def load_yaml_file(path: Path) -> dict[str, Any]:
if not path.exists():
raise ConfigError(f"Missing config file: {path}")
try:
raw = path.read_text(encoding="utf-8")
except OSError as e:
raise ConfigError(f"Cannot read config file: {path}: {e}") from e
try:
data = yaml.safe_load(raw)
except yaml.YAMLError as e:
raise ConfigError(f"Invalid YAML in {path}: {e}") from e
if data is None:
data = {}
if not isinstance(data, dict):
raise ConfigError(f"Config root must be a mapping in {path}")
return data
def load_global_config(path: Path) -> dict[str, Any]:
data = load_yaml_file(path)
try:
return validate_dict(data, GLOBAL_SCHEMA, path="global")
except ValidationError as e:
raise ConfigError(f"Invalid global config at {path}: {format_validation_error(e)}") from e
def load_host_config(path: Path) -> dict[str, Any]:
data = load_yaml_file(path)
try:
return validate_dict(data, HOST_SCHEMA, path="host")
except ValidationError as e:
raise ConfigError(f"Invalid host config at {path}: {format_validation_error(e)}") from e
def format_validation_error(err: ValidationError) -> str:
if err.path:
return f"{err.path}: {err}"
return str(err)

View File

@@ -83,7 +83,6 @@ OUTPUT_SCHEMA = Schema(
GLOBAL_SCHEMA = Schema( GLOBAL_SCHEMA = Schema(
fields={ fields={
"backup_root": FieldSpec(str, required=True), "backup_root": FieldSpec(str, required=True),
"pobsync_home": FieldSpec(str, required=False, default="/opt/pobsync"),
"ssh": FieldSpec(dict, required=False, schema=SSH_SCHEMA), "ssh": FieldSpec(dict, required=False, schema=SSH_SCHEMA),
"rsync": FieldSpec(dict, required=False, schema=RSYNC_SCHEMA), "rsync": FieldSpec(dict, required=False, schema=RSYNC_SCHEMA),
"defaults": FieldSpec(dict, required=False, schema=DEFAULTS_SCHEMA), "defaults": FieldSpec(dict, required=False, schema=DEFAULTS_SCHEMA),
@@ -95,7 +94,6 @@ GLOBAL_SCHEMA = Schema(
), ),
"logging": FieldSpec(dict, required=False, schema=LOGGING_SCHEMA), "logging": FieldSpec(dict, required=False, schema=LOGGING_SCHEMA),
"output": FieldSpec(dict, required=False, schema=OUTPUT_SCHEMA), "output": FieldSpec(dict, required=False, schema=OUTPUT_SCHEMA),
# Used by `init-host` as a convenience default
"retention_defaults": FieldSpec( "retention_defaults": FieldSpec(
dict, dict,
required=False, required=False,
@@ -131,4 +129,3 @@ HOST_SCHEMA = Schema(
}, },
allow_unknown=False, allow_unknown=False,
) )

View File

@@ -1,22 +1,8 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path
from typing import Any, Protocol from typing import Any, Protocol
from .load import load_global_config, load_host_config
from .merge import build_effective_config
class ConfigSource(Protocol): class ConfigSource(Protocol):
def effective_config_for_host(self, host: str) -> dict[str, Any]: def effective_config_for_host(self, host: str) -> dict[str, Any]:
"""Return the fully merged effective config for a host.""" """Return the fully merged effective config for a host."""
class FileConfigSource:
def __init__(self, prefix: Path) -> None:
self.prefix = prefix
def effective_config_for_host(self, host: str) -> dict[str, Any]:
global_cfg = load_global_config(self.prefix / "config" / "global.yaml")
host_cfg = load_host_config(self.prefix / "config" / "hosts" / f"{host}.yaml")
return build_effective_config(global_cfg, host_cfg)

View File

@@ -8,14 +8,6 @@ from pathlib import Path
class PobsyncPaths: class PobsyncPaths:
home: Path # usually /opt/pobsync home: Path # usually /opt/pobsync
@property
def config_dir(self) -> Path:
return self.home / "config"
@property
def hosts_dir(self) -> Path:
return self.config_dir / "hosts"
@property @property
def state_dir(self) -> Path: def state_dir(self) -> Path:
return self.home / "state" return self.home / "state"
@@ -28,11 +20,6 @@ class PobsyncPaths:
def logs_dir(self) -> Path: def logs_dir(self) -> Path:
return self.home / "logs" return self.home / "logs"
@property
def global_config_path(self) -> Path:
return self.config_dir / "global.yaml"
@property @property
def central_log_path(self) -> Path: def central_log_path(self) -> Path:
return self.logs_dir / "pobsync.log" return self.logs_dir / "pobsync.log"

View File

@@ -6,7 +6,7 @@ from django.urls import reverse
from django.utils.html import format_html from django.utils.html import format_html
from django.utils.http import urlencode from django.utils.http import urlencode
from .models import BackupRun, GlobalConfig, HostConfig, ScheduleConfig, SnapshotRecord, SshCredential from .models import BackupRun, GlobalConfig, HostConfig, PurgedSnapshot, ScheduleConfig, SnapshotRecord, SshCredential
@admin.register(SshCredential) @admin.register(SshCredential)
@@ -34,7 +34,7 @@ class GlobalConfigAdmin(admin.ModelAdmin):
list_display = ("name", "backup_root", "ssh_user", "ssh_port", "updated_at") list_display = ("name", "backup_root", "ssh_user", "ssh_port", "updated_at")
readonly_fields = ("created_at", "updated_at") readonly_fields = ("created_at", "updated_at")
fieldsets = ( fieldsets = (
(None, {"fields": ("name", "backup_root", "pobsync_home")}), (None, {"fields": ("name", "backup_root")}),
("SSH", {"fields": ("default_ssh_credential", "ssh_user", "ssh_port", "ssh_options")}), ("SSH", {"fields": ("default_ssh_credential", "ssh_user", "ssh_port", "ssh_options")}),
( (
"Rsync", "Rsync",
@@ -50,7 +50,6 @@ class GlobalConfigAdmin(admin.ModelAdmin):
), ),
("Defaults", {"fields": ("default_source_root", "default_destination_subdir", "excludes_default")}), ("Defaults", {"fields": ("default_source_root", "default_destination_subdir", "excludes_default")}),
("Retention defaults", {"fields": ("retention_daily", "retention_weekly", "retention_monthly", "retention_yearly")}), ("Retention defaults", {"fields": ("retention_daily", "retention_weekly", "retention_monthly", "retention_yearly")}),
("Legacy JSON", {"fields": ("data",), "classes": ("collapse",)}),
("Timestamps", {"fields": ("created_at", "updated_at"), "classes": ("collapse",)}), ("Timestamps", {"fields": ("created_at", "updated_at"), "classes": ("collapse",)}),
) )
@@ -76,7 +75,7 @@ class HostConfigAdmin(admin.ModelAdmin):
("Source", {"fields": ("source_root", "includes", "excludes_add", "excludes_replace")}), ("Source", {"fields": ("source_root", "includes", "excludes_add", "excludes_replace")}),
("Rsync override", {"fields": ("rsync_extra_args",)}), ("Rsync override", {"fields": ("rsync_extra_args",)}),
("Retention", {"fields": ("retention_daily", "retention_weekly", "retention_monthly", "retention_yearly")}), ("Retention", {"fields": ("retention_daily", "retention_weekly", "retention_monthly", "retention_yearly")}),
("Legacy JSON", {"fields": ("config",), "classes": ("collapse",)}), ("Runtime state", {"fields": ("config",), "classes": ("collapse",)}),
("Timestamps", {"fields": ("created_at", "updated_at"), "classes": ("collapse",)}), ("Timestamps", {"fields": ("created_at", "updated_at"), "classes": ("collapse",)}),
) )
@@ -174,6 +173,16 @@ class SnapshotRecordAdmin(admin.ModelAdmin):
return format_html('<a href="{}">{}</a>', url, count) return format_html('<a href="{}">{}</a>', url, count)
@admin.register(PurgedSnapshot)
class PurgedSnapshotAdmin(admin.ModelAdmin):
list_display = ("host_name", "kind", "dirname", "action", "reason", "triggered_by", "purged_at")
list_filter = ("action", "kind", "purged_at")
search_fields = ("host_name", "dirname", "path", "reason", "triggered_by")
list_select_related = ("host",)
readonly_fields = ("purged_at",)
date_hierarchy = "purged_at"
@admin.register(ScheduleConfig) @admin.register(ScheduleConfig)
class ScheduleConfigAdmin(admin.ModelAdmin): class ScheduleConfigAdmin(admin.ModelAdmin):
list_display = ("host", "cron_expr", "enabled", "prune", "last_status", "last_started_at", "updated_at") list_display = ("host", "cron_expr", "enabled", "prune", "last_status", "last_started_at", "updated_at")

View File

@@ -1,6 +1,8 @@
from __future__ import annotations from __future__ import annotations
from datetime import timedelta import os
import socket
from datetime import timedelta, timezone as datetime_timezone
from pathlib import Path from pathlib import Path
from django.db import transaction from django.db import transaction
@@ -107,11 +109,12 @@ def execute_backup_run(
protect_bases=bool(prune_protect_bases), protect_bases=bool(prune_protect_bases),
yes=True, yes=True,
max_delete=int(prune_max_delete), max_delete=int(prune_max_delete),
action=run.run_type,
acquire_lock=False, acquire_lock=False,
) )
except Exception as exc: except Exception as exc:
result["prune"] = {"ok": False, "error": str(exc), "type": type(exc).__name__} result["prune"] = {"ok": False, "error": str(exc), "type": type(exc).__name__}
run.status = BackupRun.Status.FAILED run.status = BackupRun.Status.WARNING
run.result = result run.result = result
run.snapshot = snapshot_record run.snapshot = snapshot_record
run.save( run.save(
@@ -125,7 +128,6 @@ def execute_backup_run(
"result", "result",
], ],
) )
raise
run.snapshot = snapshot_record run.snapshot = snapshot_record
run.result = result run.result = result
@@ -159,10 +161,10 @@ def claim_next_queued_run() -> BackupRun | None:
return run return run
def reconcile_running_runs(*, grace_seconds: int = 300) -> int: def reconcile_running_runs(*, grace_seconds: int = 300, stale_worker_seconds: int = 24 * 60 * 60) -> int:
reconciled = 0 reconciled = 0
for run in BackupRun.objects.select_related("host").filter(status=BackupRun.Status.RUNNING).order_by("started_at", "id"): for run in BackupRun.objects.select_related("host").filter(status=BackupRun.Status.RUNNING).order_by("started_at", "id"):
if _reconcile_running_run(run=run, grace_seconds=grace_seconds): if _reconcile_running_run(run=run, grace_seconds=grace_seconds, stale_worker_seconds=stale_worker_seconds):
reconciled += 1 reconciled += 1
return reconciled return reconciled
@@ -177,7 +179,9 @@ def requested_options(run: BackupRun) -> dict[str, object]:
def _running_result(*, run: BackupRun, dry_run: bool) -> dict[str, object]: def _running_result(*, run: BackupRun, dry_run: bool) -> dict[str, object]:
result = dict(run.result) if isinstance(run.result, dict) else {} result = dict(run.result) if isinstance(run.result, dict) else {}
execution = { execution = {
**_worker_execution_details(),
"started_at": (run.started_at or timezone.now()).isoformat(), "started_at": (run.started_at or timezone.now()).isoformat(),
"heartbeat_at": timezone.now().isoformat(),
} }
if dry_run: if dry_run:
execution["log"] = str(dry_run_log_path(run.host.host, run_id=run.id)) execution["log"] = str(dry_run_log_path(run.host.host, run_id=run.id))
@@ -186,24 +190,56 @@ def _running_result(*, run: BackupRun, dry_run: bool) -> dict[str, object]:
def _run_cancel_requested(run_id: int) -> bool: def _run_cancel_requested(run_id: int) -> bool:
return BackupRun.objects.filter(id=run_id, status=BackupRun.Status.CANCELLED).exists() try:
run = BackupRun.objects.only("id", "status", "result").get(id=run_id)
except BackupRun.DoesNotExist:
return True
if run.status == BackupRun.Status.CANCELLED:
return True
if run.status == BackupRun.Status.RUNNING:
_refresh_run_heartbeat(run)
return False
def _reconcile_running_run(*, run: BackupRun, grace_seconds: int) -> bool: def _reconcile_running_run(*, run: BackupRun, grace_seconds: int, stale_worker_seconds: int) -> bool:
result = run.result if isinstance(run.result, dict) else {} result = run.result if isinstance(run.result, dict) else {}
requested = result.get("requested") if isinstance(result.get("requested"), dict) else {} requested = result.get("requested") if isinstance(result.get("requested"), dict) else {}
stale_worker = _running_worker_timed_out(run=run, stale_worker_seconds=stale_worker_seconds)
if not requested.get("dry_run"): if not requested.get("dry_run"):
if stale_worker:
result.update(
{
"ok": False,
"host": run.host.host,
"failure": {
"category": "worker",
"message": "The worker heartbeat stopped before the run finished.",
"hint": "Check pobsync-worker.service logs before retrying the backup.",
},
}
)
run.status = BackupRun.Status.FAILED
run.ended_at = timezone.now()
run.result = result
run.save(update_fields=["status", "ended_at", "result"])
return True
return False return False
log_path = _execution_log_path(result) log_path = _execution_log_path(result)
log_tail = _read_log_tail(log_path) if log_path is not None else [] log_tail = _read_log_tail(log_path) if log_path is not None else []
terminal_log = _terminal_rsync_log(log_tail) terminal_log = _terminal_rsync_log(log_tail)
timed_out = _running_dry_run_timed_out(run=run, grace_seconds=grace_seconds) timed_out = _running_dry_run_timed_out(run=run, grace_seconds=grace_seconds)
if not terminal_log and not timed_out: if not terminal_log and not timed_out and not stale_worker:
return False return False
exit_code = _exit_code_from_log(log_tail) or (124 if timed_out else 255) exit_code = _exit_code_from_log(log_tail) or (124 if timed_out or stale_worker else 255)
failure = classify_rsync_failure(exit_code, log_tail) failure = classify_rsync_failure(exit_code, log_tail)
if stale_worker and not terminal_log:
failure = {
"category": "worker",
"message": "The worker heartbeat stopped before the dry-run finished.",
"hint": "Check pobsync-worker.service logs before retrying the dry-run.",
}
result.update( result.update(
{ {
"ok": False, "ok": False,
@@ -227,6 +263,30 @@ def _reconcile_running_run(*, run: BackupRun, grace_seconds: int) -> bool:
return True return True
def _worker_execution_details() -> dict[str, object]:
return {
"worker_pid": os.getpid(),
"worker_host": socket.gethostname(),
"claimed_at": timezone.now().isoformat(),
}
def _refresh_run_heartbeat(run: BackupRun, *, interval_seconds: int = 30) -> None:
result = run.result if isinstance(run.result, dict) else {}
execution = result.get("execution") if isinstance(result.get("execution"), dict) else {}
heartbeat_at = _parse_iso_datetime(execution.get("heartbeat_at"))
if heartbeat_at is not None and timezone.now() < heartbeat_at + timedelta(seconds=interval_seconds):
return
result["execution"] = {
**execution,
"worker_pid": os.getpid(),
"worker_host": socket.gethostname(),
"heartbeat_at": timezone.now().isoformat(),
}
run.result = result
run.save(update_fields=["result"])
def _execution_log_path(result: dict[str, object]) -> Path | None: def _execution_log_path(result: dict[str, object]) -> Path | None:
execution = result.get("execution") if isinstance(result.get("execution"), dict) else {} execution = result.get("execution") if isinstance(result.get("execution"), dict) else {}
log = execution.get("log") or result.get("log") log = execution.get("log") or result.get("log")
@@ -267,3 +327,28 @@ def _running_dry_run_timed_out(*, run: BackupRun, grace_seconds: int) -> bool:
if not isinstance(timeout_seconds, int) or timeout_seconds <= 0: if not isinstance(timeout_seconds, int) or timeout_seconds <= 0:
timeout_seconds = DEFAULT_DRY_RUN_TIMEOUT_SECONDS timeout_seconds = DEFAULT_DRY_RUN_TIMEOUT_SECONDS
return timezone.now() >= run.started_at + timedelta(seconds=timeout_seconds + grace_seconds) return timezone.now() >= run.started_at + timedelta(seconds=timeout_seconds + grace_seconds)
def _running_worker_timed_out(*, run: BackupRun, stale_worker_seconds: int) -> bool:
if stale_worker_seconds <= 0:
return False
result = run.result if isinstance(run.result, dict) else {}
execution = result.get("execution") if isinstance(result.get("execution"), dict) else {}
heartbeat_at = _parse_iso_datetime(execution.get("heartbeat_at"))
if heartbeat_at is None:
heartbeat_at = run.started_at
if heartbeat_at is None:
return False
return timezone.now() >= heartbeat_at + timedelta(seconds=stale_worker_seconds)
def _parse_iso_datetime(value: object):
if not isinstance(value, str) or not value:
return None
try:
parsed = timezone.datetime.fromisoformat(value)
except ValueError:
return None
if timezone.is_naive(parsed):
return timezone.make_aware(parsed, timezone=datetime_timezone.utc)
return parsed

View File

@@ -17,7 +17,7 @@ CRITICAL_ROOT_EXCLUDES = ("/proc/***", "/sys/***", "/dev/***", "/run/***", "/tmp
def collect_global_config_checks(global_config: GlobalConfig) -> list[SelfCheck]: def collect_global_config_checks(global_config: GlobalConfig) -> list[SelfCheck]:
checks = [ checks = [
_absolute_path_check("Global backup root", global_config.backup_root), _absolute_path_check("Global backup root", global_config.backup_root),
_absolute_path_check("Global pobsync home", global_config.pobsync_home), _absolute_path_check("Runtime state root", settings.POBSYNC_HOME),
_runtime_backup_root_check(global_config), _runtime_backup_root_check(global_config),
_rsync_binary_check(global_config.rsync_binary), _rsync_binary_check(global_config.rsync_binary),
_rsync_recursion_check( _rsync_recursion_check(
@@ -97,7 +97,7 @@ def _runtime_backup_root_check(global_config: GlobalConfig) -> SelfCheck:
return SelfCheck( return SelfCheck(
"Runtime backup root", "Runtime backup root",
"warning", "warning",
"Database backup root differs from runtime POBSYNC_BACKUP_ROOT.", "Database backup root differs from the runtime backup root.",
f"database={global_config.backup_root} runtime={settings.POBSYNC_BACKUP_ROOT}", f"database={global_config.backup_root} runtime={settings.POBSYNC_BACKUP_ROOT}",
) )

View File

@@ -1,13 +1,10 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path
from typing import Any from typing import Any
from django.core.exceptions import ObjectDoesNotExist from django.core.exceptions import ObjectDoesNotExist
from pobsync.config.schemas import GLOBAL_SCHEMA, HOST_SCHEMA from pobsync.config.schemas import GLOBAL_SCHEMA, HOST_SCHEMA
from pobsync.paths import PobsyncPaths
from pobsync.util import write_yaml_atomic
from pobsync.validate import validate_dict from pobsync.validate import validate_dict
from .models import GlobalConfig, HostConfig from .models import GlobalConfig, HostConfig
@@ -17,10 +14,9 @@ class ConfigRepositoryError(RuntimeError):
pass pass
def _global_yaml_data(global_config: GlobalConfig) -> dict[str, Any]: def _global_runtime_data(global_config: GlobalConfig) -> dict[str, Any]:
data = { data = {
"backup_root": global_config.backup_root, "backup_root": global_config.backup_root,
"pobsync_home": global_config.pobsync_home,
"ssh": { "ssh": {
"user": global_config.ssh_user, "user": global_config.ssh_user,
"port": global_config.ssh_port, "port": global_config.ssh_port,
@@ -48,7 +44,7 @@ def _global_yaml_data(global_config: GlobalConfig) -> dict[str, Any]:
return validate_dict(data, GLOBAL_SCHEMA, path="global") return validate_dict(data, GLOBAL_SCHEMA, path="global")
def _host_yaml_data(host_config: HostConfig) -> dict[str, Any]: def _host_runtime_data(host_config: HostConfig) -> dict[str, Any]:
data: dict[str, Any] = { data: dict[str, Any] = {
"host": host_config.host, "host": host_config.host,
"address": host_config.address, "address": host_config.address,
@@ -77,50 +73,25 @@ def _host_yaml_data(host_config: HostConfig) -> dict[str, Any]:
return validate_dict(data, HOST_SCHEMA, path="host") return validate_dict(data, HOST_SCHEMA, path="host")
def global_config_object_data(global_config: GlobalConfig) -> dict[str, Any]:
return _global_runtime_data(global_config)
def host_config_object_data(host_config: HostConfig) -> dict[str, Any]:
return _host_runtime_data(host_config)
def global_config_data(name: str = "default") -> dict[str, Any]: def global_config_data(name: str = "default") -> dict[str, Any]:
try: try:
global_config = GlobalConfig.objects.get(name=name) global_config = GlobalConfig.objects.get(name=name)
except ObjectDoesNotExist as exc: except ObjectDoesNotExist as exc:
raise ConfigRepositoryError(f"Missing GlobalConfig {name!r}") from exc raise ConfigRepositoryError(f"Missing global config {name!r}") from exc
return _global_yaml_data(global_config) return _global_runtime_data(global_config)
def host_config_data(host: str) -> dict[str, Any]: def host_config_data(host: str) -> dict[str, Any]:
try: try:
host_config = HostConfig.objects.get(host=host, enabled=True) host_config = HostConfig.objects.get(host=host, enabled=True)
except ObjectDoesNotExist as exc: except ObjectDoesNotExist as exc:
raise ConfigRepositoryError(f"Missing enabled HostConfig {host!r}") from exc raise ConfigRepositoryError(f"Missing enabled host {host!r}") from exc
return _host_yaml_data(host_config) return _host_runtime_data(host_config)
def export_global_config(prefix: Path, name: str = "default") -> Path:
try:
global_config = GlobalConfig.objects.get(name=name)
except ObjectDoesNotExist as exc:
raise ConfigRepositoryError(f"Missing GlobalConfig {name!r}") from exc
paths = PobsyncPaths(home=prefix)
write_yaml_atomic(paths.global_config_path, _global_yaml_data(global_config))
return paths.global_config_path
def export_host_config(prefix: Path, host: str) -> Path:
try:
host_config = HostConfig.objects.get(host=host, enabled=True)
except ObjectDoesNotExist as exc:
raise ConfigRepositoryError(f"Missing enabled HostConfig {host!r}") from exc
paths = PobsyncPaths(home=prefix)
target = paths.hosts_dir / f"{host_config.host}.yaml"
write_yaml_atomic(target, _host_yaml_data(host_config))
return target
def export_runtime_configs(prefix: Path, host: str | None = None) -> list[Path]:
written = [export_global_config(prefix)]
hosts = HostConfig.objects.filter(enabled=True).order_by("host")
if host is not None:
hosts = hosts.filter(host=host)
for host_config in hosts:
written.append(export_host_config(prefix, host_config.host))
return written

View File

@@ -119,7 +119,6 @@ class GlobalConfigForm(forms.ModelForm):
def save(self, commit: bool = True): def save(self, commit: bool = True):
instance = super().save(commit=False) instance = super().save(commit=False)
instance.backup_root = settings.POBSYNC_BACKUP_ROOT instance.backup_root = settings.POBSYNC_BACKUP_ROOT
instance.pobsync_home = settings.POBSYNC_HOME
if commit: if commit:
instance.save() instance.save()
self.save_m2m() self.save_m2m()
@@ -249,12 +248,18 @@ class RetentionApplyForm(forms.Form):
kind = forms.ChoiceField(choices=(("scheduled", "Scheduled"), ("manual", "Manual"), ("all", "All"))) kind = forms.ChoiceField(choices=(("scheduled", "Scheduled"), ("manual", "Manual"), ("all", "All")))
protect_bases = forms.BooleanField(required=False) protect_bases = forms.BooleanField(required=False)
max_delete = forms.IntegerField(min_value=0, initial=10) max_delete = forms.IntegerField(min_value=0, initial=10)
confirm_delete_count = forms.IntegerField(min_value=0)
confirm_host = forms.CharField() confirm_host = forms.CharField()
def __init__(self, *args, host_name: str, **kwargs) -> None: def __init__(self, *args, host_name: str, expected_delete_count: int | None = None, **kwargs) -> None:
self.host_name = host_name self.host_name = host_name
self.expected_delete_count = expected_delete_count
super().__init__(*args, **kwargs) super().__init__(*args, **kwargs)
self.fields["confirm_host"].help_text = f"Type {host_name} to confirm deletion." self.fields["confirm_host"].help_text = f"Type {host_name} to confirm deletion."
if expected_delete_count is not None:
self.fields["confirm_delete_count"].help_text = (
f"Type {expected_delete_count} to confirm the current number of planned deletions."
)
def clean_confirm_host(self) -> str: def clean_confirm_host(self) -> str:
value = self.cleaned_data["confirm_host"].strip() value = self.cleaned_data["confirm_host"].strip()
@@ -262,6 +267,42 @@ class RetentionApplyForm(forms.Form):
raise forms.ValidationError(f"Type {self.host_name} to confirm.") raise forms.ValidationError(f"Type {self.host_name} to confirm.")
return value return value
def clean_confirm_delete_count(self) -> int:
value = self.cleaned_data["confirm_delete_count"]
if self.expected_delete_count is not None and value != self.expected_delete_count:
raise forms.ValidationError(f"Type {self.expected_delete_count} to confirm the delete count.")
return value
class IncompleteCleanupForm(forms.Form):
max_delete = forms.IntegerField(min_value=0, initial=0)
confirm_delete_count = forms.IntegerField(min_value=0)
confirm_host = forms.CharField()
def __init__(self, *args, host_name: str, expected_delete_count: int, **kwargs) -> None:
self.host_name = host_name
self.expected_delete_count = expected_delete_count
super().__init__(*args, **kwargs)
self.fields["confirm_host"].help_text = f"Type {host_name} to confirm incomplete snapshot cleanup."
self.fields["confirm_delete_count"].help_text = (
f"Type {expected_delete_count} to confirm the current number of incomplete snapshots."
)
self.fields["max_delete"].help_text = (
f"Must be at least {expected_delete_count} for the incomplete snapshots shown here."
)
def clean_confirm_host(self) -> str:
value = self.cleaned_data["confirm_host"].strip()
if value != self.host_name:
raise forms.ValidationError(f"Type {self.host_name} to confirm.")
return value
def clean_confirm_delete_count(self) -> int:
value = self.cleaned_data["confirm_delete_count"]
if value != self.expected_delete_count:
raise forms.ValidationError(f"Type {self.expected_delete_count} to confirm the incomplete count.")
return value
class ScheduleConfigForm(forms.ModelForm): class ScheduleConfigForm(forms.ModelForm):
cron_expr = forms.CharField( cron_expr = forms.CharField(
@@ -277,7 +318,6 @@ class ScheduleConfigForm(forms.ModelForm):
model = ScheduleConfig model = ScheduleConfig
fields = ( fields = (
"cron_expr", "cron_expr",
"user",
"enabled", "enabled",
"prune", "prune",
"prune_max_delete", "prune_max_delete",

View File

@@ -0,0 +1,44 @@
from __future__ import annotations
from django.core.management.base import BaseCommand, CommandError
from pobsync_backend.self_check import collect_self_checks, summarize_self_checks
class Command(BaseCommand):
help = "Run pobsync runtime self checks for native installs and updates."
def add_arguments(self, parser):
parser.add_argument(
"--fail-on-warning",
action="store_true",
help="Exit with an error when warnings are present.",
)
def handle(self, *args, **options):
checks = collect_self_checks()
summary = summarize_self_checks(checks)
for check in checks:
line = f"[{check.status.upper()}] {check.name}: {check.message}"
if check.detail:
line = f"{line} ({check.detail})"
if check.status == "failed":
self.stderr.write(line)
elif check.status == "warning":
self.stderr.write(line)
else:
self.stdout.write(line)
self.stdout.write(
"Summary: "
f"{summary['ok']} ok, "
f"{summary['warning']} warning(s), "
f"{summary['failed']} failed, "
f"{summary['skipped']} skipped"
)
if summary["failed"]:
raise CommandError("pobsync install self check failed.")
if options["fail_on_warning"] and summary["warning"]:
raise CommandError("pobsync install self check reported warnings.")

View File

@@ -1,9 +1,7 @@
from __future__ import annotations from __future__ import annotations
from pathlib import Path
from typing import Any from typing import Any
from django.conf import settings
from django.core.management.base import BaseCommand, CommandError from django.core.management.base import BaseCommand, CommandError
from pobsync.config.retention import parse_retention from pobsync.config.retention import parse_retention
@@ -13,12 +11,11 @@ from pobsync_backend.models import GlobalConfig
class Command(BaseCommand): class Command(BaseCommand):
help = "Create or update the SQL-backed global pobsync configuration." help = "Create or update the default global backup configuration."
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("--name", default="default") parser.add_argument("--name", default="default")
parser.add_argument("--backup-root", required=True) parser.add_argument("--backup-root", required=True)
parser.add_argument("--pobsync-home", default=settings.POBSYNC_HOME)
parser.add_argument("--ssh-user", default="root") parser.add_argument("--ssh-user", default="root")
parser.add_argument("--ssh-port", type=int, default=22) parser.add_argument("--ssh-port", type=int, default=22)
parser.add_argument("--source-root", default="/") parser.add_argument("--source-root", default="/")
@@ -30,11 +27,9 @@ class Command(BaseCommand):
if not is_absolute_non_root(backup_root): if not is_absolute_non_root(backup_root):
raise CommandError("--backup-root must be an absolute path and must not be '/'") raise CommandError("--backup-root must be an absolute path and must not be '/'")
pobsync_home = str(Path(options["pobsync_home"]))
retention = parse_retention(options["retention"]) retention = parse_retention(options["retention"])
defaults = { defaults = {
"backup_root": backup_root, "backup_root": backup_root,
"pobsync_home": pobsync_home,
"ssh_user": options["ssh_user"], "ssh_user": options["ssh_user"],
"ssh_port": options["ssh_port"], "ssh_port": options["ssh_port"],
"ssh_options": ["-oBatchMode=yes", "-oStrictHostKeyChecking=accept-new"], "ssh_options": ["-oBatchMode=yes", "-oStrictHostKeyChecking=accept-new"],
@@ -53,8 +48,8 @@ class Command(BaseCommand):
} }
if GlobalConfig.objects.filter(name=options["name"]).exists() and not options["force"]: if GlobalConfig.objects.filter(name=options["name"]).exists() and not options["force"]:
raise CommandError(f"GlobalConfig {options['name']!r} already exists; use --force to update") raise CommandError(f"Global config {options['name']!r} already exists; use --force to update")
_obj, created = GlobalConfig.objects.update_or_create(name=options["name"], defaults=defaults) _obj, created = GlobalConfig.objects.update_or_create(name=options["name"], defaults=defaults)
action = "Created" if created else "Updated" action = "Created" if created else "Updated"
self.stdout.write(self.style.SUCCESS(f"{action} GlobalConfig {options['name']!r}.")) self.stdout.write(self.style.SUCCESS(f"{action} global config {options['name']!r}."))

View File

@@ -10,7 +10,7 @@ from pobsync_backend.models import GlobalConfig, HostConfig
class Command(BaseCommand): class Command(BaseCommand):
help = "Create or update a SQL-backed host pobsync configuration." help = "Create or update a host backup configuration."
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("host") parser.add_argument("host")
@@ -29,7 +29,7 @@ class Command(BaseCommand):
def handle(self, *args: Any, **options: Any) -> None: def handle(self, *args: Any, **options: Any) -> None:
host = sanitize_host(options["host"]) host = sanitize_host(options["host"])
if HostConfig.objects.filter(host=host).exists() and not options["force"]: if HostConfig.objects.filter(host=host).exists() and not options["force"]:
raise CommandError(f"HostConfig {host!r} already exists; use --force to update") raise CommandError(f"Host {host!r} already exists; use --force to update")
retention = self._retention(options["retention"]) retention = self._retention(options["retention"])
defaults = { defaults = {
@@ -49,7 +49,7 @@ class Command(BaseCommand):
} }
_obj, created = HostConfig.objects.update_or_create(host=host, defaults=defaults) _obj, created = HostConfig.objects.update_or_create(host=host, defaults=defaults)
action = "Created" if created else "Updated" action = "Created" if created else "Updated"
self.stdout.write(self.style.SUCCESS(f"{action} HostConfig {host!r}.")) self.stdout.write(self.style.SUCCESS(f"{action} host {host!r}."))
def _retention(self, value: str | None) -> dict[str, int]: def _retention(self, value: str | None) -> dict[str, int]:
if value: if value:

View File

@@ -9,12 +9,16 @@ from pobsync_backend.scheduler import parse_cron_expr
class Command(BaseCommand): class Command(BaseCommand):
help = "Create, update, disable, or remove a SQL-backed pobsync schedule." help = "Create, update, disable, or remove a scheduler-managed host schedule."
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("host") parser.add_argument("host")
parser.add_argument("--cron", help='Cron expression, e.g. "15 2 * * *"') parser.add_argument(
parser.add_argument("--user", default="root") "--schedule-expression",
"--cron",
dest="schedule_expression",
help='Five-field schedule expression, e.g. "15 2 * * *"',
)
parser.add_argument("--prune", action="store_true") parser.add_argument("--prune", action="store_true")
parser.add_argument("--prune-max-delete", type=int, default=10) parser.add_argument("--prune-max-delete", type=int, default=10)
parser.add_argument("--prune-protect-bases", action="store_true") parser.add_argument("--prune-protect-bases", action="store_true")
@@ -25,25 +29,25 @@ class Command(BaseCommand):
try: try:
host = HostConfig.objects.get(host=options["host"]) host = HostConfig.objects.get(host=options["host"])
except HostConfig.DoesNotExist as exc: except HostConfig.DoesNotExist as exc:
raise CommandError(f"Missing HostConfig {options['host']!r}") from exc raise CommandError(f"Missing host {options['host']!r}") from exc
if options["delete"]: if options["delete"]:
deleted, _details = ScheduleConfig.objects.filter(host=host).delete() deleted, _details = ScheduleConfig.objects.filter(host=host).delete()
self.stdout.write(self.style.SUCCESS(f"Deleted {deleted} schedule row(s) for {host.host!r}.")) self.stdout.write(self.style.SUCCESS(f"Deleted {deleted} schedule row(s) for {host.host!r}."))
return return
if not options["cron"]: schedule_expression = options["schedule_expression"]
raise CommandError("--cron is required unless --delete is used") if not schedule_expression:
raise CommandError("--schedule-expression is required unless --delete is used")
try: try:
parse_cron_expr(options["cron"]) parse_cron_expr(schedule_expression)
except ValueError as exc: except ValueError as exc:
raise CommandError(str(exc)) from exc raise CommandError(str(exc)) from exc
schedule, created = ScheduleConfig.objects.update_or_create( schedule, created = ScheduleConfig.objects.update_or_create(
host=host, host=host,
defaults={ defaults={
"cron_expr": options["cron"], "cron_expr": schedule_expression,
"user": options["user"],
"enabled": not options["disabled"], "enabled": not options["disabled"],
"prune": bool(options["prune"]), "prune": bool(options["prune"]),
"prune_max_delete": int(options["prune_max_delete"]), "prune_max_delete": int(options["prune_max_delete"]),

View File

@@ -20,14 +20,14 @@ class Command(BaseCommand):
try: try:
global_config = GlobalConfig.objects.get(name="default") global_config = GlobalConfig.objects.get(name="default")
except GlobalConfig.DoesNotExist as exc: except GlobalConfig.DoesNotExist as exc:
raise CommandError("Missing GlobalConfig 'default'") from exc raise CommandError("Missing default global config") from exc
host = None host = None
if options["host"]: if options["host"]:
try: try:
host = HostConfig.objects.get(host=options["host"], enabled=True) host = HostConfig.objects.get(host=options["host"], enabled=True)
except HostConfig.DoesNotExist as exc: except HostConfig.DoesNotExist as exc:
raise CommandError(f"Missing enabled HostConfig {options['host']!r}") from exc raise CommandError(f"Missing enabled host {options['host']!r}") from exc
kind = normalize_kind(options["kind"]) kind = normalize_kind(options["kind"])
kinds = ["scheduled", "manual", "incomplete"] if kind == "all" else [kind] kinds = ["scheduled", "manual", "incomplete"] if kind == "all" else [kind]

View File

@@ -1,23 +0,0 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
from django.conf import settings
from django.core.management.base import BaseCommand
from pobsync_backend.config_repository import export_runtime_configs
class Command(BaseCommand):
help = "Export Django database configs to pobsync runtime YAML files."
def add_arguments(self, parser) -> None:
parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Pobsync home directory")
parser.add_argument("--host", default=None, help="Export only one enabled host")
def handle(self, *args: Any, **options: Any) -> None:
written = export_runtime_configs(prefix=Path(options["prefix"]), host=options["host"])
for path in written:
self.stdout.write(str(path))
self.stdout.write(self.style.SUCCESS(f"Exported {len(written)} config file(s)."))

View File

@@ -1,81 +0,0 @@
from __future__ import annotations
from pathlib import Path
from typing import Any
from django.conf import settings
from django.core.management.base import BaseCommand, CommandError
from pobsync.config.load import load_global_config, load_host_config
from pobsync.paths import PobsyncPaths
from pobsync_backend.models import GlobalConfig, HostConfig
class Command(BaseCommand):
help = "Import pobsync YAML configs into the Django database."
def add_arguments(self, parser) -> None:
parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Pobsync home directory")
def handle(self, *args: Any, **options: Any) -> None:
paths = PobsyncPaths(home=Path(options["prefix"]))
if not paths.global_config_path.exists():
raise CommandError(f"Missing global config: {paths.global_config_path}")
global_cfg = load_global_config(paths.global_config_path)
global_ssh = global_cfg.get("ssh") or {}
global_rsync = global_cfg.get("rsync") or {}
global_defaults = global_cfg.get("defaults") or {}
retention_defaults = global_cfg.get("retention_defaults") or {}
GlobalConfig.objects.update_or_create(
name="default",
defaults={
"backup_root": global_cfg["backup_root"],
"pobsync_home": global_cfg.get("pobsync_home", str(paths.home)),
"ssh_user": global_ssh.get("user") or "root",
"ssh_port": global_ssh.get("port") or 22,
"ssh_options": global_ssh.get("options") or [],
"rsync_binary": global_rsync.get("binary") or "rsync",
"rsync_args": global_rsync.get("args") or [],
"rsync_extra_args": global_rsync.get("extra_args") or [],
"rsync_timeout_seconds": global_rsync.get("timeout_seconds") or 0,
"rsync_bwlimit_kbps": global_rsync.get("bwlimit_kbps") or 0,
"default_source_root": global_defaults.get("source_root") or "/",
"default_destination_subdir": global_defaults.get("destination_subdir") or "",
"excludes_default": global_cfg.get("excludes_default") or [],
"retention_daily": retention_defaults.get("daily", 14),
"retention_weekly": retention_defaults.get("weekly", 8),
"retention_monthly": retention_defaults.get("monthly", 12),
"retention_yearly": retention_defaults.get("yearly", 0),
"data": global_cfg,
},
)
count = 0
for host_path in sorted(paths.hosts_dir.glob("*.yaml")):
host_cfg = load_host_config(host_path)
host_ssh = host_cfg.get("ssh") or {}
host_rsync = host_cfg.get("rsync") or {}
host_retention = host_cfg.get("retention") or {}
HostConfig.objects.update_or_create(
host=host_cfg["host"],
defaults={
"address": host_cfg["address"],
"ssh_user": host_ssh.get("user") or "",
"ssh_port": host_ssh.get("port"),
"source_root": host_cfg.get("source_root") or "",
"includes": host_cfg.get("includes") or [],
"excludes_add": host_cfg.get("excludes_add") or [],
"excludes_replace": host_cfg.get("excludes_replace"),
"rsync_extra_args": host_rsync.get("extra_args") or [],
"retention_daily": host_retention.get("daily", 14),
"retention_weekly": host_retention.get("weekly", 8),
"retention_monthly": host_retention.get("monthly", 12),
"retention_yearly": host_retention.get("yearly", 0),
"config": host_cfg,
"enabled": True,
},
)
count += 1
self.stdout.write(self.style.SUCCESS(f"Imported global config and {count} host config(s)."))

View File

@@ -16,7 +16,7 @@ class Command(BaseCommand):
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("host", help="Host to back up") parser.add_argument("host", help="Host to back up")
parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Pobsync home directory") parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Runtime state root")
parser.add_argument("--dry-run", action="store_true", help="Run rsync --dry-run") parser.add_argument("--dry-run", action="store_true", help="Run rsync --dry-run")
parser.add_argument("--verbose-rsync", action="store_true", help="Write itemized rsync output to the run log") parser.add_argument("--verbose-rsync", action="store_true", help="Write itemized rsync output to the run log")
parser.add_argument("--prune", action="store_true", help="Apply retention after a successful run") parser.add_argument("--prune", action="store_true", help="Apply retention after a successful run")
@@ -30,7 +30,7 @@ class Command(BaseCommand):
try: try:
host = HostConfig.objects.get(host=host_name, enabled=True) host = HostConfig.objects.get(host=host_name, enabled=True)
except HostConfig.DoesNotExist as exc: except HostConfig.DoesNotExist as exc:
raise CommandError(f"Missing enabled HostConfig {host_name!r}") from exc raise CommandError(f"Missing enabled host {host_name!r}") from exc
run = BackupRun.objects.create( run = BackupRun.objects.create(
host=host, host=host,
@@ -60,5 +60,8 @@ class Command(BaseCommand):
if run.status == BackupRun.Status.SUCCESS: if run.status == BackupRun.Status.SUCCESS:
self.stdout.write(self.style.SUCCESS(f"Backup completed for {host.host}.")) self.stdout.write(self.style.SUCCESS(f"Backup completed for {host.host}."))
return return
if run.status == BackupRun.Status.WARNING:
self.stdout.write(self.style.WARNING(f"Backup completed with warnings for {host.host}; run id={run.id}"))
return
raise CommandError(f"Backup failed for {host.host}; run id={run.id}") raise CommandError(f"Backup failed for {host.host}; run id={run.id}")

View File

@@ -12,11 +12,11 @@ from pobsync_backend.retention import run_sql_retention_apply, run_sql_retention
class Command(BaseCommand): class Command(BaseCommand):
help = "Plan or apply retention using SQL-backed pobsync configuration." help = "Plan or apply retention using the Django backup configuration."
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("host") parser.add_argument("host")
parser.add_argument("--prefix", default=settings.POBSYNC_HOME) parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Runtime state root")
parser.add_argument("--kind", default="scheduled", choices=["scheduled", "manual", "all"]) parser.add_argument("--kind", default="scheduled", choices=["scheduled", "manual", "all"])
parser.add_argument("--protect-bases", action="store_true") parser.add_argument("--protect-bases", action="store_true")
parser.add_argument("--apply", action="store_true") parser.add_argument("--apply", action="store_true")
@@ -36,6 +36,7 @@ class Command(BaseCommand):
protect_bases=bool(options["protect_bases"]), protect_bases=bool(options["protect_bases"]),
yes=True, yes=True,
max_delete=int(options["max_delete"]), max_delete=int(options["max_delete"]),
action="cli",
) )
else: else:
result = run_sql_retention_plan( result = run_sql_retention_plan(

View File

@@ -10,7 +10,7 @@ from django.core.management.base import BaseCommand
from django.db import transaction from django.db import transaction
from django.utils import timezone from django.utils import timezone
from pobsync_backend.models import ScheduleConfig from pobsync_backend.models import BackupRun, ScheduleConfig
from pobsync_backend.scheduler import due_key, is_due from pobsync_backend.scheduler import due_key, is_due
@@ -18,7 +18,7 @@ class Command(BaseCommand):
help = "Run due pobsync schedules from the Django database." help = "Run due pobsync schedules from the Django database."
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Pobsync home directory") parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Runtime state root")
parser.add_argument("--once", action="store_true", help="Check once and exit") parser.add_argument("--once", action="store_true", help="Check once and exit")
parser.add_argument("--loop", action="store_true", help="Keep checking schedules") parser.add_argument("--loop", action="store_true", help="Keep checking schedules")
parser.add_argument("--interval", type=int, default=60, help="Loop interval in seconds") parser.add_argument("--interval", type=int, default=60, help="Loop interval in seconds")
@@ -52,12 +52,13 @@ class Command(BaseCommand):
if not is_due(schedule.cron_expr, now): if not is_due(schedule.cron_expr, now):
continue continue
schedule_started_at = timezone.now()
with transaction.atomic(): with transaction.atomic():
locked = ScheduleConfig.objects.select_for_update().get(pk=schedule.pk) locked = ScheduleConfig.objects.select_for_update().get(pk=schedule.pk)
if locked.last_due_key == current_due_key: if locked.last_due_key == current_due_key:
continue continue
locked.last_due_key = current_due_key locked.last_due_key = current_due_key
locked.last_started_at = timezone.now() locked.last_started_at = schedule_started_at
locked.last_status = "running" locked.last_status = "running"
locked.save(update_fields=["last_due_key", "last_started_at", "last_status", "updated_at"]) locked.save(update_fields=["last_due_key", "last_started_at", "last_status", "updated_at"])
@@ -72,6 +73,7 @@ class Command(BaseCommand):
prune_max_delete=schedule.prune_max_delete, prune_max_delete=schedule.prune_max_delete,
prune_protect_bases=schedule.prune_protect_bases, prune_protect_bases=schedule.prune_protect_bases,
) )
status = _latest_scheduled_run_status(host_id=schedule.host_id, started_at=schedule_started_at) or status
except Exception as exc: except Exception as exc:
status = "failed" status = "failed"
self.stderr.write(f"{schedule.host.host}: {type(exc).__name__}: {exc}") self.stderr.write(f"{schedule.host.host}: {type(exc).__name__}: {exc}")
@@ -83,3 +85,16 @@ class Command(BaseCommand):
ran += 1 ran += 1
return ran return ran
def _latest_scheduled_run_status(*, host_id: int, started_at) -> str | None:
run = (
BackupRun.objects.filter(
host_id=host_id,
run_type=BackupRun.RunType.SCHEDULED,
created_at__gte=started_at,
)
.order_by("-created_at", "-id")
.first()
)
return run.status if run is not None else None

View File

@@ -15,10 +15,16 @@ class Command(BaseCommand):
help = "Run queued pobsync backup jobs from the Django database." help = "Run queued pobsync backup jobs from the Django database."
def add_arguments(self, parser) -> None: def add_arguments(self, parser) -> None:
parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Pobsync home directory") parser.add_argument("--prefix", default=settings.POBSYNC_HOME, help="Runtime state root")
parser.add_argument("--once", action="store_true", help="Process one queued run and exit") parser.add_argument("--once", action="store_true", help="Process one queued run and exit")
parser.add_argument("--loop", action="store_true", help="Keep checking for queued runs") parser.add_argument("--loop", action="store_true", help="Keep checking for queued runs")
parser.add_argument("--interval", type=int, default=15, help="Loop interval in seconds") parser.add_argument("--interval", type=int, default=15, help="Loop interval in seconds")
parser.add_argument(
"--stale-running-seconds",
type=int,
default=24 * 60 * 60,
help="Mark running runs failed after this many seconds without a worker heartbeat; use 0 to disable",
)
def handle(self, *args: Any, **options: Any) -> None: def handle(self, *args: Any, **options: Any) -> None:
if not options["once"] and not options["loop"]: if not options["once"] and not options["loop"]:
@@ -26,14 +32,14 @@ class Command(BaseCommand):
paths = PobsyncPaths(home=Path(options["prefix"])) paths = PobsyncPaths(home=Path(options["prefix"]))
while True: while True:
count = self._run_once(prefix=paths.home) count = self._run_once(prefix=paths.home, stale_running_seconds=int(options["stale_running_seconds"]))
self.stdout.write(f"Ran {count} queued backup run(s).") self.stdout.write(f"Ran {count} queued backup run(s).")
if options["once"]: if options["once"]:
return return
time.sleep(max(1, int(options["interval"]))) time.sleep(max(1, int(options["interval"])))
def _run_once(self, *, prefix: Path) -> int: def _run_once(self, *, prefix: Path, stale_running_seconds: int = 24 * 60 * 60) -> int:
reconciled = reconcile_running_runs() reconciled = reconcile_running_runs(stale_worker_seconds=stale_running_seconds)
run = claim_next_queued_run() run = claim_next_queued_run()
if run is None: if run is None:
return reconciled return reconciled

View File

@@ -0,0 +1,27 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("pobsync_backend", "0007_filesystem_ssh_credentials"),
]
operations = [
migrations.AlterField(
model_name="backuprun",
name="status",
field=models.CharField(
choices=[
("queued", "Queued"),
("running", "Running"),
("success", "Success"),
("warning", "Warning"),
("failed", "Failed"),
("cancelled", "Cancelled"),
],
default="queued",
max_length=16,
),
),
]

View File

@@ -0,0 +1,14 @@
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("pobsync_backend", "0008_alter_backuprun_status"),
]
operations = [
migrations.RemoveField(
model_name="scheduleconfig",
name="user",
),
]

View File

@@ -0,0 +1,14 @@
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("pobsync_backend", "0009_remove_scheduleconfig_user"),
]
operations = [
migrations.RemoveField(
model_name="globalconfig",
name="pobsync_home",
),
]

View File

@@ -0,0 +1,14 @@
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
("pobsync_backend", "0010_remove_globalconfig_pobsync_home"),
]
operations = [
migrations.RemoveField(
model_name="globalconfig",
name="data",
),
]

View File

@@ -0,0 +1,30 @@
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("pobsync_backend", "0011_remove_globalconfig_data"),
]
operations = [
migrations.AddField(
model_name="backuprun",
name="reviewed_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddField(
model_name="backuprun",
name="reviewed_by",
field=models.CharField(blank=True, max_length=150),
),
migrations.AddField(
model_name="snapshotrecord",
name="reviewed_at",
field=models.DateTimeField(blank=True, null=True),
),
migrations.AddField(
model_name="snapshotrecord",
name="reviewed_by",
field=models.CharField(blank=True, max_length=150),
),
]

View File

@@ -0,0 +1,50 @@
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
("pobsync_backend", "0012_review_state"),
]
operations = [
migrations.CreateModel(
name="PurgedSnapshot",
fields=[
("id", models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name="ID")),
("host_name", models.CharField(max_length=255)),
("kind", models.CharField(max_length=16)),
("dirname", models.CharField(max_length=255)),
("path", models.CharField(max_length=1024)),
("reason", models.CharField(blank=True, max_length=512)),
(
"action",
models.CharField(
choices=[
("manual", "Manual"),
("scheduled", "Scheduled"),
("cli", "CLI"),
("incomplete_cleanup", "Incomplete cleanup"),
],
max_length=32,
),
),
("triggered_by", models.CharField(blank=True, max_length=150)),
("metadata", models.JSONField(blank=True, default=dict)),
("purged_at", models.DateTimeField(auto_now_add=True)),
(
"host",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="purged_snapshots",
to="pobsync_backend.hostconfig",
),
),
],
options={
"ordering": ["-purged_at", "host_name", "dirname"],
},
),
]

View File

@@ -14,7 +14,6 @@ class TimestampedModel(models.Model):
class GlobalConfig(TimestampedModel): class GlobalConfig(TimestampedModel):
name = models.CharField(max_length=64, default="default", unique=True) name = models.CharField(max_length=64, default="default", unique=True)
backup_root = models.CharField(max_length=512) backup_root = models.CharField(max_length=512)
pobsync_home = models.CharField(max_length=512, default="/opt/pobsync")
default_ssh_credential = models.ForeignKey( default_ssh_credential = models.ForeignKey(
"SshCredential", "SshCredential",
on_delete=models.SET_NULL, on_delete=models.SET_NULL,
@@ -37,7 +36,6 @@ class GlobalConfig(TimestampedModel):
retention_weekly = models.PositiveIntegerField(default=8) retention_weekly = models.PositiveIntegerField(default=8)
retention_monthly = models.PositiveIntegerField(default=12) retention_monthly = models.PositiveIntegerField(default=12)
retention_yearly = models.PositiveIntegerField(default=0) retention_yearly = models.PositiveIntegerField(default=0)
data = models.JSONField(default=dict, blank=True)
class Meta: class Meta:
verbose_name = "global config" verbose_name = "global config"
@@ -105,6 +103,7 @@ class BackupRun(models.Model):
QUEUED = "queued", "Queued" QUEUED = "queued", "Queued"
RUNNING = "running", "Running" RUNNING = "running", "Running"
SUCCESS = "success", "Success" SUCCESS = "success", "Success"
WARNING = "warning", "Warning"
FAILED = "failed", "Failed" FAILED = "failed", "Failed"
CANCELLED = "cancelled", "Cancelled" CANCELLED = "cancelled", "Cancelled"
@@ -125,6 +124,8 @@ class BackupRun(models.Model):
rsync_exit_code = models.IntegerField(null=True, blank=True) rsync_exit_code = models.IntegerField(null=True, blank=True)
result = models.JSONField(default=dict, blank=True) result = models.JSONField(default=dict, blank=True)
created_at = models.DateTimeField(auto_now_add=True) created_at = models.DateTimeField(auto_now_add=True)
reviewed_at = models.DateTimeField(null=True, blank=True)
reviewed_by = models.CharField(max_length=150, blank=True)
class Meta: class Meta:
ordering = ["-created_at"] ordering = ["-created_at"]
@@ -159,6 +160,8 @@ class SnapshotRecord(models.Model):
ended_at = models.DateTimeField(null=True, blank=True) ended_at = models.DateTimeField(null=True, blank=True)
metadata = models.JSONField(default=dict, blank=True) metadata = models.JSONField(default=dict, blank=True)
discovered_at = models.DateTimeField(auto_now_add=True) discovered_at = models.DateTimeField(auto_now_add=True)
reviewed_at = models.DateTimeField(null=True, blank=True)
reviewed_by = models.CharField(max_length=150, blank=True)
class Meta: class Meta:
constraints = [ constraints = [
@@ -170,10 +173,34 @@ class SnapshotRecord(models.Model):
return f"{self.host}/{self.kind}/{self.dirname}" return f"{self.host}/{self.kind}/{self.dirname}"
class PurgedSnapshot(models.Model):
class Action(models.TextChoices):
MANUAL = "manual", "Manual"
SCHEDULED = "scheduled", "Scheduled"
CLI = "cli", "CLI"
INCOMPLETE_CLEANUP = "incomplete_cleanup", "Incomplete cleanup"
host = models.ForeignKey(HostConfig, on_delete=models.SET_NULL, null=True, blank=True, related_name="purged_snapshots")
host_name = models.CharField(max_length=255)
kind = models.CharField(max_length=16)
dirname = models.CharField(max_length=255)
path = models.CharField(max_length=1024)
reason = models.CharField(max_length=512, blank=True)
action = models.CharField(max_length=32, choices=Action.choices)
triggered_by = models.CharField(max_length=150, blank=True)
metadata = models.JSONField(default=dict, blank=True)
purged_at = models.DateTimeField(auto_now_add=True)
class Meta:
ordering = ["-purged_at", "host_name", "dirname"]
def __str__(self) -> str:
return f"{self.host_name}/{self.kind}/{self.dirname}"
class ScheduleConfig(TimestampedModel): class ScheduleConfig(TimestampedModel):
host = models.OneToOneField(HostConfig, on_delete=models.CASCADE, related_name="schedule") host = models.OneToOneField(HostConfig, on_delete=models.CASCADE, related_name="schedule")
cron_expr = models.CharField(max_length=128) cron_expr = models.CharField(max_length=128)
user = models.CharField(max_length=64, default="root")
enabled = models.BooleanField(default=True) enabled = models.BooleanField(default=True)
prune = models.BooleanField(default=False) prune = models.BooleanField(default=False)
prune_max_delete = models.PositiveIntegerField(default=10) prune_max_delete = models.PositiveIntegerField(default=10)

View File

@@ -0,0 +1,232 @@
from __future__ import annotations
import shlex
import subprocess
from dataclasses import dataclass
from typing import Any
from pobsync.config.merge import build_effective_config
from pobsync.rsync import build_ssh_command
from .config_repository import global_config_object_data, host_config_object_data
from .config_source import DjangoConfigSource
from .host_ops import collect_host_checks
from .models import GlobalConfig, HostConfig
from .self_check import SelfCheck
DRY_RUN_BLOCKING_CHECKS = {
"Host global config",
"Host address",
"Host SSH key file",
"Host effective source root",
"Host effective SSH user",
"Host effective SSH port",
"Host effective SSH credential",
"Host effective rsync recursion",
}
@dataclass(frozen=True)
class BackupGate:
state: str
message: str
checks: list[SelfCheck]
real_blockers: list[SelfCheck]
dry_run_blockers: list[SelfCheck]
warnings: list[SelfCheck]
@property
def can_queue_real(self) -> bool:
return not self.real_blockers
@property
def can_queue_dry_run(self) -> bool:
return not self.dry_run_blockers
def collect_backup_gate(host: HostConfig, global_config: GlobalConfig | None = None) -> BackupGate:
checks = collect_host_checks(host, global_config)
remote_preflight_check = _remote_preflight_self_check(host)
if remote_preflight_check is not None:
checks.append(remote_preflight_check)
real_blockers = [check for check in checks if check.status == "failed"]
dry_run_blockers = [check for check in real_blockers if check.name in DRY_RUN_BLOCKING_CHECKS]
warnings = [check for check in checks if check.status == "warning"]
if real_blockers:
state = "blocked"
message = "Real backups are blocked until failed host checks are resolved."
elif warnings:
state = "warning"
message = "Backups can run, but review the warnings first."
else:
state = "ready"
message = "This host is ready for backup runs."
return BackupGate(
state=state,
message=message,
checks=checks,
real_blockers=real_blockers,
dry_run_blockers=dry_run_blockers,
warnings=warnings,
)
def run_remote_preflight(host: HostConfig, *, timeout_seconds: int = 20) -> dict[str, Any]:
config = DjangoConfigSource().effective_config_for_host(host.host)
ssh_cfg = config.get("ssh", {}) or {}
rsync_cfg = config.get("rsync", {}) or {}
address = str(config.get("address") or host.address)
user = str(ssh_cfg.get("user") or "root")
source_root = str(config.get("source_root") or (config.get("defaults", {}) or {}).get("source_root") or "/")
rsync_binary = str(rsync_cfg.get("binary") or "rsync")
target = f"{user}@{address}"
ssh_cmd = build_ssh_command(ssh_cfg)
checks = [
_run_remote_check(
name="SSH reachability",
command=[*ssh_cmd, "-oBatchMode=yes", target, "true"],
timeout_seconds=timeout_seconds,
),
_run_remote_check(
name="Remote rsync",
command=[
*ssh_cmd,
"-oBatchMode=yes",
target,
"sh",
"-lc",
f"command -v {shlex.quote(rsync_binary)} >/dev/null",
],
timeout_seconds=timeout_seconds,
),
_run_remote_check(
name="Remote source root",
command=[
*ssh_cmd,
"-oBatchMode=yes",
target,
"sh",
"-lc",
f"test -e {shlex.quote(source_root)} && test -r {shlex.quote(source_root)}",
],
timeout_seconds=timeout_seconds,
),
]
result = {
"ok": all(check["ok"] for check in checks),
"checks": checks,
"target": target,
"source_root": source_root,
"rsync_binary": rsync_binary,
"timeout_seconds": timeout_seconds,
}
host.config = {**(host.config or {}), "last_preflight": result}
host.save(update_fields=["config", "updated_at"])
return result
def effective_host_config_preview(host: HostConfig, global_config: GlobalConfig) -> dict[str, Any]:
config = build_effective_config(global_config_object_data(global_config), host_config_object_data(host))
credential = host.ssh_credential or global_config.default_ssh_credential
ssh = config.get("ssh", {}) or {}
rsync = config.get("rsync", {}) or {}
retention = config.get("retention", {}) or {}
return {
"source_root": config.get("source_root", ""),
"destination_subdir": (config.get("defaults", {}) or {}).get("destination_subdir", ""),
"includes": list(config.get("includes") or []),
"excludes": list(config.get("excludes_effective") or []),
"ssh": {
"user": ssh.get("user", ""),
"port": ssh.get("port", ""),
"options": list(ssh.get("options") or []),
"credential": str(credential) if credential else "",
},
"rsync": {
"binary": rsync.get("binary", ""),
"args": list(rsync.get("args_effective") or []),
"timeout_seconds": rsync.get("timeout_seconds", 0),
"bwlimit_kbps": rsync.get("bwlimit_kbps", 0),
},
"retention": {
"daily": retention.get("daily", 0),
"weekly": retention.get("weekly", 0),
"monthly": retention.get("monthly", 0),
"yearly": retention.get("yearly", 0),
},
}
def _run_remote_check(*, name: str, command: list[str], timeout_seconds: int) -> dict[str, Any]:
try:
result = subprocess.run(
command,
check=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
timeout=timeout_seconds,
)
except subprocess.TimeoutExpired as exc:
return {
"name": name,
"ok": False,
"exit_code": 124,
"message": f"{name} timed out after {timeout_seconds}s.",
"detail": _clip_output((exc.stderr or exc.stdout or "").strip()),
}
except OSError as exc:
return {
"name": name,
"ok": False,
"exit_code": None,
"message": f"{name} could not start.",
"detail": str(exc),
}
return {
"name": name,
"ok": result.returncode == 0,
"exit_code": result.returncode,
"message": f"{name} passed." if result.returncode == 0 else f"{name} failed.",
"detail": _clip_output((result.stderr or result.stdout or "").strip()),
}
def _remote_preflight_self_check(host: HostConfig) -> SelfCheck | None:
preflight = (host.config or {}).get("last_preflight")
if not isinstance(preflight, dict):
return SelfCheck(
"Remote preflight",
"warning",
"No remote connection preflight has been run yet.",
"Run connection preflight before the first real backup.",
)
checks = preflight.get("checks")
if not isinstance(checks, list):
return SelfCheck("Remote preflight", "failed", "Stored remote preflight result is invalid.")
failed = [str(check.get("name", "unknown")) for check in checks if isinstance(check, dict) and not check.get("ok")]
if failed:
return SelfCheck(
"Remote preflight",
"failed",
"Remote connection preflight failed.",
", ".join(failed),
)
return SelfCheck(
"Remote preflight",
"ok",
"Remote connection preflight passed.",
f"{preflight.get('target', '')} {preflight.get('source_root', '')}".strip(),
)
def _clip_output(value: str, *, max_chars: int = 800) -> str:
if len(value) <= max_chars:
return value
return f"{value[:max_chars]}..."

View File

@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import shutil import shutil
import stat
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
@@ -11,7 +12,7 @@ from pobsync.paths import PobsyncPaths
from pobsync.retention import Snapshot, apply_base_protection, build_retention_plan from pobsync.retention import Snapshot, apply_base_protection, build_retention_plan
from pobsync.util import sanitize_host from pobsync.util import sanitize_host
from .models import HostConfig, SnapshotRecord from .models import HostConfig, PurgedSnapshot, SnapshotRecord
def run_sql_retention_plan(*, host: str, kind: str, protect_bases: bool) -> dict[str, Any]: def run_sql_retention_plan(*, host: str, kind: str, protect_bases: bool) -> dict[str, Any]:
@@ -22,6 +23,7 @@ def run_sql_retention_plan(*, host: str, kind: str, protect_bases: bool) -> dict
host_config = _enabled_host_config(host) host_config = _enabled_host_config(host)
retention = _retention_for_host(host_config) retention = _retention_for_host(host_config)
snapshots = _snapshots_for_retention(host_config=host_config, kind=kind) snapshots = _snapshots_for_retention(host_config=host_config, kind=kind)
incomplete_snapshots = _incomplete_snapshots_for_host(host_config)
plan = build_retention_plan( plan = build_retention_plan(
snapshots=snapshots, snapshots=snapshots,
@@ -35,6 +37,7 @@ def run_sql_retention_plan(*, host: str, kind: str, protect_bases: bool) -> dict
keep, reasons = apply_base_protection(snapshots=snapshots, keep=keep, reasons=reasons) keep, reasons = apply_base_protection(snapshots=snapshots, keep=keep, reasons=reasons)
delete = [snapshot for snapshot in snapshots if snapshot.dirname not in keep] delete = [snapshot for snapshot in snapshots if snapshot.dirname not in keep]
keep_items = [snapshot for snapshot in snapshots if snapshot.dirname in keep]
return { return {
"ok": True, "ok": True,
@@ -44,7 +47,12 @@ def run_sql_retention_plan(*, host: str, kind: str, protect_bases: bool) -> dict
"retention": retention, "retention": retention,
"source": "sql", "source": "sql",
"keep": sorted(keep), "keep": sorted(keep),
"delete": [_snapshot_to_delete_item(snapshot) for snapshot in delete], "keep_items": [_snapshot_to_item(snapshot, reasons=reasons.get(snapshot.dirname, [])) for snapshot in keep_items],
"delete": [_snapshot_to_item(snapshot, reasons=["outside retention policy"]) for snapshot in delete],
"incomplete": [
_snapshot_to_item(snapshot, reasons=["incomplete snapshot; excluded from retention cleanup"])
for snapshot in incomplete_snapshots
],
"reasons": reasons, "reasons": reasons,
} }
@@ -57,6 +65,8 @@ def run_sql_retention_apply(
protect_bases: bool, protect_bases: bool,
yes: bool, yes: bool,
max_delete: int, max_delete: int,
action: str = PurgedSnapshot.Action.MANUAL,
triggered_by: str = "",
acquire_lock: bool = True, acquire_lock: bool = True,
) -> dict[str, Any]: ) -> dict[str, Any]:
host = sanitize_host(host) host = sanitize_host(host)
@@ -70,8 +80,11 @@ def run_sql_retention_apply(
def _do_apply() -> dict[str, Any]: def _do_apply() -> dict[str, Any]:
plan = run_sql_retention_plan(host=host, kind=kind, protect_bases=bool(protect_bases)) plan = run_sql_retention_plan(host=host, kind=kind, protect_bases=bool(protect_bases))
delete_list = plan.get("delete") or [] delete_list = plan.get("delete") or []
incomplete_list = plan.get("incomplete") or []
if not isinstance(delete_list, list): if not isinstance(delete_list, list):
raise ConfigError("Invalid retention plan output: delete is not a list") raise ConfigError("Invalid retention plan output: delete is not a list")
if not isinstance(incomplete_list, list):
raise ConfigError("Invalid retention plan output: incomplete is not a list")
if max_delete == 0 and len(delete_list) > 0: if max_delete == 0 and len(delete_list) > 0:
raise ConfigError("Deletion blocked by --max-delete=0") raise ConfigError("Deletion blocked by --max-delete=0")
if len(delete_list) > max_delete: if len(delete_list) > max_delete:
@@ -89,17 +102,28 @@ def run_sql_retention_apply(
if snap_kind not in {"scheduled", "manual"}: if snap_kind not in {"scheduled", "manual"}:
raise ConfigError(f"Refusing to delete unsupported snapshot kind: {snap_kind!r}") raise ConfigError(f"Refusing to delete unsupported snapshot kind: {snap_kind!r}")
path = Path(snap_path) path = _snapshot_delete_path(path=Path(snap_path), dirname=dirname)
reason = str(item.get("reason") or "outside retention policy")
if not path.exists(): if not path.exists():
actions.append(f"skip missing {snap_kind}/{dirname}") actions.append(f"skip missing {snap_kind}/{dirname}")
continue continue
if not path.is_dir(): if not path.is_dir():
raise ConfigError(f"Refusing to delete non-directory path: {snap_path}") raise ConfigError(f"Refusing to delete non-directory path: {path}")
shutil.rmtree(path) _remove_snapshot_tree(path)
_record_purged_snapshot(
host_config=_enabled_host_config(host),
kind=snap_kind,
dirname=dirname,
path=path,
reason=reason,
action=action,
triggered_by=triggered_by,
metadata={"source": "retention", "protect_bases": bool(protect_bases), "retention_kind": kind},
)
SnapshotRecord.objects.filter(host__host=host, kind=snap_kind, dirname=dirname).delete() SnapshotRecord.objects.filter(host__host=host, kind=snap_kind, dirname=dirname).delete()
actions.append(f"deleted {snap_kind} {dirname}") actions.append(f"deleted {snap_kind} {dirname}")
deleted.append({"dirname": dirname, "kind": snap_kind, "path": snap_path}) deleted.append({"dirname": dirname, "kind": snap_kind, "path": str(path), "reason": reason})
return { return {
"ok": True, "ok": True,
@@ -108,6 +132,8 @@ def run_sql_retention_apply(
"protect_bases": bool(protect_bases), "protect_bases": bool(protect_bases),
"max_delete": max_delete, "max_delete": max_delete,
"source": "sql", "source": "sql",
"planned_delete_count": len(delete_list),
"incomplete_ignored_count": len(incomplete_list),
"deleted": deleted, "deleted": deleted,
"actions": actions, "actions": actions,
} }
@@ -118,11 +144,92 @@ def run_sql_retention_apply(
return _do_apply() return _do_apply()
def run_incomplete_cleanup(
*,
prefix: Path,
host: str,
yes: bool,
max_delete: int,
triggered_by: str = "",
acquire_lock: bool = True,
) -> dict[str, Any]:
host = sanitize_host(host)
if not yes:
raise ConfigError("Refusing to delete incomplete snapshots without --yes")
if max_delete < 0:
raise ConfigError("--max-delete must be >= 0")
paths = PobsyncPaths(home=prefix)
def _do_cleanup() -> dict[str, Any]:
host_config = _enabled_host_config(host)
incomplete_list = [
_snapshot_to_item(snapshot, reasons=["manual incomplete cleanup"])
for snapshot in _incomplete_snapshots_for_host(host_config)
]
if max_delete == 0 and len(incomplete_list) > 0:
raise ConfigError("Incomplete cleanup blocked by --max-delete=0")
if len(incomplete_list) > max_delete:
raise ConfigError(
f"Refusing to delete {len(incomplete_list)} incomplete snapshots (exceeds --max-delete={max_delete})"
)
actions: list[str] = []
deleted: list[dict[str, Any]] = []
for item in incomplete_list:
dirname = item["dirname"]
snap_path = Path(item["path"])
path = _snapshot_delete_path(path=snap_path, dirname=dirname)
_validate_incomplete_delete_path(host=host, path=path, dirname=dirname)
if not path.exists():
actions.append(f"skip missing incomplete/{dirname}")
elif not path.is_dir():
raise ConfigError(f"Refusing to delete non-directory path: {path}")
else:
_remove_snapshot_tree(path)
actions.append(f"deleted incomplete {dirname}")
_record_purged_snapshot(
host_config=host_config,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=dirname,
path=path,
reason="manual incomplete cleanup",
action=PurgedSnapshot.Action.INCOMPLETE_CLEANUP,
triggered_by=triggered_by,
metadata={"source": "incomplete_cleanup"},
)
SnapshotRecord.objects.filter(
host__host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=dirname,
).delete()
deleted.append({"dirname": dirname, "kind": SnapshotRecord.Kind.INCOMPLETE, "path": str(path)})
return {
"ok": True,
"host": host,
"kind": SnapshotRecord.Kind.INCOMPLETE,
"max_delete": max_delete,
"source": "sql",
"planned_delete_count": len(incomplete_list),
"deleted": deleted,
"actions": actions,
}
if acquire_lock:
with acquire_host_lock(paths.locks_dir, host, command="incomplete-cleanup"):
return _do_cleanup()
return _do_cleanup()
def _enabled_host_config(host: str) -> HostConfig: def _enabled_host_config(host: str) -> HostConfig:
try: try:
return HostConfig.objects.get(host=host, enabled=True) return HostConfig.objects.get(host=host, enabled=True)
except HostConfig.DoesNotExist as exc: except HostConfig.DoesNotExist as exc:
raise ConfigError(f"Missing enabled HostConfig {host!r}") from exc raise ConfigError(f"Missing enabled host {host!r}") from exc
def _retention_for_host(host_config: HostConfig) -> dict[str, int]: def _retention_for_host(host_config: HostConfig) -> dict[str, int]:
@@ -145,6 +252,15 @@ def _snapshots_for_retention(*, host_config: HostConfig, kind: str) -> list[Snap
return [_snapshot_from_record(record) for record in records] return [_snapshot_from_record(record) for record in records]
def _incomplete_snapshots_for_host(host_config: HostConfig) -> list[Snapshot]:
records = (
SnapshotRecord.objects.filter(host=host_config, kind=SnapshotRecord.Kind.INCOMPLETE)
.select_related("base")
.order_by("-started_at", "dirname")
)
return [_snapshot_from_record(record) for record in records]
def _snapshot_from_record(record: SnapshotRecord) -> Snapshot: def _snapshot_from_record(record: SnapshotRecord) -> Snapshot:
return Snapshot( return Snapshot(
kind=record.kind, kind=record.kind,
@@ -172,11 +288,65 @@ def _base_meta_from_record(record: SnapshotRecord) -> dict[str, str] | None:
return None return None
def _snapshot_to_delete_item(snapshot: Snapshot) -> dict[str, Any]: def _snapshot_to_item(snapshot: Snapshot, *, reasons: list[str]) -> dict[str, Any]:
return { return {
"dirname": snapshot.dirname, "dirname": snapshot.dirname,
"kind": snapshot.kind, "kind": snapshot.kind,
"path": snapshot.path, "path": snapshot.path,
"dt": snapshot.dt.isoformat(), "dt": snapshot.dt.isoformat(),
"status": snapshot.status, "status": snapshot.status,
"reasons": reasons,
"reason": ", ".join(reasons),
} }
def _snapshot_delete_path(*, path: Path, dirname: str) -> Path:
if path.name == "data" and path.parent.name == dirname:
return path.parent
return path
def _record_purged_snapshot(
*,
host_config: HostConfig,
kind: str,
dirname: str,
path: Path,
reason: str,
action: str,
triggered_by: str,
metadata: dict[str, Any],
) -> None:
PurgedSnapshot.objects.create(
host=host_config,
host_name=host_config.host,
kind=kind,
dirname=dirname,
path=str(path),
reason=reason,
action=action,
triggered_by=triggered_by,
metadata=metadata,
)
def _validate_incomplete_delete_path(*, host: str, path: Path, dirname: str) -> None:
path_parts = path.parts
if path.name != dirname or ".incomplete" not in path_parts or host not in path_parts:
raise ConfigError(f"Refusing to delete unexpected incomplete snapshot path: {path}")
incomplete_index = path_parts.index(".incomplete")
if incomplete_index == 0 or path_parts[incomplete_index - 1] != host:
raise ConfigError(f"Refusing to delete incomplete snapshot outside host backup root: {path}")
def _remove_snapshot_tree(path: Path) -> None:
_make_directories_user_writable(path)
shutil.rmtree(path)
def _make_directories_user_writable(path: Path) -> None:
for directory in [path, *[child for child in path.rglob("*") if child.is_dir() and not child.is_symlink()]]:
mode = directory.stat().st_mode
if mode & stat.S_IWUSR:
continue
directory.chmod(mode | stat.S_IWUSR)

View File

@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import os import os
import pwd
import shutil import shutil
import subprocess import subprocess
import sys import sys
@@ -28,6 +29,7 @@ class SelfCheck:
def collect_self_checks() -> list[SelfCheck]: def collect_self_checks() -> list[SelfCheck]:
checks: list[SelfCheck] = [] checks: list[SelfCheck] = []
checks.extend(_django_checks()) checks.extend(_django_checks())
checks.extend(_install_checks())
checks.extend(_path_checks()) checks.extend(_path_checks())
checks.extend(_binary_checks()) checks.extend(_binary_checks())
checks.extend(_database_checks()) checks.extend(_database_checks())
@@ -36,6 +38,10 @@ def collect_self_checks() -> list[SelfCheck]:
return checks return checks
def _native_runtime_available() -> bool:
return Path("/run/systemd/system").exists() and shutil.which("systemctl") is not None
def summarize_self_checks(checks: list[SelfCheck]) -> dict[str, int]: def summarize_self_checks(checks: list[SelfCheck]) -> dict[str, int]:
return { return {
"ok": sum(1 for check in checks if check.status == "ok"), "ok": sum(1 for check in checks if check.status == "ok"),
@@ -70,10 +76,17 @@ def _django_checks() -> list[SelfCheck]:
def _path_checks() -> list[SelfCheck]: def _path_checks() -> list[SelfCheck]:
checks = [] checks = []
checks.append(_path_check("POBSYNC_HOME", Path(settings.POBSYNC_HOME), must_be_absolute=True, must_be_writable=True))
checks.append( checks.append(
_path_check( _path_check(
"POBSYNC_BACKUP_ROOT", "State root",
Path(settings.POBSYNC_HOME),
must_be_absolute=True,
must_be_writable=True,
)
)
checks.append(
_path_check(
"Backup root",
Path(settings.POBSYNC_BACKUP_ROOT), Path(settings.POBSYNC_BACKUP_ROOT),
must_be_absolute=True, must_be_absolute=True,
must_exist=True, must_exist=True,
@@ -91,18 +104,105 @@ def _path_checks() -> list[SelfCheck]:
) )
db_settings = settings.DATABASES["default"] db_settings = settings.DATABASES["default"]
if db_settings["ENGINE"] == "django.db.backends.sqlite3": if db_settings["ENGINE"] == "django.db.backends.sqlite3":
sqlite_path = Path(str(db_settings["NAME"]))
checks.append( checks.append(
_path_check( _path_check(
"SQLite directory", "SQLite directory",
Path(str(db_settings["NAME"])).parent, sqlite_path.parent,
must_be_absolute=True, must_be_absolute=True,
must_exist=True, must_exist=True,
must_be_writable=True, must_be_writable=True,
) )
) )
checks.append(_sqlite_database_check(sqlite_path))
return checks return checks
def _install_checks() -> list[SelfCheck]:
if not _native_runtime_available() and not Path(settings.POBSYNC_ENV_FILE).exists():
return [
SelfCheck(
"Environment file",
"skipped",
"Native environment file is not configured in this runtime.",
"This is expected inside Docker or local development.",
),
SelfCheck(
"Service user",
"skipped",
"Native service user check is not available in this runtime.",
"This is expected inside Docker or local development.",
),
SelfCheck(
"Backup root owner",
"skipped",
"Native backup root ownership check is not available in this runtime.",
"This is expected inside Docker or local development.",
),
]
checks = [_env_file_check(Path(settings.POBSYNC_ENV_FILE)), _service_user_check()]
checks.append(_backup_root_owner_check(Path(settings.POBSYNC_BACKUP_ROOT)))
return checks
def _env_file_check(path: Path) -> SelfCheck:
if not path.is_absolute():
return SelfCheck("Environment file", "failed", f"{path} is not absolute.")
if not path.exists():
return SelfCheck("Environment file", "failed", f"{path} does not exist.")
if not path.is_file():
return SelfCheck("Environment file", "failed", f"{path} is not a regular file.")
if not os.access(path, os.R_OK):
return SelfCheck("Environment file", "failed", f"{path} is not readable by this process.")
return SelfCheck("Environment file", "ok", str(path))
def _service_user_check() -> SelfCheck:
expected_user = settings.POBSYNC_SERVICE_USER
try:
current_user = pwd.getpwuid(os.geteuid()).pw_name
except KeyError:
return SelfCheck("Service user", "failed", f"Current uid {os.geteuid()} has no passwd entry.")
if current_user != expected_user:
return SelfCheck(
"Service user",
"warning",
f"Current process runs as {current_user}, expected {expected_user}.",
"Run terminal checks with sudo -u <service-user> pobsync-manage check_pobsync_install.",
)
return SelfCheck("Service user", "ok", current_user)
def _backup_root_owner_check(path: Path) -> SelfCheck:
if not path.exists():
return SelfCheck("Backup root owner", "failed", f"{path} does not exist.")
expected_user = settings.POBSYNC_SERVICE_USER
try:
owner = pwd.getpwuid(path.stat().st_uid).pw_name
except KeyError:
return SelfCheck("Backup root owner", "warning", f"{path} owner uid {path.stat().st_uid} has no passwd entry.")
if owner != expected_user:
return SelfCheck(
"Backup root owner",
"warning",
f"{path} is owned by {owner}, expected {expected_user}.",
)
return SelfCheck("Backup root owner", "ok", f"{path} owner={owner}")
def _sqlite_database_check(path: Path) -> SelfCheck:
if not path.is_absolute():
return SelfCheck("SQLite database", "failed", f"{path} is not absolute.")
if not path.exists():
return SelfCheck("SQLite database", "warning", f"{path} does not exist yet.")
if not path.is_file():
return SelfCheck("SQLite database", "failed", f"{path} is not a regular file.")
if not os.access(path, os.R_OK | os.W_OK):
return SelfCheck("SQLite database", "failed", f"{path} is not readable and writable by this process.")
return SelfCheck("SQLite database", "ok", str(path))
def _path_check( def _path_check(
name: str, name: str,
path: Path, path: Path,
@@ -166,7 +266,7 @@ def _config_checks() -> list[SelfCheck]:
message = "Default global config exists." message = "Default global config exists."
if global_config.backup_root != settings.POBSYNC_BACKUP_ROOT: if global_config.backup_root != settings.POBSYNC_BACKUP_ROOT:
status = "warning" status = "warning"
message = "Global config backup root differs from runtime POBSYNC_BACKUP_ROOT." message = "Global config backup root differs from the runtime backup root."
return [ return [
SelfCheck( SelfCheck(
"Global config", "Global config",
@@ -178,7 +278,7 @@ def _config_checks() -> list[SelfCheck]:
def _systemd_checks() -> list[SelfCheck]: def _systemd_checks() -> list[SelfCheck]:
if not Path("/run/systemd/system").exists() or shutil.which("systemctl") is None: if not _native_runtime_available():
return [ return [
SelfCheck( SelfCheck(
"Systemd services", "Systemd services",

View File

@@ -13,7 +13,7 @@ from .models import BackupRun, GlobalConfig, HostConfig, SnapshotRecord
def collect_dashboard_stats(*, hosts: Iterable[HostConfig], global_config: GlobalConfig | None) -> dict[str, Any]: def collect_dashboard_stats(*, hosts: Iterable[HostConfig], global_config: GlobalConfig | None) -> dict[str, Any]:
runs = list( runs = list(
BackupRun.objects.select_related("host", "snapshot") BackupRun.objects.select_related("host", "snapshot")
.filter(status=BackupRun.Status.SUCCESS) .filter(status__in=_COMPLETED_BACKUP_STATUSES)
.order_by("-started_at", "-created_at")[:100] .order_by("-started_at", "-created_at")[:100]
) )
real_runs = [_run_summary(run) for run in runs if _is_real_run(run)] real_runs = [_run_summary(run) for run in runs if _is_real_run(run)]
@@ -37,6 +37,8 @@ def collect_dashboard_stats(*, hosts: Iterable[HostConfig], global_config: Globa
available = _int_at(capacity, "available_bytes") available = _int_at(capacity, "available_bytes")
daily_literal = _average_daily_literal(real_runs) daily_literal = _average_daily_literal(real_runs)
link_dest_savings_ratio = round(total_matched / savings_basis, 4) if savings_basis else None
return { return {
"runs_sampled": len(real_runs), "runs_sampled": len(real_runs),
"avg_duration_seconds": _average(duration_values), "avg_duration_seconds": _average(duration_values),
@@ -44,7 +46,8 @@ def collect_dashboard_stats(*, hosts: Iterable[HostConfig], global_config: Globa
"avg_literal_data_bytes": avg_literal, "avg_literal_data_bytes": avg_literal,
"total_literal_data_bytes": total_literal, "total_literal_data_bytes": total_literal,
"total_matched_data_bytes": total_matched, "total_matched_data_bytes": total_matched,
"link_dest_savings_ratio": round(total_matched / savings_basis, 4) if savings_basis else None, "link_dest_savings_ratio": link_dest_savings_ratio,
"link_dest_savings_percent": round(link_dest_savings_ratio * 100, 1) if link_dest_savings_ratio is not None else None,
"estimated_runs_until_full": int(available / avg_literal) if available and avg_literal else None, "estimated_runs_until_full": int(available / avg_literal) if available and avg_literal else None,
"estimated_days_until_full": int(available / daily_literal) if available and daily_literal else None, "estimated_days_until_full": int(available / daily_literal) if available and daily_literal else None,
"capacity": capacity, "capacity": capacity,
@@ -52,9 +55,10 @@ def collect_dashboard_stats(*, hosts: Iterable[HostConfig], global_config: Globa
def collect_host_stats(*, host: HostConfig, limit: int = 8) -> dict[str, Any]: def collect_host_stats(*, host: HostConfig, limit: int = 8) -> dict[str, Any]:
runs = list(host.runs.select_related("snapshot").filter(status=BackupRun.Status.SUCCESS).order_by("-started_at", "-created_at")[:50]) runs = list(host.runs.select_related("snapshot").order_by("-started_at", "-created_at")[:50])
real_runs = [_run_summary(run) for run in runs if _is_real_run(run)] real_runs = [_run_summary(run) for run in runs if _is_real_run(run)]
trend_runs = [run for run in real_runs if run["has_stats"]][:limit] completed_real_runs = [run for run in real_runs if run["status"] in _COMPLETED_BACKUP_STATUSES]
trend_runs = [run for run in completed_real_runs if run["has_stats"]][:limit]
latest_snapshot = host.snapshots.order_by("-started_at", "-discovered_at", "-id").first() latest_snapshot = host.snapshots.order_by("-started_at", "-discovered_at", "-id").first()
latest_snapshot_stats = _snapshot_summary(latest_snapshot) if latest_snapshot else {} latest_snapshot_stats = _snapshot_summary(latest_snapshot) if latest_snapshot else {}
@@ -67,7 +71,9 @@ def collect_host_stats(*, host: HostConfig, limit: int = 8) -> dict[str, Any]:
return { return {
"runs": [_with_bar_percentages(run, max_literal=max_literal, max_matched=max_matched) for run in trend_runs], "runs": [_with_bar_percentages(run, max_literal=max_literal, max_matched=max_matched) for run in trend_runs],
"latest_run": real_runs[0] if real_runs else {}, "latest_run": completed_real_runs[0] if completed_real_runs else {},
"latest_good_run": _first_run_with_status(real_runs, {BackupRun.Status.SUCCESS}),
"latest_problem_run": _first_run_with_status(real_runs, {BackupRun.Status.WARNING, BackupRun.Status.FAILED}),
"latest_snapshot": latest_snapshot_stats, "latest_snapshot": latest_snapshot_stats,
"avg_literal_data_bytes": _average(literal_values), "avg_literal_data_bytes": _average(literal_values),
"avg_daily_literal_data_bytes": _average_daily_literal(trend_runs), "avg_daily_literal_data_bytes": _average_daily_literal(trend_runs),
@@ -87,6 +93,8 @@ def _run_summary(run: BackupRun) -> dict[str, Any]:
"ended_at": run.ended_at, "ended_at": run.ended_at,
"snapshot": run.snapshot, "snapshot": run.snapshot,
"snapshot_path": run.snapshot_path, "snapshot_path": run.snapshot_path,
"status": run.status,
"reviewed_at": run.reviewed_at,
"has_stats": bool(stats), "has_stats": bool(stats),
"duration_seconds": _int_at(stats, "duration_seconds"), "duration_seconds": _int_at(stats, "duration_seconds"),
"rsync": stats.get("rsync") if isinstance(stats.get("rsync"), dict) else {}, "rsync": stats.get("rsync") if isinstance(stats.get("rsync"), dict) else {},
@@ -121,6 +129,13 @@ def _is_real_run(run: BackupRun) -> bool:
return requested.get("dry_run") is not True return requested.get("dry_run") is not True
def _first_run_with_status(runs: list[dict[str, Any]], statuses: set[str]) -> dict[str, Any]:
for run in runs:
if run["status"] in statuses and run.get("reviewed_at") is None:
return run
return {}
def _capacity_from_system(global_config: GlobalConfig | None) -> dict[str, Any]: def _capacity_from_system(global_config: GlobalConfig | None) -> dict[str, Any]:
if global_config is None or not global_config.backup_root: if global_config is None or not global_config.backup_root:
return {} return {}
@@ -198,3 +213,6 @@ def _int_at(data: dict[str, Any], *keys: str) -> int | None:
if isinstance(value, float): if isinstance(value, float):
return int(value) return int(value)
return None return None
_COMPLETED_BACKUP_STATUSES = [BackupRun.Status.SUCCESS, BackupRun.Status.WARNING]

View File

@@ -61,7 +61,15 @@
.metric { padding: 14px; } .metric { padding: 14px; }
.metric .label { color: var(--muted); font-size: 12px; text-transform: uppercase; } .metric .label { color: var(--muted); font-size: 12px; text-transform: uppercase; }
.metric .value { font-size: 26px; font-weight: 650; margin-top: 4px; } .metric .value { font-size: 26px; font-weight: 650; margin-top: 4px; }
.metric.failed { border-color: #e8b4b4; background: #fff7f7; }
.metric.warning { border-color: #e7cf8a; background: #fffaf0; }
.metric.running { border-color: #e7cf8a; background: #fffaf0; }
.metric.queued { border-color: #b5cdea; background: #eef6ff; }
.panel { padding: 16px; margin-bottom: 18px; overflow: auto; } .panel { padding: 16px; margin-bottom: 18px; overflow: auto; }
.panel.highlight { border-left: 4px solid var(--border); }
.panel.highlight.failed { border-left-color: var(--failed); background: #fff7f7; }
.panel.highlight.warning { border-left-color: var(--running); background: #fffaf0; }
.panel.highlight.success { border-left-color: var(--success); background: #f5fbf7; }
table { width: 100%; border-collapse: collapse; min-width: 640px; } table { width: 100%; border-collapse: collapse; min-width: 640px; }
th, td { border-bottom: 1px solid var(--border); padding: 9px 8px; text-align: left; vertical-align: top; } th, td { border-bottom: 1px solid var(--border); padding: 9px 8px; text-align: left; vertical-align: top; }
th { color: var(--muted); font-size: 12px; font-weight: 650; text-transform: uppercase; } th { color: var(--muted); font-size: 12px; font-weight: 650; text-transform: uppercase; }
@@ -77,6 +85,7 @@
.status.success { color: var(--success); border-color: #a7d8b9; background: #edf8f1; } .status.success { color: var(--success); border-color: #a7d8b9; background: #edf8f1; }
.status.ok { color: var(--success); border-color: #a7d8b9; background: #edf8f1; } .status.ok { color: var(--success); border-color: #a7d8b9; background: #edf8f1; }
.status.failed { color: var(--failed); border-color: #e8b4b4; background: #fff0f0; } .status.failed { color: var(--failed); border-color: #e8b4b4; background: #fff0f0; }
.status.blocked { color: var(--failed); border-color: #e8b4b4; background: #fff0f0; }
.status.running { color: var(--running); border-color: #e7cf8a; background: #fff8df; } .status.running { color: var(--running); border-color: #e7cf8a; background: #fff8df; }
.status.warning { color: var(--running); border-color: #e7cf8a; background: #fff8df; } .status.warning { color: var(--running); border-color: #e7cf8a; background: #fff8df; }
.status.queued { color: var(--link); border-color: #b5cdea; background: #eef6ff; } .status.queued { color: var(--link); border-color: #b5cdea; background: #eef6ff; }
@@ -113,6 +122,23 @@
cursor: not-allowed; cursor: not-allowed;
} }
.inline-form { margin: 0; } .inline-form { margin: 0; }
.status-overview {
display: grid;
gap: 8px;
}
.status-summary {
align-items: center;
border: 1px solid var(--border);
border-radius: 6px;
display: flex;
flex-wrap: wrap;
gap: 8px;
padding: 10px;
}
.status-summary.failed { border-color: #e8b4b4; background: #fff7f7; color: var(--failed); }
.status-summary.warning,
.status-summary.running { border-color: #e7cf8a; background: #fffaf0; color: var(--running); }
.status-summary.queued { border-color: #b5cdea; background: #eef6ff; color: var(--link); }
.operator-state { .operator-state {
align-items: center; align-items: center;
display: flex; display: flex;
@@ -144,6 +170,151 @@
font-size: 12px; font-size: 12px;
gap: 10px; gap: 10px;
} }
.insight-grid {
display: grid;
gap: 18px 24px;
grid-template-columns: minmax(260px, 1.3fr) repeat(auto-fit, minmax(180px, 1fr));
}
.insight-main,
.insight-item {
display: grid;
gap: 4px;
min-width: 0;
}
.insight-main .label,
.insight-item .label {
color: var(--muted);
font-size: 12px;
font-weight: 650;
text-transform: uppercase;
}
.insight-main .value,
.insight-item .value {
font-size: 22px;
font-weight: 650;
overflow-wrap: anywhere;
}
.storage-meter {
background: #edf2f7;
border-radius: 999px;
height: 10px;
margin: 4px 0;
overflow: hidden;
}
.storage-meter span {
background: var(--link);
display: block;
height: 100%;
max-width: 100%;
}
.host-list {
display: grid;
gap: 12px;
}
.host-card {
border: 1px solid var(--border);
border-radius: 8px;
padding: 16px;
}
.host-card-header {
align-items: start;
display: flex;
gap: 12px;
justify-content: space-between;
margin-bottom: 14px;
}
.host-card-title {
display: grid;
gap: 3px;
min-width: 0;
}
.host-card-title a {
font-size: 17px;
font-weight: 650;
overflow-wrap: anywhere;
}
.host-card-status {
display: flex;
flex: 0 1 auto;
flex-wrap: wrap;
gap: 6px;
justify-content: flex-end;
max-width: 50%;
}
.host-card-layout {
display: grid;
gap: 24px;
grid-template-columns: minmax(0, 2fr) minmax(260px, 1fr);
}
.host-card-section {
align-content: start;
display: grid;
gap: 10px;
min-width: 0;
}
.host-card-section-title {
color: var(--muted);
font-size: 12px;
font-weight: 650;
text-transform: uppercase;
}
.host-card-timeline {
display: grid;
gap: 16px 22px;
grid-template-columns: repeat(auto-fit, minmax(210px, 1fr));
}
.host-card-stats {
align-content: start;
display: grid;
border-top: 1px solid #e6edf4;
gap: 12px 18px;
grid-template-columns: repeat(2, minmax(0, 1fr));
padding-top: 12px;
}
.host-card-item {
display: grid;
gap: 3px;
min-width: 0;
}
.host-card-item .label {
color: var(--muted);
font-size: 12px;
font-weight: 650;
text-transform: uppercase;
}
.host-card-item .value {
overflow-wrap: anywhere;
}
.host-card-stat {
display: grid;
gap: 3px;
min-width: 0;
}
.host-card-stat .label {
color: var(--muted);
font-size: 11px;
font-weight: 650;
text-transform: uppercase;
}
.host-card-stat .value {
font-size: 16px;
font-weight: 650;
overflow-wrap: anywhere;
}
.host-card-stat.wide {
grid-column: 1 / -1;
}
.host-card-warning {
background: #fffaf0;
border: 1px solid #e7cf8a;
border-radius: 6px;
color: var(--running);
display: flex;
flex-wrap: wrap;
gap: 8px;
margin-top: 14px;
padding: 10px;
}
.messages { display: grid; gap: 8px; margin-bottom: 18px; } .messages { display: grid; gap: 8px; margin-bottom: 18px; }
.message { .message {
background: var(--panel); background: var(--panel);
@@ -188,6 +359,11 @@
main { padding: 16px; } main { padding: 16px; }
nav { padding: 0; } nav { padding: 0; }
.two-col { grid-template-columns: 1fr; } .two-col { grid-template-columns: 1fr; }
.host-card-header { display: grid; }
.host-card-status { justify-content: flex-start; max-width: none; }
.host-card-layout { grid-template-columns: 1fr; }
.host-card-stats { grid-template-columns: repeat(auto-fit, minmax(140px, 1fr)); }
.insight-grid { grid-template-columns: 1fr; }
} }
</style> </style>
</head> </head>
@@ -199,6 +375,8 @@
<a href="{% url 'ssh_credentials' %}">SSH Keys</a> <a href="{% url 'ssh_credentials' %}">SSH Keys</a>
<a href="{% url 'self_check' %}">Self Check</a> <a href="{% url 'self_check' %}">Self Check</a>
<a href="{% url 'logs' %}">Logs</a> <a href="{% url 'logs' %}">Logs</a>
<a href="{% url 'purged_snapshots' %}">Purged</a>
<a href="{% url 'changelog' %}">Changelog</a>
<a href="/api/status/">Status API</a> <a href="/api/status/">Status API</a>
<span class="spacer"></span> <span class="spacer"></span>
<span class="muted">{{ request.user.username }}</span> <span class="muted">{{ request.user.username }}</span>

View File

@@ -0,0 +1,41 @@
{% extends "pobsync_backend/base.html" %}
{% block title %}Changelog - pobsync{% endblock %}
{% block content %}
<h1>Changelog</h1>
<section class="actions" aria-label="Changelog actions">
<a class="button-link secondary" href="{% url 'dashboard' %}">Back to dashboard</a>
</section>
<section class="panel">
<div class="stack spaced">
<div><strong>Installed version:</strong> {{ app_version }}</div>
<div class="muted">Source: {{ changelog_path }}</div>
{% if missing %}
<div class="status warning">missing</div>
{% endif %}
</div>
<div class="stack">
{% for block in changelog_blocks %}
{% if block.kind == "heading" %}
{% if block.level == 1 %}
<h2>{{ block.text }}</h2>
{% else %}
<h3>{{ block.text }}</h3>
{% endif %}
{% elif block.kind == "list" %}
<ul>
{% for item in block.items %}
<li>{{ item }}</li>
{% endfor %}
</ul>
{% else %}
<p>{{ block.text }}</p>
{% endif %}
{% endfor %}
</div>
</section>
{% endblock %}

View File

@@ -28,85 +28,236 @@
{% endif %} {% endif %}
<section class="grid" aria-label="Summary"> <section class="grid" aria-label="Summary">
<div class="metric"><div class="label">Global Configs</div><div class="value">{{ counts.global_configs }}</div></div>
<div class="metric"><div class="label">Hosts</div><div class="value">{{ counts.enabled_hosts }}/{{ counts.hosts }}</div></div> <div class="metric"><div class="label">Hosts</div><div class="value">{{ counts.enabled_hosts }}/{{ counts.hosts }}</div></div>
<div class="metric"><div class="label">Schedules</div><div class="value">{{ counts.enabled_schedules }}/{{ counts.schedules }}</div></div> <div class="metric"><div class="label">Schedules</div><div class="value">{{ counts.enabled_schedules }}/{{ counts.schedules }}</div></div>
<div class="metric"><div class="label">Snapshots</div><div class="value">{{ counts.snapshots }}</div></div> <div class="metric"><div class="label">Snapshots</div><div class="value">{{ counts.snapshots }}</div></div>
<div class="metric"><div class="label">Runs</div><div class="value">{{ counts.runs }}</div></div> <div class="metric"><div class="label">Runs</div><div class="value">{{ counts.runs }}</div></div>
<div class="metric"><div class="label">Running</div><div class="value">{{ counts.running_runs }}</div></div> <div class="metric {% if counts.queued_runs %}queued{% endif %}"><div class="label">Queued</div><div class="value">{{ counts.queued_runs }}</div></div>
<div class="metric"><div class="label">Failed</div><div class="value">{{ counts.failed_runs }}</div></div> <div class="metric {% if counts.running_runs %}running{% endif %}"><div class="label">Running</div><div class="value">{{ counts.running_runs }}</div></div>
<div class="metric {% if counts.warning_runs %}warning{% endif %}"><div class="label">Warnings</div><div class="value">{{ counts.warning_runs }}</div></div>
<div class="metric {% if counts.failed_runs %}failed{% endif %}"><div class="label">Failed</div><div class="value">{{ counts.failed_runs }}</div></div>
</section> </section>
{% if stats_summary.runs_sampled %} <section class="panel">
<section class="grid" aria-label="Backup trends"> <h2>Operational Status</h2>
<div class="metric"><div class="label">Backup Root Used</div><div class="value">{{ stats_summary.capacity.used_percent|default:"" }}{% if stats_summary.capacity.used_percent is not None %}%{% endif %}</div></div> {% if counts.failed_runs or counts.warning_runs or counts.running_runs or counts.queued_runs %}
<div class="metric"><div class="label">Available</div><div class="value">{{ stats_summary.capacity.available_bytes|filesizeformat }}</div></div> <div class="status-overview">
<div class="metric"><div class="label">Avg New Data</div><div class="value">{{ stats_summary.avg_literal_data_bytes|filesizeformat }}</div></div> {% if counts.failed_runs %}
<div class="metric"><div class="label">Avg Daily New</div><div class="value">{{ stats_summary.avg_daily_literal_data_bytes|filesizeformat }}</div></div> <div class="status-summary failed">
<div class="metric"><div class="label">Avg Duration</div><div class="value">{{ stats_summary.avg_duration_seconds|default:"" }}{% if stats_summary.avg_duration_seconds is not None %}s{% endif %}</div></div> <span class="status failed">failed</span>
<div class="metric"><div class="label">Link-Dest Savings</div><div class="value">{{ stats_summary.link_dest_savings_ratio|default:"" }}</div></div> <strong>{{ counts.failed_runs }} failed run{{ counts.failed_runs|pluralize }} need{{ counts.failed_runs|pluralize:"s," }} review.</strong>
<div class="metric"><div class="label">Runs Until Full</div><div class="value">{{ stats_summary.estimated_runs_until_full|default:"" }}</div></div> </div>
<div class="metric"><div class="label">Days Until Full</div><div class="value">{{ stats_summary.estimated_days_until_full|default:"" }}</div></div> {% endif %}
</section> {% if counts.warning_runs %}
{% endif %} <div class="status-summary warning">
<span class="status warning">warning</span>
<strong>{{ counts.warning_runs }} run{{ counts.warning_runs|pluralize }} completed with warnings.</strong>
</div>
{% endif %}
{% if counts.running_runs %}
<div class="status-summary running">
<span class="status running">running</span>
<strong>{{ counts.running_runs }} backup run{{ counts.running_runs|pluralize }} in progress.</strong>
</div>
{% endif %}
{% if counts.queued_runs %}
<div class="status-summary queued">
<span class="status queued">queued</span>
<strong>{{ counts.queued_runs }} backup run{{ counts.queued_runs|pluralize }} waiting for the worker.</strong>
</div>
{% endif %}
</div>
{% elif counts.hosts %}
<p><span class="status ok">ok</span> No queued, running, or unreviewed warning/failed runs.</p>
{% else %}
<p class="muted">Add a host to start tracking backup status here.</p>
{% endif %}
</section>
<section class="panel">
<h2>Backup Trends</h2>
{% if stats_summary.runs_sampled %}
<div class="insight-grid" aria-label="Backup trends">
<div class="insight-main">
<div class="label">Storage Used</div>
<div class="value">
{% if stats_summary.capacity.used_percent is not None %}
{{ stats_summary.capacity.used_percent|floatformat:1 }}%
{% else %}
unknown
{% endif %}
</div>
{% if stats_summary.capacity.used_percent is not None %}
<div class="storage-meter" aria-label="Backup root storage usage">
<span style="width: {{ stats_summary.capacity.used_percent|floatformat:0 }}%"></span>
</div>
{% endif %}
<div class="muted">
{{ stats_summary.capacity.available_bytes|filesizeformat }} available from the backup root.
</div>
</div>
<div class="insight-item">
<div class="label">Runway</div>
<div class="value">
{% if stats_summary.estimated_days_until_full %}
{{ stats_summary.estimated_days_until_full }} days
{% elif stats_summary.estimated_runs_until_full %}
{{ stats_summary.estimated_runs_until_full }} runs
{% else %}
unknown
{% endif %}
</div>
<div class="muted">Estimated from average new data per day.</div>
</div>
<div class="insight-item">
<div class="label">New Data</div>
<div class="value">{{ stats_summary.avg_daily_literal_data_bytes|filesizeformat }}/day</div>
<div class="muted">{{ stats_summary.avg_literal_data_bytes|filesizeformat }} per backup on average.</div>
</div>
<div class="insight-item">
<div class="label">Link-Dest Savings</div>
<div class="value">
{% if stats_summary.link_dest_savings_percent is not None %}
{{ stats_summary.link_dest_savings_percent|floatformat:1 }}%
{% else %}
unknown
{% endif %}
</div>
<div class="muted">{{ stats_summary.total_matched_data_bytes|filesizeformat }} reused across sampled runs.</div>
</div>
<div class="insight-item">
<div class="label">Average Duration</div>
<div class="value">{{ stats_summary.avg_duration_seconds|default:"unknown" }}{% if stats_summary.avg_duration_seconds is not None %}s{% endif %}</div>
<div class="muted">Based on {{ stats_summary.runs_sampled }} completed backup run{{ stats_summary.runs_sampled|pluralize }} with stats.</div>
</div>
</div>
{% else %}
<p class="muted">No completed backup runs with stats yet. This section will show disk usage, growth estimates, and link-dest savings after the first real backup finishes.</p>
{% endif %}
</section>
<section class="panel"> <section class="panel">
<h2>Hosts</h2> <h2>Hosts</h2>
<table> <div class="host-list">
<thead> {% for host in hosts %}
<tr> <article class="host-card">
<th>Host</th> <div class="host-card-header">
<th>Address</th> <div class="host-card-title">
<th>Enabled</th> <a href="{% url 'host_detail' host.host %}">{{ host.host }}</a>
<th>Snapshots</th> <span class="muted">{{ host.address }}</span>
<th>Latest Snapshot</th> </div>
<th>Latest Run</th> <div class="host-card-status">
<th>Next Run</th> <span class="status {% if host.enabled %}ok{% else %}skipped{% endif %}">{{ host.enabled|yesno:"enabled,disabled" }}</span>
<th>New Data</th> {% if host.queued_run_count %}
<th>Runs</th> <span class="status queued">queued {{ host.queued_run_count }}</span>
<th>Retention</th>
</tr>
</thead>
<tbody>
{% for host in hosts %}
<tr>
<td><a href="{% url 'host_detail' host.host %}">{{ host.host }}</a></td>
<td>{{ host.address }}</td>
<td>{{ host.enabled|yesno:"yes,no" }}</td>
<td>{{ host.snapshot_count }}</td>
<td>
{% if host.latest_snapshot %}
<a href="{% url 'snapshot_detail' host.latest_snapshot.id %}">{{ host.latest_snapshot.dirname }}</a>
<div class="muted">{{ host.latest_snapshot.kind }} {{ host.latest_snapshot.status }}</div>
{% else %}
<span class="muted">none</span>
{% endif %} {% endif %}
</td> {% if host.running_run_count %}
<td> <span class="status running">running {{ host.running_run_count }}</span>
{% if host.stats_summary.latest_run.id %}
<a href="{% url 'run_detail' host.stats_summary.latest_run.id %}">Run {{ host.stats_summary.latest_run.id }}</a>
<div class="muted">{{ host.stats_summary.latest_run.run_type }} {{ host.stats_summary.latest_run.duration_seconds|default:"" }}{% if host.stats_summary.latest_run.duration_seconds is not None %}s{% endif %}</div>
{% else %}
<span class="muted">none</span>
{% endif %} {% endif %}
</td> {% if host.warning_run_count %}
<td> <span class="status warning">warning {{ host.warning_run_count }}</span>
{% if host.next_run_at %}
{{ host.next_run_at|date:"Y-m-d H:i T" }}
<div class="muted">{{ scheduler_timezone }}</div>
{% else %}
<span class="muted">none</span>
{% endif %} {% endif %}
</td> {% if host.failed_run_count %}
<td>{{ host.stats_summary.latest_run.rsync.literal_data_bytes|filesizeformat }}</td> <span class="status failed">failed {{ host.failed_run_count }}</span>
<td>{{ host.run_count }}</td> {% endif %}
<td>d{{ host.retention_daily }} w{{ host.retention_weekly }} m{{ host.retention_monthly }} y{{ host.retention_yearly }}</td> </div>
</tr> </div>
{% empty %} <div class="host-card-layout">
<tr><td colspan="10" class="muted">No hosts configured yet.</td></tr> <div class="host-card-section">
{% endfor %} <div class="host-card-section-title">Backup activity</div>
</tbody> <div class="host-card-timeline">
</table> <div class="host-card-item">
<div class="label">Latest Snapshot</div>
<div class="value">
{% if host.latest_snapshot %}
<a href="{% url 'snapshot_detail' host.latest_snapshot.id %}">{{ host.latest_snapshot.dirname }}</a>
<div class="muted">{{ host.latest_snapshot.kind }} {{ host.latest_snapshot.status }}</div>
{% else %}
<span class="muted">none</span>
{% endif %}
</div>
</div>
<div class="host-card-item">
<div class="label">Last Good Backup</div>
<div class="value">
{% if host.stats_summary.latest_good_run.id %}
<a href="{% url 'run_detail' host.stats_summary.latest_good_run.id %}">Run {{ host.stats_summary.latest_good_run.id }}</a>
<div class="muted">{{ host.stats_summary.latest_good_run.run_type }} {{ host.stats_summary.latest_good_run.duration_seconds|default:"" }}{% if host.stats_summary.latest_good_run.duration_seconds is not None %}s{% endif %}</div>
{% else %}
<span class="muted">none</span>
{% endif %}
</div>
</div>
<div class="host-card-item">
<div class="label">Latest Issue</div>
<div class="value">
{% if host.stats_summary.latest_problem_run.id %}
<a href="{% url 'run_detail' host.stats_summary.latest_problem_run.id %}">Run {{ host.stats_summary.latest_problem_run.id }}</a>
<div><span class="status {{ host.stats_summary.latest_problem_run.status }}">{{ host.stats_summary.latest_problem_run.status }}</span></div>
<div class="muted">{{ host.stats_summary.latest_problem_run.run_type }} {{ host.stats_summary.latest_problem_run.duration_seconds|default:"" }}{% if host.stats_summary.latest_problem_run.duration_seconds is not None %}s{% endif %}</div>
{% else %}
<span class="muted">none</span>
{% endif %}
</div>
</div>
<div class="host-card-item">
<div class="label">Next Run</div>
<div class="value">
{% if host.next_run_at %}
{{ host.next_run_at|date:"Y-m-d H:i T" }}
<div class="muted">{{ scheduler_timezone }}</div>
{% else %}
<span class="muted">none</span>
{% endif %}
</div>
</div>
</div>
</div>
<div class="host-card-section">
<div class="host-card-section-title">Snapshot health</div>
<div class="host-card-stats">
<div class="host-card-stat">
<div class="label">Snapshots</div>
<div class="value">{{ host.snapshot_count }}</div>
</div>
<div class="host-card-stat">
<div class="label">Runs</div>
<div class="value">{{ host.run_count }}</div>
</div>
<div class="host-card-stat">
<div class="label">New Data</div>
<div class="value">{{ host.stats_summary.latest_run.rsync.literal_data_bytes|filesizeformat }}</div>
</div>
<div class="host-card-stat">
<div class="label">Retention</div>
<div class="value">d{{ host.retention_daily }} w{{ host.retention_weekly }} m{{ host.retention_monthly }} y{{ host.retention_yearly }}</div>
</div>
</div>
</div>
</div>
{% if host.retention_warning.has_warning %}
<div class="host-card-warning">
<span class="status warning">retention</span>
{% if host.retention_warning.prune_exceeded %}
Scheduled prune would delete {{ host.retention_warning.delete_count }} snapshot(s), above max {{ host.retention_warning.max_delete }}.
{% endif %}
{% if host.retention_warning.incomplete_count %}
{{ host.retention_warning.incomplete_count }} incomplete snapshot(s) need review.
<form method="post" action="{% url 'resolve_host_incomplete_reviews' host.host %}">
{% csrf_token %}
<button type="submit" class="secondary">Mark reviewed</button>
</form>
{% endif %}
{% if host.retention_warning.error %}
{{ host.retention_warning.error }}
{% endif %}
</div>
{% endif %}
</article>
{% empty %}
<p class="muted">No hosts configured yet.</p>
{% endfor %}
</div>
</section> </section>
<section class="panel"> <section class="panel">

View File

@@ -21,6 +21,10 @@
{% csrf_token %} {% csrf_token %}
<button type="submit" class="secondary">Scan SSH host key</button> <button type="submit" class="secondary">Scan SSH host key</button>
</form> </form>
<form method="post" action="{% url 'run_host_preflight' host.host %}">
{% csrf_token %}
<button type="submit" class="secondary">Run connection preflight</button>
</form>
</section> </section>
<section class="grid" aria-label="Host summary"> <section class="grid" aria-label="Host summary">
@@ -64,6 +68,74 @@
</section> </section>
</div> </div>
{% if retention_warning.has_warning %}
<section class="panel highlight warning">
<h2>Retention Warnings</h2>
<div class="stack">
{% if retention_warning.prune_exceeded %}
<div>
Scheduled pruning would delete {{ retention_warning.delete_count }} snapshot(s), above max delete
{{ retention_warning.max_delete }}. Scheduled pruning will refuse this plan until the limit or retention
selection is adjusted.
</div>
{% endif %}
{% if retention_warning.incomplete_count %}
<div>
{{ retention_warning.incomplete_count }} incomplete snapshot(s) exist. Retention does not delete incomplete
snapshots automatically; inspect them before cleanup.
</div>
<form method="post" action="{% url 'resolve_host_incomplete_reviews' host.host %}">
{% csrf_token %}
<button type="submit" class="secondary">Mark incomplete reviewed</button>
</form>
{% endif %}
{% if retention_warning.error %}
<div>{{ retention_warning.error }}</div>
{% endif %}
</div>
</section>
{% endif %}
{% if effective_config %}
<section class="panel">
<h2>Effective Config</h2>
<div class="two-col">
<div class="stack">
<div><strong>Source root:</strong> {{ effective_config.source_root }}</div>
<div><strong>Destination subdir:</strong> {{ effective_config.destination_subdir|default:"none" }}</div>
<div><strong>SSH:</strong> {{ effective_config.ssh.user }}@{{ host.address }}:{{ effective_config.ssh.port }}</div>
<div><strong>SSH key:</strong> {{ effective_config.ssh.credential|default:"none selected" }}</div>
<div><strong>SSH options:</strong> {{ effective_config.ssh.options|join:" " }}</div>
<div><strong>Rsync binary:</strong> {{ effective_config.rsync.binary }}</div>
<div><strong>Rsync args:</strong> {{ effective_config.rsync.args|join:" " }}</div>
<div><strong>Timeout:</strong> {{ effective_config.rsync.timeout_seconds }}s</div>
<div><strong>Bandwidth limit:</strong> {{ effective_config.rsync.bwlimit_kbps }} KB/s</div>
<div>
<strong>Retention:</strong>
d{{ effective_config.retention.daily }}
w{{ effective_config.retention.weekly }}
m{{ effective_config.retention.monthly }}
y{{ effective_config.retention.yearly }}
</div>
</div>
<div class="stack">
<div><strong>Includes:</strong> {{ effective_config.includes|length }}</div>
{% if effective_config.includes %}
<pre>{{ effective_config.includes|join:"&#10;" }}</pre>
{% else %}
<div class="muted">No include rules configured.</div>
{% endif %}
<div><strong>Excludes:</strong> {{ effective_config.excludes|length }}</div>
{% if effective_config.excludes %}
<pre>{{ effective_config.excludes|join:"&#10;" }}</pre>
{% else %}
<div class="muted">No exclude rules configured.</div>
{% endif %}
</div>
</div>
</section>
{% endif %}
<section class="panel"> <section class="panel">
<h2>Snapshot Discovery</h2> <h2>Snapshot Discovery</h2>
<div class="stack"> <div class="stack">
@@ -159,14 +231,47 @@
</table> </table>
</section> </section>
{% if last_preflight %}
<section class="panel">
<h2>Connection Preflight</h2>
<div class="stack spaced">
<div><strong>Status:</strong> <span class="status {% if last_preflight.ok %}ok{% else %}failed{% endif %}">{% if last_preflight.ok %}ok{% else %}failed{% endif %}</span></div>
<div><strong>Target:</strong> {{ last_preflight.target }}</div>
<div><strong>Source root:</strong> {{ last_preflight.source_root }}</div>
<div><strong>Remote rsync:</strong> {{ last_preflight.rsync_binary }}</div>
</div>
<table>
<thead>
<tr>
<th>Status</th>
<th>Check</th>
<th>Message</th>
<th>Detail</th>
</tr>
</thead>
<tbody>
{% for check in last_preflight.checks %}
<tr>
<td><span class="status {% if check.ok %}ok{% else %}failed{% endif %}">{% if check.ok %}ok{% else %}failed{% endif %}</span></td>
<td>{{ check.name }}</td>
<td>{{ check.message }}</td>
<td class="muted">{{ check.detail }}</td>
</tr>
{% endfor %}
</tbody>
</table>
</section>
{% endif %}
<section class="panel"> <section class="panel">
<h2>Backup Control</h2> <h2>Backup Control</h2>
<div class="operator-state"> <div class="operator-state">
{% if active_run %} {% if active_run %}
<span class="status {{ active_run.status }}">{{ active_run.status }}</span> <span class="status {{ active_run.status }}">{{ active_run.status }}</span>
<a href="{% url 'run_detail' active_run.id %}">Run {{ active_run.id }}</a> <a href="{% url 'run_detail' active_run.id %}">Run {{ active_run.id }}</a>
{% elif can_queue_backup %} {% elif has_global_config and host.enabled %}
<span class="status success">ready</span> <span class="status {{ backup_gate.state }}">{{ backup_gate.state }}</span>
<span class="muted">{{ backup_gate.message }}</span>
{% elif not host.enabled %} {% elif not host.enabled %}
<span class="status failed">disabled</span> <span class="status failed">disabled</span>
{% elif not has_global_config %} {% elif not has_global_config %}
@@ -180,20 +285,24 @@
<input type="hidden" name="dry_run" value="on"> <input type="hidden" name="dry_run" value="on">
<input type="hidden" name="verbose_output" value="on"> <input type="hidden" name="verbose_output" value="on">
<input type="hidden" name="prune_max_delete" value="10"> <input type="hidden" name="prune_max_delete" value="10">
<button type="submit" class="secondary" {% if not can_queue_backup %}disabled{% endif %}>Queue dry-run</button> <button type="submit" class="secondary" {% if not can_queue_dry_run %}disabled{% endif %}>Queue dry-run</button>
</form> </form>
<form class="inline-form" method="post" action="{% url 'queue_manual_backup' host.host %}"> <form class="inline-form" method="post" action="{% url 'queue_manual_backup' host.host %}">
{% csrf_token %} {% csrf_token %}
<input type="hidden" name="prune_max_delete" value="10"> <input type="hidden" name="prune_max_delete" value="10">
<button type="submit" {% if not can_queue_backup %}disabled{% endif %}>Queue backup</button> <button type="submit" {% if not can_queue_real_backup %}disabled{% endif %}>Queue backup</button>
</form> </form>
</section> </section>
{% if not can_queue_backup %} {% if active_run %}
<p class="muted">Wait for the active run to finish, or cancel it from the run detail page.</p>
{% elif not can_queue_dry_run or not can_queue_real_backup %}
{% if not has_global_config %} {% if not has_global_config %}
<p class="muted">Create the default global config before queueing backups.</p> <p class="muted">Create the default global config before queueing backups.</p>
{% elif not host.enabled %} {% elif not host.enabled %}
<p class="muted">Enable this host before queueing backups.</p> <p class="muted">Enable this host before queueing backups.</p>
{% elif backup_gate.real_blockers %}
<p class="muted">Real backups are blocked by failed preflight checks. Dry-runs may still be available when storage-only checks fail.</p>
{% endif %} {% endif %}
{% endif %} {% endif %}
@@ -212,7 +321,7 @@
{% endfor %} {% endfor %}
<div class="actions"> <div class="actions">
<button type="submit" {% if not can_queue_backup %}disabled{% endif %}>Queue with options</button> <button type="submit" {% if not can_queue_dry_run and not can_queue_real_backup %}disabled{% endif %}>Queue with options</button>
</div> </div>
</form> </form>
</section> </section>

View File

@@ -29,6 +29,22 @@
{% endfor %} {% endfor %}
</select> </select>
</div> </div>
<div class="field">
<label for="window">Time window</label>
<select id="window" name="window">
{% for value, label in time_windows.items %}
<option value="{{ value }}" {% if selected_window == value %}selected{% endif %}>{{ label }}</option>
{% endfor %}
</select>
</div>
<div class="field">
<label for="host">Host contains</label>
<input id="host" name="host" value="{{ host_filter }}" placeholder="web-01.example.test">
</div>
<div class="field">
<label for="run">Run</label>
<input id="run" name="run" value="{{ run_filter }}" inputmode="numeric" placeholder="12">
</div>
<div class="field"> <div class="field">
<label for="q">Message contains</label> <label for="q">Message contains</label>
<input id="q" name="q" value="{{ query }}"> <input id="q" name="q" value="{{ query }}">

View File

@@ -0,0 +1,74 @@
{% extends "pobsync_backend/base.html" %}
{% block title %}Purged Snapshots | pobsync{% endblock %}
{% block content %}
<h1>Purged Snapshots</h1>
<section class="actions" aria-label="Purged snapshot actions">
<a class="button-link secondary" href="{% url 'dashboard' %}">Back to dashboard</a>
</section>
<section class="panel">
<h2>Filters</h2>
<form method="get" class="form-grid">
<div class="field">
<label for="host">Host</label>
<select id="host" name="host">
<option value="">All hosts</option>
{% for host in hosts %}
<option value="{{ host.host }}" {% if selected_host == host.host %}selected{% endif %}>{{ host.host }}</option>
{% endfor %}
</select>
</div>
<div class="field">
<label for="action">Action</label>
<select id="action" name="action">
<option value="">All actions</option>
{% for value, label in actions %}
<option value="{{ value }}" {% if selected_action == value %}selected{% endif %}>{{ label }}</option>
{% endfor %}
</select>
</div>
<div class="actions">
<button type="submit">Apply filters</button>
<a class="button-link secondary" href="{% url 'purged_snapshots' %}">Clear</a>
</div>
</form>
</section>
<section class="panel">
<h2>Purged Snapshot History</h2>
<p class="muted">Showing up to 200 of {{ total_count }} purged snapshot record(s).</p>
<table>
<thead>
<tr>
<th>Purged</th>
<th>Host</th>
<th>Kind</th>
<th>Dirname</th>
<th>Action</th>
<th>Reason</th>
<th>Triggered by</th>
<th>Path</th>
</tr>
</thead>
<tbody>
{% for snapshot in purged_snapshots %}
<tr>
<td>{{ snapshot.purged_at }}</td>
<td>{% if snapshot.host %}<a href="{% url 'host_detail' snapshot.host.host %}">{{ snapshot.host_name }}</a>{% else %}{{ snapshot.host_name }}{% endif %}</td>
<td>{{ snapshot.kind }}</td>
<td>{{ snapshot.dirname }}</td>
<td><span class="status skipped">{{ snapshot.get_action_display }}</span></td>
<td>{{ snapshot.reason|default:"" }}</td>
<td>{{ snapshot.triggered_by|default:"" }}</td>
<td class="muted">{{ snapshot.path }}</td>
</tr>
{% empty %}
<tr><td colspan="8" class="muted">No purged snapshots recorded yet.</td></tr>
{% endfor %}
</tbody>
</table>
</section>
{% endblock %}

View File

@@ -18,8 +18,35 @@
<div class="metric"><div class="label">Kind</div><div class="value">{{ plan.kind }}</div></div> <div class="metric"><div class="label">Kind</div><div class="value">{{ plan.kind }}</div></div>
<div class="metric"><div class="label">Keep</div><div class="value">{{ plan.keep|length }}</div></div> <div class="metric"><div class="label">Keep</div><div class="value">{{ plan.keep|length }}</div></div>
<div class="metric"><div class="label">Would Delete</div><div class="value">{{ plan.delete|length }}</div></div> <div class="metric"><div class="label">Would Delete</div><div class="value">{{ plan.delete|length }}</div></div>
<div class="metric"><div class="label">Scheduled Limit</div><div class="value">{{ scheduled_prune_limit|default:"none" }}</div></div>
<div class="metric"><div class="label">Incomplete</div><div class="value">{{ plan.incomplete|length }}</div></div>
</section> </section>
{% if scheduled_prune_exceeded %}
<section class="panel highlight warning">
<h2>Scheduled Prune Limit</h2>
<p>
This plan would delete {{ plan.delete|length }} snapshot(s), which exceeds the scheduled prune limit of
{{ scheduled_prune_limit }}. Scheduled pruning will refuse to apply this plan until the limit or retention
selection is adjusted.
</p>
</section>
{% endif %}
{% if plan.incomplete %}
<section class="panel highlight warning">
<h2>Incomplete Snapshots</h2>
<p>
{{ plan.incomplete|length }} incomplete snapshot(s) exist for this host. Retention does not delete incomplete
snapshots automatically because they can indicate an interrupted backup that should be inspected first.
</p>
<p>
After inspection, use the dedicated cleanup form below to delete only incomplete snapshot directories and their
SQL records. Successful scheduled and manual snapshots are not touched by this cleanup.
</p>
</section>
{% endif %}
<section class="panel"> <section class="panel">
<h2>Policy</h2> <h2>Policy</h2>
<div class="stack"> <div class="stack">
@@ -28,6 +55,17 @@
<div><strong>Monthly:</strong> {{ plan.retention.monthly }}</div> <div><strong>Monthly:</strong> {{ plan.retention.monthly }}</div>
<div><strong>Yearly:</strong> {{ plan.retention.yearly }}</div> <div><strong>Yearly:</strong> {{ plan.retention.yearly }}</div>
<div><strong>Protect bases:</strong> {{ protect_bases|yesno:"yes,no" }}</div> <div><strong>Protect bases:</strong> {{ protect_bases|yesno:"yes,no" }}</div>
<div class="muted">
{% if protect_bases %}
Base snapshots referenced by kept snapshots are also kept and marked with a base-of reason.
{% else %}
Base snapshots are only kept when they match the regular retention policy.
{% endif %}
</div>
{% if schedule %}
<div><strong>Schedule pruning:</strong> {{ schedule.prune|yesno:"enabled,disabled" }}</div>
<div><strong>Schedule max delete:</strong> {{ schedule.prune_max_delete }}</div>
{% endif %}
</div> </div>
</section> </section>
@@ -40,6 +78,7 @@
<th>Dirname</th> <th>Dirname</th>
<th>Started</th> <th>Started</th>
<th>Status</th> <th>Status</th>
<th>Reason</th>
<th>Path</th> <th>Path</th>
</tr> </tr>
</thead> </thead>
@@ -50,10 +89,11 @@
<td>{{ snapshot.dirname }}</td> <td>{{ snapshot.dirname }}</td>
<td>{{ snapshot.dt }}</td> <td>{{ snapshot.dt }}</td>
<td>{{ snapshot.status|default:"" }}</td> <td>{{ snapshot.status|default:"" }}</td>
<td>{{ snapshot.reason }}</td>
<td class="muted">{{ snapshot.path }}</td> <td class="muted">{{ snapshot.path }}</td>
</tr> </tr>
{% empty %} {% empty %}
<tr><td colspan="5" class="muted">Retention would not delete snapshots for this selection.</td></tr> <tr><td colspan="6" class="muted">Retention would not delete snapshots for this selection.</td></tr>
{% endfor %} {% endfor %}
</tbody> </tbody>
</table> </table>
@@ -71,7 +111,7 @@
{{ apply_form.max_delete.errors }} {{ apply_form.max_delete.errors }}
<label for="{{ apply_form.max_delete.id_for_label }}">Max delete</label> <label for="{{ apply_form.max_delete.id_for_label }}">Max delete</label>
{{ apply_form.max_delete }} {{ apply_form.max_delete }}
<div class="helptext">Must be at least the number of snapshots shown in Would Delete.</div> <div class="helptext">Must be at least {{ plan.delete|length }} for the snapshots shown in Would Delete.</div>
</div> </div>
<div class="field"> <div class="field">
@@ -87,6 +127,13 @@
<div class="helptext">{{ apply_form.confirm_host.help_text }}</div> <div class="helptext">{{ apply_form.confirm_host.help_text }}</div>
</div> </div>
<div class="field">
{{ apply_form.confirm_delete_count.errors }}
<label for="{{ apply_form.confirm_delete_count.id_for_label }}">Confirm delete count</label>
{{ apply_form.confirm_delete_count }}
<div class="helptext">{{ apply_form.confirm_delete_count.help_text }}</div>
</div>
<div class="actions"> <div class="actions">
<button type="submit">Apply retention</button> <button type="submit">Apply retention</button>
</div> </div>
@@ -99,20 +146,85 @@
<table> <table>
<thead> <thead>
<tr> <tr>
<th>Kind</th>
<th>Dirname</th> <th>Dirname</th>
<th>Started</th>
<th>Status</th>
<th>Reasons</th> <th>Reasons</th>
</tr> </tr>
</thead> </thead>
<tbody> <tbody>
{% for dirname, reasons in plan.reasons.items %} {% for snapshot in plan.keep_items %}
<tr> <tr>
<td>{{ dirname }}</td> <td>{{ snapshot.kind }}</td>
<td>{{ reasons|join:", " }}</td> <td>{{ snapshot.dirname }}</td>
<td>{{ snapshot.dt }}</td>
<td>{{ snapshot.status|default:"" }}</td>
<td>{{ snapshot.reason }}</td>
</tr> </tr>
{% empty %} {% empty %}
<tr><td colspan="2" class="muted">No snapshots matched this retention selection.</td></tr> <tr><td colspan="5" class="muted">No snapshots matched this retention selection.</td></tr>
{% endfor %} {% endfor %}
</tbody> </tbody>
</table> </table>
</section> </section>
{% if plan.incomplete %}
<section class="panel">
<h2>Incomplete Snapshots</h2>
<table>
<thead>
<tr>
<th>Dirname</th>
<th>Started</th>
<th>Status</th>
<th>Reason</th>
<th>Path</th>
</tr>
</thead>
<tbody>
{% for snapshot in plan.incomplete %}
<tr>
<td>{{ snapshot.dirname }}</td>
<td>{{ snapshot.dt }}</td>
<td>{{ snapshot.status|default:"" }}</td>
<td>{{ snapshot.reason }}</td>
<td class="muted">{{ snapshot.path }}</td>
</tr>
{% endfor %}
</tbody>
</table>
<h3>Cleanup Incomplete Snapshots</h3>
<form method="post" action="{% url 'cleanup_host_incomplete_snapshots' host.host %}" class="form-grid">
{% csrf_token %}
{{ incomplete_cleanup_form.non_field_errors }}
<div class="field">
{{ incomplete_cleanup_form.max_delete.errors }}
<label for="{{ incomplete_cleanup_form.max_delete.id_for_label }}">Max delete</label>
{{ incomplete_cleanup_form.max_delete }}
<div class="helptext">{{ incomplete_cleanup_form.max_delete.help_text }}</div>
</div>
<div class="field">
{{ incomplete_cleanup_form.confirm_host.errors }}
<label for="{{ incomplete_cleanup_form.confirm_host.id_for_label }}">Confirm host</label>
{{ incomplete_cleanup_form.confirm_host }}
<div class="helptext">{{ incomplete_cleanup_form.confirm_host.help_text }}</div>
</div>
<div class="field">
{{ incomplete_cleanup_form.confirm_delete_count.errors }}
<label for="{{ incomplete_cleanup_form.confirm_delete_count.id_for_label }}">Confirm incomplete count</label>
{{ incomplete_cleanup_form.confirm_delete_count }}
<div class="helptext">{{ incomplete_cleanup_form.confirm_delete_count.help_text }}</div>
</div>
<div class="actions">
<button type="submit">Delete incomplete snapshots</button>
</div>
</form>
</section>
{% endif %}
{% endblock %} {% endblock %}

View File

@@ -13,15 +13,83 @@
<button type="submit" class="secondary">Cancel run</button> <button type="submit" class="secondary">Cancel run</button>
</form> </form>
{% endif %} {% endif %}
{% if run.status == "failed" or run.status == "warning" %}
{% if not run.reviewed_at %}
<form method="post" action="{% url 'resolve_run_review' run.id %}">
{% csrf_token %}
<button type="submit" class="secondary">Mark reviewed</button>
</form>
{% endif %}
{% endif %}
</section> </section>
<section class="grid" aria-label="Run summary"> <section class="grid" aria-label="Run summary">
<div class="metric"><div class="label">Host</div><div class="value">{{ run.host.host }}</div></div> <div class="metric"><div class="label">Host</div><div class="value">{{ run.host.host }}</div></div>
<div class="metric"><div class="label">Status</div><div class="value">{{ run.status }}</div></div> <div class="metric"><div class="label">Status</div><div class="value"><span class="status {{ run.status }}">{{ run.status }}</span></div></div>
<div class="metric"><div class="label">Type</div><div class="value">{{ run.run_type }}</div></div> <div class="metric"><div class="label">Type</div><div class="value">{{ run.run_type }}</div></div>
<div class="metric"><div class="label">Rsync</div><div class="value">{{ run.rsync_exit_code|default:"" }}</div></div> <div class="metric"><div class="label">Rsync</div><div class="value">{{ run.rsync_exit_code|default:"" }}</div></div>
</section> </section>
{% if failure %}
<section class="panel highlight failed">
<h2>Failure</h2>
<div class="stack">
<div><strong>Category:</strong> {{ failure.category|default:"unknown" }}</div>
<div><strong>Summary:</strong> {{ failure_summary }}</div>
<div><strong>Hint:</strong> {{ failure.hint|default:"" }}</div>
</div>
</section>
{% endif %}
{% if run.reviewed_at %}
<section class="panel highlight success">
<h2>Review</h2>
<div class="stack">
<div><strong>Reviewed:</strong> {{ run.reviewed_at }}</div>
<div><strong>Reviewed by:</strong> {{ run.reviewed_by|default:"unknown" }}</div>
</div>
</section>
{% endif %}
{% if dry_run_summary %}
<section class="panel highlight {{ dry_run_summary.highlight_class }}">
<h2>Dry Run Summary</h2>
<section class="grid" aria-label="Dry run summary">
<div class="metric"><div class="label">Status</div><div class="value">{{ dry_run_summary.status }}</div></div>
<div class="metric"><div class="label">Files Seen</div><div class="value">{{ dry_run_summary.files_seen|default:"unknown" }}</div></div>
<div class="metric"><div class="label">Would Transfer</div><div class="value">{{ dry_run_summary.files_would_transfer|default:"unknown" }}</div></div>
<div class="metric"><div class="label">Transfer Estimate</div><div class="value">{{ dry_run_summary.transfer_estimate_bytes|filesizeformat }}</div></div>
<div class="metric"><div class="label">Total Size</div><div class="value">{{ dry_run_summary.total_file_size_bytes|filesizeformat }}</div></div>
<div class="metric"><div class="label">Link-Dest Saving</div><div class="value">{{ dry_run_summary.link_dest_estimated_savings_bytes|filesizeformat }}</div></div>
</section>
<div class="stack">
{% if dry_run_summary.duration_seconds is not None %}
<div><strong>Duration:</strong> {{ dry_run_summary.duration_seconds }}s</div>
{% endif %}
<div>
<strong>Log:</strong>
{% if dry_run_summary.log_available %}
<a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a>
{% elif rsync_log_path %}
<span class="muted">{{ rsync_log_path }} (missing)</span>
{% else %}
<span class="muted">not recorded yet</span>
{% endif %}
</div>
{% if dry_run_summary.warnings %}
<div><strong>Warnings:</strong></div>
<ul>
{% for warning in dry_run_summary.warnings %}
<li>{{ warning }}</li>
{% endfor %}
</ul>
{% else %}
<div><strong>Warnings:</strong> none recorded</div>
{% endif %}
</div>
</section>
{% endif %}
<div class="two-col"> <div class="two-col">
<section class="panel"> <section class="panel">
<h2>Timing</h2> <h2>Timing</h2>
@@ -29,6 +97,10 @@
<div><strong>Created:</strong> {{ run.created_at }}</div> <div><strong>Created:</strong> {{ run.created_at }}</div>
<div><strong>Started:</strong> {{ run.started_at|default:"" }}</div> <div><strong>Started:</strong> {{ run.started_at|default:"" }}</div>
<div><strong>Ended:</strong> {{ run.ended_at|default:"" }}</div> <div><strong>Ended:</strong> {{ run.ended_at|default:"" }}</div>
{% if execution %}
<div><strong>Worker:</strong> {{ execution.worker_host|default:"unknown" }}{% if execution.worker_pid %} pid {{ execution.worker_pid }}{% endif %}</div>
<div><strong>Worker heartbeat:</strong> {{ execution.heartbeat_at|default:"" }}</div>
{% endif %}
</div> </div>
</section> </section>
@@ -37,6 +109,16 @@
<div class="stack"> <div class="stack">
<div><strong>Snapshot:</strong> {% if run.snapshot %}<a href="{% url 'snapshot_detail' run.snapshot.id %}">{{ run.snapshot.dirname }}</a>{% else %}{{ run.snapshot_path|default:"" }}{% endif %}</div> <div><strong>Snapshot:</strong> {% if run.snapshot %}<a href="{% url 'snapshot_detail' run.snapshot.id %}">{{ run.snapshot.dirname }}</a>{% else %}{{ run.snapshot_path|default:"" }}{% endif %}</div>
<div><strong>Base:</strong> {{ run.base_path|default:"" }}</div> <div><strong>Base:</strong> {{ run.base_path|default:"" }}</div>
<div>
<strong>Rsync log:</strong>
{% if rsync_log_exists %}
<a href="{% url 'run_rsync_log' run.id %}">{{ rsync_log_path }}</a>
{% elif rsync_log_path %}
<span class="muted">{{ rsync_log_path }} (missing)</span>
{% else %}
<span class="muted">none</span>
{% endif %}
</div>
</div> </div>
</section> </section>
</div> </div>
@@ -54,6 +136,36 @@
</section> </section>
{% endif %} {% endif %}
<section class="panel">
<h2>Rsync Command</h2>
{% if rsync_command %}
<pre>{% for part in rsync_command %}{{ part }}{% if not forloop.last %}
{% endif %}{% endfor %}</pre>
{% else %}
<p class="muted">No rsync command recorded yet.</p>
{% endif %}
</section>
<section class="panel">
<h2>Rsync Log</h2>
<div class="stack spaced">
{% if rsync_log_exists %}
<div><a href="{% url 'run_rsync_log' run.id %}">Open full rsync log</a></div>
<div class="muted">{{ rsync_log_path }}</div>
{% elif rsync_log_path %}
<div class="muted">{{ rsync_log_path }} (missing)</div>
{% else %}
<div class="muted">No rsync log path recorded yet.</div>
{% endif %}
</div>
{% if rsync_log_tail %}
<pre>{% for line in rsync_log_tail %}{{ line }}{% if not forloop.last %}
{% endif %}{% endfor %}</pre>
{% else %}
<p class="muted">No recent rsync log output recorded yet.</p>
{% endif %}
</section>
{% if stats %} {% if stats %}
<section class="panel"> <section class="panel">
<h2>Stats</h2> <h2>Stats</h2>
@@ -76,8 +188,57 @@
</section> </section>
{% endif %} {% endif %}
{% if has_prune_result %}
<section class="panel highlight {% if prune_result.ok %}success{% else %}warning{% endif %}">
<h2>Retention</h2>
<div class="stack">
<div><strong>Status:</strong> {% if prune_result.ok %}ok{% else %}warning{% endif %}</div>
{% if prune_result.source %}<div><strong>Source:</strong> {{ prune_result.source }}</div>{% endif %}
{% if prune_result.kind %}<div><strong>Kind:</strong> {{ prune_result.kind }}</div>{% endif %}
{% if prune_result.planned_delete_count is not None %}<div><strong>Planned deletions:</strong> {{ prune_result.planned_delete_count }}</div>{% endif %}
{% if prune_result.deleted %}<div><strong>Deleted:</strong> {{ prune_result.deleted|length }}</div>{% endif %}
{% if prune_result.max_delete is not None %}<div><strong>Max delete:</strong> {{ prune_result.max_delete }}</div>{% endif %}
{% if prune_result.protect_bases is not None %}<div><strong>Protect bases:</strong> {{ prune_result.protect_bases|yesno:"yes,no" }}</div>{% endif %}
{% if prune_result.incomplete_ignored_count %}<div><strong>Incomplete ignored:</strong> {{ prune_result.incomplete_ignored_count }}</div>{% endif %}
{% if prune_result.actions %}
<div><strong>Actions:</strong></div>
<ul>
{% for action in prune_result.actions %}
<li>{{ action }}</li>
{% endfor %}
</ul>
{% endif %}
{% if prune_result.error %}<div><strong>Error:</strong> {{ prune_result.error }}</div>{% endif %}
{% if prune_result.type %}<div><strong>Type:</strong> {{ prune_result.type }}</div>{% endif %}
</div>
</section>
{% endif %}
{% if retention_warning.has_warning %}
<section class="panel highlight warning">
<h2>Retention Warnings</h2>
<div class="stack">
{% if retention_warning.prune_exceeded %}
<div>
Scheduled pruning for this host would delete {{ retention_warning.delete_count }} snapshot(s), above max
delete {{ retention_warning.max_delete }}.
</div>
{% endif %}
{% if retention_warning.incomplete_count %}
<div>
{{ retention_warning.incomplete_count }} incomplete snapshot(s) exist for this host and are excluded from
retention cleanup.
</div>
{% endif %}
{% if retention_warning.error %}
<div>{{ retention_warning.error }}</div>
{% endif %}
</div>
</section>
{% endif %}
<section class="panel"> <section class="panel">
<h2>Result</h2> <h2>Raw Result</h2>
<pre>{{ result_json }}</pre> <pre>{{ result_json }}</pre>
</section> </section>
{% endblock %} {% endblock %}

View File

@@ -60,6 +60,48 @@
</section> </section>
{% endif %} {% endif %}
<section class="panel">
<h2>Restore Guidance</h2>
<div class="stack spaced">
<div><strong>Snapshot data source:</strong> {{ restore.source_path }}</div>
<div><strong>Example staging destination:</strong> {{ restore.destination_path }}</div>
<div class="muted">
Restore from the snapshot's <code>data/</code> directory. Start with a dry run, restore to a staging path first,
and only then copy data back to a live host or service path.
</div>
</div>
<div class="stack spaced">
<div><strong>Inspect the snapshot:</strong></div>
<pre>{{ restore.inspect_command }}</pre>
</div>
<div class="stack spaced">
<div><strong>Dry-run restore to staging:</strong></div>
<pre>{{ restore.dry_run_command }}</pre>
</div>
<div class="stack spaced">
<div><strong>Restore to staging:</strong></div>
<pre>{{ restore.local_command }}</pre>
</div>
<div class="stack spaced">
<div><strong>Dry-run a directory restore:</strong></div>
<pre>{{ restore.partial_dry_run_command }}</pre>
<div class="muted">Replace <code>{{ restore.example_relative_path }}</code> with the path you want to restore.</div>
</div>
<div class="stack spaced">
<div><strong>Dry-run a single file restore:</strong></div>
<pre>{{ restore.file_dry_run_command }}</pre>
<div class="muted">Replace <code>{{ restore.example_file_relative_path }}</code> with the file you want to restore.</div>
</div>
<div class="stack spaced">
<div><strong>Dry-run restore back to the source host:</strong></div>
<pre>{{ restore.remote_dry_run_command }}</pre>
</div>
<p class="muted">
Snapshots can contain hardlinks to files shared with earlier snapshots. Treat snapshot directories as read-only:
copy data out with rsync instead of editing files in place.
</p>
</section>
<section class="panel"> <section class="panel">
<h2>Backup Runs</h2> <h2>Backup Runs</h2>
<table> <table>

View File

@@ -45,9 +45,21 @@
{% if credential %} {% if credential %}
<section class="panel"> <section class="panel">
<h2>Delete SSH Key</h2> <h2>Delete SSH Key</h2>
{% if credential.hosts.exists or credential.global_configs.exists %}
<p class="muted">
This SSH key is still selected by {{ credential.hosts.count }} host(s) or
{{ credential.global_configs.count }} global config(s). Select another key there before deleting it.
</p>
{% else %}
<p class="muted">Type <strong>{{ credential.name }}</strong> to confirm deletion.</p>
{% endif %}
<form method="post" action="{% url 'delete_ssh_credential' credential.id %}"> <form method="post" action="{% url 'delete_ssh_credential' credential.id %}">
{% csrf_token %} {% csrf_token %}
<button type="submit" class="danger">Delete SSH key</button> <div class="field">
<label for="confirm_name">Confirm key name</label>
<input id="confirm_name" name="confirm_name" type="text" {% if credential.hosts.exists or credential.global_configs.exists %}disabled{% endif %}>
</div>
<button type="submit" class="danger" {% if credential.hosts.exists or credential.global_configs.exists %}disabled{% endif %}>Delete SSH key</button>
</form> </form>
</section> </section>
{% endif %} {% endif %}

View File

@@ -23,6 +23,7 @@
<th>Known hosts</th> <th>Known hosts</th>
<th>Hosts</th> <th>Hosts</th>
<th>Updated</th> <th>Updated</th>
<th>Actions</th>
</tr> </tr>
</thead> </thead>
<tbody> <tbody>
@@ -35,9 +36,10 @@
<td>{{ credential.known_hosts|yesno:"yes,no" }}</td> <td>{{ credential.known_hosts|yesno:"yes,no" }}</td>
<td>{{ credential.hosts.count }}</td> <td>{{ credential.hosts.count }}</td>
<td>{{ credential.updated_at }}</td> <td>{{ credential.updated_at }}</td>
<td><a class="button-link secondary" href="{% url 'edit_ssh_credential' credential.id %}">Edit</a></td>
</tr> </tr>
{% empty %} {% empty %}
<tr><td colspan="7" class="muted">No SSH credentials configured yet.</td></tr> <tr><td colspan="8" class="muted">No SSH credentials configured yet.</td></tr>
{% endfor %} {% endfor %}
</tbody> </tbody>
</table> </table>

View File

@@ -5,11 +5,27 @@ from datetime import datetime, timezone
from django.contrib.admin.sites import AdminSite from django.contrib.admin.sites import AdminSite
from django.test import TestCase from django.test import TestCase
from pobsync_backend.admin import BackupRunAdmin, HostConfigAdmin, SnapshotRecordAdmin from pobsync_backend.admin import BackupRunAdmin, GlobalConfigAdmin, HostConfigAdmin, SnapshotRecordAdmin
from pobsync_backend.models import BackupRun, HostConfig, ScheduleConfig, SnapshotRecord from pobsync_backend.models import BackupRun, GlobalConfig, HostConfig, ScheduleConfig, SnapshotRecord
class AdminDisplayTests(TestCase): class AdminDisplayTests(TestCase):
def test_admin_hides_old_global_state_fields_and_labels_host_runtime_state(self) -> None:
site = AdminSite()
global_admin = GlobalConfigAdmin(GlobalConfig, site)
host_admin = HostConfigAdmin(HostConfig, site)
global_fieldsets = list(global_admin.fieldsets)
host_fieldsets = list(host_admin.fieldsets)
global_fields = [field for _name, options in global_fieldsets for field in options["fields"]]
fieldset_names = [name for name, _options in [*global_fieldsets, *host_fieldsets]]
self.assertNotIn("pobsync_home", global_fields)
self.assertNotIn("data", global_fields)
self.assertIn("Runtime state", fieldset_names)
self.assertNotIn("Compatibility data", fieldset_names)
self.assertNotIn("Legacy JSON", fieldset_names)
def test_host_admin_links_to_related_snapshots_and_runs(self) -> None: def test_host_admin_links_to_related_snapshots_and_runs(self) -> None:
site = AdminSite() site = AdminSite()
admin = HostConfigAdmin(HostConfig, site) admin = HostConfigAdmin(HostConfig, site)

View File

@@ -61,6 +61,9 @@ class BackupWorkerTests(TestCase):
def fake_run_scheduled(**kwargs): def fake_run_scheduled(**kwargs):
run.refresh_from_db() run.refresh_from_db()
self.assertIn("execution", run.result) self.assertIn("execution", run.result)
self.assertIn("worker_pid", run.result["execution"])
self.assertIn("worker_host", run.result["execution"])
self.assertIn("heartbeat_at", run.result["execution"])
return { return {
"ok": True, "ok": True,
"dry_run": False, "dry_run": False,
@@ -82,6 +85,57 @@ class BackupWorkerTests(TestCase):
self.assertEqual(SnapshotRecord.objects.count(), 1) self.assertEqual(SnapshotRecord.objects.count(), 1)
self.assertEqual(run.snapshot, SnapshotRecord.objects.get()) self.assertEqual(run.snapshot, SnapshotRecord.objects.get())
def test_worker_refreshes_heartbeat_while_run_is_active(self) -> None:
with TemporaryDirectory() as tmp:
GlobalConfig.objects.create(name="default", backup_root=str(Path(tmp) / "backups"))
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
run = queue_backup_run(host=host)
with patch("pobsync_backend.backup_runner.run_scheduled") as run_scheduled:
def fake_run_scheduled(**kwargs):
run.refresh_from_db()
old_heartbeat = timezone.now() - timedelta(seconds=120)
run.result["execution"]["heartbeat_at"] = old_heartbeat.isoformat()
run.save(update_fields=["result"])
self.assertFalse(kwargs["cancel_check"]())
run.refresh_from_db()
self.assertGreater(
timezone.datetime.fromisoformat(run.result["execution"]["heartbeat_at"]),
old_heartbeat,
)
return {
"ok": True,
"dry_run": False,
"host": host.host,
"snapshot": "",
"base": None,
"rsync": {"exit_code": 0},
}
run_scheduled.side_effect = fake_run_scheduled
Command()._run_once(prefix=Path(tmp) / "home")
def test_worker_reconciles_stale_real_run_after_heartbeat_timeout(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
run = queue_backup_run(host=host)
run.status = BackupRun.Status.RUNNING
run.started_at = timezone.now() - timedelta(seconds=120)
run.result["execution"] = {
"worker_pid": 123,
"worker_host": "backup",
"heartbeat_at": (timezone.now() - timedelta(seconds=90)).isoformat(),
}
run.save(update_fields=["status", "started_at", "result"])
reconciled = reconcile_running_runs(stale_worker_seconds=30)
self.assertEqual(reconciled, 1)
run.refresh_from_db()
self.assertEqual(run.status, BackupRun.Status.FAILED)
self.assertEqual(run.result["failure"]["category"], "worker")
self.assertIn("heartbeat stopped", run.result["failure"]["message"])
def test_worker_records_dry_run_log_path_while_running(self) -> None: def test_worker_records_dry_run_log_path_while_running(self) -> None:
with TemporaryDirectory() as tmp: with TemporaryDirectory() as tmp:
GlobalConfig.objects.create(name="default", backup_root=str(Path(tmp) / "backups")) GlobalConfig.objects.create(name="default", backup_root=str(Path(tmp) / "backups"))

View File

@@ -1,71 +1,63 @@
from __future__ import annotations from __future__ import annotations
import tempfile
from pathlib import Path
from django.test import TestCase from django.test import TestCase
from pobsync.config.load import load_global_config, load_host_config from pobsync_backend.config_repository import ConfigRepositoryError, global_config_data, host_config_data
from pobsync_backend.config_repository import export_runtime_configs
from pobsync_backend.models import GlobalConfig, HostConfig from pobsync_backend.models import GlobalConfig, HostConfig
class ConfigRepositoryTests(TestCase): class ConfigRepositoryTests(TestCase):
def test_exports_database_configs_to_engine_yaml(self) -> None: def test_builds_runtime_config_from_database_fields(self) -> None:
with tempfile.TemporaryDirectory() as tmp: GlobalConfig.objects.create(
prefix = Path(tmp) name="default",
GlobalConfig.objects.create( backup_root="/backups",
name="default", ssh_user="backup",
backup_root="/backups", ssh_port=2222,
pobsync_home=str(prefix), rsync_args=["--archive"],
ssh_user="backup", excludes_default=["/proc/***"],
ssh_port=2222, retention_daily=7,
rsync_args=["--archive"], retention_weekly=4,
excludes_default=["/proc/***"], retention_monthly=3,
retention_daily=7, retention_yearly=1,
retention_weekly=4, )
retention_monthly=3, HostConfig.objects.create(
retention_yearly=1, host="web-01",
data={ address="web-01.example.test",
"backup_root": "/ignored", ssh_user="root",
"pobsync_home": "/ignored", includes=[],
"ssh": {"user": "ignored", "port": 22, "options": []}, excludes_add=["/tmp/***"],
"unknown": "must-not-leak", retention_daily=7,
"retention_defaults": {"daily": 99, "weekly": 99, "monthly": 99, "yearly": 99}, retention_weekly=4,
}, retention_monthly=3,
) retention_yearly=1,
HostConfig.objects.create( config={
host="web-01", "host": "ignored",
address="web-01.example.test", "address": "ignored",
ssh_user="root", "retention": {"daily": 99, "weekly": 99, "monthly": 99, "yearly": 99},
includes=[], "excludes_add": ["/ignored/***"],
excludes_add=["/tmp/***"], "unknown": "must-not-leak",
retention_daily=7, },
retention_weekly=4, )
retention_monthly=3,
retention_yearly=1,
config={
"host": "ignored",
"address": "ignored",
"retention": {"daily": 99, "weekly": 99, "monthly": 99, "yearly": 99},
"excludes_add": ["/ignored/***"],
"unknown": "must-not-leak",
},
)
written = export_runtime_configs(prefix=prefix, host="web-01") global_cfg = global_config_data()
host_cfg = host_config_data("web-01")
self.assertEqual(len(written), 2) self.assertEqual(global_cfg["backup_root"], "/backups")
global_cfg = load_global_config(prefix / "config" / "global.yaml") self.assertEqual(global_cfg["ssh"]["user"], "backup")
host_cfg = load_host_config(prefix / "config" / "hosts" / "web-01.yaml") self.assertEqual(global_cfg["ssh"]["port"], 2222)
self.assertEqual(global_cfg["backup_root"], "/backups") self.assertEqual(global_cfg["retention_defaults"]["daily"], 7)
self.assertEqual(global_cfg["pobsync_home"], str(prefix)) self.assertEqual(host_cfg["host"], "web-01")
self.assertEqual(global_cfg["ssh"]["user"], "backup") self.assertEqual(host_cfg["address"], "web-01.example.test")
self.assertEqual(global_cfg["ssh"]["port"], 2222) self.assertEqual(host_cfg["retention"]["daily"], 7)
self.assertEqual(global_cfg["retention_defaults"]["daily"], 7) self.assertEqual(host_cfg["excludes_add"], ["/tmp/***"])
self.assertEqual(host_cfg["host"], "web-01") self.assertNotIn("unknown", global_cfg)
self.assertEqual(host_cfg["address"], "web-01.example.test") self.assertNotIn("unknown", host_cfg)
self.assertEqual(host_cfg["retention"]["daily"], 7)
self.assertEqual(host_cfg["excludes_add"], ["/tmp/***"]) def test_missing_config_errors_use_operator_labels(self) -> None:
self.assertNotIn("unknown", global_cfg) with self.assertRaisesMessage(ConfigRepositoryError, "Missing global config 'default'"):
self.assertNotIn("unknown", host_cfg) global_config_data()
GlobalConfig.objects.create(name="default", backup_root="/backups")
with self.assertRaisesMessage(ConfigRepositoryError, "Missing enabled host 'web-01'"):
host_config_data("web-01")

View File

@@ -16,7 +16,6 @@ class ConfigureCommandsTests(TestCase):
call_command( call_command(
"configure_pobsync_global", "configure_pobsync_global",
backup_root="/backups", backup_root="/backups",
pobsync_home="/opt/pobsync",
retention="daily=3,weekly=2,monthly=1,yearly=0", retention="daily=3,weekly=2,monthly=1,yearly=0",
stdout=out, stdout=out,
) )
@@ -24,7 +23,7 @@ class ConfigureCommandsTests(TestCase):
config = GlobalConfig.objects.get(name="default") config = GlobalConfig.objects.get(name="default")
self.assertEqual(config.backup_root, "/backups") self.assertEqual(config.backup_root, "/backups")
self.assertEqual(config.retention_daily, 3) self.assertEqual(config.retention_daily, 3)
self.assertIn("Created GlobalConfig", out.getvalue()) self.assertIn("Created global config", out.getvalue())
def test_configure_host_uses_global_retention_defaults(self) -> None: def test_configure_host_uses_global_retention_defaults(self) -> None:
GlobalConfig.objects.create( GlobalConfig.objects.create(
@@ -62,7 +61,7 @@ class ConfigureCommandsTests(TestCase):
call_command( call_command(
"configure_pobsync_schedule", "configure_pobsync_schedule",
host.host, host.host,
cron="15 2 * * *", schedule_expression="15 2 * * *",
prune=True, prune=True,
stdout=out, stdout=out,
) )

View File

@@ -9,6 +9,14 @@ from pobsync.cli import main
class ConsoleEntrypointTests(SimpleTestCase): class ConsoleEntrypointTests(SimpleTestCase):
def test_version_prints_package_version(self) -> None:
stdout = StringIO()
with patch("sys.stdout", stdout):
exit_code = main(["--version"])
self.assertEqual(exit_code, 0)
self.assertEqual(stdout.getvalue().strip(), "pobsync 1.0.0")
def test_maps_backup_alias_to_django_command(self) -> None: def test_maps_backup_alias_to_django_command(self) -> None:
with patch("pobsync.cli.execute_from_command_line") as execute: with patch("pobsync.cli.execute_from_command_line") as execute:
exit_code = main(["backup", "web-01", "--dry-run"]) exit_code = main(["backup", "web-01", "--dry-run"])
@@ -31,15 +39,6 @@ class ConsoleEntrypointTests(SimpleTestCase):
self.assertEqual(exit_code, 0) self.assertEqual(exit_code, 0)
execute.assert_called_once_with(["pobsync", "check"]) execute.assert_called_once_with(["pobsync", "check"])
def test_maps_schedule_alias_to_django_command(self) -> None:
with patch("pobsync.cli.execute_from_command_line") as execute:
exit_code = main(["schedule", "web-01", "--cron", "15 2 * * *"])
self.assertEqual(exit_code, 0)
execute.assert_called_once_with(
["pobsync", "configure_pobsync_schedule", "web-01", "--cron", "15 2 * * *"]
)
def test_maps_discover_snapshots_alias_to_django_command(self) -> None: def test_maps_discover_snapshots_alias_to_django_command(self) -> None:
with patch("pobsync.cli.execute_from_command_line") as execute: with patch("pobsync.cli.execute_from_command_line") as execute:
exit_code = main(["discover-snapshots", "--host", "web-01"]) exit_code = main(["discover-snapshots", "--host", "web-01"])
@@ -53,3 +52,12 @@ class ConsoleEntrypointTests(SimpleTestCase):
self.assertEqual(exit_code, 0) self.assertEqual(exit_code, 0)
execute.assert_called_once_with(["pobsync", "run_pobsync_worker", "--once"]) execute.assert_called_once_with(["pobsync", "run_pobsync_worker", "--once"])
def test_configuration_aliases_are_not_public_commands(self) -> None:
stderr = StringIO()
with patch("sys.stderr", stderr):
exit_code = main(["schedule", "web-01", "--cron", "15 2 * * *"])
self.assertEqual(exit_code, 2)
self.assertIn("Unknown pobsync command", stderr.getvalue())
self.assertIn("pobsync django <management-command>", stderr.getvalue())

View File

@@ -15,7 +15,6 @@ class DjangoConfigSourceTests(TestCase):
GlobalConfig.objects.create( GlobalConfig.objects.create(
name="default", name="default",
backup_root="/backups", backup_root="/backups",
pobsync_home="/opt/pobsync",
rsync_args=["--archive"], rsync_args=["--archive"],
rsync_extra_args=["--numeric-ids"], rsync_extra_args=["--numeric-ids"],
excludes_default=["/proc/***"], excludes_default=["/proc/***"],
@@ -23,21 +22,6 @@ class DjangoConfigSourceTests(TestCase):
retention_weekly=4, retention_weekly=4,
retention_monthly=3, retention_monthly=3,
retention_yearly=1, retention_yearly=1,
data={
"backup_root": "/ignored",
"pobsync_home": "/ignored",
"ssh": {"user": "root", "port": 22, "options": []},
"rsync": {
"binary": "rsync",
"args": ["--archive"],
"timeout_seconds": 0,
"bwlimit_kbps": 0,
"extra_args": ["--numeric-ids"],
},
"defaults": {"source_root": "/", "destination_subdir": ""},
"excludes_default": ["/proc/***"],
"retention_defaults": {"daily": 7, "weekly": 4, "monthly": 3, "yearly": 1},
},
) )
HostConfig.objects.create( HostConfig.objects.create(
host="web-01", host="web-01",
@@ -72,7 +56,6 @@ class DjangoConfigSourceTests(TestCase):
GlobalConfig.objects.create( GlobalConfig.objects.create(
name="default", name="default",
backup_root="/backups", backup_root="/backups",
pobsync_home="/opt/pobsync",
default_ssh_credential=credential, default_ssh_credential=credential,
ssh_options=["-oBatchMode=yes"], ssh_options=["-oBatchMode=yes"],
) )
@@ -99,7 +82,6 @@ class DjangoConfigSourceTests(TestCase):
GlobalConfig.objects.create( GlobalConfig.objects.create(
name="default", name="default",
backup_root="/backups", backup_root="/backups",
pobsync_home="/opt/pobsync",
default_ssh_credential=global_credential, default_ssh_credential=global_credential,
) )
HostConfig.objects.create( HostConfig.objects.create(
@@ -127,7 +109,6 @@ class DjangoConfigSourceTests(TestCase):
GlobalConfig.objects.create( GlobalConfig.objects.create(
name="default", name="default",
backup_root="/backups", backup_root="/backups",
pobsync_home="/opt/pobsync",
default_ssh_credential=credential, default_ssh_credential=credential,
) )
HostConfig.objects.create(host="web-01", address="web-01.example.test") HostConfig.objects.create(host="web-01", address="web-01.example.test")
@@ -146,7 +127,6 @@ class DjangoConfigSourceTests(TestCase):
GlobalConfig.objects.create( GlobalConfig.objects.create(
name="default", name="default",
backup_root="/backups", backup_root="/backups",
pobsync_home="/opt/pobsync",
default_ssh_credential=credential, default_ssh_credential=credential,
) )
HostConfig.objects.create(host="web-01", address="web-01.example.test") HostConfig.objects.create(host="web-01", address="web-01.example.test")

View File

@@ -7,6 +7,7 @@ from tempfile import TemporaryDirectory
from django.test import SimpleTestCase from django.test import SimpleTestCase
from pobsync.commands.retention_plan import run_retention_plan from pobsync.commands.retention_plan import run_retention_plan
from pobsync.errors import ConfigError
from pobsync.util import write_yaml_atomic from pobsync.util import write_yaml_atomic
@@ -24,6 +25,15 @@ class FakeConfigSource:
class RetentionConfigSourceTests(SimpleTestCase): class RetentionConfigSourceTests(SimpleTestCase):
def test_retention_plan_requires_explicit_config_source(self) -> None:
with self.assertRaisesMessage(ConfigError, "A Django config source is required."):
run_retention_plan(
prefix=Path("/missing-prefix"),
host="web-01",
kind="scheduled",
protect_bases=False,
)
def test_retention_plan_uses_injected_config_source(self) -> None: def test_retention_plan_uses_injected_config_source(self) -> None:
with TemporaryDirectory() as tmp: with TemporaryDirectory() as tmp:
root = Path(tmp) / "backups" root = Path(tmp) / "backups"

View File

@@ -96,13 +96,14 @@ class RunBackupRecordsSnapshotTests(TestCase):
protect_bases=True, protect_bases=True,
yes=True, yes=True,
max_delete=3, max_delete=3,
action=BackupRun.RunType.SCHEDULED,
acquire_lock=False, acquire_lock=False,
) )
run = BackupRun.objects.get() run = BackupRun.objects.get()
self.assertEqual(run.status, BackupRun.Status.SUCCESS) self.assertEqual(run.status, BackupRun.Status.SUCCESS)
self.assertEqual(run.result["prune"], {"ok": True, "source": "sql", "deleted": []}) self.assertEqual(run.result["prune"], {"ok": True, "source": "sql", "deleted": []})
def test_prune_failure_is_recorded_on_backup_run(self) -> None: def test_prune_failure_marks_backup_run_as_warning(self) -> None:
with TemporaryDirectory() as tmp: with TemporaryDirectory() as tmp:
backup_root = Path(tmp) / "backups" backup_root = Path(tmp) / "backups"
GlobalConfig.objects.create(name="default", backup_root=str(backup_root)) GlobalConfig.objects.create(name="default", backup_root=str(backup_root))
@@ -128,19 +129,20 @@ class RunBackupRecordsSnapshotTests(TestCase):
} }
retention_apply.side_effect = ConfigError("Deletion blocked by --max-delete=0") retention_apply.side_effect = ConfigError("Deletion blocked by --max-delete=0")
with self.assertRaises(ConfigError): output = StringIO()
call_command( call_command(
"run_pobsync_backup", "run_pobsync_backup",
host.host, host.host,
prefix=str(Path(tmp) / "home"), prefix=str(Path(tmp) / "home"),
prune=True, prune=True,
prune_max_delete=0, prune_max_delete=0,
stdout=StringIO(), stdout=output,
) )
run = BackupRun.objects.get() run = BackupRun.objects.get()
self.assertEqual(run.status, BackupRun.Status.FAILED) self.assertEqual(run.status, BackupRun.Status.WARNING)
self.assertIsNotNone(run.snapshot) self.assertIsNotNone(run.snapshot)
self.assertIn("completed with warnings", output.getvalue())
self.assertEqual(run.result["prune"]["ok"], False) self.assertEqual(run.result["prune"]["ok"], False)
self.assertEqual(run.result["prune"]["type"], "ConfigError") self.assertEqual(run.result["prune"]["type"], "ConfigError")
self.assertEqual(run.result["prune"]["error"], "Deletion blocked by --max-delete=0") self.assertEqual(run.result["prune"]["error"], "Deletion blocked by --max-delete=0")

View File

@@ -7,6 +7,7 @@ from unittest.mock import patch
from django.test import SimpleTestCase from django.test import SimpleTestCase
from pobsync.commands.run_scheduled import run_scheduled from pobsync.commands.run_scheduled import run_scheduled
from pobsync.errors import ConfigError
from pobsync.rsync import RsyncResult from pobsync.rsync import RsyncResult
@@ -34,6 +35,10 @@ class FakeConfigSource:
class RunScheduledConfigSourceTests(SimpleTestCase): class RunScheduledConfigSourceTests(SimpleTestCase):
def test_requires_explicit_config_source(self) -> None:
with self.assertRaisesMessage(ConfigError, "A Django config source is required."):
run_scheduled(prefix=Path("/missing-prefix"), host="web-01", dry_run=True)
def test_dry_run_uses_injected_config_source(self) -> None: def test_dry_run_uses_injected_config_source(self) -> None:
with patch("pobsync.commands.run_scheduled.run_rsync") as run_rsync: with patch("pobsync.commands.run_scheduled.run_rsync") as run_rsync:
run_rsync.return_value = RsyncResult(exit_code=0, command=["rsync", "--archive"]) run_rsync.return_value = RsyncResult(exit_code=0, command=["rsync", "--archive"])
@@ -242,6 +247,7 @@ class RunScheduledConfigSourceTests(SimpleTestCase):
meta_text = meta_path.read_text(encoding="utf-8") meta_text = meta_path.read_text(encoding="utf-8")
self.assertTrue(result["ok"]) self.assertTrue(result["ok"])
self.assertEqual(result["log"], str(Path(result["snapshot"]) / "meta" / "rsync.log"))
self.assertEqual(result["stats"]["rsync"]["files_total"], 10) self.assertEqual(result["stats"]["rsync"]["files_total"], 10)
self.assertEqual(result["stats"]["rsync"]["files_transferred"], 2) self.assertEqual(result["stats"]["rsync"]["files_transferred"], 2)
self.assertEqual(result["stats"]["rsync"]["link_dest_estimated_savings_bytes"], 1500) self.assertEqual(result["stats"]["rsync"]["link_dest_estimated_savings_bytes"], 1500)

View File

@@ -8,7 +8,7 @@ from zoneinfo import ZoneInfo
from django.test import SimpleTestCase, TestCase from django.test import SimpleTestCase, TestCase
from pobsync_backend.management.commands.run_pobsync_scheduler import Command from pobsync_backend.management.commands.run_pobsync_scheduler import Command
from pobsync_backend.models import HostConfig, ScheduleConfig from pobsync_backend.models import BackupRun, HostConfig, ScheduleConfig
from pobsync_backend.scheduler import due_key, is_due, next_due_after from pobsync_backend.scheduler import due_key, is_due, next_due_after
@@ -64,3 +64,30 @@ class SchedulerCommandTests(TestCase):
self.assertEqual(call.call_count, 1) self.assertEqual(call.call_count, 1)
schedule = ScheduleConfig.objects.get(host=host) schedule = ScheduleConfig.objects.get(host=host)
self.assertEqual(schedule.last_status, "success") self.assertEqual(schedule.last_status, "success")
def test_run_due_records_warning_status_from_scheduled_backup_run(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
ScheduleConfig.objects.create(host=host, cron_expr="* * * * *", prune=True, prune_max_delete=1)
def create_warning_run(*args, **kwargs) -> None:
BackupRun.objects.create(
host=host,
run_type=BackupRun.RunType.SCHEDULED,
status=BackupRun.Status.WARNING,
result={
"ok": True,
"prune": {
"ok": False,
"type": "ConfigError",
"error": "Refusing to delete 2 snapshots (exceeds --max-delete=1)",
},
},
)
command = Command()
with patch("pobsync_backend.management.commands.run_pobsync_scheduler.call_command", side_effect=create_warning_run):
count = command._run_due(prefix=Path("/opt/pobsync"), dry_run=False)
self.assertEqual(count, 1)
schedule = ScheduleConfig.objects.get(host=host)
self.assertEqual(schedule.last_status, "warning")

View File

@@ -1,11 +1,16 @@
from __future__ import annotations from __future__ import annotations
import subprocess import subprocess
from io import StringIO
from pathlib import Path
from tempfile import TemporaryDirectory
from unittest.mock import patch from unittest.mock import patch
from django.test import SimpleTestCase from django.core.management import call_command
from django.core.management.base import CommandError
from django.test import SimpleTestCase, override_settings
from pobsync_backend.self_check import _systemd_checks from pobsync_backend.self_check import SelfCheck, _install_checks, _sqlite_database_check, _systemd_checks
class SystemdSelfCheckTests(SimpleTestCase): class SystemdSelfCheckTests(SimpleTestCase):
@@ -40,3 +45,92 @@ class SystemdSelfCheckTests(SimpleTestCase):
journal_check = next(check for check in checks if check.name == "Journal access") journal_check = next(check for check in checks if check.name == "Journal access")
self.assertEqual(journal_check.status, "failed") self.assertEqual(journal_check.status, "failed")
self.assertEqual(journal_check.message, "pobsync cannot read service logs.") self.assertEqual(journal_check.message, "pobsync cannot read service logs.")
class InstallSelfCheckTests(SimpleTestCase):
def test_install_checks_skip_native_paths_in_development_runtime(self) -> None:
with override_settings(POBSYNC_ENV_FILE="/missing/pobsync.env"), patch(
"pobsync_backend.self_check._native_runtime_available",
return_value=False,
):
checks = _install_checks()
self.assertEqual([check.status for check in checks], ["skipped", "skipped", "skipped"])
self.assertEqual(checks[0].name, "Environment file")
self.assertEqual(checks[1].name, "Service user")
self.assertEqual(checks[2].name, "Backup root owner")
def test_service_user_warns_when_current_user_differs(self) -> None:
with override_settings(
POBSYNC_ENV_FILE="/etc/pobsync/pobsync.env",
POBSYNC_SERVICE_USER="pobsync",
POBSYNC_BACKUP_ROOT="/backups",
), patch("pobsync_backend.self_check._native_runtime_available", return_value=True), patch(
"pobsync_backend.self_check._env_file_check",
return_value=SelfCheck("Environment file", "ok", "/etc/pobsync/pobsync.env"),
), patch(
"pobsync_backend.self_check._backup_root_owner_check",
return_value=SelfCheck("Backup root owner", "ok", "/backups owner=pobsync"),
), patch(
"pobsync_backend.self_check.os.geteuid",
return_value=0,
), patch(
"pobsync_backend.self_check.pwd.getpwuid",
) as getpwuid:
getpwuid.return_value.pw_name = "root"
checks = _install_checks()
service_user_check = next(check for check in checks if check.name == "Service user")
self.assertEqual(service_user_check.status, "warning")
self.assertIn("expected pobsync", service_user_check.message)
def test_sqlite_database_check_reports_existing_database(self) -> None:
with TemporaryDirectory() as tmp:
db_path = Path(tmp) / "pobsync.sqlite3"
db_path.write_text("", encoding="utf-8")
check = _sqlite_database_check(db_path)
self.assertEqual(check.status, "ok")
self.assertEqual(check.name, "SQLite database")
class CheckPobsyncInstallCommandTests(SimpleTestCase):
def test_command_prints_summary_for_successful_checks(self) -> None:
stdout = StringIO()
stderr = StringIO()
checks = [
SelfCheck("Database connection", "ok", "django.db.backends.sqlite3"),
SelfCheck("Systemd services", "skipped", "systemd is not available in this runtime."),
]
with patch("pobsync_backend.management.commands.check_pobsync_install.collect_self_checks", return_value=checks):
call_command("check_pobsync_install", stdout=stdout, stderr=stderr)
self.assertIn("[OK] Database connection", stdout.getvalue())
self.assertIn("[SKIPPED] Systemd services", stdout.getvalue())
self.assertIn("Summary: 1 ok, 0 warning(s), 0 failed, 1 skipped", stdout.getvalue())
self.assertEqual(stderr.getvalue(), "")
def test_command_fails_when_checks_fail(self) -> None:
stdout = StringIO()
stderr = StringIO()
checks = [
SelfCheck("POBSYNC_BACKUP_ROOT", "failed", "/backups does not exist."),
]
with patch("pobsync_backend.management.commands.check_pobsync_install.collect_self_checks", return_value=checks):
with self.assertRaisesMessage(CommandError, "pobsync install self check failed."):
call_command("check_pobsync_install", stdout=stdout, stderr=stderr)
self.assertIn("[FAILED] POBSYNC_BACKUP_ROOT", stderr.getvalue())
self.assertIn("Summary: 0 ok, 0 warning(s), 1 failed, 0 skipped", stdout.getvalue())
def test_command_can_fail_on_warnings(self) -> None:
checks = [
SelfCheck("Global config", "warning", "Default global config has not been created yet."),
]
with patch("pobsync_backend.management.commands.check_pobsync_install.collect_self_checks", return_value=checks):
with self.assertRaisesMessage(CommandError, "pobsync install self check reported warnings."):
call_command("check_pobsync_install", "--fail-on-warning", stdout=StringIO(), stderr=StringIO())

View File

@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import json import json
import stat
from datetime import datetime, timezone from datetime import datetime, timezone
from io import StringIO from io import StringIO
from pathlib import Path from pathlib import Path
@@ -10,8 +11,8 @@ from django.core.management import call_command
from django.test import TestCase from django.test import TestCase
from pobsync.errors import ConfigError from pobsync.errors import ConfigError
from pobsync_backend.models import HostConfig, SnapshotRecord from pobsync_backend.models import HostConfig, PurgedSnapshot, SnapshotRecord
from pobsync_backend.retention import run_sql_retention_apply, run_sql_retention_plan from pobsync_backend.retention import run_incomplete_cleanup, run_sql_retention_apply, run_sql_retention_plan
class SqlRetentionTests(TestCase): class SqlRetentionTests(TestCase):
@@ -31,7 +32,10 @@ class SqlRetentionTests(TestCase):
self.assertEqual(plan["source"], "sql") self.assertEqual(plan["source"], "sql")
self.assertEqual(plan["keep"], [new.dirname]) self.assertEqual(plan["keep"], [new.dirname])
self.assertEqual([item["dirname"] for item in plan["keep_items"]], [new.dirname])
self.assertEqual([item["dirname"] for item in plan["delete"]], [old.dirname]) self.assertEqual([item["dirname"] for item in plan["delete"]], [old.dirname])
self.assertEqual(plan["delete"][0]["reason"], "outside retention policy")
self.assertEqual(plan["incomplete"], [])
def test_plan_can_protect_base_snapshot_from_sql_relation(self) -> None: def test_plan_can_protect_base_snapshot_from_sql_relation(self) -> None:
host = HostConfig.objects.create( host = HostConfig.objects.create(
@@ -83,7 +87,72 @@ class SqlRetentionTests(TestCase):
self.assertTrue(new_dir.exists()) self.assertTrue(new_dir.exists())
self.assertTrue(SnapshotRecord.objects.filter(pk=new.pk).exists()) self.assertTrue(SnapshotRecord.objects.filter(pk=new.pk).exists())
self.assertFalse(SnapshotRecord.objects.filter(pk=old.pk).exists()) self.assertFalse(SnapshotRecord.objects.filter(pk=old.pk).exists())
self.assertEqual(result["deleted"], [{"dirname": old.dirname, "kind": "scheduled", "path": str(old_dir)}]) self.assertEqual(
result["deleted"],
[
{
"dirname": old.dirname,
"kind": "scheduled",
"path": str(old_dir),
"reason": "outside retention policy",
}
],
)
self.assertEqual(result["planned_delete_count"], 1)
self.assertEqual(result["max_delete"], 1)
self.assertEqual(result["incomplete_ignored_count"], 0)
purged = PurgedSnapshot.objects.get(dirname=old.dirname)
self.assertEqual(purged.host_name, host.host)
self.assertEqual(purged.kind, "scheduled")
self.assertEqual(purged.path, str(old_dir))
self.assertEqual(purged.reason, "outside retention policy")
self.assertEqual(purged.action, PurgedSnapshot.Action.MANUAL)
def test_apply_deletes_snapshot_with_readonly_data_directory(self) -> None:
with TemporaryDirectory() as tmp:
prefix = Path(tmp) / "home"
host = HostConfig.objects.create(
host="web-01",
address="web-01.example.test",
retention_daily=0,
retention_weekly=0,
retention_monthly=0,
retention_yearly=0,
)
old_dir = Path(tmp) / "backups" / host.host / "scheduled" / "20260518-021500Z__OLD"
old_data = old_dir / "data"
old_data.mkdir(parents=True)
old_data.joinpath("etc").mkdir()
old_data.joinpath("etc", "config").write_text("preserved permissions\n")
old_data.chmod(stat.S_IREAD | stat.S_IEXEC | stat.S_IRGRP | stat.S_IXGRP | stat.S_IROTH | stat.S_IXOTH)
new_dir = Path(tmp) / "backups" / host.host / "scheduled" / "20260519-021500Z__NEW"
new_dir.mkdir(parents=True)
old = self._snapshot(host, old_dir.name, path=str(old_data))
self._snapshot(host, new_dir.name, path=str(new_dir))
result = run_sql_retention_apply(
prefix=prefix,
host=host.host,
kind="scheduled",
protect_bases=False,
yes=True,
max_delete=1,
acquire_lock=False,
)
self.assertFalse(old_dir.exists())
self.assertFalse(SnapshotRecord.objects.filter(pk=old.pk).exists())
self.assertEqual(
result["deleted"],
[
{
"dirname": old.dirname,
"kind": "scheduled",
"path": str(old_dir),
"reason": "outside retention policy",
}
],
)
def test_apply_respects_max_delete(self) -> None: def test_apply_respects_max_delete(self) -> None:
host = HostConfig.objects.create( host = HostConfig.objects.create(
@@ -109,6 +178,81 @@ class SqlRetentionTests(TestCase):
acquire_lock=False, acquire_lock=False,
) )
def test_incomplete_cleanup_deletes_directory_and_record(self) -> None:
with TemporaryDirectory() as tmp:
prefix = Path(tmp) / "home"
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
incomplete_dir = Path(tmp) / "backups" / host.host / ".incomplete" / "20260519-031500Z__BROKEN01"
incomplete_dir.mkdir(parents=True)
incomplete_dir.joinpath("partial-file").write_text("interrupted\n")
record = SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname=incomplete_dir.name,
path=str(incomplete_dir),
status="failed",
started_at=datetime(2026, 5, 19, 3, 15, tzinfo=timezone.utc),
)
result = run_incomplete_cleanup(
prefix=prefix,
host=host.host,
yes=True,
max_delete=1,
acquire_lock=False,
)
self.assertFalse(incomplete_dir.exists())
self.assertFalse(SnapshotRecord.objects.filter(pk=record.pk).exists())
self.assertEqual(
result["deleted"],
[{"dirname": incomplete_dir.name, "kind": SnapshotRecord.Kind.INCOMPLETE, "path": str(incomplete_dir)}],
)
self.assertEqual(result["planned_delete_count"], 1)
purged = PurgedSnapshot.objects.get(dirname=incomplete_dir.name)
self.assertEqual(purged.action, PurgedSnapshot.Action.INCOMPLETE_CLEANUP)
self.assertEqual(purged.reason, "manual incomplete cleanup")
def test_incomplete_cleanup_respects_max_delete(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname="20260519-031500Z__BROKEN01",
path=f"/backups/{host.host}/.incomplete/20260519-031500Z__BROKEN01",
status="failed",
started_at=datetime(2026, 5, 19, 3, 15, tzinfo=timezone.utc),
)
with self.assertRaisesRegex(ConfigError, "blocked by --max-delete=0"):
run_incomplete_cleanup(
prefix=Path("/tmp/pobsync-test"),
host=host.host,
yes=True,
max_delete=0,
acquire_lock=False,
)
def test_incomplete_cleanup_rejects_unexpected_path(self) -> None:
host = HostConfig.objects.create(host="web-01", address="web-01.example.test")
SnapshotRecord.objects.create(
host=host,
kind=SnapshotRecord.Kind.INCOMPLETE,
dirname="20260519-031500Z__BROKEN01",
path=f"/backups/{host.host}/scheduled/20260519-031500Z__BROKEN01",
status="failed",
started_at=datetime(2026, 5, 19, 3, 15, tzinfo=timezone.utc),
)
with self.assertRaisesRegex(ConfigError, "unexpected incomplete snapshot path"):
run_incomplete_cleanup(
prefix=Path("/tmp/pobsync-test"),
host=host.host,
yes=True,
max_delete=1,
acquire_lock=False,
)
def test_management_command_plans_from_sql(self) -> None: def test_management_command_plans_from_sql(self) -> None:
host = HostConfig.objects.create( host = HostConfig.objects.create(
host="web-01", host="web-01",

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,7 @@
from __future__ import annotations from __future__ import annotations
import json import json
import shlex
import shutil import shutil
import subprocess import subprocess
from pathlib import Path from pathlib import Path
@@ -8,11 +9,13 @@ from pathlib import Path
from django.contrib import messages from django.contrib import messages
from django.contrib.admin.views.decorators import staff_member_required from django.contrib.admin.views.decorators import staff_member_required
from django.conf import settings from django.conf import settings
from django.db.models import Count from django.http import FileResponse, Http404
from django.db.models import Count, Q
from django.shortcuts import get_object_or_404, redirect, render from django.shortcuts import get_object_or_404, redirect, render
from django.utils import timezone from django.utils import timezone
from django.views.decorators.http import require_POST from django.views.decorators.http import require_POST
from pobsync import __version__
from pobsync.errors import PobsyncError from pobsync.errors import PobsyncError
from pobsync.config.defaults import DEFAULT_EXCLUDES, DEFAULT_RSYNC_ARGS from pobsync.config.defaults import DEFAULT_EXCLUDES, DEFAULT_RSYNC_ARGS
@@ -22,15 +25,17 @@ from .forms import (
CreateHostConfigForm, CreateHostConfigForm,
GlobalConfigForm, GlobalConfigForm,
HostConfigForm, HostConfigForm,
IncompleteCleanupForm,
ManualBackupForm, ManualBackupForm,
RetentionApplyForm, RetentionApplyForm,
SshCredentialGenerateForm, SshCredentialGenerateForm,
ScheduleConfigForm, ScheduleConfigForm,
SshCredentialForm, SshCredentialForm,
) )
from .host_ops import collect_host_checks, ensure_host_directories from .host_ops import ensure_host_directories
from .models import BackupRun, GlobalConfig, HostConfig, ScheduleConfig, SnapshotRecord, SshCredential from .models import BackupRun, GlobalConfig, HostConfig, PurgedSnapshot, ScheduleConfig, SnapshotRecord, SshCredential
from .retention import run_sql_retention_apply, run_sql_retention_plan from .preflight import collect_backup_gate, effective_host_config_preview, run_remote_preflight
from .retention import run_incomplete_cleanup, run_sql_retention_apply, run_sql_retention_plan
from .self_check import collect_self_checks, summarize_self_checks from .self_check import collect_self_checks, summarize_self_checks
from .scheduler import next_due_after from .scheduler import next_due_after
from .snapshot_discovery import discover_snapshots, inspect_snapshot_discovery from .snapshot_discovery import discover_snapshots, inspect_snapshot_discovery
@@ -43,7 +48,22 @@ def dashboard(request):
global_config = GlobalConfig.objects.filter(name="default").first() global_config = GlobalConfig.objects.filter(name="default").first()
hosts = list( hosts = list(
HostConfig.objects.select_related("schedule") HostConfig.objects.select_related("schedule")
.annotate(snapshot_count=Count("snapshots", distinct=True), run_count=Count("runs", distinct=True)) .annotate(
snapshot_count=Count("snapshots", distinct=True),
run_count=Count("runs", distinct=True),
queued_run_count=Count("runs", filter=Q(runs__status=BackupRun.Status.QUEUED), distinct=True),
running_run_count=Count("runs", filter=Q(runs__status=BackupRun.Status.RUNNING), distinct=True),
warning_run_count=Count(
"runs",
filter=Q(runs__status=BackupRun.Status.WARNING, runs__reviewed_at__isnull=True),
distinct=True,
),
failed_run_count=Count(
"runs",
filter=Q(runs__status=BackupRun.Status.FAILED, runs__reviewed_at__isnull=True),
distinct=True,
),
)
.order_by("host") .order_by("host")
) )
for host_config in hosts: for host_config in hosts:
@@ -53,6 +73,7 @@ def dashboard(request):
.first() .first()
) )
host_config.next_run_at = _next_run_for_host(host_config) host_config.next_run_at = _next_run_for_host(host_config)
host_config.retention_warning = _retention_warning_for_host(host_config, _schedule_for_host(host_config))
stats_summary = collect_dashboard_stats(hosts=hosts, global_config=global_config) stats_summary = collect_dashboard_stats(hosts=hosts, global_config=global_config)
context = { context = {
"hosts": hosts, "hosts": hosts,
@@ -68,13 +89,43 @@ def dashboard(request):
"enabled_schedules": ScheduleConfig.objects.filter(enabled=True).count(), "enabled_schedules": ScheduleConfig.objects.filter(enabled=True).count(),
"snapshots": SnapshotRecord.objects.count(), "snapshots": SnapshotRecord.objects.count(),
"runs": BackupRun.objects.count(), "runs": BackupRun.objects.count(),
"queued_runs": BackupRun.objects.filter(status=BackupRun.Status.QUEUED).count(),
"running_runs": BackupRun.objects.filter(status=BackupRun.Status.RUNNING).count(), "running_runs": BackupRun.objects.filter(status=BackupRun.Status.RUNNING).count(),
"failed_runs": BackupRun.objects.filter(status=BackupRun.Status.FAILED).count(), "warning_runs": BackupRun.objects.filter(
status=BackupRun.Status.WARNING,
reviewed_at__isnull=True,
).count(),
"failed_runs": BackupRun.objects.filter(
status=BackupRun.Status.FAILED,
reviewed_at__isnull=True,
).count(),
}, },
} }
return render(request, "pobsync_backend/dashboard.html", context) return render(request, "pobsync_backend/dashboard.html", context)
@staff_member_required
def changelog(request):
changelog_path = Path(settings.BASE_DIR) / "CHANGELOG.md"
try:
changelog_text = changelog_path.read_text(encoding="utf-8")
missing = False
except FileNotFoundError:
changelog_text = "CHANGELOG.md was not found in this installation."
missing = True
return render(
request,
"pobsync_backend/changelog.html",
{
"app_version": __version__,
"changelog_blocks": _parse_changelog(changelog_text),
"changelog_path": changelog_path,
"missing": missing,
},
)
@staff_member_required @staff_member_required
def self_check(request): def self_check(request):
checks = collect_self_checks() checks = collect_self_checks()
@@ -94,6 +145,27 @@ def logs(request):
return render(request, "pobsync_backend/logs.html", context) return render(request, "pobsync_backend/logs.html", context)
@staff_member_required
def purged_snapshots(request):
host = request.GET.get("host", "").strip()
action = request.GET.get("action", "").strip()
purged = PurgedSnapshot.objects.select_related("host").order_by("-purged_at", "host_name", "dirname")
if host:
purged = purged.filter(host_name=host)
if action:
purged = purged.filter(action=action)
context = {
"purged_snapshots": purged[:200],
"hosts": HostConfig.objects.order_by("host"),
"actions": PurgedSnapshot.Action.choices,
"selected_host": host,
"selected_action": action,
"total_count": purged.count(),
}
return render(request, "pobsync_backend/purged_snapshots.html", context)
@staff_member_required @staff_member_required
def ssh_credentials(request): def ssh_credentials(request):
context = { context = {
@@ -188,6 +260,9 @@ def delete_ssh_credential(request, credential_id: int):
if credential.hosts.exists() or credential.global_configs.exists(): if credential.hosts.exists() or credential.global_configs.exists():
messages.error(request, f"SSH key {credential.name} is still in use and cannot be deleted.") messages.error(request, f"SSH key {credential.name} is still in use and cannot be deleted.")
return redirect("edit_ssh_credential", credential_id=credential.id) return redirect("edit_ssh_credential", credential_id=credential.id)
if request.POST.get("confirm_name", "").strip() != credential.name:
messages.error(request, f"Type {credential.name} to confirm SSH key deletion.")
return redirect("edit_ssh_credential", credential_id=credential.id)
name = credential.name name = credential.name
try: try:
@@ -259,26 +334,32 @@ def create_host_config(request):
@staff_member_required @staff_member_required
def host_detail(request, host: str): def host_detail(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
global_config = GlobalConfig.objects.filter(name="default").first()
schedule = _schedule_for_host(host_config) schedule = _schedule_for_host(host_config)
queued_runs = host_config.runs.filter(status=BackupRun.Status.QUEUED) queued_runs = host_config.runs.filter(status=BackupRun.Status.QUEUED)
running_runs = host_config.runs.filter(status=BackupRun.Status.RUNNING) running_runs = host_config.runs.filter(status=BackupRun.Status.RUNNING)
active_run = host_config.runs.filter( active_run = host_config.runs.filter(
status__in=[BackupRun.Status.QUEUED, BackupRun.Status.RUNNING] status__in=[BackupRun.Status.QUEUED, BackupRun.Status.RUNNING]
).order_by("created_at", "id").first() ).order_by("created_at", "id").first()
has_global_config = GlobalConfig.objects.filter(name="default").exists() has_global_config = global_config is not None
host_checks = collect_host_checks(host_config) backup_gate = collect_backup_gate(host_config, global_config)
stats_summary = collect_host_stats(host=host_config, limit=10) stats_summary = collect_host_stats(host=host_config, limit=10)
context = { context = {
"host": host_config, "host": host_config,
"schedule": schedule, "schedule": schedule,
"retention_warning": _retention_warning_for_host(host_config, schedule),
"next_run_at": _next_run_for_schedule(schedule, host_config), "next_run_at": _next_run_for_schedule(schedule, host_config),
"scheduler_timezone": timezone.get_current_timezone_name(), "scheduler_timezone": timezone.get_current_timezone_name(),
"discovery": inspect_snapshot_discovery(host=host_config), "discovery": inspect_snapshot_discovery(host=host_config),
"host_checks": host_checks, "host_checks": backup_gate.checks,
"host_check_summary": summarize_self_checks(host_checks), "host_check_summary": summarize_self_checks(backup_gate.checks),
"backup_gate": backup_gate,
"last_preflight": (host_config.config or {}).get("last_preflight") if isinstance(host_config.config, dict) else {},
"effective_config": effective_host_config_preview(host_config, global_config) if global_config else {},
"stats_summary": stats_summary, "stats_summary": stats_summary,
"manual_backup_form": ManualBackupForm(initial=_default_manual_backup_initial(host_config)), "manual_backup_form": ManualBackupForm(initial=_default_manual_backup_initial(host_config)),
"can_queue_backup": host_config.enabled and has_global_config, "can_queue_dry_run": host_config.enabled and has_global_config and backup_gate.can_queue_dry_run and active_run is None,
"can_queue_real_backup": host_config.enabled and has_global_config and backup_gate.can_queue_real and active_run is None,
"has_global_config": has_global_config, "has_global_config": has_global_config,
"active_run": active_run, "active_run": active_run,
"latest_runs": host_config.runs.select_related("snapshot").order_by("-created_at")[:10], "latest_runs": host_config.runs.select_related("snapshot").order_by("-created_at")[:10],
@@ -288,7 +369,10 @@ def host_detail(request, host: str):
"runs": host_config.runs.count(), "runs": host_config.runs.count(),
"queued_runs": queued_runs.count(), "queued_runs": queued_runs.count(),
"running_runs": running_runs.count(), "running_runs": running_runs.count(),
"failed_runs": host_config.runs.filter(status=BackupRun.Status.FAILED).count(), "failed_runs": host_config.runs.filter(
status=BackupRun.Status.FAILED,
reviewed_at__isnull=True,
).count(),
"incomplete_snapshots": host_config.snapshots.filter(kind=SnapshotRecord.Kind.INCOMPLETE).count(), "incomplete_snapshots": host_config.snapshots.filter(kind=SnapshotRecord.Kind.INCOMPLETE).count(),
}, },
} }
@@ -330,6 +414,34 @@ def scan_host_known_key(request, host: str):
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@staff_member_required
@require_POST
def run_host_preflight(request, host: str):
host_config = get_object_or_404(HostConfig, host=host)
if not host_config.enabled:
messages.error(request, f"Cannot run preflight for disabled host {host_config.host}.")
return redirect("host_detail", host=host_config.host)
if not GlobalConfig.objects.filter(name="default").exists():
messages.error(request, "Create the default global config before running preflight.")
return redirect("host_detail", host=host_config.host)
try:
result = run_remote_preflight(host_config)
except Exception as exc:
messages.error(request, f"Connection preflight failed for {host_config.host}: {exc}")
else:
if result.get("ok"):
messages.success(request, f"Connection preflight passed for {host_config.host}.")
else:
failed = [
str(check.get("name"))
for check in result.get("checks", [])
if isinstance(check, dict) and not check.get("ok")
]
messages.error(request, f"Connection preflight failed for {host_config.host}: {', '.join(failed)}.")
return redirect("host_detail", host=host_config.host)
@staff_member_required @staff_member_required
@require_POST @require_POST
def queue_manual_backup(request, host: str): def queue_manual_backup(request, host: str):
@@ -337,7 +449,8 @@ def queue_manual_backup(request, host: str):
if not host_config.enabled: if not host_config.enabled:
messages.error(request, f"Cannot queue backup for disabled host {host_config.host}.") messages.error(request, f"Cannot queue backup for disabled host {host_config.host}.")
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
if not GlobalConfig.objects.filter(name="default").exists(): global_config = GlobalConfig.objects.filter(name="default").first()
if global_config is None:
messages.error(request, "Create the default global config before queueing backups.") messages.error(request, "Create the default global config before queueing backups.")
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
@@ -346,6 +459,17 @@ def queue_manual_backup(request, host: str):
messages.error(request, "Manual backup options are invalid.") messages.error(request, "Manual backup options are invalid.")
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
backup_gate = collect_backup_gate(host_config, global_config)
if form.cleaned_data["dry_run"]:
if not backup_gate.can_queue_dry_run:
blockers = ", ".join(check.name for check in backup_gate.dry_run_blockers)
messages.error(request, f"Cannot queue dry-run until failed preflight checks are resolved: {blockers}.")
return redirect("host_detail", host=host_config.host)
elif not backup_gate.can_queue_real:
blockers = ", ".join(check.name for check in backup_gate.real_blockers)
messages.error(request, f"Cannot queue real backup until failed preflight checks are resolved: {blockers}.")
return redirect("host_detail", host=host_config.host)
run = queue_backup_run( run = queue_backup_run(
host=host_config, host=host_config,
dry_run=form.cleaned_data["dry_run"], dry_run=form.cleaned_data["dry_run"],
@@ -361,17 +485,53 @@ def queue_manual_backup(request, host: str):
@staff_member_required @staff_member_required
def run_detail(request, run_id: int): def run_detail(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id) run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id)
run_stats = run.result.get("stats") if isinstance(run.result, dict) else {} result = run.result if isinstance(run.result, dict) else {}
run_stats = result.get("stats") if isinstance(result.get("stats"), dict) else {}
rsync_result = result.get("rsync") if isinstance(result.get("rsync"), dict) else {}
failure = result.get("failure") if isinstance(result.get("failure"), dict) else {}
prune_result = result.get("prune") if isinstance(result.get("prune"), dict) else {}
execution = result.get("execution") if isinstance(result.get("execution"), dict) else {}
rsync_log_path = _run_rsync_log_path(run)
rsync_log_tail = _run_rsync_log_tail(rsync_result, rsync_log_path)
requested = result.get("requested") if isinstance(result.get("requested"), dict) else {}
context = { context = {
"run": run, "run": run,
"can_cancel": run.status in {BackupRun.Status.QUEUED, BackupRun.Status.RUNNING}, "can_cancel": run.status in {BackupRun.Status.QUEUED, BackupRun.Status.RUNNING},
"requested": run.result.get("requested") if isinstance(run.result, dict) else {}, "requested": requested,
"execution": execution,
"stats": run_stats if isinstance(run_stats, dict) else {}, "stats": run_stats if isinstance(run_stats, dict) else {},
"rsync": rsync_result,
"rsync_command": _run_rsync_command(rsync_result),
"failure": failure,
"failure_summary": failure.get("message") or failure.get("summary") or "",
"prune_result": prune_result,
"has_prune_result": bool(prune_result),
"retention_warning": _retention_warning_for_host(run.host, _schedule_for_host(run.host)),
"rsync_log_path": str(rsync_log_path) if rsync_log_path is not None else "",
"rsync_log_exists": bool(rsync_log_path and rsync_log_path.exists()),
"rsync_log_tail": rsync_log_tail,
"dry_run_summary": _dry_run_summary(
result=result,
requested=requested,
stats=run_stats if isinstance(run_stats, dict) else {},
failure=failure,
rsync_log_tail=rsync_log_tail,
rsync_log_exists=bool(rsync_log_path and rsync_log_path.exists()),
),
"result_json": _pretty_json(run.result), "result_json": _pretty_json(run.result),
} }
return render(request, "pobsync_backend/run_detail.html", context) return render(request, "pobsync_backend/run_detail.html", context)
@staff_member_required
def run_rsync_log(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host", "snapshot"), id=run_id)
log_path = _run_rsync_log_path(run)
if log_path is None or not log_path.is_file():
raise Http404("Rsync log not found")
return FileResponse(log_path.open("rb"), content_type="text/plain; charset=utf-8")
@staff_member_required @staff_member_required
@require_POST @require_POST
def cancel_run(request, run_id: int): def cancel_run(request, run_id: int):
@@ -396,18 +556,54 @@ def cancel_run(request, run_id: int):
return redirect("run_detail", run_id=run.id) return redirect("run_detail", run_id=run.id)
@staff_member_required
@require_POST
def resolve_run_review(request, run_id: int):
run = get_object_or_404(BackupRun.objects.select_related("host"), id=run_id)
if run.status not in {BackupRun.Status.FAILED, BackupRun.Status.WARNING}:
messages.warning(request, f"Run {run.id} does not need review.")
return redirect("run_detail", run_id=run.id)
if run.reviewed_at:
messages.info(request, f"Run {run.id} was already marked reviewed.")
return redirect("run_detail", run_id=run.id)
run.reviewed_at = timezone.now()
run.reviewed_by = request.user.get_username()
run.save(update_fields=["reviewed_at", "reviewed_by"])
messages.success(request, f"Run {run.id} marked reviewed.")
return redirect("run_detail", run_id=run.id)
@staff_member_required
@require_POST
def resolve_host_incomplete_reviews(request, host: str):
host_config = get_object_or_404(HostConfig, host=host)
reviewed_count = host_config.snapshots.filter(
kind=SnapshotRecord.Kind.INCOMPLETE,
reviewed_at__isnull=True,
).update(reviewed_at=timezone.now(), reviewed_by=request.user.get_username())
if reviewed_count:
messages.success(request, f"Marked {reviewed_count} incomplete snapshot(s) reviewed for {host_config.host}.")
else:
messages.info(request, f"No incomplete snapshots needed review for {host_config.host}.")
return redirect("host_detail", host=host_config.host)
@staff_member_required @staff_member_required
def snapshot_detail(request, snapshot_id: int): def snapshot_detail(request, snapshot_id: int):
snapshot = get_object_or_404( snapshot = get_object_or_404(
SnapshotRecord.objects.select_related("host", "base").prefetch_related("derived_snapshots", "backup_runs"), SnapshotRecord.objects.select_related("host", "base").prefetch_related("derived_snapshots", "backup_runs"),
id=snapshot_id, id=snapshot_id,
) )
restore = _snapshot_restore_guidance(snapshot)
context = { context = {
"snapshot": snapshot, "snapshot": snapshot,
"stats": snapshot.metadata.get("stats") if isinstance(snapshot.metadata, dict) else {}, "stats": snapshot.metadata.get("stats") if isinstance(snapshot.metadata, dict) else {},
"metadata_json": _pretty_json(snapshot.metadata), "metadata_json": _pretty_json(snapshot.metadata),
"backup_runs": snapshot.backup_runs.select_related("host").order_by("-created_at"), "backup_runs": snapshot.backup_runs.select_related("host").order_by("-created_at"),
"derived_snapshots": snapshot.derived_snapshots.select_related("host").order_by("-started_at", "dirname"), "derived_snapshots": snapshot.derived_snapshots.select_related("host").order_by("-started_at", "dirname"),
"restore": restore,
} }
return render(request, "pobsync_backend/snapshot_detail.html", context) return render(request, "pobsync_backend/snapshot_detail.html", context)
@@ -446,17 +642,34 @@ def host_retention_plan(request, host: str):
except PobsyncError as exc: except PobsyncError as exc:
messages.error(request, str(exc)) messages.error(request, str(exc))
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
schedule = _schedule_for_host(host_config)
scheduled_prune_limit = schedule.prune_max_delete if schedule and schedule.prune else None
delete_count = len(plan["delete"])
incomplete_count = len(plan["incomplete"])
context = { context = {
"host": host_config, "host": host_config,
"kind": kind, "kind": kind,
"protect_bases": protect_bases, "protect_bases": protect_bases,
"plan": plan, "plan": plan,
"schedule": schedule,
"scheduled_prune_limit": scheduled_prune_limit,
"scheduled_prune_exceeded": scheduled_prune_limit is not None and delete_count > scheduled_prune_limit,
"apply_form": RetentionApplyForm( "apply_form": RetentionApplyForm(
host_name=host_config.host, host_name=host_config.host,
expected_delete_count=delete_count,
initial={ initial={
"kind": kind, "kind": kind,
"protect_bases": protect_bases, "protect_bases": protect_bases,
"max_delete": len(plan["delete"]), "max_delete": delete_count,
"confirm_delete_count": delete_count,
},
),
"incomplete_cleanup_form": IncompleteCleanupForm(
host_name=host_config.host,
expected_delete_count=incomplete_count,
initial={
"max_delete": incomplete_count,
"confirm_delete_count": incomplete_count,
}, },
), ),
} }
@@ -467,7 +680,26 @@ def host_retention_plan(request, host: str):
@require_POST @require_POST
def apply_host_retention(request, host: str): def apply_host_retention(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
form = RetentionApplyForm(request.POST, host_name=host_config.host) raw_kind = request.POST.get("kind", "scheduled")
raw_protect_bases = request.POST.get("protect_bases") in {"1", "true", "on", "yes"}
expected_delete_count = None
if raw_kind in {"scheduled", "manual", "all"}:
try:
plan = run_sql_retention_plan(
host=host_config.host,
kind=raw_kind,
protect_bases=raw_protect_bases,
)
except PobsyncError as exc:
messages.error(request, str(exc))
return redirect("host_retention_plan", host=host_config.host)
expected_delete_count = len(plan.get("delete") or [])
form = RetentionApplyForm(
request.POST,
host_name=host_config.host,
expected_delete_count=expected_delete_count,
)
if not form.is_valid(): if not form.is_valid():
messages.error(request, "Retention apply confirmation is invalid.") messages.error(request, "Retention apply confirmation is invalid.")
return redirect("host_retention_plan", host=host_config.host) return redirect("host_retention_plan", host=host_config.host)
@@ -482,6 +714,8 @@ def apply_host_retention(request, host: str):
protect_bases=protect_bases, protect_bases=protect_bases,
yes=True, yes=True,
max_delete=form.cleaned_data["max_delete"], max_delete=form.cleaned_data["max_delete"],
action=PurgedSnapshot.Action.MANUAL,
triggered_by=request.user.get_username(),
) )
except PobsyncError as exc: except PobsyncError as exc:
messages.error(request, str(exc)) messages.error(request, str(exc))
@@ -496,6 +730,41 @@ def apply_host_retention(request, host: str):
return target return target
@staff_member_required
@require_POST
def cleanup_host_incomplete_snapshots(request, host: str):
host_config = get_object_or_404(HostConfig, host=host)
try:
plan = run_sql_retention_plan(host=host_config.host, kind="all", protect_bases=True)
except PobsyncError as exc:
messages.error(request, str(exc))
return redirect("host_retention_plan", host=host_config.host)
incomplete_count = len(plan.get("incomplete") or [])
form = IncompleteCleanupForm(
request.POST,
host_name=host_config.host,
expected_delete_count=incomplete_count,
)
if not form.is_valid():
messages.error(request, "Incomplete cleanup confirmation is invalid.")
return redirect("host_retention_plan", host=host_config.host)
try:
result = run_incomplete_cleanup(
prefix=Path(settings.POBSYNC_HOME),
host=host_config.host,
yes=True,
max_delete=form.cleaned_data["max_delete"],
triggered_by=request.user.get_username(),
)
except PobsyncError as exc:
messages.error(request, str(exc))
else:
messages.success(request, f"Deleted {len(result['deleted'])} incomplete snapshot(s) for {host_config.host}.")
return redirect("host_retention_plan", host=host_config.host)
@staff_member_required @staff_member_required
def edit_host_config(request, host: str): def edit_host_config(request, host: str):
host_config = get_object_or_404(HostConfig, host=host) host_config = get_object_or_404(HostConfig, host=host)
@@ -535,7 +804,11 @@ def edit_host_schedule(request, host: str):
messages.success(request, f"Schedule saved for {host_config.host}.") messages.success(request, f"Schedule saved for {host_config.host}.")
return redirect("host_detail", host=host_config.host) return redirect("host_detail", host=host_config.host)
else: else:
form = ScheduleConfigForm(instance=schedule, initial=_default_schedule_initial()) form = (
ScheduleConfigForm(instance=schedule)
if schedule
else ScheduleConfigForm(initial=_default_schedule_initial())
)
return render( return render(
request, request,
@@ -568,10 +841,49 @@ def _next_run_for_schedule(schedule: ScheduleConfig | None, host_config: HostCon
return None return None
def _retention_warning_for_host(host_config: HostConfig, schedule: ScheduleConfig | None) -> dict[str, object]:
incomplete_count = host_config.snapshots.filter(
kind=SnapshotRecord.Kind.INCOMPLETE,
reviewed_at__isnull=True,
).count()
warning: dict[str, object] = {
"has_warning": incomplete_count > 0,
"incomplete_count": incomplete_count,
}
if schedule is None or not schedule.prune or not host_config.enabled:
return warning
try:
plan = run_sql_retention_plan(
host=host_config.host,
kind="scheduled",
protect_bases=bool(schedule.prune_protect_bases),
)
except PobsyncError as exc:
warning.update(
{
"has_warning": True,
"error": str(exc),
}
)
return warning
delete_count = len(plan.get("delete") or [])
warning.update(
{
"delete_count": delete_count,
"max_delete": schedule.prune_max_delete,
"protect_bases": bool(schedule.prune_protect_bases),
"prune_exceeded": delete_count > schedule.prune_max_delete,
}
)
if warning["prune_exceeded"]:
warning["has_warning"] = True
return warning
def _default_schedule_initial() -> dict[str, object]: def _default_schedule_initial() -> dict[str, object]:
return { return {
"cron_expr": "15 2 * * *", "cron_expr": "15 2 * * *",
"user": "root",
"enabled": True, "enabled": True,
"prune_max_delete": 10, "prune_max_delete": 10,
} }
@@ -631,6 +943,158 @@ def _pretty_json(value: object) -> str:
return json.dumps(value or {}, indent=2, sort_keys=True) return json.dumps(value or {}, indent=2, sort_keys=True)
def _parse_changelog(text: str) -> list[dict[str, object]]:
blocks: list[dict[str, object]] = []
paragraph: list[str] = []
list_items: list[str] = []
def flush_paragraph() -> None:
if paragraph:
blocks.append({"kind": "paragraph", "text": " ".join(paragraph)})
paragraph.clear()
def flush_list() -> None:
if list_items:
blocks.append({"kind": "list", "items": list(list_items)})
list_items.clear()
for raw_line in text.splitlines():
line = raw_line.strip()
if not line:
flush_paragraph()
flush_list()
continue
if line.startswith("#"):
flush_paragraph()
flush_list()
marker, _space, heading = line.partition(" ")
level = min(max(len(marker), 1), 3)
blocks.append({"kind": "heading", "level": level, "text": heading.strip() or line.lstrip("#").strip()})
continue
if line.startswith("- "):
flush_paragraph()
list_items.append(line[2:].strip())
continue
flush_list()
paragraph.append(line)
flush_paragraph()
flush_list()
return blocks
def _snapshot_restore_guidance(snapshot: SnapshotRecord) -> dict[str, str]:
source_path = Path(snapshot.path) / "data"
destination_path = Path("/restore") / snapshot.host.host
example_relative_path = Path("etc") / "nginx"
example_file_relative_path = Path("home") / "example" / "site" / "public_html" / "index.php"
quoted_source = _quote_path_with_trailing_slash(source_path)
quoted_destination = _quote_path_with_trailing_slash(destination_path)
quoted_partial_source = _quote_path_with_trailing_slash(source_path / example_relative_path)
quoted_partial_destination = _quote_path_with_trailing_slash(destination_path / example_relative_path)
quoted_file_source = shlex.quote(str(source_path / example_file_relative_path))
quoted_file_destination = shlex.quote(str(destination_path / example_file_relative_path))
quoted_remote_destination = shlex.quote(f"root@{snapshot.host.address or snapshot.host.host}:/")
common_args = "rsync -aHAX --numeric-ids --info=progress2"
return {
"source_path": str(source_path),
"destination_path": str(destination_path),
"example_relative_path": str(example_relative_path),
"example_file_relative_path": str(example_file_relative_path),
"inspect_command": f"ls -la {quoted_source}",
"dry_run_command": f"{common_args} --dry-run {quoted_source} {quoted_destination}",
"local_command": f"{common_args} {quoted_source} {quoted_destination}",
"partial_dry_run_command": f"{common_args} --dry-run {quoted_partial_source} {quoted_partial_destination}",
"file_dry_run_command": f"{common_args} --dry-run {quoted_file_source} {quoted_file_destination}",
"remote_dry_run_command": f"{common_args} --dry-run {quoted_source} {quoted_remote_destination}",
}
def _quote_path_with_trailing_slash(path: Path) -> str:
return shlex.quote(str(path).rstrip("/") + "/")
def _run_rsync_log_path(run: BackupRun) -> Path | None:
if isinstance(run.result, dict):
log = run.result.get("log")
if isinstance(log, str) and log:
return Path(log)
execution = run.result.get("execution")
if isinstance(execution, dict):
execution_log = execution.get("log")
if isinstance(execution_log, str) and execution_log:
return Path(execution_log)
if run.snapshot_path:
return Path(run.snapshot_path) / "meta" / "rsync.log"
return None
def _run_rsync_command(rsync_result: dict) -> list[str]:
command = rsync_result.get("command")
if not isinstance(command, list):
return []
return [str(part) for part in command]
def _run_rsync_log_tail(rsync_result: dict, log_path: Path | None, *, max_lines: int = 30) -> list[str]:
log_tail = rsync_result.get("log_tail")
if isinstance(log_tail, list):
return [str(line) for line in log_tail[-max_lines:]]
if log_path is None or not log_path.is_file():
return []
try:
with log_path.open("r", encoding="utf-8", errors="replace") as handle:
return handle.read().splitlines()[-max_lines:]
except OSError:
return []
def _dry_run_summary(
*,
result: dict,
requested: dict,
stats: dict,
failure: dict,
rsync_log_tail: list[str],
rsync_log_exists: bool,
) -> dict[str, object]:
if not (result.get("dry_run") or requested.get("dry_run")):
return {}
rsync_stats = stats.get("rsync") if isinstance(stats.get("rsync"), dict) else {}
warnings = []
if failure:
message = failure.get("message") or failure.get("summary")
hint = failure.get("hint")
if message:
warnings.append(str(message))
if hint:
warnings.append(str(hint))
for line in rsync_log_tail:
lowered = line.lower()
if "warning" in lowered or "permission denied" in lowered or "failed" in lowered:
warnings.append(line)
return {
"ok": result.get("ok"),
"status": "passed" if result.get("ok") else ("failed" if result.get("ok") is False else "running"),
"highlight_class": "success" if result.get("ok") else ("failed" if result.get("ok") is False else "warning"),
"files_seen": rsync_stats.get("files_total"),
"files_would_transfer": rsync_stats.get("files_transferred"),
"total_file_size_bytes": rsync_stats.get("total_file_size_bytes"),
"transfer_estimate_bytes": rsync_stats.get("total_transferred_file_size_bytes")
or rsync_stats.get("literal_data_bytes"),
"link_dest_estimated_savings_bytes": rsync_stats.get("link_dest_estimated_savings_bytes"),
"duration_seconds": stats.get("duration_seconds"),
"log_available": rsync_log_exists,
"warnings": list(dict.fromkeys(warnings)),
}
def _log_context(request) -> dict[str, object]: def _log_context(request) -> dict[str, object]:
units = ("pobsync-web.service", "pobsync-worker.service", "pobsync-scheduler.service") units = ("pobsync-web.service", "pobsync-worker.service", "pobsync-scheduler.service")
priorities = { priorities = {
@@ -641,8 +1105,26 @@ def _log_context(request) -> dict[str, object]:
"6": "Info", "6": "Info",
"7": "Debug", "7": "Debug",
} }
time_windows = {
"1h": "Last hour",
"6h": "Last 6 hours",
"24h": "Last 24 hours",
"7d": "Last 7 days",
"": "All available",
}
since_values = {
"1h": "1 hour ago",
"6h": "6 hours ago",
"24h": "24 hours ago",
"7d": "7 days ago",
}
selected_unit = request.GET.get("unit", "") selected_unit = request.GET.get("unit", "")
priority = request.GET.get("priority", "0..4") priority = request.GET.get("priority", "0..4")
time_window = request.GET.get("window", "24h")
if time_window not in time_windows:
time_window = "24h"
host_filter = request.GET.get("host", "").strip()
run_filter = request.GET.get("run", "").strip()
query = request.GET.get("q", "").strip() query = request.GET.get("q", "").strip()
lines = [] lines = []
error = "" error = ""
@@ -651,6 +1133,8 @@ def _log_context(request) -> dict[str, object]:
error = "journalctl is not available in this runtime." error = "journalctl is not available in this runtime."
else: else:
command = ["journalctl", "--no-pager", "-n", "300", "-o", "short-iso"] command = ["journalctl", "--no-pager", "-n", "300", "-o", "short-iso"]
if time_window:
command.extend(["--since", since_values[time_window]])
if selected_unit in units: if selected_unit in units:
command.extend(["-u", selected_unit]) command.extend(["-u", selected_unit])
else: else:
@@ -663,16 +1147,38 @@ def _log_context(request) -> dict[str, object]:
error = result.stderr.strip() or "Could not read journal logs." error = result.stderr.strip() or "Could not read journal logs."
else: else:
lines = result.stdout.splitlines() lines = result.stdout.splitlines()
if query: lines = _filter_log_lines(lines, query=query, host=host_filter, run_id=run_filter)
lowered_query = query.lower()
lines = [line for line in lines if lowered_query in line.lower()]
return { return {
"units": units, "units": units,
"priorities": priorities, "priorities": priorities,
"time_windows": time_windows,
"selected_unit": selected_unit, "selected_unit": selected_unit,
"selected_priority": priority, "selected_priority": priority,
"selected_window": time_window,
"host_filter": host_filter,
"run_filter": run_filter,
"query": query, "query": query,
"lines": lines, "lines": lines,
"error": error, "error": error,
} }
def _filter_log_lines(lines: list[str], *, query: str, host: str, run_id: str) -> list[str]:
filters = []
if query:
filters.append(lambda line: query.lower() in line.lower())
if host:
filters.append(lambda line: host.lower() in line.lower())
if run_id:
run_tokens = (
f"run {run_id}",
f"run={run_id}",
f"run_id={run_id}",
f"run-{run_id}",
f"#{run_id}",
)
filters.append(lambda line: any(token in line.lower() for token in run_tokens))
if not filters:
return lines
return [line for line in lines if all(matches(line) for matches in filters)]

View File

@@ -99,3 +99,6 @@ DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
POBSYNC_HOME = os.getenv("POBSYNC_HOME", "/opt/pobsync") POBSYNC_HOME = os.getenv("POBSYNC_HOME", "/opt/pobsync")
POBSYNC_BACKUP_ROOT = os.getenv("POBSYNC_BACKUP_ROOT", "/backups") POBSYNC_BACKUP_ROOT = os.getenv("POBSYNC_BACKUP_ROOT", "/backups")
POBSYNC_ENV_FILE = os.getenv("POBSYNC_ENV_FILE", "/etc/pobsync/pobsync.env")
POBSYNC_SERVICE_USER = os.getenv("POBSYNC_SERVICE_USER", "pobsync")
POBSYNC_SERVICE_GROUP = os.getenv("POBSYNC_SERVICE_GROUP", "pobsync")

View File

@@ -8,8 +8,10 @@ from pobsync_backend import api, views
urlpatterns = [ urlpatterns = [
path("", views.dashboard, name="dashboard"), path("", views.dashboard, name="dashboard"),
path("changelog/", views.changelog, name="changelog"),
path("self-check/", views.self_check, name="self_check"), path("self-check/", views.self_check, name="self_check"),
path("logs/", views.logs, name="logs"), path("logs/", views.logs, name="logs"),
path("purged-snapshots/", views.purged_snapshots, name="purged_snapshots"),
path("config/global/", views.edit_global_config, name="edit_global_config"), path("config/global/", views.edit_global_config, name="edit_global_config"),
path("ssh-credentials/", views.ssh_credentials, name="ssh_credentials"), path("ssh-credentials/", views.ssh_credentials, name="ssh_credentials"),
path("ssh-credentials/new/", views.create_ssh_credential, name="create_ssh_credential"), path("ssh-credentials/new/", views.create_ssh_credential, name="create_ssh_credential"),
@@ -21,13 +23,26 @@ urlpatterns = [
path("hosts/<str:host>/config/", views.edit_host_config, name="edit_host_config"), path("hosts/<str:host>/config/", views.edit_host_config, name="edit_host_config"),
path("hosts/<str:host>/prepare-directories/", views.prepare_host_directories, name="prepare_host_directories"), path("hosts/<str:host>/prepare-directories/", views.prepare_host_directories, name="prepare_host_directories"),
path("hosts/<str:host>/scan-known-key/", views.scan_host_known_key, name="scan_host_known_key"), path("hosts/<str:host>/scan-known-key/", views.scan_host_known_key, name="scan_host_known_key"),
path("hosts/<str:host>/preflight/", views.run_host_preflight, name="run_host_preflight"),
path("hosts/<str:host>/queue-backup/", views.queue_manual_backup, name="queue_manual_backup"), path("hosts/<str:host>/queue-backup/", views.queue_manual_backup, name="queue_manual_backup"),
path("hosts/<str:host>/discover-snapshots/", views.discover_host_snapshots, name="discover_host_snapshots"), path("hosts/<str:host>/discover-snapshots/", views.discover_host_snapshots, name="discover_host_snapshots"),
path("hosts/<str:host>/retention-apply/", views.apply_host_retention, name="apply_host_retention"), path("hosts/<str:host>/retention-apply/", views.apply_host_retention, name="apply_host_retention"),
path("hosts/<str:host>/retention-plan/", views.host_retention_plan, name="host_retention_plan"), path("hosts/<str:host>/retention-plan/", views.host_retention_plan, name="host_retention_plan"),
path(
"hosts/<str:host>/incomplete-cleanup/",
views.cleanup_host_incomplete_snapshots,
name="cleanup_host_incomplete_snapshots",
),
path("hosts/<str:host>/schedule/", views.edit_host_schedule, name="edit_host_schedule"), path("hosts/<str:host>/schedule/", views.edit_host_schedule, name="edit_host_schedule"),
path("runs/<int:run_id>/", views.run_detail, name="run_detail"), path("runs/<int:run_id>/", views.run_detail, name="run_detail"),
path("runs/<int:run_id>/rsync-log/", views.run_rsync_log, name="run_rsync_log"),
path("runs/<int:run_id>/cancel/", views.cancel_run, name="cancel_run"), path("runs/<int:run_id>/cancel/", views.cancel_run, name="cancel_run"),
path("runs/<int:run_id>/resolve-review/", views.resolve_run_review, name="resolve_run_review"),
path(
"hosts/<str:host>/resolve-incomplete-reviews/",
views.resolve_host_incomplete_reviews,
name="resolve_host_incomplete_reviews",
),
path("snapshots/<int:snapshot_id>/", views.snapshot_detail, name="snapshot_detail"), path("snapshots/<int:snapshot_id>/", views.snapshot_detail, name="snapshot_detail"),
path("api/", api.api_index), path("api/", api.api_index),
path("api/status/", api.status), path("api/status/", api.status),