Commit Graph

28 Commits

Author SHA1 Message Date
29f455a153 (bugfix) Enable rsync progress output for live real runs
Default queued and management-command backups to verbose rsync output so live
run views show progress for long-running real backups, matching dry-run
visibility.

Add a quiet-rsync escape hatch for operators who intentionally want less noisy
real-run logs.
2026-05-28 21:42:40 +02:00
515330c436 ## Summary
- Add per-host rsync bandwidth limit overrides with inherit/unlimited semantics.
- Store the effective bwlimit in run metadata/results and show it in host/run detail views.
- Document recommended starting values for VPN and remote backups.

## Tests
- `.venv/bin/python manage.py makemigrations --check --dry-run`
- `.venv/bin/python manage.py test src.pobsync_backend.tests.test_django_config_source.DjangoConfigSourceTests.test_returns_effective_config_from_database src.pobsync_backend.tests.test_django_config_source.DjangoConfigSourceTests.test_host_can_disable_global_rsync_bandwidth_limit src.pobsync_backend.tests.test_configure_commands.ConfigureCommandsTests.test_configure_host_uses_global_retention_defaults src.pobsync_backend.tests.test_run_scheduled_config_source.RunScheduledConfigSourceTests.test_dry_run_applies_configured_bandwidth_limit src.pobsync_backend.tests.test_run_scheduled_config_source.RunScheduledConfigSourceTests.test_real_run_can_request_verbose_output_args --verbosity 2`
- `.venv/bin/python manage.py test src.pobsync_backend.tests.test_views.ViewTests.test_create_host_config_form_creates_host src.pobsync_backend.tests.test_views.ViewTests.test_host_detail_renders_effective_config_preview src.pobsync_backend.tests.test_views.ViewTests.test_run_detail_renders_result_payload src.pobsync_backend.tests.test_views.ViewTests.test_host_config_form_updates_host_config --verbosity 2`
- `.venv/bin/python manage.py check`

Closes #51
2026-05-23 00:59:55 +02:00
ea9e3e41e3 (release) Add purged snapshot audit overview
Record snapshot purge history whenever retention or incomplete cleanup removes
snapshot directories and SQL records. Store the purge reason, original kind,
path, action source, and triggering operator so manual, scheduled, CLI, and
incomplete cleanup actions remain auditable after the original snapshot record
is deleted.

Add a staff-only Purged Snapshots page with host/action filters and register
the audit model in Django admin.

Refs #16
Refs #8
2026-05-21 03:46:38 +02:00
4c8ed24561 (release) Track worker heartbeat for running jobs
Record worker pid, host, claim time, and heartbeat metadata on running
backup jobs so operators can see which worker owns a run.

Refresh the heartbeat while rsync is active and reconcile stale running
runs when the worker heartbeat stops. Add a worker option to tune or
disable stale-run reconciliation.

Refs #11
2026-05-21 03:16:38 +02:00
1c8cbd96ca (refactor) Normalize maintainer command labels
Prefer --schedule-expression for scripted schedule updates while keeping
--cron as a compatibility alias.

Clean up management command help, errors, and output so operator-facing
text talks about hosts, global config, and Django backup configuration
instead of model names or old SQL-backed pobsync wording.
2026-05-21 02:52:42 +02:00
2642f14e49 (refactor) Remove pobsync_home from global config
Drop the obsolete pobsync_home field from GlobalConfig and remove it from
runtime config generation, form saves, and configuration commands.

The runtime state root now comes exclusively from POBSYNC_HOME/settings,
which keeps the Django model focused on backup behavior instead of install
layout.
2026-05-21 02:41:02 +02:00
bb62382e18 (refactor) Remove YAML config import and export path
Drop the pre-Django YAML import/export management commands and remove the
file-based config loader fallback from the backup and retention engines.

Keep the runtime config bridge backed by Django models, and add tests that
ensure engine operations require an explicit Django config source.
2026-05-21 02:34:09 +02:00
c5865a5379 (refactor) Normalize runtime config labels
Hide the old pobsync_home field from the Django admin and replace legacy
operator-facing labels with runtime state root and backup root terminology.

Rename admin compatibility fieldsets, update self-check/config-check text,
and refresh management command help so Django/systemd stays the primary
mental model.
2026-05-21 02:24:55 +02:00
994f7f66c4 (bugfix) Preserve scheduled backup warning status
Update the scheduler to reflect the actual scheduled BackupRun status after
a run completes, so prune warnings are shown as schedule warnings instead
of being reported as successful schedule executions.
2026-05-21 01:22:06 +02:00
c97c595253 (ops) Add terminal self-check command for native installs
Add check_pobsync_install so native deployments can run the same runtime
diagnostics from the terminal that are available in the Django Self Check view.

The command prints every check with status, returns a failing exit code when
install-critical checks fail, supports fail-on-warning for stricter automation,
and is documented in the installer output and README update flow.
2026-05-20 01:37:07 +02:00
c2d7342a47 (refactor) Remove unused schedule user field
Drop the legacy schedule user setting from the Django model, form, defaults,
and configure command.

Schedules are executed by the pobsync scheduler service under the configured
systemd service user, while remote SSH login users are configured separately
on global or host backup config.

Add a migration to remove the unused database column and update schedule
view tests around the simplified form.
2026-05-19 23:41:55 +02:00
8bff241b12 (bugfix) Mark retention failures as backup warnings
Add a warning status for BackupRun records so successful snapshots are not
reported as failed when post-run SQL retention fails.

Keep the prune error in the run result, link the successful snapshot, and
let the management command complete with a warning instead of raising a
backup failure.

Include warning runs in backup trend summaries and add a regression test
for successful backups with failed retention cleanup.
2026-05-19 23:20:52 +02:00
728e5c740a (feature) Add optional verbose rsync output for manual backups
Expose a verbose rsync output option in the Django manual backup form and
store the selected value with the queued run request.

Propagate the option through the worker, direct management command, and
rsync command builder so real backups can emit itemized changes, file-list
progress, and stats when requested. Dry-runs continue to use verbose output
by default and report that consistently in requested options.

Cover the queue, worker, view, and rsync command behavior with focused
tests.
2026-05-19 22:13:33 +02:00
d52a9167d1 (bugfix) Reconcile failed dry-runs from rsync terminal logs
Classify rsync failures in run results so transport issues such as exit
255 and broken pipes show clearer diagnostic hints.

Teach the worker to reconcile running dry-runs when their log already
contains a terminal rsync error, and to fail stale dry-runs after their
timeout window. This prevents failed rsync processes from leaving runs
stuck in the running state indefinitely.
2026-05-19 21:10:08 +02:00
df3dcc47c9 (feature) Generate filesystem-backed SSH credentials
Add filesystem-backed SSH credentials for the native systemd deployment
path. Generated keys are stored below POBSYNC_HOME with 0600
permissions, while Django keeps the public key, fingerprint, path, and
selection metadata.

Add a Django SSH key generation view, delete action for unused generated
keys, and a management command used by the installer to ensure a default
backup key exists.

Update runtime config to use generated key paths directly as IdentityFile,
extend host checks to verify key readability, and keep legacy uploaded
keys available for compatibility.
2026-05-19 19:41:40 +02:00
fe8e65e12e (feature) add queued backup worker foundation
Move backup execution out of the management command into a reusable
backup runner service that can execute an existing BackupRun record.

Add queue primitives and a run_pobsync_worker command so manual backup
requests can be recorded as queued SQL state and processed outside the
web request path.

Add a worker Docker service and pobsync worker CLI alias, with tests for
queued run creation, worker execution, manual run typing, and command
mapping.
2026-05-19 13:00:12 +02:00
797619acd9 Run post-backup pruning through SQL retention
Stop passing prune options into the legacy scheduled backup engine from the
Django backup command. Record the completed snapshot first, then apply retention
through the SQL-backed retention service so pruning sees the same SnapshotRecord
state as the admin and retention command.

Also record prune failures on BackupRun.result instead of leaving the run in an
ambiguous state.
2026-05-19 11:32:32 +02:00
254f915051 Plan Django retention from snapshot records 2026-05-19 11:24:48 +02:00
5808800981 feat: link backup runs to snapshot records
Add a nullable SnapshotRecord foreign key to BackupRun and populate it
when run_pobsync_backup records a completed or failed snapshot. Keep the
existing snapshot_path for audit compatibility while making run-to-snapshot
navigation explicit in the database and admin.
2026-05-19 11:13:06 +02:00
0a49c5719c feat: record backup snapshots during run completion
Upsert SnapshotRecord rows directly from run_pobsync_backup results so
new successful and failed backup runs are reflected in the database
without requiring a separate discovery pass. Keep discovery for existing
snapshots and repair workflows, and cover success, failure, and dry-run
behavior with tests.
2026-05-19 11:09:20 +02:00
336fb1a5be feat: discover snapshots into Django records
Add a Django-native snapshot discovery service and management command
that scans backup directories, reads snapshot metadata, and idempotently
upserts SnapshotRecord rows. Expose it through the pobsync command
wrapper, update admin/docs, and cover discovery behavior with tests.
2026-05-19 05:18:01 +02:00
e564262c72 refactor: replace legacy CLI with Django command surface
Retire the old YAML and cron oriented pobsync CLI commands and expose a
SQL-first Django-backed command surface instead. Add schedule and
retention management commands, move shared defaults/parsing out of legacy
commands, remove obsolete command modules, and update documentation and
tests for the new workflow.
2026-05-19 05:14:29 +02:00
6d9ddc4457 refactor: stop using legacy JSON for runtime config
Build runtime pobsync configuration exclusively from structured SQL
fields, leaving legacy JSON only for import and audit context. Add
SQL-first management commands for global and host configuration and
cover them with tests.
2026-05-19 05:08:37 +02:00
a0eb5dcc8f refactor: promote backup configuration to structured SQL fields
Add explicit Django model fields for global and host backup settings,
including SSH, rsync, source, excludes, and retention configuration.
Populate them from legacy JSON during migration, make the config
repository prefer structured fields, and update import/admin/tests around
the SQL-first configuration model.
2026-05-19 05:04:49 +02:00
100215bf11 refactor: use injected config sources for retention
Allow retention planning and pruning to use the same ConfigSource
abstraction as scheduled backups. This removes the remaining SQL-to-YAML
export dependency from Django backup runs with pruning, keeping YAML only
as a legacy CLI compatibility path.
2026-05-19 05:00:15 +02:00
bb44f8a09c refactor: inject config sources into scheduled backups
Introduce a ConfigSource interface so scheduled backups no longer need
to load host configuration directly from runtime YAML. Add a Django-backed
config source for SQL-driven backup runs, keep file-based config as the
CLI default, and make scheduled prune execution actually apply retention
after successful runs.
2026-05-19 04:57:10 +02:00
18082496e4 feat: make Django configs drive backups and scheduling
Treat SQL-backed Django models as the source of truth for pobsync
configuration, exporting runtime YAML only as a compatibility layer for
the existing engine. Add a database-driven scheduler command, Docker
scheduler services, schedule run-state fields, and tests for scheduler,
config export, and retention behavior.
2026-05-19 04:53:47 +02:00
1a51c3e448 feat: add Django backend foundation and Docker runtime
Add a Django admin-backed management layer for pobsync configs, runs,
snapshots, and schedules. Keep the existing CLI engine as the execution
source of truth, add import/run management commands, and provide SQLite
default plus optional MariaDB Docker Compose support.
2026-05-19 04:48:13 +02:00