Commit Graph

50 Commits

Author SHA1 Message Date
515330c436 ## Summary
- Add per-host rsync bandwidth limit overrides with inherit/unlimited semantics.
- Store the effective bwlimit in run metadata/results and show it in host/run detail views.
- Document recommended starting values for VPN and remote backups.

## Tests
- `.venv/bin/python manage.py makemigrations --check --dry-run`
- `.venv/bin/python manage.py test src.pobsync_backend.tests.test_django_config_source.DjangoConfigSourceTests.test_returns_effective_config_from_database src.pobsync_backend.tests.test_django_config_source.DjangoConfigSourceTests.test_host_can_disable_global_rsync_bandwidth_limit src.pobsync_backend.tests.test_configure_commands.ConfigureCommandsTests.test_configure_host_uses_global_retention_defaults src.pobsync_backend.tests.test_run_scheduled_config_source.RunScheduledConfigSourceTests.test_dry_run_applies_configured_bandwidth_limit src.pobsync_backend.tests.test_run_scheduled_config_source.RunScheduledConfigSourceTests.test_real_run_can_request_verbose_output_args --verbosity 2`
- `.venv/bin/python manage.py test src.pobsync_backend.tests.test_views.ViewTests.test_create_host_config_form_creates_host src.pobsync_backend.tests.test_views.ViewTests.test_host_detail_renders_effective_config_preview src.pobsync_backend.tests.test_views.ViewTests.test_run_detail_renders_result_payload src.pobsync_backend.tests.test_views.ViewTests.test_host_config_form_updates_host_config --verbosity 2`
- `.venv/bin/python manage.py check`

Closes #51
2026-05-23 00:59:55 +02:00
c7e9e69345 (release) Prepare pobsync 1.1.0
Bump the package and runtime version to 1.1.0, add release notes for
the UI-focused control panel update, and refresh version assertions for
the CLI entrypoint.

Refs 1.1
2026-05-21 15:23:50 +02:00
c5865a5379 (refactor) Normalize runtime config labels
Hide the old pobsync_home field from the Django admin and replace legacy
operator-facing labels with runtime state root and backup root terminology.

Rename admin compatibility fieldsets, update self-check/config-check text,
and refresh management command help so Django/systemd stays the primary
mental model.
2026-05-21 02:24:55 +02:00
20a9f93378 (docs) Add targeted restore examples
Extend the restore guidance with directory and single-file dry-run
examples so operators can restore a focused path without copying an
entire snapshot.

Render the examples on snapshot detail pages using the selected
snapshot's data path and the host-specific staging destination.
2026-05-21 02:05:19 +02:00
b78f102e9d (docs) Add manual restore guidance for snapshots
Document the manual restore workflow in the README and surface snapshot-
specific restore commands on the snapshot detail page.

The guidance keeps restores intentionally manual for now: inspect the
snapshot data directory, run rsync with --dry-run, restore to staging
first, and treat hardlinked snapshot files as read-only.
2026-05-21 02:01:40 +02:00
851f967f12 (ops) Expand native install self checks and recovery docs
Extend the runtime self check with native install diagnostics for the
environment file, service user, backup root ownership, and SQLite database
path. Export install metadata from the systemd units and pobsync-manage wrapper
so custom env files and service users are visible to Django checks.

Document restart, journal log inspection, and rollback steps in the README so
production updates have a clear recovery path.
2026-05-20 01:44:51 +02:00
c97c595253 (ops) Add terminal self-check command for native installs
Add check_pobsync_install so native deployments can run the same runtime
diagnostics from the terminal that are available in the Django Self Check view.

The command prints every check with status, returns a failing exit code when
install-critical checks fail, supports fail-on-warning for stricter automation,
and is documented in the installer output and README update flow.
2026-05-20 01:37:07 +02:00
e6ed7954de (ops) Add native update wrapper for production deploys
Add scripts/update-systemd as a safer routine deploy entrypoint for native
systemd installations. The wrapper keeps updates non-interactive, preserves the
existing environment file, skips OS package installation, and avoids superuser
creation prompts while still reusing the installer refresh flow.

Document the update path in the README and capture the maintenance expectation
in the development notes.
2026-05-20 01:30:02 +02:00
73e6bb7285 (ops) Install pobsync-manage for native management commands
Add a pobsync-manage wrapper that loads the native environment file before
running Django management commands, so production commands use the same
database and runtime settings as the systemd services.

Install the wrapper from the systemd installer, use it for migrations,
static collection, SSH key setup, and superuser creation, and document it
in the README for operational commands.
2026-05-20 01:27:08 +02:00
d451d01fe2 (docs) Move 1.0 roadmap tracking to Gitea
Create the 1.0 milestone and roadmap issues in Gitea, making Gitea the
leading tracker for release work.

Remove the repository roadmap document and README reference now that the
roadmap is tracked as milestone issues.
2026-05-20 01:17:35 +02:00
bfe17969e6 (docs) Add pobsync 1.0 roadmap
Document the remaining 1.0 work as ordered, reviewable chunks with
checkboxes for install/update flow, logging, preflight validation,
retention UX, dashboard polish, restore docs, config cleanup, and release
hardening.

Link the roadmap from the README and note how it can later be mirrored into
Gitea milestones and issues.
2026-05-20 01:07:17 +02:00
39b6cf3469 (feature) Add scheduler timezone setup to the native installer
Prompt for the scheduler timezone during interactive installs and support
a --time-zone flag for scripted installs.

Detect a sensible default from POBSYNC_TIME_ZONE, timedatectl, or
/etc/timezone before falling back to UTC, and validate the value with
Python zoneinfo before writing it to pobsync.env.

Document that schedules are evaluated in POBSYNC_TIME_ZONE so production
installs can avoid UTC/local-time ambiguity.
2026-05-19 23:09:15 +02:00
df3dcc47c9 (feature) Generate filesystem-backed SSH credentials
Add filesystem-backed SSH credentials for the native systemd deployment
path. Generated keys are stored below POBSYNC_HOME with 0600
permissions, while Django keeps the public key, fingerprint, path, and
selection metadata.

Add a Django SSH key generation view, delete action for unused generated
keys, and a management command used by the installer to ensure a default
backup key exists.

Update runtime config to use generated key paths directly as IdentityFile,
extend host checks to verify key readability, and keep legacy uploaded
keys available for compatibility.
2026-05-19 19:41:40 +02:00
ccacad3d37 (bugfix) Grant service user backup and journal access
Update the native installer so the pobsync service user gets journal
read access when the host exposes systemd-journal or adm groups.

Apply ownership and private directory modes to the configured backup
root, and reuse the existing environment backup root on reinstall so
production updates do not fall back to /backups.

Add a self-check for journal access and a host detail action that can
prepare missing backup directories for existing host configurations.
2026-05-19 19:25:05 +02:00
90f28410ce (feature) Add host doctor checks and Django log viewer
Add host-level checks for address, enabled state, SSH credential
selection, and backup directory readiness, and show them on the host
detail page.

Create host backup directories during host creation and prefill new
hosts from the default global config.

Add a staff-only logs view backed by journalctl with filtering by
pobsync unit, priority, and message text.

Improve runtime checks for gunicorn in virtualenv installs and ensure
the native installer grants the service user access to the backup root.
2026-05-19 19:11:57 +02:00
bb7907846e (bugfix) Restart systemd services during native updates
Change the native installer to enable services and then explicitly
restart web, worker, and scheduler so updated Django code is loaded
after each install or update.

Document that the installer refreshes app files and restarts systemd
services as part of the normal update flow.
2026-05-19 18:58:41 +02:00
96b91b2a69 (refactor) Quiet native installer output by default
Add step-based installer logging that reports pobsync actions as OK,
FAILED, or SKIPPED while suppressing noisy apt, pip, Django, and
systemd output during successful runs.

Show the captured output for the failed step when something breaks,
and add a --verbose flag to restore full command output for debugging.

Document the quieter installer behavior and verbose mode in the README
and development notes.
2026-05-19 18:52:31 +02:00
c7cfb603b0 (bugfix) Improve SSH credential validation feedback
Normalize pasted private keys before validation and detect common SSH
credential mistakes, including public keys pasted into the private key
field and public keys that do not match the supplied private key.

Translate OpenSSH libcrypto parse failures into a clearer user-facing
message and disable browser spellcheck/autocomplete on SSH key fields.

Document the native update flow as git pull followed by the
non-interactive installer so deployments refresh cleanly.
2026-05-19 18:35:39 +02:00
38f946d1c4 (feature) Make the native installer interactive
Add an interactive setup flow to the systemd installer with defaults
for install paths, service identity, backup storage, bind address,
allowed hosts, CSRF origins, OS package installation, MariaDB support,
nginx setup, and first superuser creation.

Keep scripted installs supported through non-interactive mode, existing
overrides, environment variables, and explicit superuser flags.

Print a user-facing completion summary with the control panel URL,
Self Check reminder, first setup steps, and useful service log commands.
2026-05-19 18:22:18 +02:00
44d821c638 (docs) Reframe documentation around Django-first operations
Update the README to describe pobsync as a Django-first, SQL-backed
service with the control panel as the primary operational interface.

Move CLI examples out of the normal workflow and document them as
maintainer tooling for debugging, services, and migration tasks.
2026-05-19 18:17:43 +02:00
b1789d8621 (config) Install native runtime requirements and split docs
Teach the systemd installer to install required Debian/Ubuntu
packages by default, with an opt-out for users who manage system
dependencies themselves.

Add explicit MariaDB install-extra handling so native installs can
pull in both the Python extra and required client build packages.

Slim down the README to production setup and operations, and move
development, Docker, migration helper, and architecture notes into
docs/development.md.
2026-05-19 18:15:34 +02:00
372a857f15 (feature) Add full native installer and self-check page
Expand the systemd installer so it can perform a complete native
installation with sensible defaults: copy the checkout into the target
app directory, create runtime directories, write the environment file,
install dependencies, configure systemd units, and optionally configure
nginx.

Add a staff-only Django self-check page that verifies runtime settings,
required binaries, writable paths, database connectivity, global config
state, and systemd service status when available.

Document installer overrides and expose the self-check from the main
navigation.
2026-05-19 16:05:03 +02:00
b93e19a7c8 (refactor) Add native systemd production deployment
Make native systemd services the recommended production path for
pobsync while keeping Docker Compose available for development and
optional test installs.

Add web, worker, and scheduler systemd unit templates, a native
environment example, an optional nginx reverse proxy template, and an
installer that creates the venv, service user, env file, units, and
runs migrations/static collection.

Allow native deployments to configure POBSYNC_BACKUP_ROOT directly and
document the new production layout and update flow.
2026-05-19 15:59:07 +02:00
1297a839d4 (config) Harden Docker deployment for remote servers
Run the Django control panel with Gunicorn instead of the development
runserver and serve static files through WhiteNoise.

Add restart policies, healthchecks, .env-driven production settings, and
a sample .env file for single-server deployments. Update the Docker
entrypoint to collect static assets and document the remote server
deployment and update flow in the README.
2026-05-19 15:33:09 +02:00
e65537c6de (feature) Add Django-managed SSH credentials
Add SSH credentials as first-class Django data so backup keys can be
uploaded through the control panel instead of mounted into containers.

Credentials can be selected globally or overridden per host. At runtime
the selected key is materialized inside the container with restrictive
file permissions and injected into the rsync SSH command via IdentityFile.
Known hosts entries are handled the same way when configured.

Add control panel views for creating and listing SSH keys, expose the
fields in config forms and admin, document the workflow, and cover global
and host credential selection with tests.
2026-05-19 14:37:38 +02:00
573177e118 (refactor) make Docker backup root static in Django setup
Remove backup_root from the normal Django global config form and display
the fixed container path /backups instead.

Always persist /backups from the setup form so Docker deployments do not
mix host paths with container paths.

Update tests and docs to clarify that the host backup directory is chosen
through the Docker mount, while Django always uses /backups internally.
2026-05-19 13:14:22 +02:00
fe8e65e12e (feature) add queued backup worker foundation
Move backup execution out of the management command into a reusable
backup runner service that can execute an existing BackupRun record.

Add queue primitives and a run_pobsync_worker command so manual backup
requests can be recorded as queued SQL state and processed outside the
web request path.

Add a worker Docker service and pobsync worker CLI alias, with tests for
queued run creation, worker execution, manual run typing, and command
mapping.
2026-05-19 13:00:12 +02:00
aea22597ba (bugfix) preserve saved global backup root in Django setup form
Fix the global config edit view so default initial values are only used
when creating a new config, preventing saved backup_root values from
being hidden by form defaults.

Keep pobsync_home as an internal runtime setting instead of exposing it
in the normal Django setup form.

Mount a host backup directory into Docker at /backups and document
POBSYNC_BACKUP_ROOT so backup_root behaves predictably in containers.
2026-05-19 12:48:32 +02:00
4dbde43465 (feature) Add host config editing view
Add a staff-only Django form for editing operational host settings while keeping
host identity stable. Support address, enablement, SSH/source overrides,
include/exclude lists, rsync extra args, and retention settings using the same
SQL-backed HostConfig model consumed by backup and scheduler flows.

Parse newline-separated list fields into JSON lists, preserve nullable
excludes_replace semantics, and cover rendering plus update behavior with view
tests.
2026-05-19 12:17:17 +02:00
6d7bf531ac (feature) Add schedule editing view for hosts
Add a staff-only Django form for creating and updating host schedules using the
SQL-backed ScheduleConfig model. Link the form from host detail pages, validate
cron expressions with the existing scheduler parser, and preserve scheduler/CLI
behavior by writing to the same source of truth.

Cover default rendering, schedule creation, updates, and invalid cron handling
with view tests.
2026-05-19 12:13:12 +02:00
123583a502 (feature) Add read-only retention plan view
Add a staff-only retention plan page for each host using the SQL-backed
retention service. Link it from the host detail page and show policy settings,
keep reasons, and snapshots that would be deleted for scheduled, manual, or all
snapshot kinds.

Keep the flow non-destructive for now, validate query parameters, and cover the
view with tests for rendering, base protection, and invalid kind handling.
2026-05-19 12:00:19 +02:00
3f3bdf2d45 (feature) Add snapshot discovery action to host view
Add a staff-only POST action on host detail pages to discover existing snapshots
for that host and record them into SQL. Show success or failure feedback through
Django messages, and keep the action non-destructive before adding heavier
backup or retention controls.

Cover the action with view tests for successful discovery, redirect behavior,
and method safety.
2026-05-19 11:56:45 +02:00
b0c6afad09 (feature) Add staff-only Django dashboard views
Add a small template-based UI for inspecting pobsync state through Django. The
dashboard shows host, schedule, snapshot, and backup run summaries, while host
detail pages show config, schedule, recent runs, and discovered snapshots.

Keep the views read-only and staff-protected, document the new dashboard URL,
and cover the routes with focused view tests.
2026-05-19 11:53:32 +02:00
2778a589ea (feature) Add staff-only service status API
Add /api/status/ for quick inspection of database backend, object counts, latest
backup run, and latest scheduler activity. Link it from the API index and reuse
schedule serialization between host summaries and status output.

Cover the endpoint with a focused API test and document the new status URL.
2026-05-19 11:46:22 +02:00
ccd89119da (feature) Add staff-only JSON inspection API
Expose lightweight Django JSON endpoints for hosts, snapshots, and backup runs
using the existing admin/staff authentication boundary. Include filters for
snapshot and run inspection, return resolved snapshot base metadata, and document
the new /api/ entrypoint.

Add endpoint tests for authentication, host summaries, snapshot lineage payloads,
and run filtering.
2026-05-19 11:43:50 +02:00
e16c13a1e7 Move Docker web admin port to 8010
Publish the Django web container on host port 8010 while keeping the internal
runserver port at 8000. Update the Docker README URL so the admin location
matches the running compose setup.
2026-05-19 11:34:42 +02:00
797619acd9 Run post-backup pruning through SQL retention
Stop passing prune options into the legacy scheduled backup engine from the
Django backup command. Record the completed snapshot first, then apply retention
through the SQL-backed retention service so pruning sees the same SnapshotRecord
state as the admin and retention command.

Also record prune failures on BackupRun.result instead of leaving the run in an
ambiguous state.
2026-05-19 11:32:32 +02:00
254f915051 Plan Django retention from snapshot records 2026-05-19 11:24:48 +02:00
659377d894 Track snapshot base lineage in Django 2026-05-19 11:19:22 +02:00
336fb1a5be feat: discover snapshots into Django records
Add a Django-native snapshot discovery service and management command
that scans backup directories, reads snapshot metadata, and idempotently
upserts SnapshotRecord rows. Expose it through the pobsync command
wrapper, update admin/docs, and cover discovery behavior with tests.
2026-05-19 05:18:01 +02:00
e564262c72 refactor: replace legacy CLI with Django command surface
Retire the old YAML and cron oriented pobsync CLI commands and expose a
SQL-first Django-backed command surface instead. Add schedule and
retention management commands, move shared defaults/parsing out of legacy
commands, remove obsolete command modules, and update documentation and
tests for the new workflow.
2026-05-19 05:14:29 +02:00
6d9ddc4457 refactor: stop using legacy JSON for runtime config
Build runtime pobsync configuration exclusively from structured SQL
fields, leaving legacy JSON only for import and audit context. Add
SQL-first management commands for global and host configuration and
cover them with tests.
2026-05-19 05:08:37 +02:00
a0eb5dcc8f refactor: promote backup configuration to structured SQL fields
Add explicit Django model fields for global and host backup settings,
including SSH, rsync, source, excludes, and retention configuration.
Populate them from legacy JSON during migration, make the config
repository prefer structured fields, and update import/admin/tests around
the SQL-first configuration model.
2026-05-19 05:04:49 +02:00
100215bf11 refactor: use injected config sources for retention
Allow retention planning and pruning to use the same ConfigSource
abstraction as scheduled backups. This removes the remaining SQL-to-YAML
export dependency from Django backup runs with pruning, keeping YAML only
as a legacy CLI compatibility path.
2026-05-19 05:00:15 +02:00
bb44f8a09c refactor: inject config sources into scheduled backups
Introduce a ConfigSource interface so scheduled backups no longer need
to load host configuration directly from runtime YAML. Add a Django-backed
config source for SQL-driven backup runs, keep file-based config as the
CLI default, and make scheduled prune execution actually apply retention
after successful runs.
2026-05-19 04:57:10 +02:00
18082496e4 feat: make Django configs drive backups and scheduling
Treat SQL-backed Django models as the source of truth for pobsync
configuration, exporting runtime YAML only as a compatibility layer for
the existing engine. Add a database-driven scheduler command, Docker
scheduler services, schedule run-state fields, and tests for scheduler,
config export, and retention behavior.
2026-05-19 04:53:47 +02:00
1a51c3e448 feat: add Django backend foundation and Docker runtime
Add a Django admin-backed management layer for pobsync configs, runs,
snapshots, and schedules. Keep the existing CLI engine as the execution
source of truth, add import/run management commands, and provide SQLite
default plus optional MariaDB Docker Compose support.
2026-05-19 04:48:13 +02:00
2fc26df1e5 Update readme.md 2026-02-04 01:26:01 +01:00
7caaf46588 add new deploy script and cutting out pip as the installer 2026-02-03 22:39:13 +01:00
bc5fb21762 Commit first skeleton code for pobsync 2026-02-02 22:15:54 +01:00