Commit Graph

114 Commits

Author SHA1 Message Date
96b91b2a69 (refactor) Quiet native installer output by default
Add step-based installer logging that reports pobsync actions as OK,
FAILED, or SKIPPED while suppressing noisy apt, pip, Django, and
systemd output during successful runs.

Show the captured output for the failed step when something breaks,
and add a --verbose flag to restore full command output for debugging.

Document the quieter installer behavior and verbose mode in the README
and development notes.
2026-05-19 18:52:31 +02:00
98d152da06 (bugfix) Add SSH private key file upload
Allow SSH credentials to be created from an uploaded private key file
as an alternative to pasting the key into a textarea.

Use multipart form handling in the credential views so server-side
keys can be imported without copy/paste wrapping or formatting damage.

Cover the upload path with a view test while keeping existing pasted
key validation behavior intact.
2026-05-19 18:48:17 +02:00
97797c574d (bugfix) Normalize pasted OpenSSH private keys
Canonicalize uploaded OpenSSH private keys before validation by
normalizing line endings, removing whitespace from the base64 body,
and re-wrapping it between the BEGIN and END markers.

Add SSH credential tests that generate a real ed25519 key, damage its
wrapping, and verify that validation succeeds after normalization.

Return a clearer validation error for PEM private keys, which are not
supported by the current credential flow.
2026-05-19 18:42:02 +02:00
c7cfb603b0 (bugfix) Improve SSH credential validation feedback
Normalize pasted private keys before validation and detect common SSH
credential mistakes, including public keys pasted into the private key
field and public keys that do not match the supplied private key.

Translate OpenSSH libcrypto parse failures into a clearer user-facing
message and disable browser spellcheck/autocomplete on SSH key fields.

Document the native update flow as git pull followed by the
non-interactive installer so deployments refresh cleanly.
2026-05-19 18:35:39 +02:00
38f946d1c4 (feature) Make the native installer interactive
Add an interactive setup flow to the systemd installer with defaults
for install paths, service identity, backup storage, bind address,
allowed hosts, CSRF origins, OS package installation, MariaDB support,
nginx setup, and first superuser creation.

Keep scripted installs supported through non-interactive mode, existing
overrides, environment variables, and explicit superuser flags.

Print a user-facing completion summary with the control panel URL,
Self Check reminder, first setup steps, and useful service log commands.
2026-05-19 18:22:18 +02:00
44d821c638 (docs) Reframe documentation around Django-first operations
Update the README to describe pobsync as a Django-first, SQL-backed
service with the control panel as the primary operational interface.

Move CLI examples out of the normal workflow and document them as
maintainer tooling for debugging, services, and migration tasks.
2026-05-19 18:17:43 +02:00
b1789d8621 (config) Install native runtime requirements and split docs
Teach the systemd installer to install required Debian/Ubuntu
packages by default, with an opt-out for users who manage system
dependencies themselves.

Add explicit MariaDB install-extra handling so native installs can
pull in both the Python extra and required client build packages.

Slim down the README to production setup and operations, and move
development, Docker, migration helper, and architecture notes into
docs/development.md.
2026-05-19 18:15:34 +02:00
372a857f15 (feature) Add full native installer and self-check page
Expand the systemd installer so it can perform a complete native
installation with sensible defaults: copy the checkout into the target
app directory, create runtime directories, write the environment file,
install dependencies, configure systemd units, and optionally configure
nginx.

Add a staff-only Django self-check page that verifies runtime settings,
required binaries, writable paths, database connectivity, global config
state, and systemd service status when available.

Document installer overrides and expose the self-check from the main
navigation.
2026-05-19 16:05:03 +02:00
b93e19a7c8 (refactor) Add native systemd production deployment
Make native systemd services the recommended production path for
pobsync while keeping Docker Compose available for development and
optional test installs.

Add web, worker, and scheduler systemd unit templates, a native
environment example, an optional nginx reverse proxy template, and an
installer that creates the venv, service user, env file, units, and
runs migrations/static collection.

Allow native deployments to configure POBSYNC_BACKUP_ROOT directly and
document the new production layout and update flow.
2026-05-19 15:59:07 +02:00
1297a839d4 (config) Harden Docker deployment for remote servers
Run the Django control panel with Gunicorn instead of the development
runserver and serve static files through WhiteNoise.

Add restart policies, healthchecks, .env-driven production settings, and
a sample .env file for single-server deployments. Update the Docker
entrypoint to collect static assets and document the remote server
deployment and update flow in the README.
2026-05-19 15:33:09 +02:00
c018011e83 (bugfix) Validate Django-managed SSH private keys
Validate uploaded SSH private keys with ssh-keygen before saving them so
invalid, malformed, or unsupported key material is rejected in the
control panel instead of failing later during rsync.

Auto-populate the public key when it is omitted, add an edit flow for
existing SSH credentials, and cover create, update, and invalid-key
paths with view tests.
2026-05-19 15:22:40 +02:00
e65537c6de (feature) Add Django-managed SSH credentials
Add SSH credentials as first-class Django data so backup keys can be
uploaded through the control panel instead of mounted into containers.

Credentials can be selected globally or overridden per host. At runtime
the selected key is materialized inside the container with restrictive
file permissions and injected into the rsync SSH command via IdentityFile.
Known hosts entries are handled the same way when configured.

Add control panel views for creating and listing SSH keys, expose the
fields in config forms and admin, document the workflow, and cover global
and host credential selection with tests.
2026-05-19 14:37:38 +02:00
91ce7ad4c5 (feature) Add host backup control actions
Turn the host detail page into a more useful operator surface for
starting greenfield backups from Django.

Add quick actions for dry-run and real backup runs, keep the advanced
manual options available, and show whether a host is ready, disabled, or
blocked by missing global config. Surface queued and running counts plus
a direct link to the active run.

Expose requested backup options on the run detail page and cover the new
control flow with view tests.
2026-05-19 14:25:28 +02:00
4fb33eca6c (feature) Add Django retention apply flow
Expose retention apply from the host retention plan page so planned
snapshot deletions can be executed from the Django UI.

The form requires explicit host confirmation, carries through the
selected retention kind and base-protection setting, and uses max_delete
as a deletion guard. The view delegates to the SQL retention apply
service and reports predictable pobsync errors back through Django
messages instead of surfacing a server error.

Add view coverage for confirmed deletion, invalid confirmation, and
POST-only enforcement.
2026-05-19 13:54:15 +02:00
83334803b9 (feature) show latest snapshot on the Django dashboard
Add latest snapshot context to each dashboard host row so imported legacy
snapshots are visible without opening every host page.

Link the latest snapshot directly to its detail page and show its kind
and status beside the host snapshot count.

Cover the dashboard latest-snapshot selection with a view test.
2026-05-19 13:44:28 +02:00
5c469f723a edit docker compose file 2026-05-19 13:33:57 +02:00
1d90454109 (feature) improve snapshot discovery visibility in Django
Add a discovery preflight that reports the configured backup root, host
root, and snapshot directory counts before importing anything.

Show discovery status on host detail pages so missing mounts or mismatched
host directories are visible from the UI.

Warn clearly when discovery scans zero snapshots, including whether the
host backup directory is missing or simply empty.
2026-05-19 13:21:31 +02:00
573177e118 (refactor) make Docker backup root static in Django setup
Remove backup_root from the normal Django global config form and display
the fixed container path /backups instead.

Always persist /backups from the setup form so Docker deployments do not
mix host paths with container paths.

Update tests and docs to clarify that the host backup directory is chosen
through the Docker mount, while Django always uses /backups internally.
2026-05-19 13:14:22 +02:00
3da877eb8a (feature) queue manual backups from the Django host page
Add a staff-only manual backup form to host detail pages with safe
dry-run defaults and optional retention settings.

Queue manual BackupRun records through the existing worker-backed runner
path instead of executing backups inside the web request.

Validate disabled hosts, missing global config, and invalid methods with
view tests covering the new UI flow.
2026-05-19 13:04:50 +02:00
fe8e65e12e (feature) add queued backup worker foundation
Move backup execution out of the management command into a reusable
backup runner service that can execute an existing BackupRun record.

Add queue primitives and a run_pobsync_worker command so manual backup
requests can be recorded as queued SQL state and processed outside the
web request path.

Add a worker Docker service and pobsync worker CLI alias, with tests for
queued run creation, worker execution, manual run typing, and command
mapping.
2026-05-19 13:00:12 +02:00
aea22597ba (bugfix) preserve saved global backup root in Django setup form
Fix the global config edit view so default initial values are only used
when creating a new config, preventing saved backup_root values from
being hidden by form defaults.

Keep pobsync_home as an internal runtime setting instead of exposing it
in the normal Django setup form.

Mount a host backup directory into Docker at /backups and document
POBSYNC_BACKUP_ROOT so backup_root behaves predictably in containers.
2026-05-19 12:48:32 +02:00
66e1f549b9 (feature) add Django detail views for backup runs and snapshots
Add staff-only run and snapshot detail pages so scheduler and command
output can be inspected from the Django UI.

Link dashboard and host detail tables to the new detail views, including
snapshot/base relationships and linked backup runs.

Render stored result and metadata JSON in readable form and cover the new
inspection views with tests.
2026-05-19 12:31:47 +02:00
6bcc15c174 (feature) add Django setup flow for initial pobsync configuration
Add staff-only UI routes for creating/editing the default GlobalConfig
and creating the first HostConfig from the dashboard.

Improve the empty dashboard state so a fresh database guides the user
towards the next useful setup action instead of only showing empty tables.

Cover the setup flow with view tests for empty state prompts, global
config creation, and host creation.
2026-05-19 12:25:45 +02:00
4dbde43465 (feature) Add host config editing view
Add a staff-only Django form for editing operational host settings while keeping
host identity stable. Support address, enablement, SSH/source overrides,
include/exclude lists, rsync extra args, and retention settings using the same
SQL-backed HostConfig model consumed by backup and scheduler flows.

Parse newline-separated list fields into JSON lists, preserve nullable
excludes_replace semantics, and cover rendering plus update behavior with view
tests.
2026-05-19 12:17:17 +02:00
6d7bf531ac (feature) Add schedule editing view for hosts
Add a staff-only Django form for creating and updating host schedules using the
SQL-backed ScheduleConfig model. Link the form from host detail pages, validate
cron expressions with the existing scheduler parser, and preserve scheduler/CLI
behavior by writing to the same source of truth.

Cover default rendering, schedule creation, updates, and invalid cron handling
with view tests.
2026-05-19 12:13:12 +02:00
123583a502 (feature) Add read-only retention plan view
Add a staff-only retention plan page for each host using the SQL-backed
retention service. Link it from the host detail page and show policy settings,
keep reasons, and snapshots that would be deleted for scheduled, manual, or all
snapshot kinds.

Keep the flow non-destructive for now, validate query parameters, and cover the
view with tests for rendering, base protection, and invalid kind handling.
2026-05-19 12:00:19 +02:00
3f3bdf2d45 (feature) Add snapshot discovery action to host view
Add a staff-only POST action on host detail pages to discover existing snapshots
for that host and record them into SQL. Show success or failure feedback through
Django messages, and keep the action non-destructive before adding heavier
backup or retention controls.

Cover the action with view tests for successful discovery, redirect behavior,
and method safety.
2026-05-19 11:56:45 +02:00
b0c6afad09 (feature) Add staff-only Django dashboard views
Add a small template-based UI for inspecting pobsync state through Django. The
dashboard shows host, schedule, snapshot, and backup run summaries, while host
detail pages show config, schedule, recent runs, and discovered snapshots.

Keep the views read-only and staff-protected, document the new dashboard URL,
and cover the routes with focused view tests.
2026-05-19 11:53:32 +02:00
2778a589ea (feature) Add staff-only service status API
Add /api/status/ for quick inspection of database backend, object counts, latest
backup run, and latest scheduler activity. Link it from the API index and reuse
schedule serialization between host summaries and status output.

Cover the endpoint with a focused API test and document the new status URL.
2026-05-19 11:46:22 +02:00
ccd89119da (feature) Add staff-only JSON inspection API
Expose lightweight Django JSON endpoints for hosts, snapshots, and backup runs
using the existing admin/staff authentication boundary. Include filters for
snapshot and run inspection, return resolved snapshot base metadata, and document
the new /api/ entrypoint.

Add endpoint tests for authentication, host summaries, snapshot lineage payloads,
and run filtering.
2026-05-19 11:43:50 +02:00
d158644567 Improve Django admin navigation for backup data
Add linked admin summaries for hosts, snapshots, and backup runs so the SQL-first
backup state is easier to inspect from the Django admin. Hosts now link to their
filtered snapshot and run lists, backup runs link back to their snapshot, and
snapshots show base/run relationships without requiring filesystem inspection.

Cover the new admin display helpers with focused tests.
2026-05-19 11:39:10 +02:00
e16c13a1e7 Move Docker web admin port to 8010
Publish the Django web container on host port 8010 while keeping the internal
runserver port at 8000. Update the Docker README URL so the admin location
matches the running compose setup.
2026-05-19 11:34:42 +02:00
797619acd9 Run post-backup pruning through SQL retention
Stop passing prune options into the legacy scheduled backup engine from the
Django backup command. Record the completed snapshot first, then apply retention
through the SQL-backed retention service so pruning sees the same SnapshotRecord
state as the admin and retention command.

Also record prune failures on BackupRun.result instead of leaving the run in an
ambiguous state.
2026-05-19 11:32:32 +02:00
254f915051 Plan Django retention from snapshot records 2026-05-19 11:24:48 +02:00
659377d894 Track snapshot base lineage in Django 2026-05-19 11:19:22 +02:00
5808800981 feat: link backup runs to snapshot records
Add a nullable SnapshotRecord foreign key to BackupRun and populate it
when run_pobsync_backup records a completed or failed snapshot. Keep the
existing snapshot_path for audit compatibility while making run-to-snapshot
navigation explicit in the database and admin.
2026-05-19 11:13:06 +02:00
0a49c5719c feat: record backup snapshots during run completion
Upsert SnapshotRecord rows directly from run_pobsync_backup results so
new successful and failed backup runs are reflected in the database
without requiring a separate discovery pass. Keep discovery for existing
snapshots and repair workflows, and cover success, failure, and dry-run
behavior with tests.
2026-05-19 11:09:20 +02:00
336fb1a5be feat: discover snapshots into Django records
Add a Django-native snapshot discovery service and management command
that scans backup directories, reads snapshot metadata, and idempotently
upserts SnapshotRecord rows. Expose it through the pobsync command
wrapper, update admin/docs, and cover discovery behavior with tests.
2026-05-19 05:18:01 +02:00
e564262c72 refactor: replace legacy CLI with Django command surface
Retire the old YAML and cron oriented pobsync CLI commands and expose a
SQL-first Django-backed command surface instead. Add schedule and
retention management commands, move shared defaults/parsing out of legacy
commands, remove obsolete command modules, and update documentation and
tests for the new workflow.
2026-05-19 05:14:29 +02:00
6d9ddc4457 refactor: stop using legacy JSON for runtime config
Build runtime pobsync configuration exclusively from structured SQL
fields, leaving legacy JSON only for import and audit context. Add
SQL-first management commands for global and host configuration and
cover them with tests.
2026-05-19 05:08:37 +02:00
a0eb5dcc8f refactor: promote backup configuration to structured SQL fields
Add explicit Django model fields for global and host backup settings,
including SSH, rsync, source, excludes, and retention configuration.
Populate them from legacy JSON during migration, make the config
repository prefer structured fields, and update import/admin/tests around
the SQL-first configuration model.
2026-05-19 05:04:49 +02:00
100215bf11 refactor: use injected config sources for retention
Allow retention planning and pruning to use the same ConfigSource
abstraction as scheduled backups. This removes the remaining SQL-to-YAML
export dependency from Django backup runs with pruning, keeping YAML only
as a legacy CLI compatibility path.
2026-05-19 05:00:15 +02:00
bb44f8a09c refactor: inject config sources into scheduled backups
Introduce a ConfigSource interface so scheduled backups no longer need
to load host configuration directly from runtime YAML. Add a Django-backed
config source for SQL-driven backup runs, keep file-based config as the
CLI default, and make scheduled prune execution actually apply retention
after successful runs.
2026-05-19 04:57:10 +02:00
18082496e4 feat: make Django configs drive backups and scheduling
Treat SQL-backed Django models as the source of truth for pobsync
configuration, exporting runtime YAML only as a compatibility layer for
the existing engine. Add a database-driven scheduler command, Docker
scheduler services, schedule run-state fields, and tests for scheduler,
config export, and retention behavior.
2026-05-19 04:53:47 +02:00
1a51c3e448 feat: add Django backend foundation and Docker runtime
Add a Django admin-backed management layer for pobsync configs, runs,
snapshots, and schedules. Keep the existing CLI engine as the execution
source of truth, add import/run management commands, and provide SQLite
default plus optional MariaDB Docker Compose support.
2026-05-19 04:48:13 +02:00
27acd790bd fix an issue with link-dest pointing towards the wrong directory 2026-02-05 11:55:35 +01:00
2fc26df1e5 Update readme.md 2026-02-04 01:26:01 +01:00
1ac326eada remove some installation logic from install.py 2026-02-04 01:01:41 +01:00
7caaf46588 add new deploy script and cutting out pip as the installer 2026-02-03 22:39:13 +01:00
f30d37632b add new install commands so they align better with the doctor code and will set the bin to the correct path 2026-02-03 16:36:18 +01:00