pobsync/docs/development.md

Development Notes

This document contains development and optional Docker workflows. The recommended production path is the native systemd installer documented in the README.

Local Development

python3 -m venv .venv
. .venv/bin/activate
python3 -m pip install -e .
mkdir -p var
python3 manage.py migrate
python3 manage.py createsuperuser
python3 manage.py runserver

The admin is available at:

• http://127.0.0.1:8000/
• http://127.0.0.1:8000/admin/

Staff-only JSON endpoints are available at:

• http://127.0.0.1:8000/api/
• http://127.0.0.1:8000/api/status/

Running Tests

The project test suite is currently run through the Docker image so the runtime dependencies match deployment:

docker compose build web scheduler worker
docker compose run --rm web python manage.py test pobsync_backend --verbosity 2

Maintainer CLI

The Django UI is the normal operating surface. The pobsync entrypoint and direct manage.py commands are kept for debugging, automated maintenance, and migrations. Prefer using the control panel for day-to-day host configuration, schedule changes, manual backup queueing, snapshot discovery, retention planning, and SSH credential management.

Useful checks:

pobsync django check
python3 manage.py showmigrations pobsync_backend

The short pobsync aliases are limited to operational actions that are useful while debugging a running install. Configuration aliases are intentionally not public commands; use the Django UI or explicit management commands instead.

UI Refresh Pattern

The control panel stays Django-template-first. Pages that need live status should expose a small server-rendered partial view and opt into refresh with data-refresh-url and data-refresh-interval on the container that should be replaced. The shared script in base.html polls only those explicit regions, skips refreshes while the browser tab is hidden, and lets the partial response turn polling off with the X-Pobsync-Refresh-Active: false header.

Use this for operational status surfaces such as running backup details. Avoid refreshing form-heavy sections while an operator might be typing.

Worker and scheduler commands are normally run by systemd services:

pobsync worker --loop --interval 15
pobsync scheduler --loop --interval 60

One-off maintenance commands are still available when the UI is not the right tool:

pobsync backup <host> --dry-run
pobsync discover-snapshots --host <host>
pobsync retention <host>

For scripted configuration changes, call the Django management command explicitly so it is clear that this is an automation/debugging path rather than the normal UI workflow:

pobsync django configure_pobsync_host <host> --address <host.example>
pobsync django configure_pobsync_schedule <host> --schedule-expression "15 2 * * *"

Installer Development

The native installer is interactive by default when stdin is a terminal. It should keep every prompt backed by a command line flag or environment variable so production installs remain scriptable.

Useful modes:

sudo scripts/install-systemd
sudo scripts/install-systemd --non-interactive
sudo scripts/install-systemd --verbose
sudo scripts/install-systemd --create-superuser --superuser-username admin
sudo scripts/update-systemd

The installer should print a short completion summary with the control panel URL, Self Check reminder, and service log commands. Keep normal output user-facing: pobsync step names with OK, FAILED, or SKIPPED. Full apt, pip, Django, and systemd output belongs behind --verbose or in the failed step output.

The updater is intentionally a small wrapper around the installer for routine production deploys. It should stay non-interactive, preserve the existing environment file, skip OS package installation, skip superuser creation, and still run the Django/runtime refresh steps needed after a code update.

Docker With SQLite

Docker Compose is useful for local development and disposable test installs. Native systemd is preferred for production backup servers.

docker compose up --build web

This starts Django on:

• http://127.0.0.1:8010/
• http://127.0.0.1:8010/admin/
• http://127.0.0.1:8010/api/
• http://127.0.0.1:8010/api/status/

Run the scheduler alongside the web admin:

docker compose up --build web scheduler worker

The web service runs Django through Gunicorn and serves static files with WhiteNoise. The container persists /opt/pobsync and the SQLite database in Docker volumes.

Backup data is always available at /backups inside the containers. By default this uses ./backups on the host. Override the host-side mount with POBSYNC_BACKUP_ROOT:

POBSYNC_BACKUP_ROOT=/mnt/backups/pobsync docker compose up --build web scheduler worker

Docker With MariaDB

docker compose --profile mariadb up --build web-mariadb

With the scheduler:

docker compose --profile mariadb up --build web-mariadb scheduler-mariadb worker-mariadb

SQLite remains the default because it is enough for a single backup server and keeps deployment simple.

For native systemd installs with MariaDB client support, run the installer with:

sudo scripts/install-systemd --install-extras mariadb

Current Architecture

The public operating surface is Django-first. The CLI is now a maintainer layer around Django management commands and the old YAML/cron workflow has been retired from the pobsync entrypoint. Discovered snapshots are stored in SnapshotRecord, including the base snapshot metadata and a nullable SQL link to the base record when it is known.

The Django retention command plans from SnapshotRecord instead of rediscovering snapshots from the filesystem. Post-backup pruning from Django also uses the SQL retention service after the completed snapshot is recorded. Staff-only JSON endpoints expose service status, hosts, snapshots, and backup runs for lightweight inspection. Staff-only dashboard views expose the same operational state through Django templates. Host pages include a safe snapshot discovery action that records existing snapshots into SQL. Host pages also include a read-only SQL retention plan view before any destructive pruning action. Schedules can be created or updated from host pages using the same SQL-backed scheduler model. Host config can be edited from host pages while keeping host identity stable.

The remaining internal engine code still contains reusable backup primitives:

• snapshot naming and metadata
• rsync command construction and execution
• retention planning and pruning
• host locking

Next refactor targets:

• Move more snapshot lifecycle details into typed domain objects.
• Replace remaining dictionary-shaped config at engine boundaries.