Peter van Arkel
3cac7b61ac
Add a server-rendered run detail partial and a small vanilla JavaScript refresh hook so active backup runs update status, controls, timing, and rsync log output without a full page reload. Document the Django-template-first refresh pattern for future control panel work. Refs #36
188 lines
6.6 KiB
Markdown
188 lines
6.6 KiB
Markdown
# Development Notes
|
|
|
|
This document contains development and optional Docker workflows. The recommended production path is the native
|
|
systemd installer documented in the README.
|
|
|
|
## Local Development
|
|
|
|
```
|
|
python3 -m venv .venv
|
|
. .venv/bin/activate
|
|
python3 -m pip install -e .
|
|
mkdir -p var
|
|
python3 manage.py migrate
|
|
python3 manage.py createsuperuser
|
|
python3 manage.py runserver
|
|
```
|
|
|
|
The admin is available at:
|
|
|
|
- http://127.0.0.1:8000/
|
|
- http://127.0.0.1:8000/admin/
|
|
|
|
Staff-only JSON endpoints are available at:
|
|
|
|
- http://127.0.0.1:8000/api/
|
|
- http://127.0.0.1:8000/api/status/
|
|
|
|
## Running Tests
|
|
|
|
The project test suite is currently run through the Docker image so the runtime dependencies match deployment:
|
|
|
|
```
|
|
docker compose build web scheduler worker
|
|
docker compose run --rm web python manage.py test pobsync_backend --verbosity 2
|
|
```
|
|
|
|
## Maintainer CLI
|
|
|
|
The Django UI is the normal operating surface. The `pobsync` entrypoint and direct `manage.py` commands are kept for
|
|
debugging, automated maintenance, and migrations. Prefer using the control panel for day-to-day host configuration,
|
|
schedule changes, manual backup queueing, snapshot discovery, retention planning, and SSH credential management.
|
|
|
|
Useful checks:
|
|
|
|
```
|
|
pobsync django check
|
|
python3 manage.py showmigrations pobsync_backend
|
|
```
|
|
|
|
The short `pobsync` aliases are limited to operational actions that are useful while debugging a running install.
|
|
Configuration aliases are intentionally not public commands; use the Django UI or explicit management commands instead.
|
|
|
|
## UI Refresh Pattern
|
|
|
|
The control panel stays Django-template-first. Pages that need live status should expose a small server-rendered partial
|
|
view and opt into refresh with `data-refresh-url` and `data-refresh-interval` on the container that should be replaced.
|
|
The shared script in `base.html` polls only those explicit regions, skips refreshes while the browser tab is hidden, and
|
|
lets the partial response turn polling off with the `X-Pobsync-Refresh-Active: false` header.
|
|
|
|
Use this for operational status surfaces such as running backup details. Avoid refreshing form-heavy sections while an
|
|
operator might be typing.
|
|
|
|
Worker and scheduler commands are normally run by systemd services:
|
|
|
|
```
|
|
pobsync worker --loop --interval 15
|
|
pobsync scheduler --loop --interval 60
|
|
```
|
|
|
|
One-off maintenance commands are still available when the UI is not the right tool:
|
|
|
|
```
|
|
pobsync backup <host> --dry-run
|
|
pobsync discover-snapshots --host <host>
|
|
pobsync retention <host>
|
|
```
|
|
|
|
For scripted configuration changes, call the Django management command explicitly so it is clear that this is an
|
|
automation/debugging path rather than the normal UI workflow:
|
|
|
|
```
|
|
pobsync django configure_pobsync_host <host> --address <host.example>
|
|
pobsync django configure_pobsync_schedule <host> --schedule-expression "15 2 * * *"
|
|
```
|
|
|
|
## Installer Development
|
|
|
|
The native installer is interactive by default when stdin is a terminal. It should keep every prompt backed by a command
|
|
line flag or environment variable so production installs remain scriptable.
|
|
|
|
Useful modes:
|
|
|
|
```
|
|
sudo scripts/install-systemd
|
|
sudo scripts/install-systemd --non-interactive
|
|
sudo scripts/install-systemd --verbose
|
|
sudo scripts/install-systemd --create-superuser --superuser-username admin
|
|
sudo scripts/update-systemd
|
|
```
|
|
|
|
The installer should print a short completion summary with the control panel URL, Self Check reminder, and service log
|
|
commands. Keep normal output user-facing: pobsync step names with OK, FAILED, or SKIPPED. Full apt, pip, Django, and
|
|
systemd output belongs behind `--verbose` or in the failed step output.
|
|
|
|
The updater is intentionally a small wrapper around the installer for routine production deploys. It should stay
|
|
non-interactive, preserve the existing environment file, skip OS package installation, skip superuser creation, and still
|
|
run the Django/runtime refresh steps needed after a code update.
|
|
|
|
## Docker With SQLite
|
|
|
|
Docker Compose is useful for local development and disposable test installs. Native systemd is preferred for production
|
|
backup servers.
|
|
|
|
```
|
|
docker compose up --build web
|
|
```
|
|
|
|
This starts Django on:
|
|
|
|
- http://127.0.0.1:8010/
|
|
- http://127.0.0.1:8010/admin/
|
|
- http://127.0.0.1:8010/api/
|
|
- http://127.0.0.1:8010/api/status/
|
|
|
|
Run the scheduler alongside the web admin:
|
|
|
|
```
|
|
docker compose up --build web scheduler worker
|
|
```
|
|
|
|
The web service runs Django through Gunicorn and serves static files with WhiteNoise. The container persists
|
|
`/opt/pobsync` and the SQLite database in Docker volumes.
|
|
|
|
Backup data is always available at `/backups` inside the containers. By default this uses `./backups` on the host.
|
|
Override the host-side mount with `POBSYNC_BACKUP_ROOT`:
|
|
|
|
```
|
|
POBSYNC_BACKUP_ROOT=/mnt/backups/pobsync docker compose up --build web scheduler worker
|
|
```
|
|
|
|
## Docker With MariaDB
|
|
|
|
```
|
|
docker compose --profile mariadb up --build web-mariadb
|
|
```
|
|
|
|
With the scheduler:
|
|
|
|
```
|
|
docker compose --profile mariadb up --build web-mariadb scheduler-mariadb worker-mariadb
|
|
```
|
|
|
|
SQLite remains the default because it is enough for a single backup server and keeps deployment simple.
|
|
|
|
For native systemd installs with MariaDB client support, run the installer with:
|
|
|
|
```
|
|
sudo scripts/install-systemd --install-extras mariadb
|
|
```
|
|
|
|
## Current Architecture
|
|
|
|
The public operating surface is Django-first. The CLI is now a maintainer layer around Django management commands and
|
|
the old YAML/cron workflow has been retired from the `pobsync` entrypoint.
|
|
Discovered snapshots are stored in `SnapshotRecord`, including the base snapshot metadata and a nullable SQL link to the
|
|
base record when it is known.
|
|
|
|
The Django retention command plans from `SnapshotRecord` instead of rediscovering snapshots from the filesystem.
|
|
Post-backup pruning from Django also uses the SQL retention service after the completed snapshot is recorded.
|
|
Staff-only JSON endpoints expose service status, hosts, snapshots, and backup runs for lightweight inspection.
|
|
Staff-only dashboard views expose the same operational state through Django templates.
|
|
Host pages include a safe snapshot discovery action that records existing snapshots into SQL.
|
|
Host pages also include a read-only SQL retention plan view before any destructive pruning action.
|
|
Schedules can be created or updated from host pages using the same SQL-backed scheduler model.
|
|
Host config can be edited from host pages while keeping host identity stable.
|
|
|
|
The remaining internal engine code still contains reusable backup primitives:
|
|
|
|
- snapshot naming and metadata
|
|
- rsync command construction and execution
|
|
- retention planning and pruning
|
|
- host locking
|
|
|
|
Next refactor targets:
|
|
|
|
- Move more snapshot lifecycle details into typed domain objects.
|
|
- Replace remaining dictionary-shaped config at engine boundaries.
|