pobsync/README.md

# pobsync

`pobsync` is a pull-based backup service. It runs on a central backup server and pulls data from remote machines via
rsync over SSH.

The current refactor is Django-first and SQL-backed:

- The Django control panel is the primary interface for setup and operations.
- The database is the source of truth for hosts, schedules, runs, snapshots, credentials, and retention settings.
- SQLite is the default database; MariaDB is optional.
- Backups use the existing rsync snapshot engine internally.
- Scheduling is handled by a Django scheduler service, not host cron.
- SSH keys can be managed from Django and selected globally or per host.

## Recommended Production Install

The recommended production deployment is native systemd services on the backup server. Docker Compose remains available
for development and disposable test installs, but native systemd avoids Docker friction around SSH, filesystem mounts,
large backup storage, and host-level service logs.

Recommended layout:

```
/opt/pobsync/app          # installed app checkout
/opt/pobsync/venv         # Python virtualenv
/etc/pobsync/pobsync.env  # settings and secrets
/var/lib/pobsync          # SQLite database, state, runtime SSH key files, static files
/backups                  # backup storage, or set another absolute path
```

From a checked-out copy of this repository, run:

```
sudo scripts/install-systemd
```

When run from a terminal, the installer asks for the important paths and settings with sensible defaults already filled
in. It can also create the first Django superuser and prints the next steps when installation is complete.

The installer will, by default:

- install required Debian/Ubuntu OS packages with `apt-get`
- copy the checkout to `/opt/pobsync/app`
- create `/opt/pobsync/venv`
- write `/etc/pobsync/pobsync.env` if it does not exist
- install `pobsync-manage`, a Django management wrapper that loads `/etc/pobsync/pobsync.env`
- create `/var/lib/pobsync`, `/var/log/pobsync`, and the backup root
- install Python dependencies
- run migrations and collect static files
- generate a default SSH key for the service user if one does not exist yet
- install and start `pobsync-web`, `pobsync-worker`, and `pobsync-scheduler`
- guide you through the first login and setup steps

Common overrides:

```
sudo scripts/install-systemd \
  --backup-root /mnt/backups/pobsync \
  --time-zone Europe/Amsterdam \
  --allowed-hosts backup.example.com,localhost,127.0.0.1 \
  --csrf-trusted-origins https://backup.example.com
```

Use `--no-install-os-packages` if you want to manage system packages yourself. Use `--force-env` only when you want the
installer to rewrite an existing `/etc/pobsync/pobsync.env`.
Use `--non-interactive` for scripted installs. Use `--verbose` when you want to see the underlying apt, pip, Django, and
systemd output.

Schedules are evaluated in `POBSYNC_TIME_ZONE`. The installer defaults this to the server timezone when it can detect
one, otherwise `UTC`; override it with `--time-zone Europe/Amsterdam` or by editing `/etc/pobsync/pobsync.env`.

For MariaDB support, add:

```
sudo scripts/install-systemd --install-extras mariadb
```

## Services

The installer creates:

- `pobsync-web.service`: Gunicorn Django control panel on `127.0.0.1:8010`
- `pobsync-worker.service`: queued backup worker
- `pobsync-scheduler.service`: SQL-backed schedule dispatcher

Check service state and logs:

```
systemctl status pobsync-web pobsync-worker pobsync-scheduler
journalctl -u pobsync-worker -f
```

Restart after configuration changes:

```
sudo systemctl restart pobsync-web pobsync-worker pobsync-scheduler
```

## Reverse Proxy

Use an existing reverse proxy by forwarding to:

```
http://127.0.0.1:8010
```

To install a starter nginx site file:

```
sudo scripts/install-systemd --with-nginx --server-name backup.example.com
```

For HTTPS behind a reverse proxy, set:

```
POBSYNC_DJANGO_ALLOWED_HOSTS=backup.example.com,localhost,127.0.0.1
POBSYNC_DJANGO_CSRF_TRUSTED_ORIGINS=https://backup.example.com
```

## Django UI

After install, open the control panel through your reverse proxy or directly at:

```
http://127.0.0.1:8010/
```

Create a superuser if needed:

```
sudo -u pobsync pobsync-manage createsuperuser
```

For other Django management commands on native installs, use `pobsync-manage` so the production environment file is
loaded before Django starts:

```
sudo -u pobsync pobsync-manage showmigrations pobsync_backend
sudo -u pobsync pobsync-manage check
sudo -u pobsync pobsync-manage check_pobsync_install
```

The UI includes:

- dashboard and host detail pages
- global and per-host config forms
- schedule editing
- manual backup queueing
- snapshot discovery
- host checks for backup directories and SSH readiness
- host directory preparation for new or existing hosts
- SQL retention planning and apply flow
- Django-managed SSH keys
- `/self-check/` for runtime checks
- `/logs/` for filtered pobsync service logs

## Bandwidth Limits

Global config can set an rsync bandwidth limit in KB/s. The default `0` means unlimited. Each host can inherit the
global value, set `0` to explicitly run unlimited, or set its own limit for slower remote links.

For VPN-backed or remote backups, start conservatively and adjust after watching normal traffic:

- `2500` KB/s is roughly 20 Mbit/s
- `5000` KB/s is roughly 40 Mbit/s
- `10000` KB/s is roughly 80 Mbit/s

pobsync passes the effective value to rsync as `--bwlimit=<KB/s>` and shows it on the host detail and run detail pages.

## Restoring Data

pobsync treats restores as an explicit manual operation. The control panel shows restore guidance on each snapshot
detail page, but it does not run restore commands for you yet. That is deliberate: restores should be inspected and
tested before data is copied back into a live system.

Each snapshot directory contains:

```
<snapshot>/data/  # backed-up filesystem contents
<snapshot>/meta/  # metadata and rsync logs
```

Use the `data/` directory as the rsync source. Start with a dry run and restore to a staging path first:

```
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/ /restore/example.org/
rsync -aHAX --numeric-ids --info=progress2 /backups/example.org/scheduled/<snapshot>/data/ /restore/example.org/
```

After validating the staged files, copy the specific files or directories back to the target machine. For a full-host
restore, use another dry run before writing to the remote root:

```
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/ root@example.org:/
```

For most incidents, prefer a targeted restore instead of copying the whole snapshot. Keep paths relative to the
snapshot's `data/` directory:

```
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/etc/nginx/ /restore/example.org/etc/nginx/
rsync -aHAX --numeric-ids --info=progress2 --dry-run /backups/example.org/scheduled/<snapshot>/data/home/example/site/public_html/index.php /restore/example.org/home/example/site/public_html/index.php
```

Snapshots may use hardlinks for files that are unchanged between backups. That saves disk space and is safe for normal
restore copies, but do not edit files inside snapshot directories. Treat snapshots as read-only and copy data out with
rsync.

## SSH Keys

SSH keys can be managed from `/ssh-credentials/`. The recommended flow is to generate keys from Django or during the
installer. pobsync stores the private key on disk under the runtime state root (`POBSYNC_HOME`), keeps the public key
visible in the UI, and lets you select a credential either as the global default or as a per-host override.

Generated private keys are stored at:

```
$POBSYNC_HOME/state/ssh-credentials/<id>/identity
```

The key file is written with `0600` permissions and injected into the rsync SSH command with `IdentityFile`. Copy the
public key shown in Django to the target host's `authorized_keys`.

Existing private keys can still be added manually, but generated filesystem keys are preferred for native systemd
production installs.

## Updates

From a fresh checkout or the existing app directory:

```
git pull
sudo scripts/update-systemd
```

The updater is a thin wrapper around the installer for normal production deploys. It preserves the existing
`/etc/pobsync/pobsync.env`, skips OS package installation, skips superuser creation, refreshes the installed app, updates
Python dependencies, runs migrations, collects static files, and restarts the systemd services so new Django code is
loaded.

Use the full installer again when you intentionally want to change install-time settings, install OS packages, enable
nginx, or rewrite the environment file:

```
sudo scripts/install-systemd --non-interactive
sudo scripts/install-systemd --force-env
```

Then check:

```
systemctl status pobsync-web pobsync-worker pobsync-scheduler
sudo -u pobsync pobsync-manage check
sudo -u pobsync pobsync-manage check_pobsync_install
```

Restart services manually after environment or reverse proxy changes:

```
sudo systemctl restart pobsync-web pobsync-worker pobsync-scheduler
```

Inspect service logs with:

```
journalctl -u pobsync-web -n 100 --no-pager
journalctl -u pobsync-worker -f
journalctl -u pobsync-scheduler -n 100 --no-pager
```

Rollback to a previous revision by checking out the known-good commit or tag, then running the updater again:

```
git switch master
git pull
git checkout <known-good-commit-or-tag>
sudo scripts/update-systemd
sudo -u pobsync pobsync-manage check_pobsync_install
```

## Development

Development, Docker, maintainer tooling, and architecture notes live in:

- [docs/development.md](docs/development.md)